Commit Graph

2321 Commits

Author SHA1 Message Date
Rusty Russell 3380f559f9 memleak: simplify API.
Mainly renaming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell caecd1ee0a lightningd: don't log JSON ids as debug, use log io.
They are cute, sure, but they do spam the logs.

@Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell f1f2c1322d contrib/pyln-client: construct JSON ID correctly.
They can set their name explicitly, but if they don't we extract it from argv[0].

We also set it around callbacks, so it will be expanded by default.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell d360075d22 libplugin: use string ids correctly.
Build them from the command which caused them, and take plugin name
as basename with extension stripped.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell e8ef42b741 plugin: wire JSON id for commands which caused hooks to fire.
Most obvious one is the "connect" hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell eceb9f4328 lightningd: wire plugin command JSON id through to plugin commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell ea7903f69a lightningd: trace JSON id prefixes through sendrawtx.
First, merge the _ahf_ and non-ahf interfaces.
Second, remove the always-NULL txs->cmd field.

Then, add optional id_prefix for bitcoind_sendrawx, so if it's
triggered by a command (e.g. "withdraw") it's shown correctly in logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell a9557d5194 lightningd: derive JSONRPC ids from incoming id (append /cln:<method>#NNN).
Usually the calls are spontanous, so it's just "cln:<method>#NNN", but
json_invoice() calls listincoming, and json_checkmessage calls
listnodes, so those become "cli:invoice-<pid>/cln:listincoming#NNN".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell 8fcf880e0f lightningd: explicitly remember if JSON id was a string.
This lets us use 'cmd->id' as an unquoted string (for building
new ids!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell ed3f700991 lightningd: use string as json req ids when we create them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell 8711241535 lightning-cli: use cli:<method>-<pid> for all requests.
This is the format we should standardize on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Michael Schmoock c8ab8192ca peer_control: getinfo show correct port on discovered IPs
Changelog-Fixed: peer_control: getinfo shows the correct port on discovered IPs
2022-09-15 13:30:06 +09:30
Rusty Russell 4ca6b36439 lightningd: refuse to upgrade db on non-released versions by default.
This is a good sanity check that users understand that if they upgrade
to master mid-cycle they can't go back!

Suggested-by: @wtogami
Changelog-Added: Config: `--database-upgrade=true` required if a non-release version wants to (irrevocably!) upgrade the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-15 13:25:58 +09:30
fiatjaf 1ef8fb7ef8 rename `block_processed` to `block_added` 2022-09-14 13:50:38 -05:00
fiatjaf 9b33a921f0 Add plugin notification topic "block_processed".
Changelog-Added: Plugins: Added notification topic "block_processed".
2022-09-14 13:50:38 -05:00
Rusty Russell a6d4756d08 commando: make rune alternatives a JSON array.
This avoids having to escape | or &, though we still allow that for
the deprecation period.

To detect deprecated usage, we insist that alternatives are *always*
an array (which could be loosened later), but that also means that
restrictions must *always* be an array for now.

Before:

```
# invoice, description either A or B
lightning-cli commando-rune '["method=invoice","pnamedescription=A|pnamedescription=B"]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '["method=invoice","pnamedescription=A\\|B"]'
```

After:

```
# invoice, description either A or B
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A", "pnamedescription=B"]]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A|B"]]'
```

Changelog-Deprecated: JSON-RPC: `commando-rune` restrictions is always an array, each element an array of alternatives.  Replaces a string with `|`-separators, so no escaping necessary except for `\\`.
2022-09-14 17:46:43 +02:00
Rusty Russell d57d87ea3a commando: unmangle JSON.
JSON needs to escape \, since it can't be in front of anything unexpected,
so no \|.  So we need to return \\ to \, and in theory handle \n etc.

Changelog-Fixed: JSON-RPC: `commando-rune` now handles \\ escapes properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-14 17:46:43 +02:00
Rusty Russell 897245e3b7 pytest: test for escapes in commando values.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-14 17:46:43 +02:00
Christian Decker cb3ee0ac2e wallet: Load and value `completed_at` timestamp from DB 2022-09-14 13:14:10 +02:00
niftynei 3ad8347969 bkpr-test: maybe fix race in test_bookkeeping_closing_trimmed_htlcs
test_bookkeeping_closing_trimmed_htlcs fails to find 'all outputs
resolved' occassionally, seems like it's because the
OUR_DELAYED_TO_WALLET doesn't make it into the mempool before we start
mining blocks?

So here make sure there's something in the mempool before before we
start making new blocks.
2022-09-14 15:59:37 +09:30
niftynei efad09f966 bkpr: confirm that replaying the open+lock-in txs at start is ok
Make sure that we're not issuing duplicate lease_fee events!
2022-09-14 15:59:37 +09:30
niftynei c143914ebf bkpr: migration to delete any duplicate lease_fee entries
Clean up for #5557.

If you've got duplicate 'lease_fee' entries, we delete them!
2022-09-14 15:59:37 +09:30
niftynei 1980ba420b notif: dont send balance snapshot for not yet opened channel
We were double counting channel lease fees because we were double firing
the channel open event sequence (so to speak). If we don't report
balances for unopened channels, we don't have this problem?

Changelog-Changed: Plugins: `balance_snapshot` notification does not send balances for channels that aren't locked-in/opened yet
2022-09-14 15:59:37 +09:30
niftynei 8452d903b4 bkpr: failing test for bookkeeper crash
Reproduce crash for #5557!

If we record the channel open because bookkeeper was added after the
channel open request started but the channel confirms later, we end up
with re-recording any associated push or leased fees (paid or rcvd).

In the case where you've paid for these fees, your channel balance goes
negative and the node crashes the next time you call `listbalances`.

Reported-by: @chrisguida
2022-09-14 15:59:37 +09:30
Michael Schmoock e0d6f3ceb1 connectd: DNS Bolt7 #911 no longer EXPERIMENTAL
Changelog-Changed: Bolt7 #911 DNS annoucenent support is no longer EXPERIMENTAL
2022-09-13 06:42:20 +09:30
Rusty Russell 375215a141 lightningd: more graceful shutdown.
Be more graceful in shutting down: this should fix the issue where
bookkeeper gets upset that its commands are rejected during shutdown,
and generally make things more graceful.

1. Stop any new RPC connections.
2. Stop any per-peer daemons (channeld, etc).
3. Shut down plugins.
4. Stop all existing RPC connections.
5. Stop global daemons.
6. Free up peer, chanen HTLC datastructures.
7. Close database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: RPC operations are now still available during shutdown.
2022-09-12 14:00:41 +02:00
Rusty Russell 5b7f14a7cb channeld/dualopend/lightningd: use channel_ready everywhere.
This alters the billboard, but that's a human-readable thing so not
noted in CHANGELOG.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listpeers` `status` now refers to "channel ready" rather than "funding locked" (BOLT language change for zeroconf channels)
Changelog-Added: JSON-RPC: `channel_opened` notification `channel_ready` flag.
Changelog-Deprecated: JSON-RPC: `channel_opened` notification `funding_locked` flag (use `channel_ready`: BOLTs namechange).
2022-09-12 09:34:52 +09:30
Rusty Russell 1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell 6cf3d47505 offers: remove backwards-compatiblity invoice_request signatures.
We changed the field name in v0.11.0, so this breaks compat with
v0.10.2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell c4203e7de6 pyln-client: allow 'msat' fields to be 'null'
This happens with deprecated-apis and listconfigs, breaking some
python plugins!

Fixes: #5546
Fixes: #5563
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-07 11:01:36 +09:30
Matt Morehouse 0d5808b6f6 pytest: fix test_channel_state_change_history
The test fails because the closed channel is deleted by the time we
check the state change history.

It is unnecessary to mine any blocks after the close, since the state
changes up to CLOSINGD_COMPLETE occur without onchain changes.
2022-08-31 12:23:43 +03:00
Alex Myers 0abe2e3af1 pytest: Add debugging to test_gossip_store_compact_on_load
This flake has been difficult to reproduce, so let's dump the gossip
store to aid in debugging. See issue #5410.

Changelog-None
2022-08-26 07:03:10 +09:30
adi2011 8f78a76d1a tests/test_misc.py: check logs for already existing channel. 2022-08-19 10:27:09 -04:00
Christian Decker 65549a2931 ld: Fix a log message assuming that the `channel->scid` was set
This is no longer true after we introduce `zeroconf`, so use the alias
local alias instead if not set.

Signed-off-by: Christian Decker <@cdecker>
2022-08-16 14:52:52 -05:00
Christian Decker ab1ca7f159 pytest: Reproduce a crash when we have multiple channels and 0conf
This is a case where we assume that the `channel->scid` is set, which
is no longer true with `zeroconf`

Changelog-None: Zeroconf was not yet released at the time the discovery was made

Reported-by: Yaacov Akiba Slama <@yaslama>
Reported-by: Roei Erez <@roeierez>
2022-08-16 14:52:52 -05:00
niftynei 4984014578 signpsbt: add utxo info to inputs
If you build a PSBT externally from CLN and attempt to sign for the
output, we would crash. Now we don't crash.

Changelog-Changed: JSON-RPC: `signpsbt` will now add redeemscript + witness-utxo to the PSBT for an input that we can sign for, before signing it.

Fixes #5499 ?
2022-08-13 12:57:03 -05:00
Ken Sedgwick 0878002fe6 Fix derived_secret, use correct size of secretstuff.derived secret
[ Updated tests to match -- RR]
2022-08-10 12:41:27 -05:00
Rusty Russell b173b29346 pytest: test loading lease_chan_max_msat from channel_funding_inflights
It crashes with:

```
s32 field doesn't match size: expected 4, actual 8
```

Reported-by: @zerofeerouting
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-10 10:34:12 -05:00
niftynei 9d3bf4a1b5 bkpr: let channel reconnect, flake? 2022-08-10 10:30:45 -05:00
niftynei 72a30fc750 bkpr: dont flake, wait til pay done before mining blocks 2022-08-10 10:30:45 -05:00
niftynei 23cd58402a bkpr: create accounts for zero sat channels
we weren't making records for 'missed' accounts that had a zero balance
at snapshot time (if peer opens channel and is unused)

Fixes: #5502
Reported-By: https://github.com/niftynei/cln-logmaid
2022-08-10 10:30:45 -05:00
Rusty Russell 9543204b79 pytest: don't use bogus scids for first hop of route.
This was a legacy from when it was redundant: with multiple channels, it
no longer is!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:33:24 -05:00
Rusty Russell 8a9ce55345 lightningd: don't route private channels via real scid.
Again, we should use the real channel_type, but we approximate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: private channels will only route using short-channel-ids if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00
Rusty Russell cfe6b06fb5 lightnind: use aliases in routehints for private channels.
We *should* remember the channel type, since this is only required
if they set the channel_type to include option_scid_alias.

However, since we support channel upgrade, channel_type really needs
a new table.  I have a patch for that, from my abandoned original
"fastopen" branch for aliases, but it's too big a chance for rc2 IMHO.

Meanwhile, we allow exposeprivatechannels's scids to be either real or
the aliases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: invoice routehints will use fake short-channel-ids for private channels if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00
Rusty Russell b479e9a9fa pytest: test that we implement option_scid_alias privacy.
Spoiler: we don't! :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:31:36 -05:00
niftynei 3fcf60ab7c bkpr: track channel rebalances, display in listincome
Track rebalances, and report income events for them.

Previously `listincome` would report:
	- invoice event, debit, outgoing channel
	- invoice_fee event, debit, outgoing channel
	- invoice event, credit, inbound channel

Now reports:
	- rebalance_fee, debit, outgoing channel
	(same value as invoice_fee above)

Note: only applies on channel events; if a rebalance falls to chain
we'll use the older style of accounting.

Changelog-None
2022-08-09 11:57:18 +09:30
Rusty Russell 80a6d9b58e lightningd: set the channel_type feature.
AFAICT we should have been doing this since we started sending and
receiving it, but didn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now advertize the `option_channel_type` feature (which we actually supported since v0.10.2)
2022-08-08 11:49:56 -05:00
Rusty Russell 5260ea2911 pytest: make sure we never break channels in multhtlc test.
It's perfectlty safe to extend the timeout to 100 blocks, so let's do
it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00
Rusty Russell 93303ffdad pytest: change multihtlc topology for simpler testing.
We were seeing flakes due to channel closures on intermidiary nodes if
the blocks came too fast.

If we use a star topology it's actually what we want and it's much simpler to
figure out what's going on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00
Rusty Russell b5ee5e7fb1 pytest: simplify test_onchain_multihtlc_our_unilateral/their_unilateral
We don't actually care how it does it, just that it ends the HTLCs,
so simplify this logic which tries to match it exactly.

This also fixes the flake where we would sometimes close the upstream
channels, simply because we now wait at least one second per block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00