Commit Graph

2905 Commits

Author SHA1 Message Date
Dusty Daemon 6f4a3c86e9 splice: Agressive restart testing during splices
Test node restarts in lots of phases of splice.

Changelog-None
2023-12-15 09:42:45 +10:30
Erik De Smedt 557b5fee46 Allow dynamic option in python plugin 2023-12-14 16:48:02 +01:00
Rusty Russell 36b631699b plugins: re-enable listchannels local info in deprecated mode.
We only deprecated this, we didn't actually remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell 58a1c4c012 topology: don't show private nodes in listchannels.
This alters a few remaining tests, as well.

[ Inclused even more test fixes from Alex Myers <alex@endothermic.dev>! ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listchannels` listing private channels: use listpeerchannels
2023-12-14 09:16:56 +10:30
Alex Myers 3a6b9f2ec6 pytest: skip test_hook_crash on liquid CI 2023-12-14 09:16:56 +10:30
Rusty Russell b318811a2d pytest: fix tests in assumption that listchannels will no longer show private gossip.
Some can only be changed once that is true, but some can be removed/amended already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell ea0d00e677 topology: don't show private nodes in listnodes.
This breaks our tests a bit, which assumed we can always see ourselves
even if we don't have a proper channel.

Usually we would deprecate this first, but it's unlikely to break
anyone since it's a bit obscure that this worked at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listnodes` no longer shows private (peer) nodes: use listpeers
2023-12-14 09:16:56 +10:30
Rusty Russell f61ba7b332 pytest: remove test waits for routehints.
Now listincoming sees local channels in listpeerchannels, we don't need these waits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell f3b4821d36 pytest: fix test_zeroconf_open()
l2 gave us a routehint, but it should have seen l1 as a dead-end.  It
didn't due to the presence of a channel alias, which was a bug.  We're
about to fix this, which breaks the test.

Add a dummy node off l1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell c3f33eb6dd listpeerchannels: show gossip updates.
This is redundant if it's a public channel, but vital if it's not.  Publishing unconditionally makes
it easier for gossmap: we create a local modification all the time, even if redundant (and we can
have the actual capacity ceiling accurate in this case, since we know it for local channels).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-Added: JSON-RPC: `listpeerchannels` now shows gossip update contents (even if channel unannounced).
2023-12-14 09:16:56 +10:30
Alex Myers a8995caa8b pytest: use wait_local_channel_active 2023-12-14 09:16:56 +10:30
Rusty Russell 0b23133ab2 lightningd: don't print out notification msat fields as strings.
Reported-by: Shahana Farooqui
Changelog-Fixed: JSON-RPC: Plugin notification `msat` fields in `invoice_payment` and `invoice_created` hooks now a number, not a string with "msat" suffix.
Changelog-Fixed: JSON-RPC: Plugin hook `payment` `msat` field is now a number, not a string with "msat" suffix.
2023-11-21 08:23:39 +01:00
ShahanaFarooqui fdc7fcbf34 tests/clnrest: Test to confirm that `msat` suffix has been removed from notifications 2023-11-21 08:23:39 +01:00
ShahanaFarooqui 2a7a3d2371 tests/clnrest: Added rpc method name and params to use the same `notifications_received_via_websocket` function for other rpc requests too 2023-11-21 08:23:39 +01:00
Dusty Daemon a6a9e5b1e3 splice: Reestablish when commit or sig sends fail
Adds tests for when the connection fails during
1) splice tx_signature
2) splice commitment_signed

Fleshed out the reestablish flow for these two cases and implemented the fixes to make these reestablish flows work.

Part of this work required changing commit process for splices: Now we send a single commit_part for the splice where previously we sent all commits, and accordingly, we no longer revoke in response.

Changelog-Fixed: Implemented splicing restart logic for tx_signature and commitment_signed. Splice commitments are reworked in a manner incompatible with the last version.
2023-11-20 07:35:22 +01:00
Rusty Russell eae0d1da1e lightningd: fix up deprecated `rest-port`, `rest-protocol`, `rest-host` and `rest-certs` option if we would otherwise fail.
Since these worked in v23.08, we can't just rename them.  So if they are
used and unclaimed, we should rename them internally (if they're claimed,
it's probably clightning-rest, and we should *NOT* touch them!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `clnrest` parameters `rest-port`, `rest-protocol`, `rest-host` and `rest-certs`: prefix `cln` to them
2023-11-17 10:51:41 +01:00
ShahanaFarooqui cc2665da7f clnrest: prefixing all rest config options with `cln`
This will allow users to use clnrest with c-lightning-REST without conflicts.
It was required for applications to have enough time for migrating from c-lightning-REST to clnrest.

Changelog-Changed:
config option `rest-certs` changed to `clnrest-certs`
config option `rest-protocol` changed to `clnrest-protocol`
config option `rest-host` changed to `clnrest-host`
config option `rest-port` changed to `clnrest-port`
config option `rest-cors-origins` changed to `clnrest-cors-origins`
config option `rest-csp` changed to `clnrest-csp`
2023-11-16 11:21:04 +01:00
Rusty Russell 8b84c8945b clnrest: don't convert *_msat fields to strings.
We have a global JSON encoder hack, which means that any field ending in msat gets special treatment (so we can safely talk to lightningd, even if a field expects satoshi amounts, we are explicit).  However, requests uses the JSON parser and DOES NOT want this conversion when sending it out as an HTTP response!

The simplest local fix we could find was Shahana's suggestion to iterate and covert away from Millisatoshi(): the reverse of what our JSON encoder does.

Fixes: https://github.com/ElementsProject/lightning/issues/6848
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-16 09:07:07 +01:00
Rusty Russell f2f6c73977 clnrest: don't try to handle plugin.rpc.call results.
It already throws an exception or error, or decodes the JSON response:
we can simply pass it through.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-16 09:07:07 +01:00
Tony Aldon b3f122a2c5 pytest: adds test suite for clnrest plugin
- cln-grpc certificate reuse
- new certificate generation
- `GET` and `POST` requests
- websocket server
- config options and HTTP headers

Link to #6436.
2023-11-16 09:07:07 +01:00
niftynei 3190c26bc9 dualfund: error on out of order sigs
We weren't blocking if the tx-sigs arrived before the commitment sigs.

This was causing problems in the openchannel (spender plugin)

spenderp: FATAL SIGNAL 11 (version v23.08.1-404-g62ff475-modded)
0x559836dc98ba send_backtrace
	common/daemon.c:33
0x559836dc9951 crashdump
	common/daemon.c:75
0x7f37f42c351f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f37f441ac92 ???
	../sysdeps/x86_64/multiarch/memcmp-avx2-movbe.S:83
0x559836db7760 bitcoin_txid_eq
	./bitcoin/tx.h:29
0x559836db7760 collect_sigs
	plugins/spender/openchannel.c:509
0x559836db81de check_sigs_ready
	plugins/spender/openchannel.c:531
0x559836db84dd json_peer_sigs
	plugins/spender/openchannel.c:611
0x559836dbcad7 ld_command_handle
	plugins/libplugin.c:1611
0x559836dbcd9d ld_read_json_one
	plugins/libplugin.c:1721
0x559836dbce29 ld_read_json
	plugins/libplugin.c:1741
0x559836ef3bff next_plan
	ccan/ccan/io/io.c:59
0x559836ef40da do_plan
	ccan/ccan/io/io.c:407
0x559836ef4177 io_ready
	ccan/ccan/io/io.c:417
0x559836ef5b14 io_loop
	ccan/ccan/io/poll.c:453
0x559836dbd48d plugin_main
	plugins/libplugin.c:1948
0x559836db22bf main
	plugins/spender/main.c:35
0x7f37f42aad8f __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7f37f42aae3f __libc_start_main_impl
	../csu/libc-start.c:392
0x559836da3774 ???
	???:0
0xffffffffffffffff ???
	???:0
2023-10-31T15:15:57.458Z INFO    plugin-spenderp: Killing plugin: exited during normal operation
2023-10-31T15:15:57.458Z **BROKEN** plugin-spenderp: Plugin marked as important, shutting down lightningd!
2023-10-31T15:15:57.458Z DEBUG   lightningd: io_break: lightningd_exit
2023-10-31T15:15:57.458Z DEBUG   lightningd: io_loop_with_timers: main
2023-10-31T15:15:57.458Z DEBUG   connectd: REPLY WIRE_CONNECTD_START_SHUTDOWN_REPLY with 0 fds
2023-10-31T15:15:57.458Z DEBUG   lightningd: io_break: connectd_start_shutdown_reply
2023-10-31T15:15:57.458Z DEBUG   021ccce7bc396996c8f3b7bfeb1e30c6600269517026a74adfe2217b7187879797-dualopend-chan#1: Status closed, but not exited. Killing
2023-10-31T15:15:57.458Z DEBUG   lightningd: Command returned result after jcon close
2023-10-31T15:15:57.458Z INFO    021ccce7bc396996c8f3b7bfeb1e30c6600269517026a74adfe2217b7187879797-chan#1: Unsaved peer failed. Deleting channel.
2023-10-31T15:15:57.464Z DEBUG   lightningd: io_break: destroy_plugin
2023-10-31T15:15:57.464Z DEBUG   connectd: Shutting down
2023-10-31T15:15:57.464Z DEBUG   gossipd: Shutting down
2023-10-31T15:15:57.464Z DEBUG   hsmd: Shutting down

Reported-By: @t-bast
2023-11-02 19:32:05 +10:30
niftynei fa8458c00a dualfund: add test to make sure that tx-sigs sent before commitment
results in an error.
2023-11-02 19:32:05 +10:30
niftynei 30ec8cbf8e dualfund, test: add test for dropping to chain during RBF
Here we make sure we can drop the initial tx to chain, and that an
inflight txid that's missing its commitment sigs is properly ignored.
2023-11-02 19:32:05 +10:30
niftynei da34c369f3 dualfund, tests: break out "peer forgets" test
Now that we save the commitment sigs immediately, we have to drop the
connection elsewhere in the flow to get the state where only one peer
remembers.
2023-11-02 19:32:05 +10:30
niftynei 5417312911 tests: update opening tests for new reconnect behavior
Let's test that things stay together!

One cool thing to note is that now we sort of "magically" recover from
pretty brutal disconnects!

Very nice!
2023-11-02 19:32:05 +10:30
niftynei 20c77419dc dualfund: split 'commit-received' into two parts
Here, we split up what was "commit_received" into two phases:
	- commit-ready, where we're about to send our commitment tx to
	  peer
	- commit-received, when we've gotten the commitment tx from our
          peer

This lets us do the right thing (as far as the spec is concerned) with
returning the correct 'next_funding_txid' on reconnect (later commits).
2023-11-02 19:32:05 +10:30
Rusty Russell f004952442 lightningd: wumbo is now the default, setting has no effect.
"Patrick, I'm sorry I doubted you."

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `large-channels` is now the default, wumbology for all.
2023-11-02 08:16:51 +01:00
Alex Myers 10bac49dac ld: add commit-fee-offset option, update config schema
Changelog-Added: Added option --commit-fee-offset to potentially reduce feerate update disagreements
2023-11-02 09:49:59 +10:30
Alex Myers 4265699fcd lightningd: add a feerate offset when updating feerates as opener
Adding a fee offset as the channel opener reduces the likelihood of a
disconnect by the peer do to slight variation in feerate calculation
between nodes.

Changelog-Fixed: Some peer disconnects due to update_fee disagreements are avoided.
2023-11-02 09:49:59 +10:30
Rusty Russell 28fd70a3d8 lightningd: rewrite anchor spend to use multiple UTXOs if needed.
Closes: #6747
Changelog-EXPERIMENTAL: Fixed anchor spending to be able to use more than one UTXO.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Rusty Russell b901f885e0 pytest: force anchor tests to use more than one UTXO.
They fail now, since one UTXO is not enough.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Rusty Russell 1d677bcba7 addpsbtoutput: allow command to specify output address.
Default is still to generate it.

Changelog-None: Introduced this release anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Chris Guida c6262189b7 cln-plugin: Add test for cln-plugin send_custom_notification
Also fix Makefile for rust plugin examples

Also add in a missing assert in the test_plugin_start test
2023-10-30 18:24:23 +01:00
Rusty Russell af971fd025 wait forwards: add in_htlc_id
Without this, we have no unique identifier for which forward happened.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 0317348479 pytest: add test for wait forwards.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 1d8af90b56 listforwards: add created_index and updated_index fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` fields `created_index` (old: `id`) and `updated_index`.
2023-10-28 15:48:26 +10:30
Rusty Russell c70a326f1f lightningd: add ordering and pagination to listsendpays.
Changelog-Added: JSON-RPC: `listsendpays` new parameters `index`, `start` and `limit`.
2023-10-28 15:48:26 +10:30
Rusty Russell 00ac1a9475 listsendpays: add created_index and updated_index fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sendpay`, `listsendpays`, `delpay` new fields `created_index` (old: `id`) and `updated_index`.
2023-10-28 15:48:26 +10:30
Rusty Russell 9ec6ac9922 lightningd: don't log that we're trying to pay if we're not.
Looking through logs I was surprise to see:

```
lightningd-1 2023-10-26T03:42:36.824Z INFO    lightningd: Sending 200000000msat over 1 hops to deliver 200000000msat
```

On a re-payment where we simply returned from sendpay immediately!  Move that log to later.
2023-10-28 15:48:26 +10:30
Rusty Russell f65c3cab75 wallet: use standard-style iterators for payments.
We used to have "unsaved" payments: now we don't we can use
our normal "iterator" pattern rather than returning arrays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Aditya Sharma 34692ec3c8 tests/test_misc.py: Add test_decode to test the decryption of bech32 encoding of emergency.recover 2023-10-27 16:05:28 +10:30
Dusty Daemon 990f4d0dad splice: Update test for new logs
The log messages were changed but the test fields weren’t updated, resulting in some test flakiness. Being more explicit with the log message we’re looking for should help.

Changelog-None
2023-10-27 13:20:33 +10:30
Rusty Russell f8e036b22d channeld: use anchors on peer's commitment(s) if we can't broadcast our own.
This means refactoring out some of the generic anchor info, from the
per-commitment-tx info (we can have at least two, perhaps more with
splicing!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell 2bf21daba7 pytest: test (fails!) that we push anchor on peer's commitment tx.
We try to use anchors to CPFP our own commitment, but what if they
get there first?  We also need to use anchors on the commitment
txs they broadcast.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Christian Decker 0c32183e8d pytest: Mark `test_plugin_startdir_lol` as `@unittest.skip()`
It isn't that flaky, but when it gets stuck we'll be waiting for 30m
for it to complete.
2023-10-26 15:51:07 +02:00
Christian Decker 23ce9a947d pytest: Mark test_exclude_adjacent_routehint as skip
It has about a 7% flakyness rate.
2023-10-26 15:51:07 +02:00
Christian Decker 8f4c997248 pytest: Mark `test_splice` as falky 2023-10-26 15:51:07 +02:00
Christian Decker 7deeff8404 pytest: Fix a falke in test_gossip_pruning
Fixes cause of https://github.com/ElementsProject/lightning/actions/runs/6518612194/job/17704479411
2023-10-26 15:51:07 +02:00
Christian Decker 5c12d14e11 pytest: Skip `test_commit_crash_splice` as it is too flaky 2023-10-26 15:51:07 +02:00
Chris Guida 88ac2dcb7e add test, update schemas, fix warning_custom_fallbacks 2023-10-26 19:11:17 +10:30