Commit Graph

12800 Commits

Author SHA1 Message Date
dependabot[bot] 48c334dc81 build(deps): bump werkzeug from 2.2.2 to 2.2.3
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3.
- [Release notes](https://github.com/pallets/werkzeug/releases)
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/werkzeug/compare/2.2.2...2.2.3)

---
updated-dependencies:
- dependency-name: werkzeug
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-22 10:31:21 +10:30
Rusty Russell 03c153ac0b channeld: don't spin trying to send commitment while waiting.
We would sleep for 10msec (default) and try again, spamming the logs
every second.  But we're waiting for revoke_and_ack, and that handler
already sets off the timer, so there's no need to spin at all!

Fixes: #6077
Changelog-Fixed: `channeld`: no longer spin and spam logs when waiting for revoke_and_ack.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 10:30:48 +10:30
Rusty Russell 2f188622b7 pytest: add timeout to test_feerate_stress.
This seems to be getting stuck in CI, so make sure we time out if it happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 10:30:23 +10:30
Michael Schmoock 9a3f69aecf connectd: log status_failed on TOR problems
This changes connectd to use `status_fail()` on TOR problems during statup
instead of `err()`. Using `err()` did not write to the logfile.

To find out TOR problems during startup, the user needed to stop the system
daemon and call `lightningd` manually in console to see the error.

`status_fail()` logs and exits, but also prints a whole stacktrace,
which is a bit too much imho on config errors. But currently there is
no `status_SOMETHING` method that logs, prints and exists on an error
without stacktrace.

Changelog-None
2023-03-22 09:42:13 +10:30
Kristaps Kaupe 906279a46e Output channel_id in listfunds
Changelog-Added: JSON-RPC: `listfunds` now has a `channel_id` field.
2023-03-21 10:29:33 +00:00
Vincenzo Palazzo fca62113f5 plugin: fetchinvoice: set the quantity in invreq
While the user trying to fetch an invoice by specifing the quantity we do
not work as expected.

Running the command

```
lightning-cli fetchinvoice -k offer='lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqffqszsk2p6hycmgv9ek2grpyphxjcm9ypmkjer8v46pyzmhd9jxwet5wvhxxmmdzsqs593pq0ylsvakdua5h976f4g3eautgjt3udvtyga47eaw7339sjrhpwpwz' quantity=2
```

and we answer back with

```json
{
   "code": -32602,
   "message": "quantity parameter required"
}
```

This is caused because we forget to bind the `quanity` field from the
RPC into the `invrequest`.

Reported-by: @aaronbarnardsound
Link: https://github.com/ElementsProject/lightning/issues/6089
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: fetchinvoice: fix: do not ignore the `quantity` field
into the invreq field.
2023-03-20 20:25:02 +00:00
Greg Sanders 6c641bdbbb test_backfill_scriptpubkeys: stop first cln node before second sub-test 2023-03-20 10:47:29 +10:30
Vincenzo Palazzo ba4f0c8dab ci: add timeout field to 2h for each task
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-20 10:43:35 +10:30
Justin Moon 57d21206db cln_plugin: add `shutdown()` method to `Plugin`
When plugins receive a "shutdown" notification, then can call this
method which will shutdown `cln_plugin`.

Then they can await `plugin.join()` and do any remaining cleanup
there.

This helps avoid a pain-point where plugin authors need to handle
2 separate plugin shutdown mechanisms https://github.com/ElementsProject/lightning/issues/6040
2023-03-18 11:52:29 +00:00
Rusty Russell acf01f4c09 pytest: don't run test_backfill_scriptpubkeys under valgrind in CI.
It seems that bitcoind frequently dies on this test.  I assume running
the multiple nodes under valgrind with the extra 214 blocks is too
memory-hungry?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 658bae30d5 lightningd: require "jsonrpc": "2.0" as per JSONRPC spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: require the `"jsonrpc": "2.0"` property (requests without this deprecated in v0.10.2).
2023-03-18 15:55:49 +10:30
Rusty Russell 9366e6b39f cleanup: rename json_add_amount_msat_only to json_add_amount_msat
Now there's no compat variant, we can rename this function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 983542f2a7 global: remove deprecated "msat" suffix on msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: the "msat" suffix on millisatoshi fields, as deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 780f32dfc6 global: remove deprecated non-msat-named msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: all the non-msat-named millisatoshi fields deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 67f23c19f7 lightningd: remove deprecated local_msat, remote_msat from listpeers.
Changelog-Removed: JSON-RPC: `listpeers`.`local_msat` and `listpeers`.`remote_msat` (deprecated v0.12.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 06b9009dd8 lightningd: remove deprecated behavior where checkmessage would fail quietly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `checkmessage` now always returns an error when the pubkey is not specified and it is unknown in the network graph (deprecated v0.12.0)
2023-03-18 15:55:49 +10:30
Rusty Russell 1c4f6ab2c5 hsmd: deprecate reply_v1.
We promised two versions after v0.12, and here we are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell fd04f46a92 common/hsm_version: list sha256 for every known version.
Makes it easier when we remove support for a version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Vincenzo Palazzo 4f3f3deab6 fix: partial fix lnprototest runner
This reintroduce lnprototest after 2 releases,
there was a lot of breaking around it and this
will patch them (most of them)!

However, there are some issue related to channel opening and closing
that need some additional love and are disabled for now, but I think it
is good to introduce lnprototest now again in the CI, to be able to
stress the fix for now and see if there are other problem around.

I will take care of it!

Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-18 15:53:29 +10:30
Alex Myers dcc66d58ab doc: update release procedure
Added clarification for sums signing, file ownership, and pyln publishing
as well as a reminder to update pyln version for the release.

Changelog-None
2023-03-17 14:26:56 +10:30
Rusty Russell bfc6fedfbf CHANGELOG.md: v23.03.2
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-14 15:44:50 +10:30
Rusty Russell cdf803cd6f plugins/pay: revert removal of paying invoice without description.
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT #11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1856.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-14 15:44:50 +10:30
Alex Myers be8ed8c7f0 meta: update changelog for v23.02.1
Changelog-None
2023-03-10 15:27:24 -06:00
Alex Myers 9e2287415f offers: enable label for invoicerequest 2023-03-09 06:56:06 -06:00
Rusty Russell cfbfe5d7ee doc: update documentation for fetchinvoice(7) and offer(7).
1. Don't refer to obsolete send_invoice flag.
2. Don't refer to obsolete quantity_min field.
3. Don't refer to unsigned vs signed offers: they're all unsigned.
4. Add references to invoicerequest(7).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell aea8184e58 doc: fix modern usage of sendinvoice (changed in v22.11)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell 5e394ef53f doc: add documentation for invoicerequest commands.
As reported on Discord, these are undocumented.  And thus, um, hard to find!

Reported-by: Aaron Barnard
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell df10c62508 chanbackup: even if they enable experimental-peer-storage, check peers
Seems like LND is hanging up on receiving these messages, even though
they're odd :(

So, when a peer connects, check if it supplies or wants peer backup
(even if it doesn't support both, it shouldn't hang up, and I didn't
want to separate the two paths).

And when we go to send our own, updated backup, check features before
sending.

Fixes: #6065
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `experimental-peer-storage` caused LND to hang up on us, so only send to peers which support it.
2023-03-08 18:46:21 -06:00
Rusty Russell 6c4a438afd wallet: really allow broken migrations.
e778ebb9af ("wallet: only log broken if we
have duplicate scids in channels.") downgraded the fatal() to a broken
log message, but the user reports it still won't start up.

Perhaps they're hitting the fatal() outside the loop?  (And we're
not getting that output).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-08 16:20:09 -06:00
Rusty Russell b5c614069b connectd: fix crash on freed context for new connections.
ccan/io stores the context pointer for io_new_conn, but we were using
`daemon->listeners` which we reallocate, so it can use a stale pointer.

```
0x3e1700 call_error
	ccan/ccan/tal/tal.c:93
0x3e1700 check_bounds
	ccan/ccan/tal/tal.c:165
0x3e1700 to_tal_hdr
	ccan/ccan/tal/tal.c:174
0x3e1211 to_tal_hdr_or_null
	ccan/ccan/tal/tal.c:186
0x3e1211 tal_alloc_
	ccan/ccan/tal/tal.c:426
0x3db8f4 io_new_conn_
	ccan/ccan/io/io.c:91
0x3dd2e1 accept_conn
	ccan/ccan/io/poll.c:277
0x3dd2e1 io_loop
	ccan/ccan/io/poll.c:444
0x3419fa main
	connectd/connectd.c:2081
```

Fixes: #6060
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 17:14:22 -06:00
Rusty Russell 194d37b70f gossipd: don't make new zombies, just prune channels as we did before.
This reverts us to the v22.11 behaviour, pending a revisit for the
next release.

Changelog-Changed: gossipd: revert zombification change, keep all gossip for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:15:22 -06:00
Rusty Russell aaa14846c6 gossipd: ignore zombie flag when loading gossip_store.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:15:22 -06:00
Rusty Russell 4bc5d6a0c5 pytest: remove zombie test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:15:22 -06:00
Rusty Russell 1e2bc665ae sql: fix nodes table update.
Without this patch, we only ever loaded the "nodes" table once, then
didn't see updates.

How this ever got past CI is a mystery; perhaps valgrind was so slow that
the updated node_announcement hit the gossmap before we even asked sql
on l3 about the nodes table?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `sql` nodes table now gets refreshed when gossip changes.
2023-03-06 16:14:28 -06:00
Rusty Russell df0661ce22 sql: fix bug where nodes table would get duplicate entries.
As soon as we apply the next commit, we get a new problem: the
delete code didn't work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:14:28 -06:00
Alex Myers 8c3baa98cf gossipd: remove zombie spam cupdate when resurrecting
Changelog-Fixed: gossip_store is no longer corrupted when resurrecting channels
2023-03-03 09:41:06 -06:00
Alex Myers 538a8d5c57 meta: update changelog and pyln version for 23.02 release
Changelog-None
2023-03-02 12:14:50 -06:00
Alex Myers 0707ffcab4 reprobuild: use pyenv for python installation
python-setuptools installation was flakey on some systems. Installing
with pyenv should provide a more reproducible build.

Changelog-None
2023-03-02 12:11:02 -06:00
Christian Decker c7fd13a460 repro: Add `protoc` dependency to repro-build 2023-03-02 12:11:02 -06:00
Alex Myers d1402e06f9 gossipd: load and store node_announcements correctly
Loading the gossip_store would not create a pending node announcement
when the node already had a zombie channel.  This would cause the node
announcement to attempt to be loaded, but fail because it had no
broadcastable channels.  Accepting a pending node announcement as when
normally loading from the channel corrects this.
`node_has_public_channels` taking into account zombie channels enables
this behavior.

Separately, node_announcements were still being flagged as zombies
in the gossip store despite that feature being removed.

Changelog-None
2023-03-01 15:36:13 -06:00
Alex Myers d5246e43bb gossipd: flag zombie channels when loading from gossip_store
Without inheriting zombie status, gossipd would allow regular channel updates
into the store until the pruning cycle hits (and the channel is properly
flagged) which is 3.5 days. Applying zombie status when reading channel
updates from the store prevents this.

Changelog-None
2023-03-01 15:36:13 -06:00
Alex Myers 07c04d247e gossipd: correct node_announcement order when zombifying channels
remove_chan_from_node already corrects the ordering if a node_announcement
is left ahead of the next oldest channel_announcement, but zombifying should
do that check (and reorder if necessary) too.

Changelog-None
2023-03-01 15:36:13 -06:00
Christian Decker f1c29aa3bd keysend: Do not strip even TLV types that were allowlisted
Changelog-Fixed: keysend: Keysend would strip even allowed extra TLV types before resolving, this is no longer the case.
2023-02-28 15:00:44 -06:00
Christian Decker 5dc85d185a keysend: Extract `accept-extra-tlv-types` from `listconfigs`
Since it is an optional field in the `listconfigs` output we can't use
the `rpc_scan` mechanism (doesn't handle optionals yet). We'll use
that list of accepted types later to avoid stripping them.
2023-02-28 15:00:44 -06:00
Christian Decker 29031c02ca libplugin: Expose the `jsonrpc_request_sync` method
This one was mostly used in libplugin, but not available outside. It
is rather useful, so let's expose it.
2023-02-28 15:00:44 -06:00
Christian Decker 4a38e37b59 json: Add method to parse a u64 array
This will be used to parse the extratlvs from `listconfigs` in
`keysend`, so we don't accidentally strip values we'd like to keep.
2023-02-28 15:00:44 -06:00
Rusty Russell 1426ac881b pytest: remove openchannel('v2') marker from test_sql
Now we'll test it in CI!  Manually set experimental-dual-fund.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Rusty Russell f3baa3e510 sql: fix crash on fresh node_announcment.
Missing quotes when we delete the old one!

Reported-by: Alex Myers
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Rusty Russell 855980641c sql: fix schema tests since num_channels added to listpeers.
Commit a418615b7f ("rpc: adds
num_channels to listpeers") broke the sql tests.  Turns out, no
openchannel v2 tests are run in CI!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Vincenzo Palazzo bcc94b2d43 fix: do not send send peerstorage msg when disabled
This commit will disable the peerstorage plugins
when the feature is not enabled.

I found this issue with lnprototest, and I guess
we did not find it with normal run because
other the unknown messages are ingored?

Changelog-Fixed: Disable the protocol messages when peerstorage is disabled.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-02-27 11:18:56 -06:00