Commit Graph

12645 Commits

Author SHA1 Message Date
Vincenzo Palazzo fca62113f5 plugin: fetchinvoice: set the quantity in invreq
While the user trying to fetch an invoice by specifing the quantity we do
not work as expected.

Running the command

```
lightning-cli fetchinvoice -k offer='lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqffqszsk2p6hycmgv9ek2grpyphxjcm9ypmkjer8v46pyzmhd9jxwet5wvhxxmmdzsqs593pq0ylsvakdua5h976f4g3eautgjt3udvtyga47eaw7339sjrhpwpwz' quantity=2
```

and we answer back with

```json
{
   "code": -32602,
   "message": "quantity parameter required"
}
```

This is caused because we forget to bind the `quanity` field from the
RPC into the `invrequest`.

Reported-by: @aaronbarnardsound
Link: https://github.com/ElementsProject/lightning/issues/6089
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: fetchinvoice: fix: do not ignore the `quantity` field
into the invreq field.
2023-03-20 20:25:02 +00:00
Greg Sanders 6c641bdbbb test_backfill_scriptpubkeys: stop first cln node before second sub-test 2023-03-20 10:47:29 +10:30
Vincenzo Palazzo ba4f0c8dab ci: add timeout field to 2h for each task
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-20 10:43:35 +10:30
Justin Moon 57d21206db cln_plugin: add `shutdown()` method to `Plugin`
When plugins receive a "shutdown" notification, then can call this
method which will shutdown `cln_plugin`.

Then they can await `plugin.join()` and do any remaining cleanup
there.

This helps avoid a pain-point where plugin authors need to handle
2 separate plugin shutdown mechanisms https://github.com/ElementsProject/lightning/issues/6040
2023-03-18 11:52:29 +00:00
Rusty Russell acf01f4c09 pytest: don't run test_backfill_scriptpubkeys under valgrind in CI.
It seems that bitcoind frequently dies on this test.  I assume running
the multiple nodes under valgrind with the extra 214 blocks is too
memory-hungry?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 658bae30d5 lightningd: require "jsonrpc": "2.0" as per JSONRPC spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: require the `"jsonrpc": "2.0"` property (requests without this deprecated in v0.10.2).
2023-03-18 15:55:49 +10:30
Rusty Russell 9366e6b39f cleanup: rename json_add_amount_msat_only to json_add_amount_msat
Now there's no compat variant, we can rename this function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 983542f2a7 global: remove deprecated "msat" suffix on msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: the "msat" suffix on millisatoshi fields, as deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 780f32dfc6 global: remove deprecated non-msat-named msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: all the non-msat-named millisatoshi fields deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 67f23c19f7 lightningd: remove deprecated local_msat, remote_msat from listpeers.
Changelog-Removed: JSON-RPC: `listpeers`.`local_msat` and `listpeers`.`remote_msat` (deprecated v0.12.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 06b9009dd8 lightningd: remove deprecated behavior where checkmessage would fail quietly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `checkmessage` now always returns an error when the pubkey is not specified and it is unknown in the network graph (deprecated v0.12.0)
2023-03-18 15:55:49 +10:30
Rusty Russell 1c4f6ab2c5 hsmd: deprecate reply_v1.
We promised two versions after v0.12, and here we are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell fd04f46a92 common/hsm_version: list sha256 for every known version.
Makes it easier when we remove support for a version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Vincenzo Palazzo 4f3f3deab6 fix: partial fix lnprototest runner
This reintroduce lnprototest after 2 releases,
there was a lot of breaking around it and this
will patch them (most of them)!

However, there are some issue related to channel opening and closing
that need some additional love and are disabled for now, but I think it
is good to introduce lnprototest now again in the CI, to be able to
stress the fix for now and see if there are other problem around.

I will take care of it!

Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-18 15:53:29 +10:30
Alex Myers dcc66d58ab doc: update release procedure
Added clarification for sums signing, file ownership, and pyln publishing
as well as a reminder to update pyln version for the release.

Changelog-None
2023-03-17 14:26:56 +10:30
Rusty Russell bfc6fedfbf CHANGELOG.md: v23.03.2
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-14 15:44:50 +10:30
Rusty Russell cdf803cd6f plugins/pay: revert removal of paying invoice without description.
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT #11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1856.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-14 15:44:50 +10:30
Alex Myers be8ed8c7f0 meta: update changelog for v23.02.1
Changelog-None
2023-03-10 15:27:24 -06:00
Alex Myers 9e2287415f offers: enable label for invoicerequest 2023-03-09 06:56:06 -06:00
Rusty Russell cfbfe5d7ee doc: update documentation for fetchinvoice(7) and offer(7).
1. Don't refer to obsolete send_invoice flag.
2. Don't refer to obsolete quantity_min field.
3. Don't refer to unsigned vs signed offers: they're all unsigned.
4. Add references to invoicerequest(7).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell aea8184e58 doc: fix modern usage of sendinvoice (changed in v22.11)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell 5e394ef53f doc: add documentation for invoicerequest commands.
As reported on Discord, these are undocumented.  And thus, um, hard to find!

Reported-by: Aaron Barnard
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell df10c62508 chanbackup: even if they enable experimental-peer-storage, check peers
Seems like LND is hanging up on receiving these messages, even though
they're odd :(

So, when a peer connects, check if it supplies or wants peer backup
(even if it doesn't support both, it shouldn't hang up, and I didn't
want to separate the two paths).

And when we go to send our own, updated backup, check features before
sending.

Fixes: #6065
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `experimental-peer-storage` caused LND to hang up on us, so only send to peers which support it.
2023-03-08 18:46:21 -06:00
Rusty Russell 6c4a438afd wallet: really allow broken migrations.
e778ebb9af ("wallet: only log broken if we
have duplicate scids in channels.") downgraded the fatal() to a broken
log message, but the user reports it still won't start up.

Perhaps they're hitting the fatal() outside the loop?  (And we're
not getting that output).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-08 16:20:09 -06:00
Rusty Russell b5c614069b connectd: fix crash on freed context for new connections.
ccan/io stores the context pointer for io_new_conn, but we were using
`daemon->listeners` which we reallocate, so it can use a stale pointer.

```
0x3e1700 call_error
	ccan/ccan/tal/tal.c:93
0x3e1700 check_bounds
	ccan/ccan/tal/tal.c:165
0x3e1700 to_tal_hdr
	ccan/ccan/tal/tal.c:174
0x3e1211 to_tal_hdr_or_null
	ccan/ccan/tal/tal.c:186
0x3e1211 tal_alloc_
	ccan/ccan/tal/tal.c:426
0x3db8f4 io_new_conn_
	ccan/ccan/io/io.c:91
0x3dd2e1 accept_conn
	ccan/ccan/io/poll.c:277
0x3dd2e1 io_loop
	ccan/ccan/io/poll.c:444
0x3419fa main
	connectd/connectd.c:2081
```

Fixes: #6060
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 17:14:22 -06:00
Rusty Russell 194d37b70f gossipd: don't make new zombies, just prune channels as we did before.
This reverts us to the v22.11 behaviour, pending a revisit for the
next release.

Changelog-Changed: gossipd: revert zombification change, keep all gossip for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:15:22 -06:00
Rusty Russell aaa14846c6 gossipd: ignore zombie flag when loading gossip_store.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:15:22 -06:00
Rusty Russell 4bc5d6a0c5 pytest: remove zombie test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:15:22 -06:00
Rusty Russell 1e2bc665ae sql: fix nodes table update.
Without this patch, we only ever loaded the "nodes" table once, then
didn't see updates.

How this ever got past CI is a mystery; perhaps valgrind was so slow that
the updated node_announcement hit the gossmap before we even asked sql
on l3 about the nodes table?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `sql` nodes table now gets refreshed when gossip changes.
2023-03-06 16:14:28 -06:00
Rusty Russell df0661ce22 sql: fix bug where nodes table would get duplicate entries.
As soon as we apply the next commit, we get a new problem: the
delete code didn't work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-06 16:14:28 -06:00
Alex Myers 8c3baa98cf gossipd: remove zombie spam cupdate when resurrecting
Changelog-Fixed: gossip_store is no longer corrupted when resurrecting channels
2023-03-03 09:41:06 -06:00
Alex Myers 538a8d5c57 meta: update changelog and pyln version for 23.02 release
Changelog-None
2023-03-02 12:14:50 -06:00
Alex Myers 0707ffcab4 reprobuild: use pyenv for python installation
python-setuptools installation was flakey on some systems. Installing
with pyenv should provide a more reproducible build.

Changelog-None
2023-03-02 12:11:02 -06:00
Christian Decker c7fd13a460 repro: Add `protoc` dependency to repro-build 2023-03-02 12:11:02 -06:00
Alex Myers d1402e06f9 gossipd: load and store node_announcements correctly
Loading the gossip_store would not create a pending node announcement
when the node already had a zombie channel.  This would cause the node
announcement to attempt to be loaded, but fail because it had no
broadcastable channels.  Accepting a pending node announcement as when
normally loading from the channel corrects this.
`node_has_public_channels` taking into account zombie channels enables
this behavior.

Separately, node_announcements were still being flagged as zombies
in the gossip store despite that feature being removed.

Changelog-None
2023-03-01 15:36:13 -06:00
Alex Myers d5246e43bb gossipd: flag zombie channels when loading from gossip_store
Without inheriting zombie status, gossipd would allow regular channel updates
into the store until the pruning cycle hits (and the channel is properly
flagged) which is 3.5 days. Applying zombie status when reading channel
updates from the store prevents this.

Changelog-None
2023-03-01 15:36:13 -06:00
Alex Myers 07c04d247e gossipd: correct node_announcement order when zombifying channels
remove_chan_from_node already corrects the ordering if a node_announcement
is left ahead of the next oldest channel_announcement, but zombifying should
do that check (and reorder if necessary) too.

Changelog-None
2023-03-01 15:36:13 -06:00
Christian Decker f1c29aa3bd keysend: Do not strip even TLV types that were allowlisted
Changelog-Fixed: keysend: Keysend would strip even allowed extra TLV types before resolving, this is no longer the case.
2023-02-28 15:00:44 -06:00
Christian Decker 5dc85d185a keysend: Extract `accept-extra-tlv-types` from `listconfigs`
Since it is an optional field in the `listconfigs` output we can't use
the `rpc_scan` mechanism (doesn't handle optionals yet). We'll use
that list of accepted types later to avoid stripping them.
2023-02-28 15:00:44 -06:00
Christian Decker 29031c02ca libplugin: Expose the `jsonrpc_request_sync` method
This one was mostly used in libplugin, but not available outside. It
is rather useful, so let's expose it.
2023-02-28 15:00:44 -06:00
Christian Decker 4a38e37b59 json: Add method to parse a u64 array
This will be used to parse the extratlvs from `listconfigs` in
`keysend`, so we don't accidentally strip values we'd like to keep.
2023-02-28 15:00:44 -06:00
Rusty Russell 1426ac881b pytest: remove openchannel('v2') marker from test_sql
Now we'll test it in CI!  Manually set experimental-dual-fund.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Rusty Russell f3baa3e510 sql: fix crash on fresh node_announcment.
Missing quotes when we delete the old one!

Reported-by: Alex Myers
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Rusty Russell 855980641c sql: fix schema tests since num_channels added to listpeers.
Commit a418615b7f ("rpc: adds
num_channels to listpeers") broke the sql tests.  Turns out, no
openchannel v2 tests are run in CI!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-28 14:11:12 -06:00
Vincenzo Palazzo bcc94b2d43 fix: do not send send peerstorage msg when disabled
This commit will disable the peerstorage plugins
when the feature is not enabled.

I found this issue with lnprototest, and I guess
we did not find it with normal run because
other the unknown messages are ingored?

Changelog-Fixed: Disable the protocol messages when peerstorage is disabled.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-02-27 11:18:56 -06:00
Rusty Russell 355a7ae827 pay: fix delpay to actually delete.
It works for the trivial case, where groupid and partid are the same,
but silently deletes nothing in the other cases (or worse, deletes the
wrong entry!).

See: #5835
Changelog-Fixed: `delpay`: actually delete the specified payment (mainly found by `autoclean`).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-27 11:08:12 -06:00
Rusty Russell 38d90b2505 autoclean: fix timer crash when we're cleaning two things at once.
If we had two things to clean, we fired off two requests (eg.
listforwards and listinvoices) and both marked the timer as finished,
triggering an assert.

We already have a refcount for outstanding requests to avoid this
for e.g. outstanding del commands, so use it here too!

```
Jan 19 19:20:00  lightningd[748044]: autoclean: plugins/libplugin.c:445: timer_complete: Assertion `p->in_timer > 0' failed.
Jan 19 19:20:00  lightningd[748044]: autoclean: FATAL SIGNAL 6 (version v22.11.1)
Jan 19 19:20:00  lightningd[748044]: 0x562c388136e4 send_backtrace
Jan 19 19:20:00  lightningd[748044]:         common/daemon.c:33
Jan 19 19:20:00  lightningd[748044]: 0x562c3881376c crashdump
Jan 19 19:20:00  lightningd[748044]:         common/daemon.c:46
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0898d5f ???
Jan 19 19:20:00  lightningd[748044]:         ./signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0898ce1 __GI_raise
Jan 19 19:20:00  lightningd[748044]:         ../sysdeps/unix/sysv/linux/raise.c:51
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0882536 __GI_abort
Jan 19 19:20:00  lightningd[748044]:         ./stdlib/abort.c:79
Jan 19 19:20:00  lightningd[748044]: 0x7f26d088240e __assert_fail_base
Jan 19 19:20:00  lightningd[748044]:         ./assert/assert.c:92
Jan 19 19:20:00  lightningd[748044]: 0x7f26d0891661 __GI___assert_fail
Jan 19 19:20:00  lightningd[748044]:         ./assert/assert.c:101
Jan 19 19:20:00  lightningd[748044]: 0x562c3880355d timer_complete
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:445
Jan 19 19:20:00  lightningd[748044]: 0x562c38800b54 clean_finished
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:122
Jan 19 19:20:00  lightningd[748044]: 0x562c388010ed clean_finished_one
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:132
Jan 19 19:20:00  lightningd[748044]: 0x562c388011b6 del_done
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:149
Jan 19 19:20:00  lightningd[748044]: 0x562c388058b5 handle_rpc_reply
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:768
Jan 19 19:20:00  lightningd[748044]: 0x562c38805a39 rpc_read_response_one
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:944
Jan 19 19:20:00  lightningd[748044]: 0x562c38805ad7 rpc_conn_read_response
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:968
Jan 19 19:20:00  lightningd[748044]: 0x562c38876b60 next_plan
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/io.c:59
Jan 19 19:20:00  lightningd[748044]: 0x562c38876fe7 do_plan
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/io.c:407
Jan 19 19:20:00  lightningd[748044]: 0x562c38877080 io_ready
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/io.c:417
Jan 19 19:20:00  lightningd[748044]: 0x562c3887823c io_loop
Jan 19 19:20:00  lightningd[748044]:         ccan/ccan/io/poll.c:453
Jan 19 19:20:00  lightningd[748044]: 0x562c38805d11 plugin_main
Jan 19 19:20:00  lightningd[748044]:         plugins/libplugin.c:1801
Jan 19 19:20:00  lightningd[748044]: 0x562c38801c7a main
Jan 19 19:20:00  lightningd[748044]:         plugins/autoclean.c:613
```

Fixes: #5912
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-27 11:05:36 -06:00
niftynei e315f30728 db-fix: update NULL lease_satoshi fields to zero
Missed a DEFAULT in the db clause.

Feb 15 16:02:12 citrine lightningd[902093]: Accessing a null column lease_satoshi/15 in query SELECT  funding_tx_id, funding_tx_outnum, funding_feerate, funding_satoshi, our_funding_satoshi, funding_psbt, last_tx, last_sig, funding_tx_remote_sigs_received, lease_expiry, lease_commit_sig, lease_chan_max_msat, lease_chan_max_ppt, lease_blockheight_start, lease_fee, lease_satoshi FROM channel_funding_inflights WHERE channel_id = ? ORDER BY funding_feerate

Fixes #6016
2023-02-15 18:08:25 -06:00
Alex Myers 7079fb506f meta: Update changelog for 23.02rc3
Changelog-None
2023-02-15 16:07:45 -06:00
Alex Myers 463355de59 gossipd: remember to squelch node announcements when shuffling
Closing channels would previously require moving the node announcements
in the gossip store on occasion. They incorrectly lost their spam flag
during this process (would no longer be squelched.)

Changelog-None
2023-02-15 14:19:42 -06:00