Commit Graph

13461 Commits

Author SHA1 Message Date
Rusty Russell e6d23b5677 pytest: test for emergency minimum fund violations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 3e57496af9 plugins: don't let multifundchannel open a new anchor channel without reserves.
If we're opening a channel with a peer which support anchors (and
we do), we tell fundpsbt/utxopsbt to enforce the emergency reserve;
this matters, as it doesn't know about the channel yet, and thus
won't (if it's our first anchor channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `fundchannel` and `multifundchannel` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (or are opening one).
2023-06-29 11:28:47 -04:00
Rusty Russell 0402e645f0 lightningd: add option `opening_anchor_channel` to fundpsbt, utxopsbt.
This is needed when we know we're *opening* an anchor channel, to
override the "do we already have an anchor channel open?" logic.

Also, document the nonwrapped arg added in v23.02.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundpsbt` and `utxopsbt` new parameter `opening_anchor_channel` so lightningd knowns it needs emergency reserve for anchors.
2023-06-29 11:28:47 -04:00
Rusty Russell 391da2f440 lightningd: don't let them fundpsbt below emergency reserve.
This is the simple version which always tries to keep some sats if we
have an anchor channel.  Turns out that we need something more
sophisticated for multifundchannel, so that's next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (and `all` will be reduced appropriately).
Changelog-Changed: JSON-RPC: `fundpsbt` and `utxopsbt` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels.
2023-06-29 11:28:47 -04:00
Rusty Russell 75aca3cbb6 config: add min-emergency-msat option.
For anchors, we need some sats sitting around in case we need to CPFP
a close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `min-emergency-msat` setting for (currently experimental!) anchor channels, to keep funds in reserve for forced closes.
2023-06-29 11:28:47 -04:00
Rusty Russell 2f0e13e793 multifundchannel: use `excess_as_change` flag to simplify logic.
This was added to fundpsbt/utxopsbt in v0.10, but the spender plugin
didn't take advantage of it, instead calculating its own change amount
and output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 37005b93d3 txprepare: use `excess_as_change` flag to simplify withdraw/txprepare logic.
This was added to fundpsbt/utxopsbt in v0.10, but the txprepare plugin
didn't take advantage of it, instead calculating its own change amount
and output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell e20ceba9e0 fundpsbt/utxopsbt: handle `excess_as_change` and `all` correctly.
If you did call fundpsbt with amount 'all' and `excess_as_change`
true, you would get everything going to the change output.  That's
obviously not the intention, and we'd like to use this to add change
outputs even for "all" when have keep emergency reserves.

And change the finish_psbt() API to take an explicit change amount:
at the moment it's either all or nothing, but that will change with
emergency-sat reserves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell d882a38619 plugin: make sure multifundchannel / fundchannel unreserve correctly on failure.
We were marking our inputs very late, which means any early failure
would not know to unreserve them.

This becomes particularly bad when we start enforcing emergency reserves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 7894d7136f pytest: adapt all the anchor-iff-EXPERIMENTAL tests to --experimental-anchors.
We use parameterization here.  The old `anchor_expected()` was for
non-zero-fee anchors, and have bitrotted so there are some other
changes as well.

Unfortunately, all the anchor accounting seems to be broken, but I
cannot understand these tests at all.  I had to simply disable them
for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell f64188f925 pytest: make dual-funding tests using option_anchors_zero_fee_htlc_tx.
They used to force option_anchor_outputs, so switch them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 825173f1b4 pytest: test CPFP using anchors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 17821da80b lightningd: turn on experimental support for zero-fee htlc anchors.
We disabled experimental support for opening non-zero-fee anchor
channels (though old nodes may still have such channels if they turned
that on!).

So we simply call this `experimental-anchors`, since this is the variant
which we expect to be used widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: protocol: added support for zero-fee-htlc anchors (`option_anchors_zero_fee_htlc_tx`), using `--experimental-anchors`.
2023-06-29 11:28:47 -04:00
Rusty Russell af6d7c0779 global: thread zero fee option everywhere.
In most cases, it's the same as option_anchor_outputs, but for
fees it's different.  This transformation is the simplest:
pass it as a pair, and test it explicitly.

In future we could rationalize some paths, but this was nice
and mechanical.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell e1a9e25412 common/channel_type: add zero-fee-anchor type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 73691100bf onchaind: don't grind for htlc fees if option_anchors_zero_fee_htlc_tx set.
The answer, it's right in the name of the option!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell a2039472c1 onchaind: use our own inputs to add to HTLC txs (iff using anchors).
This lets us RBF htlc txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 662b2687e2 hsmd: routine to sign HTLC tx merged with our own tx.
Since HTLC txs when using anchors are
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY, we can attach other inputs to
give it a higher feerate.  But we need the HSMd to actually sign the
combo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell a9d8f84c7e channeld: for anchors, use minrelayfee as minimum commit tx fee to allow.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 6087decec3 dualopend: use a lower feerate for first commitment tx for anchors.
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 96e198f751 openingd: use a lower feerate for first commitment tx for anchors.
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell e45bf14300 lightningd: use lowball feerate for commit_tx on anchor channels.
Since we can CPFP, we don't have to track the feerate as closely.  But
it still needs to get in the mempool, so we use 10 sat/byte, or the
100 block estimate if that is higher.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates` has new fields `unilateral_anchor_close` to show the feerate used for anchor channels (currently experimental), and `unilateral_close_nonanchor_satoshis`.
Changelog-Changed: JSON-RPC: `feerates` `unilateral_close_satoshis` now assumes anchor channels if enabled (currently experimental).
2023-06-29 11:28:47 -04:00
Rusty Russell dd76d60b0d dual-funding: don't override default if multifundchannel doesn't set feerate.
mfc->feerate_str is *never* NULL, since we set it in getfeerate; this is
confusing, as many places check for NULL.

Indeed, the logic in perform_fundpsbt() was *wrong* in this case: it used
`normal` (if it was NULL, which it never was) instead of `opening` to fundpsbt.

And the correct thing is for multifundchannel to not use a string here at
all, but to use the exact feerate it is counting on (even the same
string may have different values now if a block has come in).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 5332a8a67a lightningd: code to spend anchor outputs if we want to boost commitment tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell e036080fc5 common: simple helpers to test for either anchor feature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 0587cf324e lightningd/chaintopology: change form of finished callback.
We don't actually use it anywhere, but we actually want to now for
CPFP.  So give it more parameters and make it return bool so it can
be set without necessarily suppressing rexmit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 313354329d bitcoin/psbt: handle anchor spends.
Turns out it's a single sig, identical to the already-handled
case where we spend a to_remote output.

We also close a temporary memleak: stack was unused, but
tallocated off the psbt, so it lives as long as the PSBT.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell c2dffcab0f hsmd: command to sign anchor spends.
This is kind of a withdrawl to ourselves, except we also spend a
channel to-local anchor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell d895ef4fee pytest: test for commitment feerate.
Interesting we didn't actually test that the feerate we use is
actually delivered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell e47e51edf2 lightningd: don't make htlc_timeout_satoshis/htlc_success_satoshis zero if we support anchors.
It depends on whether we negotiated anchors: just document that this
field doesn't apply for anchors (it becomes zero by the end of this
patch series, which is weird).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 47fd31e8b4 lightningd: don't spam with RBF messages if fee hasn't changed.
Reported-by: @19710405 on GitHub
Fixes: #6283
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell cba310b983 bitcoin: set PSBT amount inside tx for bitcoin_tx_output_set_amount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell a5232659bf wallet: fix psbt_using_utxos when used with base PSBT.
We were setting the wrong input number: don't assume it's the
same as the UTXO number, but simply the last-appended input.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 3b1652842e msggen: add new version string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 7f265300c7 lightningd: ignore any new options we add in deprecated output.
It's weird to add new options into the deprecated section.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 5c6e706026 lightningd: initialize channel_type field on dualopend channel creation.
Otherwise a badly-timed listpeerchannels will crash.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 336dcef541 wallet: don't try to set a timer past 2038 on 32-bit platforms.
It'll wrap, probably be in the past, and infinite loop.  This was caused by an invoice
with expiry set at 2076.  This wrap caused us to think the expiry has already
passed, and keep looping!

Reported-by: @telelvis
Fixes: #6339
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightnind: don't infinite loop on 32 bit platforms if only invoices are expiring after 2038.
2023-06-29 06:54:58 -04:00
Bitkarrot 910630c497
add m1 native lib paths to Makefile
add m1 native lib paths to Makefile (also need PKG_CONFIG_PATH)

[ Squashed into a single commit --RR ]
2023-06-26 08:25:21 +09:30
avatar4d 856808aa2d Correct INSTALL.md requirements for OpenBSD
GNU xgettext is required to compile. It is found in the gettext-tools package in OpenBSD. Tested on OpenBSD 7.3
2023-06-26 08:18:28 +09:30
Christian Decker 6507d34b50 cln-grpc: Add some missing states to HtlcState 2023-06-23 14:38:23 +09:30
Rusty Russell b5845afd43 pytest: fix another flake in test_restorefrompeer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell 71adeccfbf pytest: fix reconnect flake in test_plugin_connected_hook_chaining
```
         l1.rpc.reject(l3.info['id'])
     
         l2.connect(l1)
         l1.daemon.wait_for_logs([
             f"peer_connected_logger_a {l2id}",
             f"{l2id} is allowed",
             f"peer_connected_logger_b {l2id}"
         ])
         assert len(l1.rpc.listpeers(l2id)['peers']) == 1
     
 >       l3.connect(l1)
 tests/test_plugin.py:468: 
... 
 >           raise RpcError(method, payload, resp['error'])
 E           pyln.client.lightning.RpcError: RPC call failed: method: connect, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'host': '127.0.0.1', 'port': 42391}, error: {'code': 402, 'message': 'disconnected during connection'}
 
 contrib/pyln-client/pyln/client/lightning.py:422: RpcError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell ffb0f03e1a lightningd: fix crash on shutdown while if channel being cancelled at same time.
```
 DEBUG   lightningd: Command returned result after jcon close
 DEBUG   connectd: Shutting down
 DEBUG   gossipd: Shutting down
 DEBUG   hsmd: Shutting down
 **BROKEN** lightningd: FATAL SIGNAL 6 (version b29955a-modded)
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/common/daemon.c:38 (send_backtrace) 0x55fea4a13a08
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/common/daemon.c:75 (crashdump) 0x55fea4a140a1
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea680651f
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea685aa7c
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea6806475
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea67ec7f2
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:95 (call_error) 0x55fea4bbef68
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:169 (check_bounds) 0x55fea4bbfa68
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:180 (to_tal_hdr) 0x55fea4bbdefe
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:193 (to_tal_hdr_or_null) 0x55fea4bbd9cf
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:461 (tal_alloc_) 0x55fea4bbd893
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:506 (tal_alloc_arr_) 0x55fea4bbdce6
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/str/str.c:91 (tal_vfmt_) 0x55fea4bbc95e
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/jsonrpc.c:502 (command_fail) 0x55fea499c427
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/channel.c:81 (destroy_channel) 0x55fea4976a31
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:246 (notify) 0x55fea4bbdc18
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:437 (del_tree) 0x55fea4bbe028
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:521 (tal_free) 0x55fea4bbde84
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/lightningd.c:577 (free_all_channels) 0x55fea49a2660
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/lightningd.c:1280 (main) 0x55fea49a1530
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea67edd8f
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea67ede3f
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55fea496ef04
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell f0a9f1100a poetry, CI: insist in protobuf v21.12 in both Python and CI.
And re-ran poetry update which updated the lock file ofc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 13:58:31 +09:30
Rusty Russell 69fc9c28bc Makefile: correctly erase generated contrib/pyln-testing/pyln/testing/ in distclean, and rebuild by default.
And add contrib/pyln-testing/pyln/testing/grpc2py.py since we didn't previously know
how to build it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 13:58:31 +09:30
Matt Whitlock 8d737cc4bf Makefile: use grouped targets for recipes with multiple fixed outputs
See the section headed "Rules with Grouped Targets" on the Texinfo page
`(make)Multiple Targets`.

Without this fix, Make does not know that these recipes unconditionally
make *all* of their named targets regardless of which target triggers
their execution, and Make will blissfully execute multiple instances of
any such recipe in parallel, not only wasting CPU cycles but potentially
producing incorrect results if the recipe is not atomic in its effects
on the file system.  With this fix, Make understands that it need only
execute such a recipe once to make all of its targets.

In pursuit of the above, move and combine two redundant msggen recipes
into the top-level Makefile, and populate its grouped targets from the
subordinate Makefiles.

Changelog-None
2023-06-23 13:58:31 +09:30
Vincenzo Palazzo fdb676491f pyln-proto: expose the tlv types
Exposing the tlv types to allow public access to it.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-06-23 13:32:25 +09:30
Matt Morehouse 8991f27497 fuzz: initial fuzz-cryptomsg seeds
Corpus generated by 300+ CPU hours of fuzzing and then minimized.
2023-06-23 13:12:42 +09:30
Matt Morehouse 1d94d2cd5d fuzz: test message encryption and decryption
Add a fuzz test for BOLT 8 message encryption and decryption. The fuzz
test is based on the unit test at common/test/run-cryptomsg.c and uses a
static initial state with fuzzer-generated messages to encrypt or
decrypt.
2023-06-23 13:12:42 +09:30
Matt Morehouse fca50eaebc common: make hdr parameter const
cryptomsg_decrypt_header doesn't modify hdr, so we should make it const.
2023-06-23 13:12:42 +09:30