Commit Graph

12916 Commits

Author SHA1 Message Date
Rusty Russell 62fa91e23b lightningd: rebroadcast all pending txs each 30-60 seconds.
We also do it on every block, but since bitcoind can't always be counted
to rebroadcast for us, we might as well be aggressive!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 3754e283f8 lightningd: remember if they set "allowhighfees" when we rebroadcast.
We would only set it the first time, which was OK for how we were using it
before.  Now we want to also set it for rebroadcast.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 5582970715 lightningd: split the simple onchain tx signing code.
Splitting into onchaind_tx() into onchaind_tx_unsigned() and
sign_and_get_witness() makes it easier to reuse for RBF.

Adding more information in onchain_signing_info is required too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 819d9882aa lightningd: base feerate for onchain txs on deadlines, not fixed fees. 2023-04-10 07:31:12 +09:30
Rusty Russell 3a3370f4c1 feerates: add `floor` field for the current minimum feerate bitcoind will accept
Changelog-Added: JSON-RPC: `feerates`: added `floor` field for current minimum feerate bitcoind will accept
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 812a5a14c0 plugins/bcli: use the new feerate levels, and the floor.
Fixes: #4473
Changelog-Deprecated: Plugins: `estimatefees` returning feerates by name (e.g. "opening"); use `fee_floor` and `feerates`.
Changelog-Fixed: Plugins: `bcli` now tells us the minimal possible feerate, such as with mempool congestion, rather than assuming 1 sat/vbyte.
2023-04-10 07:31:12 +09:30
Rusty Russell 9e2d4240b1 lightningd: handle bcli plugins returning fee_floor and feerates parameters.
Changelog-Added: Plugins: `estimatefees` can return explicit `fee_floor` and `feerates` by block number.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell c46473e615 lightningd: allow "NNblocks" and "minimum" as feerates.
And consolidate descriptions into lightning-feerates().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` now allow "minimum" and NN"blocks" as `feerate` (`feerange` for `close`).
2023-04-10 07:31:12 +09:30
Rusty Russell 64b1ddd761 lightningd: clean up feerate handling, deprecate old terms.
Drop try_get_feerate() in favor of explicit feerate_for_deadline() and
smoothed_feerate_for_deadline().

This shows us everywhere we deal with old-style feerates by names.

`delayed_to_us` and `htlc_resolution` will be moving to dynamic fees,
so deprecate those.

Note that "penalty" is still used for generating penalty txs for
watchtowers, and "unilateral_close" still used until we get zero-fee
anchors.

Changelog-Added: JSON-RPC: `feerates` `estimates` array shows fee estimates by blockcount from underlying plugin (usually *bcli*).
Changelog-Changed: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) value *slow* is now 100 block-estimate, not half of 100-block estimate.
Changelog-Deprecated: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) expressed as, "delayed_to_us", "htlc_resolution", "max_acceptable" or "min_acceptable".  Use explicit block counts or *slow*/*normal*/*urgent*/*minimum*.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell cdb85d5618 lightningd: handle fees as blockcount + range.
Rather than have specific-purpose levels, have an array of
[blockcount, feerate], and rebuild the specific-purpose levels
for now on top.

We also keep a *separate* smoothed feerate, so you can ask for that
explicitly.

Since all the plugins used the same formula to derive the different
named fee levels, we apply the reverse to return to the underlying
estimates: updating the interface comes next.

This is ugly for now, but various specific-purpose levels will be
going away, as we shift to deadline-driven fees.

This temporarily breaks the floor calculation, so that test is
disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell faae44713b lightningd: clarify uses of dynamic (mempool) feerate floor, and static.
We have the FEERATE_FLOOR constant if you don't care, but usually you want
to use the current bitcoind lower limit, so call get_feerate_floor()
(which is currently the same, but coming!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 7aa8c76002 pytest: test parsefeerate explicitly.
Since we're messing with feerates, it's good to test this directly upfront.

Also, fix documentation!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell a2ca34ccf5 common: add tal_arr_insert helper to utils.h
We have tal_arr_remove and tal_arr_append already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 6799cd5d0b plugins/bcli: move commit-fee (dev-max-fee-multiplier) and into core.
Turns out the two bcli replacements I checked (`sauron` and
`trustedcoin`) don't even implement this, and the multiplier makes
more sense in lightningd, especially as we move to bcli just providing
raw feerate estimates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell b53cc69cfd msggen: fix incorrect assertion.
Adding a new field with `added` fails:

```
AssertionError: Field Feerates.perkb.estimates[] does not have an `added` annotation
```

Looks like this assertion is wrong: we should get an added from the field itself or
from the .msggen.json file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Matt Morehouse ca80dee514 doc: add section about improving fuzzing corpora
We want to encourage contributions to the seed corpora that improve
coverage.
2023-04-09 12:41:27 +09:30
Matt Morehouse 6e11a2e416 doc: document "make check-fuzz" 2023-04-09 12:41:27 +09:30
Matt Morehouse 14afa6efe7 make: add check-fuzz target
The target builds and runs each fuzz target on its seed corpus.
2023-04-09 12:41:27 +09:30
Matt Morehouse fa988d2942 fuzz: add check-fuzz.sh
The script runs each fuzz target on its seed corpus and prints any
failures.
2023-04-09 12:41:27 +09:30
Matt Morehouse a1a1373090 fuzz: improve corpus merging
The following arguments were copied from Bitcoin Core's corpus merging
script https://github.com/bitcoin/bitcoin/blob/master/test/fuzz/test_runner.py:

-shuffle=0
-prefer_small=1
-use_value_profile=1
2023-04-09 12:41:27 +09:30
Matt Morehouse 9384692e2a fuzz: add initial seed corpora
These corpora were generated with default libFuzzer flags with 30+ hours
of CPU time, and then minimized with:
  ./fuzz-TARGET -merge=1 -shuffle=0 -prefer_small=1 -use_value_profile=1 corpora/fuzz-TARGET UNMINIMIZED_CORPUS
2023-04-09 12:41:27 +09:30
Alex Myers 2f050621b0 pytest: add blackbox tests for reckless
A canned lightningd/plugins is used to test against. This allows faster
and more deterministic outcomes.

Changelog-None
2023-04-09 12:41:11 +09:30
Alex Myers cf203369bc reckless: use environment variable redirects
This will be used during CI testing in the following commit.
2023-04-09 12:41:11 +09:30
Alex Myers 55cddcd350 reckless: add support for additional networks
This should have been added earlier as @cdecker suggested, but is needed
to enable CI testing.

Changelog-Changed: Reckless - added support for networks beyond bitocoin and regtest
2023-04-09 12:41:11 +09:30
Alex Myers e61401aab9 reckless: don't crash on subprocess calls
They prefer Paths to be explicitly cast as strings
2023-04-09 12:41:11 +09:30
Rusty Russell 88905e8372 tests: split fetchinvoice recurrence tests into separate test.
This is for VLS, which doesn't implement signing for the non-standard
recurrence fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:23:14 +09:30
Rusty Russell d4ffc75691 Makefile: update to latest BOLT text.
In particular:
	- Bolt 4: add route blinding construction
	- Bolt 4: add blinded payments

And this means it's not experimental, so we can turn it on
by default!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: blinded payments are now supported by default (not just with `--experimental-onion-messages`)
2023-04-07 21:22:56 +09:30
Rusty Russell 458a85042b Makefile: update to BOLT 20066dc2aba906f37f3be5a810ae67040f265377
"BOLT 03: fix static-remote same amt and pre-image test vector"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell f26b1166b7 Makefile: update bolts a0bbe47b0278b4f152dbaa4f5fab2562413a217c
"BOLT 04: remove associated data from test vector"

(We actually use merge point).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell dfa6c0ca52 Makefile: bolt version b38156b9510c0562cf50f8758a64602cc0315c19
"Allow nodes to overshoot final htlc amount and expiry (#1032)"

Note that this also renamed `min_final_cltv_expiry` to the more-correct
`min_final_cltv_expiry_delta`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell fdf9b13bdb Makefile: update bolts fc40879995ebc61cc50dfd729512f17afb15b355.
"Allow nodes to overshoot the MPP `total_msat` when paying (#1031)"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Allow slight overpaying, even with MPP, as spec now recommends.
2023-04-07 21:22:56 +09:30
Rusty Russell 15f8e1e63c Makefile: update bolts to 60cfb5972ad4bec4c49ee0f9e729fb3352fcdc6a.
"BOLT 4: Remove legacy format, make var_onion_optin compulsory."

This also renamed the redundant "tlv_payload" to "payload", so we
replace "tlv_tlv_payload" with "tlv_payload" everyhere!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell 06d42694d5 wire: fix extracted patch.
Too much context, it didn't apply if you try to rebuild from source!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell a3ebc1bac4 lightningd: update comments now channel-type is merged.
It's in the main bolt, so remove qualifies from the bolt quote so they
are checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Vincenzo Palazzo bf9c4df0de test: add the timeout to the waitpay command
Inside our integration testing we get another timeout,
so this commit adds a timeout to the waitpay command to avoid waiting forever.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-04-07 11:55:31 +02:00
Anmol Agrawal dae92c5830 Update INSTALL.md 2023-04-07 16:01:47 +09:30
Rusty Russell a3b81ba17f onchaind: no longer need information about current feerates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell c1bc4d0ead onchaind: remove now-unused direct tx creation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 9496e9fbef onchaind: propose_ignore specifically to ignore if output reaches depth.
We do this for HTLCs which will timeout to them: we watch them in case we
want to fulfill them as a preimage comes in, but once they reach depth we
can forget about them.

We change the message, which causes some more test churn.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell c5b7dbcd98 pytest: clean up wait_for_onchaind_tx interface, remove wait_for_onchaind_broadcast
Using single tuples in Python is ugly, so:
1. Rename wait_for_onchaind_tx to wait_for_onchaind_txs.
2. Make it take tuples explicitly.
3. Make wait_for_onchaind_tx a simpler wrapper/unwrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 0c27acc705 onchaind: use lightningd to sign and broadcast htlc expired txs.
This is when they closed the channel, we can simply make our own tx to
expire the HTLC.  (The other case is where we closed the channel, and
we have a special htlc_timeout tx which we have their signature for).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 5bdd532e70 onchaind: use lightningd to sign and broadcast htlc_timeout transactions.
This breaks tests/test_closing.py::test_onchain_all_dust's accouting
checks.

That test doesn't really test what it claims to test; sure, onchaind
*says* it's going to ignore the output due to high fees, but the tx
still gets mined.

I cannot figure out what the test is supposed to look like, so I
simply disabled the accounting checks :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 868fa8ae81 onchaind: use lightningd to sign and broadcast htlc spending txs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell a9dfec0e71 onchaind: use lightningd to sign and broadcast htlc_success transactions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 2f6be4e6bb common: expose low-level htlc_tx function.
We'll want this, as lightningd will want to produce htlc txs based on
what it's told from onchaind, so we need a lower-level accessor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 3e53c6e359 onchaind: have lightningd create our penalty txs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 36dd70e677 onchaind, pytest: disable RBF logic.
We'll reimplement it once lightningd makes all the onchain txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 80cd6f0afe lightningd: remember depth of closing transaction.
We'll use this later to calculate deadlines for spending txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 9d5dfa7bdd onchaind: use lightningd for spending our unilateral "to us" output.
This follows the same pattern as the "spend htlc tx" in fact.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell 07413c20b9 onchaind: use lightningd to send "delayed_output_to_us" from HTLC txs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30