To reduce the surface area of amount of a channel balance that can be
eaten up as htlc dust, we introduce a new config
'--max-dust-htlc-exposure-msat', which sets the max amount that any
channel's balance can be added as dust
Changelog-Added: config: new option --max-dust-htlc-exposure-msat, which limits the total amount of sats to be allowed as dust on a channel
And initialize max to current height max when htlcs are already dead.
Turns out (thanks CI!) that MAX() of multiple columns is GREATEST() in
Postgres. That's clearer (MAX is used elsewhere for single columns),
so translate on the sqlite3 side.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We keep the now-removed chains field, and in deprecated mode, we set it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
Before:
Ten builds, laptop -j5, no ccache:
```
real 0m36.686000-38.956000(38.608+/-0.65)s
user 2m32.864000-42.253000(40.7545+/-2.7)s
sys 0m16.618000-18.316000(17.8531+/-0.48)s
```
Ten builds, laptop -j5, ccache (warm):
```
real 0m8.212000-8.577000(8.39989+/-0.13)s
user 0m12.731000-13.212000(12.9751+/-0.17)s
sys 0m3.697000-3.902000(3.83722+/-0.064)s
```
After:
Ten builds, laptop -j5, no ccache: 8% faster
```
real 0m33.802000-35.773000(35.468+/-0.54)s
user 2m19.073000-27.754000(26.2542+/-2.3)s
sys 0m15.784000-17.173000(16.7165+/-0.37)s
```
Ten builds, laptop -j5, ccache (warm): 1% faster
```
real 0m8.200000-8.485000(8.30138+/-0.097)s
user 0m12.485000-13.100000(12.7344+/-0.19)s
sys 0m3.702000-3.889000(3.78787+/-0.056)s
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds). This scheme
follows the latest draft, in which features are flagged compulsory.
We also add several helper functions.
Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This includes the new bolt11 test vectors, and also removes the
requirement that HTLCs be less than 2^32 msat. We keep that for now
because Electrum enforced it on receive: in two releases we will stop
that too.
So no longer warn about needing mpp in that case either.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Protocol: No longer restrict HTLCs to
@shesek points out that we called this field created_at in bolt11 decode,
which makes more sense anyway.
Changelog-EXPERIMENTAL: bolt12 decode `timestamp` field deprecated in favor of new name `created_at`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't support it (yet), but update the spec to include it.
We include the previous field (recurrence_signature) as a shim for the
moment, for compat with existing nodes. It's ugly, but next release
we'll stop *sending* it, then finally we'll stop accepting it!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As per latest spec revision.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: BOLT12 offers can now be unsigned, for really short QR codes.
This allows us to ensure a packet is read by the other end, but we
don't read anything else from them or write anything to them.
Using '+' is similar, but because it closes the connection, the peer
might notice before receiving the packet (such as if it does a write).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The test vectors in BIP-320 are for different networks. Write a quick
tool to turn them into regtest ones.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Tor v2 hidden services have been deprecated for a while:
https://blog.torproject.org/v2-deprecation-timeline .
This prevents user from being able to set them in the configuration
and to connect to them while still letting us be able to parse them
for gossip.
Changelog-Deprecated: lightningd: v2 Tor addresses. Use v3. See https://blog.torproject.org/v2-deprecation-timeline.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers. But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.
Weaken that check, and include config.h in C files before assert.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The links were rather annoying in the flowing text, so extract them
and add them to the footer of the release instead. This uses the
shorthand and footnote style of links.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
Don't include exp directly, use an ifdef in common/bolt12
(like we do for peer and onion wiregen files).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Still asserts that it's the standard size, but makes it a dynamic
member. For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.
We also include the plugin headers in ALL_C_HEADERS so they get
checked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys). This makes it far
easier for gossmap to handle local channels.
Backwards compatible update, since we update old stores.
We also fix devtools/dump-gossipstore to know about the tombstone markers.
Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were always ordering heap by distance, not score (which are different
if we are routing by cheapest, not shortest!).
This simplifies our callbacks, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.
includes a migration for existing channels
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids overwriting the ones in git, and generally makes things neater.
We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS. Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to change the API on the more complete JSON parser, so
make and use a simple API for the easy cases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This also means we subtract 660 satoshis more everywhere we subtract
the base fee (except for mutual close, where the base fee is still
used).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Update the `bitcoin_tx_add_input` interface to accept a witness script
and or scriptPubkey.
We save the amount + witness script + witness program (if known) to
the PSBT object for a transaction when creating an input.
now that witness script data is saved into the tx/psbt which is
serialized across the wire, there's no reason to use witscript to do
this. good bye witscript!
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward. This changes it to directly
access those fields.
This is not only more elegant (60 fewer lines), it would also be
more cache friendly. That's right: cache hot singles!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV. But how it does ecdh is daemon-specific.
We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh(). This just makes it more general.
So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently abuse the added_htlc and failed_htlc messages to tell channeld
about existing htlcs when it restarts. It's clearer to have an explicit
'existing_htlc' type which contains all the information for this case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This will help with the next patch, where we wean off using a global
for features: connectd.c has access to the feature bits.
Since connectd might now want to send a message, it needs the crypto_state
non-const, which makes this less trivial than it would otherwise be.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Normal output is suitable for feeding to devtools/onion, but for python tests
we want something simpler.
Ideally, we'd simply generate blinded paths in pyln.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
e.g.
$ PUBKEY1=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
$ PRIVKEY1=41bfd2660762506c9933ade59f1debf7e6495b10c14a92dbcd2d623da2507d3d
$ PUBKEY2=022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59
$ PRIVKEY1=c4a813f81ffdca1da6864db81795ad2d320add274452cafa1fb2ac2d07d062bd
# First line is blinding, second is contents and nodeids for onion.
$ ./devtools/blindedpath create $PUBKEY1 $PUBKEY2
03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00
# Generate the onion
$ /devtools/onion generate 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00 > /tmp/onion.dat
# First node unwraps it, gives next blinding and onion
$ ./devtools/blindedpath --first-node unwrap $PRIVKEY1 `cat /tmp/onion.dat` 03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a
Contents: 04210326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911
Next blinding: 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b
Next onion: 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0
# Feed that onion and blinding to second node
$ ./devtools/blindedpath unwrap $PRIVKEY2 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup':
fixup! devtool/blindedpath: primitive tool to make blinded onions.
On decode, don't mess with op.ephemeralkey, since it will be used to derive
the next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Christian Decker
niftynei
Rusty Russell
nathanael
Antoine Poinsot
fiatjaf
Jan Sarenik
Matt Whitlock
Sebastian Falbesoner
Dr. Maxim Orlovsky
Vasil Dimov