Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids overwriting the ones in git, and generally makes things neater.
We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS. Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV. But how it does ecdh is daemon-specific.
We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh(). This just makes it more general.
So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
e.g.
$ PUBKEY1=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
$ PRIVKEY1=41bfd2660762506c9933ade59f1debf7e6495b10c14a92dbcd2d623da2507d3d
$ PUBKEY2=022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59
$ PRIVKEY1=c4a813f81ffdca1da6864db81795ad2d320add274452cafa1fb2ac2d07d062bd
# First line is blinding, second is contents and nodeids for onion.
$ ./devtools/blindedpath create $PUBKEY1 $PUBKEY2
03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00
# Generate the onion
$ /devtools/onion generate 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00 > /tmp/onion.dat
# First node unwraps it, gives next blinding and onion
$ ./devtools/blindedpath --first-node unwrap $PRIVKEY1 `cat /tmp/onion.dat` 03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a
Contents: 04210326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911
Next blinding: 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b
Next onion: 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0
# Feed that onion and blinding to second node
$ ./devtools/blindedpath unwrap $PRIVKEY2 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup':
fixup! devtool/blindedpath: primitive tool to make blinded onions.
On decode, don't mess with op.ephemeralkey, since it will be used to derive
the next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now that we have json_stream in common/, we can move all the related
helpers from lightningd/json to common/json. This way everyone can
benefit of them (including libplugin, the plugins themselves,
potentially lightning-cli), not lightningd alone!
Note that the Makefile of the common/test/ had to be modified, because
the new helpers make use of common/wireaddr... Which turns out to
\#include <lightingd/lightningd.h> ! So we couldnt just include the .c
and add mocks if we redefined some structs (hello run-param).
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is an intermediary step: we still don't save it to the database,
but we do use the fee_states struct to track it internally.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).
This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
anything about "style".
The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This encoding scheme is no longer just used for short_channel_ids, so make
the names more generic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than reaching into data structures, let them register their own
callbacks. This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.
Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The `runtest` command takes a JSON onion spec, creates the onion and decodes
it with the provided private keys. It is fully configurable and can be used
for the test-vectors in the spec.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We need to hand -s to both header and body generation, or neither:
wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here
We also need it for printwire, otherwise we get static unused functions for subtypes:
devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows for complete channel simulation, including HTLC
transactions, but means we use higher-level primitives to
make the easy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These utilities allow us to create valid test txs and information given both
sides' complete set of secrets.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.
this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
Encapsulating the peer state was a win for lightningd; not surprisingly,
it's even more of a win for the other daemons, especially as we want
to add a little gossip information.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
added sanity check to make sure scid of csv is the same as scid in gossip.
Revised style, mem allocation, and error checks
[ Minor fixups, and updated benchmark script -- RR ]
With data.tar.gz: 456609740 Apr 2 12:33
store_load_msec:35300-42354(37118.2+/-2.7e+03)
vsz_kb:582832
store_rewrite_sec:12.700000-13.430000(12.988+/-0.27)
listnodes_sec:3.000000-3.160000(3.076+/-0.057)
listchannels_sec:30.790000-31.690000(31.03+/-0.34)
routing_sec:0.00
peer_write_all_sec:63.640000-67.860000(66.294+/-1.4)
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.
We didn't get much space saving in gossipd, even though we should save
24 bytes per node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.
The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
niftynei
Matt Whitlock
darosior
Christian Decker
Joe Netti