Commit Graph

8870 Commits

Author SHA1 Message Date
Sergi Delgado Segura 3779c0b6ee pyln: Moves old zbase32 test from primitives 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura f497b90ee8 pyln: Adds tests to zbase32 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura 26f651f71f pyln: Adds one-liner docs to zbase32 functions 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura 1da29305fc pyln: Improve zbase32 encoding / decoding
- Adds bitarray filling so mesages of any length can be encoded, instead of forcing the encoding to be of messages with length multiple of 5.
- Adds checks for encoding / decoding and raises expections if the inputs are not as expected.
- Flags functions that are supposed to be internal as "private".
2020-10-22 13:56:57 +02:00
Rusty Russell 375040a3d9 Makefile: fix dependencies for check-pyln-%
$(BIN_PROGRAMS) is defined afterwards, leading to the problem that
lightningd doens't get rebuilt (in fact, it was running my installed
lightningd instead!).

It also needs $(PLUGINS) and the subdaemons, so hoist them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell 5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Antoine Poinsot 49dcb90b4e doc: add a document about our fuzz testing integration
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 85aa53c778 fuzz: add a runner Python script for fuzz targets
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot b3594690d9 configure: allow to run with UB sanitizer
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot c149a88bbe fuzz: add a fuzz target for initial_channel
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 31e50c9c25 contrib: remove ASAN suppressions
My bad, i comitted them as part of the LSAN suppressions while this data
race could have and had been fixed.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 1aa94ccd9e fuzz: add a fuzz target for the close tx
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 33c7eccb3b fuzz: add a fuzz target for channel_id derivation
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot f0061d0606 fuzz: add a fuzz target for common/bip32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot a89a018475 fuzz: add a fuzz target for common/bigsize
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 5d987f2dec fuzz: add a target for base32/64 encoding
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 07bb931fed common: remove b64_decode
We don't use it, and it's buggy (will always return NULL)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 63b1ae54a1 fuzz: add a target for common/bech32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot ae4dc231c1 amount: correctly parse amount strings we generate
This:
	- Allows `.*btc` amounts (without post-decimal)
	- Avoids creating decimals when amount is 0 btc
	- Corrects our handling of the suffixes (memeqstr would
	  sometimes return false because of null-termination)

Changelog-Fixed: We are now able to parse any amount string (XXXmsat, XX.XXXbtc, ..) we create.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 2f90c45454 fuzz: add a fuzz target for amounts parsing
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
niftynei ede5f5be3c mfc: blackbox test for commitment vs funding tx feerates
Liquid is excluded because the mempool entry doesn't contain a 'weight'
field.
2020-10-21 11:24:16 +10:30
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
niftynei f8c4cc73ae mfc: use declared constants
reduce, reuse, recycle for a greener world
2020-10-21 11:24:16 +10:30
niftynei e66937e012 mfc: Add a `commitment_feerate` optional parameter to multifundchannel
Technically there *are* two feerates that we need to know:
  - the feerate to use for the funding transaction, and
  - the feerate to tell our peer to use for our commitment txs/htlc txs

As written, `multifundchannel` uses the same feerate for both. This
optional parameter will allow us to differentiate between the two, which
will be exceedingly handy for anchor output worlds. ;)

FIXME: test this

Changelog-Added: JSON API: `multifundchannel` has a new optional argument, 'commitment_feerate', which can be used to differentiate between the funding feerate and the channel's initial commitment feerate
2020-10-21 11:24:16 +10:30
niftynei ae825fff26 opening: use correct dust_limit for reserve floor
Fixes #4140

Reported-By: @PsySc0rpi0n
Changelog-Fixed: openingd now uses the correct dust limit for determining the allowable floor for a channel open (affects fundee only)
2020-10-21 11:18:31 +10:30
niftynei 1b3a9be416 df, channeld: cleanup how psbt signalling works
We used to send our tx_sigs before we got to channeld existing. We
changed how this worked so that multifundchannel could live, but failed
to clean up the logic of what "having a psbt around" means wrt channeld
and messaging our peer.

The general idea is that we want to send `tx_signatures` to our peer on
reconnect until they've sent us `funding_locked`.

Note that it's an error to
  - send `funding_locked` without having sent `tx_signatures`
  - send `tx_signatures` after sending `funding_locked`

We use the 'finalized' state of the peer's inputs/outputs to help signal
where we are in receiving their sigs -- but this doesn't work at all for
opens where the peer doesn't contribute inputs at all.

This isn't really a huge deal, but it does mean that if we receive a
peer's `tx_sigs` more than once (will happen for a reconnect before
`funding_locked`), then we'll issue a notification about receiving their
sigs multiple times. /shrug
2020-10-21 09:04:32 +10:30
niftynei d535a27104 df, bugfix: wait til after we've saved the channel to do this
this cleans up `cmd` and we're not done with it yet (we need it for
saving the channel updates to the database)
2020-10-21 09:04:32 +10:30
niftynei c6d4bd676f dual-open,openchannel_update: include the index of the funding output
This allows us to do correct reporting via multiopenchannel :)
2020-10-21 09:04:32 +10:30
niftynei 9d412718df psbt: save the index of the change on the 'parent'
Note that for removals, the index will be on the original; for
additions, the index will be on the new. Yes this is implicit.
2020-10-21 09:04:32 +10:30
niftynei daa55d1221 df: add notification for receiving peer's funding tx sigs
This will allow us to build complex, multi-peer transactions, with
easeTM!

Changelog-Added: EXPERIMENTAL, Plugins: `openchannel_peer_sigs` notification, which contains a peer's signatures for the funding transaction (`opt_dual_fund`)
2020-10-21 09:04:32 +10:30
niftynei f9aab50ee8 dual-fund: rework where we send our tx-sigs message, allow peers in
Prior to this patch update, we expected a client to call
`openchannel_signed` before checking for peer's tx-sigs messages on the
wire.

When moving to a 'multifundchannel' approach, we'll need to be able to
collect sigs from our peers before sending our tx_sigs message. There's
no strict ordering on when tx-sigs messages are sent/received, so this
is fine.

To do this, we go ahead and start up channeld as soon as
commitment_sigs are secured, so that we process incoming tx-sigs from
our peers as soon as we get them.
2020-10-21 09:04:32 +10:30
niftynei c6ad4f9b20 channel.psbt: make non-const
We update it in the next patch, which technically breaks this contract.
So we shouldn't have the contract of const on this in the first place
then.
2020-10-21 09:04:32 +10:30
niftynei bdf1cc2f93 channeld-df: only send our sigs if we've got them 2020-10-21 09:04:32 +10:30
niftynei d1c7c78159 channeld-df: actually check serial_id of input when setting sigs
We're about to totally upset the order that sigs are set on our PSBTs
for new channel opens, making it such that our peer's sigs may arrive
before ours do.

We can no longer rely on the 'set witness means this is our input' since
there's no guarantee that our input sigs have been added yet, so we
check the serial_id and only set the stack on their (odd) inputs.
2020-10-21 09:04:32 +10:30
Rusty Russell f3bd57a088 common: cleanups suggested by Christian Decker's review.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell b470ae2c73 plugins/libplugin-pay: use gossmap.
This is a fairly direct translation.  Even so, it should be faster in
most cases, and and we can do more sophisticated things if we want.

This also handles disabled channels better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.
2020-10-21 08:58:34 +10:30
Rusty Russell eadf2c91fe libplugin-pay: incorporate gossip store.
So we can use this for routing determinations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 92f2461b5d plugins/pay: fix leak on failed new payments.
Start with attaching the payment to cmd (in case of failure), then steal
onto the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 639eddf840 common/gossmap: digest private channel information too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 83aea6b2bb gossip_store: make private channels more similar to channel_announcement
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys).  This makes it far
easier for gossmap to handle local channels.

Backwards compatible update, since we update old stores.

We also fix devtools/dump-gossipstore to know about the tombstone markers.

Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell c6625943b5 pytest: test that route can see private channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell b80342b928 gossmap: implement feature tests
Faster than pulling the announce msg and parsing.  We need this to test
if the node supports TLV.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 52c465fef0 common/gossmap: fix gossmap_node_get_announce() on unannounced nodes.
We would return junk before.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell bb9ad57a03 gossip_store: don't copy old delete markers on startup compact.
So we don't have to handle them at load time, either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 30bf6706b7 route: return NULL if destination is unreachable.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 1bf3eebbf6 dijkstra: fix heap ordering.
We were always ordering heap by distance, not score (which are different
if we are routing by cheapest, not shortest!).

This simplifies our callbacks, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
niftynei 4a1843a151 df, bugfix: use tal_wally around an allocation 2020-10-21 06:22:18 +10:30
niftynei 6d650064a0 df, nit: make this error message a little bit more informative 2020-10-21 06:22:18 +10:30