Commit Graph

8840 Commits

Author SHA1 Message Date
niftynei 7a2e72f56c dualopend: handle dev-memleaks; fix placement of return on DEV options 2020-10-26 13:04:42 +10:30
niftynei 396c406983 dualopend: re-order 'handle_dev_memleak' occurrence 2020-10-26 13:04:42 +10:30
niftynei fe56f572ed df: finalize redeemscript at the same time as witness stack
libwally has a quirk where the finalize method will fail to 'completely'
finalize an input's parts if either the final_scriptsig or
final_redeemscript fields are set

since we manually set the final_witness stack here, we also need to
fully finalize the redeemscript -> final_scriptsig here as well.
2020-10-26 13:04:42 +10:30
niftynei 0ca6c3cc84 script: break out redeemscript->scriptsig function
We're going to finalze some redeemscripts here shortly, so break out the
ability to render an arbitrary redeemscript -> scriptsig
2020-10-26 13:04:42 +10:30
Michael Schmoock 6697ebcebb doc: update HACKING.md 2020-10-23 12:47:47 -05:00
Michael Schmoock 5b542195d3 doc: typo in lightning-setchannelfee.7
Changelog-None
2020-10-23 12:47:47 -05:00
Rusty Russell 1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell f395404a10 lightningd: infrastructure for internal notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 44f6b8a746 pytest: add notifications to tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 41290a436f lightning-cli: print notifications (with '# ' prefix) if received.
This can be suppressed with -N.

Note that we wull get an error with older lightningd, but we ignore it
anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: print notifications and progress bars if commands provide them.
2020-10-23 13:53:16 +10:30
Rusty Russell 806f208295 pyln: handle (ignore) notifications, and add notify_msg to send them.
We also sanity check that response id matches our request.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: pyln.client handles and can send progress notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell 572c849873 libplugin: ignore incoming notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 127811757f libplugin: support for sending notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: routines to send notification updates and progress.
2020-10-23 13:53:16 +10:30
Rusty Russell 9f687d60d9 lightningd: forward notifications from plugins if enabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell c732d8707a JSON-RPC: notifications command.
This lets callers enable notifications; we won't send any if they don't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `notifications` command to enable notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell a95205c25c common/json_stream: add generic double-cr helper.
And make caller of json_stream_forward_change_id use it, since
we're going to reuse that.

Also call json_out_finished here, so next object doesn't have a ","
prepended.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell d970addd4b ccan: update to latest version, get json_out_finished update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell becd4fe576 common: add routines for log level names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 0dec593aa8 lightningd: clean up close code now force is always true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Sergi Delgado Segura 9de5f438c4 pyln: Adds type annotations to zbase32 functions 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura 3779c0b6ee pyln: Moves old zbase32 test from primitives 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura f497b90ee8 pyln: Adds tests to zbase32 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura 26f651f71f pyln: Adds one-liner docs to zbase32 functions 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura 1da29305fc pyln: Improve zbase32 encoding / decoding
- Adds bitarray filling so mesages of any length can be encoded, instead of forcing the encoding to be of messages with length multiple of 5.
- Adds checks for encoding / decoding and raises expections if the inputs are not as expected.
- Flags functions that are supposed to be internal as "private".
2020-10-22 13:56:57 +02:00
Rusty Russell 375040a3d9 Makefile: fix dependencies for check-pyln-%
$(BIN_PROGRAMS) is defined afterwards, leading to the problem that
lightningd doens't get rebuilt (in fact, it was running my installed
lightningd instead!).

It also needs $(PLUGINS) and the subdaemons, so hoist them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell 5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Antoine Poinsot 49dcb90b4e doc: add a document about our fuzz testing integration
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 85aa53c778 fuzz: add a runner Python script for fuzz targets
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot b3594690d9 configure: allow to run with UB sanitizer
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot c149a88bbe fuzz: add a fuzz target for initial_channel
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 31e50c9c25 contrib: remove ASAN suppressions
My bad, i comitted them as part of the LSAN suppressions while this data
race could have and had been fixed.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 1aa94ccd9e fuzz: add a fuzz target for the close tx
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 33c7eccb3b fuzz: add a fuzz target for channel_id derivation
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot f0061d0606 fuzz: add a fuzz target for common/bip32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot a89a018475 fuzz: add a fuzz target for common/bigsize
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 5d987f2dec fuzz: add a target for base32/64 encoding
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 07bb931fed common: remove b64_decode
We don't use it, and it's buggy (will always return NULL)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 63b1ae54a1 fuzz: add a target for common/bech32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot ae4dc231c1 amount: correctly parse amount strings we generate
This:
	- Allows `.*btc` amounts (without post-decimal)
	- Avoids creating decimals when amount is 0 btc
	- Corrects our handling of the suffixes (memeqstr would
	  sometimes return false because of null-termination)

Changelog-Fixed: We are now able to parse any amount string (XXXmsat, XX.XXXbtc, ..) we create.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 2f90c45454 fuzz: add a fuzz target for amounts parsing
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
niftynei ede5f5be3c mfc: blackbox test for commitment vs funding tx feerates
Liquid is excluded because the mempool entry doesn't contain a 'weight'
field.
2020-10-21 11:24:16 +10:30
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
niftynei f8c4cc73ae mfc: use declared constants
reduce, reuse, recycle for a greener world
2020-10-21 11:24:16 +10:30
niftynei e66937e012 mfc: Add a `commitment_feerate` optional parameter to multifundchannel
Technically there *are* two feerates that we need to know:
  - the feerate to use for the funding transaction, and
  - the feerate to tell our peer to use for our commitment txs/htlc txs

As written, `multifundchannel` uses the same feerate for both. This
optional parameter will allow us to differentiate between the two, which
will be exceedingly handy for anchor output worlds. ;)

FIXME: test this

Changelog-Added: JSON API: `multifundchannel` has a new optional argument, 'commitment_feerate', which can be used to differentiate between the funding feerate and the channel's initial commitment feerate
2020-10-21 11:24:16 +10:30
niftynei ae825fff26 opening: use correct dust_limit for reserve floor
Fixes #4140

Reported-By: @PsySc0rpi0n
Changelog-Fixed: openingd now uses the correct dust limit for determining the allowable floor for a channel open (affects fundee only)
2020-10-21 11:18:31 +10:30
niftynei 1b3a9be416 df, channeld: cleanup how psbt signalling works
We used to send our tx_sigs before we got to channeld existing. We
changed how this worked so that multifundchannel could live, but failed
to clean up the logic of what "having a psbt around" means wrt channeld
and messaging our peer.

The general idea is that we want to send `tx_signatures` to our peer on
reconnect until they've sent us `funding_locked`.

Note that it's an error to
  - send `funding_locked` without having sent `tx_signatures`
  - send `tx_signatures` after sending `funding_locked`

We use the 'finalized' state of the peer's inputs/outputs to help signal
where we are in receiving their sigs -- but this doesn't work at all for
opens where the peer doesn't contribute inputs at all.

This isn't really a huge deal, but it does mean that if we receive a
peer's `tx_sigs` more than once (will happen for a reconnect before
`funding_locked`), then we'll issue a notification about receiving their
sigs multiple times. /shrug
2020-10-21 09:04:32 +10:30
niftynei d535a27104 df, bugfix: wait til after we've saved the channel to do this
this cleans up `cmd` and we're not done with it yet (we need it for
saving the channel updates to the database)
2020-10-21 09:04:32 +10:30
niftynei c6d4bd676f dual-open,openchannel_update: include the index of the funding output
This allows us to do correct reporting via multiopenchannel :)
2020-10-21 09:04:32 +10:30