Commit Graph

1517 Commits

Author SHA1 Message Date
Rusty Russell 1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 44f6b8a746 pytest: add notifications to tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Antoine Poinsot 85aa53c778 fuzz: add a runner Python script for fuzz targets
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot c149a88bbe fuzz: add a fuzz target for initial_channel
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 1aa94ccd9e fuzz: add a fuzz target for the close tx
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 33c7eccb3b fuzz: add a fuzz target for channel_id derivation
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot f0061d0606 fuzz: add a fuzz target for common/bip32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot a89a018475 fuzz: add a fuzz target for common/bigsize
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 5d987f2dec fuzz: add a target for base32/64 encoding
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 63b1ae54a1 fuzz: add a target for common/bech32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 2f90c45454 fuzz: add a fuzz target for amounts parsing
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
niftynei ede5f5be3c mfc: blackbox test for commitment vs funding tx feerates
Liquid is excluded because the mempool entry doesn't contain a 'weight'
field.
2020-10-21 11:24:16 +10:30
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
Rusty Russell 639eddf840 common/gossmap: digest private channel information too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 83aea6b2bb gossip_store: make private channels more similar to channel_announcement
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys).  This makes it far
easier for gossmap to handle local channels.

Backwards compatible update, since we update old stores.

We also fix devtools/dump-gossipstore to know about the tombstone markers.

Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell c6625943b5 pytest: test that route can see private channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
niftynei a7d4b8780e elementsd: disable dual-funding test; don't add non-witness utxo
Elementsd needs to be updated to v0.20.0 in order for this to work as
expected
2020-10-20 12:50:31 +10:30
niftynei b696ec89a5 df-open: use channel_id for openchannel_update and openchannel_signed
Be as specific as possible is a good rule for things, I think
2020-10-20 12:50:31 +10:30
niftynei 5fbd25f938 dual-fund: test (turned off) and plugin for executing an openchannel
Proof that v2 openchannel works, plus a plugin for sewing all the
requisite bits together
2020-10-20 12:50:31 +10:30
niftynei e2a6fd7112 common: pull up `param_psbt`
Usable other places, not just in wallet
2020-10-20 12:50:31 +10:30
niftynei 8f253b2adc test: simplify test for close_to
Since `fundchannel` now supports the 'close_to' argument, we can remove
all the logic needed to call fundchannel_start here.

Underneath, we're still calling `fundchannel_start`, we're just one (or
two, if you count multifundchannel) call levels away from it now.
2020-10-16 13:51:57 +10:30
niftynei de34f08b82 tests,fundchannel: return the result from the fundchannel rpc call
We need this so we can verify the 'close_to' result
2020-10-16 13:51:57 +10:30
Rusty Russell a3c30441d3 channeld: order htlcs by id before retransmission.
We had one report of this, and then Eugene and Roasbeef of Lightning
Labs confirmed it; they saw misordered HTLCs on reconnection too.

Since we didn't enforce this when we receive HTLCs, we never noticed :(

Fixes: #3920
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)
2020-10-14 19:58:57 +10:30
Rusty Russell d04597cbb6 pytest: add test for HTLC ordering.
[ Includes tidyups by Christian Decker ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-14 19:58:57 +10:30
Rusty Russell d5c91a347a channeld: log broken message if we receive HTLCs out of order.
We didn't care, but other implementations (particularly lnd) do.  And it
does violate the spec.

(We need to use skip not xfail on the test which catches this, since
xfail doesn't seem to stop errors reported by cleanup)

(Includes Christian's typo fix!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-14 19:58:57 +10:30
Rusty Russell 6195d953cc plugins: use "slow" feerate for mutual close negotiation.
We're rarely in a hurry here, and bitcoind is aggressive with fees.
You can always spend this output if you really have to, using CPFP.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: mutual closing feerate reduced to "slow" to avoid overpaying.
2020-10-13 20:53:34 +02:00
Christian Decker 9f8c9d6406 pytest: Check for connection close and reset errors in timeout test
I get both errors from time to time, seems like a timing issue.
2020-10-13 14:30:45 +02:00
Rusty Russell 79a21755c4 pytest: fix flaky test if one side hasn't processed close yet.
```
    @unittest.skipIf(TEST_NETWORK != 'regtest', "External wallet support doesn't work with elements yet.")
    def test_funding_close_upfront(node_factory, bitcoind):
...

        # check that you can provide a closing address upfront
        addr = l1.rpc.newaddr()['bech32']
>       _fundchannel(l1, l2, amt_normal, addr)
...
pyln.client.lightning.RpcError: RPC call failed: method: fundchannel_start, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 100000, 'announce': True, 'close_to': 'bcrt1qctx2k9cu9fd7nk449mzphqjcvvpyc4rxh6826x'}, error: {'code': -1, 'message': 'They sent error channel 2a1ca624cd1127761cb7a4395df2c3fd6d0abb3732c1f85a5345b0da716540d0: Multiple channels unsupported'}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-12 13:14:39 +02:00
Christian Decker 7d6892abba pytest: Verification mode of expanded stmts for the db_write hook
This can be used to find instances of unparseable expanded SQL statements like
the one fixed in #4090
2020-10-10 11:34:43 +10:30
Rusty Russell 711133e3c7 pytest: fix flaky test.
l2 only gives up 100 blocks after it ignores the HTLC, so sometimes
this test fails.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-09 15:20:03 +10:30
Rusty Russell 2d897be10a pytest: fix experimental test.
Broken back in 3b8c0a7397 which
changed the behavior of funding not to mark as immediately spent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-08 11:00:23 +10:30
Christian Decker ee0ee046ff pytest: Stabilize test_penalty_htlc_tx_timeout
lightning-5 can sometimes see itself sweeping the unilateral output resulting
in this weird line:

```log
HTLC already resolved by SELF when we found preimage
```
2020-10-07 09:36:30 +10:30
Christian Decker 9021bb26d1 pyln: Decode process output once before storing it 2020-10-07 09:36:30 +10:30
Christian Decker 81569de709 pytest: Stabilize test_setchannelfee_{restart,zero}
In both cases the flakyness arises from the destination not knowing about the
modified fees of the forwarding node, thus including the outdated details in
the routehint, and the sender being unlucky and always trying with the
routehint anyway.

The long-term solutions to this is going to be #4111, this commit just reduces
the flakyness to get back to business.
2020-10-07 09:36:30 +10:30
Christian Decker 1563bbc2fa pytest: Cleanup test_gossip_ratelimit
Drive-by code cleanups I stumbled over while investigating the issue fixed in
the previous commit.
2020-10-07 09:36:30 +10:30
Christian Decker 05934724dd pytest: Don't start 3 nodes in test_gossip_ratelimit
We really are just interested in their on-chain footprint, so actually
starting the nodes is pointless overhead, and caused a lot of flakyness due to
the output ordering sometimes not matching up. We now just use the `bitcoind`
API to fund, sign and send a raw transaction that matches the stashed gossip
messages.
2020-10-07 09:36:30 +10:30
Jan Sarenik a777d21fb7 pytest: Rename all fund_channel to fundchannel 2020-09-25 21:04:56 +02:00
Christian Decker 5a87e88ee5 pytest: Add test for failcode conversion
We added a conversion of failcodes that do not have sufficient information in
faac4b28ad. That means that a failcode that'd require additional information
in order to be a correct error to return in an onion is mapped to a generic
one since we can't backfill the information.

This tests that the mapping is performed correctly and replicates the
situation in #4070
2020-09-24 11:23:33 +09:30
Michael Schmoock 980a9517c6 fix: openchannel_hook log BROKEN on dup close_to
Changelog-None
2020-09-24 11:22:47 +09:30
Christian Decker d9ba39d2cb pytest: Add test for key-value style postgresql DSNs 2020-09-23 20:20:40 +09:30
niftynei 53ba34fe3a doc: correct weight numbers for fundpsbt 2020-09-17 10:14:22 +09:30
Rusty Russell f658dd0d78 pytest: test connection timeout.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-11 21:27:45 +09:30
Vincenzo Palazzo f62d7bbe45 Added additional fix to code formatting and English spelling.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
Vincenzo Palazzo 9fbeb9bcdc delpay: code style changes and fixed docs
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
ZmnSCPxj jxPCSnmZ 0eb1e7e0ca plugins/libplugin-pay.c: Add new payee_incoming_limit to limit number of HTLCs based on payee connectivity.
Fixes: #3926

(probably)

Changelog-Fixed: pay: Also limit the number of splits if the payee seems to have a low number of channels that can enter it, given the max-concurrent-htlcs limit.
2020-09-10 16:50:52 +09:30
ZmnSCPxj jxPCSnmZ 2d86014e53 tests/test_pay.py: Test for overload of number of HTLCs at the payee. 2020-09-10 16:50:52 +09:30
ZmnSCPxj jxPCSnmZ 9587425f2d plugins/spender/fundchannel.c: Make fundchannel a thin layer around multifundchannel.
Changelog-Changed: protocol: `fundchannel` now shuffles inputs and outputs, and no longer follows BIP69.
2020-09-10 15:28:50 +09:30
niftynei 9771273c4d elements: these can coin moves can arrive in either order 2020-09-10 12:31:36 +09:30
niftynei eab7fcf8e5 test_utxopsbt: account for fee output diff when elements 2020-09-10 12:31:36 +09:30