Commit Graph

1533 Commits

Author SHA1 Message Date
Rusty Russell 4745e7e3d0 pytest: make test_hook_dep tests more stable.
I think this is what Travis is having an issue with, but it work
fine locally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Michael Schmoock f3804daabb pytest: simplify test_openchannel_hook_chaining now order is deterministic. 2020-11-09 15:22:33 -06:00
Rusty Russell fb295ffb51 plugin: sort topological candidates by specified order.
We previously registered hooks up in who-replies-to-getmanifest-first
order, but then if any had dependencies it would scatter that order.

This allows users to manually set dependencies developers have
forgotten by specifying the plugins manually in their configuration or
cmdline.  This was an excellent consideration by @mschmook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell d429e21db3 pytest: test that we maintain load order unless hook deps require a change.
Suggested-by: @mschmook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell 852e14c947 plugins: check order once all plugins have returned from getmanifest.
This means we need to stop at this stage even in the runtime-loaded
case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell a4f79e8e7e pytest: test that we don't complain about missing dependencies at startup.
We fail this, because we check dependencies as they come in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell 62c52fe868 libplugin: add support for before and after deps on hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell e16ed0e207 pyln: add support for dependent hooks.
And use that to add simple tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
ZmnSCPxj jxPCSnmZ 0fd87b85da openingd/: Fail `fundchannel_start` if we already are, or will become, the fundee.
Fixes: #4108

Changelog-Fixed: Network: Fixed a race condition when us and a peer attempt to make channels to each other at nearly the same time.
2020-11-07 17:18:49 +00:00
ZmnSCPxj jxPCSnmZ 712e0796fc tests/test_connection.py: Test for #4108. 2020-11-07 17:18:49 +00:00
Michael Schmoock 4a8722bf1d pytest: state_change history 2020-11-06 14:47:04 -06:00
Michael Schmoock 68ce25c92d feat: adds timestamp to state_change notification 2020-11-06 14:47:04 -06:00
Michael Schmoock 67f4970eb1 pytest: test opener and closer of listpeers 2020-11-06 14:47:04 -06:00
Michael Schmoock c52efe0f37 pytest: channel state change cause and message 2020-11-06 14:47:04 -06:00
Rusty Russell ae4139ba10 lightningd: note whether payment secret is used or not.
This will allow nodes (with log-level=debug) to gather how many payments
are made without payment_secrets.  We need to know this so we know when
we can make them compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 12:03:14 +01:00
niftynei 4ea9d9e928 df: Pass new feerate options through to plugin, set reasonable bounds
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.

We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.

In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
2020-10-27 19:52:05 -05:00
Rusty Russell 1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 44f6b8a746 pytest: add notifications to tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Antoine Poinsot 85aa53c778 fuzz: add a runner Python script for fuzz targets
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot c149a88bbe fuzz: add a fuzz target for initial_channel
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 1aa94ccd9e fuzz: add a fuzz target for the close tx
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 33c7eccb3b fuzz: add a fuzz target for channel_id derivation
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot f0061d0606 fuzz: add a fuzz target for common/bip32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot a89a018475 fuzz: add a fuzz target for common/bigsize
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 5d987f2dec fuzz: add a target for base32/64 encoding
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 63b1ae54a1 fuzz: add a target for common/bech32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 2f90c45454 fuzz: add a fuzz target for amounts parsing
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
niftynei ede5f5be3c mfc: blackbox test for commitment vs funding tx feerates
Liquid is excluded because the mempool entry doesn't contain a 'weight'
field.
2020-10-21 11:24:16 +10:30
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
Rusty Russell 639eddf840 common/gossmap: digest private channel information too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 83aea6b2bb gossip_store: make private channels more similar to channel_announcement
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys).  This makes it far
easier for gossmap to handle local channels.

Backwards compatible update, since we update old stores.

We also fix devtools/dump-gossipstore to know about the tombstone markers.

Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell c6625943b5 pytest: test that route can see private channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
niftynei a7d4b8780e elementsd: disable dual-funding test; don't add non-witness utxo
Elementsd needs to be updated to v0.20.0 in order for this to work as
expected
2020-10-20 12:50:31 +10:30
niftynei b696ec89a5 df-open: use channel_id for openchannel_update and openchannel_signed
Be as specific as possible is a good rule for things, I think
2020-10-20 12:50:31 +10:30
niftynei 5fbd25f938 dual-fund: test (turned off) and plugin for executing an openchannel
Proof that v2 openchannel works, plus a plugin for sewing all the
requisite bits together
2020-10-20 12:50:31 +10:30
niftynei e2a6fd7112 common: pull up `param_psbt`
Usable other places, not just in wallet
2020-10-20 12:50:31 +10:30
niftynei 8f253b2adc test: simplify test for close_to
Since `fundchannel` now supports the 'close_to' argument, we can remove
all the logic needed to call fundchannel_start here.

Underneath, we're still calling `fundchannel_start`, we're just one (or
two, if you count multifundchannel) call levels away from it now.
2020-10-16 13:51:57 +10:30
niftynei de34f08b82 tests,fundchannel: return the result from the fundchannel rpc call
We need this so we can verify the 'close_to' result
2020-10-16 13:51:57 +10:30
Rusty Russell a3c30441d3 channeld: order htlcs by id before retransmission.
We had one report of this, and then Eugene and Roasbeef of Lightning
Labs confirmed it; they saw misordered HTLCs on reconnection too.

Since we didn't enforce this when we receive HTLCs, we never noticed :(

Fixes: #3920
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)
2020-10-14 19:58:57 +10:30
Rusty Russell d04597cbb6 pytest: add test for HTLC ordering.
[ Includes tidyups by Christian Decker ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-14 19:58:57 +10:30
Rusty Russell d5c91a347a channeld: log broken message if we receive HTLCs out of order.
We didn't care, but other implementations (particularly lnd) do.  And it
does violate the spec.

(We need to use skip not xfail on the test which catches this, since
xfail doesn't seem to stop errors reported by cleanup)

(Includes Christian's typo fix!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-14 19:58:57 +10:30
Rusty Russell 6195d953cc plugins: use "slow" feerate for mutual close negotiation.
We're rarely in a hurry here, and bitcoind is aggressive with fees.
You can always spend this output if you really have to, using CPFP.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: mutual closing feerate reduced to "slow" to avoid overpaying.
2020-10-13 20:53:34 +02:00
Christian Decker 9f8c9d6406 pytest: Check for connection close and reset errors in timeout test
I get both errors from time to time, seems like a timing issue.
2020-10-13 14:30:45 +02:00
Rusty Russell 79a21755c4 pytest: fix flaky test if one side hasn't processed close yet.
```
    @unittest.skipIf(TEST_NETWORK != 'regtest', "External wallet support doesn't work with elements yet.")
    def test_funding_close_upfront(node_factory, bitcoind):
...

        # check that you can provide a closing address upfront
        addr = l1.rpc.newaddr()['bech32']
>       _fundchannel(l1, l2, amt_normal, addr)
...
pyln.client.lightning.RpcError: RPC call failed: method: fundchannel_start, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 100000, 'announce': True, 'close_to': 'bcrt1qctx2k9cu9fd7nk449mzphqjcvvpyc4rxh6826x'}, error: {'code': -1, 'message': 'They sent error channel 2a1ca624cd1127761cb7a4395df2c3fd6d0abb3732c1f85a5345b0da716540d0: Multiple channels unsupported'}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-12 13:14:39 +02:00
Christian Decker 7d6892abba pytest: Verification mode of expanded stmts for the db_write hook
This can be used to find instances of unparseable expanded SQL statements like
the one fixed in #4090
2020-10-10 11:34:43 +10:30
Rusty Russell 711133e3c7 pytest: fix flaky test.
l2 only gives up 100 blocks after it ignores the HTLC, so sometimes
this test fails.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-09 15:20:03 +10:30
Rusty Russell 2d897be10a pytest: fix experimental test.
Broken back in 3b8c0a7397 which
changed the behavior of funding not to mark as immediately spent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-08 11:00:23 +10:30
Christian Decker ee0ee046ff pytest: Stabilize test_penalty_htlc_tx_timeout
lightning-5 can sometimes see itself sweeping the unilateral output resulting
in this weird line:

```log
HTLC already resolved by SELF when we found preimage
```
2020-10-07 09:36:30 +10:30
Christian Decker 9021bb26d1 pyln: Decode process output once before storing it 2020-10-07 09:36:30 +10:30