Commit Graph

2954 Commits

Author SHA1 Message Date
Ken Sedgwick 33c838212e tests: augment test_onchain_their_unilateral_out to check to_remote witness generation
Changelog-Added: added a withdraw all to the end of test_onchain_their_unilateral_out to ensure that the unilateral close info is correct with anchors.  Tests https://github.com/Blockstream/greenlight/issues/348
2024-01-31 15:51:31 +01:00
Rusty Russell 1f9e9777f0 lightningd: don't generate node_announcements with identical timestamps.
This caused a flake in test_gossip_lease_rates, since the peer would ignore
the node_announcement due to duplicate timestamps!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 38ff9c6d74 pytest: don't trigger bad gossip message in test_routing_gossip_reconnect.
```
2024-01-29T21:26:50.9785559Z lightningd-1 2024-01-29T21:14:09.709Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Ignoring future channel_announcment for 110x1x0 (current block 109)
2024-01-29T21:26:50.9786945Z lightningd-1 2024-01-29T21:14:09.710Z UNUSUAL lightningd: Bad gossip order: could not find channel 110x1x0 for peer's channel update
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell fa7c0a7809 pytest: fix flake in test_wumbo_channels
We mine blocks too fast, and l3 discard the channel_announcment as too far in the future:

```
lightningd-3 2023-12-14T06:40:04.744Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 103x1x1 (current block 102)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 9f05250ee7 lightingd: corrections from Alex Myers's review.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell dd59d0617d lightningd: expose last_stable_connection in RPC.
This isuseful to find completely dead channels which are worthy of
closing.

Changelog-Added: JSON-RPC: `listpeerchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Changelog-Added: JSON-RPC: `listclosedchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 81a280d8a3 lightningd: expose `reestablished` field.
This is useful to see if we've really made progress, or just bounced
off the peer (e.g. it doesn't know the channel, or we have fallen
behind somehow).

Changelog-Added: JSON-RPC: `listpeerchannels` new field `reestablished` set once we've exchanged `channel_reestablish` messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell e4b21b467a lightningd: refine heuristics for remote address selection.
Rather than take the first two from peers with committed channels, use
the most common address given by at least 2 peers, and accept the majority
from non-committed peers if there are no committed peers.

This works well with the node_announcement rework, which waits until
everyone has a chance to connect before creating the node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 97ccf05117 lightningd: ignore gossip messages from channeld, switch to our own.
This commit is a bit messy, but it tries to do the minimal switchover.

Some tests change, so those are included here. 

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell d2e7831d87 pytest: fix flake in test_upgrade_statickey_onchaind
We need to make sure l1 has closed previous channel:

```
wait_for(lambda: only_one(l1.rpc.listpeerchannels()['channels'])['state'] == 'CHANNELD_NORMAL')

tests/test_connection.py:3866: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
tests/test_connection.py:3866: in <lambda>
    wait_for(lambda: only_one(l1.rpc.listpeerchannels()['channels'])['state'] == 'CHANNELD_NORMAL')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = [{'alias': {'local': '7000028x5013088x54781', 'remote': '3360520x6431919x56666'}, 'channel_id': 'fb198b27b1be882937eea...: ['static_remotekey/even']}, 'close_to': '5120888ab14b6e1655d1d00039b836d70b66e3351543ab6cd2f94166255f3d5e6cb5', ...}]

    def only_one(arr):
        """Many JSON RPC calls return an array; often we only expect a single entry
        """
>       assert len(arr) == 1
E       AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 13:39:23 +10:30
Rusty Russell 82819433bc pytest: test_gossip_pruning should not assume redundant node_announcement.
This will change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell 9d9c30331a pytest: prepare sql test for faster gossip generation.
Mining 13 blocks to close the l1->l2 channel also causes the l2->l3
channel to confirm, and I was reliably getting the
channel_announcement to l3 in time to confuse this wait_for test.  So
query the channel we want directly to make it more robust.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell a27bbbb734 pytest: fix up test_zeroconf_multichan_forward once routehints fixed.
After this, the invoice (correctly!) gives the zeroconf channel as a routehint,
so this test fails.  Simple workaround: make invoice before creating zeroconf
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell 412bb137e0 pytest: fix splicing tests to be more explicit, and not rely in private gossip messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell e30eadd6a9 pytest: check for short_channel_id, not status.
Status actually changes in this series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell 11fa3da4cc pytest: Remove check for to-be-removed message about msg from channeld.
This race is resolved with the gossipd changes (and this message is
also removed!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell 8f9af62228 pytest: make sure listincoming sees remote update in test_zeroconf_forward.
Otherwise we get no routehint.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell 64678c43ab pytest: make sure gossipd sees channel before we close it.
This was fine when gossipd saw local channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell bfa81f30e1 pytest: ignore private updates in test_addgossip.
This was triggering on private updates after gossip changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

1diff --git a/tests/test_gossip.py b/tests/test_gossip.py
index 677ec4825..285290b71 100644
2024-01-31 09:15:46 +10:30
Rusty Russell 0a8003a4d2 pytest: fix psbt db test.
When we rewrite, we actually use the remote channel_announcement signatures from the db, and notice that l1 here is actually l2:

```
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: gossipd rejected our channel announcement: WARNING: Bad node_signature_1 304402204ad9f0ca9b5f1f8eec61290b6008bf2c15f011e35543d25c35c5b89d2e1a183402201aaca5883714dbe11d6ae00a146fac7302352e1b9e987eee5ce98e8eb6a82909 hash e713698e14d3d8f973210e8d93c96d1a7c117c58340ccb195785a672fd0b5ef1 on channel_announcement 01004ad9f0ca9b5f1f8eec61290b6008bf2c15f011e35543d25c35c5b89d2e1a18341aaca5883714dbe11d6ae00a146fac7302352e1b9e987eee5ce98e8eb6a829095c05509264fcf72e19180f0285cf0ca0d786f280a75d79e46e5627be20c9dd9a4e9105aa61d7faf3a375454db94376b9b9df29cf078130ed827836b0849f119014ec750ab163baef701e7889f430ee37ea7e38857aa2b2745440600999c85f2903abefde5210dd06ae12f24b8801416800db84f5c73d35594af594f9001800025dee15519d29eb81e9d1ac761deebdad91ba1d08a05c027c16ea5fc08f3a9b663f8433209392395f0107f6b4d266c46b0946bd098f88c0693817fb560e0526a8000006226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00006900000100000266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c035180266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c0351802324266de8403b3ab157a09f1f784d587af61831c998c151bcc21bb74c2b2314b0206540493a98599967a61c45172df49681e398a6cb0890a506e74f76c2d2be242
```

Explicitly remove those messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Christian Decker 948ee4dbc5 pyln: Consolidate on pyln.testing.db
We had a complete copy of the `db_provider` and  associated classes in
teh `tests/` directory, causing them to drift apart. Consolidating on
one version makes this more maintainable.

Changelog-None
2024-01-29 19:13:12 +01:00
Tushar Vyas 464dc02b94 wipe_db() implementation 2024-01-29 19:13:12 +01:00
Rusty Russell af4b244f2f pytest: fix flake in test_peer_anchor_push
If l1 is one block behind, l2 goes onchain one block early:

```
        # Drops to chain
>       wait_for(lambda: only_one(l2.rpc.listpeerchannels(l3.info['id'])['channels'])['state'] == 'AWAITING_UNILATERAL')

tests/test_closing.py:3951:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

success = <function test_peer_anchor_push.<locals>.<lambda> at 0x7f799f9cd1f0>
timeout = 180

    def wait_for(success, timeout=TIMEOUT):
        start_time = time.time()
        interval = 0.25
        while not success():
            time_left = start_time + timeout - time.time()
            if time_left <= 0:
>               raise ValueError("Timeout while waiting for {}".format(success))
E               ValueError: Timeout while waiting for <function test_peer_anchor_push.<locals>.<lambda> at 0x7f799f9cd1f0>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 16:47:33 +10:30
Rusty Russell bf9cdc597e pytest: fix flake in test_custommsg_triggers_notification
If listpeers on l2 is called too fast, then it won't return any peers, and
we will fail:

```
        # Connect l1 to l2
        l1.connect(l2)
>       wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])

tests/test_misc.py:2690:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])
E   IndexError: list index out of range
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 16:47:33 +10:30
Rusty Russell 6778f320d2 openingd: make channel-type rejection message clearer.
For example, lnprototest got the error 'You gave bad parameters: Did not support channel_type ' which doesn't make it clear that it's rejecting the empty channel type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell b805899b06 pytest: check that listpeerchannels gives the same channel type as returned from fund/openchannel
Could have done this in an earlier commit, but that would be a messy rebase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell 17d0d006d2 fundchannel_start & multifundchannel: add channel_type.
Let's tell the caller what channel_type they got!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel`, `fundchannel_start` and `openchannel_init`: new field `channel_type`.
2024-01-29 13:40:34 +10:30
Rusty Russell 08e29ab47f lightningd: make option_static_remotekey compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

We still support channels opened without it, but you can no longer open new ones without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell e531ff83b3 lightningd: add dev-any-channel-type to allow obsolete channel types.
We're about to make static_remotekey compulsory, but we still want to
do tests for pre-existing channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell 3c2a57598f spenderp: add channel_type parameter to fundchannel / multifundchannel.
And add a request schema for multifundchannel.

Changelog-Added: JSON-RPC: `fundchannel` and `multifundchannel` now take an optional `channel_type` parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell e749aebbff fundchannel_start / openchannel_init: add a channel_type parameter to force channel type.
And add request schemas for openchannel_init and fundchannel_start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_start` and `openchannel_init` now take an optional `channel_type` parameter.
2024-01-29 13:40:34 +10:30
Rusty Russell f450dfeb55 lightningd: make option_gossip_queries compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell 88a2146664 lightningd: make option_data_loss_protect compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_data_loss_protect` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Lagrang3 b0054aad46 renepay: accomodate fees in the payment flow
Min. Cost Flow does not take into account fees when computing a flow
    with liquidity constraints.
    This is a work-around solution that reduces the amount on every route to
    respect the liquidity bound. The deficity in the delivered amount is
    solved by running MCF once again.

    Changes:

    1. the function `flow_complete` allocates amounts to send over the set of routes
       computed by the MCF algorithm, but it does not allocate more than liquidity
       bound of the route. For this reason `minflow` returns a set of routes that
       satisfy the liquidity bounds but it is not guaranteed that the total payment
       reaches the destination therefore there could a deficit in the delivery:
       `deficit = amount_to_deliver - delivering`.

    2. in the function `add_payflows` after `minflow` returns a set of routes we
       call `flows_fit_amount` that tries to a allocate the `deficit` in the routes
       that the MCF have computed.

    3. if the resulting flows pass all payment constraints then we update
        `amount_to_deliver = amount_to_deliver - delivering`, and the loop
        repeats as long as `amount_to_deliver` is not zero.

    In other words, the excess amount, beyond the liquidity bound,
    in the routes is removed and then we try to allocate it
    into known routes, otherwise we do a whole MCF again just for the
    remaining amount.

    Fixes issue #6599
2024-01-29 10:48:24 +10:30
niftynei a4f92eac81 bkpr: now that we're not doing empty acct logging, we dont need this bool
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.

Should make bookkeeper less chatty and remove noisy logs

Changelog-None
2024-01-29 10:05:03 +10:30
niftynei 5484aaee33 bkpr: dont log unknown accounts with zero balances
We were putting out a lot of empty journal entries. Let's
stop doing that.

Now the wallet balance stays uninitialized until/unless you have
funds in it.

Fixes #5672
2024-01-29 10:05:03 +10:30
Rusty Russell 277ed2ccbf libplugin: support version strings for deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell 4764ec794c pytest: note where we use very deprecated features, so tests don't rbeak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell 5ef4779edc lightningd: remove msatoshi alias for amount_msat.
Changelog-Removed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument (deprecated 0.12.0). Use `amount_msat`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Lagrang3 7c7d13d6dc renepay: add self-pay feature 2024-01-24 14:42:45 +10:30
Matt Morehouse 3260d7dd5e fuzz: initial fuzz-bolt12-invoice-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse 1c18269cd1 fuzz: initial fuzz-bolt12-invrequest-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse 2073d9771d fuzz: initial fuzz-bolt12-offer-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse 3af61e087c fuzz: initial fuzz-bolt12-bech32-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse 7d05f07a2b fuzz: test bolt12 decoding
Add fuzz targets for decoding of offers, invoice requests, and invoices.
Mutators are shared in bolt12.h.
2024-01-23 20:21:48 +10:30
Matt Morehouse b73f921a91 fuzz: test bolt12 bech32 decoding 2024-01-23 20:21:48 +10:30
Rusty Russell 09c1cfde39 pyln-client: don't automatically turn JSON into Millisatoshi class.
Now _msat fields are all integers (last conversion 23.08) we can simply
leave them alone, rather than trying to convert them.

And for turning Millisatoshi into JSON, we simply globally replace the
default encoding function to try ".to_json()" on items, which allows
anything to be marshalled.

The global replacement was interfering with other uses of JSON, such
as the clnrest plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-client: no longer autoconverts _msat field to Millisatoshi class (leaves as ints).
2023-12-16 10:22:01 +01:00
Rusty Russell 8b224b6ec2 pytest: don't assume that msat fields get turned into Millisatoshi.
Since the class interacts with int very well now, we don't have to
make explicit accesses, so it's easy to write code which works with
Millisatoshi or int.

Also, don't access rpc.decoder in one test, it's unnecessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-16 10:22:01 +01:00
Erik De Smedt 375985bf6c Implement tests for notifications on `custommsg` 2023-12-16 11:36:42 +10:30
Dusty Daemon 6f4a3c86e9 splice: Agressive restart testing during splices
Test node restarts in lots of phases of splice.

Changelog-None
2023-12-15 09:42:45 +10:30