Commit Graph

14383 Commits

Author SHA1 Message Date
Rusty Russell f2cf353431 common: gossmap method to load fd directly, not filename.
And helpers to tell if a node_announcement exists, and get a
full channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-04 09:24:44 +10:30
Rusty Russell 870c996628 gossipd: new routines to support gossmap compatibility.
gossip_store_del - takes a gossmap-style offset-of-msg not offset-of-hdr.
gossip_store_flag: set an arbitrary flag on a gossip_store hdr.
gossip_store_get_timestamp/gossip_store_set_timestamp: access gossip_store hdr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-04 09:24:44 +10:30
Rusty Russell 37ccca5d69 common/gossmap: remove now-unused private flag.
The only way you'll see private channel_updates is if you put them
there yourself with localmods.

I also renamed the confusing gossmap_chan_capacity to gossmap_chan_has_capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-04 09:24:44 +10:30
evansmj 5544911387 Convert fee result into msats 2024-02-02 17:31:23 +01:00
evansmj c90ca104ae Remove update_count from find_account_onchain_fees
update_count is just the count of the records for a tx.  To calculate onchain fees
for an account we must sum all credits vs debits.  We don't need to GROUP BY update_count
nor ORDER BY update_count since it is just a running index of updates to this tx.
2024-02-02 17:31:23 +01:00
evansmj 78ba4138d4 Remove grouping by update_count in finding onchain fees
Remove grouping by update_count which resulted in a crash due to bad arithmetic
caused by fee calculation returned rows not being consolidated.
Remove xfail.
2024-02-02 17:31:23 +01:00
evansmj 3b0cc28d4c Add xfail tests for onchain fees lookup query
Add test for single funded channels.
Add test for dual funded channels.
2024-02-02 17:31:23 +01:00
Lagrang3 9abd5ecf45 regtest: error message if bitcoin-cli is not found
If $BITCOIN_BIN is empty and bitcoin-cli or bitcoind are not found
startup_regtest will shutdown and display a corresponding error message.

Similarly if lightning-cli and lightningd are not found.
2024-02-02 17:25:50 +01:00
Rusty Russell 3c9fd0ca5b pytest: fix another gossip flake.
```
2024-02-02T10:30:38.6479143Z __________________ ERROR at teardown of test_multifunding_one __________________
...
2024-02-02T10:30:38.6491895Z >           raise ValueError(str(errors))
2024-02-02T10:30:38.6492208Z E           ValueError:
2024-02-02T10:30:38.6492465Z E           Node errors:
2024-02-02T10:30:38.6492833Z E            - lightningd-2: had bad gossip messages
2024-02-02T10:30:38.6493535Z E           Global errors:
...
2024-02-02T10:30:38.7458545Z lightningd-2 2024-02-02T10:25:48.410Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 103x2x1 (current block 102)
2024-02-02T10:30:38.7460140Z lightningd-2 2024-02-02T10:25:48.411Z UNUSUAL lightningd: Bad gossip order: could not find channel 103x2x1 for peer's channel update
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-02 16:14:53 +01:00
Rusty Russell a04fdfbe89 pytest: fix gossip_store load test for EXPERIMENTAL_SPLICING=1
It can change the node_announcement size, so don't check that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-02 16:14:53 +01:00
Rusty Russell 858c084f54 lightningd: fix reference to uninitialized channel_type during opening.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-02 16:14:53 +01:00
Lagrang3 59935c535c fixed source (using raw millisatohis) 2024-02-02 11:28:47 +01:00
Lagrang3 997d5aa970 renepay: flow routes limited by htlc_max/min
The flow routes returned by minflow are bounded by the htlc_min and
    htlc_max along the route whenever possible.
2024-02-02 11:28:47 +01:00
Christian Decker 5924509813 rpc: Regenerate derived files after adding bkpr-listincome 2024-02-01 20:00:52 +01:00
Christian Decker 19af808f45 msggen: Add classes for MethodName and TypeName
This is required for types and methods with names that need
post-processing (`bkpr-listincome`).
2024-02-01 20:00:52 +01:00
Harshit933 85b79bc2e1 fix : implements the `bkpr-listincome` plugin for grpc interface.
Changelog-added support for the `bkpr-listincome` for grpc interface

Signed-off-by: harshit933 <harshitverma933@gmail.com>
2024-02-01 20:00:52 +01:00
Christian Decker 6bf97ac259 ci: Split out the fuzz testing
Tacking the fuzz testing onto the PR testing not a good idea. It
causes completely unrelated tests to fail, and doesn't guarantee it
will find newly introduced ones. Running daily is likely better.

Changelog-None No user visible change
2024-02-01 19:46:57 +01:00
Christian Decker f39a43cf14 pyln: Include py.typed file 2024-02-01 17:55:10 +01:00
Matt Morehouse 968d6d6d24 configure: abort on UBSan failure
By default, UBSan reports runtime errors but does not stop execution.
We already abort in debug builds, and this commit makes us also abort in
regular builds when UBSan is enabled. Arguably, this is what users
expect when they enable UBSan, so it is a good default.

I know I've missed some UBSan bugs in the past because of this issue,
and dergoegge mentioned that this also happened to him.
2024-02-01 14:52:39 +10:30
Ken Sedgwick d759f3b758 channeld: use explicit revoke on retry when HSM_VERSION >= 5
Changelog-None: fixes hole in ([#7010]), see ([#7026])

Explicit hsmd_revoke_commitment_tx instead of get_per_commitment_point
needed when revokes are retried. The hsmd_revoke_commitment_tx message
is idempotent.
2024-02-01 14:38:58 +10:30
Erik De Smedt 085960b054 Rename `FeatureBitsPlace` to `FeatureBitsKind` 2024-01-31 21:27:47 +01:00
Erik De Smedt 870e25e180 cln-plugin: Allow user to set featurebits 2024-01-31 21:27:47 +01:00
ok300 a38d81dee0 cln-grpc: Derive serde Serialize/Deserialize for types 2024-01-31 16:44:02 +01:00
Ken Sedgwick 33c838212e tests: augment test_onchain_their_unilateral_out to check to_remote witness generation
Changelog-Added: added a withdraw all to the end of test_onchain_their_unilateral_out to ensure that the unilateral close info is correct with anchors.  Tests https://github.com/Blockstream/greenlight/issues/348
2024-01-31 15:51:31 +01:00
22388o⚡️ c7e5d8049e Update LICENSE 2024-01-31 13:33:30 +01:00
Erik De Smedt 71c343e2d1 cln_plugin: custommessages in `get_manifest` 2024-01-31 13:26:19 +01:00
Lagrang3 42a7b63d30 update the documentation 2024-01-31 13:02:51 +01:00
Lagrang3 28c28e4ea9 add --show-secrets option for hsmtool dumponchain 2024-01-31 13:02:51 +01:00
Lagrang3 a1c02e8098 hsmtool: add feature to dump private keys 2024-01-31 13:02:51 +01:00
Vincenzo Palazzo 12a285f5dd ci: fix the CI configuration for rust release
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-01-31 12:09:01 +01:00
Rusty Russell 1f9e9777f0 lightningd: don't generate node_announcements with identical timestamps.
This caused a flake in test_gossip_lease_rates, since the peer would ignore
the node_announcement due to duplicate timestamps!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 38ff9c6d74 pytest: don't trigger bad gossip message in test_routing_gossip_reconnect.
```
2024-01-29T21:26:50.9785559Z lightningd-1 2024-01-29T21:14:09.709Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Ignoring future channel_announcment for 110x1x0 (current block 109)
2024-01-29T21:26:50.9786945Z lightningd-1 2024-01-29T21:14:09.710Z UNUSUAL lightningd: Bad gossip order: could not find channel 110x1x0 for peer's channel update
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell fa7c0a7809 pytest: fix flake in test_wumbo_channels
We mine blocks too fast, and l3 discard the channel_announcment as too far in the future:

```
lightningd-3 2023-12-14T06:40:04.744Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 103x1x1 (current block 102)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 9f05250ee7 lightingd: corrections from Alex Myers's review.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 6031efe145 gossipd: make seeker uintmap visible to memleak code.
```
gossipd: MEMLEAK: 0x55e22ac0f8e8
gossipd:   label=gossipd/seeker.c:621:u8
gossipd:   alloc:
gossipd:     ccan/ccan/tal/tal.c:477 (tal_alloc_)
gossipd:     gossipd/seeker.c:621 (check_timestamps)
gossipd:     gossipd/seeker.c:645 (process_scid_probe)
gossipd:     gossipd/queries.c:849 (handle_reply_channel_range)
gossipd:     gossipd/gossipd.c:436 (handle_recv_gossip)
gossipd:     gossipd/gossipd.c:509 (connectd_req)
gossipd:     common/daemon_conn.c:35 (handle_read)
gossipd:     ccan/ccan/io/io.c:59 (next_plan)
gossipd:     ccan/ccan/io/io.c:407 (do_plan)
gossipd:     ccan/ccan/io/io.c:417 (io_ready)
gossipd:     ccan/ccan/io/poll.c:453 (io_loop)
gossipd:     gossipd/gossipd.c:950 (main)
gossipd:   parents:
gossipd:     gossipd/seeker.c:132:struct seeker
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell dd59d0617d lightningd: expose last_stable_connection in RPC.
This isuseful to find completely dead channels which are worthy of
closing.

Changelog-Added: JSON-RPC: `listpeerchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Changelog-Added: JSON-RPC: `listclosedchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 95bc730894 lightningd: set last_stable_connection 1 minute after channel reestablished.
This is a nice reflection of channel stability: in particular, worse case
ping time is 45 seconds, so we should have had some traffic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 0575f8a544 lightningd: add last_stable_connection field to db, channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 81a280d8a3 lightningd: expose `reestablished` field.
This is useful to see if we've really made progress, or just bounced
off the peer (e.g. it doesn't know the channel, or we have fallen
behind somehow).

Changelog-Added: JSON-RPC: `listpeerchannels` new field `reestablished` set once we've exchanged `channel_reestablish` messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 40d063cda4 channeld: notify lightningd once we've exchanged reestablish messages.
In particular, we were sending `announcement_signatures` before
`channel_reestablish`; we allow this because LND used to do it, but
it's not spec compliant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 58d0cc12c4 gossipd: remove node_announcement generation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 418ac4ed90 lightningd: turn on node_announcement generation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell b76d589a6e lightningd: move node_announcement unit test from gossipd/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 06d59839ec connectd: channel_gossip when we've tried to connect to all peers.
It then waits 10 more seconds (for plugins to call setlease, especially)
before it will update a node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell e4b21b467a lightningd: refine heuristics for remote address selection.
Rather than take the first two from peers with committed channels, use
the most common address given by at least 2 peers, and accept the majority
from non-committed peers if there are no committed peers.

This works well with the node_announcement rework, which waits until
everyone has a chance to connect before creating the node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell b03b049c57 lightningd: don't create node_announcement until gossipd is finished.
It might be redundant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell dd86e817b4 gossipd: tell lightningd its node_announcement on startup, if any.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 48b9b6a05c lightningd: add routines to generate node_announcement.
This is currently still done in gossipd, but we should generate it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 3508331fc4 lightningd: check channel_announcement signatures we get from peer.
We hoise check_signed_hash_nodeid from gossipd's internals, into common/.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell 7bdbb62b09 gossipd: remove channel update/annoounce creation and local update handling.
Now lightningd just doesn't tell us about private channels, doesn't
expect us to generate channel gossip, and tells us about public channels
via the addgossip call, we don't need any of this in gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30