The gossip_store_load is now basically a noop, since gossmap
does that.
gossipd removes a pile of routines dealing with messages,
in favor of just handing them to gossmap_manage.
The stub gossmap_manage constructor is removed entirely.
We simplified behaviour around channel_announcements with
no channel update: we now add them to the store, and go
back to fix the timestamp later. This changes a test,
which explicitly tests for the old behaviour.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a bit less optimal than before, where we had an ordered map of
channels and could easily serve "channels between scids 800000x and
900000x". We now iterate all of them.
The rest is fairly mechanical.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We add a temporary stub gossmap_manage constructor, which simply opens
the gossmap and doesn't do anything else.
Then seeker uses this, rather than routing.c, to probe.
We optimize our "get random node announcements" a bit by traversing a
random set of nodes directly, and seeing if we have no
node_announcement, then querying their first channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we've got more than 10000 pending channel_announcements,
complain and stop processing any more.
If this becomes a problem, we can limit individual peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's an unnecessary round-trip, and can cause us to complain in CI, in
the case where the channel has been closed by the time we ask.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
At initialization, gossipd is supposed to send all the local
channel_updates and any node_announcement it knows, so lightningd
doesn't generate fresh ones unnecessarily.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a fair amount of code, but much is taken from
the old routing.c, with the difference that this uses
common/gossmap instead of our own structures.
The interfaces are fairly clear:
1. gossmap_manage_new - allocator
2. gossmap_manage_channel_announcement
- handle new channel announcement msg
- if too early, keeps it in early map
- queues it, asks lightingd about UTXO.
3. gossmap_manage_handle_get_txout_reply
- handle response from lightningd for above.
4. gossmap_manage_channel_update
- handle channel_update message
- may have to wait on pending channel_announcement
5. gossmap_manage_node_announcement
- handle node_announcement msg
- may have to wait on pending channel_announcement
6. gossmap_manage_new_block
- see if early announces can now be processed.
7. gossmap_manage_channel_spent
- lightningd tells us UTXO is spent
- may prepare channel for closing in 12 blocks.
8. gossmap_manage_channel_dying
- gossip_store load tells us channel was spent earlier.
- like gossmap_manage_channel_spent, but maybe < 12.
9. gossmap_manage_get_gossmap
- gossmap accessor: seeker and queries will need this.
10. gossmap_manage_new_peer
- a new peer has connected, give them all our gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We weakened this progressively over time, and gossip v1.5 makes spam
impossible by protocol, so we can wait until then.
Removing this code simplifies things a great deal!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer ratelimit gossip messages by channel, making our code far simpler.
We never enabled it, because we seemed to be eliminating valid
channels. We discard zombie-marked records on loading.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Most nodes don't really care about exact timestamps on gossip filters,
so just keep a flag on whether we have anything in the gossip_store,
and use that to determine whether we ask peers for everything.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is easier for future callers, which don't have a convenient peer
structure: in particular, asynchronous processing of gossip for peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was an obscure dev command, as it never worked reliably.
It would be much easier to re-implement once this is done.
This turned out to reveal a tiny leak on
tests/test_gossip.py::test_gossip_store_load_amount_truncated where we
didn't immedately free chan_ann if it was dangling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, allow callers to see unknown records we ignore (and let
them fail as a result), and get called if we can't pack a
channel_update into our internal format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
gossip_store_del - takes a gossmap-style offset-of-msg not offset-of-hdr.
gossip_store_flag: set an arbitrary flag on a gossip_store hdr.
gossip_store_get_timestamp/gossip_store_set_timestamp: access gossip_store hdr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The only way you'll see private channel_updates is if you put them
there yourself with localmods.
I also renamed the confusing gossmap_chan_capacity to gossmap_chan_has_capacity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
update_count is just the count of the records for a tx. To calculate onchain fees
for an account we must sum all credits vs debits. We don't need to GROUP BY update_count
nor ORDER BY update_count since it is just a running index of updates to this tx.
Remove grouping by update_count which resulted in a crash due to bad arithmetic
caused by fee calculation returned rows not being consolidated.
Remove xfail.
If $BITCOIN_BIN is empty and bitcoin-cli or bitcoind are not found
startup_regtest will shutdown and display a corresponding error message.
Similarly if lightning-cli and lightningd are not found.
```
2024-02-02T10:30:38.6479143Z __________________ ERROR at teardown of test_multifunding_one __________________
...
2024-02-02T10:30:38.6491895Z > raise ValueError(str(errors))
2024-02-02T10:30:38.6492208Z E ValueError:
2024-02-02T10:30:38.6492465Z E Node errors:
2024-02-02T10:30:38.6492833Z E - lightningd-2: had bad gossip messages
2024-02-02T10:30:38.6493535Z E Global errors:
...
2024-02-02T10:30:38.7458545Z lightningd-2 2024-02-02T10:25:48.410Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 103x2x1 (current block 102)
2024-02-02T10:30:38.7460140Z lightningd-2 2024-02-02T10:25:48.411Z UNUSUAL lightningd: Bad gossip order: could not find channel 103x2x1 for peer's channel update
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Tacking the fuzz testing onto the PR testing not a good idea. It
causes completely unrelated tests to fail, and doesn't guarantee it
will find newly introduced ones. Running daily is likely better.
Changelog-None No user visible change
By default, UBSan reports runtime errors but does not stop execution.
We already abort in debug builds, and this commit makes us also abort in
regular builds when UBSan is enabled. Arguably, this is what users
expect when they enable UBSan, so it is a good default.
I know I've missed some UBSan bugs in the past because of this issue,
and dergoegge mentioned that this also happened to him.
Changelog-None: fixes hole in ([#7010]), see ([#7026])
Explicit hsmd_revoke_commitment_tx instead of get_per_commitment_point
needed when revokes are retried. The hsmd_revoke_commitment_tx message
is idempotent.
Changelog-Added: added a withdraw all to the end of test_onchain_their_unilateral_out to ensure that the unilateral close info is correct with anchors. Tests https://github.com/Blockstream/greenlight/issues/348
Rusty Russell
evansmj
Lagrang3
Christian Decker
Harshit933
Matt Morehouse
Ken Sedgwick
Erik De Smedt
ok300
22388o⚡️