This avoids us gossiping about nodes which don't have live channels.
Interstingly, we previously tested that we *did* gossip such node
announcements, and now we fix that test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Add checking for and sending tx_abort to channeld.
When receiving it we first ACK it back, send a request to restart to lightningd, and then shutdown channeld.
We also must update the splice tests that relied on reconnect checks for splice warnings (as some are now tx_aborts instead).
If we try to reuse the same UTXO for the HTLC as the anchor, it will clash.
One block later we can spend the anchor change, all good.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still want to test non-anchor channels, as we still support them, but
we've made it non-experimental. To test non-anchor channels, we
use dev-force-features: -23.
Changelog-Added: Protocol: `option_anchors_zero_fee_htlc_tx` enabled, no longer experimental.
Changelog-Changed: Config: `experimental-anchors` now does nothing (it's enabled by default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup!_options__make_anchors_enabled_by_default,_ignore_experimental-anchors.patch':
fixup! options: make anchors enabled by default, ignore experimental-anchors.
This uses pip to install to a venv when the dependencies are specified
with requirements.txt and poetry when it's present on the system and the
plugin has a pyproject.toml.
The directory structure will be:
reckless/
source/ (original plugin code here)
plugin_entrypoint (original entrypoint)
plugin_name (symlink or wrapper to activate venv)
.metadata (installation information)
.venv/ (python virtual environment)
The wrapper matches the naming of the original plugin entrypoint. The
shebang is modified to use the virtual environment's python binary,
then the original plugin is imported as a module and executed as though
it was run directly.
Changelog-Changed: reckless installs python plugins in virtual environments
Christian points out that this makes the type harder, and it's just awkward.
Changelog-EXPERIMENTAL: JSON-RPC: Deprecated `offer` parameter `recurrence_base` with `@` prefix: use `recurrence_start_any_period`.
Changelog-EXPERIMENTAL: JSON-RPC: Added `offer` parameter `recurrence_start_any_period`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For better performance it is recommended to use the modern OpenSSL
EVP_MD_fetch API to load digest algorithms (i.e. explicit fetching),
instead of the older implicit fetching API.
As a side effect, using this API seems to avoid memory leaks with some
versions of OpenSSL.
We can get bad gossip if a node processes a gossip message after we've closed:
```
_________________________________________ ERROR at teardown of test_closing_specified_destination _________________________________________
...
> raise ValueError(str(errors))
E ValueError:
E Node errors:
E - lightningd-1: had warning messages
E - lightningd-4: had bad gossip messages
E Global errors:
...
lightningd-1 2024-02-03T00:29:02.299Z INFO 0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199-connectd: Received WIRE_WARNING: WARNING: channel_announcement: no unspent txout 105x1x0
lightningd-1 2024-02-03T00:29:02.300Z DEBUG 0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199-connectd: peer_in WIRE_WARNING
lightningd-1 2024-02-03T00:29:02.300Z INFO 0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199-connectd: Received WIRE_WARNING: WARNING: channel_announcement: no unspent txout 103x1x0
lightningd-1 2024-02-03T00:29:02.339Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-connectd: peer_in WIRE_WARNING
lightningd-1 2024-02-03T00:29:02.339Z INFO 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-connectd: Received WIRE_WARNING: WARNING: channel_announcement: no unspent txout 103x1x0
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Instead of "new" and "load", we don't really need to "load" anything,
so do everything in gossip_store_new.
Have it do the compaction/rewrite, and collect the dying records
The gossip_store_load is now basically a noop, since gossmap
does that.
gossipd removes a pile of routines dealing with messages,
in favor of just handing them to gossmap_manage.
The stub gossmap_manage constructor is removed entirely.
We simplified behaviour around channel_announcements with
no channel update: we now add them to the store, and go
back to fix the timestamp later. This changes a test,
which explicitly tests for the old behaviour.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We weakened this progressively over time, and gossip v1.5 makes spam
impossible by protocol, so we can wait until then.
Removing this code simplifies things a great deal!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer ratelimit gossip messages by channel, making our code far simpler.
We never enabled it, because we seemed to be eliminating valid
channels. We discard zombie-marked records on loading.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was an obscure dev command, as it never worked reliably.
It would be much easier to re-implement once this is done.
This turned out to reveal a tiny leak on
tests/test_gossip.py::test_gossip_store_load_amount_truncated where we
didn't immedately free chan_ann if it was dangling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
update_count is just the count of the records for a tx. To calculate onchain fees
for an account we must sum all credits vs debits. We don't need to GROUP BY update_count
nor ORDER BY update_count since it is just a running index of updates to this tx.
Remove grouping by update_count which resulted in a crash due to bad arithmetic
caused by fee calculation returned rows not being consolidated.
Remove xfail.
```
2024-02-02T10:30:38.6479143Z __________________ ERROR at teardown of test_multifunding_one __________________
...
2024-02-02T10:30:38.6491895Z > raise ValueError(str(errors))
2024-02-02T10:30:38.6492208Z E ValueError:
2024-02-02T10:30:38.6492465Z E Node errors:
2024-02-02T10:30:38.6492833Z E - lightningd-2: had bad gossip messages
2024-02-02T10:30:38.6493535Z E Global errors:
...
2024-02-02T10:30:38.7458545Z lightningd-2 2024-02-02T10:25:48.410Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 103x2x1 (current block 102)
2024-02-02T10:30:38.7460140Z lightningd-2 2024-02-02T10:25:48.411Z UNUSUAL lightningd: Bad gossip order: could not find channel 103x2x1 for peer's channel update
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: added a withdraw all to the end of test_onchain_their_unilateral_out to ensure that the unilateral close info is correct with anchors. Tests https://github.com/Blockstream/greenlight/issues/348
This caused a flake in test_gossip_lease_rates, since the peer would ignore
the node_announcement due to duplicate timestamps!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We mine blocks too fast, and l3 discard the channel_announcment as too far in the future:
```
lightningd-3 2023-12-14T06:40:04.744Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 103x1x1 (current block 102)
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This isuseful to find completely dead channels which are worthy of
closing.
Changelog-Added: JSON-RPC: `listpeerchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Changelog-Added: JSON-RPC: `listclosedchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is useful to see if we've really made progress, or just bounced
off the peer (e.g. it doesn't know the channel, or we have fallen
behind somehow).
Changelog-Added: JSON-RPC: `listpeerchannels` new field `reestablished` set once we've exchanged `channel_reestablish` messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than take the first two from peers with committed channels, use
the most common address given by at least 2 peers, and accept the majority
from non-committed peers if there are no committed peers.
This works well with the node_announcement rework, which waits until
everyone has a chance to connect before creating the node_announcement.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit is a bit messy, but it tries to do the minimal switchover.
Some tests change, so those are included here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Mining 13 blocks to close the l1->l2 channel also causes the l2->l3
channel to confirm, and I was reliably getting the
channel_announcement to l3 in time to confuse this wait_for test. So
query the channel we want directly to make it more robust.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
After this, the invoice (correctly!) gives the zeroconf channel as a routehint,
so this test fails. Simple workaround: make invoice before creating zeroconf
channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was triggering on private updates after gossip changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1diff --git a/tests/test_gossip.py b/tests/test_gossip.py
index 677ec4825..285290b71 100644
When we rewrite, we actually use the remote channel_announcement signatures from the db, and notice that l1 here is actually l2:
```
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: gossipd rejected our channel announcement: WARNING: Bad node_signature_1 304402204ad9f0ca9b5f1f8eec61290b6008bf2c15f011e35543d25c35c5b89d2e1a183402201aaca5883714dbe11d6ae00a146fac7302352e1b9e987eee5ce98e8eb6a82909 hash e713698e14d3d8f973210e8d93c96d1a7c117c58340ccb195785a672fd0b5ef1 on channel_announcement 01004ad9f0ca9b5f1f8eec61290b6008bf2c15f011e35543d25c35c5b89d2e1a18341aaca5883714dbe11d6ae00a146fac7302352e1b9e987eee5ce98e8eb6a829095c05509264fcf72e19180f0285cf0ca0d786f280a75d79e46e5627be20c9dd9a4e9105aa61d7faf3a375454db94376b9b9df29cf078130ed827836b0849f119014ec750ab163baef701e7889f430ee37ea7e38857aa2b2745440600999c85f2903abefde5210dd06ae12f24b8801416800db84f5c73d35594af594f9001800025dee15519d29eb81e9d1ac761deebdad91ba1d08a05c027c16ea5fc08f3a9b663f8433209392395f0107f6b4d266c46b0946bd098f88c0693817fb560e0526a8000006226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00006900000100000266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c035180266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c0351802324266de8403b3ab157a09f1f784d587af61831c998c151bcc21bb74c2b2314b0206540493a98599967a61c45172df49681e398a6cb0890a506e74f76c2d2be242
```
Explicitly remove those messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
niftynei
Dusty Daemon
Erik De Smedt
Christian Decker
Alex Myers
Matt Morehouse
evansmj
Lagrang3
Ken Sedgwick