Commit Graph

12911 Commits

Author SHA1 Message Date
Christian Decker 6d67eb934d py: Ignore missing whitespace after keyword for now 2023-01-16 13:53:48 +10:30
Christian Decker 4d668e76a0 gci: Split out the stages better
We want to compile with one set of dependencies, and run the tests
with another. This also helps us cut down on the times we compile CLN
itself, by sharing them among stages, and simplifies the logic of each
stage to have one specific goal.
2023-01-16 13:53:48 +10:30
Christian Decker a00190dce5 gci: Update Github Actions steps to their latest versions
They were causing warnings.
2023-01-16 13:53:48 +10:30
Christian Decker 8c075c4cda gci: Stop uploading unit test results
We never used them, and they were creating distracting warnings.
2023-01-16 13:53:48 +10:30
Christian Decker 1fbf774e04 gci: Split out the lnprototest from the larger CI run
The lnprototests are often blocking PRs, due to them failing, but we
can't restart them when valgrind runs are still ongoing, since they'd
also restart. Splitting allows us to rerun them selectively and waste
less time.

Ideally we'd just fix them, but I am by no means knowledgeable enough
to fix them now.
2023-01-16 13:53:48 +10:30
Rusty Russell 9d5841a0e6 pytest: fix flake in tests/test_closing.py::test_closing_specified_destination
The close call can fail, since we already unilaterally closed since we mined blocks
too fast:

```
2023-01-14T01:00:10.2502199Z E           pyln.client.lightning.RpcError: RPC call failed: method: close, payload: ['107x1x1', None, 'bcrt1qeyyk6sl5pr49ycpqyckvmttus5ttj25pd0zpvg'], error: {'code': -32602, 'message': "Short channel ID not active: '107x1x1'"}
...
2023-01-14T01:00:10.5288050Z lightningd-4 2023-01-14T00:59:59.650Z UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer permanent failure in CHANNELD_NORMAL: Fulfilled HTLC 0 SENT_REMOVE_COMMIT cltv 113 hit deadline
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell 932ca9e91f lightningd: deprecate p2sh-segwit addresses for `newaddr` `addresstype`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `newaddr`: `addresstype` `p2sh-segwit` (use default, or `bech32`).
2023-01-14 21:17:45 +10:30
Rusty Russell 2f36c03307 pytest: use bech32 addresses everywhere.
We used to create some p2sh-segwit addresses just to mix things up.  This
streamlines back to just bech32.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell 85fc46f76c pyln-testing: don't default openchannel and fundwallet to p2sh-segwit, use bech32.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Christian Decker e5d384a427 gossip: Do not send warnings if we fail to parse a `channel_update`
We removed a warning about the channel_update being malformed since
the warning could cause lnd to disconnect (seems they treat
channel-unrelated warnings as fatal?). This was caused by lnd not
enforcing the `htlc_maximum`, thus the parsing would fail. We can
re-add the warning once our assumption that `htlc_maximum` being set
is valid.

Changelog-Fixed: gossip: We no longer send warning that lnd would not understand if we get outdated gossip
2023-01-14 12:39:10 +10:30
Rusty Russell 2b5f4d14d1 CI: fix schema diff check.
GitHub's master branch is called "main" apparently.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 10:13:28 +10:30
Riccardo Casatta 6b977f0292 cln-grpc: update listpeers json fixing tests 2023-01-13 21:42:27 +01:00
Rusty Russell f1373fd98c doc: remove manual field descriptions from listpeerchannels(7).
Some are best copied into the schema, but some are already
out-of-date, so cleanest to remove them and rely on the generated (and
thus, checked!) fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 6c0b9b0c78 lightningd: deprecate listpeers.channels
Changelog-Deprecated: JSON-RPC: `listpeers` `channels` array: use `listpeerchannels`

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Rusty Russell 9ffaab7d22 pytest: fix race in test_bookkeeping_closing_subsat_htlcs
With the next change (which, as a side-effect, speeds up listpeers),
we seem to hit a race in this test.  The bookkeeper doesn't get to
process the final payment before the node is shutdown.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo a2347c7452 tests: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 1fa32333b9 tests/utils.py: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo f08d3516f7 contrib/pyln-testing: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo c488561282 plugins/topology: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Rusty Russell a56c890ae5 plugins: use listpeerchannels instead of listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell ff2d7e6833 pay: use json_to_listpeers_channels() for local_channel_hints.
Don't parse the listpeers.channels output ourselves: with two extra fields
we can simply reuse json_to_listpeers_channels().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell 5d5b9c6812 libplugin: don't return unopened channels from json_to_listpeers_channels().
This way we always have an SCID and a direction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell 57dcf68c0b plugins/libplugin: flatten return from json_to_listpeers_result.
Instead of returning a peers -> channels heirarchy, return (as callers
want!) a flat array of channels.

This is actually most of the transition work to make them work with
listpeerchannels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo cb5ee7e49c plugins: make bookkeeper use the new listpeerchannels command. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 6fa904b4fb lightningd: add listpeerchannels command
Changelog-Added: JSON-RPC: new command `listpeerchannels` now contains information on direct channels with our peers.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Rusty Russell 1d8b899551 lightningd: prepare internal json routines for listpeerchannels.
We're soon going to call json_add_unsaved_channel and
json_add_uncommitted_channel from a new place, where we want the peer
state directly included.

Based on patch by @vincenzopalazzo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Greg Sanders 300f732bbe proposal_meets_depth tracked output always has a proposal 2023-01-12 14:17:38 +10:30
Rusty Russell 6044184323 lightningd: don't call memcpy with NULL.
Thanks C committee!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 5dfcd15782 all: no longer need to call htable_clear to free htable contents.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell f07e37018d setup: make all htables use tal.
This makes them easier to clean up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 0d93841cc7 plugins/command: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 3c4ce9e448 plugins/pay: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 94e8ce030a gossipd: use pointer to hash table for channels in node.
We actually reduce the size of struct node by 1 pointer, which
is mildly smaller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 4200371020 gossipd: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 851cbf6c83 memleak: prepare for htable to be a tal object.
Since it gets resized during traverse, we would crash by
keeping a pointer to the old one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 81e57dce52 connectd: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 4a570c9419 gossmap: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 763d02e424 lightningd: ensure htlc htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell da5eb03bae lightningd/chaintopology: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
niftynei 42c6d49082 tests: add account_id's and match by acct id, not test ordering
There's no guarantee as to iteration order for accounts/channels, but
this test was relying on them.

Adding account attribution and comparing by account_ids fixes

Fixes: #5869
Reported-By: @rustyrussell
2023-01-12 11:40:17 +10:30
niftynei 66a4d50098 tests: mark test as dev (times out otherwise) 2023-01-12 11:40:17 +10:30
Rusty Russell 3a39c635b6 CI: rough check that schema changes seem to mention added, don't delete non-deprcated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 717cb03f51 doc: add recent additions, fix annotation on listpeers to actually deprecate.
TODO: It would be great to similarly annotate new/deprecated commands and
their parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: doc: we now annotate what versions JSON field additions and deprecations happenened.
2023-01-11 12:51:18 +10:30
Rusty Russell b2148d0eab docs: handle "added": "version" and "deprecated": "version" from schemas.
This means we will document deprecations and additions, rather than just
pretending they've always been that way!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 3f8199bbfd doc: document that we should annotate added and deprecated schemas.
No longer just delete them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 404e961bad cli: add -c/--commando support.
It's easier to type:

```
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== invoice any "Invoice Label" "Invoice Description"
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== commando amount_msat=100000 label="invoice label" description="invoice description"
```

Than:

```
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '["any", "Invoice Label", "Invoice Description"]' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '{"amount_msat": "100000", "label": "invoice label", "description": "invoice description"}' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: `--commando=peerid:rune` (or `-c peerid:rune`) as convenient shortcut for running commando commands.
2023-01-11 11:13:27 +10:30
Rusty Russell 3f0c5b985b commando: add filtering support.
1. When we receive a commando command from a remote using the `filter`
   field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
   more efficient than using filtering locally.

Of course, older remote nodes will ignore the filter, but that's
harmless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
2023-01-11 11:13:27 +10:30
Rusty Russell 1250806060 commando: correctly replace the `id` field in responses.
This was reported a while ago: now do it properly.

Fixes: #5637
Changelog-Fixed: Plugins: `commando` now responds to remote JSON calls with the correct JSON `id` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell b75ada7017 commando: track incoming and outgoing JSON IDs. Get upset if they don't match!
They currently don't, so we get some BROKEN messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell 0201e6977f commando: build ID of command based on the id they give us.
We change the libplugin API so commando can provide its own ID base.

This id chaining enables much nicer diagnostics!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30