Commit Graph

12425 Commits

Author SHA1 Message Date
Rusty Russell 611795beee listtransactions: get rid of per-tx type annotations.
We didn't actually populate them properly, and the real annotations
are on inputs and outputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `listtransactions` `channel` and `type` field removed at top level.
2023-01-30 15:15:41 -06:00
Rusty Russell 578f075407 wallet: remove unused TX_ANNOTATION type in transaction_annotations table.
We only ever use this table for output and input transactions: indeed, my node
doesn't have any annotation types 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 4b9cb7eb76 doc: remove unused offerout schema.
We removed the command for v22.11.

Also, we removed the `refund_for` offer parameter, so remove its description
from the manpage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Alex Myers 98bfd23fff pytest: test_channel_resurrection now succeeds 2023-01-30 16:33:03 +10:30
Alex Myers 1bae8cd28a gossipd: zombify inactive channels instead of pruning
Though BOLT 7 says a channel may be pruned when one side becomes inactive
and fails to refresh their channel_update, in practice, the
channel_announcement can be difficult to recover if deleted entirely.
Here the channel_announcement is tagged as zombie such that gossip_store
consumers may safely ignore it, but it may be retained should the channel
come back online in the future. Node_announcements and channel_updates may
also be retained in such a fashion until the channel is ready to be
resurrected.

Changelog-Fixed: Pruned channels are more reliably restored.
2023-01-30 16:33:03 +10:30
Alex Myers 6bff10cd40 gossip_store: add a flag for zombie entries
This will allow gossipd to store and persist gossip for channels rather
than deleting them entirely when the channels are pruned from the
network.
2023-01-30 16:33:03 +10:30
Alex Myers ed4815527a gossipd: avoid gossipd crash due to double freeing timer 2023-01-30 16:33:03 +10:30
Alex Myers f2f05117aa pytest: gossipd: test resumption of pruned channels 2023-01-30 16:33:03 +10:30
Rusty Russell a3ca3fb047 common/bolt11: fix 32-bit compilation.
Fixes d9fed06b900368e59f4d1f432b87d40fd28ce8d3:

```
common/bolt11.c:868:31: error: format specifies type 'size_t' (aka 'unsigned long') but the argument has type 'u64' (aka 'unsigned long long') [-Werror,-Wformat]
                                           bech32_charset[type], field_len);
                                                                 ^~~~~~~~~
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 16:30:46 +10:30
Rusty Russell 2dec805465 decode: fix handling of blinded_payinfo.
Our pay code handles this correctly, but decode was still using an old model
where there was a payinfo per hop, not per path.

Reported-by: @t-bast
See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 6eb7a4cbf2 plugins: update to match latest offers text.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-offers" BOLTVERSION=guilt/offers
```

In this case, only trivial mods.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 9aefe3d40a common: update to latest onion-message spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-onion-message" BOLTVERSION=guilt/offers
```

Mainly textual, though I neatened the extra fields check for TLVs with
blinding, and implemented the "no other fields" requirement for
non-final onion message hops.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 8e630e7c53 common/test: fix up name of test file to match latest version.
Otherwise it's skipped!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell e9eb5f493b common: update to latest route-blinding spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-route-blinding" BOLTVERSION=guilt/offers
```

Other than textual changes, this does:

1. Ensures we put total_amount_msat in onion final hop (reported by @t-bast).
2. Require that they put total_amount_msat in onion final hop.
3. Return `invalid_onion_blinding` exactly as defined by the spec (i.e. less
   aggressive when we're the final hop) (also reported by @t-bast, but I already
   knew).

See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `offers` breaking blinded payments change (total_amount_sat required, Eclair compat)
2023-01-30 13:24:29 +10:30
Rusty Russell d5c19b23d8 common/onion_decode: put final flag in onion_payload.
You can use rs->nextcase, but we don't always keep that around, so
keep a flag in onion_payload.

We'll use this in the "do we need to return a blinded error code"
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 885506765e tools/check-bolt.c: don't leak open directory.
Thanks valgrind!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell da3506e6a0 wire: use correct number of update_add_tlvs blinding field.
See: #5823
Reported-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `offers` breaking blinded payments change (update_add_tlvs fix, Eclair compat)
2023-01-30 13:24:29 +10:30
Rusty Russell 5958c9c3d6 common/test: remove unused padding in bolt04/blinded-onion-message-onion-test.json
This was reported by @valentinewallace: Dave would only use padding to
make all his own encrypted_recipient_data equal-length.  We did it
across the entire path, which includes the hop added by Alice, which
Dave wouldn't know about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Christian Decker 13fe27c65f gossipd: Do not send warning when node_announcement parsing fails
Changelog-Fixed: gossip: We removed a warning for old `node_announcement` that was causing LND peers to disconnect
2023-01-27 12:58:49 -06:00
Ken Sedgwick 7b2c5617c1 hsmd: increase HSM_MAX_VERSION to 3 2023-01-26 21:10:15 -06:00
Ken Sedgwick a4dc714cdc hsmd: add hsmd_preapprove_keysend and check_preapprovekeysend pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_keysend` is added.
Changelog-added: JSON-RPC: A new command `preapprovekeysend` is added.
2023-01-26 21:10:15 -06:00
Ken Sedgwick f29343d740 hsmd: add hsmd_preapprove_invoice and check_preapproveinvoice pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_invoice` is added.
Changelog-added: JSON-RPC: A new command `preapproveinvoice` is added.
2023-01-26 21:10:15 -06:00
Braydon 9482e0619c docker: Install protobuf-compiler for builder 2023-01-26 15:22:45 +00:00
Riccardo Casatta 182a9cdcb6 cln-rpc: use serde rename instead of alias
rename is necessary to roundtrip, otherwise the rust name is used.

This also remove the rename if they are not necessary.

Note that:
```
 #[serde(rename="foo", skip_serializing_if=="bar")]
 pub field: bool,
```

is equivalent to:
```
 #[serde(rename="foo")]
 #[serde(skip_serializing_if=="bar")]
 pub field: bool,
```

and for simplicity of construction the latter is used
2023-01-26 15:03:54 +01:00
Rusty Russell fa4b61d13d common/bolt11: convert to table-driven.
Decode functions are now almost entirely uniform, so just use a table.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-25 13:59:34 -06:00
Rusty Russell cbd0ef4192 common/bolt11: add pull_all helper for common case of entire field.
And make pull_bits return a uniform error message, since that's what
callers want, rather than asserting success.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: pay: don't assert() on malformed BOLT11 strings.
2023-01-25 13:59:34 -06:00
Rusty Russell d9fed06b90 common/bolt11: const cleanup, fix parsing errors.
Also, we don't need to pass the total length to the field parsers,
just the length for this field (confusingly, this was called
"data_length").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-25 13:59:34 -06:00
Michael Schmoock 1fb1e0eec4 pytest: test ip discovery for custom port 2023-01-25 17:57:04 +01:00
Michael Schmoock a62c74be7b doc: usage of --announce-addr-discovered-port option 2023-01-25 17:57:04 +01:00
Michael Schmoock ca9e3e4cc1 opts: adds --announce-addr-discovered-port config option
This will give the user an option to set a custom port when using
discovered IPs for node_announcents. Without this, only the selected
networks default port can used.

Changelog-Added: Adds --announce-addr-discovered-port config option to set custom port for IP discovery.
2023-01-25 17:57:04 +01:00
Michael Schmoock 34cfc93939 cli: getinfo output to regard --ip-discovery 2023-01-25 14:37:56 +01:00
Michael Schmoock 30dea0a431 opts: deprecate --disable-ip-discovery switch
This switch was not doing anything useful anymore.
We deprecate it anyways to notify the user about the new switch.

Changelog-Deprecated: The old --disable-ip-discovery config switch
2023-01-25 14:37:56 +01:00
Michael Schmoock a2b94f16f8 pytest: fix and adapt test_remote_addr_disabled 2023-01-25 14:37:56 +01:00
Michael Schmoock ee046662a7 doc: announce-addr-discovered config switch 2023-01-25 14:37:56 +01:00
Michael Schmoock 3babbc5201 opts: announce-addr-discovered on/off/auto switch
This adds the option to explicitly enable ip-discovery, which maybe
helpful for example when a user wants TOR announced along with
discovered IPs to improve connectivity and have TOR just as a fallback.

Changelog-Added: Adds config switch 'announce-addr-discovered': on/off/auto
2023-01-25 14:37:56 +01:00
Michael Schmoock bd75f8ea6c opts: adds the autobool on/off/auto feature 2023-01-25 14:37:56 +01:00
Michael Schmoock e932f05bc8 ci: adds git fetch before doing schema checks
This fixes the CI errors when doing `make check-source`
steps 'schema-added-check' and 'schema-removed-check'.
These errors prevented CI from performing these steps correctly:

```
fatal: ambiguous argument 'main': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
```

I changed it so that CI does a `git fetch origin` at first and do
the `git diff` against 'origin/master' (which then exist).
Also fixed a bug in the script that was missing $$master in the same line.

Also I added that the script shows the actual diff before failing,
so the user quickly sees whats wrong.
2023-01-25 14:37:56 +01:00
Rusty Russell 0e25d56329 lightningd: use a hash table for peer->dbid.
Otherwise, loading up when we have 100k peers is *painful*!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Rusty Russell cfa632b0e9 lightningd: use hash map for peers instead of linked list.
After connecting 100,000 peers with one channel each (not all at
once!), we see various places where we exhibit O(N^2) behaviour.

Fix these by keeping a map of id->peer instead of a simple
linked-list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Rusty Russell 17aa047b17 pytest: fix output order assumption in test_setchannel_all
Hash table coming up makes it random!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Rusty Russell 6a95d3a25e common: expose node_id_hash functions.
They're used in several places, and we're about to add more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Rusty Russell 0faa8397c3 wallet: add dependency on lightningd/ headers.
wallet should really be a subdir of lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Matt Whitlock acfb63e4bf channeld: remove dead HTLCs from htable and free them (eventually)
The channel->htlcs map was exhibiting unbounded growth, as elements were
never removed from it. This was causing lightning_channeld processes to
consume ever-increasing amounts of memory, and iterating over the map
was causing ever-increasing CPU utilization. There were FIXME comments
suggesting that the intention was to remove HTLCs from the map upon
their deaths. This commit implements that intention.

Changelog-Fixed: channeld no longer retains dead HTLCs in memory.
2023-01-20 14:04:08 -06:00
daywalker90 82c94330a5 add PartialEq to ShortChannelId
PartialEq makes it possible to compare 2 ShortChannelIds for equality which i found often helpful
2023-01-17 14:58:07 +01:00
Vincenzo Palazzo 8f94e8b943 comm: make sure that our version check is reliable
Rework the logic of the version check used in the
database migration, and make sure
that it is full functional to avoid confusion
at release time.

Changelog-Fixed: database: Correctly identity official release versions for database upgrade.

Reported-by: @urza
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-17 14:15:24 +10:30
Rusty Russell 8d825ef0b7 lightningd: fix valgrind reported leak when we exit early.
```
E           Global errors:
E            - Node /tmp/ltests-adkwu44c/test_logging_1/lightning-2/ has memory leaks: [
E               {
E                   "backtrace": [
E                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E                       "lightningd/peer_control.c:2203 (load_channels_from_wallet)",
E                       "lightningd/lightningd.c:1105 (main)"
E                   ],
E                   "label": "lightningd/peer_control.c:2203:struct htlc_in_map",
E                   "parents": [
E                       "lightningd/lightningd.c:107:struct lightningd"
E                   ],
E                   "value": "0x55d920a345e8"
E               }
E           ]
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-17 14:13:45 +10:30
Rusty Russell 288f5df8d1 ccan: update to fix recent gcc "comparison will always evaluate as 'false'" warning
```
lightningd/jsonrpc.c: In function ‘destroy_json_command’:
lightningd/jsonrpc.c:1180:63: error: the comparison will always evaluate as ‘false’ for the address of ‘canary’ will never be NULL [-Werror=address]
lightningd/jsonrpc.c:108:53: note: ‘canary’ declared here
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-17 14:13:45 +10:30
Christian Decker 1f21796454 gha: Temporarily disable `test_notify`
It is failing due to an error somewhere:

```
*** buffer overflow detected ***: terminated
```
2023-01-16 13:53:48 +10:30
Christian Decker e3a9bda301 ci: Downgrade the upload-artifact action to v2.2.4
There seem to be a couple of issues with Bad Requests with later
versions, trying this :-)
2023-01-16 13:53:48 +10:30
Christian Decker b0e3d483e6 gci: Add a test for the postgres backend 2023-01-16 13:53:48 +10:30