Commit Graph

1456 Commits

Author SHA1 Message Date
Rusty Russell 153b7bf192 common/gossip_store: move subdaemon-only routines to connectd.
connectd is the only one who uses these routines now.  The
rest can be linked into a plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 7e8b93daa1 common/gossip_store: expose routine to read one header.
This is useful when you're writing routines to scan it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 0274d88bad common/gossip_store: clean up header.
It's actually two separate u16 fields, so actually treat it as
such!

Cleans up zombie handling code a bit too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 578f075407 wallet: remove unused TX_ANNOTATION type in transaction_annotations table.
We only ever use this table for output and input transactions: indeed, my node
doesn't have any annotation types 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Alex Myers 1bae8cd28a gossipd: zombify inactive channels instead of pruning
Though BOLT 7 says a channel may be pruned when one side becomes inactive
and fails to refresh their channel_update, in practice, the
channel_announcement can be difficult to recover if deleted entirely.
Here the channel_announcement is tagged as zombie such that gossip_store
consumers may safely ignore it, but it may be retained should the channel
come back online in the future. Node_announcements and channel_updates may
also be retained in such a fashion until the channel is ready to be
resurrected.

Changelog-Fixed: Pruned channels are more reliably restored.
2023-01-30 16:33:03 +10:30
Alex Myers 6bff10cd40 gossip_store: add a flag for zombie entries
This will allow gossipd to store and persist gossip for channels rather
than deleting them entirely when the channels are pruned from the
network.
2023-01-30 16:33:03 +10:30
Rusty Russell a3ca3fb047 common/bolt11: fix 32-bit compilation.
Fixes d9fed06b900368e59f4d1f432b87d40fd28ce8d3:

```
common/bolt11.c:868:31: error: format specifies type 'size_t' (aka 'unsigned long') but the argument has type 'u64' (aka 'unsigned long long') [-Werror,-Wformat]
                                           bech32_charset[type], field_len);
                                                                 ^~~~~~~~~
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 16:30:46 +10:30
Rusty Russell 9aefe3d40a common: update to latest onion-message spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-onion-message" BOLTVERSION=guilt/offers
```

Mainly textual, though I neatened the extra fields check for TLVs with
blinding, and implemented the "no other fields" requirement for
non-final onion message hops.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 8e630e7c53 common/test: fix up name of test file to match latest version.
Otherwise it's skipped!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell e9eb5f493b common: update to latest route-blinding spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-route-blinding" BOLTVERSION=guilt/offers
```

Other than textual changes, this does:

1. Ensures we put total_amount_msat in onion final hop (reported by @t-bast).
2. Require that they put total_amount_msat in onion final hop.
3. Return `invalid_onion_blinding` exactly as defined by the spec (i.e. less
   aggressive when we're the final hop) (also reported by @t-bast, but I already
   knew).

See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `offers` breaking blinded payments change (total_amount_sat required, Eclair compat)
2023-01-30 13:24:29 +10:30
Rusty Russell d5c19b23d8 common/onion_decode: put final flag in onion_payload.
You can use rs->nextcase, but we don't always keep that around, so
keep a flag in onion_payload.

We'll use this in the "do we need to return a blinded error code"
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell 5958c9c3d6 common/test: remove unused padding in bolt04/blinded-onion-message-onion-test.json
This was reported by @valentinewallace: Dave would only use padding to
make all his own encrypted_recipient_data equal-length.  We did it
across the entire path, which includes the hop added by Alice, which
Dave wouldn't know about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Ken Sedgwick 7b2c5617c1 hsmd: increase HSM_MAX_VERSION to 3 2023-01-26 21:10:15 -06:00
Ken Sedgwick a4dc714cdc hsmd: add hsmd_preapprove_keysend and check_preapprovekeysend pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_keysend` is added.
Changelog-added: JSON-RPC: A new command `preapprovekeysend` is added.
2023-01-26 21:10:15 -06:00
Ken Sedgwick f29343d740 hsmd: add hsmd_preapprove_invoice and check_preapproveinvoice pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_invoice` is added.
Changelog-added: JSON-RPC: A new command `preapproveinvoice` is added.
2023-01-26 21:10:15 -06:00
Rusty Russell fa4b61d13d common/bolt11: convert to table-driven.
Decode functions are now almost entirely uniform, so just use a table.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-25 13:59:34 -06:00
Rusty Russell cbd0ef4192 common/bolt11: add pull_all helper for common case of entire field.
And make pull_bits return a uniform error message, since that's what
callers want, rather than asserting success.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: pay: don't assert() on malformed BOLT11 strings.
2023-01-25 13:59:34 -06:00
Rusty Russell d9fed06b90 common/bolt11: const cleanup, fix parsing errors.
Also, we don't need to pass the total length to the field parsers,
just the length for this field (confusingly, this was called
"data_length").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-25 13:59:34 -06:00
Rusty Russell 6a95d3a25e common: expose node_id_hash functions.
They're used in several places, and we're about to add more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Vincenzo Palazzo 8f94e8b943 comm: make sure that our version check is reliable
Rework the logic of the version check used in the
database migration, and make sure
that it is full functional to avoid confusion
at release time.

Changelog-Fixed: database: Correctly identity official release versions for database upgrade.

Reported-by: @urza
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-17 14:15:24 +10:30
Christian Decker bcc75b6e98 tests: Fix a small memory leak in the onion test vector tester
Changelog-None
2023-01-16 13:53:48 +10:30
Rusty Russell 5dfcd15782 all: no longer need to call htable_clear to free htable contents.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell f07e37018d setup: make all htables use tal.
This makes them easier to clean up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 851cbf6c83 memleak: prepare for htable to be a tal object.
Since it gets resized during traverse, we would crash by
keeping a pointer to the old one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 4a570c9419 gossmap: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell 2d8fff6b57 libplugin: don't turn non-string JSON ids into strings.
When called with `"id": 1` we replied with `"id": "1"`.  lightningd doesn't
actually care, but it's weird.

Copy the entire token: this way we don't have to special case anything.

Also, remove the doubled test in json_add_jsonstr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell efd7a5868b common/test: fix typo in bolt12 test vector generation.
Reported-by: @jkczyz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-06 20:44:50 +01:00
tony 4b98df186d common: update comments documenting the use of param()
The file common/json_tok.c has been removed in a previous
commit.

Changelog-None
2023-01-04 15:13:42 +01:00
Rusty Russell 66bde4bd9f lightningd: only allow closing to native segwit
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `close` `destination` no longer allows p2pkh or p2sh addresses (deprecated v0.11.0)
2022-12-13 08:28:12 +10:30
Michael Schmoock 0ae6f4d6fb wireaddr: allow for UpperCase DNS names
This further relaxes the DNS hostname checks to allow for uppercase
input.

Changelog-None
2022-12-07 13:20:33 +01:00
Michael Schmoock fb9d6684ad wireaddr: adds test for punycode
Changelog-None
2022-12-06 14:26:19 +01:00
Michael Schmoock 29f81baac9 wireaddr: is_dnsaddr allow underscore in hostname
The hostname part of a DNS FQDN can allow for additional characters
other than specified in `man 7 hostname`.

This extends is_dnsaddr and the test issue #5657.
Also fixes a typo in a comment.

Changelog-Fixed: wireaddr: #5657 allow '_' underscore in hostname part of DNS FQDN
2022-12-06 14:26:19 +01:00
Rusty Russell 37590eeeb2 common: fix arm32 compile breakage.
It's obviously the incorrect type, while our CI didn't catch it, Nicholas did:

```
plugins/fetchinvoice.c:1362:30: error: conversion from 'long long unsigned int' to 'size_t' {aka 'unsigned int'} changes value from '18446744073709551615' to '4294967295' [-Werror=overflow]
 1362 |      || tlv_span(wire, 1001, UINT64_MAX, NULL) != 0) {
```

Reported-by: @NicholasDorier
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Build: arm32 compiler error in fetchinvoice, due to bad types on 32-bit platforms.
Fixes: #5776
2022-12-06 10:44:39 +01:00
Rusty Russell 8a217f13cf bolt12: update comments to match latest spec.
No code changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 02d7454226 db: add invoicerequests table.
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly.  Create a new table for them, and
associated functions.

The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
2022-11-09 15:08:03 +01:00
Rusty Russell 891cef7b2b bolt12: routines to hash the invreq parts.
This gives us a unique identifier, by which we can match an invoice to
their invoice_request.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 179f573e45 lightningd/invoice.c, plugins/fetchinvoice.c: use tlv_make_fields() instead of towire/fromwire hack.
I forgot this existed!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 505356145b bolt12: update to modern signature scheme.
This changed, by popular demands.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 1e3cb01546 bolt12: import the latest spec, update to fit.
I know this is an unforgivably large diff, but the spec has changed so
much that most of this amounts to a rewrite.

Some points:
* We no longer have "offer_id" fields, we generate that locally, as all
  offer fields are mirrored into invoice_request and then invoice.
* Because of that mirroring, field names all have explicit offer/invreq/invoice
  prefixes.
* The `refund_for` fields have been removed from spec: will re-add locally later.
* quantity_min was removed, max == 0 now mean "must specify a quantity".
* I have put recurrence fields back in locally.

This brings us to 655df03d8729c0918bdacac99eb13fdb0ee93345 ("BOLT 12:
add explicit invoice_node_id.")

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 9a0d2040d3 common/features: add explicit bolt12 feature sets.
The spec only specifies the mpp bit for invoices, but in
general they are separate spaces.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 6e755d6fe8 common/bolt12: code to initialize invreqs from offers, invs from invreqs.
This is an important part of the coming spec: we mirror all fields,
known and unknown.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell aa73878831 common/bolt12: add code to generate offer_id, extract parts of streams.
The new spec removes the offer_id, in favor of mirroring all the
fields.  So we need a way of generating a convenient identifier to
identify the offer, and this works.

We also want to extract parts of streams elsewhere, so expose that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 4bc10579e6 listincoming: add htlc_min_msat, public and peer_features fields.
This is needed for offers to generate blinded paths.

No documentation changes since listincoming is an undocumented
internal hack interface which topology presents for production
of routehints.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell c6f50220e1 common/onion_decode: put the path_id into onion_payload->payment_secret.
And check it in invoice.c, insead of a hack where we compare against invhash.
Restore checking, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 595fbd2a19 createinvoice: make a minimal blinded "path" in bolt12 invoice if none presented.
The "path" is just a message to ourselves.  This meets the minimal
requirement for bolt12 invoices: that there be a blinded path (at
least so we can use the path_id inside in place of "payment_secret").

We expose the method to make this path_id to a common routine: offers
will need this for generating more sophisticated paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 5becfa6ee1 onion_message: don't use general secret, use per-message secret.
We had a scheme where lightningd itself would put a per-node secret in
the blinded path, then we'd tell the caller when it was used.  Then it
simply checks the alias to determine if the correct path was used.

But this doesn't work when we start to offer multiple blinded paths.
So go for a far simpler scheme, where the secret is generated (and
stored) by the caller, and hand it back to them.

We keep the split "with secret" or "without secret" API, since I'm
sure callers who don't care about the secret won't check that it
doesn't exist!  And without that, someone can use a blinded path for a
different message and get a response which may reveal the node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 4cfd972407 common/blindedpath: expose API at a lower level.
We actually want lightningd to create these, since it wants to put the
path_id secret in the last element.  So best API is actually a generic
one, rather than separate APIs to create first and last ones.

And really, the more explicit initialization makes the users clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 8720bbedae common/onion: split into decode and encode routines.
Some places (e.g. the pay plugin) only need to construct onions,
not decode them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell c5656ec90a common/onion: handle payment by node_id.
In a blinded path, you can specify node_id instead of scid.  Handle
that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 2760490d5d common: catch up on latest routeblinding spec.
This makes us match eed2ab0c30ad7f93e3b2641ca9d7ade32f3d121d
("Use `invalid_onion_blinding` everywhere").

1. Numerous typographical changes.
2. Make sure we *always* return WIRE_INVALID_ONION_BLINDING if
   we're in a blinded path.
3. Handle p->total_msat correctly (MPP payments).
4. Reorganize blinding handling just like spec order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00