Commit Graph

347 Commits

Author SHA1 Message Date
Rusty Russell 93caef3584 common/tlvstream: allow fromwire_tlv to accept *any* unknown type.
This is used for keysend if EXPERIMENTAL_FEATURES is set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 83ee68ab06 common/tlvstream: put TLV checking back in the generic function.
Callers were supposed to call "tlv_fields_valid" after fromwire_tlv,
but few did.  Make this the default, and call the underlying function
directly where we want to be more flexible (one place).

This loses the ability to allow misordered fields, or to pass through
*any* even fields.  We restore that for special cases in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell a770f51d0e tools/generate_wire.py: make functions allocate the TLV.
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 2b422272e5 wire: generate printwire_ routines for bolt12 CSV.
Good exercise, and we need it when bolt12 messages appear in onion
messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell fa0c29f959 tools/generate_wire.py: tlvs should start with tlv_
No more "towire_offer", but "towire_tlv_offer".

This means we double-up on the unfortunately-named `tlv_payload` inside
the onion, but we should rename that in the spec when we remove
old payloads.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell 4d0a97e1bd offers: import latest variant from draft, part 2.
Notably, the latest draft uses the correct tlv types inside the onion,
but we don't want to (we'd prefer to demarshal those as a separate step,
for better diagnostics), so we change it, then add a spec patch to change
it back.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 726b6878d1 offers: import latest variant from draft.
In particular, this changes the name of a field in invoice_request:
`payer_signature` becomes simply `signature`.  So we allow both for
now, and send the old one unless deprecated_apis is disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 6f7d51ee68 common/onion: always fill in failtlvtype and failtlvpos parameters.
When we did fill them in, we filled them in wrong: the offset should be
the offset in the message, not the field number!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Michael Schmoock 38e2abf68a peer_exchange: set, read and log remote_addr
Changelog-Added: Protocol: set remote_addr on init tlvs
2022-02-22 05:45:47 +10:30
Michael Schmoock 03a1df074a peer_wire: add remote_addr to init_tlv
Unfortunately we can't do any smart parsing here since
wiregen does not support switch/type cases for different
substructure unions yet. So just give us a pointer we can use.
2022-02-22 05:45:47 +10:30
Rusty Russell 1c71c9849b connectd: handle custom messages.
This is neater than what we had before, and slightly more general.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `sendcustommsg` now works with any connected peer, even when shutting down a channel.
2022-02-08 11:15:52 +10:30
Rusty Russell 8782d39476 connectd: handle onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell a2b3d335bb connectd: do decryption for peers.
We temporarily hack to sync_crypto_write/sync_crypto_read functions to
not do any crypto, and do it all in connectd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 35e3c1866e common: generalize extract_channel_id().
connectd is going to end up using this do demux; make it fast and complete.

Fixing this reveals a problem in openingd: it now extracts the channel_id
from funding_signed (which is where we transition off the temporary), and
gets upset.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 6ce7e7ea60 common: is_msg_gossip_broadcast() for putting things in filter.
We don't need the other "gossip" messages in our echo-suppression filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 9a85b02c6f wire: add missing spec patch.
You have to manually run "make extract-bolt-csv" to recreate the
spec files, so we didn't notice that I forgot to check this in :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-15 06:53:39 +10:30
Rusty Russell 484222b0a1 daemons: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell d9968bbc0c bitcoin: remove unused functions, or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 296437c655 bolt12: remove deprecated `chains` fields
We also move recurrence fields into a separate spec patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 1ec6346f3d common: rename current onion message structures to obs2_.
Yes, we changed the spec again.  Hopefully for the last time!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell b74848f6f6 common: remove support for pre v0.10.2 onionmessages.
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.

Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell 1c85b27b4c gossipd: remove ping/pong handling
To minimize the diffs, we #if 0 the code.  We'll reenable it once
channeld is ready.

We also temporarily disable the ping tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell 7157a92ea6 channeld: import updated channel_upgrade spec.
It now uses raw bitfields instead of a subtype, and only allows a single
option for any upgrade.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell c92ce59892 BOLT 12: switch invoice_request/invoice to singular `chain` field.
We keep the now-removed chains field, and in deprecated mode, we set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
2021-10-08 13:47:30 +02:00
Rusty Russell 45bf7a3974 bolt12: update to latest spec.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
2021-10-08 13:47:30 +02:00
Rusty Russell 79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Vincenzo Palazzo eb103c15df review 2/2: Use generic query to make the code more readable.
Suggested by @cdecker

P.S: Also this include an API refactoring from my previous solution, also this it is suggested by @cdecker.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Rusty Russell f2a4bd6ad8 wire: import new onion message spec.
One change from the obsolete version handling, gossipd will no longer send
forwarding onion msgs to lightningd, but will forward it directly.
That was the effect before, anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell e3ae7883bb channeld: rename onion_message to obs_onion_message.
This splits the existing old-spec pathways to prepare for the
new ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell 24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Jan Sarenik 3f05dff89d wire: Fix generated files
Changelog-None
2021-09-20 18:17:56 +02:00
Rusty Russell 7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
niftynei 23a6c00496 Makefile: import latest spec, which includes quickclose.
It also gets rid of the requirement that close negotiation fee maximum
is the old commitment transaction.  We still do that, however, to
avoid surprising old peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell 19413a821f wire/tlvstream: don't leak in tlvstream_set_tu64/tu32/short_channel_id.
Another tiny leak, which happened in keysend which uses these routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 0a05bcd05e wire: fix patch file to match actual peer_wire.csv.
Running `make extract-peer-csv` changes peer_wire.csv, @niftynei
points out that the patch file is wrong (peer_wire.csv is correct).

(Any attempt to regenerate the csv files from the specs hits this issue)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-02 14:04:32 +09:30
Rusty Russell ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
Rusty Russell 7769903f1a bolt12: import latest spec (timestamp -> created_at).
@shesek points out that we called this field created_at in bolt11 decode,
which makes more sense anyway.

Changelog-EXPERIMENTAL: bolt12 decode `timestamp` field deprecated in favor of new name `created_at`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
niftynei 265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei 5b3c02f401 liquidity-ads: import from spec
Import the wires from spec. Here we go!
2021-07-20 13:28:38 -04:00
niftynei 376e6f8bd1 dual-funding: update fee_step to be a feerate
Using a 'feestep' is more restrictive than you'd want, instead we
enforce that the next feerate must be at least 1/64th more than the
last, but put no upper limit on it

Includes update to lnprototest changes

Contributed-By: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: Protocol: Replaces init_rbf's `fee_step` for RBF of v2 opens with `funding_feerate_perkw`, breaking change
2021-07-19 16:13:24 -04:00
Rusty Russell 424ac84263 patch remove-payer-backcompat.patch 2021-07-14 14:33:18 -05:00