Commit Graph

384 Commits

Author SHA1 Message Date
Rusty Russell 27ef4b98c1 devtools: remove warning message from dump-gossipstore.
It prints a message to stderr, but actually it's fine with this version:

```
dump-gossipstore: UNKNOWN GOSSIP minor VERSION 14 (expected 12)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-12 11:43:33 +01:00
Rusty Russell 07cd4a809b gossipd: remove spam handling.
We weakened this progressively over time, and gossip v1.5 makes spam
impossible by protocol, so we can wait until then.

Removing this code simplifies things a great deal!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer ratelimit gossip messages by channel, making our code far simpler.
2024-02-04 09:24:44 +10:30
Rusty Russell e7ceffd565 gossipd: remove zombie handling.
We never enabled it, because we seemed to be eliminating valid
channels.  We discard zombie-marked records on loading.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-04 09:24:44 +10:30
Rusty Russell f2f43eeffa gossipd: strip private updates from gossip_store on startup.
We rename them to _obs, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Peter Neuroth 3cbe35e7da devtools: Deduplicate links from linkify
The method linkify just added links for an entry to a list and produced
duplicates if we got more than one changelog line in a PR. We now only
produce a link once per PR.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-01-15 18:36:01 +01:00
Rusty Russell e609bc934e channeld: tell lightningd about local anchor for each commitment tx.
It's going to want to remember these, in case it encounters peers'
commitment tx and needs to boost it with CPFP on the anchor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell 9b3965c805 channeld: rename channel_splice_txs to channel_txs.
channel_txs was a thin wrapper around channel_splice_txs, but that's
just confusing.  Rename channel_splice_txs to channel_txs, and just
call it everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell fea4eb8968 common: helper to generate bolt12 test vector bolt12/offers-test.json.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 13:31:03 +09:30
Rusty Russell 16e0d38fff bolt12-cli: add encode routine to make bolt12 vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 13:31:03 +09:30
Alex Myers bd4a001279 cli: don't crash when there's no argument
This should provide the default help message and exit, but was
resulting in a segmentation fault from freeing pointers passed to
the default config.

Changelog-Fixed: lightning-cli properly returns help without argument
2023-08-16 12:25:25 +09:30
Rusty Russell 5ba4713b21 devtools/credit: ! not ^ to invert the match set in sh.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-31 17:17:15 +09:30
grubles a1d25961de devtools/print_wire: long long instead of longing
adding PRId64 magic instead

Changelog-None
2023-07-24 15:24:05 +09:30
Rusty Russell 8178b7389f gossip_store: add "dying" flag to indicate not to gossip dying channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-20 11:47:32 +09:30
Dustin Dettmer aba4d18ed1 signed types: add handlers for signed types
We're adding signed types to the spec! This adds the support mechanisms
for them.
2023-07-13 19:34:47 +09:30
Rusty Russell af6d7c0779 global: thread zero fee option everywhere.
In most cases, it's the same as option_anchor_outputs, but for
fees it's different.  This transformation is the simplest:
pass it as a pair, and test it explicitly.

In future we could rationalize some paths, but this was nice
and mechanical.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell c1d3da889c tools/gossipwith: shutdown, don't close, TCP socket.
This isn't the cause of the test_gossip_ratelimit flake I saw (since
the final gossip msg clearly was received), but it's still good to fix
since it means we might not send the final messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 17:24:48 +09:30
Matt Morehouse e81bf8b89f devtools: call common_setup in main
Fixes nullability errors detected by UBSan:

wire/fromwire.c:173:46: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:432:3: note: nonnull attribute specified here
    #0 0x65214a in fromwire_secp256k1_ecdsa_signature wire/fromwire.c:173:6
    #1 0x659500 in printwire_secp256k1_ecdsa_signature devtools/print_wire.c:331:1
    #2 0x646ba2 in printwire_channel_update wire/peer_printgen.c:1900:7
    #3 0x637182 in printpeer_wire_message wire/peer_printgen.c:128:11
    #4 0x65a097 in main devtools/decodemsg.c:85:10
2023-06-05 16:16:21 +02:00
Rusty Russell edbaa944da lightningd: switch parsing to common/configvar
Now we wire in the code which gathers configvars and parses from there;
lightningd keeps the array of configuration variables for future use.

Note that lightning-cli also needs to read the config, but it has its
own options (including short ones!) and doesn't want to use this
configvar mechanism, so we have a different API for that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell f45c17450b ccan: update to latest ccan/opt
This adds:
1. ability to search for an option by name.
2. allowance to set our own bits when registering options.
3. show callbacks which can say "don't show", and variable length.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell a6772e9dec common: add new internal type for websockets.
Now it's not a public type, we need a way to refer to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell 3f35d48fe4 common: remove websocket type from wireaddr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell d40379885d common/wireaddr.h: simplify parse_wireaddr API.
1. Make it the standard "return the error" pattern.
2. Rather than flags to indicate what types are allowed, have the callers
   check the return explicitly.
3. Document the APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell 4deb552fe9 build: don't generate experimental variants of wire files.
We no longer have any experimental-only wire definitions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell 295557ac50 connectd: don't try to set TCP_CORK on websocket pipe.
Most of this is piping the flag through so we know it's a websocket!

Reported-by: @ShahanaFarooqui
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 09:41:56 +09:30
Christian Decker 67a39b59e7 tools: Add yml mode to `blockreplace.py` 2023-03-25 15:40:35 +10:30
Rusty Russell 73e50b26f9 devtools: fix ZOMBIE detection in devtools/dump-gossipstore.
A victim of simultaneous changes, and I didn't pick it up :(
    
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-15 14:19:42 -06:00
Rusty Russell 0274d88bad common/gossip_store: clean up header.
It's actually two separate u16 fields, so actually treat it as
such!

Cleans up zombie handling code a bit too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Alex Myers 6bff10cd40 gossip_store: add a flag for zombie entries
This will allow gossipd to store and persist gossip for channels rather
than deleting them entirely when the channels are pruned from the
network.
2023-01-30 16:33:03 +10:30
Christian Decker a31575ca0b tools: Add multi-language support to blockreplace.py
Suggested-by: Rusty Russell <@rustyrussell>
2022-12-06 11:39:53 +10:30
Christian Decker 744d111cea doc: Create a blockreplace tool to update generated blocks in docs
We introduced a minor issue in #5757 that was causing the manpages to
be added every time we regenerate instead of replacing them. The
script was a bit unscrutable, and we do this block-replacement in
several places I thought it might be a good idea to have a dedicated
tool to do it.

This allows us to have simpler Makefiles whenever we update a
generated block inside another file. It's python, but does not have
dependencies :-)

Changelog-None
2022-12-06 11:39:53 +10:30
Christian Decker dfb963e249 db: Backfill missing HTLC IDs in the forwards table
We have a primary key that is spanning the `in_channel_id` and the
`in_htcl_id`. The latter gets set to NULL when the HTLC and channel
gets deleted, so we coalesce with a random large number that is
unlikely to collide for the primary key.
2022-11-28 14:47:43 +01:00
Rusty Russell 2e4a58efac check: fix warnings from shellcheck 0.8.0
And make errors gcc-style, so emacs can jump through the automatically.

```
In devtools/reduce-includes.sh line 21:
	    echo -n "-$LINE"
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In devtools/reduce-includes.sh line 25:
	    echo -n "."
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In tools/rel.sh line 6:
prefix=$(printf '%s\n' "${from#$common}" | sed 's@[^/][^/]*@..@g')
                               ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
prefix=$(printf '%s\n' "${from#"$common"}" | sed 's@[^/][^/]*@..@g')


In tools/rel.sh line 7:
printf '%s\n' "$prefix/${to#$common}"
                            ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
printf '%s\n' "$prefix/${to#"$common"}"

For more information:
  https://www.shellcheck.net/wiki/SC3037 -- In POSIX sh, echo flags are undef...
  https://www.shellcheck.net/wiki/SC2295 -- Expansions inside ${..} need to b...
make: *** [Makefile:553: check-shellcheck] Error 123
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 8a217f13cf bolt12: update comments to match latest spec.
No code changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 37bc4603b8 lightningd: re-add 'offerout' functionality, as 'invoicerequest'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell fdb3d9186a devtools/bolt12-cli: fix decode to understand modern fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 1e3cb01546 bolt12: import the latest spec, update to fit.
I know this is an unforgivably large diff, but the spec has changed so
much that most of this amounts to a rewrite.

Some points:
* We no longer have "offer_id" fields, we generate that locally, as all
  offer fields are mirrored into invoice_request and then invoice.
* Because of that mirroring, field names all have explicit offer/invreq/invoice
  prefixes.
* The `refund_for` fields have been removed from spec: will re-add locally later.
* quantity_min was removed, max == 0 now mean "must specify a quantity".
* I have put recurrence fields back in locally.

This brings us to 655df03d8729c0918bdacac99eb13fdb0ee93345 ("BOLT 12:
add explicit invoice_node_id.")

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 8720bbedae common/onion: split into decode and encode routines.
Some places (e.g. the pay plugin) only need to construct onions,
not decode them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
niftynei 7bd0d7641c mkfunding: no scriptPubKey on utxo causing crash, so we add one
Changelog-Fixed: devtools: `mkfunding` command no longer crashes (abort)

Fixes #5363

Assisted-By: @TKChattoraj
2022-11-01 18:23:00 +01:00
niftynei 87616b7ffc mkfunding: add missing common_setup
Was crashing on allocating wally memory, b/c was missing the wally_init
that's wrapped up in common_setup

Assisted-By: @TKChattoraj
2022-11-01 18:23:00 +01:00
Rusty Russell 5cf86a1a2e common: update to latest onion message spec.
Mainly, field name changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Protocol: Support for forwarding blinded payments (as per latest draft)
2022-10-26 11:29:06 +10:30
Rusty Russell 511e8e6477 common/blindedpay: routines to construct a blinded payment.
Don't shoehorn it into onion_nonfinal_hop() and onion_final_hop(), but
provide an explicit routine "blinded_onion_hops" and an onion helper
"onion_blinded_hop()" for it to call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 077ec99788 common/onion: blinded payment support.
We make it look like a normal payment for the caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 662c6931f3 Remove point32.
The x-only dream is dead.  Remove all trace.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell e30ea91908 BOLTs: update to more recent bolt12 spec.
It's 2b7ad577d7a790b302bd1aa044b22c809c76e49d, which reverts the
point32 changes.

It also restores send_invoice in `invoice`, which we had removed
from spec and put into the recurrence patch.

I originally had implemented compatibility, but other changes
which followed this are far too widespread.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: complete rework of spec from other teams (yay!) breaks previous compatibility (boo!)
2022-10-26 11:29:06 +10:30
Rusty Russell bed905a394 lightningd: use 33 byte pubkeys internally.
We still use 32 bytes on the wire, but internally don't use x-only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 125b17b7fc devtools: enhance bolt12-cli to convert to/from hex
This makes it easier to add/delete fields manually to a bolt12 encoding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell f00cc23f67 sphinx: rename confusing functions, ensure valid payloads.
"sphinx_add_hop" takes a literal hop to include,
"sphinx_add_modern_hop" prepends the length.  Now we always prepend a
length, make it clear that the literal version is a shortcut:

* sphinx_add_hop -> sphinx_add_hop_has_length
* sphinx_add_modern_hop -> sphinx_add_hop

In addition, we check that length is actually correct!  This means
`createonion` can no longer create legacy or otherwise-invalid onions:
fix tests and update man page to remove legacy usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `createonion` no longer allows non-TLV-style payloads.
2022-09-28 13:40:57 +02:00
Rusty Russell 8771c86379 common/onion: expunge all trace of different onion styles.
In particular, remove special routines to pull length: it's there,
take it and check it yourself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell a1f62ba0e7 gossipd: don't close non-local channels immediately, add 12 block delay.
This adds a new "chan_dying" message to the gossip_store, but since we
already changed the minor version in this PR, we don't bump it again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now delay forgetting funding-spent channels for 12 blocks (as per latest BOLTs, to support splicing in future).
2022-09-24 15:22:27 +09:30
Rusty Russell 253b25522b BOLT: update to version which requires option_channel_htlc_max.
We will now simply reject old-style ones as invalid.  Turns out the
only trace we could find is a channel between two nodes unconnected to
the rest of the network.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now require all channel_update messages include htlc_maximum_msat (as per latest BOLTs)
2022-09-24 15:22:27 +09:30