Commit Graph

944 Commits

Author SHA1 Message Date
Rusty Russell 39c93ee6e5 connectd: get addresses from lightningd, not gossipd.
It's weird to have connectd ask gossipd, when lightningd can just do it
and hand all the addresses together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 6115ed02e8 subdaemons: don't stream gossip_store at all.
We now let gossipd do it.

This also means there's nothing left in 'struct per_peer_state' to
send across the wire (the fds are sent separately), so that gets
removed from wire messages too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 9c0bb444b7 per_peer_state: remove struct crypto_state
Now that connectd does the crypto, no need to hand around crypto_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 71f736678f lightningd: make sure gossipd knows before we update blockheight.
Without this, we can get spurious failures from lnprototest, which
is waiting for lightningd to acknowledge the blockheight in getinfo:

```
2021-12-21T00:56:21.460Z DEBUG   lightningd: Adding block 109: 57a7bd3ade3a88a899e5b442075e9722ccec07e0205f9a913b304a3e2e038e26
2021-12-21T00:56:21.470Z DEBUG   lightningd: Adding block 110: 11a280eb76f588e92e20c39999be9d2baff016c3c6bac1837b649a270570b7dd
2021-12-21T00:56:21.479Z DEBUG   lightningd: Adding block 111: 02977fc9529b2ab4e0a805c4bc1bcfbff5a4e6577a8b31266341d22e204a1d27
2021-12-21T00:56:21.487Z DEBUG   lightningd: Adding block 112: 2402f31c5ddfc9e847e8bbfb7df084d29a5d5d936a4358c109f2f4cf9ea8d828
2021-12-21T00:56:21.496Z DEBUG   lightningd: Adding block 113: 5a561fe9423b4df33f004fc09985ee3ef38364d692a56a8b27ecbc6098a16d39
2021-12-21T00:56:21.505Z DEBUG   lightningd: Adding block 114: 4502f5ec23c89177872846848848322e8fa6c3fb6f5eb361194e4cd47596dfe9
2021-12-21T00:56:21.511Z DEBUG   02f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9-gossipd: Ignoring future channel_announcment for 109x1x0 (current block 108)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 562e6c4a93 update mocks
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell d287d7d229 gossipd: make request handlers return void.
They all returned the next io_plan, but it was always the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 0aad222c2d gossipd: rewrite update generation, remove local_chan.
local_chan was mainly around so we could "soft" disable channels (and
really disable them once we used the channel_update in an error
message).

Instead we introduce the idea of a "deferred_update": it's either
deferred indefinitely (a peer goes offline, if we need to send it in
an error we'll apply it immediatly), or simply delayed to avoid
spamming everyone.

The resulting rewrite is much clearer, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 8e37eb0028 gossipd: local_direction helper to generalize is_local_channel.
Increasingly we want to know is it local, and get the direction: it's
more efficient to do both at once.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Simon Vrouwe 605fda7214 common: cleanup unsused parameter in timer_expired() 2021-12-14 09:33:10 +10:30
Rusty Russell d22fd59997 gossipd: remove gossip_msg.[ch]
This was a remnant from when we used to get routing from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 484222b0a1 daemons: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 1c230eee86 test: hack in test for (now-obsolete!) test vector Thomas emailed me.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 5361107c9c gossipd: handle receipt of modern onion message.
And wire it through to the hook; update the plugins to recognize
modern vs obs2 onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell e7b263304e lightningd: Send updated onion spec messages.
It's very similar to the previous, but there are a few changes:

1. The enctlv fields are numbered differently.
2. The message itself is a different number.

The onionmsg_path type is the same, however, so we keep that constant
at least.

The result is a lot of cut & paste, but we will delete the old one
next release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 1ec6346f3d common: rename current onion message structures to obs2_.
Yes, we changed the spec again.  Hopefully for the last time!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell b74848f6f6 common: remove support for pre v0.10.2 onionmessages.
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.

Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell 57328fe59e tests: use common_setup/common_shutdown to avoid leaks.
It also does more checks (like taken() checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell 9c254c8387 gossipd: when we refresh channel, don't override pending updates.
We can have an update pending because it's too fast, but
refresh_local_channel is supposed to make sure we're up-to-date, so
force immediate application in that case.

Otherwise, we call update_local_channel at the bottom which frees the
pending update.  This can mean that we miss a change in fees, for example.

Changelog-Fixed: errors: Errors returning a `channel_update` no longer return an outdated one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-21 20:05:54 +02:00
Rusty Russell 1c85b27b4c gossipd: remove ping/pong handling
To minimize the diffs, we #if 0 the code.  We'll reenable it once
channeld is ready.

We also temporarily disable the ping tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell 6c8c6a7c7d gossipd: handle modern onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell f9a21d9fc9 lightningd: handle modern onion termination.
This adds a new hook: onion_message_ourpath for when we know a message
came in via a blinded path we created.  The onion_message_blinded hook
is now called for all other messages, since all messages are now
blinded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Rusty Russell f2a4bd6ad8 wire: import new onion message spec.
One change from the obsolete version handling, gossipd will no longer send
forwarding onion msgs to lightningd, but will forward it directly.
That was the effect before, anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell 191798bb56 gossipd: rename onionmsg messages to lightningd to obs_onionmsg.
The new ones are slightly different.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell e3ae7883bb channeld: rename onion_message to obs_onion_message.
This splits the existing old-spec pathways to prepare for the
new ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell 8b694907e6 gossipd: fix blinded onion forwarding.
We never tested that we can correctly unwrap on the next step after
unblinding: it failed because we mangled the onion in place!  Fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell 24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell 7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell a5fee67b91 common/memleak: take over dump_memleak(), allow print pointer.
This will let plugins use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
Rusty Russell 87239dc691 gossipd: de-duplicate code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-20 13:28:38 -04:00
niftynei 8819278a88 lease rates: helpers for parsing, serializing + equating lease_rates
We're gonna need them
2021-07-20 13:28:38 -04:00
niftynei 265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei 9e839b177c gossipd: turn off lease offers if the rates are empty
If received lease rates are empty (all zeroes), turn them off
2021-07-20 13:28:38 -04:00
niftynei 876ff6821e setleaserates: new RPC to pass in lease rates
Changelog-Experimental: JSON-RPC: new RPC `setleaserates`, for passing in the rates to advertise for a channel lease (option_will_fund)
2021-07-20 13:28:38 -04:00
niftynei 48d244c304 gossipd: signal whether node_ann is updated, are we starting up?
If there's a rate-card for liquidity, we don't know about it until
after startup (the plugin *should* call us at init to tell us what their
current rates are)
2021-07-20 13:28:38 -04:00
niftynei 55f5a5caba gossip/liquidity-ad: node_ann comparison, optional TLV check
Correctly mark whether or not the TLV's are the same/different, given two
node announcements
2021-07-20 13:28:38 -04:00
niftynei 49bae2a873 lease_rates: parse them out of node_announcements
When we get a node announcement out, we can now pull out its offer
characteristic
2021-07-20 13:28:38 -04:00
niftynei 5b3c02f401 liquidity-ads: import from spec
Import the wires from spec. Here we go!
2021-07-20 13:28:38 -04:00
Christian Decker b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Rusty Russell 75720ad0e1 lightningd: wait for gossipd to finish initalizing before starting plugins.
This mainly helps our CI under valgrind, which starts a fresh instance
and immediately calls the invoice command.  This can cause the topology
plugin to try to access the gossmap file before it's created.

We can also move the gossmap reading in topology to init time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 7e7ab4cb3b gossipd: remove routing, listchannels and listnodes infrastructure.
This involves removing some fields from the now-misnamed routing.h
datastructures, and various internal messages.

One non-obvious change is to our "keepalive" logic which refreshes
channels every 13 days: instead of using the 'enabled' flag on the
last channel broadcast to decide whether to refresh it, we use the
local connected status directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30