Commit Graph

281 Commits

Author SHA1 Message Date
Rusty Russell b5a1715c2b connectd: also fail without a scary backtrace when listen fails.
For example, if you do:

```
./lightningd/lightningd --network=regtest --experimental-websocket-port=19846
```

Then you're trying to reuse the normal port as the websocket port, but this
only fails at *listen* time, when we activate connectd.  Catch this too.

Fixes incorrect fatal() message, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Rusty Russell 885a6f50ae connectd: make sure we announce websocket addr which succeeded.
By accessing `addr` after the loop, it's possible that it's one which
failed, in complex scenarios.

Also gives us a chance to warn if they specify a websocket but don't
actually end up advertizing it (you *must* advertize a normal addr as
well).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Rusty Russell c075d78431 connectd: use listen_fd array directly, rather than returning binding arr.
We always added to both arrays, might as well just keep one.  

We make mayfail an explicit flag, rather than relying on the presence
of errstr, which is never NULL now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Rusty Russell a62f5e5d82 connectd: hoist find_local_address so we can give more graceful Tor erros.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Rusty Russell 200a8a985b connectd: add is_websocket and wireaddr to struct listen_fd.
This lets us give a better error message if listen fails, and also
moved the callback closer to where it's needed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Rusty Russell f1ed373c97 connectd: be more graceful when we an address is in use.
Aditya had this issue due to a config line, and the result was hard
to diagnose even for me.

It's now:

```
$ ./lightningd/lightningd --network=regtest --addr=:18444
2022-02-26T05:01:28.705Z **BROKEN** connectd: Failed to bind socket for 0.0.0.0:18444: Address already in use
```

Whereas before it doesn't even give the address it's trying to bind:

```
rusty@rusty-XPS-13-9370:~/devel/cvs/lightning (master)$ ./lightningd/lightningd --network=regtest --addr=:18444
lightning_connectd: Failed to bind on 2 socket: Address already in use (version v0.10.2-331-g86b83e4)
0x558a8b8d9a12 send_backtrace
	common/daemon.c:33
0x558a8b8e91e1 status_failed
	common/status.c:221
0x558a8b8c8e4f make_listen_fd
	connectd/connectd.c:1090
0x558a8b8c8f55 handle_wireaddr_listen
	connectd/connectd.c:1129
0x558a8b8c993d setup_listeners
	connectd/connectd.c:1312
0x558a8b8ca344 connect_init
	connectd/connectd.c:1517
0x558a8b8cbb57 recv_req
	connectd/connectd.c:1896
0x558a8b8d9f9f handle_read
	common/daemon_conn.c:31
0x558a8b9247c1 next_plan
	ccan/ccan/io/io.c:59
0x558a8b9253c9 do_plan
	ccan/ccan/io/io.c:407
0x558a8b92540b io_ready
	ccan/ccan/io/io.c:417
0x558a8b9276fe io_loop
	ccan/ccan/io/poll.c:453
0x558a8b8cbf36 main
	connectd/connectd.c:2033
0x7fe4d02940b2 ???
	???:0
0x558a8b8c285d ???
	???:0
0xffffffffffffffff ???
	???:0
2022-02-26T05:02:27.547Z **BROKEN** connectd: Failed to bind on 2 socket: Address already in use (version v0.10.2-331-g86b83e4)
2022-02-26T05:02:27.547Z **BROKEN** connectd: backtrace: common/daemon.c:38 (send_backtrace) 0x558a8b8d9a68
2022-02-26T05:02:27.547Z **BROKEN** connectd: backtrace: common/status.c:221 (status_failed) 0x558a8b8e91e1
2022-02-26T05:02:27.547Z **BROKEN** connectd: backtrace: connectd/connectd.c:1090 (make_listen_fd) 0x558a8b8c8e4f
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1129 (handle_wireaddr_listen) 0x558a8b8c8f55
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1312 (setup_listeners) 0x558a8b8c993d
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1517 (connect_init) 0x558a8b8ca344
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1896 (recv_req) 0x558a8b8cbb57
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: common/daemon_conn.c:31 (handle_read) 0x558a8b8d9f9f
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x558a8b9247c1
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x558a8b9253c9
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x558a8b92540b
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x558a8b9276fe
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:2033 (main) 0x558a8b8cbf36
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: (null):0 ((null)) 0x7fe4d02940b2
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: (null):0 ((null)) 0x558a8b8c285d
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: (null):0 ((null)) 0xffffffffffffffff
2022-02-26T05:02:27.548Z **BROKEN** connectd: STATUS_FAIL_INTERNAL_ERROR: Failed to bind on 2 socket: Address already in use
lightningd: connectd failed (exit status 242), exiting.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Michael Schmoock df9a34b81e chore: use EXPERIMENTAL for BOLT1 remote_addr #917 2022-02-22 05:45:47 +10:30
Michael Schmoock 38e2abf68a peer_exchange: set, read and log remote_addr
Changelog-Added: Protocol: set remote_addr on init tlvs
2022-02-22 05:45:47 +10:30
Rusty Russell d4fee837c2 misc: clarifications from cdecker review.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 727b486d49 connectd: don't received useless peer fd if we're told to send final msg.
We don't need the connection to ourselves, just to free it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell ca08f27d54 connectd: remove second gossip fd.
Now we only send and receive gossip messages on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 3121cebf4c gossipd: don't hand out fds.
Gossipd now simply gets told by channeld when peers arrive or leave.
(it only needs to know for the seeker).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 3c5d27e3e9 subdaemons: remove gossipd fd from per-peer daemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 1c71c9849b connectd: handle custom messages.
This is neater than what we had before, and slightly more general.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `sendcustommsg` now works with any connected peer, even when shutting down a channel.
2022-02-08 11:15:52 +10:30
Rusty Russell 960e911986 connectd: do io logging properly for msgs we make.
We don't need to log msgs from subds, but we do our own, and we weren't.

1. Rename queue_peer_msg to inject_peer_msg for clarity, make it do logging
2. In the one place where we're relaying, call msg_queue() directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 8782d39476 connectd: handle onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 50eccb6a12 connectd: handle pings and pongs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `ping` now works with connected peers, even without a channel.
2022-02-08 11:15:52 +10:30
Rusty Russell d7cf38a80a connectd: divert gossip messages directly to gossipd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 9983c2fd8e gossipd: add routines to send gossip messages to and from connectd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell bba468a51c connectd: temporarily have two fds to gossipd.
We want to stream gossip through this, but currently connectd treats the
fd as synchronous.  While we work on getting rid of that, it's easiest to
have two fds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell c98734e0a4 connectd: don't ignore requests to connect if we're shutting down.
We used to shut down peers atomically, but now we flush the
connections there's a delay.  If we are asked to connect in that time,
we ignore it, as we are already connected, but that's wrong: we need
to remember that we were told to connect and reconnect.

This should solve a few weird test failures where "connect" would hang
indefinitely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 4584066a1e connectd: make sure we io_log msgs doing to gossipd.
test_gossip_no_empty_announcements relies on this!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 1ae3172409 connectd: flush queues before hanging up.
This is critical in the common case where peer sends an error and
hangs up: we almost never get to relay the error to the subd in time.

This also applies in the other direction: we need to flush the queue
to the peer when the subd closes.  Note we only free the actual peer
struct when lightningd reaps us with connectd_peer_disconnected().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 0841e4190b connectd: also do the shutdown()-close for final_msg sends.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell d29795a198 connectd: don't just close to peer, but use shutdown().
We would lose packets sometimes due to this previously, but it
doesn't happen over localhost so our tests didn't notice.  However,
now we have connectd being sole thing talking to peers, we can do
a more elegant shutdown, which should fix closing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: Always flush sockets to increase chance that final message get to peer (esp. error packets).
2022-01-20 15:24:06 +10:30
Rusty Russell d51fb5207a msg_queue: don't allow magic MSG_PASS_FD message for peers.
msg_queue was originally designed for inter-daemon comms, and so it has
a special mechanism to mark that we're trying to send an fd.  Unfortunately,
a peer could also send such a message, confusing us!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell a93c49ca65 connectd: implement @ correctly.
dev_blackhole_fd was a hack, and doesn't work well now we are async
(it worked for sync comms in per-peer daemons, but now we could sneak
through a read before we get to the next write).

So, make explicit flags and use them.  This is much easier now we
have all peer comms in one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell bb5beeddd7 connectd: drop support (unused) for @ during handshake.
We could implement it, but we don't have to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 26b9384fd0 various: minor cleanups from Christian's review.
More significant things have been folded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 39c93ee6e5 connectd: get addresses from lightningd, not gossipd.
It's weird to have connectd ask gossipd, when lightningd can just do it
and hand all the addresses together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 6d4c56e8b6 connectd: put more stuff into struct gossip_state.
We're the only ones who use it now, so put our fields inside it and
make it local.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 407a89a400 connectd: remove per_peer_state in favor of keeping gossip_fd directly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 6115ed02e8 subdaemons: don't stream gossip_store at all.
We now let gossipd do it.

This also means there's nothing left in 'struct per_peer_state' to
send across the wire (the fds are sent separately), so that gets
removed from wire messages too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 029d65cf2e connectd: serve gossip_store file for the peer.
We actually intercept the gossip_timestamp_filter, so the gossip_store
mechanism inside the per-peer daemon never kicks off for normal connections.

The gossipwith tool doesn't set OPT_GOSSIP_QUERIES, so it gets both, but
that only effects one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell e37a638c0c connectd: do nagle by packet type.
channeld can't do it any more: it's using local sockets.  Connectd
can do it, and simply does it by type.

Amazingly, on my machine the timing change *always* caused
test_channel_receivable() to fail, due to a latent race.

Includes feedback from @cdecker.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 7a514112ec connectd: do dev_disconnect logic.
As connectd handles more packets itself, or diverts them to/from gossipd,
it's the only place we can implement the dev_disconnect logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 9c0bb444b7 per_peer_state: remove struct crypto_state
Now that connectd does the crypto, no need to hand around crypto_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell a2b3d335bb connectd: do decryption for peers.
We temporarily hack to sync_crypto_write/sync_crypto_read functions to
not do any crypto, and do it all in connectd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell e683649004 connectd: maintain connection with peer, shuffle data.
Instead of passing the incoming socket to lightningd for the
subdaemon, create a new one and simply shuffle data between them,
keeping connectd in the loop.

For the moment, we don't decrypt at all, just shuffle.  This means our
buffer code is kind of a hack, but that goes away once we start
actually decrypting and understanding message boundaries.

This implementation is naive: it closes the socket to the local daemon
as soon as the peer closes the socket to us.  This is fixed in a
successive patch series (along with many other similar issues).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 7e7a63a20d connectd: keep timeout timer around so we can disable it.
connectd will be keeping the conn open, so it needs to free this
"conn_timeout" timer.  Pass it through, so we can do that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell 4ef2367c04 connectd: rename struct peer in peer_exchange_initmsg to early_peer.
We want to have a real (persistent) struct peer eventually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 888745be16 dev_disconnect: remove @ marker.
Once connectd is doing this, we can't close as soon as we send,
and in fact we can't do 'fail write' either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 5111f39d2a connectd: clean up lightningd connection handling.
They all returned the same thing anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Simon Vrouwe 605fda7214 common: cleanup unsused parameter in timer_expired() 2021-12-14 09:33:10 +10:30
Michael Schmoock c2d2cc1274 connectd: fix empty error message
1. Adds the missing DNS error massages so they can be handled by
   connect_control.
2. Prepends a 'All addresses failed' to code 401 message, so we
   always have at least some error message to the user.

Changelog-None
2021-12-08 13:52:24 +01:00
Rusty Russell d22fd59997 gossipd: remove gossip_msg.[ch]
This was a remnant from when we used to get routing from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell d9968bbc0c bitcoin: remove unused functions, or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 5284ee4dae connectd: don't advertize websocket support if we have no other (non-Tor) addrs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Rusty Russell ff556fefc6 connectd: fix websocket binding when we're doing both IPv4 and IPv6 on same port.
We would fail connectd when listening on the IPv6 version failed; instead we should
allow that.

Changelog-Experimental: experimental-websocket-port fixed to work with default addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00