Commit Graph

270 Commits

Author SHA1 Message Date
Rusty Russell 74521b3fb7 gossipd: don't delay the very first channel_update.
Lightning charge tests stopped working without a timeout, being unable
to find a route.  The 15 second delay doesn't matter in real life, but
in these scenarios it does.  This fixes it by making sure the channel
is usable immediately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:49:12 +02:00
conanoc b1900b18ab Fix DEVELOPER guard for ping
ping_req() should be outside of DEVELOPER guard now.
2018-08-15 06:48:55 +00:00
practicalswift 7969cc335e Allocate off ctx instead of tmpctx in encode_short_channel_ids_start(const tal_t *ctx) 2018-08-01 13:09:16 +09:30
practicalswift b5682a773b Remove dead stores 2018-07-31 12:45:02 +02:00
Rusty Russell 5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Rusty Russell 36730ddb6d gossipd: dev-suppress-gossip.
Useful for testing that we only get an update via the error message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 73b3782943 gossipd: send latest update in error message, even if delayed.
We delay internally to reduce broadcastig route flap, but errors are
a special case: we want to send the latest, otherwise we might send an
old (non-disabled) update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 3c66d5fa03 gossipd: add flag for locally disabling channel.
We used to just manually set ROUTING_FLAGS_DISABLED, but that means we
then suppressed the real channel_update because we thought it was a
duplicate!

So use a local flag: set it for the channel when the peer disconnects,
and clear it when channeld sends a local update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell d241bd762c connectd: don't use gossip_getnodes_entry.
gossip_getnodes_entry was used by gossipd for reporting nodes, and for
reporting peers.  But the local_features field is only available for peers,
and most other fields are only available from node_announcement.

Note that the connectd change actually means we get less information
about peers: gossipd used to do the node lookup for peers and include the
node_announcement information if it had it.

Since generate_wire.py can't create arrays-of-arrays, we add a 'struct
peer_features' to encapsulate the two feature arrays for each peer, and
for convenience we add it to lightningd/gossip_msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 7b2641ed0d gossipd: remove peer-related fields and wire messages.
This completes the removal of peer-related messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 0d442b5ff2 gossipd: move files into connectd.
These source files are only used for peer-related things, so move them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell dba7f9002f gossipd: provide connectd with address resolution.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 3d3d2ef9af gossipd: remove connectd functionality, enable connectd.
This patch guts gossipd of all peer-related functionality, and hands
all the peer-related requests to channeld instead.

gossipd now gets the final announcable addresses in its init msg, since
it doesn't handle socket binding any more.

lightningd now actually starts connectd, and activates it.  The init
messages for both gossipd and connectd still contain redundant fields
which need cleaning up.

There are shims to handle the fact that connectd's wire messages are
still (mostly) gossipd messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 92d66a5451 gossipd: take connectd fd on initialization.
connectd has a dedicated fd to gossipd, so it can ask for a new gossip_fd
for a peer.

gossipd has a standalone routine to create a remote peer (this will
eventually be the only way gossipd creates a new peer).

For now lightningd creates a socketpair but doesn't run connectd, so
gossipd never sees any requests on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell e1dfb1b178 gossipd: simplify per-peer features.
Store the two we care about as booleans.  Once connectd is complete we won't
even have the feature bitmaps for peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell f747ad8f73 common/daemon_conn: add daemon_conn_wake() helper.
We've been open-coding it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 16b8f1eb83 gossipd: actually use global features to create our own node_announcement.
It's currently empty, but I was surprised we still used "NULL".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell a52d522525 gossipd: handle ping messages for remote peers too.
This simplifies our ping handling: make gossipd always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 019ba86b91 gossipd: use optional fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Christian Decker 14c6310a4f gossip: Fix concurrent PR merge issue with structeq
PR #1618 in parallel with the migration to macro `structeq` created this.

Fixes #1674
2018-07-08 19:04:46 +02:00
Rusty Russell ed83bbe623 pytest: fix flaky race in test_gossip_query_channel_range.
We weren't waiting for gossipd to actually process the
dev_set_max_scids_encode_size message, so under Travis it sometimes
split the reply before processing that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell 57794b9285 gossipd: also delay locally-generated disables when peer vanishes.
Note that we mark both directions of the channel disabled immediately,
it's just the broadcast of the update which is delayed, just like the
ones generated when channeld tells us to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell f9b8237d50 gossipd: delay generation of local updates.
We disable the channel every time the peer disconnects; if it reconnects
we get two updates.

The simplest solution: delay all updates by 15 seconds.  Replace any
pending delayed update.  If update is redundant after 15 seconds,
discard.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell ef59a8f4aa gossipd: suppress redundant local updates which we would generate.
This doesn't do anything for us now, since we actually tend to produce
DISABLE/ENABLE update pairs.  But the infrastructure is useful for the
next patch.

We also add more details to the trace message in the core update code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 8e571ba688 listnodes: expose global features.
Since nobody sets these yet, it's a bit moot, but it will be great in
future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 9fa738a741 listpeers: expose peer features as 'local_features' and 'global_features'
For now, just the connected peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 7b735fbeee gossipd: fix json_listpeers printing node information.
json_listpeers returns an array of peers, and an array of nodes: the latter
is a subset of the former, and is used for printing alias/color information.

This changes it so there is a 1:1 correspondance between the peer information
and nodes, meaning no more O(n^2) search.

If there is no node_announce for a peer, we use a negative timestamp
(already used to indicate that the rest of the gossip_getnodes_entry
is not valid).

Other fixes:
1. Use get_node instead of iterating through the node map.
2. A node without addresses is perfectly valid: we have to use the timestamp
   to see if the alias/color are set.  Previously we wouldn't print that
   if it didn't also advertize an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell 82ff891202 Update to latest BOLT version.
And remove the FIXMEs now that the gossip_query extension is merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 17:37:03 +02:00
Rusty Russell f67182ff20 gossipd: order node_announcement addresses correctly, remove duplicate types.
Fixes: #1596
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell 284f0a04c9 gossipd: don't announce bound address if given with --bind-addr, even if public.
Only --addr implies announce-if-public: --bind-addr does not.

It's also possible to have --bind-addr to an automatic Tor address:
you'd have to dig the onion address out of the logs or getinfo to use
it, but it's possible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell 9d3ce87700 decode_short_ids: move to common.
We want to use it in devtools/decodemsg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Christian Decker 4a5cff8490 gossip: Try to detect broken ISP resolvers and discard broken replies
This is a best effort attempt to skip connection attempts if we detect a broken
ISP resolver. A broken ISP resolver is a resolver that will replace NXDOMAIN
replies with a dummy response. This is best effort in that it'll only detect a
single fixed dummy reply, it'll check only on startup, and will not detect if we
switched networks. It should be good enough for most cases, and in the worst
case it will result in a connection attempt that does not complete.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Glenn Willen <@gwillen>
2018-06-21 11:21:16 +02:00
Christian Decker 91c2416657 gossip: Do not use DNS if we were told not to
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 11:21:16 +02:00
Christian Decker ceef61dbbd gossip: Pass use_dns option down to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 11:21:16 +02:00
Rusty Russell 833e8387aa gossipd: fix up BOLT references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell f5ebf8e231 gossipd: send correct channel_update in response to query_short_channel_ids
Cut & paste means we sometimes sent NULL:

```
2018-06-15T00:13:51.908Z lightningd(23653): lightning_closingd-03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f chan #436: Gossipd gave us bad send_gossip message 0bc80000
```

Fixes: #1581
Reported-by: @Xian001
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 15:39:30 +02:00
Rusty Russell 60b3f0e376 gossipd: remove oververbose logging when we uncompress short_channel_id array
Reported-by: Xian001 (#1581)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 15:39:30 +02:00
Rusty Russell 9d721ecb99 gossipd: add assertions to try to catch mysterious crash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell 5c19c55841 gossipd: fix take leak when peer is dying.
In this case, local and remote are *both* NULL; so if someone tries to
send a packet with take(), we need to free it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell a7e6cdb418 gossipd: peer->local->peer_out queue should have lifetime of peer->local.
The current code attaches it to peer, which is a slight leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell e098578731 gossipd: fix leak when we fail to dup fds.
In this case, peer would stay around, but conn would be freed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell f6ff89e596 gossipd: fix use-after-free when we fail to make connection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Christian Decker 4279e5cdbd gossip: Fix "already reaching" issue
I think this is what is causing #1536: getting disconnected causes gossipd to
attempt to reach the peer again, unconditionally setting the flag to tell the
master. At the same time the master also issues a reaching command (which is
allowed since it is its first), but then it clashes on the already set
flag. Setting this flag only when the master actually needs to be told should
fix this.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-15 01:06:42 +00:00
Christian Decker e6ab594904 gossip: Have gossip_store annotate gossip messages
This makes the exposed interface much smaller, cleaner and will allow us to just
replay gossip messages from the broadcast.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Rusty Russell a38c619486 gossipd: keep index of node and channel announcements.
This lets detect if a node announce preceeds a channel announce once we
delete the node announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell 1bb7713274 gossipd: minor cleanups.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 035d6067e4 Rename consider_own_node_announce to maybe_send_own_node_announce.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell f52245d442 gossipd: support and use zlib encoding in short_channel_id encoding.
We still use uncompressed if zlib turns out to be larger.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 9e51e196c1 gossipd: dev-set-max-scids-encode-size to artificially force "full" replies.
We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00