Commit Graph

376 Commits

Author SHA1 Message Date
Christian Decker 4e7fc99ae1 gossip: Duplicate removes can result in null pointers in broadcast
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-08 20:00:27 +02:00
Rusty Russell 5d6a9f3fb0 gossipd: check consistency.
This is a hack to check that our gossip state is consistent on every
insert and delete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell da55d3c0ff gossipd: handle node_announcement when channel_announcement removed.
Two cases:
1. Node no longer has any public channels: remove node_announcement.
2. Node's node_announcement now preceeds all the channel_announcements:
   move node_announcement to the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell def18a7bc1 gossipd: implement broadcast_del to delete a specific index.
Required if we want to reorder node_announcement broadcasts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell a38c619486 gossipd: keep index of node and channel announcements.
This lets detect if a node announce preceeds a channel announce once we
delete the node announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell 1bb7713274 gossipd: minor cleanups.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 035d6067e4 Rename consider_own_node_announce to maybe_send_own_node_announce.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 5ec454c7b2 gossipd: don't queue node_announce unless we've queued channel_announce.
We *accept* a node_announce if we have a channel_announce, but we
can't queue it until we queue the channel_announce, which we only do
once we have recieved a channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell f52245d442 gossipd: support and use zlib encoding in short_channel_id encoding.
We still use uncompressed if zlib turns out to be larger.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 9e51e196c1 gossipd: dev-set-max-scids-encode-size to artificially force "full" replies.
We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 118f099dd8 gossip: dev-query-channel-range to test query_channel_range.
We keep a crappy bitmap, and finish when their replies cover
everything we asked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 0dda5d4e1c gossipd: handle query_channel_range
We send them all the short_channel_ids we have in a given range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell c34b49c356 gossipd: add dev-send-timestamp-filter command for testing timestamp filtering.
Since we currently only (ab)use it to send everything, we need a way to
generate boutique queries for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell db6a6442cb gossipd: single-thread the gossip timer.
We have a function called 'wake_pkt_out' which is really 'start
gossiping', so rename it to 'wake_gossip_out'.

In addition, it's fired both on a timer, and in response to our first
gossip_timestamp_filter, which leads to very confusing (though,
technically, not incorrect) behavior.

Keep a single timer at all times, which now doubles as the flag to
indicating we're syncing right now.  Set it once we're done syncing
gossip.

Technically this means we got from once-every-60-seconds to
quiet-for-60-seconds-between-gossip, but that's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 531c82b6ad gossipd: handle gossip_timestamp_filter message.
And initialize filter (to "never") when we negotiated LOCAL_GOSSIP_QUERIES,
and send initial filter message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 97bb6c5a28 gossipd: ensure incoming timestamps are reasonable.
This is kind of orthogonal to the other changes, but makes sense: if we
would instantly or never prune the message, don't accept it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 7a32637b5f gossipd: add timestamp to each broadcast message.
This lets us filter by timestamp.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 4d8b29089b gossipd: wire up infrastructure to generate query_short_channel_ids msg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 7ee5da858c gossipd: handle query_short_channel_ids message.
This doesn't handle zlib yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 32c39c2979 gossipd: send node announcements after short_channel_id replies.
We use the same system as for gossip: we trickle out replies when we're
otherwise idle.

As we trickle out replies to query_short_channel_ids, we remember the
pubkeys of nodes we mention.  At the end, we sort and uniquify, and
then send any node_announcements we have for those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 5864415d31 gossipd: infrastructure to handle short_channel_id replies.
We use the same system as for gossip: we trickle out replies when we're
otherwise idle.

This is minimal infrastructure: we don't actually process the
query_short_channel_ids message yet, nor do we append node
announcements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 6c6da45f53 wire: Update to lastest BOLT draft.
This includes the gossip query messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 803e4f8895 gossipd: announce nodes after channel announcement.
In general, we need to only publish node announcements after
publishing channel announcements, though we can accept node
announcements as soon as we see channel announcements.  So we keep a
flag for those node_announcement which haven't been broadcast yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell c2cc3823db gossipd: announce own node only after channel announcement actually broadcast.
handle_pending_cannouncement might not actually add the announcment,
as it could be waiting for a channel_update.  We need to wait for
the actual announcement before considering announcing our node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell c2189229ca gossipd: only broadcast channel_announcement once we have a channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 2431742285 gossipd: don't publish private updates after channel_announce.
We generate new ones anyway; removing this code changes fixes coming
up which now only need to change one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker c550fd1752 gossip: Clean up the code to disable a local channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker c17848a3f3 gossip: Disable local channels after loading the gossip_store
We don't have any connection yet, so how could they be active? Disable both
sides to avoid trying to route through them or telling others to use them as
`contact_points` in invoices.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker f2dc406172 moveonly: Hoist gossip_disable_channel higher up
We'll need it in the next commit

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker ba31dd2d9d gossip: Avoid sending duplicate disable messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker 8e278044e3 gossip: Disable channels when we lose the connection to the peer
We're telling gossipd about disconnections anyway, so let's just use that signal
to disable both sides of the channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker 3e5b798c60 gossip: Fix disable flags in handle_disable_channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker 9982e24a1c gossip: Add local_channel_close message to disable channels upon close
This was failing some of our integration tests, i.e., the ones closing a channel
and not waiting for sigexchange. The remote node would often not be quick enough
to send us its disabling channel_update, and hence we'd still remember the
incoming direction. That could then be sent out as part of an invoice, and fail
subsequently. So just set both directions to be disabled and let the onchain
spend clean up once it happens.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker 402125a70e gossip: Add CRC32 checksum to the gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell @rustyrussell
2018-05-29 12:16:00 +00:00
Rusty Russell 88053bd1ca gossipd: remove too-loose timestamp workaround.
Now timestamps always increment, we don't have to allow them to do the
wrong thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-21 09:17:57 -07:00
Rusty Russell 6454d7af84 gossip: cleanup keepalive updates to use the same create_channel_update() code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-21 09:17:57 -07:00
Rusty Russell fca5a9ef30 channeld: tell gossipd to generate channel_updates.
This resolves the problem where both channeld and gossipd can generate
updates, and they can have the same timestamp.  gossipd is always able
to generate them, so can ensure timestamp moves forward.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-21 09:17:57 -07:00
Rusty Russell adbe02c6be gossip: temporarily allow replacement of updates with same timestamp.
We erroneously create updates with the same timestamps when tests run
quickly, and the second one is ignored.

We've already noted that this should be fixed: gossipd should generate
all the updates, as it already has to do the case where channeld
crashed, for example.  But that's a bigger change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell c546b1bbb6 gossipd: specify origin of updates in errors.
@cdecker points out that in test_forward, where we manually create a route,
we get an error back which contains an update for an unknown channel.

We should still note this, but it's not an error for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 8ee60e2d8e testing: make sure we don't see gossip in bad order.
This is something which generally shouldn't happen, but we didn't
notice it previously.

We ignore this warning in the case where a channel was deleted: this
happens because one side can send an update while the other notices
that the channel is closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 177a1fc88e gossipd: handle local channel creation separately from update.
Note: this will break the gossip_store if they have current channels,
but it will fail to parse and be discarded.

Have local_add_channel do just that: the update is logically separate
and can be sent separately.

This removes the ugly 'bool add_to_store' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 540c68d7ca gossipd/gossip_constants.h: Single place for BOLT constants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell b965ef7d1d routing: make sure we fail if we can't unmarshal announcements.
This is how we notice if the gossip store is corrupt!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
practicalswift fab3b214b4 Avoid static analyzer warning about integer wraparound 2018-05-15 05:26:29 +00:00
Rusty Russell 1125682ceb wireaddr: new type, ADDR_INTERNAL_FORPROXY, use it if we can't/wont resolve.
Tor wasn't actually working for me to connect to anything, but it worked
for 'ssh -D' testing.

Note that the resulting 'netaddr' is a bit weird, but I guess it's honest.

    $ ./cli/lightning-cli connect 021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b
    {
      "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b"
    }
    $ ./cli/lightning-cli listpeers
    {
      "peers": [
        {
          "state": "GOSSIPING", 
          "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b", 
          "netaddr": [
            "ln1qg0je0lugpzu5ttsv78vlrkhteyg9yy8fjw68qr57mfhsfyrxurzkq522ah.lseed.bitcoinstats.com:9735"
          ], 
          "connected": true, 
          "owner": "lightning_gossipd"
        }
      ]
    }

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 2a0acd3492 tor: log proxy communications using status_io.
Good for debugging (you have to send SIGUSR1 to lightning_gossipd to turn
it on though, and --log-level=io on the lightningd cmdline to have it
output IO messages by default).

I also noticed that io_tor_connect_after_req_host() does a useless
test on reach->buffer[0] after it's *written*: remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 570283bc76 gossipd: don't use fake addrhint for non-addrhint resolutions.
Use a wireaddr_internal directly (which is what we want).

Also, don't hardcode 9735, use DEFAULT_PORT internally in
seed_resolve_addr().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell de063edb54 gossip: extract function to derive seedname.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 0d23f4fb4a gossipd: hand io_tor_connect the host as a string.
Previously it converted the wireaddr to a string internally: to support
unresolved names we need that done externally.

We actually tell the SOCKS5 proxy to do a domain lookup already, even
though we give use IP/IPv6 address, so this change is sufficient to
support connect-by-name.

Note replacement of assert() with an explicit case statement, which
has the benefit that the compiler complains when we add new
ADDR_INTERNAL types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell a1dc4eef56 wireaddr: tell caller that we failed due to wanting DNS lookup, don't try.
This is useful for the next patch, where we want to hand the unresolved
name through to the proxy.

This also addresses @Saibato's worry that we still called getaddrinfo()
(with the AI_NUMERICHOST option) even if we didn't want a lookup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00