Commit Graph

196 Commits

Author SHA1 Message Date
Rusty Russell ab735dcbe6 gossipd: wire up memleak detection.
For simplicity we dump leaks to logs, and just return a bool to master.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 29b672b117 gossipd: hear no wumbo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell 9620393109 gossipd: store chainparams internally.
We keep a chain_hash in struct daemon, becayse otherwise we end up with
`&peer->daemon->rstate->chainparams->genesis_blockhash` which is a bit
ridiculous.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell 5c60d7ffb2 gossipd: split wire types into msgs from lightningd and msgs from per-peer daemons
This avoids some very ugly switch() statements which mixed the two,
but we also take the chance to rename 'towire_gossip_' to
'towire_gossipd_' for those inter-daemon messages; they're messages to
gossipd, not gossip messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 363564301f gossipd: be more rigorous in handling peer messages vs. daemon requests.
Messages from a peer may be invalid in many ways: we send an error
packet in that case.  Rather than internally calling peer_error,
however, we make it explicit by having the handle_ functions return
NULL or an error packet.

Messages from the daemon itself should not be invalid: we log an error
and close the fd to them if it is.  Previously we logged an error but
didn't kill them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell c236361efd wireaddr: update bolt version, remove 'padding' from addresses.
Nobody used this, so it was removed from the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 23:51:05 +00:00
lisa neigut 762c795c9b gossip: reject channel_update with invalid `htlc_max_msat`
If the channel update signals an invalid `htlc_maximum_msat` value,
we ignore the update.
2018-10-09 23:22:52 +00:00
lisa neigut 1b6bd3fded wire: add test for parsing optional version of channel_update 2018-10-09 23:22:52 +00:00
lisa neigut b9331e5ac8 gossipd: parse and respect optional `htlc_maximum_msat`
If another channel has set the optional `htlc_maximum_msat` field,
we should correctly parse that field and respect it when drawing up
routes for payments.
2018-10-09 23:22:52 +00:00
Rusty Russell df27fc55af More renaming of gfeatures to globalfeatures.
Use the BOLT #1 naming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell afc92dd757 gossipd: use array[32] not pointer for alias.
And use ARRAY_SIZE() everywhere which will break compile if it's not a
literal array, plus assertions that it's the same length.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell e450c6bbdb gossipd: remove time-delayed local channel_update, produce DISABLE on-demand.
We have a lot of infrastructure to delay local channel_updates to
avoid spamming on each peer reconnect; we had to keep tracking of
pending ones though, in case we needed the very latest for sending an
error when failing an HTLC.

Instead, it's far simpler to set the local_disabled flag on a channel
when we disconnect, but only send a disabling channel_update if we
actually fail an HTLC.

Note: handle_channel_update() TAKES update (due to tal_arr_dup), but we
didn't use that before.  Now we do, add annotation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell 16e16a725e gossipd: apply private updates to announce channel.
We trade channel_update before channel_announce makes the channel
public, and currently forget them when we finally get the
channel_announce.  We should instead apply them, and not rely on
retransmission (which we remove in the next patch!).

This earlier channel_update means test_gossip_jsonrpc triggers too
early, so have that wait for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell 66105e83ea gossipd: simplify "broadcast channel_announcement now we have channel_update" logic
It's simpler and more robust to just check that it's not yet announced
(the broadcast index will be 0).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell 8455b12781 Revert "gossipd: handle premature node_announcements in the store."
This reverts commit e2f426903d.

With the new store version, this can't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
lisa neigut b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
Rusty Russell 97c7ba2f80 gossipd: fix reordering of node_announcements in presence of a unannounced channel.
If we receive a channel_announce but not a channel_update, we store the announce
but don't put it in the broadcast map.

When we delete a channel, we check if the node_announcement broadcast
now preceeds all channel_announcements, and if so, we move it to the
end of the map.  However, with a channel_announcement at index '0',
this test fails.

This is at least one potential cause of the node map getting out of order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:36:05 +02:00
Rusty Russell e2f426903d gossipd: handle premature node_announcements in the store.
These happen after we compact the store; every log I've seen of a
restart on a real node has a message about truncating the store,
because node_announcements predate channel_announcements.

I extracted one such case from testnet, and reduced it to test here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:36:05 +02:00
Rusty Russell 1c81486b48 routing: fix falsely flagged leak.
pending goes away on a timer, sure, but might as well use tmpctx here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 83eadb3548 gossipd: fix SUPERVERBOSE usage, enhance, when turned on.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Christian Decker 6627da5eb5 routing: Do not consider risk when capping transfers
Reported-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2018-08-06 22:46:02 +02:00
Christian Decker 84905eac2b routing: Make the capacity a parameter to new_chan
As pointed out by @rustyrussell the capacity is now always defined, so we can
fold that into the construction of the channel itself.

Reported-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2018-08-06 22:46:02 +02:00
Christian Decker 8201764117 routing: Skip channels that require larger HTLCs than we are routing
The `htlc_minimum_msat` parameter was ignored so far, and we'd be attempting to
pay and hitting a brick wall by doing so. This patch just skips channels that
are not eligible anyway.
2018-08-06 22:46:02 +02:00
Christian Decker 14000a22bc routing: Skip channels that don't have sufficient capacity
We know the total channel capacity after checking for its existence on-chain, so
we can actually make use of that information to discard channels that don't have
a sufficient capacity anyway, reducing the number of failed attempts.
2018-08-06 22:46:02 +02:00
Christian Decker 8a34933c1a gossip: Annotate locally added channels with their capacity
We were adding channels without their capacity, and eventually annotated them
when we exchanged `channel_update`s. This worked as long as we weren't
considering the channel capacity, but would result in local-only channels to be
unusable once we start checking.
2018-08-06 22:46:02 +02:00
Rusty Russell 584ee26200 gossipd: fix thinko in node_announcement address parsing which made us miss final address
'cursor < ser + max' isn't valid because we reduce 'max' as we go!  Effectively
we'll stop once we're past halfway, which can only happen with ipv6 + a torv2
address.

Ths fix is one-line, but we rename 'max' to 'len' which makes its purpose
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-06 19:33:46 +02:00
Rusty Russell 5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Rusty Russell 3c66d5fa03 gossipd: add flag for locally disabling channel.
We used to just manually set ROUTING_FLAGS_DISABLED, but that means we
then suppressed the real channel_update because we thought it was a
duplicate!

So use a local flag: set it for the channel when the peer disconnects,
and clear it when channeld sends a local update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 7b2641ed0d gossipd: remove peer-related fields and wire messages.
This completes the removal of peer-related messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell ef59a8f4aa gossipd: suppress redundant local updates which we would generate.
This doesn't do anything for us now, since we actually tend to produce
DISABLE/ENABLE update pairs.  But the infrastructure is useful for the
next patch.

We also add more details to the trace message in the core update code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 8e571ba688 listnodes: expose global features.
Since nobody sets these yet, it's a bit moot, but it will be great in
future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell 4a1ca0fb99 gossipd: don't use raw secp256k1_pubkey in routing.
We wrap it in 'struct pubkey' for typesafety and consistency, and the
next patch takes advantage of that when we move to pubkey_eq.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell 833e8387aa gossipd: fix up BOLT references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Christian Decker 985af483cf gossip: Wrap insert_broadcast and gossip_store_add in persistent_broadcast
They should sync up nicely otherwise we may be overestimating the stale rate.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker e6ab594904 gossip: Have gossip_store annotate gossip messages
This makes the exposed interface much smaller, cleaner and will allow us to just
replay gossip messages from the broadcast.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 0546ca446d gossip: Pass routing_state to the gossip_store
We'll need it later to annotate the raw gossip messages, e.g., the capacity of a
channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 552ddb8dfd gossip: Pass broadcast_state to gossip_store
We'll be sourcing messages from this `broadcast_state` when rewriting the
`gossip_store`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Rusty Russell da55d3c0ff gossipd: handle node_announcement when channel_announcement removed.
Two cases:
1. Node no longer has any public channels: remove node_announcement.
2. Node's node_announcement now preceeds all the channel_announcements:
   move node_announcement to the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell a38c619486 gossipd: keep index of node and channel announcements.
This lets detect if a node announce preceeds a channel announce once we
delete the node announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell 5ec454c7b2 gossipd: don't queue node_announce unless we've queued channel_announce.
We *accept* a node_announce if we have a channel_announce, but we
can't queue it until we queue the channel_announce, which we only do
once we have recieved a channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 97bb6c5a28 gossipd: ensure incoming timestamps are reasonable.
This is kind of orthogonal to the other changes, but makes sense: if we
would instantly or never prune the message, don't accept it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 7a32637b5f gossipd: add timestamp to each broadcast message.
This lets us filter by timestamp.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 803e4f8895 gossipd: announce nodes after channel announcement.
In general, we need to only publish node announcements after
publishing channel announcements, though we can accept node
announcements as soon as we see channel announcements.  So we keep a
flag for those node_announcement which haven't been broadcast yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell c2cc3823db gossipd: announce own node only after channel announcement actually broadcast.
handle_pending_cannouncement might not actually add the announcment,
as it could be waiting for a channel_update.  We need to wait for
the actual announcement before considering announcing our node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell c2189229ca gossipd: only broadcast channel_announcement once we have a channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 2431742285 gossipd: don't publish private updates after channel_announce.
We generate new ones anyway; removing this code changes fixes coming
up which now only need to change one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell 88053bd1ca gossipd: remove too-loose timestamp workaround.
Now timestamps always increment, we don't have to allow them to do the
wrong thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-21 09:17:57 -07:00
Rusty Russell adbe02c6be gossip: temporarily allow replacement of updates with same timestamp.
We erroneously create updates with the same timestamps when tests run
quickly, and the second one is ignored.

We've already noted that this should be fixed: gossipd should generate
all the updates, as it already has to do the case where channeld
crashed, for example.  But that's a bigger change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00