Commit Graph

3501 Commits

Author SHA1 Message Date
Christian Decker 7fc06d9b3e wallet: Add outpointfilter to wallet so we can pass it all outputs
Will be used later to filter out outputs we are interested in, and
trigger db updates with them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 5fd19483a9 txfilter: Add an outpoint filter
This can be used both for our own outputs as well as the utxos we are
tracking.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 247aa13a58 db: Add index covering the output heights
Avoids performing a table scan, now deletes on blocks are a lot
faster.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker c9985805cd pytest: Add blockchain tracking test
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 4e2c1dd038 jsonrpc: Add confirmation status to `listfunds` output
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 227dc36146 utxo: Add blockheight and spendheight to outputs to track state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 7f00230657 db: Add confirmation_height and spend_height columns to outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 1aa21498c3 topology: Track blockchain changes in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 9440b23852 wallet: Add primitive to roll back to a specific height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 504202973f wallet: Add primitives to store blockchain internally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker fea03dbca9 db: Add a table to track blockheaders
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
ZmnSCPxj 8e8d7c2aba pay/sendpay: Use spec names for rhash and r. 2018-03-05 20:21:37 +00:00
Rusty Russell dace9bfdcf gossipd: the great renaming.
We already have 'struct node', so rename 'struct routing_channel' to
'struct chan', and 'struct node_connection' to 'struct half_chan'.

Other minor changes:
1. rstate->channels -> rstate->chanmap.
2. 'connections' -> 'half'.
3. connection_to -> half_chan_to
4. connection_from -> half_chan_from

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell 61bcb054e0 routing: remove redundant fields from struct node_connection.
The containing `struct routing_channel` contains src and dst, so
remove them.  However, the channel_update msgidx does belong int
`struct node_connection` along with the channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell 172af04247 gossip: remove short_channel_id from struct node_connection.
It's in the containing routing_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell 21b2f37dfb routing: return entire routing_channel array from find_route.
Returning the separate first routing_channel was a weird API: just
return the entire array.  Sure, we have to treat the first node a bit
differently (because we don't charge ourselves fees), but it's still
simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell 56349ab008 routing: work with struct routing_channel not struct node_connection.
To remove the redundant fields in `struct node_connection` (ie. 'src'
and 'dst' pointers) we need to deal with `struct routing_channel`.
This means we get a series of channels, from which the direction is
implied, so it's a bit more complex to decode.  We add a helper
`other_node` to help with this, and since we're the only user of
`connection_to` we change that function to return the index.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
John Barboza 31b9b6bc1c clear onchaind subd in channel on error
When the subdaemon closes unexpectedly, clear the reference and
update the channel billboard with the description
2018-03-04 16:29:44 +01:00
Jan Sarenik 9b3e8f0bd3 bitcoin.bib: Add Satoshi paper URL and date
Date according to Wikipedia which cites other source.
2018-03-04 03:22:32 +01:00
Jan Sarenik 424ca8cf59 doc: Modify README.md code sections 2018-03-04 03:21:59 +01:00
Jan Sarenik fd338c0689 doc: More plain-text improvements to HACKING.md and INSTALL.md
This time the rendered output is slightly different, but mostly
because long preformatted lines are wrapped and contain an extra
continuation backslash now.
2018-03-04 03:21:59 +01:00
Jan Sarenik 960548f311 doc: Reflow HACKING.md and INSTALL.md without textual change
See previous commit 9504a77b for a script to prove there is no
change in the rendered file, just readability plain-text
improvements.
2018-03-04 03:21:59 +01:00
ZmnSCPxj c03f584e19 test_lightningd.py: Adapt to new help output. 2018-03-03 19:40:22 +01:00
Mark Beckwith 38e94e883f Improved display of lightning-cli help for humans
The display format is now:

command1
    description1

command2
    description2

    .
    .
    .
2018-03-03 19:40:22 +01:00
Rusty Russell 30faa6485a subdaemons: copy backtraces to stderr.
We didn't get the entire thing in the parent when gossipd crashed: stderr
is suitable for these I think.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell fd9c0c8543 routing: move struct node_connection into struct routing_channel.
No need to have pointers since they're always there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell be14b52423 routing: connections are now never null; simplify.
Failure and pruning were the two places where a node_connection could
be freed; now they both deal with entire channels, we can remove the
NULL checks, and the destructor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell d7b5882f61 routing: don't free a single nc on prune, only entire channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell d0f68ffb15 routing: free entire channel, not just node_connection on PERM error.
And do it by reparenting onto the tmpctx, restoring some simpliciy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 00194b6130 handle_disable_channel: don't use get_connection_by_scid.
This removes the final user, so we remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 74ee448bda routing: expose setter for struct node_connection fields.
And use it in gossip's handle_local_add_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 1e4959af1f gossipd: fix case where we get an update on a pruned channel.
We discarded it; we should populate it.  The comment is wrong, since
local_add_channel() doesn't add public channels, and we test that above.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 961321a6d8 gossipd: move route pruning to routing.c.
And reimplement it to walk the channels map rather than the nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 33726b0a08 gossip: instead of refresh interval, have routing know prune_timeout.
This is twice the 'update_channel_interval' we get handed.

We delete the non-existent channel_add_connection and delete_connection
declarations from the header too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell b7bf414ac4 gossipd: prune announced-but-not-updated channels eventually.
We currently give them a free pass.  The simplest fix is to give them
an old timestamp on initialization.

We still skip unannounced channels, on the assumption that they're
ours.  And we set the last_update_timestamp to -1 when we convert to
gossip_getchannels_entry to indicate no update.

This breaks the DEVELOPER=1 pruning test, since we hardcode the 1
week timeout.  That's fixed in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell b7ec2c8c9c node_connection: move channel_announcement field into struct routing_channel.
We don't actually *use it*, mind you: the copy in the broadcast message
is the one we use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 942d04ba87 gossipd: simplify channel_announce handling.
We make new_routing_channel() populate both connections
(active=false), so local_add_channel becomes simpler.  We also
suppress listchannels output of active=false unannounced channels, to
avoid breaking tests (also, these are unusable, so it makes sense to
omit them)

It also seems the logic in add_channel_direction is legacy: a
channel_announce cannot replace the scid (that would be a different
channel), we don't allow duplicate announcements, and the announcement
is never NULL.

And since we disallow repeated channel_announce already, I believe
'forward' is always true, greatly simplifying the logic in
handle_pending_cannouncement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 6dec2ed1c1 gossipd: hoist new_node_connection above new_routing_channel.
This is moveonly, wanted for next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell fc410eff60 resolve_channel_req: use get_channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell a79879637a gossipd: local_add_channel can just use get_channel()
Also upgrade the status messages in the weird cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 6bc634badf gossip: handle_get_update can just use get_channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell a9b1d73148 getchannels_req: iterate over channel map rather than nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 5f5d0b3e25 gossip_local_add_channel: remove unused flags field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 9b900138d0 gossip: put 'routing_channel' in charge of 'node_connection'.
This makes 'routing_channel' the primary object in the system; it can have
one or two 'node_connection's attached, and points to two nodes.

The nodes are freed when no more routing_channel refer to them.  The
routing_channel are freed when they contain no more 'node_connection'.
This fixes #1072 which I surmise was caused by a dangling
routing_channel after pruning.

Each node contains a single array of 'routing_channel's, not one for
each direction.  The 'routing_channel' itself orders nodes in key
order (conveniently the index is equal to the direction flag we use),
and 'node_connection' with source in the same order.

There are helpers to assist with common questions like "which
'node_connection' leads out of this node?".

There are now two ways to find a channel:
1. Direct scid lookup via rstate->channels map.
2. Node key lookup, followed by channel traversal.

Several FIXMEs are inserted for where we can now do things more optimally.

Fixes: #1072
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell f8426600a6 gossipd: don't create a routing_channel while we're waiting.
We're going to make it a first-class citizen, and pending routing_channel
are not real ones (in particular, we don't want to create pending nodes).

We had a linked list called rstate->pending_cannouncement which we didn't
actually use, so put that back for now and add a FIXME to use a faster
data structure.

We need to check that list now in handle_channel_update, but we never
have a real routing_channel and a pending, unless the routing_channel
isn't public.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 2ec38ec687 gossip: factor out appending code in getchannels_req().
This moveonly makes a coming cleanup easier.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 629c3db421 ccan: update to get tal_free fix.
This allows us to make heirarchy loops, such as reference counting to
free our own parent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
ZmnSCPxj 1f6008689d payalgo: Throttle pay command if failure is due to blockheight disagreement. 2018-03-03 18:19:53 +01:00
Douglas Schilling Landgraf ef34efa93f pylightning: label is required for waitinvoice()
This patch make sure label don't have default value.
2018-03-03 18:12:26 +01:00
John Barboza 35ce13179c Add tests for peer information containing alias 2018-03-02 02:25:30 +00:00