Commit Graph

3596 Commits

Author SHA1 Message Date
Christian Decker 23984ecde4 chaintopology: Use the DB to locate transactions and rebroadcast txs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 86b6402e5c chaintopology: Refactor get_tx_depth to use the DB backed tx store
We are slowly hollowing out the in-memory blockchain representation to make
restarts easier.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker aa696370af txwatch: Switch to passing only txid into the depth callbacks
All of the callback functions were only using the tx to generate the txid again,
so we just pass that in directly and save passing the tx itself.

This is a simplification to move to the DB backed depth callbacks. It'd be
rather wasteful to read the rawtx and deserialize just to serialize right away
again to find the txid, when we already searched the DB for exactly that txid.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 85fbab2fab wallet: Add function to retrieve a watched transaction's blockheight
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 50600ae241 wallet: Store transactions we are watching, broadcast or own
This will later allow us to determine the transaction confirmation count, and
recover transactions for rebroadcasts.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 633ffeec6f wallet: Add methods to add transactions to the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 0f89653ce8 db: Add table for transactions we are interested in
Currently these are either transactions we sent ourselves or transactions that
we are watching because they are part of a channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Rusty Russell b0c2e3cd5c gossipd: use a separate CSV file for the gossip_store types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell 57b38cac71 gossip_store: empty, don't truncate, on error.
Christian points out that we don't get spend notifications for old
channels if we truncate the store.  We'd need more work to do this,
either validating the channels are still unspent, or replaying old
blocks from the truncation point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell d5767fb3bb gossipd: print stats even if we truncate store.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell 2b8293c9f6 gossipd: don't use pwrite, better error messaging on init.
Since we open with O_APPEND, any write() will append as we want it to.

But we want to distinguish a new store creation from a truncation due
to bad version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell 7d0a76c533 goossipd: make store load truncate on errors.
We don't need pread, we just need read, and we can loop internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell 3e1b584e73 gossipd: always add message internally before store.
If something goes (fatally) wrong, we won't add it to the store.

This reveals a latent bug in routing_add_channel_announcement() and
friend which did a take() on msg, which it doesn't own.  TAKES means
that it will take ownership IF the caller requests, not an unconditional
ownership transfer (which is an antipattern).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell abbbfac8e2 gossipd: return bool from message announce routines.
Now we can tell if they fail, so we can respond appropriately if we're
loading from the store.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell e8a052eb6d routing: add more debugging to announcement replaced fail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell 30c1ab424f gossipd: reorder handle_node_announcement
I found the logic a bit confusing, so this reworks to bunch the
"no node" cases together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell 4aca909acb routing: don't store node_announce unannounced nodes.
We enter nodes in the map when we create channels, but those channels
could be local and unannounced.  This triggered a failure in
test_gossip_persistence since the store truncated when it saw the
first thing was a node_announce.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
ZmnSCPxj 957513666c closing_control: Fix loop limit in better_closing_fee. 2018-04-10 20:45:16 +00:00
ZmnSCPxj 86290b54d4 routing: Use 64-bit msatoshi for messages to and from routing.
Internally both payment and routing use 64-bit, but the interface
between them used 32-bit.
Since both components already support 64-bit we should use that.
2018-04-09 20:45:26 +02:00
Christian Decker 0ba687732f bitcoind: Do not copy the newline character when asking for a block
In the short_channel_id check we were copying the entire result into the next
bitcoin-cli call, including the newline character.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-By: @gdassori
2018-04-09 00:21:20 +00:00
Christian Decker ba3ceb2abf wallet: Lowerbound the rescan by going at most back to LNs origin
Repeated crashes could result in the `last_processed_block` variable being
pushed further and further into the past (in some cases going as far back as
scanning blocks from 2012...). This is a stop-gap solution that just lower
bounds the value to what is the first possible block we might be interested in
LN, until we have the 0-rescan fix I'm working on.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker a121b7dbc3 gossip: Make gossipd less noisy when receiving requests
This is very noisy when syncing with the blockchain

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker 2de7f622cb gossip: Add an explicit debug message when handing back a peer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker 1757732236 pytest: Fix flaky test_blockchaintrack test
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker ffbe56847e db: Fixing some clang warnings
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker 36c335fb09 contrib: Updated the builder images to include shellcheck
As requested in #1319

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker 723b64036f jsonrpc: Pretty-print the json results
Just a small cleanup of the indentation code, so we don't have to reformat all
the issue reports to become readable. This is much closer to what `jq` or
`json_pp` spit out and doesn't have those infinitely long lines.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-08 08:26:00 +00:00
ZmnSCPxj 8a9fef2127 peer_control: Indicate how much money in the channel is spendable, given the reserve. 2018-04-08 08:23:34 +00:00
Rusty Russell a1cf7897c0 shellcheck: restore the check.
Accidentally disabled in 9c3691340f.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-07 23:38:03 +02:00
Christian Decker a41ab650e5 master: Move pid-file creation after the daemonization
Creating the pid-file before daemonizing results in the pid-file containing the
pid of the process that started the daemon, but is now dead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-By: Torkel Rogstad @torkelrogstad
2018-04-07 19:49:40 +02:00
ZmnSCPxj 249b1ac6df .gitignore: Ignore onchaind test binary. 2018-04-06 19:42:59 +02:00
Rusty Russell 09c4203767 bolt11: allow multiple fallback addresses.
We can have more than one; eg we might offer both bech32 and a p2sh
address, and in future we might offer v1 segwit, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-06 14:26:53 +02:00
Rusty Russell 5b7fcab766 tools: fix shellcheck errors.
Not sure how this got through Travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-06 14:26:53 +02:00
Rusty Russell 9c3691340f ccan: update to more recent version.
In particular, this gets some MacOS fixes from #1327.
It also includes a major intmap update which fixes corner cases in traversals,
and requires ccan/bitops.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-06 13:34:27 +02:00
Christian Decker 9ab28d1659 topology: Two off-by-one error when catching up with the blockchain
There are two very hard problems in software engineering:

 1. Off-by-one errors

In this case we were rolling back further than needed and we were starting the
catchup one block further than expected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:55:41 +00:00
Christian Decker 4da3d407b4 pytest: Use a dict to pass options to lightningd
This allows us to have some default options that can then be overridden easily
on a per-test basis.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:52:22 +00:00
Christian Decker 992d395b42 Update walletrpc.c 2018-04-05 19:37:26 +02:00
Zhen Zhang 9423896092 Add funding_address to listfunds RPC response, fixes #1227 2018-04-05 19:37:26 +02:00
Rusty Russell daa14f48f2 peer_control: don't list opening channels as connected=false.
I saw a failure in test_funding_fail():
	assert l2.rpc.listpeers()['peers'][0]['connected']

This can happen if l2 hasn't yet handed back to gossipd.  Turns out
we didn't mark uncommitted channels as connected:

	[{'id': '03afa3c78bb39217feb8aac308852e6383d59409839c2b91955b2d992421f4a41e', 'connected': False, 'channels': [{'state': 'OPENINGD', 'owner': 'lightning_openingd', 'funder': 'REMOTE', 'status': ['Incoming channel: accepted, now waiting for them to create funding tx']}]}]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 429c853fac opening: clearer messages for negotiation failures.
'negotiation_failed' is currently just a useless wrapper around
peer_failed (a vestige from when peer_failed would close the
connection).  Change it to send different local and remote messages,
and use it wherever we dislike their parameters: stick with
peer_failed if we dislike our own parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 21fbae6df8 openingd: ensure that initial channel can cover fees and reserve.
This is probably covered by our "channel capacity" heuristic which
requires the channel be significant, but best to be explicit and sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 441a5b8835 openingd: update check to latest bolt revision.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 83f83f7685 openingd: make sure our their reserve isn't our dust, and vice versa.
This quotes from the BOLT proposal at https://github.com/lightningnetwork/lightning-rfc/pull/389

Don't try to fund channels which would do this, and don't allow others
to fund channels which would do this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 0c2447ed77 openingd: don't allow reserve less than dust.
This quotes from the BOLT proposal at https://github.com/lightningnetwork/lightning-rfc/pull/389

Don't try to fund channels with reserve less than dust, nor allow them
to fund channels with reserve less than dust.

Fixes: #632
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
ZmnSCPxj f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
Justin Litchfield d3d195dd85 bt_print and bt_exit is enclosed in the conditional 2018-04-05 18:17:55 +02:00
practicalswift 1a55147870 Be consistent in choice of shell across all shell scripts in repo 2018-04-04 15:06:30 +02:00
practicalswift bc4a099bff Add shell script linting: Check for shellcheck warnings in shell scripts 2018-04-04 15:06:30 +02:00
practicalswift b95d3b8f54 Fix shellcheck warnings 2018-04-04 15:06:30 +02:00
ZmnSCPxj 5a267eb831 pay, payalgo: Show erring_node as compressed DER pubkey.
For consistency with other node pubkeys.
2018-04-04 14:17:07 +02:00