Commit Graph

412 Commits

Author SHA1 Message Date
Christian Decker 876d698f3c wallet: Add primitives to store onchaind transactions in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker d2dc93e3cb wallet: Add a struct to represent an onchaind transaction
This will be used to replay transactions that were witnessed in the blockchain
during startup, so that onchaind can be recreate its state.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 28feb2eb7d db: Add table for onchaind transactions
These transactions being seen on the blockchain triggered some action in
onchaind so we need to replay them when we restore the onchaind.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Rusty Russell d2b4e09e27 lightningd: re-allow closing negotiation when CLOSINGD_COMPLETE
d822ba1ee accidentally removed this case, which is important: if the
other side didn't get our final matching closing_signed, it will
reconnect and try again.  We consider the channel no longer "active"
and thus ignore it, and get upset when it send the
`channel_reestablish` message.

We could just consider CLOSINGD_COMPLETE to be active, but then we'd
have to wait for the closing transaction to be mined before we'd allow
another connection.

We can't special case it when the peer reconnects, because there
could be (in theory) multiple channels for that peer in CLOSINGD_COMPLETE,
and we don't know which one to reestablish.

So, we need to catch this when they send the reestablish, and hand
that msg to closingd to do negotiation again.  We already have code
to note that we're in CLOSINGD_COMPLETE and thus ignore any result
it gives us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell 8e976150ad json_fundchannel: fix release vs connect/nongossip race.
The new connect code revealed an existing race: we tell gossipd to
release the peer, but at the same time it connects in.  gossipd fails
the release because the peer is remote, and json_fundchannel fails.

Instead, we catch this race when we get peer_connected() and we were
trying to open a channel.  It means keeping a list of fundchannels which
are awaiting a gossipd response though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell bee795ed68 channeld: don't do explicit state update.
We missed it in some corner cases where we crashed/were killed between
being told of the lockin and sending the channel_normal_operation message.
When we were restarted, we were told both sides were locked in already,
so we never updated the state.

Pull the entire "tell channeld" logic into channel_control.c, and make
it clear that we need to keep waching if we cant't tell channeld.  I think
we did get this correct in practice, since funding_announce_cb has the
same test, but it's better to be clear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
ZmnSCPxj 2cee1ab20f peer_control: Make close wait for complete closure, with timeout.
Also report tx and txid, and whether we closed unilaterally or
bilaterally, if we could close the channel.

Also make a manpage.

Fixes: #1207
Fixes: #714
Fixes: #622
2018-04-23 05:24:46 +00:00
ZmnSCPxj 990ef729b0 walletrpc: Refactor out encoding of pubkey to address. 2018-04-22 12:51:03 +02:00
ZmnSCPxj 3bf626a750 walletrpc: Correct generation of P2SH-P2WPKH addresses for listfunds. 2018-04-22 12:51:03 +02:00
conanoc 0733770559 Adjust indents 2018-04-21 15:55:00 +02:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ 11ca729d85 wallet, payalgo: Save detail of payment failures for later reporting. (#1345)
Pointless for remote failures as those are never sent by
the erring node, but for local failures we can give more
detail.
2018-04-16 15:29:40 +02:00
Christian Decker f27cd3e43f topo: Remove in-memory txs from the block struct
The only use for these was to compute their txids so we could notify depth
in case of reorgs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 23984ecde4 chaintopology: Use the DB to locate transactions and rebroadcast txs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker aa696370af txwatch: Switch to passing only txid into the depth callbacks
All of the callback functions were only using the tx to generate the txid again,
so we just pass that in directly and save passing the tx itself.

This is a simplification to move to the DB backed depth callbacks. It'd be
rather wasteful to read the rawtx and deserialize just to serialize right away
again to find the txid, when we already searched the DB for exactly that txid.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 85fbab2fab wallet: Add function to retrieve a watched transaction's blockheight
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 633ffeec6f wallet: Add methods to add transactions to the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 0f89653ce8 db: Add table for transactions we are interested in
Currently these are either transactions we sent ourselves or transactions that
we are watching because they are part of a channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker ba3ceb2abf wallet: Lowerbound the rescan by going at most back to LNs origin
Repeated crashes could result in the `last_processed_block` variable being
pushed further and further into the past (in some cases going as far back as
scanning blocks from 2012...). This is a stop-gap solution that just lower
bounds the value to what is the first possible block we might be interested in
LN, until we have the 0-rescan fix I'm working on.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker ffbe56847e db: Fixing some clang warnings
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker 9ab28d1659 topology: Two off-by-one error when catching up with the blockchain
There are two very hard problems in software engineering:

 1. Off-by-one errors

In this case we were rolling back further than needed and we were starting the
catchup one block further than expected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:55:41 +00:00
Christian Decker 992d395b42 Update walletrpc.c 2018-04-05 19:37:26 +02:00
Zhen Zhang 9423896092 Add funding_address to listfunds RPC response, fixes #1227 2018-04-05 19:37:26 +02:00
ZmnSCPxj f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
Rusty Russell 19d5305658 wallet/test: fix Makefile so test correctly depend on wallet files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 6bb47276ce lightningd: put min/max feerates into db, struct channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 8975fc2ceb libwally: use their secp context for all our daemons.
I didn't convert all tests: they can still use a standalone context.
It's just marginally more efficient to share the libwally one for all
our daemons which link against it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Christian Decker 1133820ad6 db: Add a cleanup migration to clear any dangling utxoset entries
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker eb531e6da7 db: Make sure to enable foreign keys even after forking
This may be causing #1280, since with `--daemon` the DB is being reopened
without enabling the foreign key relations and hence the delete cascades.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker a8d587c418 wallet: Return any eventual outpoint scid when marking it spent
Just return the short_channel_id matching the outpoint that we just marked as
spent.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker ba7341ec87 cleanup: Make blockheights unsigned
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
Christian Decker 286cfefc62 wallet: Remove pruned outpoints from utxoset outpoint_filter
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 17:14:58 +02:00
practicalswift 6269a4c55d Remove unused functions not covered by unit tests 2018-03-26 23:35:56 +00:00
ZmnSCPxj b914062465 pay: Also record how much we actually ended up sending. 2018-03-26 05:01:49 +00:00
practicalswift 7e9750ffee Reduce variable scopes 2018-03-26 01:31:21 +00:00
ZmnSCPxj 0bb9bcc0f1 wallet: Track some channel usage statistics.
Fixes: #1049
2018-03-26 01:08:52 +00:00
Rusty Russell 76e8a11380 wallet: use json_escaped for invoice label.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell d92579f627 common/json_escaped: new type which explicitly notes a string is already JSON.
Trivial to use as a string, but it still means you should be careful
around it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift 98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift 0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
ZmnSCPxj d5d31864cf wallet: Remove onion-decoding information from db on payment success/fail.
Fixes: #1177
2018-03-21 13:56:18 +01:00
Mark Beckwith 489d8d04a6 Fund all (#1249)
* Funding channel with "all"
* Added test for fund all
2018-03-21 00:06:04 +01:00
ZmnSCPxj 159e736fc1 invoices: Implement autocleaninvoice. 2018-03-20 17:25:51 +01:00
ZmnSCPxj 3b0081aebb invoices: Add delexpiredinvoice command. 2018-03-20 17:25:51 +01:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ 044705a547 peer_control: Fix a use-after-free bug. (#1237)
This bug is a classic case of being lazy:
1. peer_accept_channel() allocated its return off the input message,
   rather than taking an explicit allocation context.  This concealed the
   lifetime nature of the return.
2. The context for sanitize_error was the error itself, rather than the
   more obvious tmpctx (connect_failed does not take).

The global tmpctx removes the "efficiency" excuse for grabbing a random
object to use as context, and is also nice and explicit.

All-the-hard-work-by: @ZmnSCPxj
2018-03-19 09:27:09 +00:00
ZmnSCPxj 9efe123a0d lightningd/json: Move json helpers specific for lightningd to new module. 2018-03-19 00:27:55 +00:00
Rusty Russell 1b9791f0f5 pay: delete HTLC when we delete payment.
This fixes the root cause of https://github.com/ElementsProject/lightning/issues/1212
where we deleted the payment because we wanted to retry, then retry failed
so we had an (old) HTLC without a matching payment.  We then fed that
HTLC to onchaind, which tells us it's missing, and we try to fail the
payment and deref a NULL pointer.

Fixes: #1212
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 04:01:06 +00:00
practicalswift 03e2e404f2 Mark unused parameters as such (using "UNUSED") 2018-03-16 01:17:40 +00:00
practicalswift 9970419b3a Remove redundant return from void function 2018-03-16 01:17:40 +00:00
Rusty Russell 0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell de5d84097e lightningd: remove ltmp context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
ZmnSCPxj 60f5abdc9b wallet: Make wallet_payment_store idempotent. 2018-03-14 05:33:09 +00:00
ZmnSCPxj 693c718160 wallet: Add DB fields for recording routing failures. 2018-03-14 05:33:09 +00:00
Rusty Russell 46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Christian Decker 387ba76920 wallet: Add method to retrieve outpoints from the utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker d363a6870d wallet: Fix header ordering
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-07 18:55:51 +01:00
Rusty Russell f8c8d55565 db: migrate old shutdown_keyidx_local -1 values onto key 0.
It would be better to give them unique values, but we don't fully support
db migrate anyway and this is simple (though they will end up using the
same key for multiple channel closes if created before this commit).

Note that even if bip32_max_index is currently unset, it defaults to 0
so it will be found.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell 6c1233de44 channel: reserve a bip32 index as soon as channel is opened.
This simplifies things, and means it's always in the database.  Our
previous approach to creating it on the fly had holes when it was
created for onchaind, causing us to use another every time we
restarted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell 242a934b8c wallet: add check-source to makefile.
And fix up resulting breakage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell e76f863329 txfilter: clean up prototypes.
Use const, add TAKES to declaration.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell 9d9c1c3db6 wallet: wrap missing last_processed_block handling in COMPAT_V052.
Our testing also reveals a bug: we start lightningd and shut it down
before fully processing the blockchain, so we don't set
last_processed_block.  Fix that by setting it immediately once we have
a block: worst case it goes backwards a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
John Barboza 30b290cb8f Explicit error message when disconnect fails
Whether the peer is not connected or not gossiping.
2018-03-07 16:14:01 +01:00
John Barboza 0a1351fd51 gossipd: command to disconnect from a gossiping peer
lightning-cli disconnect <peer id>
2018-03-07 16:14:01 +01:00
Christian Decker aba3d5f34d wallet: Clean up spent UTXOs once they are buried deeply enough
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 8958a082ec wallet: Mark utxo outpoints as spent
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 50a8e32e3f db: Add index on utxoset.spendheight to speed up blockchain reorgs
Otherwise we would be doing a table scan per block being reorged/rescanned.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 27e0ec694c wallet: Add primitive to register new utxoset outpoint to the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker d8d11e5689 wallet: Add outpointfilter for the utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker ec014be2a8 db: Add utxoset table
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 5a1b4b91e2 wallet: Fix output extraction when we own multiple outputs
When we already know about an output we would stop scanning the remaining
outputs. Known outputs happen whenever we extracted from our own transactions
and then extracted again from blocks. We would not update if the first update
fails.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker ae30942e3f chaintopo: Record outpoint spends for owned outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 2680e6d9ff wallet: Move txfilter into wallet
Transaction filters are strongly related to the wallet, this move just
makes it a bit more explicit.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker b3c94a5256 wallet: Update existing owned outputs with their confirmation height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 7fc06d9b3e wallet: Add outpointfilter to wallet so we can pass it all outputs
Will be used later to filter out outputs we are interested in, and
trigger db updates with them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 247aa13a58 db: Add index covering the output heights
Avoids performing a table scan, now deletes on blocks are a lot
faster.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 4e2c1dd038 jsonrpc: Add confirmation status to `listfunds` output
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 227dc36146 utxo: Add blockheight and spendheight to outputs to track state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 7f00230657 db: Add confirmation_height and spend_height columns to outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 9440b23852 wallet: Add primitive to roll back to a specific height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker 504202973f wallet: Add primitives to store blockchain internally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker fea03dbca9 db: Add a table to track blockheaders
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
John Barboza b50912d8b1 Print node alias when listing peers 2018-03-02 02:25:30 +00:00
Rusty Russell c5d41a23d7 short_channel_id: just use structeq.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Igor Cota cb6820d445 Do preimage, hash and consequentially bolt11 generation before we store the invoice in db.
This way we store bolt11 to the table as well, in case a client needs it later
2018-03-01 03:55:48 +00:00
Igor Cota 3371168d54 Add bolt11 to the invoice struct.
We store this in case an RPC client asks for it
2018-03-01 03:55:48 +00:00
Igor Cota 2bb9797ddb Add bolt11 to the invoices db table as TEXT.
Saving in case an RPC client needs it later
2018-03-01 03:55:48 +00:00
ZmnSCPxj efc5da9ad4 invoices: Use common code for loading r and rhash. 2018-02-28 11:17:08 +01:00
ZmnSCPxj 978e5c67d8 invoices: Remove persistent in-memory invoice structures. 2018-02-28 11:17:08 +01:00
ZmnSCPxj f05c86618c invoices: Change iteration interface to be more abstract. 2018-02-28 11:17:08 +01:00
ZmnSCPxj e9112af0bc invoices: Move expiration timer to invoices object.
In preparation for removing in-memory invoices objects.
2018-02-28 11:17:08 +01:00
ZmnSCPxj 7a6fe0704e invoices: Unify waitone/any waiters to invoices object.
In preparation for removing in-memory invoice objects.
Since they will be removed, there will no longer be any
individual invoice objects to attach to.
2018-02-28 11:17:08 +01:00
ZmnSCPxj 9b4c6699f9 invoices: Semantically separate invoice details from invoice.
In preparation for removing in-memory invoice structures.
Invoice details are requested rarely anyway.
2018-02-28 11:17:08 +01:00
Rusty Russell b7ed5670d5 lightningd: close and reopen db across fork for daemonize
Fixes: #1092
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-27 00:09:34 +01:00
ZmnSCPxj f7f55edcdb wireaddr: Do hostname resolution in parse_wireaddr.
Fixes: #834
2018-02-26 03:13:55 +00:00
ZmnSCPxj bd07a91782 jsonrpc: Let Base58 '2..' address work on both regtest and testnet. 2018-02-26 03:09:15 +00:00
John Barboza b028a0a439 invoice: add fallback address to invoice command
* Modifies invoice command to have the following format
  invoice <msatoshi> <label> <desc> <?expiry> <?fallbackaddr>
* Adds support for Segwit bcrt1 addresses for withdraw
* Add test case for fallback address in invoice creation
* Create a common json_tok_address_scriptpubkey to be used
  by invoice and withdraw commands.
2018-02-26 03:09:15 +00:00
Christian Decker 7c8883ae7d wallet: Fix an off by one error in dev-listaddrs
Reported-By: @Xian001
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-23 21:53:40 +00:00
Rusty Russell cccdb53bec channel_states: fold all the ONCHAIND states into one.
The billboard is now far more useful to tell what's going on, and this
gets us closer to a state == owner mapping.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell 86a04c59d4 lightningd: maintain a status billboard for each channel.
Each state (effectively, each daemon) has two slots: a permanent slot
if something permanent happens (usually, a failure), and a transient
slot which summarizes what's happening right now.

Uncommitted channels only have a transient slot, by their very nature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell 655ae90e5e wallet: remove obsolete peers.
Fixes: #1029
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-22 12:06:39 +01:00