Commit Graph

480 Commits

Author SHA1 Message Date
Rusty Russell c919551109 onchaind: include htlc id in htlc_stub so we agree on what HTLC we're closing.
If there are two HTLCs with the same preimage, lightningd would always
find the first one.  By including the id in the `struct htlc_stub`
it's both faster (normal HTLC lookup) and allows lightningd to detect
that onchaind wants to fail both of them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:55:35 +02:00
Rusty Russell 0dcd66880c Rename `struct json_result` to `struct json_stream` (RENAMEONLY)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell e46ce0fc84 jsonrpc: declare up front whether a response is success or fail.
Such an API is required for when we stream it directly.  Almost all our
handlers fit this pattern already, or nearly do.

We remove new_json_result() in favor of explicit json_stream_success()
and json_stream_fail(), but still allowing command_fail() if you just
want a simple all-in-one fail wrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell 305795b01e common/json: move JSON creation routines into lightningd/
It's the only user of them, and it's going to get optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossip.pydiff --git a/common/test/run-json.c b/common/test/run-json.c
index 956fdda35..db52d6b01 100644
2018-10-19 22:02:11 +00:00
Rusty Russell 604638712b jsonrpc: Only show total fees collected in getinfo.
And use wallet_forward_status_in_db() everywhere in db code.
And clean up extra CHANGELOG.md entry (looks like rebase error?)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 21:58:27 +00:00
Christian Decker 6d333f16cc wallet: Correctly handle forwards when channels or htlcs are deleted
The left join should make sure we still get the results but
referencing the fields and/or attempting to write them to the JSON-RPC
result will cause unforeseen problems. So just omit if we forgot
something.
2018-10-19 21:58:27 +00:00
Christian Decker facd7d16aa json-rpc: Add `listforwardings` command 2018-10-19 21:58:27 +00:00
Christian Decker 5b924a7eb7 wallet: Add primitives to add forwarded payments to the DB
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker 07ebc525e9 wallet: Add new `forwards` table to keep track of forwarded payments
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Rusty Russell 3991425111 gossipd: don't accept forwarding short_channel_ids we don't own.
Gossipd provided a generic "get endpoints of this scid" and we only
use it in one place: to look up htlc forwards.  But lightningd just
assumed that one would be us.

Instead, provide a simpler API which only returns the peer node
if any, and now we handle it much more gracefully.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-15 23:04:17 +00:00
Christian Decker 81f5b89fe3 wallet: Actually load wallet stats when asked to
The call to `sqlite3_step` is actually needed, otherwise we'll always
get the default values for all types.
2018-10-13 10:58:53 +00:00
lisa neigut b9331e5ac8 gossipd: parse and respect optional `htlc_maximum_msat`
If another channel has set the optional `htlc_maximum_msat` field,
we should correctly parse that field and respect it when drawing up
routes for payments.
2018-10-09 23:22:52 +00:00
Rusty Russell 0226ef0572 htlc: rename local flag to am_origin, add FIXME.
Noted by @cdecker, the term 'local' is grossly overused, and the hout
preimage is basically only used as a sanity check (though I've just put
a FIXME there for now).

Also eliminated spurious blank line which crept into wallet.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 4146950496 lightningd: don't access htlc_in's failoutchannel on db restore.
failoutchannel tells us which channel to send an update for (specifically
for temporary_channel_failure); but we don't save it into the db.  It's
not even clear we should, since it's a corner case and the channel might
not even exist when we come back.

So on db restore, change such errors to WIRE_TEMPORARY_NODE_FAILURE
which doesn't need an update.

We also don't memset it to 0 in the normal case (we only access if it
failcode has the UPDATE bit set) so valgrind will trigger if we're
wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 79ebb8a92e db: save the failcode / failuremsg into db.
Now we can finally move the fixup code under COMPAT_V061, so it's only
for old nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 2293f4ce9e db: restore failuremsg/failcode from db.
We don't write it in there yet, so this change currently has no effect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell d85251ac6c db: fix up HTLCs which are missing failure information.
We don't save them to the database, so fix things up as we load them.

Next patch will actually save them into the db, and this will become
COMPAT code.

Also: call htlc_in_check() with NULL on db load, as otherwise it aborts
internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 4040c53258 lightningd: handle case where incoming HTLC vanished before fulfilled outgoing.
We now need an explicit 'local' flag, rather than relying on the existence
of the 'in' pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell c8c3294a23 json-invoice: add routeboost, warnings.
We split json_invoice(), as it now needs to round-trip to the gossipd,
and uniqueness checks need to happen *after* gossipd replies to avoid
a race.

For every candidate channel gossipd gives us, we check that it's in
state NORMAL (not shutting down, not still waiting for lockin), that
it's connected, and that it has capacity.  We then choose one with
probability weighted by excess capacity, so larger channels are more
likely.

As a side effect of this, we can tell if an invoice is unpayble (no
channels have sufficient incoming capacity) or difficuly (no *online*
channels have sufficient capacity), so we add those warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell 9fa7f5e30e listpeers: include current htlc information.
This enables the next patch, which allows us to wait until all HTLCs are
completely resolved.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell 96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Mark Beckwith 30b67c0334 param: call param() all the time
Now call param() even for commands that don't accept any parameters.

This is a bugfix of sorts.  For example, before you could call:

	bitcoin-cli getinfo blah

and the blah parameter would be ignored.

Now you will get an error: "too many parameters: got 1, expected 0"

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Rusty Russell 0855422110 gossip_control: when searching for a txout, make sure it's not spent!
There's no reason for the db to ever return non-NULL if it's spent.  And there's
only one caller, for which that is definitely true.

Suggested-by: @cdecker
Fixes: #1934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell c2e56fbb1b wallet: fix Makefile to include correct dependencies.
It didn't depend on its own headers, it should also depend on
lightningd/lightningd.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-14 22:11:48 +02:00
Rusty Russell cefb6925b2 db: save and restore last_sent_commit correctly.
It's an array: we were only saving the single element; if there was more than
one changed HTLC we'd get a bad signature!

The report in #1907 is probably caused by the other side re-requesting
something we considered already finalized; to avoid this particular error,
we should set the field to NULL if there's no last_sent_commit.

I'm increasingly of the opinion we want to just save all the update
packets to the db and blast them out, instead of doing this
second-guessing dance.

Fixes: #1907
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:43:27 +02:00
Rusty Russell 168bec0974 lightningd: move channel/peer/htlc load into own function.
Also, wallet has no business wiring up HTLCs; move that code to
peer_htlcs.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell 3e53a63cf2 wallet: do wallet_invoice init during preparation.
We have a transaction anyway, and it's simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell db3c387264 feerate: allow names 'urgent' 'normal' and 'slow'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell e0952ceff2 feerate: use suffix, not separate argument.
And, reluctantly, default to bitcoind style.
"It's wrong to be right too soon."

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell 14dc1c37ab fundchannel / withdraw: allow explicit feerate setting.
These are the two cases where we'll refuse without a fee estimate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell 175db926c2 chaintopology: expose when we don't actually know feerate.
We use feerate in several places, and each one really should react
differently when it's not available (such as when bitcoind is still
catching up):

1. For general fee-enforcement, we use the broadest possible limits.
2. For closingd, we use it as our opening negotiation point: just use half
   the last tx feerate.
3. For onchaind, we can use the last tx feerate as a guide for our own txs;
   it might be too high, but at least we know it was sufficient to be mined.
4. For withdraw and fund_channel, we can simply refuse.

Fixes: #1836
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell a5ecc95c42 db: store claimed per_commitment_point from option_data_loss_protect.
This means we don't try to unilaterally close after a restart, *and*
we can tell onchaind to try to use the point to recover funds when the
peer unilaterally closes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 05f12edf60 txwatch: hand ld to callback, don't assume channel is non-NULL.
We're about to use the txwatch facility for UTXOs, where there's no channel,
so allow that the be NULL, and hand the struct lightningd which callers
want anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 5d23698665 wallet: expose function to confirm a tx.
Note that we don't actually need the output number: it's the tx itself
which is confirmed.  And the next caller doesn't have it convenient, so
eliminate it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 27835df8fd wallet: add accessor getting close-info-needed unconfirmed UTXOs.
These are not confirmed by the normal methods (wallet_can_spend is false!),
so we'll deal with them manually.

We use UTXO_FIELDS in wallet_add_utxo, too, for consistency.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Mark Beckwith a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith fa55e2cab0 param: upgraded json_tok_loglevel
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 1fb203891d param: upgraded json_tok_short_channel_id
Made previous version a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 9b28ecf8fc param: upgraded json_tok_pubkey
Also add json_to_pubkey as utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 2a0c2601c0 param: upgraded json_tok_newaddr
Made it a local static since its a one-off.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8f17191099 param: upgraded json_tok_tok to advanced callback
This was a very simple change and allowed us to remove the special
`json_opt_tok` macro.

Moved the callback out of `common/json.c` to `lightningd/json.c` because the new
callbacks are dependent on `struct command` etc.
(I already started on `json_tok_number`)

My plan is to:
	1. upgrade json_tok_X one a time, maybe a PR for each one.
	2. When done, rename macros (i.e, remove "_tal").
	3. Remove all vestiges of the old callbacks
	4. Add new callbacks so that we no longer need json_tok_tok!
	   (e.g., json_tok_label, json_tok_str, json_tok_msat)

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-13 23:46:35 +00:00
Rusty Russell 74568a1c50 lightningd: peer_start_channeld always returns true; make it void.
It is always true, and we always ignore it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell 7c856470e2 wallet: add buildtime and runtime assertions on db enums.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:44:28 +02:00
Rusty Russell 1dde233a48 wallet: put explicit hook in for fatal error testing.
We're currently overriding fatal() with something that actually
returns, which contrasts with its declaration as NORETURN.

This breaks in the next patch which wants a real fatal() in wallet.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:44:28 +02:00
Rene Pickhardt 8b902729b9 added a test case so that the enum wallet_payment_status which is critical for the communitcation with the data store will not accidentely being changed by some developer
[ Folded "cleaned up the test code" -- RR ]
[ Fixed prototype -- RR ]
2018-08-10 12:44:28 +02:00
Rusty Russell d8d4b19f3a connectd: remove separate address hint message.
Include it as an optional field in the connect_to_peer message (it was
added before we had optional fields).

The only issue is that reconnects want it too, so again connectd hands
it back to master in connectctl_connect_failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00