Commit Graph

563 Commits

Author SHA1 Message Date
Rusty Russell 4040c53258 lightningd: handle case where incoming HTLC vanished before fulfilled outgoing.
We now need an explicit 'local' flag, rather than relying on the existence
of the 'in' pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell c8c3294a23 json-invoice: add routeboost, warnings.
We split json_invoice(), as it now needs to round-trip to the gossipd,
and uniqueness checks need to happen *after* gossipd replies to avoid
a race.

For every candidate channel gossipd gives us, we check that it's in
state NORMAL (not shutting down, not still waiting for lockin), that
it's connected, and that it has capacity.  We then choose one with
probability weighted by excess capacity, so larger channels are more
likely.

As a side effect of this, we can tell if an invoice is unpayble (no
channels have sufficient incoming capacity) or difficuly (no *online*
channels have sufficient capacity), so we add those warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell 9fa7f5e30e listpeers: include current htlc information.
This enables the next patch, which allows us to wait until all HTLCs are
completely resolved.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell 96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Mark Beckwith 30b67c0334 param: call param() all the time
Now call param() even for commands that don't accept any parameters.

This is a bugfix of sorts.  For example, before you could call:

	bitcoin-cli getinfo blah

and the blah parameter would be ignored.

Now you will get an error: "too many parameters: got 1, expected 0"

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Rusty Russell 0855422110 gossip_control: when searching for a txout, make sure it's not spent!
There's no reason for the db to ever return non-NULL if it's spent.  And there's
only one caller, for which that is definitely true.

Suggested-by: @cdecker
Fixes: #1934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell c2e56fbb1b wallet: fix Makefile to include correct dependencies.
It didn't depend on its own headers, it should also depend on
lightningd/lightningd.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-14 22:11:48 +02:00
Rusty Russell cefb6925b2 db: save and restore last_sent_commit correctly.
It's an array: we were only saving the single element; if there was more than
one changed HTLC we'd get a bad signature!

The report in #1907 is probably caused by the other side re-requesting
something we considered already finalized; to avoid this particular error,
we should set the field to NULL if there's no last_sent_commit.

I'm increasingly of the opinion we want to just save all the update
packets to the db and blast them out, instead of doing this
second-guessing dance.

Fixes: #1907
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:43:27 +02:00
Rusty Russell 168bec0974 lightningd: move channel/peer/htlc load into own function.
Also, wallet has no business wiring up HTLCs; move that code to
peer_htlcs.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell 3e53a63cf2 wallet: do wallet_invoice init during preparation.
We have a transaction anyway, and it's simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell db3c387264 feerate: allow names 'urgent' 'normal' and 'slow'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell e0952ceff2 feerate: use suffix, not separate argument.
And, reluctantly, default to bitcoind style.
"It's wrong to be right too soon."

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell 14dc1c37ab fundchannel / withdraw: allow explicit feerate setting.
These are the two cases where we'll refuse without a fee estimate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell 175db926c2 chaintopology: expose when we don't actually know feerate.
We use feerate in several places, and each one really should react
differently when it's not available (such as when bitcoind is still
catching up):

1. For general fee-enforcement, we use the broadest possible limits.
2. For closingd, we use it as our opening negotiation point: just use half
   the last tx feerate.
3. For onchaind, we can use the last tx feerate as a guide for our own txs;
   it might be too high, but at least we know it was sufficient to be mined.
4. For withdraw and fund_channel, we can simply refuse.

Fixes: #1836
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell a5ecc95c42 db: store claimed per_commitment_point from option_data_loss_protect.
This means we don't try to unilaterally close after a restart, *and*
we can tell onchaind to try to use the point to recover funds when the
peer unilaterally closes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 05f12edf60 txwatch: hand ld to callback, don't assume channel is non-NULL.
We're about to use the txwatch facility for UTXOs, where there's no channel,
so allow that the be NULL, and hand the struct lightningd which callers
want anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 5d23698665 wallet: expose function to confirm a tx.
Note that we don't actually need the output number: it's the tx itself
which is confirmed.  And the next caller doesn't have it convenient, so
eliminate it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 27835df8fd wallet: add accessor getting close-info-needed unconfirmed UTXOs.
These are not confirmed by the normal methods (wallet_can_spend is false!),
so we'll deal with them manually.

We use UTXO_FIELDS in wallet_add_utxo, too, for consistency.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Mark Beckwith a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith fa55e2cab0 param: upgraded json_tok_loglevel
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 1fb203891d param: upgraded json_tok_short_channel_id
Made previous version a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 9b28ecf8fc param: upgraded json_tok_pubkey
Also add json_to_pubkey as utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 2a0c2601c0 param: upgraded json_tok_newaddr
Made it a local static since its a one-off.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8f17191099 param: upgraded json_tok_tok to advanced callback
This was a very simple change and allowed us to remove the special
`json_opt_tok` macro.

Moved the callback out of `common/json.c` to `lightningd/json.c` because the new
callbacks are dependent on `struct command` etc.
(I already started on `json_tok_number`)

My plan is to:
	1. upgrade json_tok_X one a time, maybe a PR for each one.
	2. When done, rename macros (i.e, remove "_tal").
	3. Remove all vestiges of the old callbacks
	4. Add new callbacks so that we no longer need json_tok_tok!
	   (e.g., json_tok_label, json_tok_str, json_tok_msat)

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-13 23:46:35 +00:00
Rusty Russell 74568a1c50 lightningd: peer_start_channeld always returns true; make it void.
It is always true, and we always ignore it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell 7c856470e2 wallet: add buildtime and runtime assertions on db enums.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:44:28 +02:00
Rusty Russell 1dde233a48 wallet: put explicit hook in for fatal error testing.
We're currently overriding fatal() with something that actually
returns, which contrasts with its declaration as NORETURN.

This breaks in the next patch which wants a real fatal() in wallet.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:44:28 +02:00
Rene Pickhardt 8b902729b9 added a test case so that the enum wallet_payment_status which is critical for the communitcation with the data store will not accidentely being changed by some developer
[ Folded "cleaned up the test code" -- RR ]
[ Fixed prototype -- RR ]
2018-08-10 12:44:28 +02:00
Rusty Russell d8d4b19f3a connectd: remove separate address hint message.
Include it as an optional field in the connect_to_peer message (it was
added before we had optional fields).

The only issue is that reconnects want it too, so again connectd hands
it back to master in connectctl_connect_failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 8939a5001b connectd: rely on the master to tell us to reconnect.
connectd tells master about every disconnection, and master knows
whether it's important to reconnect.  Just get the master to invoke a new
connect command if it considers the peer important!

The only twist is timeouts: we don't want to immediately reconnect if
we've failed to connect.  To solve this, connectd passes a 'delaytime'
to the master when a connection fails, and the master passes it back
when it asks for a connection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 30f08cc2b0 connectd: always tell master when connection fails/succeeded.
We used to separate implicit connection requests (ie. timed retries
for important peers) and explicit ones, and send a
WIRE_CONNECTCTL_CONNECT_TO_PEER_RESULT for the latter.

In the success case, that's now redundant, since we hand the connected
peer to the master using WIRE_CONNECT_PEER_CONNECTED; we just need a
message for the failure case.  And we might as well tell the master
every failure, so we don't have to distinguish internally.

This also solves a race we had before: connectd would send
WIRE_CONNECTCTL_CONNECT_TO_PEER_RESULT which completes the incoming
JSON connect command, then send WIRE_CONNECT_PEER_CONNECTED.  So
there's a window where the JSON command can return, but the peer isn't
known to lightningd yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 909cd4136b openingd: get told if we can't let them open a new channel.
Previously master would fail once the channel has been negotiated,
which is terrible, since the funder will have already broadcast tx.

Now we tell them if we have an active channel, and update if it goes away.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 02966a4857 connectd: remove unused handback APIs and code.
We now simply maintain a pubkey set for connected peers (we only care
if there's a reconnect), not the entire peer structure.

lightningd no longer queries us for getpeers: it knows more than we do
already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 50f5eb34b4 openingd: take peer before we're opening, wait for explicit funding msg.
Prior to this, lightningd would hand uninteresting peers back to connectd,
which would then return it to lightningd if it sent a non-gossip msg,
or if lightningd asked it to release the peer.

Now connectd hands the peer to lightningd once we've done the init
handshake, which hands it off to openingd.

This is a deep structural change, so we do the minimum here and cleanup
in the following patches.

Lightningd:
1. Remove peer_nongossip handling from connect_control and peer_control.
2. Remove list of outstanding fundchannel command; it was only needed to
   find the race between us asking connectd to release the peer and it
   reconnecting.
3. We can no longer tell if the remote end has started trying to fund a
   channel (until it has succeeded): it's very transitory anyway so not
   worth fixing.
4. We now always have a struct peer, and allocate an uncommitted_channel
   for it, though it may never be used if neither end funds a channel.
5. We start funding on messages for openingd: we can get a funder_reply
   or a fundee, or an error in response to our request to fund a channel.
   so we handle all of them.
6. A new peer_start_openingd() is called after connectd hands us a peer.
7. json_fund_channel just looks through local peers; there are none
   hidden in connectd any more.
8. We sometimes start a new openingd just to send an error message.

Openingd:
1. We always have information we need to accept them funding a channel (in
   the init message).
2. We have to listen for three fds: peer, gossip and master, so we opencode
   the poll.
3. We have an explicit message to start trying to fund a channel.
4. We can be told to send a message in our init message.

Testing:
1. We don't handle some things gracefully yet, so two tests are disabled.
2. 'hand_back_peer .*: now local again' from connectd is no longer a message,
   openingd says 'Handed peer, entering loop' once its managing it.
3. peer['state'] used to be set to 'GOSSIPING' (otherwise this field doesn't
   exist; 'state' is now per-channel.  It doesn't exist at all now.
4. Some tests now need to turn on IO logging in openingd, not connectd.
5. There's a gap between connecting on one node and having connectd on
   the peer hand over the connection to openingd.  Our tests sometimes
   checked getpeers() on the peer, and didn't see anything, so line_graph
   needed updating.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Christian Decker 2d1190d929 wallet: Add missing description field to the invoices iteration 2018-08-04 23:49:28 +00:00
practicalswift 0f7b11bdc2 Remove redundant code 2018-08-02 15:58:14 +09:30
practicalswift 09ec0ae990 Return true or false from sqlite3_bind_* based on result from SQLite instead of always returning true 2018-07-31 12:44:44 +02:00
Rusty Russell 52303029aa fundchannel: cap 'all' at 2^24-1 satoshi.
The easiest way to do this is to play with the 'wallet_tx' semantics
and have 'amount' have meaning even when 'all_funds' is set.

Note that we change the string 'Cannot afford funding transaction' to
'Cannot afford transaction' as this code is also used for withdrawls.

Inspired-by: molz on #c-lightning
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:46:41 +02:00
Rusty Russell 337075dc8c tal: don't access low-level tal functions.
In several places we use low-level tal functions because we want the
label to be something other than the default.  ccan/tal is adding
tal_*_label so replace them and shim it for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Rusty Russell 5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Rusty Russell eae9b81099 json: json_add_hex_talarr for common case of dumping a tal object in hex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Christian Decker fbbc5899e4 invoices: Make the invoice_details more idiomatic
This seems like a premature optimization: it tried to cut down the number of
allocations by reusing the same `struct invoice_details` while iterating through
a number of results. But this sidesteps the checks by `valgrind` and we'd miss a
missing field that was set by the previous iteration.

Reported-by: @rustyrussell
Signed-off-by: Christian Decker <@cdecker>
2018-07-30 03:04:45 +00:00
Christian Decker 259a69994d json-rpc: Retrieve the invoice description in listinvoices 2018-07-30 03:04:45 +00:00
Christian Decker 7f45947523 wallet: Store the description in the database for invoices 2018-07-30 03:04:45 +00:00
Christian Decker 6bbe5b60f6 wallet: Extract the payment fields into a define
`wallet_stmt2payment` always expects the same fields in the same order, so we
should make sure that we always fetch them in that order and all of them.
2018-07-30 03:04:45 +00:00
Christian Decker 60e0eec967 wallet: Store payment description in the database 2018-07-30 03:04:45 +00:00
Christian Decker ff5ca37f3c wallet: Add descriptions to invoice and payment structs 2018-07-30 03:04:45 +00:00
Christian Decker 7119a02932 db: Add description fields to invoices and payments 2018-07-30 03:04:45 +00:00
Christian Decker 55d450ff00 pay: Exempt a fee from the maxfeepercent rule it is still tiny
Several users have noticed that they cannot pay satoshis.place or similar places
that have tiny payment amounts if they are not directly connected. This is due
to the forwarding fee dominating the transferred amount.

This commit adds a new option, exempting tiny fees (up to 5 satoshis by default)
from having to pass the maxfeepercent flag. While we could have told users to
tweak maxfeepercent I think it is usefull to have a default exemption.

[Squashed --RR]
2018-07-30 01:05:43 +00:00
Rusty Russell 722dd05e9d lightningd: keep features arrays for connected peers.
As a side-effect, we only print them for connected peers (which avoids
an O(n^2) traversal).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell d241bd762c connectd: don't use gossip_getnodes_entry.
gossip_getnodes_entry was used by gossipd for reporting nodes, and for
reporting peers.  But the local_features field is only available for peers,
and most other fields are only available from node_announcement.

Note that the connectd change actually means we get less information
about peers: gossipd used to do the node lookup for peers and include the
node_announcement information if it had it.

Since generate_wire.py can't create arrays-of-arrays, we add a 'struct
peer_features' to encapsulate the two feature arrays for each peer, and
for convenience we add it to lightningd/gossip_msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 5c066b9a25 connectd: wean off gossip_wire message types, use our own.
This involves much renaming of gossip -> channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Mark Beckwith 369bd274a1 Updated docs to reflect bech32 default
The manpage did not specify the name of the named parameter (addresstype).
It does now.

Fixed manpage grammar errors.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-23 23:46:06 +00:00
Rusty Russell 231f14e645 lightningd: get basepoints from hsmd, don't ever get seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 93b8217eb0 lightningd: get HSM to sign the last commitment tx for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell f456fdfab1 lightningd: keep local_basepoints and local_funding_pubkey.
For now we can always regenerate them, but eventually they'll be given
to us by the HSM.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Mark Beckwith f850849486 Modern param style for all remaining files
Removed `json_get_params`.

Also added json_tok_percent and json_tok_newaddr. Probably should
have been a separate PR but it was so easy.

[ Squashed comment update for gcc workaround --RR ]
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-20 01:14:02 +00:00
Rusty Russell 28c3706f87 hsmd: fix missing status messages.
I crashed the HSMD, and it gave no output at all.  That's because we
were only reading the status fd when we were waiting for a reply.

Fix this by using a separate request fd and status fd, which also means
that hsm_sync_read() is no longer required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell e217bc1220 per-commit-secret is a struct secret, not a sha256.
Well, it's generated by shachain, so technically it is a sha256, but
that's an internal detail.  It's a secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell 6c98457ef2 per-peer seed is a 'struct secret' not a 'struct privkey'.
They're both 32 bytes, but it's not a privkey at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
William Casarin 4a1b68502e wallet: fix up semi-cryptic wallet error message
There is evidence that this message can be confusing[1].
Suggest a way to resolve the error in the message.

[1] https://www.reddit.com/r/Bitcoin/comments/7ybcq9/programming_how_can_i_resolve_the_below_error/

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-03 23:58:55 +00:00
Jan Sarenik cb53ad0706 wallet/wallet.c: bind_null if (payment_key) 2018-07-02 05:00:08 +00:00
Jan Sarenik da80447ba3 wallet/wallet.com: bind_null if (out->preimage) 2018-07-02 05:00:08 +00:00
Jan Sarenik 5205de7b8e wallet/wallet.c: bind_null if (in->preimage) 2018-07-02 05:00:08 +00:00
Jan Sarenik 580b8ba8a9 wallet/wallet.c: bind_null if (chan->remote_shutdown_scriptpubkey) 2018-07-02 05:00:08 +00:00
Jan Sarenik 2090ddd1fa wallet/wallet.c: bind_null if (chan->scid) 2018-07-02 05:00:08 +00:00
Jan Sarenik 0640ac23cb wallet/wallet.c: Add missing bind_null
In order to address #338
2018-07-02 05:00:08 +00:00
Rusty Russell 0e6c0dbba2 bitcoin: expose feerate_floor.
Onchaind will want it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 13:43:32 +02:00
Christian Decker d43403257d wallet: Cleanup db files if the tests succeed
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Christian Decker 024dca0fff wallet: Return both min and max block heights
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Christian Decker 8e278044e3 gossip: Disable channels when we lose the connection to the peer
We're telling gossipd about disconnections anyway, so let's just use that signal
to disable both sides of the channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Mark Beckwith 7f437715d5 Added error code parameter to command_fail
Until now, `command_fail()` reported an error code of -1 for all uses.
This PR adds an `int code` parameter to `command_fail()`, requiring the
caller to explicitly include the error code.

This is part of #1464.

The majority of the calls are used during parameter validation and
their error code is now JSONRPC2_INVALID_PARAMS.

The rest of the calls report an error code of LIGHTNINGD, which I defined to
-1 in `jsonrpc_errors.h`.  The intention here is that as we improve our error
reporting, all occurenaces of LIGHTNINGD will go away and we can eventually
remove it.

I also converted calls to `command_fail_detailed()` that took a `NULL` `data`
parameter to use the new `command_fail()`.

The only difference from an end user perspecive is that bad input errors that
used to be -1 will now be -32602 (JSONRPC2_INVALID_PARAMS).
2018-05-26 12:17:36 +02:00
Rusty Russell 323472225c channeld: simplify announce/locked-in callback,
Just have a "new depth" callback, and let channeld do the right thing.

This makes the channeld paths a bit more straightforward.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 1125682ceb wireaddr: new type, ADDR_INTERNAL_FORPROXY, use it if we can't/wont resolve.
Tor wasn't actually working for me to connect to anything, but it worked
for 'ssh -D' testing.

Note that the resulting 'netaddr' is a bit weird, but I guess it's honest.

    $ ./cli/lightning-cli connect 021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b
    {
      "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b"
    }
    $ ./cli/lightning-cli listpeers
    {
      "peers": [
        {
          "state": "GOSSIPING", 
          "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b", 
          "netaddr": [
            "ln1qg0je0lugpzu5ttsv78vlrkhteyg9yy8fjw68qr57mfhsfyrxurzkq522ah.lseed.bitcoinstats.com:9735"
          ], 
          "connected": true, 
          "owner": "lightning_gossipd"
        }
      ]
    }

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 5a0bc83b20 Tor: don't do seed queries if we're supposed to always use proxy.
Risks leakage.  We could do lookup via the proxy, but that's a TODO.

There's only one occurance of getaddrinfo (and no gethostbyname), so
we add a flag to the callers.

Note: the use of --always-use-proxy suppresses *all* DNS lookups, even
those from connect commands and the command line.

FIXME: An implicit setting of use_proxy_always is done in gossipd if it
determines that we are announcing nothing but Tor addresses, but that
does *not* suppress 'connect'.

This is fixed in a later patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato 877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 73cd009a4c gossipd/lightningd: use wireaddr_internal.
This replacement is a little menial, but it explicitly catches all
the places where we allow a local socket.  The actual implementation of
opening a AF_UNIX socket is almost hidden in the patch.

The detection of "valid address" is now more complex:

	p->addr.itype != ADDR_INTERNAL_WIREADDR || p->addr.u.wireaddr.type != ADDR_TYPE_PADDING

But most places we do this, we should audit: I'm pretty sure we can't
get an invalid address any more from gossipd (they may be in db, but
we should fix that too).

Closes: #1323
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
ZmnSCPxj e588737511 peer_control: Have `close` accept channel IDs also. 2018-05-03 22:47:07 +02:00
Mark Beckwith 7767b68ee9 Removed redundancies in withdraw and fundchannel.
No new functionality, just a continuation of my work toward completing #665.

I removed the common members of `struct withdrawal` and `struct fund_channel`
and placed them in a new `struct wallet_tx`.  Then it was fairly straightforward
to reimplement the existing code in terms of `wallet_tx`.

Since I made some structural changes I wanted to get this approved before I
go any farther.

Added 'all' to fundchannel help message.
2018-05-03 18:20:20 +02:00
Rusty Russell c6af2a8cb2 lightningd: loosen feerate minimum.
We're getting spurious closures, even on mainnet.  Using --ignore-fee-limits
is dangerous; it's slightly less so to lower the minimum (which is the
usual cause of problems).

So let's halve it, but beware the floor.

This is a workaround, until we get independent feerates in the spec.

Fixes: #613
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-01 18:45:04 +02:00
Rusty Russell 0f0c045aca wallet: fix statement leaks.
We do this before enabling leak checking, but of course I wrote the
commits in the other order!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Rusty Russell ae17c64c4a db: full location tags for callers, make it implicit.
For better leak tracking.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Rusty Russell b036948219 db: track open sqlite3_stmt in DEVELOPER mode.
I would have liked to make it a tal object, then we'd catch most
things with our memleak detection.  However, sqlite3 doesn't seem to
allow allocator overrides.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
practicalswift abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
Rusty Russell aa71a822b3 wallet: free sql statement in wallet_onchaind_channels.
This is the cause of 'sqlite3_close: unable to close due to unfinalized statements or unfinished backups' with the --daemon option.

Fixes: #1420
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 09:59:13 +02:00
Rusty Russell be1f33b265 gossipd: have master explicitly tell us when peer is disconnected.
Currently we intuit it from the fd being closed, but that may happen out
of order with when the master thinks it's dead.

So now if the gossip fd closes we just ignore it, and we'll get a
notification from the master when the peer is disconnected.

The notification is slightly ugly in that we have to disable it for
a channel when we manually hand the channel back to gossipd.

Note: as stands, this is racy with reconnects.  See the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell ab9d9ef3b8 gossipd: drain fd instead of passing around gossip index.
(This was sitting in my gossip-enchancement patch queue, but it simplifies
this set too, so I moved it here).

In 94711969f we added an explicit gossip_index so when gossipd gets
peers back from other daemons, it knows what gossip it has sent (since
gossipd can send gossip after the other daemon is already complete).

This solution is insufficient for the more general case where gossipd
wants to send other messages reliably, so replace it with the other
solution: have gossipd drain the "gossip fd" which the daemon returns.

This turns out to be quite simple, and is probably how I should have
done it originally :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell 72c459dd6c gossipd: keep reaching struct only when we're actively connecting, and don't retry
1. Lifetime of 'struct reaching' now only while we're actively doing connect.
2. Always free after a single attempt: if it's an important peer, retry
   on a timer.
3. Have a single response message to master, rather than relying on
   peer_connected on success and other msgs on failure.
4. If we are actively connecting and we get another command for the same
   id, just increment the counter

The result is much simpler in the master daemon, and much nicer for
reconnection: if they say to connect they get an immediate response,
rather than waiting for 10 retries.  Even if it's an important peer,
it fires off another reconnect attempt, unless it's actively
connecting now.

This removes exponential backoff: that's restored in next patch.  It
also doesn't handle multiple addresses for a single peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell a1f77cab3c lightningd: tell gossipd that peers we load from db are important.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell 8c2c1fe1c2 openingd: tell gossipd that the peer is important once funding tx in place.
And on channel_fail_permanent and closing (the two places we drop to
chain), we tell gossipd it's no longer important.

Fixes: #1316
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
ZmnSCPxj 079778e357 invoice: Check duplicate preimage when explicitly sprcified.
Reported-by: @mcudev
2018-04-26 05:47:09 +00:00
Christian Decker 89ff46f1e6 db: Added DB migrations to get the correct sync height
The no-rescan change requires us to rescan one last time from the first_blocknum
of our channels (if we have any). The migrations just drop blocks that are
higher, then insert a dummy with the first_blocknum, and then clean up after
us. If we don't have any channels we don't go back at all.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 96352858d6 chaintopology: Simplify rescan offset computation
Simplification of the offset calculation to use the rescan parameter, and rename
of `wallet_first_blocknum`. We now use either relative rescan from our last
known location, or absolute if a negative rescan was given. It's all handled in
a single location (except the case in which the blockcount is below our
precomputed offset), so this should reduce surprises.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 7406a5b614 wallet: Report current blockheight as the offset to continue from
This is a big simplification, we just report the DBs current blockchain height
as the point to continue scanning, or the passed in default. No more guessing
where to continue from or whether the wallet was used and when it first saw the
light of day.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 5e505e9c53 onchaind: Add a level of indirection to txwatches and txowatches
This will allow us in the next commit to store the transactions that triggered
this event in the DB and thus allowing us to replay them later on.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 876d698f3c wallet: Add primitives to store onchaind transactions in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker d2dc93e3cb wallet: Add a struct to represent an onchaind transaction
This will be used to replay transactions that were witnessed in the blockchain
during startup, so that onchaind can be recreate its state.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 28feb2eb7d db: Add table for onchaind transactions
These transactions being seen on the blockchain triggered some action in
onchaind so we need to replay them when we restore the onchaind.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Rusty Russell d2b4e09e27 lightningd: re-allow closing negotiation when CLOSINGD_COMPLETE
d822ba1ee accidentally removed this case, which is important: if the
other side didn't get our final matching closing_signed, it will
reconnect and try again.  We consider the channel no longer "active"
and thus ignore it, and get upset when it send the
`channel_reestablish` message.

We could just consider CLOSINGD_COMPLETE to be active, but then we'd
have to wait for the closing transaction to be mined before we'd allow
another connection.

We can't special case it when the peer reconnects, because there
could be (in theory) multiple channels for that peer in CLOSINGD_COMPLETE,
and we don't know which one to reestablish.

So, we need to catch this when they send the reestablish, and hand
that msg to closingd to do negotiation again.  We already have code
to note that we're in CLOSINGD_COMPLETE and thus ignore any result
it gives us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell 8e976150ad json_fundchannel: fix release vs connect/nongossip race.
The new connect code revealed an existing race: we tell gossipd to
release the peer, but at the same time it connects in.  gossipd fails
the release because the peer is remote, and json_fundchannel fails.

Instead, we catch this race when we get peer_connected() and we were
trying to open a channel.  It means keeping a list of fundchannels which
are awaiting a gossipd response though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell bee795ed68 channeld: don't do explicit state update.
We missed it in some corner cases where we crashed/were killed between
being told of the lockin and sending the channel_normal_operation message.
When we were restarted, we were told both sides were locked in already,
so we never updated the state.

Pull the entire "tell channeld" logic into channel_control.c, and make
it clear that we need to keep waching if we cant't tell channeld.  I think
we did get this correct in practice, since funding_announce_cb has the
same test, but it's better to be clear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
ZmnSCPxj 2cee1ab20f peer_control: Make close wait for complete closure, with timeout.
Also report tx and txid, and whether we closed unilaterally or
bilaterally, if we could close the channel.

Also make a manpage.

Fixes: #1207
Fixes: #714
Fixes: #622
2018-04-23 05:24:46 +00:00
ZmnSCPxj 990ef729b0 walletrpc: Refactor out encoding of pubkey to address. 2018-04-22 12:51:03 +02:00
ZmnSCPxj 3bf626a750 walletrpc: Correct generation of P2SH-P2WPKH addresses for listfunds. 2018-04-22 12:51:03 +02:00
conanoc 0733770559 Adjust indents 2018-04-21 15:55:00 +02:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ 11ca729d85 wallet, payalgo: Save detail of payment failures for later reporting. (#1345)
Pointless for remote failures as those are never sent by
the erring node, but for local failures we can give more
detail.
2018-04-16 15:29:40 +02:00
Christian Decker f27cd3e43f topo: Remove in-memory txs from the block struct
The only use for these was to compute their txids so we could notify depth
in case of reorgs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 23984ecde4 chaintopology: Use the DB to locate transactions and rebroadcast txs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker aa696370af txwatch: Switch to passing only txid into the depth callbacks
All of the callback functions were only using the tx to generate the txid again,
so we just pass that in directly and save passing the tx itself.

This is a simplification to move to the DB backed depth callbacks. It'd be
rather wasteful to read the rawtx and deserialize just to serialize right away
again to find the txid, when we already searched the DB for exactly that txid.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 85fbab2fab wallet: Add function to retrieve a watched transaction's blockheight
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 633ffeec6f wallet: Add methods to add transactions to the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker 0f89653ce8 db: Add table for transactions we are interested in
Currently these are either transactions we sent ourselves or transactions that
we are watching because they are part of a channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker ba3ceb2abf wallet: Lowerbound the rescan by going at most back to LNs origin
Repeated crashes could result in the `last_processed_block` variable being
pushed further and further into the past (in some cases going as far back as
scanning blocks from 2012...). This is a stop-gap solution that just lower
bounds the value to what is the first possible block we might be interested in
LN, until we have the 0-rescan fix I'm working on.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker ffbe56847e db: Fixing some clang warnings
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
Christian Decker 9ab28d1659 topology: Two off-by-one error when catching up with the blockchain
There are two very hard problems in software engineering:

 1. Off-by-one errors

In this case we were rolling back further than needed and we were starting the
catchup one block further than expected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:55:41 +00:00
Christian Decker 992d395b42 Update walletrpc.c 2018-04-05 19:37:26 +02:00
Zhen Zhang 9423896092 Add funding_address to listfunds RPC response, fixes #1227 2018-04-05 19:37:26 +02:00
ZmnSCPxj f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
Rusty Russell 19d5305658 wallet/test: fix Makefile so test correctly depend on wallet files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 6bb47276ce lightningd: put min/max feerates into db, struct channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 8975fc2ceb libwally: use their secp context for all our daemons.
I didn't convert all tests: they can still use a standalone context.
It's just marginally more efficient to share the libwally one for all
our daemons which link against it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Christian Decker 1133820ad6 db: Add a cleanup migration to clear any dangling utxoset entries
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker eb531e6da7 db: Make sure to enable foreign keys even after forking
This may be causing #1280, since with `--daemon` the DB is being reopened
without enabling the foreign key relations and hence the delete cascades.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker a8d587c418 wallet: Return any eventual outpoint scid when marking it spent
Just return the short_channel_id matching the outpoint that we just marked as
spent.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker ba7341ec87 cleanup: Make blockheights unsigned
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
Christian Decker 286cfefc62 wallet: Remove pruned outpoints from utxoset outpoint_filter
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 17:14:58 +02:00
practicalswift 6269a4c55d Remove unused functions not covered by unit tests 2018-03-26 23:35:56 +00:00
ZmnSCPxj b914062465 pay: Also record how much we actually ended up sending. 2018-03-26 05:01:49 +00:00
practicalswift 7e9750ffee Reduce variable scopes 2018-03-26 01:31:21 +00:00
ZmnSCPxj 0bb9bcc0f1 wallet: Track some channel usage statistics.
Fixes: #1049
2018-03-26 01:08:52 +00:00
Rusty Russell 76e8a11380 wallet: use json_escaped for invoice label.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell d92579f627 common/json_escaped: new type which explicitly notes a string is already JSON.
Trivial to use as a string, but it still means you should be careful
around it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift 98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift 0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
ZmnSCPxj d5d31864cf wallet: Remove onion-decoding information from db on payment success/fail.
Fixes: #1177
2018-03-21 13:56:18 +01:00
Mark Beckwith 489d8d04a6 Fund all (#1249)
* Funding channel with "all"
* Added test for fund all
2018-03-21 00:06:04 +01:00
ZmnSCPxj 159e736fc1 invoices: Implement autocleaninvoice. 2018-03-20 17:25:51 +01:00
ZmnSCPxj 3b0081aebb invoices: Add delexpiredinvoice command. 2018-03-20 17:25:51 +01:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ 044705a547 peer_control: Fix a use-after-free bug. (#1237)
This bug is a classic case of being lazy:
1. peer_accept_channel() allocated its return off the input message,
   rather than taking an explicit allocation context.  This concealed the
   lifetime nature of the return.
2. The context for sanitize_error was the error itself, rather than the
   more obvious tmpctx (connect_failed does not take).

The global tmpctx removes the "efficiency" excuse for grabbing a random
object to use as context, and is also nice and explicit.

All-the-hard-work-by: @ZmnSCPxj
2018-03-19 09:27:09 +00:00
ZmnSCPxj 9efe123a0d lightningd/json: Move json helpers specific for lightningd to new module. 2018-03-19 00:27:55 +00:00
Rusty Russell 1b9791f0f5 pay: delete HTLC when we delete payment.
This fixes the root cause of https://github.com/ElementsProject/lightning/issues/1212
where we deleted the payment because we wanted to retry, then retry failed
so we had an (old) HTLC without a matching payment.  We then fed that
HTLC to onchaind, which tells us it's missing, and we try to fail the
payment and deref a NULL pointer.

Fixes: #1212
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 04:01:06 +00:00
practicalswift 03e2e404f2 Mark unused parameters as such (using "UNUSED") 2018-03-16 01:17:40 +00:00
practicalswift 9970419b3a Remove redundant return from void function 2018-03-16 01:17:40 +00:00