Commit Graph

1513 Commits

Author SHA1 Message Date
Rusty Russell d93be58bd0 pytest: remove use dev-override-feerates.
Manipulate fees via fake-bitcoin-cli.  It's not quite the same, as
these are pre-smoothing, so we need a restart to override that where
we really need an exact change.  Or we can wait until it reaches a
certain value in cases we don't care about exact amounts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell a75de62477 chaintopology: always initialize smoothed feerate if it's the first entry.
Not just during startup: we could have bitcoind not give estimates until
later, but we don't want to smooth with zero.

The test changes in next patch trigger this, so I didn't write a test
with this patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell 807e62b05d moveonly: move feerate routines from peer_htlcs.c to channel_control.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell 607d4bf9d2 channel: update fees after lockin.
We don't respond to fee changes until we're locked in: make sure we catch
up at that point.

Note that we use NORMAL fees during opening, but IMMEDIATE after, so
this often sends a fee update.  The tests which break, we set those
feerates to be equal.

This (sometimes) changes the behavior of test_permfail, as we now
get an immediate commit, so that is fixed too so we always wait for
that to complete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell 6338ae8a44 channeld: update fees if we're restarting.
This is a noop if we're opening a new channel (channel_fees_can_change(channel)
is false until funding locked in), but important if we're restarting.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell 43fe7f034e chaintopology: try to get a feerate estimate before we complete startup.
It may fail, but it's better than having a window where we're using
the default feerate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell b6a63024c1 lightningd: fix double-free on multiple HTLC timeouts.
We can close a connection with a peer to timeout an HTLC, but
we need to clear the pointer otherwise next time we try, we'll
free an expired pointer:

```
lightningd: Fatal signal 6 (version v0.6-336-gfcd1eb5-modded)
0x13ce86 crashdump
	common/daemon.c:37
0x5739f1f ???
	???:0
0x5739e97 ???
	???:0
0x573b800 ???
	???:0
0x1850c3 call_error
	ccan/ccan/tal/tal.c:93
0x18528b check_bounds
	ccan/ccan/tal/tal.c:165
0x1852ca to_tal_hdr
	ccan/ccan/tal/tal.c:174
0x185bfb tal_free
	ccan/ccan/tal/tal.c:472
0x1343a8 peer_sending_commitsig
	lightningd/peer_htlcs.c:1035
0x114f25 channel_msg
	lightningd/channel_control.c:159
0x13756b sd_msg_read
	lightningd/subd.c:474
0x177c1f next_plan
	ccan/ccan/io/io.c:59
0x178717 do_plan
	ccan/ccan/io/io.c:387
0x178755 io_ready
	ccan/ccan/io/io.c:397
0x17a336 io_loop
	ccan/ccan/io/poll.c:310
0x120589 main
	lightningd/lightningd.c:455
0x571cb96 ???
	???:0
0x10e6d9 ???
	???:0
0xffffffffffffffff ???
	???:0
2018-08-16T06:41:21.249Z lightningd(869): FATAL SIGNAL 6 (version v0.6-336-gfcd1eb5-modded)
2018-08-16T06:41:21.250Z lightningd(869): backtrace: common/daemon.c:42 (crashdump) 0x13ceda
2018-08-16T06:41:21.250Z lightningd(869): backtrace: (null):0 ((null)) 0x5739f1f
2018-08-16T06:41:21.250Z lightningd(869): backtrace: (null):0 ((null)) 0x5739e97
2018-08-16T06:41:21.251Z lightningd(869): backtrace: (null):0 ((null)) 0x573b800
2018-08-16T06:41:21.251Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x1850c3
2018-08-16T06:41:21.251Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x18528b
2018-08-16T06:41:21.252Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x1852ca
2018-08-16T06:41:21.252Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:472 (tal_free) 0x185bfb
2018-08-16T06:41:21.252Z lightningd(869): backtrace: lightningd/peer_htlcs.c:1035 (peer_sending_commitsig) 0x1343a8
2018-08-16T06:41:21.252Z lightningd(869): backtrace: lightningd/channel_control.c:159 (channel_msg) 0x114f25
2018-08-16T06:41:21.253Z lightningd(869): backtrace: lightningd/subd.c:474 (sd_msg_read) 0x13756b
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x177c1f
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:387 (do_plan) 0x178717
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:397 (io_ready) 0x178755
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/poll.c:310 (io_loop) 0x17a336
2018-08-16T06:41:21.253Z lightningd(869): backtrace: lightningd/lightningd.c:455 (main) 0x120589
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0x571cb96
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0x10e6d9
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0xffffffffffffffff
Log dumped in crash.log
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 16:55:15 +02:00
Rusty Russell 36b1cac6e6 lightningd: new state AWAITING_UNILATERAL.
When in this state, we send a canned error "Awaiting unilateral close".
We enter this both when we drop to chain, and when we're trying to get
them to drop to chain due to option_data_loss_protect.

As this state (unlike channel errors) is saved to the database, it means
we will *never* talk to a peer again in this state, so they can't
confuse us.

Since we set this state in channel_fail_permanent() (which is the only
place we call drop_to_chain for a unilateral close), we don't need to
save to the db: channel_set_state() does that for us.

This state change has a subtle effect: we return WIRE_UNKNOWN_NEXT_PEER
instead of WIRE_TEMPORARY_CHANNEL_FAILURE as soon as we get a failure
with a peer.  To provoke a temporary failure in test_pay_disconnect we
take the node offline.

Reported-by: Christian Decker @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell a5ecc95c42 db: store claimed per_commitment_point from option_data_loss_protect.
This means we don't try to unilaterally close after a restart, *and*
we can tell onchaind to try to use the point to recover funds when the
peer unilaterally closes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 1a4084442b onchaind: use a point-of-last-resort if we see an unknown transaction.
This may have been supplied by the peer if it's nice and supports
option_data_loss_protect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 43156643b4 lightningd: message for channeld to tell us that channel risks penalty.
For option_data_loss_protect, the peer can prove to us that it's ahead;
it gives us the (hopefully honest!) per_commitment_point it will use,
and we make sure we don't broadcast the commitment transaction we have.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell ebaf5eaf2e channeld: send option_data_loss_protect fields.
We ignore incoming for now, but this means we advertize the option and
we send the required fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 162adfdf12 listpeers: correctly display features on reconnect.
peer features are only kept for connected peers (as they can change),
but we didn't update them on reconnect.  The main effect was that
after a restart we displayed the features as empty, even after
reconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Christian Decker 8f56d64a1f log: Append the current time to the crash log filename
This should make it easier to identify the latest crash file and correlate
crashes with external monitoring tools.
2018-08-23 12:51:08 +02:00
Rusty Russell 00696277d2 logging: always dump a crash log, but make files per-pid.
Someone had a 21GB crash.log, which doesn't help anyone!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 12:51:08 +02:00
Rusty Russell 213be90e77 log: implement reopening log-file on SIGHUP
Closes: #1623
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 12:51:08 +02:00
Rusty Russell 9f175deecd lightningd: update feerate upon receiving revoke_and_ack from fundee.
1. l1     update_fee ->    l2
2. l1 commitment_signed -> l2 (using new feerate)
3. l1  <- revoke_and_ack   l2
4. l1 <- commitment_signed l2 (using new feerate)
5. l1  -> revoke_and_ack   l2

When we break the connection after #3, the reconnection causes #4 to
be retransmitted, but it turns out l1 wasn't telling the master to set
the local feerate until it received the commitment_signed, so on
reconnect it uses the old feerate, with predictable results (bad
signature).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 18:54:53 +02:00
Rusty Russell 3d8836c1e5 bitcoind: don't use double in extracting feerate.
It introduces imprecision (took 1 satoshi off results in the coming
tests), and we have a helper for this already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell ab0fa7a1bd chaintopology: always cap max block to bitcoind's block height.
We only did this when we were first creating a wallet, or when we
asked for a relative rescan, not in the normal case!

Fixes: #1843
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 01:00:37 +02:00
Rusty Russell f8052a6c1a chaintopology: watch UTXOs which need closeinfo when we remove blocks.
Normal wallet txs get reconfirmed as blocks come in, but ones which need
closeinfo are more fragile, so we do it manually using txwatch for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 05f12edf60 txwatch: hand ld to callback, don't assume channel is non-NULL.
We're about to use the txwatch facility for UTXOs, where there's no channel,
so allow that the be NULL, and hand the struct lightningd which callers
want anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Mark Beckwith a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith fa55e2cab0 param: upgraded json_tok_loglevel
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 1fb203891d param: upgraded json_tok_short_channel_id
Made previous version a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 9b28ecf8fc param: upgraded json_tok_pubkey
Also add json_to_pubkey as utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 6254d15efd param: upgraded json_tok_sha256
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 2a0c2601c0 param: upgraded json_tok_newaddr
Made it a local static since its a one-off.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 947752b9cc param: upgraded json_tok_percent
Made it a local static since its only used once.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 47555efc0d param: rebased master
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith e5918f4e5a param: upgraded json_tok_double
Also renamed old version to json_to_double for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Simon Vrouwe 795e0e1b21 do not start fee estimation loop with option: --dev-override-fee-rates 2018-08-14 06:52:57 +00:00
Simon Vrouwe d309a7d1b9 in ./lightningd --help, tell for which network the default values
are shown
2018-08-14 00:07:10 +00:00
Mark Beckwith 8f17191099 param: upgraded json_tok_tok to advanced callback
This was a very simple change and allowed us to remove the special
`json_opt_tok` macro.

Moved the callback out of `common/json.c` to `lightningd/json.c` because the new
callbacks are dependent on `struct command` etc.
(I already started on `json_tok_number`)

My plan is to:
	1. upgrade json_tok_X one a time, maybe a PR for each one.
	2. When done, rename macros (i.e, remove "_tal").
	3. Remove all vestiges of the old callbacks
	4. Add new callbacks so that we no longer need json_tok_tok!
	   (e.g., json_tok_label, json_tok_str, json_tok_msat)

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-13 23:46:35 +00:00
Rusty Russell 65c882ca3a Minor cleanups.
1. connect convenience variable for improved readabilty.
2. a comment explaining that timer is on channel, not HTLC.
3. use modern python style in test_htlc_send_timeout

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell 223cd97c94 lightningd: kill channeld if we added an HTLC and it didn't commit in 30 seconds.
This effectively constrains how long we'll delay an outgoing HTLC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell 71575b2115 ping: no longer a dev_ command.
Fixes: #1407
Suggested-by: conanoc@gmail.com
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell be7a27a765 connect: randomize backoff a little.
Since we now fixed the bug where nodes receiving a connection would
try to reconnect to the source IP/port of that connection, we now expose
an issue mentioned by other implementers: we can continually cross over
reconnections unless we add some fuzz.  One second should be sufficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell 74568a1c50 lightningd: peer_start_channeld always returns true; make it void.
It is always true, and we always ignore it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Mark Beckwith bd5bf1f168 Enhanced param parsing
[ Squashed into single commit --RR ]

This adds two new macros, `p_req_tal()` and `p_opt_tal()`. These support
callbacks that take a `struct command *` context.  Example:

	static bool json_tok_label_x(struct command *cmd,
                                      const char *name,
				      const char *buffer,
				      const jsmntok_t *tok,
				      struct json_escaped **label)

The above is taken from the run-param unit test (near the bottom of the diff).
The return value is true on success, or false (and it calls command_fail itself).

We can pretty much remove all remaining usage of `json_tok_tok` in the codebase
with this type of callback.
2018-08-10 02:15:30 +00:00
Rusty Russell b4e6a0fcad peer_failed: write error message to peer directly.
We currently hand the error back to the master, who then stores it for
future connections and hands it back to another openingd to send and exit.

Just send directly; it's more reliable and simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell d8d4b19f3a connectd: remove separate address hint message.
Include it as an optional field in the connect_to_peer message (it was
added before we had optional fields).

The only issue is that reconnects want it too, so again connectd hands
it back to master in connectctl_connect_failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 8939a5001b connectd: rely on the master to tell us to reconnect.
connectd tells master about every disconnection, and master knows
whether it's important to reconnect.  Just get the master to invoke a new
connect command if it considers the peer important!

The only twist is timeouts: we don't want to immediately reconnect if
we've failed to connect.  To solve this, connectd passes a 'delaytime'
to the master when a connection fails, and the master passes it back
when it asks for a connection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 30f08cc2b0 connectd: always tell master when connection fails/succeeded.
We used to separate implicit connection requests (ie. timed retries
for important peers) and explicit ones, and send a
WIRE_CONNECTCTL_CONNECT_TO_PEER_RESULT for the latter.

In the success case, that's now redundant, since we hand the connected
peer to the master using WIRE_CONNECT_PEER_CONNECTED; we just need a
message for the failure case.  And we might as well tell the master
every failure, so we don't have to distinguish internally.

This also solves a race we had before: connectd would send
WIRE_CONNECTCTL_CONNECT_TO_PEER_RESULT which completes the incoming
JSON connect command, then send WIRE_CONNECT_PEER_CONNECTED.  So
there's a window where the JSON command can return, but the peer isn't
known to lightningd yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 684d60dbda lightningd: don't call connectd if we already know about peer.
The semantic here are that we 'succeed' if we're already connected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 8b5c80f42a opening_control.c: make sure we always clean up in error cases.
Especially by closing the file descriptors we were handed!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 174c79acad openingd: tell master if funding failed, but don't exit.
We don't want to exit just because channel parameter negotiation
failed, but we do want to tell the master if it was a channel we were
trying to fund.

Note that lightningd still needs to fail the funding cmd if it gets a
fromwire_opening_fundee (they raced us and won), or an outright
failure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 909cd4136b openingd: get told if we can't let them open a new channel.
Previously master would fail once the channel has been negotiated,
which is terrible, since the funder will have already broadcast tx.

Now we tell them if we have an active channel, and update if it goes away.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00