Commit Graph

4490 Commits

Author SHA1 Message Date
Rusty Russell 6aed936799 channeld: check option_data_loss_protect fields.
Firstly, if they claim to know a future value, we ask the HSM; if
they're right, we tell master what the per-commitment-secret it gave
us (we have no way to validate this, though) and it will not broadcast
a unilateral (knowing it will cause them to use a penalty tx!).

Otherwise, we check the results they sent were valid.  The spec says
to do this (and close the channel if it's wrong!), because otherwise they
could continually lie and give us a bad per-commitment-secret when we
actually need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell e7116284f0 channeld: move reestablish retransmission below checks.
This makes it a bit clearer, but also means we do all checks before
sending any packets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 43156643b4 lightningd: message for channeld to tell us that channel risks penalty.
For option_data_loss_protect, the peer can prove to us that it's ahead;
it gives us the (hopefully honest!) per_commitment_point it will use,
and we make sure we don't broadcast the commitment transaction we have.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 18297a173c hsm: routine to validate that a future secret is correct.
Currently it works for any secret (we don't know the current secret),
but importantly it doesn't leak timing information when checking.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 7f37fa3263 derive_basepoints: harden checking.
I managed to crash the HSM by asking for point -1 (shachain_index has an
assert).  Fail in this case, instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 8340d8c070 secret_eq: remove in favor of constant time variant.
To be safe, we should never memcmp secrets.  We don't do this
currently outside tests, but we're about to.

The tests to prove this as constant time are the tricky bit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell da8d620907 pytest: check that we advertise and send option_data_loss_protect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 4c891f4661 common: log when we toggle IO logging, don't edit env in tests!
Tests were failing when in the same thread after a test which set
log_all_io=True, because SIGUSR1 seemed to be turning logging *off*.

This is due to Python using references not copies for assignment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 9f044305db pytest: dev env var LIGHTNINGD_DEV_LOG_IO turns io logging on immediately.
This is required for the next test, which has to log messages from channeld
as soon as it starts (so might be too late if it sends SIGUSR1).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell ebaf5eaf2e channeld: send option_data_loss_protect fields.
We ignore incoming for now, but this means we advertize the option and
we send the required fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 61c6b8b25a tools/generate-wire.py: mark your_last_per_commitment_secret as a struct secret.
Without an override, the tool assumes it's a sha256.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell b123b1867d shachain: shachain_get_secret helper.
This is a wrapper around shachain_get_hash, which converts the
commit_num to an index and returns a 'struct secret' rather than a
'struct sha256' (which is really an internal detail).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 692bae7873 channeld: create get_per_commitment_point helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 162adfdf12 listpeers: correctly display features on reconnect.
peer features are only kept for connected peers (as they can change),
but we didn't update them on reconnect.  The main effect was that
after a restart we displayed the features as empty, even after
reconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell f5143d9549 pytest: fail if we see 'bad reestablish' in the logs.
Really, we should use log-level more cleverly here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 83eadb3548 gossipd: fix SUPERVERBOSE usage, enhance, when turned on.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 28977435a3 channeld: fix incorrect comment on reestablish.
We quote BOLT 2 on *local* above the *remote* checks (we quote it
again below when we do the local checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 2a73450818 connectd: fix leak of peer.
We no longer need to keep 'struct peer' around: we free it as soon as
we hand off to the master daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Christian Decker 8f56d64a1f log: Append the current time to the crash log filename
This should make it easier to identify the latest crash file and correlate
crashes with external monitoring tools.
2018-08-23 12:51:08 +02:00
Rusty Russell 00696277d2 logging: always dump a crash log, but make files per-pid.
Someone had a 21GB crash.log, which doesn't help anyone!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 12:51:08 +02:00
Rusty Russell 213be90e77 log: implement reopening log-file on SIGHUP
Closes: #1623
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 12:51:08 +02:00
Rusty Russell 9f175deecd lightningd: update feerate upon receiving revoke_and_ack from fundee.
1. l1     update_fee ->    l2
2. l1 commitment_signed -> l2 (using new feerate)
3. l1  <- revoke_and_ack   l2
4. l1 <- commitment_signed l2 (using new feerate)
5. l1  -> revoke_and_ack   l2

When we break the connection after #3, the reconnection causes #4 to
be retransmitted, but it turns out l1 wasn't telling the master to set
the local feerate until it received the commitment_signed, so on
reconnect it uses the old feerate, with predictable results (bad
signature).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 18:54:53 +02:00
Rusty Russell c106fa1b4f pytest: add test for reconnect immediately after feerate change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 18:54:53 +02:00
Rusty Russell cda6c97a57 pytest: allow NodeFactory to create non-started peers.
Useful it we want to intercept bitcoin-cli first.

We move the getinfo() caching into start(), as that's when we can actually
use RPC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell 7eed9cba90 pytest: always override estimatesmartfee.
It (almost?) always fails for regtest; best to override it directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell 7a77b81dff pytest: always use the fake-bitcoin-cli, name it more appropriately.
We're going to use it to override specific commands.  It's non-valgrinded
already since we use '--trace-children-skip=*bitcoin-cli*' so the overhead
should be minimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell 5e20eedb41 pytest: allow more elaborate bitcoin-cli override.
In particular, this lets us intercept individual commands, such as
estimatesmartfee.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell 3d8836c1e5 bitcoind: don't use double in extracting feerate.
It introduces imprecision (took 1 satoshi off results in the coming
tests), and we have a helper for this already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell 2314a4aa5d contrib/short_channel_id-to-txid.sh: simple mapping util.
I use this to look up on smartbit.com.au to see if a channel is spent,
for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 22:53:45 +02:00
Christian Decker c2aa4e51dd docker: Make the bitcoind version configurable at build time
This was suggested via mail: the SHA256 sums should be extracted from the
sha256sums file we are checking against, which also allows us to switch bitcoind
version at build time.

Suggested-by: Giles Hall <@vishnubob>
Signed-off-by: Christian Decker <@cdecker>
2018-08-21 22:03:43 +02:00
Rusty Russell ab0fa7a1bd chaintopology: always cap max block to bitcoind's block height.
We only did this when we were first creating a wallet, or when we
asked for a relative rescan, not in the normal case!

Fixes: #1843
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 01:00:37 +02:00
Rusty Russell 74521b3fb7 gossipd: don't delay the very first channel_update.
Lightning charge tests stopped working without a timeout, being unable
to find a route.  The 15 second delay doesn't matter in real life, but
in these scenarios it does.  This fixes it by making sure the channel
is usable immediately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:49:12 +02:00
Rusty Russell bc7ca34a38 pytest: fix race in test_closing_different_fees
We got an index error, because status had only one field (onchaind not
started yet).

    >   wait_for(lambda: only_one(p.rpc.listpeers(l1.info['id'])['peers'][0]['channels'])['status'][1] == 'ONCHAIN:Tracking mutual close transaction')
    E   IndexError: list index out of range1

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell f8052a6c1a chaintopology: watch UTXOs which need closeinfo when we remove blocks.
Normal wallet txs get reconfirmed as blocks come in, but ones which need
closeinfo are more fragile, so we do it manually using txwatch for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 05f12edf60 txwatch: hand ld to callback, don't assume channel is non-NULL.
We're about to use the txwatch facility for UTXOs, where there's no channel,
so allow that the be NULL, and hand the struct lightningd which callers
want anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 5d23698665 wallet: expose function to confirm a tx.
Note that we don't actually need the output number: it's the tx itself
which is confirmed.  And the next caller doesn't have it convenient, so
eliminate it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell 27835df8fd wallet: add accessor getting close-info-needed unconfirmed UTXOs.
These are not confirmed by the normal methods (wallet_can_spend is false!),
so we'll deal with them manually.

We use UTXO_FIELDS in wallet_add_utxo, too, for consistency.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Rusty Russell d280eabc1e pytest: add a resart to test_permfail that we don't forget on restart.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-21 00:48:02 +02:00
Mark Beckwith a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith fa55e2cab0 param: upgraded json_tok_loglevel
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 1fb203891d param: upgraded json_tok_short_channel_id
Made previous version a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 9b28ecf8fc param: upgraded json_tok_pubkey
Also add json_to_pubkey as utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 6254d15efd param: upgraded json_tok_sha256
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 2a0c2601c0 param: upgraded json_tok_newaddr
Made it a local static since its a one-off.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 947752b9cc param: upgraded json_tok_percent
Made it a local static since its only used once.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 47555efc0d param: rebased master
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith e5918f4e5a param: upgraded json_tok_double
Also renamed old version to json_to_double for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00