Commit Graph

2279 Commits

Author SHA1 Message Date
Christian Decker 48796f4f39 cli: Add --no-reconnect cli flag
Especially when testing we might want to disable the automatic
reconnection logic in order not to masquerade bugs that disappear when
reconnecting.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Rusty Russell 243fc2c05a utils: save log file on failure.
This broke somewhere in the recent changes, because we override
TailalbleProc stop().  Break out log extractor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-03 12:56:08 +02:00
Christian Decker f35e29607d wire: Fix include order in wire_io.h
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 14:40:34 +02:00
Rusty Russell 29b83aed2a wire: restore BE endian to wire headers for internal messages.
We don't anticipate daemons across machines, but you never know.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell 3d316518fd wire: use 26-bit lengths for inter-daemon messaging.
Fixes: #289
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell 32631b4278 generate-wire.py: add --bolt arg, use size->type hacks only when that's specified.
For our own internal comms CSVs, we should always name explicit types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell 1b31d1c5a0 bitcoind: handle up to 60 seconds of outage.
Seems to go out to lunch on reorgs:

   +136792.168286138 lightningd(9465):BROKEN: bitcoin-cli getchaintips exited 28: 'error code: -28
   error message:
   Rewinding blocks...

Closes: #286
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell a2d4e09b80 bitcoind: wait for startup.
We don't hit this in testing, since we wait for startup already.  Hacking
tests to avoid that, I tested this code by hand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell 66c1da795f lightningd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 8bb20d127d channeld: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell c850e3c0a4 gossipd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 1398a208f8 hsmd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 3404c393b7 common/io_debug: replacement for ccan/io's poll which does sanity checks.
For now we just check for outstanding take() or tal_tmpctx().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell e587ec3bd3 tal_tmpctx: keep information around so we can find leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Christian Decker 5029f95066 pytest: Use futures to wait for failing sendpays
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker 0b350d2f5f pytest: Integrate with known/allowed failures
Moved the flagging for allowed failures into the factory getter, and
renamed into `may_fail`. Also stopped the teardown of a node from
throwing an exception if we are allowed to exit non-cleanly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker c1f4c86589 pytest: Make sure to clean up all lightningds after failures
A failed returncode check could result in the cleanup for other
lightningds to be skipped. Now make sure to cleanup all and then
rethrow an exception that contains all returncodes.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker 11eaabdbe6 pytest: Stopping daemon cleanly
We used to simply kill the daemon, which in some cases could result in
half-written crashlogs and similar artifacts such as half-completed
RPC calls. Now we ask lightningd to stop nicely, give it some time and
only then kill it. We also return the returncode of the daemon.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell 3f9ec6c2fa jsonrpc: use-after-free bug due to unspecified free behavior 2/2
Using pc after free in the pay_command_destroyed destructor, so
we just steal cmd onto pc so free order is the one we want.

[ Edit: expanded comment, split commit ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker eac770760c jsonrpc: Make sure we handle the case where jcon outlives command
So far only happens during normal shutdown, but it may happen in other
cases as well. We simply define a new destructor that unregisters the
`cmd` from the `jcon`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell 3564263e12 jsonrpc: use-after-free bug due to unspecified free behavior 1/2
These were fun to hunt down. The jcon and the conn are allocated off
of ld, so the free order is unspecified and if conn is freed before
conn then the finish_jcon destructor uses conn after free.

[ Edit: split commit, modified to use a destructor directly on jcon,
  which is more robust than relying on it only being freed via conn --RR ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell 4275ca5da3 lightningd: don't free log on shutdown until after lightningd freed.
Based-on-patch-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:06:56 +09:30
Rusty Russell 2fe2a0bcf9 peer_control: don't double-free on permanent fail of non-persistent peer.
peer_fail_permanent() frees peer->owner, but for bad_peer() we're
being called by the sd->badpeercb(), which then goes on to
io_close(conn) which is a child of sd.

We need to detach the two for this case, so neither tries to free the
other.

This leads to a corner case when the subd exits after the peer is gone:
subd->peer is NULL, so we have to handle that too.

Fixes: #282
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell b7bb0be944 subd: remove context arg, as we're always owned by lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell 8741043a0e test_lightningd: rename has_failed to allow_failure.
Note that it should really be a flag to daemon on construction, too,
but that may interfere with another concurrent branch so I've deferred.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell f3025ce147 onchaind: simplify reasonable-depth code a little.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell f082c7b80e lightningd: add FIXMEs for future work.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 96a85a786c opening: save the correct commitment tx when we are the fundee.
We were saving *our* commitment tx, not theirs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell d04b7af56e onchain: add initial onchain state.
We have a race where we start onchaind, but state is unchanged, so checks
like peer_control.c's:

	peer_ready = (peer->owner && peer->state == CHANNELD_AWAITING_LOCKIN);
	if (!peer_ready) {
		log_unusual(peer->log,
			    "Funding tx confirmed, but peer state %s %s",
			    peer_state_name(peer->state),
			    peer->owner ? peer->owner->name : "unowned");
	} else {
		subd_send_msg(peer->owner,
			      take(towire_channel_funding_locked(peer,
								 peer->scid)));
	}

Can send to the wrong daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 72b215f6fe Make all internal message numbers unique.
We were sending a channeld message to onchaind, which was v. confusing
due to overlap.  We make all the numbers distinct, which means we can
also add an assert() that it's valid for that daemon, which catches
such errors immediately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell a8c60ed289 towire: remove useless double-invert in towire_bool.
GCC optimizes it out anyway: I sent an uninitialized var and it sent 8!
The receiver checks the value is 0 or 1 anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell aea77653d3 onchaind: update bolt #5, and implement failure of timed-out onchain HTLCs.
We re-use the value for reasonable_depth given by the master, and we
tell it when our timeout transactions reach that depth.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 5666ba551f onchaind: fail htlcs which are not included in commitment tx.
As per update 149cf020d6

Fixes: #249
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 4ce9f73e00 BOLT update to fae35903ae600d61124a3920363094b02d2ae8e5
This details handling of onchain HTLCs, and we document how we
implement it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 1ac9e0b532 onchain: implement penalty transaction.
Fixes: #242
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell d640dc3a6f test_lightningd.py: fix wait_for_logs with duplicate entries.
In the next test, we wait for multiple 'sendrawtx exit 0' which
doesn't work because we use a set not a list, and the current code
would match multiple against the same thing.  The result was we didn't
wait for the final sendrawtransaction, and occasionally had test
failures as a result.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 0e569209ec onchaind: immediately broadcast proposals which don't need to wait.
This removes an extra call and simplifies handling of "immediate" txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell ab8251c214 lightningd: dev-reenable-commit RPC command to re-enable commit timer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell ce160d9b17 lightnind: _ dev-disconnect argument to suppress commit timer.
Required for catching daemon in exact state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 5957dd9303 lightnind: dev-sign-last-tx RPC command for testing penalty txs.
We don't normally let daemons send invalid txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell cbe72b658e onchaind: extract payment_preimage from onchain HTLC redemption.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell f96a04e2a6 onchaind: resolve HTLC transactions.
When we sent out an HTLC-Timeout or HTLC-Success tx, we need to spend
it after the timeout so it's safely in our wallet.

We generalize the tx_type OUR_UNILATERAL_TO_US_RETURN_TO_WALLET to
OUR_DELAYED_RETURN_TO_WALLET, since we use it for HTLC transactions too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell a76a53a933 onchaind: fulfill HTLCs onchain.
When we see an offered HTLC onchain, we need to use the preimage if we
know it.  So we dump all the known HTLC preimages at startup, and send
new ones as we discover them.

This doesn't cover preimages we know because we're the final
recipient; that can happen if an HTLC hasn't been irrevocably
committed yet.  We'll do that in a followup patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 6ef64cd52b script: make "sig_and_empty" more generic, make htlc tx witness fns clearer.
For non-delayed HTLC success spends, we have a similar pattern ("<sig>
<preimage> <wscript>") so a we want to use the same function.

The other routines don't say "witness" in them, and should.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 86ecc6a058 onchaind: move wallet pubkey and private keys into globals.
As we start handling HTLCs, we need access to these from the
"wait_for_resolved" loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 2259faf3a9 onchaind: move feerate grinding into its own function.
We'll want to use this for htlc_success txs as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 561a73328b onchaind: move keyset, feerate and dustlimit into global vars.
As we start handling HTLCs, we need access to these from the
"wait_for_resolved" loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 664ce8bfc2 watch: remove peer_debug helper.
This was part of the transition; we can now just log directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 9123d6d45e type_to_string: support preimage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30