Commit Graph

829 Commits

Author SHA1 Message Date
Rusty Russell d577769350 invoice: provide bolt11 invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 8d09734536 bolt11: add decodepay RPC helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 15e1e4b099 bolt11: support for encoding/decoding and checking.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 9ec5cb7ba2 script: enhance is_p2sh/is_p2pkh/is_p2wsh/is_p2wpkh to extract addr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 7ed81cdf7b pay: factor out actual payment sending from json part of sendpay.
We're going to reuse this for the new 'pay' all-in-one command.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 79df10f782 bech32: pull in bech32 sample code, almost untouched.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell c2a0c51c30 lightningd: check for excessive HTLC locktimes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell 0000dc5908 lightningd: Allow in excess of ctlv_expiry.
We add a test which both overpays and over-sets the delta on the
intermediate and final nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell 71e794a046 lightningd: split ctlv_expiry and final_cltv.
These need to be different for testing the example in BOLT 11.

We also use the cltv_final instead of deadline_blocks in the final hop:
various tests assumed 5 was OK, so we tweak utils.py.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell 68dae5648d gossipd: route correctly using final CLTV value.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell bc9918ad46 dev: option not to do backtracing.
It crashes under valgrind, causing a valgrind error: valgrind gives us a
backtrace anyway, so we don't need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 04:14:33 +00:00
Rusty Russell 21305c0d28 fatal: cause a backtrace.
Much nicer for debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 04:14:33 +00:00
Rusty Russell 82f252c79a test_permfail_new_commit: fix intermittant failure.
Normally, we get an error as soon as we send WIRE_REVOKE_AND_ACK.  But if the
commit timer goes off, we get some extra cycles, during which the other side
can reconnect.  In this case, we simply kill the channeld before it fails,
and never check for the permfail string.

    b'lightning_channeld(18613): TRACE: dev_disconnect: -WIRE_REVOKE_AND_ACK'
    b'lightning_channeld(18613): TRACE: Trying commit'
    b'lightning_channeld(18613): TRACE: htlc 0: SENT_ADD_REVOCATION->SENT_ADD_ACK_COMMIT'
    b'lightning_channeld(18613): TRACE: htlc added REMOTE: local +0 remote -200000000'
    b'lightning_channeld(18613): TRACE: sending_commit: HTLC REMOTE 0 = SENT_ADD_ACK_COMMIT/RCVD_ADD_ACK_COMMIT'
    b'lightning_gossipd(18590): TRACE: Responder: Act 1'
    b'lightning_channeld(18613): TRACE: Derived key 034aab0b5cb755de836cffb34c053ba115fba6fe75414e8f56261e23c80eabb1fe from basepoint 03e0a7bb422b254f54bc954be05bd6823a7b7a4b996ff8d3079ca211590fb5df39, point 02f3bf525b6ca595bf85d63e89c95fc59c0fde3ae434b55c8093bbb5c64849da37'
    b'lightningd(18465): Connected json input'
    b'lightningd(18465):jcon fd 16: Success'
    b'lightningd(18465):jcon fd 16: Closing (Bad file descriptor)'
    b'lightning_gossipd(18590): TRACE: Responder: Act 2'
    b'lightning_gossipd(18590): TRACE: Responder: Act 3'
    b'lightning_gossipd(18590): UPDATE WIRE_GOSSIP_PEER_CONNECTED'
    b'lightning_gossipd(18590): UPDATE WIRE_GOSSIP_PEER_CONNECTED'
    b'lightningd(18465): peer 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518: Peer has reconnected, state CHANNELD_NORMAL'
    b'lightning_channeld(18613): Status closed, but not exited. Killing'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-28 13:33:00 +02:00
Rusty Russell 0c7ca9ab7c gossipd: call to return all connected peers.
And we report these through the getpeers JSON RPC again (carefully: in
our reconnect tests we can get duplicates which this patch now filters
out).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell a7d6326bef type_to_string: format wireaddr.
Good for printing, and removes some code from peer_control.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell 78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell 4bd0352951 lightningd: try to figure out our own IP automatically.
Most of the code is from bitcoind, to handle the weird different non-public
IP ranges.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell 329269d9d0 lightningd: support multiple addresses.
Currently only ipv4 and ipv6.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell bd1cac34ce netaddr: remove.
We use ipaddr everywhere now, so we can remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell dfd60a2047 gossipd: tell the master the peer's address.
This will let us remove peer->netaddr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell ac92138603 common: remove unused assert() headers.
Auditing for assert/abort in common/ code used by lightningd, this is all
that showed up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 81db5896e1 common/json: remove asserts() which may trigger from user input.
They don't currently, since callers check, but be safe.  In addition,
handle NULL returns from these in the bitcoind code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 11b43a422b lightningd: close one possibly-reachable abort.
There are others, but they really are casued by bad failure.  We need a
parachute system for these.

Closes: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell a2dc71b0a1 lightningd: close a take() leak.
test_routing_gossip (__main__.LightningDTests) ... lightningd: Outstanding taken pointers: lightningd/peer_control.c:2352:towire_errorfmt(ld, ((void *)0), "Can't resolve your address")

This caused by the other end closing due to the next bug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell 7d62de8632 lightningd: fix typo in fatal error.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell b6a2b8c58b Add --rgb and --alias options.
And derive random ones from nodeid if they don't choose.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 09:16:14 +00:00
Rusty Russell ebdecebb1a channeld: send channel_announce and initial update to master, not gossipd.
There is a race we see sometimes under valgrind on Travis which shows
gossipd receiving the node_announce from master before it reads the
channel_announce from channeld, and thus fails.  The simplest solution
is to send the channel_announce and channel_update to master as well,
so it can ensure it sends them to gossipd in order

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell 7f38943956 options: show the default network setting in --help.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell 7e022b522c gossipd: don't try to handle padding inside fromwire_ipaddr.
It makes it impossible to embed an ipaddr in another structure, since we
always try to skip over any zeroes, which may swallow a following field.

Do the skip specially for the case where we're parsing routing messages:
we never use padding for our own internal messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell 79962b3588 lightningd: return transaction from fundchannel RPC.
Lets tests figure out the short channel name, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell 48cedef756 peer_control: remove unique_id field.
It's now completely useless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell ffaa15c7da hsm: remove unique_id.
It was only for error messages, so replace it with pubkey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell c3bed51b2d test_lightningd.py: make HSM seeds constant for tests.
Makes it easier to compare before/after failures.  Ideally, we should
run under Travis both with this option and with the seed based on the
entire tmp path (which is still reproducible with determination, but
not fixed every run like this is).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell 4c9f7542b2 subd: Clarify description of subd_release_peer.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-22 16:24:10 +02:00
Rusty Russell 1954844fbf lightningd: make peer_fail_permanent() only save the first error for peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 0b953b86fe subd: automatically detect if callback frees subd.
This involves a tricky callback internally, but far less error-prone.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 5a256c724a subd: simplify and cleanup lifetime handling.
There are now only two kinds of subdaemons: global ones (hsmd, gossipd) and
per-peer ones.  We can handle many callbacks internally now.

We can have a handler to set a new peer owner, and automatically do
the cleanup of the old one if necessary, since we now know which ones
are per-peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell a117d595a4 subd: allow callbacks to free sd.
We'll need this for the next patch; we'll be freeing the old subd whenever
peer->owner changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell cb82bf7aa2 onchaind: send message when peer's transactions are irrevocably committed.
We currently rely on a zero exit status.  That's the only difference between
onchain finished handling and other per-peer daemons, so instead we should
have an explicit "done" message.  This is both clearer, and allows us to
unify.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell f83ee6d5ea dev_disconnect: don't permfail more than once.
The coming tests trigger this latent bug under travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell ebba5f85a2 handshaked: remove.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 474887512d gossipd: rewrite to do the handshake internally.
Now the flow is much simpler from a lightningd POV:

1. If we want to connect to a peer, just send gossipd `gossipctl_reach_peer`.
2. Every new peer, gossipd hands up to lightningd, with global/local features
   and the peer fd and a gossip fd using `gossip_peer_connected`
3. If lightningd doesn't want it, it just hands the peerfd and global/local
   features back to gossipd using `gossipctl_handle_peer`
4. If a peer sends a non-gossip msg (eg `open_channel`) the gossipd sends
   it up using `gossip_peer_nongossip`.
5. If lightningd wants to fund a channel, it simply calls `release_channel`.

Notes:
* There's no more "unique_id": we use the peer id.
* For the moment, we don't ask gossipd when we're told to list peers, so
  connected peers without a channel don't appear in the JSON getpeers API.
* We add a `gossipctl_peer_addrhint` for the moment, so you can connect to
  a specific ip/port, but using other sources is a TODO.
* We now (correctly) only give up on reaching a peer after we exchange init
  messages, which changes the test_disconnect case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 98ad6b9231 lightningd: change connect RPC args.
We're going to make the ip/port optional, so they should go at the end.
In addition, using ip:port is nicer, for gethostbyaddr().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell e11553fc55 lightningd: expose ipaddr parsing.
We don't do DNS lookups, but hack in localhost for the moment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 79ebb9dfd0 json: helper to parse pubkeys.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 871d0b1d74 lightningd: simplify peer destruction.
We have to do a dance when we get a reconnect in openingd, because we
don't normally expect to free both owner and peer.  It's a layering
violation: freeing a peer should clean up the owner's pointer to it,
to avoid a double free, and we can eliminate this dance.

The free order is now different, and the test_reconnect_openingd was
overprecise.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 61786b9c90 subd: don't leak fds if we fail to create subdaemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 4fa36c585d gossipd: receive hsm fd from master.
We'll need this soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell f172be71dc gossipd: fail peer for the master daemon.
This fixes the only case where the master currently has to write directly
to the peer: re-sending an error.  We make gossipd do it, by adding
a new gossipctl_fail_peer message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 2394c9a2e7 crypto_state: move to its own file.
In particular, the main daemon needs to pass it about (marshal/unmarshal)
but it won't need to actually use it after the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 399b5f61bc gossipd: rename fail_peer to drop_peer.
We don't actually send it a failure message, we just close it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 8f057f7fc7 Revert "gossip: send the *other* node's cltv_expiry_delta in channel_announce."
This reverts commit 297e278132.
2017-10-11 11:54:50 +02:00
Rusty Russell f32e0b35ad Revert "Send/receive cltv_expiry_delta in open/accept channel."
This reverts commit 18e3f9820f.
2017-10-11 11:54:50 +02:00
Christian Decker cfac9a1f73 htlcs: Remove in-memory htlc_stubs
We pull them from the database on-demand, where we're storing them
anyway. No need to keep them in memory as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker 896a67d1d6 peer: Pass in the htlc_stub directly to tell_if_missing
No idea why we were iterating over the list of stubs and then passing
in the index instead of a pointer to the stub directly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker a668cb4f2b onchain: Load HTLC stubs upon starting onchaind
This wires in the loading of `struct htlc_stub`s on-demand when
starting `onchaind` so that we don't need to keep them in memory.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker 98f63856f7 wallet: Remove FIXME
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Rusty Russell 297e278132 gossip: send the *other* node's cltv_expiry_delta in channel_announce.
Include tests from example doc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell 18e3f9820f Send/receive cltv_expiry_delta in open/accept channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell 2a28173891 Typo fix: CTLV -> CLTV.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell e137e2527f Update BOLT references with typo fixes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Christian Decker 9a106bf799 wallet: Add invoice removal support
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker 11903aed6c wallet: Wiring in invoice persistence into JSON-RPC and master
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker 5a2242cde7 invoice: Refactor to make the invoice status explicit
So far we were tracking the status by including it either in the paid
or the unpaid list. This refactor makes the state explicit, which
matches the planned DB schema much better.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker c1493ae60c lightningd: Added loading of HTLCs upon startup
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker c6f210a46c htlcs: Needed to add origin_htlc_id to outgoing HTLCs
While loading HTLCs from the database we might not yet have all the
incoming HTLCs loaded when loading a dependent htlc_out. So we defer
the wiring of the HTLCs until we are sure we have them loaded.

This is also the first step towards keeping that association only in
the database, since otherwise we cannot selectively load channels from
DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker 53763ba6a4 wallet: Store HTLCs to the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker b8fa765c3f wallet: Add primitives to store htlcs in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker 249083bcc1 options: Do not schedule a reconnect attempt with `--no-reconnect`
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Christian Decker 359c83b855 options: Move non-legacy options into options.c
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Christian Decker 48796f4f39 cli: Add --no-reconnect cli flag
Especially when testing we might want to disable the automatic
reconnection logic in order not to masquerade bugs that disappear when
reconnecting.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Rusty Russell 3d316518fd wire: use 26-bit lengths for inter-daemon messaging.
Fixes: #289
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell 1b31d1c5a0 bitcoind: handle up to 60 seconds of outage.
Seems to go out to lunch on reorgs:

   +136792.168286138 lightningd(9465):BROKEN: bitcoin-cli getchaintips exited 28: 'error code: -28
   error message:
   Rewinding blocks...

Closes: #286
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell a2d4e09b80 bitcoind: wait for startup.
We don't hit this in testing, since we wait for startup already.  Hacking
tests to avoid that, I tested this code by hand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell 66c1da795f lightningd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 3404c393b7 common/io_debug: replacement for ccan/io's poll which does sanity checks.
For now we just check for outstanding take() or tal_tmpctx().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 3f9ec6c2fa jsonrpc: use-after-free bug due to unspecified free behavior 2/2
Using pc after free in the pay_command_destroyed destructor, so
we just steal cmd onto pc so free order is the one we want.

[ Edit: expanded comment, split commit ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker eac770760c jsonrpc: Make sure we handle the case where jcon outlives command
So far only happens during normal shutdown, but it may happen in other
cases as well. We simply define a new destructor that unregisters the
`cmd` from the `jcon`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell 3564263e12 jsonrpc: use-after-free bug due to unspecified free behavior 1/2
These were fun to hunt down. The jcon and the conn are allocated off
of ld, so the free order is unspecified and if conn is freed before
conn then the finish_jcon destructor uses conn after free.

[ Edit: split commit, modified to use a destructor directly on jcon,
  which is more robust than relying on it only being freed via conn --RR ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell 4275ca5da3 lightningd: don't free log on shutdown until after lightningd freed.
Based-on-patch-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:06:56 +09:30
Rusty Russell 2fe2a0bcf9 peer_control: don't double-free on permanent fail of non-persistent peer.
peer_fail_permanent() frees peer->owner, but for bad_peer() we're
being called by the sd->badpeercb(), which then goes on to
io_close(conn) which is a child of sd.

We need to detach the two for this case, so neither tries to free the
other.

This leads to a corner case when the subd exits after the peer is gone:
subd->peer is NULL, so we have to handle that too.

Fixes: #282
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell b7bb0be944 subd: remove context arg, as we're always owned by lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell f082c7b80e lightningd: add FIXMEs for future work.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell d04b7af56e onchain: add initial onchain state.
We have a race where we start onchaind, but state is unchanged, so checks
like peer_control.c's:

	peer_ready = (peer->owner && peer->state == CHANNELD_AWAITING_LOCKIN);
	if (!peer_ready) {
		log_unusual(peer->log,
			    "Funding tx confirmed, but peer state %s %s",
			    peer_state_name(peer->state),
			    peer->owner ? peer->owner->name : "unowned");
	} else {
		subd_send_msg(peer->owner,
			      take(towire_channel_funding_locked(peer,
								 peer->scid)));
	}

Can send to the wrong daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 72b215f6fe Make all internal message numbers unique.
We were sending a channeld message to onchaind, which was v. confusing
due to overlap.  We make all the numbers distinct, which means we can
also add an assert() that it's valid for that daemon, which catches
such errors immediately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell aea77653d3 onchaind: update bolt #5, and implement failure of timed-out onchain HTLCs.
We re-use the value for reasonable_depth given by the master, and we
tell it when our timeout transactions reach that depth.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 5666ba551f onchaind: fail htlcs which are not included in commitment tx.
As per update 149cf020d6

Fixes: #249
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 4ce9f73e00 BOLT update to fae35903ae600d61124a3920363094b02d2ae8e5
This details handling of onchain HTLCs, and we document how we
implement it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell ab8251c214 lightningd: dev-reenable-commit RPC command to re-enable commit timer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell ce160d9b17 lightnind: _ dev-disconnect argument to suppress commit timer.
Required for catching daemon in exact state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 5957dd9303 lightnind: dev-sign-last-tx RPC command for testing penalty txs.
We don't normally let daemons send invalid txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell a76a53a933 onchaind: fulfill HTLCs onchain.
When we see an offered HTLC onchain, we need to use the preimage if we
know it.  So we dump all the known HTLC preimages at startup, and send
new ones as we discover them.

This doesn't cover preimages we know because we're the final
recipient; that can happen if an HTLC hasn't been irrevocably
committed yet.  We'll do that in a followup patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 664ce8bfc2 watch: remove peer_debug helper.
This was part of the transition; we can now just log directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Christian Decker e418f2a7fe gossip: Make sure timestamp does not change while creating sig
If the HSM is slow it might happen that the timestamp has changed the
second time we come around, so we generate the timestamp externally
and pass it in so we're sure it won't change between calls.

Reported-by: Rusty Russell
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-16 09:02:06 +09:30
Rusty Russell d39c27a304 lightningd: search path for our own directory.
Needed slight reformatting of log.h for stubs autogeneration.

Fixes: #277
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-15 13:44:34 +02:00
Rusty Russell db19873ee4 lightningd: dump backtrace on crash or fatal().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell 584b160cdc lightningd: avoid errors on shutdown.
lightningd can crash on shutdown if it's in the middle of getchaintips;
we free the conn, the finished callback is called (process_chaintips),
and it reports that it received an empty result.

The simplest fix is to set a flag in the struct bitcoind destructor,
and avoid the callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell ec63c0d10b lightningd: give option to crash if a subdaemon fails.
Either when it exits with a signal, or sends an error status message.
Then we make test_lightningd.py use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell ef28b6112c status: use common status codes for all the failures.
This change is really to allow us to have a --dev-fail-on-subdaemon-fail option
so we can handle failures from subdaemons generically.

It also neatens handling so we can have an explicit callback for "peer
did something wrong" (which matters if we want to close the channel in
that case).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell f219955dc6 wallet: fix spending of change.
We recorded our own change as P2SH; it's not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 22:59:24 +02:00
Rusty Russell be90364f7f log: remove struct helpers.
They predated (and inspired) type_to_string(), which is more general.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 22:59:24 +02:00
Rusty Russell afabbe0cb0 log: add ltmp convenience context.
It's sometimes nicer where you don't have a convenient tmpctx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 22:59:24 +02:00
Rusty Russell 4f5e7a4d92 lightningd/Makefile: clean up.
1. Remove reference to old $(LIGHTNINGD_OLD_LIB_OBJS) var (in handshaked too).
2. Make check depend directly on unit tests, insteadof weird lightningd/tests
   variable.
3. check-source-bolt and check-whitespace are automatic for $(ALL_TEST_PROGRAMS)
   so we don't need them here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-11 14:29:20 +02:00
Christian Decker fa13190595 wallet: Hook into the hsm_funding_sig to extract change outputs
This is the step where we broadcast the transaction to the network and
a nice place to extract the change from the transaction.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-10 10:48:53 +09:30
Christian Decker 6d1bcc2c32 Removing the --ignore-dbversion option
It's no longer used and we definitely do not want to run with an
outdated or future db, so we'll terminate if we can't upgrade or
the version is newer than what we understand.

Signed-off-by: Christian Decker <decker.christian@gmai.com>
2017-09-08 17:02:04 +09:30
Christian Decker 641a3b64cf channel: Use min_cltv_expiry in the channel announcements
So far we were always using the deadline in the announcements, that's
obviously not good, so this introduces the parameter as per spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Christian Decker d75733f7fb fix: Make sure peer->owner is always set 2017-09-08 17:02:04 +09:30
Christian Decker 0137aea7fc test: Don't assume size_t == u64 in tests 2017-09-08 16:56:07 +09:30
Rusty Russell b19a4516d4 lightningd: fix crash when old openingd still around.
We weren't killing it.  Eventually it would die, and peer_owner_finished()
would access subd->peer->owner, but that peer was freed already.

Closes: #261
Reported-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 19:11:21 +02:00
Rusty Russell 5889ad5fc4 dev-disconnect: add blackhole option.
To reproduce the next bug, I had to ensure that one node keeps thinking it's
disconnected, then the other node reconnects, then the first node realizes
it's disconnected.

This code does that, adding a '0' dev-disconnect modifier.  That means
we fork off a process which (due to pipebuf) will accept a little
data, but when the dev_disconnect file is truncated (a hacky, but
effective, signalling mechanism) will exit, as if the socket finally
realized it's not connected any more.

The python tests hang waiting for the daemon to terminate if you leave
the blackhole around; to give a clue as to what's happening in this
case I moved the log dump to before killing the daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 19:11:21 +02:00
Rusty Russell e8dadbc0fb lightningd: fix reconnect while openingd still live.
In this case, we unset the old subd->peer, then freed subd.
peer_owner_finished dereferenced subd->peer->owner, and boom:

test_disconnect_funder (__main__.LightningDTests) ... Fatal signal 11. Log dumped in crash.log

------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.2882
==2882== Invalid read of size 8
==2882==    at 0x413F74: peer_owner_finished (peer_control.c:679)
==2882==    by 0x41EA2C: destroy_subd (subd.c:381)
==2882==    by 0x459700: notify (tal.c:240)
==2882==    by 0x459BB1: del_tree (tal.c:400)
==2882==    by 0x459FC0: tal_free (tal.c:509)
==2882==    by 0x413796: peer_reconnected (peer_control.c:493)
==2882==    by 0x413A6A: add_peer (peer_control.c:592)
==2882==    by 0x40ED1F: handshake_succeeded (new_connection.c:186)
==2882==    by 0x41E3DD: sd_msg_reply (subd.c:262)
==2882==    by 0x41E6BB: sd_msg_read (subd.c:318)
==2882==    by 0x41E4E6: read_fds (subd.c:283)
==2882==    by 0x44DEB4: next_plan (io.c:59)
==2882==  Address 0x838 is not stack'd, malloc'd or (recently) free'd
==2882==

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 05:14:38 +09:30
Christian Decker b0c0e28a43 gossip: Simplify announce_signature exchange
The logic of dispatching the announcement_signatures message was
distributed over several places and daemons. This aims to simplify it
by moving it all into `channeld`, making peer_control only report
announcement depth to `channeld`, which then takes care of the
rest. We also do not reuse the funding_locked tx watcher since it is
easier to just fire off a new watcher with the specific purpose of
waiting for the announcement_depth.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-05 12:47:25 +09:30
Christian Decker 1047e891b7 cleanup: Remove IRC flags from default options
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-04 08:47:13 +02:00
Rusty Russell 40fc95921d Cleanup: remove unused IRC flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-04 08:47:13 +02:00
Rusty Russell ee9e300da0 gossip: fix address descriptor handling.
1. The code to skip over padding didn't take into account max.
2. It also didn't use symbolic names.
3. We are not supposed to fail on unknown addresses, just stop parsing.
4. We don't use the read_ip/write_ip code, so get rid of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell 4be7e94e0b gossip: make rpc responses correct.
It's not fee_per_kw, it's fee-per-millionth and a base in msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell 28553308e2 gossip: make channel_announce-set fields unset until channel announce.
Use a negative timestamp as the flag for this, making the test simple.
This allows valgrind to detect that we're accessing them prematurely,
including across the wire on gossip_getchannels_entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell 1cf33eefe2 lightningd: handle case where channeld fails locally-generated HTLC.
jl777 reported a crash when we try to pay past reserve.  Fix that (and
a whole class of related bugs) and add tests.

In test_lightning.py I had to make non-async path for sendpay() non-threaded
to get the exception passed through for testing.

Closes: #236
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-30 11:36:37 +02:00
Rusty Russell bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 739b163f8b Makefiles: simplify dependencies.
Gather all binaries and objects and make the depend on external
requirements and common headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell f42f34b82d external: new subdirectory for all external libraries and submodules.
You will want to 'make distclean' after this.

I also removed libsecp; we use the one in in libwally anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell c6976cd947 shachain: always build 48 bit version.
No more special Makefile hacks required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 153c622157 lightningd: remove lightningd_state.
Some fields were redundant, some are simply moved into 'struct lightningd'.
All routines updated to hand 'struct lightningd *ld' now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 77bcaf0a25 getinfo: remove testnet flag and have network flag instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 0170224fd0 dev_restart: remove this command.
We'll end up doing this via packet traffic (like permfail) I imagine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8c22bd9ee1 headers: fix up header idempotent lines.
For future reference, done via:
	for f in `find wire/ bitcoin/ common/ lightningd -name '*.h' ! -name 'gen*'`; do ID=`echo -n LIGHTNING/$f | tr 'a-z' 'A-Z' | tr -cs 'A-Z0-9' _`; sed 's/^#\(ifndef\|define\) .*_H$/#\1 '$ID/ < $f | sed 's,#endif /..*_H ./$,#endif /* '$ID' */,' | bagto $f; done

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell a37c165cb9 common: move some files out of lightningd/
Basically all files shared by different daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8375857116 common: absorb remaining files from daemon/
Also, we split the more sophisticated json_add helpers to avoid pulling in
everything into lightning-cli, and unify the routines to print struct
short_channel_id (it's ':',  not '/' too).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell a3c51168db routing: use status() instead of log-to-nowhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 85ff95e829 common: new directory for any shared objects.
To avoid everything pulling in HTLCs stuff to the opening daemon, we
split the channel and commit_tx routines into initial_channel and
initial_commit_tx (no HTLC support) and move full HTLC supporting versions
into channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8088e5cadb protobuf: remove
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 3c22aaa213 htlc: move htlc structure into channeld.
Other places require the flags and states, but the structure is
only needed in channeld, and even then we can remove several fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 052c9f42d4 bitcoin/script: remove routines unnecessary for modern daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Christian Decker 4b64b7f2aa lightningd: Fix channel-persistence for channels with commits
I was hoping to defer HTLC updates until we actually store HTLCs, but
we need to flush to DB whenever balances update as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-26 16:37:53 +09:30
Rusty Russell d7db0be5da lightningd: start onchaind when we see funding spent.
We're very simple about it: if there's a reorganization, we restart.  Otherwise
we tell it about everything.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-23 17:38:22 +02:00
Rusty Russell 1b2fb18944 onchaind: basic daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-23 17:38:22 +02:00
Rusty Russell 1396cf6249 lightningd: remove num_revocations_received.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-23 17:38:22 +02:00
Rusty Russell a5d4a3eb2c lightningd: derive num_revocations_received.
It's in the shachain, so storing it is completely redundant.  We leave
it in for the moment so we can assert() that nothing has changed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-23 17:38:22 +02:00
Christian Decker 9c3ac38544 lighthningd: Make sure peer->htlcs is initialized
When loading from DB the list of htlcs was not being initialized which
caused a segfault when the first commit came around, this fixes it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-23 10:23:54 +09:30
Christian Decker def3d77a70 lightningd: Base peer->seed off of the channel ID
The peer->seed needs to be unique for each channel, since bitcoin
pubkeys and the shachain are generated from it. However we also need
to guarantee that the same seed is generated for a given channel every
time, e.g., upon a restart. The DB channel ID is guaranteed to be
unique, and will not change throughout the lifetime of a channel, so
we simply mix it in, instead of a separate increasing counter.

We also needed to make sure to store in the DB before deriving the
seed, in order to get an ID assigned by the DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-23 10:23:54 +09:30
Christian Decker e51d261f51 lightningd: Load persisted channels on startup
This is the big one, and it's completely anticlimactic: it loads all
channels that have reached opening and are not marked as
closingd_complete into memory, that's it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-23 10:23:54 +09:30
Christian Decker 1a94e7282b db: Make peers unique by node_id
We should only ever have a single entry for each node_id, enforcing
this on the DB.
2017-08-23 10:23:54 +09:30
Christian Decker 4bc0750882 wallet: Moving wallet_channel->peer_id into the peer struct
This was supposed to be a temporary solution anyway, and I had a
rather annoying mixup between peer_id and unique_id, the latter of
which is actually a connection identifier.
2017-08-23 10:23:54 +09:30
Christian Decker 3fefd5f46d master: Only persist the peer/channel if we have reached openingd
There is no point in doing so before, and it'd be creating empty
entries for each connection.
2017-08-23 10:23:54 +09:30
Christian Decker e9b69b195a wallet: Store channel to db whenever something changes
This is as coarse grained as it can get, and therefore slow, but it'll
store the channel (sans HTLCs) to database so we can recover them upon
restart.
2017-08-23 10:23:54 +09:30
Christian Decker 406d52042b peer: Add channel to peer and save upon opening a channel
Add the channel to the peer on the two open paths (fundee and funder)
and store it into the database. Currently fails when opening a channel
to a known peer after loading from DB because we attempt to insert a
new peer with the same node_id. Will fix later.
2017-08-23 10:23:54 +09:30
Christian Decker 17173a6322 jsonrpc: Add `connected` field to getpeers
This was implicitly indicated by the presence of the owner field,
making it explicit.
2017-08-23 10:23:54 +09:30
Christian Decker 221fe8c5f3 reconnect: Re-assign peer->netaddr on reconnect, could have changed
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-23 10:23:54 +09:30
Rusty Russell 91116fe67c gossip: include chain_hash in gossip messages.
As per lightning-rfc change 956e8809d9d1ee87e31b855923579b96943d5e63
"BOLT 7: add chain_hashes values to channel_update and channel_announcment"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell ffddb91e3e gossipd: hand in chain_hash.
This will be needed for gossip messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell 2b9d3bb0b7 channeld: hand in chain_hash.
This will be needed for gossip messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell b3514d3430 generate-wire.py: generate chain_hash fields as sha256_double.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell 1ecd687eec sphinx: update padding as per latest BOLT 4
BOLT 04: increase max size of onion payload messages (#227)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell fea412dd94 BOLT: Update text to latest commit.
This brings us up to 955e874acc535ab2c74c1cf0eab61896ea4224ff in
https://github.com/lightningnetwork/lightning-rfc

This doesn't actually change anything; the only actual change is held back
for the next commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell 8ffdeea522 daemon/chaintopology: hand full txs to watch_tx()
This is required for onchaind: we want to watch all descendents by default,
as to do otherwise would be racy, which means we need to traverse the outputs
when a tx appears.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 04db39558d htlc_tx: use keyset abstraction.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 4bfaaef408 keyset: abstraction over what keys we need for a specific commitment.
onchaind will need to do similar logic to channeld, so this allows them
to share much more code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell edd27d21b5 lightningd: save last htlc sigs.
This belongs in the db, but until then...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell b6d7ad0614 lightningd: store the htlc stubs.
These belong in the database, but for now just keep an array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 893335244d openingd: supply initial tx as well as signature.
And store in peer->last_tx/peer->last_sig like all other places,
that way we broadcast it if we need to.

Note: the removal of tmpctx in funder_channel() is needed because we
use txs[0], which was allocated off tmpctx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 80e28707a3 channeld: send commit tx and signature to master.
This also means we can simply drop it to chain on error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell c389215a35 closingd: send the tx to master as we negotiate.
We can derive fees from this, and also broadcast at any time without
having to reconstruct it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell dbfac68c3f lightningd: keep last valid tx, and its signature.
This avoids us having to reconstruct it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 8e0c19c76a lightningd: add dev-fail command to inject permenant failure.
A couple of double-free bugs founnd doing this, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 524e56cf56 dev_disconnect: support '*N' syntax for disconnecting on the N'th occurance.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 99581bd709 dev_disconnect: support 'permfail' line to permanently fail peer.
The master daemon checks for this after a subdaemon dies.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell f5f8ca5352 onchain/onchain_wire: routines to marshal/unmarshal htlc stubs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell f765e0e846 bip32: marshal/unmarshal routines.
Neater than using a u8 array as we do now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell bcc9ed9aa6 lightningd/htlc_wire: marshal/unmashal bitcoin tx.
We want to keep the last valid tx, and its signature, for broadcast.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell a4f290daba htlc_wire: marshal/unmarshal shachain object.
We want to hand it to onchaind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 217df2d2e8 channeld: fix shutdown when we have to complete HTLCs.
We need to check if we exit after sending a revoke_and_ack, otherwise
channeld ends up getting the closing_signed packet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell f5624af965 lightningd: fix error packet we send due to permanent error.
tal_strdup() doesn't set tal_count(), so we end up sending an ERROR
packet with an empty message.  Wrap this and get it right.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell ce24731da9 lightningd: use internal_error rather than freeing peer.
Should never free peer directly: we need to log error, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell 709991fed1 subd: fix case where we replycb calls peer_internal_error().
It frees the sd, destroy_subd() frees sd->conn, then we call io_close(conn).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Christian Decker 6dbd99ddc6 gossip: Fix a race condition between release_peer and fail_peer
There was a race condition that would cause an assertion to segfault
if a call to release_peer was interleaved with a fail_peer. The
release_peer was making the peer non-local, which was then causing the
assertion in fail_peer to fail. Now we just have 3 cases: not found,
local, and non-local.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-17 10:31:55 +09:30
Christian Decker cc5af13e05 wallet: Implement channel_config persistence 2017-08-10 12:34:58 +09:30
Christian Decker ca00ec0bb8 htlc: Include htlc_wire.h in peer_htlcs.h to avoid forward decl 2017-08-10 12:34:58 +09:30
Christian Decker 85d2256898 wallet: Use persistent shachain for remote secrets 2017-08-10 12:34:58 +09:30
Christian Decker 28cc92cd15 gossip: Use bit 3 (mask 0x08) to signal initial routing sync
After quite some back and forth we seem to finally agree on the bit
3 (mask 0x08) to signal optional initial_routing_sync.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-09 10:01:33 +09:30
Rusty Russell 40895e4550 Update to match spec: "BOLT7: Reorder feature bitmaps in order to allow future changes"
aka. a257554456cda98afd1532c302c0e5e84de0455e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-03 14:26:41 +02:00
Rusty Russell 2b7c091e70 closing: update to match proposal to restart negotiation on reconnect.
This simplifies significantly, as we don't need to remember what we offered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-02 11:45:30 +09:30
Christian Decker 35847206b2 fix: Allow pings while waiting for the channel to lock
This was causing failures on testnet where confirmations are not
immediate.

Reported-by: Fabrice Drouin @sstone
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-07-22 09:52:14 +09:30
Christian Decker 2bf92c9063 peer: Check localfeatures and globalfeatures against what we support (#195)
We support a number of features already, so failing connections
whenever we see an even bit set is not a good idea. This turned out to
kill our connections to eclair.

Also, the spec says that the LSB / bit 0 is to be counted as index 0, and
therefore even. So we need to check the lower of each 2-bit-tuple not
the higher one.
2017-07-16 10:38:26 +09:30
Christian Decker 7dc693963d script: Consolidate scripts to use pubkey_to_hash160 2017-07-15 10:19:33 +09:30
sstone f371b6df20 sphinx: fix payload amount encoding
it was changed to 64 bits
2017-07-14 19:27:33 +02:00
Christian Decker 83c8c3fc52 opening: Use the correct chainparams to open a channel
We were using the bitcoin genesis blockhash for all networks, which is
not correct, and would result in the open being aborted when talking
to other implementations.

Reported-by: @sstone and @pm47
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-07-12 11:30:23 +09:30
Christian Decker fbc2b65a3d opening: Pass network_index through to openingd
This is needed in order to open channels with the correct
genesis_blockhash.

Reported-By: @sstone
2017-07-12 11:30:23 +09:30
Christian Decker 843e21826a opts: Remove --regtest flag and pass chainparams to bitcoind 2017-07-12 11:30:23 +09:30
Christian Decker cf16b5faea options: Add option to set the network 2017-07-12 11:30:23 +09:30
Christian Decker 9370a3cdd2 opening: Fix switched parameters for towire_accept_channel
Turns out we were serializing two parameters in the wrong order.
2017-07-12 11:00:26 +09:30
Christian Decker 1a1e29a4bc gossip: Re-initiate the broadcast timer upon reconnect
We weren't registering reconnecting peers for broadcasts. Just
starting a timer is enough. Also added an integration test to check
that the gossip sync is being resumed.
2017-07-12 11:00:26 +09:30
Rusty Russell 03be40ea8b lightningd: don't reset balance when we reconnect!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 8b71ea924a lightningd: don't crash when simply using closingd just for retransmission.
test_closing_negotiation_reconnect (__main__.LightningDTests) ... peer state CLOSINGD_COMPLETE should be CLOSINGD_SIGEXCHANGE

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell a87d50acc9 Makefile: clean daemons on clean.
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 7d3e074863 Misc feedback minor fixes.
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 87232ec25c lightningd: new peer state CLOSINGD_COMPLETE.
This is a transitional state, while we're waiting to see the
closing tx onchain (which is To Be Implemented).

The simplest way to do re-transmission is to re-use closingd, and just
disallow any updates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell c1f21b9fce lightningd/peer_control.c: rename peer->balance.
I made the mistake of thinking it was a [NUM_SIDES] array, but
it's actually our balance, and it's in millisatoshi.  Rename
for clarity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 4223a91842 lightningd: update daemon list.
This is used to check that versions are correct on startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 0c5e238a48 lightningd: handle reconnect during shutdown and closing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 0c07856ae4 lightningd/peer_control.c: remove unnecessary reconnected field.
It can be on the stack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell a827d2b2bb lightningd: add in support for closingd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell e5a8a7502c lightningd/closing: subdaemon for closing negotiation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 6fa90c926a channeld: exit after shutdown when no more HTLCs.
Ready for the introduction of closingd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 80886cda8a daemon_conn: fix daemon_conn_sync_flush.
We need to set fd to blocking before trying to sync write.  Use
io_fd_block() elsewhere, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 94a33a7d15 channeld: use an enum side not a bool for funder/
This predated our handling of enums.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 7884343c15 tools/generate-wire.py: enums are assignable.
Simplfies their marshalling/unmarshalling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell 9ec804341c channeld: fix intermittant failure on reading init msg from master.
As tracked down by Christian; by setting up the master conn first,
we make the master fd async.  This means that the synchronous read
(in init_channel) can fail with -EAGAIN, and indeed, Christian
saw this when not running under valgrind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-03 19:43:34 +09:30
Rusty Russell 249085ac10 channeld: reorder functions more logically.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-03 19:43:34 +09:30
Rusty Russell c05f63a4db peer_control: fix double-state transition on reconnect.
We actually don't need to transition if we're reconnecting, and logic
to go to CHANNELD_NORMAL was wrong: we checked that we'd seen funding tx
locked, but not that we'd received a msg from the remote peer.

We need to fix the tests now we no longer double-transition, too.

Fixes: #188
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-03 19:43:34 +09:30
Christian Decker f9a2f73072 Addressing feedback from PR #187 2017-06-30 13:42:00 +02:00
Christian Decker 70f4484ba8 channeld: Pass through the agreed feerate_per_kw to channeld
We were erroneously using the fee_base which refers to the fees we
require for forwarding transactions instead of the on-chain fees.
2017-06-30 19:55:17 +09:30
Christian Decker 5c80963577 bug: channel_update parameters slipped by one 2017-06-30 19:55:17 +09:30
Christian Decker 5e1dcea5b8 opening: Passing feerate_per_kw through to master 2017-06-30 19:55:17 +09:30
Christian Decker 02aba77698 channel: Passing channel_flags through to channeld 2017-06-30 19:55:17 +09:30
Rusty Russell ac1172c7b0 lightningd: check scriptpubkey in shutdown.
Important: a non-standard one can make the closing tx not propagate.

Drive-by cut&paste message fix, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell 4881129f54 peer_fail_permanent: handle take() arg.
We assume this in at least one caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell ee9ebbc184 channeld: fix `funding_locked` retransmission using latest revision of BOLT2.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell 42c9cb58c7 channeld: rename commit_index to next_index.
This is what it actually is, and makes it clearer when we refer to the
spec.  It's the commitment we're currently updating, which is the next
commitment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell d411b07ba2 channeld: fix make_revocation_msg() to take index of revoke.
It currently takes index + 1 (ie. current commit), which is confusing
in this context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell 1679f2da06 channeld: update to send next commitment number on connection reestablish.
As per latest BOLT revision.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell 05b9e0d538 channeld: do connection reestablishment synchronously.
We currently have the problem that the master can send new HTLCs before
we've processed the incoming reestablish message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell 456fa39380 sync_crypto_write: support take(msg)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell de5bf56ffa opening: update to new open_channel with channel_flags.
While we're there, make the announcement conditional on it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell d492f3872c wire/peer_wire: rename gossip_msg / unknown_msg / unknown_msg_discardable
The next patch includes wire/peer_wire.h and causes a compile error
as lightningd/gossip_control.c defined its own gossip_msg function.

New names are clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-29 00:08:21 +02:00
Rusty Russell fc4e2d8b48 lightningd: fix typo.
If we're going to die, at least we can do so with dignity.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 99289bece7 lightningd/peer_control.c: fix memleak.
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell ac703ff9dc lgihtningd: remove extraneous whitespace.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell e2c5104aab lightningd: immediately fail HTLCs coming in from shutting down channels.
This can happen even without a protocol violation, if the incoming
update_add_htlc crosses over our outgoing shutdown.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 866ac79850 channeld: shutdown support.
We keep the scriptpubkey to send until after a commitment_signed (or,
in the corner case, if there's no pending commitment).  When we
receive a shutdown from the peer, we pass it up to the master.

It's up to the master not to add any more HTLCs, which works because
we move from CHANNELD_NORMAL to CHANNELD_SHUTTING_DOWN.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell ee00c2c508 lightningd: remove peer->gossip_client_fd
We don't need to keep this around any more: by handing it to
subdaemons we ensure we'll close it if the peer disconnects, and we
also add code to get a new one on reconnection.

Because getting a gossip_fd is async, we re-check the peer state after
it gets back.  This is kind of annoying: perhaps if we were to hand
the reconnected peer through gossipd (with a flag to immediately
return it) we could get the gossip fd that way and unify the paths?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 4185153d81 gossipd: interface to get a client gossip_fd for a reconnect.
At the moment, master simply keeps the gossip fd open when peer
disconnects.  That's inefficient, and wrong anyway (it may want a
complete new sync, or may not, but we'll currently send all the
messages including stale ones).

This interface will be required for restart anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell ea52b0a8ff openingd: pass back gossip fd on completion.
For the moment, master still keeps a dup of gossip_fd, but that's the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 2ff50107ac lightningd/hsm_control: use a simple fd for HSM.
Now we're always sync, just use an fd.  Put the hsm_sync_read() helper
here, too, and do HSM init sync which makes things much simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 6e59f85666 subd: expose raw API for getting a single fd to a subdaemon.
We're going to use this for the HSM.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 4ef3e9e3fb lightningd/peer_control: get announcement signature from HSM synchronously.
We temporarily play games with the hsm fd; those will go away once
we're done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell d1d84a81d2 lightningd/new_connection: get handshake fd from HSM synchronously.
We temporarily play games with the hsm fd; those will go away once
we're done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 7213312f57 lightningd: remove fd from struct peer.
With no async calls left, we can just use a stack variable for the fd.

And we're now *always* in the hands of some daemon, unless we're
disconnected, so owner is only NULL in that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell f2d4309add lightningd/subd: explicit failure reply support.
We had a terrible hack in gossip when a peer didn't exist.  Formalize
a pattern when code+200 is a failure (with no fds passed), and use it
here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 31ff5a49f4 lightningd/peer_control: remove cs from struct peer.
Now we're not async, we don't need to pack it into struct peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 232a182b25 lightningd: get fd from HSM synchronously.
This means there's no GETTING_HSMFD state at all any more.  We
temporarily play games with the hsm fd; those will go away once we're
done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 77620ea06f lightningd: get funding signature from HSM synchronously.
This means there's no GETTING_SIG_FROM_HSM state at all any more.  We
temporarily play games with the hsm fd; those will go away once we're
done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell 40ce29beac wire_sync_write: support take()
We often want it to free the message after writing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Christian Decker 938ab67a01 wallet: move can_spend to wallet.c and json_addfunds to walletrpc.c 2017-06-23 16:02:20 +09:30
Christian Decker d6656358b5 wallet: Add bip32_base to the wallet struct
Wallet should really be the container for anything bip32 related, so
I'd like to slowly wean off of `ld->bip32_base` in favor of
`ld->wallet->bip32_base`
2017-06-23 16:02:20 +09:30
Christian Decker 8cdab9da1e wallet: Move json_newaddr to wallet.c 2017-06-23 16:02:20 +09:30
Christian Decker 8c4f381afb hsm: Sign withdrawal transactions 2017-06-23 16:02:20 +09:30
Christian Decker a775b52941 wallet: Return change satoshis when selecting coins
We'd be computing them later most of the time anyway.
2017-06-23 16:02:20 +09:30
Christian Decker 3f594172b2 hsm: Add messages for withdrawal signature requests 2017-06-23 16:02:20 +09:30