Commit Graph

4373 Commits

Author SHA1 Message Date
William Casarin 8869204730 test: add pytest-xdist to the python test requirements
Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-26 23:18:37 +02:00
William Casarin 6e4745b28a doc: update python test requirements in HACKING
Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-26 23:18:37 +02:00
Rene Pickhardt 88ef2246e8 exchanged code containing goto with a do while loop (#1758)
exchanged code containing goto with a do while loop
2018-07-26 23:17:45 +02:00
Christian Decker d6048de100 json-rpc: Shutdown the JSON-RPC in the context of a DB transaction
This needs to be done separately from the rest of the daemon since we can
otherwise not make sure that it happens before the DB is freed and we might
still need the DN, and be running in a DB transaction, for some destructors to
run.
2018-07-26 19:14:56 +02:00
Christian Decker ead0c8e17a pytest: Add test for issue 1737 that is expected to fail
This reproduces the issue described in issue #1737 and will be fixed by the next
commit.
2018-07-26 19:14:56 +02:00
Simon Vrouwe a80622edab at startup initialize smoothed feerate to polled feerate
fix indentation
2018-07-26 19:08:13 +02:00
Rusty Russell 378d73cd96 channeld: fix dev_disconnect doublefree crash.
We shouldn't unconditionally free msg in enqueue_peer_msg:

DEBUG: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: dev_disconnect: @WIRE_REVOKE_AND_ACK
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: FATAL SIGNAL 6 (version 8aae6a8)
...
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:98 (call_error) 0x80855d1
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:170 (check_bounds) 0x8085730
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:181 (to_tal_hdr) 0x8085791
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:504 (tal_free) 0x8085fe6
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: channeld/channel.c:2651 (main) 0x8050639

For additional safety, handle each msg allocation separately, rather than
freeing at bottom of large branch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-26 14:33:23 +02:00
Mark Beckwith e8edecfbf7 Add lightningd-config(5) to install
The man5 section wasn't being added in the Makefile.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-26 05:41:48 +00:00
ueno 21e61a5232 fix: make devtools-clean 2018-07-25 15:44:25 +02:00
Rusty Russell 641b33349b peer_control: annotate feature bitfields as being taken.
Because tal_dup_arr takes, this does too.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell f2e0c35916 connectd: hack in a gossip flush as a workaround.
We were failing test_closing_torture, with gossipd complaining that it
received a malformed packet.  This makes it pass, but the real fix is
in the next series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 722dd05e9d lightningd: keep features arrays for connected peers.
As a side-effect, we only print them for connected peers (which avoids
an O(n^2) traversal).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell b5fcd54ef0 channeld: don't read from gossipfd while we're reconnecting.
That was the cause of the bad gossip order failures: gossipd thought our
channel was live, but the other end didn't receive message last time.

Now gossipd doesn't use fd to kill us (connectd tells master to do so), we
can implement read_peer_msg_nogossip().

Fixes: #1706
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell d241bd762c connectd: don't use gossip_getnodes_entry.
gossip_getnodes_entry was used by gossipd for reporting nodes, and for
reporting peers.  But the local_features field is only available for peers,
and most other fields are only available from node_announcement.

Note that the connectd change actually means we get less information
about peers: gossipd used to do the node lookup for peers and include the
node_announcement information if it had it.

Since generate_wire.py can't create arrays-of-arrays, we add a 'struct
peer_features' to encapsulate the two feature arrays for each peer, and
for convenience we add it to lightningd/gossip_msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell c9435787cb connectd: remove gossip-related fields.
This almost completes the removal of the last vestages of gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 7b2641ed0d gossipd: remove peer-related fields and wire messages.
This completes the removal of peer-related messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 0d442b5ff2 gossipd: move files into connectd.
These source files are only used for peer-related things, so move them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 5c066b9a25 connectd: wean off gossip_wire message types, use our own.
This involves much renaming of gossip -> channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell dba7f9002f gossipd: provide connectd with address resolution.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 3d3d2ef9af gossipd: remove connectd functionality, enable connectd.
This patch guts gossipd of all peer-related functionality, and hands
all the peer-related requests to channeld instead.

gossipd now gets the final announcable addresses in its init msg, since
it doesn't handle socket binding any more.

lightningd now actually starts connectd, and activates it.  The init
messages for both gossipd and connectd still contain redundant fields
which need cleaning up.

There are shims to handle the fact that connectd's wire messages are
still (mostly) gossipd messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 92d66a5451 gossipd: take connectd fd on initialization.
connectd has a dedicated fd to gossipd, so it can ask for a new gossip_fd
for a peer.

gossipd has a standalone routine to create a remote peer (this will
eventually be the only way gossipd creates a new peer).

For now lightningd creates a socketpair but doesn't run connectd, so
gossipd never sees any requests on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell e1dfb1b178 gossipd: simplify per-peer features.
Store the two we care about as booleans.  Once connectd is complete we won't
even have the feature bitmaps for peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell b4ed954435 connectd: do socket binding during initial setup.
gossipd does a two-step initialization: it only tries to create the
listening sockets when it's activated.  This means it doesn't know the
addresses to announce until this point.

Now connectd is doing this, this would mean we'd have to tell gossipd
later ("oh, BTW here are your addresses") since we need to start gossipd
before connectd activation.

So make connectd do all the setup, but only actually listen on the fds
once we activate it.  We clone the gossip_init message into
connect_wire.csv.  The master daemon still waits for a reply from
connectd for the activate message, since it wants to be listening
before it prints "Server started".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 5a8091c853 connectd: cleanup gossip connection properly
This still has a problem, but we can't fix that easily here; per-peer
daemons don't have this trouble, however.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell eab83ca79b connectd: new daemon to handle connections.
This is just copying most of gossipd/gossip.c into connectd/connect.c.
It shares the same wire format as gossipd during transition, and changes
are deliberately minimal.

It also has an additional message 'connect_reconnected' which it sends
to the master daemon to tell it to kill a peer; gossipd relied on
closing the gossipfd to do this, but connectd doesn't maintain an fd
with remote peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 8e3bc84634 lightningd: tell gossipd when we kill an opening channel.
And unify the code paths.  Without this, a reconnect during opening
might hang.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell f747ad8f73 common/daemon_conn: add daemon_conn_wake() helper.
We've been open-coding it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 16b8f1eb83 gossipd: actually use global features to create our own node_announcement.
It's currently empty, but I was surprised we still used "NULL".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell f4e2a9c921 pytest: don't expect alias and color in listpeers.
Gossipd combines the information if it knows it, but that's really the
job of 'listnodes'.  More importantly, channeld won't have access to
this information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell eee84b198b listnodes: display global_features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell a52d522525 gossipd: handle ping messages for remote peers too.
This simplifies our ping handling: make gossipd always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
William Casarin f8fa4213f1 tools: use /usr/bin/env bash instead of /bin/bash
These commands fail on systems that do not have bash under /bin

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-24 00:25:43 +00:00
Mark Beckwith 369bd274a1 Updated docs to reflect bech32 default
The manpage did not specify the name of the named parameter (addresstype).
It does now.

Fixed manpage grammar errors.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-23 23:46:06 +00:00
Rusty Russell 231f14e645 lightningd: get basepoints from hsmd, don't ever get seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 9bf238e001 hsmd: provide message for master to get basepoints & funding pubkey for a channel
This is only used by the master daemon, but it's not secret information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 429aad8ac7 closingd: use hsmfd to get signatures, don't use seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 6b700f904d hsmd: add message to sign the mutual close transaction.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell cdc97f5114 channeld: use HSM for signatures and to get per_commitment_point, remove seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 8f9408a072 channeld: store next per_commit_point.
This will avoid us having to round-trip to the HSM each time we want it.
For now we still derive it, too, and assert it's correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 909bda938b channeld: add hsm_req() helper.
We're going to be making more HSM requests as we move signing over to the HSM.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 89b1cc69ef openingd: take hsmfd, use it to sign commitment transaction and get commitment_point
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell b9ff130e9e hsmd: routines to sign remote commitment and HTLC txs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 613b65eede onchaind: use the HSM to get the per-commitment-point.
This means onchaind doesn't need the per-channel secret at all (aka. peer seed)
so we remove that from the onchaind_init message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 2575dbf493 hsmd: provide routine to get the per-commitment point.
This will be used by onchaind for now, but also for openingd and channeld
in future, so it returns the old revocation secret as well.

Of course, the HSM should refuse to sign a commitment transaction if it
has handed out the revocation secret previously!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 14d6fc4a31 onchaind: use HSM for signing htlc transactions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 522b9dc96b hsmd: add routing to sign a local htlc transaction for onchaind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 4098f47cfc onchaind: use HSM to sign "to-us" transactions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 437b65eba1 onchaind: clean up types for penalty transactions.
We can use pubkey_from_secret() to avoid the ugly sha->secret->privkey
conversion.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell dfaf74d972 hsmd: add routines to sign onchain transactions, part 1.
This handles the "to-us" transactions which return funds to the wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell 93b8217eb0 lightningd: get HSM to sign the last commitment tx for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00