Commit Graph

2622 Commits

Author SHA1 Message Date
Rusty Russell 43c97187a4 test_lightningd: fix test_gossip_jsonrpc.
A watch constant never changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 10b40c524d gossipd: attach updates to pending transactions in preference.
Otherwise, we otherwise end up with out-of-order updates
(ie. preceeding announcements).

I assume that is because of the locally-inserted connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 96ed75d8b2 gossipd: handle channel_update while we're still checking channel_announce.
We only need to keep one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell db30411ba5 gossipd: look up short_channel_id when we see a channel_announcement.
This is done it two parts, since we have to ask the main daemon to do
the lookup for us.

If this becomes a bottleneck, we can have a separate daemon, or even
an RPC pipe to bitcoind ourselves.

Fixes: #403
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell e22dd5ad71 lightningd: implement gossip_get_txout/gossip_get_txout_reply.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 2b1eb6a677 bitcoind: getoutput so we can check short_channel_ids.
It would be nice if bitcoind had an RPC to do this in one, but that's
a bit much to ask for.  We could also hand around proofs, for lite nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 61be62ddb7 gossipd: ensure node_announcement timestamps always increment.
Covers a possible source of subtle bugs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell bc631166f0 channeld: ensure channel_update timestamps always increment.
Covers a possible source of subtle bugs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 86e1a61165 json: fix json_tok_bitcoin_amount()
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 9a8eca4fee Remove obsolete FIXME messages.
1. htlc->fail has been changed to a u8 *.
2. wallet_get_newindex saves to the db.
3. peer->next_htlc_id is saved to the db in peer_save_commitsig_sent() below.
4. We do store commit in peer_save_commitsig_received(peer, commitnum),
   and the fixme below talks about HTLC sigs.
5. We do commit shachain and next_per_commit_point in wallet_shachain_add_hash
   and update_per_commit_point respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 27e0ea2358 wire: remove special zero-key handling.
We used to use a hack for gossip_resolve_channel_reply, where we'd send
a NULL key on failure.  It's now been neatened to use a counter, so we
don't need this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 55909a7a39 lightningd: call peer_last_tx() to update peer->last_tx before saving to db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 8cf97e904d lightningd: remove unused "forever_confirms" parameter.
It's 100, but we never wired this up, so remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 7e0bc88e2a gossip: fix default broadcast interval, move option.
This now makes a few more gossip tests time out without --dev-broadcast-interval

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 171ce689dc lightningd/chaintopology: remove unneeded chain_topology pointer from block.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 3332cd6395 chain_topology: remove unneeded struct outgoing_tx topo pointer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell cc60735aee channeld: check short_channel_ids match before announcing channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell b1e93c8838 channeld: remove redundant funding_locked assignment.
This makes no sense here, and yet it was in the original HTLC patch
791927936.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
ZmnSCPxj fdbf2f41b4 generate-wire.py: Add scare comments to output.
Discourage developers from modifying files generated by generate-wire.py
2018-01-10 23:53:12 +00:00
practicalswift ab0c26be91 Check return value of json_get_params(...) call in json_getlog(...) and json_getpeers(...)
All other users of json_get_params(...) check the return value:

```
lightningd/chaintopology.c:     if (!json_get_params(buffer, params,
lightningd/chaintopology.c:     if (!json_get_params(buffer, params,
lightningd/dev_ping.c:  if (!json_get_params(buffer, params,
lightningd/gossip_control.c:    if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params, "label", &labeltok, NULL)) {
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/jsonrpc.c:   if (!json_get_params(buffer, params,
lightningd/pay.c:       if (!json_get_params(buffer, params,
lightningd/pay.c:       if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
wallet/walletrpc.c:     if (!json_get_params(buffer, params,
wallet/walletrpc.c:     if (!json_get_params(buffer, params, "tx", &txtok, NULL)) {
```
2018-01-10 23:43:50 +00:00
Rusty Russell 467ca0a996 lightningd: suppress feerate debug messages when nothing changes.
Every 30 seconds, these flood the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-10 16:55:36 +01:00
practicalswift c53dfc3fde Check return value of derive_simple_privkey(...) call in hsm_unilateral_close_privkey(...)
All other users of derive_simple_privkey(...) check the return value:

```
channeld/channel.c:     if (!derive_simple_privkey(&peer->our_secrets.htlc_basepoint_secret,
lightningd/test/run-commit_tx.c:        if (!derive_simple_privkey(&x_remote_htlc_basepoint_secret,
lightningd/test/run-commit_tx.c:        if (!derive_simple_privkey(&x_local_delayed_payment_basepoint_secret,
lightningd/test/run-commit_tx.c:        if (!derive_simple_privkey(&x_local_htlc_basepoint_secret,
lightningd/test/run-key_derive.c:       if (!derive_simple_privkey(&base_secret, &base_point,
onchaind/onchain.c:     if (!derive_simple_privkey(&secrets->delayed_payment_basepoint_secret,
onchaind/onchain.c:     if (!derive_simple_privkey(&secrets->payment_basepoint_secret,
onchaind/onchain.c:     if (!derive_simple_privkey(&secrets->htlc_basepoint_secret,
onchaind/onchain.c:     if (!derive_simple_privkey(&secrets->payment_basepoint_secret,
onchaind/onchain.c:     if (!derive_simple_privkey(&secrets->htlc_basepoint_secret,
```
2018-01-10 04:05:49 +00:00
practicalswift e8c94c2e29 Check return value of read_all(...) call
All other users of read_all(...) check the return value:

```
hsmd/hsm.c:    if (!read_all(fd, &secretstuff.hsm_secret, sizeof(secretstuff.hsm_secret)))
test/test_protocol.c:    if (!read_all(fd, p, len))
wire/wire_sync.c:    if (!read_all(fd, &len, sizeof(len)))
wire/wire_sync.c:    if (!read_all(fd, msg, wirelen_to_cpu(len)))
```
2018-01-10 04:03:58 +00:00
William Casarin 5043e201b8 emacs: add .dir-locals.el for linux style C
This will automatically configure proper indent settings for new contributors
using emacs.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-10 04:01:56 +00:00
Rusty Russell 758317387e HACKING.md: describe take() in more detail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-10 03:58:46 +00:00
Rusty Russell 9b129f7fb5 doc/HACKING.md: update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-10 03:58:46 +00:00
Christian Decker 90527498bc pytest: Do not re-print logs if we failed to find a log entry
CI always runs with TEST_DEBUG=1 which prints logs anyway, and testing
locally should also be done this way, combined with pytest which
captures the logs. No need to duplicate the functionality of pytest.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-10 03:52:46 +00:00
Christian Decker d0da8a1035 pytest: Remove unused breakpoint shorthand
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-10 03:52:46 +00:00
Christian Decker 55d2e4b1d0 pytest: Spawn a new bitcoind for each test
Since we seem to have some isolation concerns when re-generating the
same HSM secret and re-parsing the blockchain some blocks in the past.

This also alleviates the problem of printing to a logging stream that
has been closed. Previously bitcoind would keep running despite a test
had failed and continue logging to the, now closed, StringIO that
py.test uses when capturing stdout.

The performance impact seems to be 1-3 second per test, not too bad
IMHO for increased test isolation and cleaner logs:

|--------------------+---------------+----------|
|                    | No_valgrind   | Valgrind |
|--------------------+---------------+----------|
| bitcoind per suite | 10 min 24 sec | 46:15.31 |
| bitcoind per test  | 11 min 38 sec | 49:21.64 |
|--------------------+---------------+----------|

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-10 03:52:46 +00:00
Christian Decker 00e75d3d81 pylightning: Bumped version to 0.0.2 2018-01-10 03:52:46 +00:00
ZmnSCPxj 72841056f5 Makefile: Install subdaemons into pkglibexecdir instead of bindir.
Fixes: #434
2018-01-09 14:52:48 +01:00
ZmnSCPxj e717963210 lightningd: Do not search for ourself in the daemon dir.
In preparation for putting our daemons into pkglibexecdir; lightningd
itself will be in bindir, separate from our subdaemons.
2018-01-09 14:52:48 +01:00
ZmnSCPxj 7f88585d84 lightningd: When searching subd fall back to pkglibexecdir. 2018-01-09 14:52:48 +01:00
ZmnSCPxj 1946f0c288 Makefile: Add BINTOPKGLIBEXECDIR macro. 2018-01-09 14:52:48 +01:00
ZmnSCPxj bba8a3cab7 Makefile: Add installcheck target to check installation procedure. 2018-01-09 14:52:48 +01:00
ZmnSCPxj c572c3bbee lightningd: Add --test-daemons-only option. 2018-01-09 14:52:48 +01:00
ZmnSCPxj e559f2cb2e Implement "install" and "uninstall" targets for Makefile.
Fixes: #383
2018-01-09 14:52:48 +01:00
practicalswift 4bdd2452f2 Make sure fsync, connect and close are never accidentally passed negative arguments 2018-01-09 14:50:50 +01:00
Rusty Russell 795e42d757 chaintopology: fix use-after-free which sometimes causes fatal() in clear_otx_peer
I've only seen this under travis, so I can't verify that this fixes it,
but it's certainly a bug which could cause that issue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-09 14:49:52 +01:00
practicalswift dcb4039a96 Check lseek(...) return value 2018-01-09 13:52:12 +01:00
Rusty Russell 0f97b8cf36 subdaemon.c: subdaemon_setup() routine for all daemons.
Our handling of SIGPIPE was incoherent and inconsistent, and we had much
cut & paste between the daemons.  They should *ALL* ignore SIGPIPE, and
much of the rest of the boilerplate can be shared, so should be.

Reported-by: @ZmnSCPxj
Fixes: #528
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:16:39 +01:00
Rusty Russell a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Rusty Russell 7975a1cbfc json_getchannels: short_id -> short_channel_id, use standard format.
We include flags separately, so name this field as per spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Christian Decker 5c91a3449a pytest: Add withdraw test after permfail to test unilateral_close
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 539c1485f2 hsm: Implement private key derivation needed for unilateral closes
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 3db4474465 hsm: Generate fully signed funding transaction in the HSM
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker d9df831a2f Don't attempt to permute a single input
This was annoying valgrind since it would attempt to swap an input
with itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker ce845853b0 hsm: Generate fully signed transactions and return them to caller
So far we have been generating the tx both in the HSM and in the
caller, and had to rely on them generating exactly the same
transaction. This makes it a lot simpler by fully signing and
serializing the TX on the HSM side and the caller just needs to unpack
and broadcast it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 248879e203 wire: Move bitcoin_tx serialization from htlc_wire to wire
Removes the need to keep a second transaction around and marking it as
`noleak`, just to make sure that dependencies are not free'd along
with the original tx.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 524849bc11 bitcoin: Split pull_bitcoin_tx
The deserialization of bitcoin transactions in wire/ is rather
annoying in that we first allocate a new bitcoin_tx, then copy it's
contents onto the destination and then still carry the newly allocated
one around due to the tal-tree. This splits `pull_bitcoin_tx` into
two: one part that does the allocation and another one that proceeds
to parse.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00