Commit Graph

4103 Commits

Author SHA1 Message Date
Rusty Russell ff6a6f8deb lightningd: create hsm_get_client_fd() helper.
We're going to use this more when we hand hsm fds to openingd,
onchaind and closingd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell f9e5dc7ee8 hsmd: take dbid for client.
We need this later, to generate its seed.  When we switch to lnd's key system,
we'll only need this, and not peerid.

Note also that the peerid is not just for messages any more, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell 64008e275a common/derive_basepoints: add routines to get a specific secret.
Often we only need a single secret, so it's clearer to have routines
to do just that.  When we change to the lnd key scheme, there will be
no benefit in calculating them all together.

This also adds a test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell b2b85100d7 common/derive_basepoints: add routines for marshal/unmarshal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell ad9dbaaa3f openingd: rename confusing 'our_commit' and 'their_commit'.
We use 'our_commit' for the commit we sign (ie. the remote commitment tx),
and vice versa.  Use local/remote nomenclature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell e217bc1220 per-commit-secret is a struct secret, not a sha256.
Well, it's generated by shachain, so technically it is a sha256, but
that's an internal detail.  It's a secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell 6c98457ef2 per-peer seed is a 'struct secret' not a 'struct privkey'.
They're both 32 bytes, but it's not a privkey at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell 019ba86b91 gossipd: use optional fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell ac4c6b1a82 tools/generate-wire.py: support for optional fields, with ? before typename.
We already work around this by using an array with a 0/1 length convention,
but better to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Hiroki Gondo 070aa08709 fix: compile error with clang 2018-07-17 05:08:29 +00:00
Rusty Russell edf043c89b pytest: make utils.py read config.vars.
I could not figure out why test_announce_address suddenly stopped working:
I had previously been using DEVELOPER=1 on the cmdline for historical
reasons when testing locally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-16 23:16:34 +02:00
Rusty Russell a37e692af6 pytest: mark test_reconnect_normal flaky for now.
The bad gossip order bug is going to take a while to fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-16 22:48:41 +02:00
Mark Beckwith 19a0b78fec param: renamed files
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-16 03:51:21 +00:00
Christian Decker 356dcbba97 pylightning: Don't attempt to deserialize after every read
We make use of the structure of the final read to decide when to deserialize.
2018-07-16 00:24:04 +00:00
Bertrand Marlier 760b053d7b pylightning: fix incorrect description for disconnect() method 2018-07-15 18:40:48 +02:00
SimonVrouwe f2ffb6d03e improves exponential smoothing of feerate estimates (#1699)
- fixes problem with polling interval > 150 * 0.9
- fixes log message 'feerate hit floor' at every feerate change
- smoothed fee now reaches 90% of (exp weighted) fee estimates polled in last
120s, independent of polling interval
- only apply smoothing when effect > 10 percent so it doesn't correct forever
- fix indentation
2018-07-15 18:30:43 +02:00
Rusty Russell e0c21debc2 pytest: fix sleep(1) in test_channel_persistence.
It's obviously not enough under load, as Travis demonstrates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-15 17:37:29 +02:00
Rusty Russell b1182702ae pytest: increase timeout for test_payment_failed_persistence
This happened under travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-15 17:37:29 +02:00
arowser 97118c558e add --conf parameter 2018-07-15 09:45:55 +00:00
Mark Beckwith cf12130627 params: shortened names
This is a cosmetic change only. No functional changes.

I shortened the names of macros and changed param_parse() to param().

Also went through params.h with a fine-toothed comb and updated the comments
to reflect the current API.

I wanted to change the files:

	params.c -> param.c
	params.h -> param.h
	run-params.c -> run->param.c

but that confused `git diff` for params.h so its best left for another PR.

I'm keeping #1682 updated locally with all these changes.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 23:44:50 +00:00
Mark Beckwith 891dee739d params: Fix arg size
Fixes #1668

Reported-by: @jsarenik

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 11:30:52 +00:00
Mark Beckwith 7ce19b0bc8 Improved comment.
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 11:21:39 +00:00
Rusty Russell cf86c74870 params: add helper to provide default initialization.
@wythe points out that many cases want a default value, not NULL.
Nicer to do it in the param_parse() call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-13 11:21:39 +00:00
gallizoltan 1220a4eb1d pylightning: add missing arguments to getroute 2018-07-13 13:12:49 +02:00
Christian Decker 14c6310a4f gossip: Fix concurrent PR merge issue with structeq
PR #1618 in parallel with the migration to macro `structeq` created this.

Fixes #1674
2018-07-08 19:04:46 +02:00
Jonathan Zernik ec55bf3b3f README.md: 18332 is default bitcoind rpc port for testnet. 2018-07-08 16:08:56 +02:00
Rusty Russell 68a8eeea21 htlc_wire: rename malformed to failcode in struct failed_htlc.
I'm not completely convinced that it's only ever set to a failcode
with the BADONION bit set, especially after the previous patches in
this series.  Now that channeld can handle arbitrary failcodes passed
this way, simply rename it.

We add marshalling assertions that only one of failcode and failreason
is set, and we unmarshal an empty 'fail' to NULL (just the the
generated unmarshalling code does).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell 5a184c24e8 channeld: add extra check to channel_force_htlcs.
None of these sanity checks should fail, but let's be thorough: we
were testing for htlc->fail but not failcode when fulfilling an HTLC.
The failing-htlc case had this correct already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell efee948d3a channeld: handle HTLCs failed by failcode uniformly.
'struct htlc' in channeld has a 'malformed' field, which is really only
used in the "retransmit updates on reconnect" case.  That's quite confusing,
and I'm not entirely convinced that it can only be set to a failcode
with the BADONION bit set.

So generalize it, using the same logic we use in the master daemon:

failcode: a locally generated error, for channeld to turn into the appropriate
          error message.
fail: a remotely generated onion error, for forwarding.

Either of these being non-zero/non-NULL means we've failed, and only one
should be set at any time.

We unify the "send htlc fail/fulfill update due to retransmit" and the
normal send update paths, by always calling send_fail_or_fulfill.

This unification revealed that we accidentally skipped the
onion-wrapping stage when we retransmit failed htlcs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell 8155bfcf18 channeld: make channel_fulfill_htlc return the HTLC it fulfulled.
This is the same pattern as channel_fail_htlc, and in fact one caller
wanted it already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell e92f244b80 channeld: hoist make_failmsg above send_fail_or_fulfill.
Move only.  Needed for next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell 6e9ae98e1e lightningd: don't send uninialized malformed fields to channeld.
==1224== Uninitialised byte(s) found during client check request
==1224==    at 0x152CAD: memcheck_ (mem.h:247)
==1224==    by 0x152D18: towire (towire.c:17)
==1224==    by 0x152DA1: towire_u16 (towire.c:28)
==1224==    by 0x142189: towire_failed_htlc (htlc_wire.c:29)
==1224==    by 0x16343F: towire_channel_init (gen_channel_wire.c:596)
==1224==    by 0x115C2C: peer_start_channeld (channel_control.c:249)
==1224==    by 0x131701: peer_connected (peer_control.c:503)
==1224==    by 0x117820: gossip_msg (gossip_control.c:182)
==1224==    by 0x139D97: sd_msg_read (subd.c:500)
==1224==    by 0x139676: read_fds (subd.c:327)
==1224==    by 0x179D52: next_plan (io.c:59)
==1224==    by 0x17A84F: do_plan (io.c:387)
==1224==  Address 0x1ffefffabe is on thread 1's stack
==1224==  in frame #2, created by towire_u16 (towire.c:26)

Followed by:

2018-06-18T21:53:04.129Z lightningd(1224): 03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134 chan #1: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d0101486543e1a8b6871556a4fe1fba4ad4d83ce7f6f92919fd17bd1545d2fd5: UpdateFailMalformedHtlc message doesn't have BADONION bit set

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell b14cc0c9f7 lightningd/params: fix typesafe check.
typesafe_cb isn't suitable here, as it is simply a conditional cast,
and the result is passed through '...' and doesn't matter.

Reported-by: @wythe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 23:49:25 +00:00
Rusty Russell dd7afc33ee pytest: fix flaky test in test_htlc_sig_persistence
We can shutdown before the tx hits bitcoind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell ed83bbe623 pytest: fix flaky race in test_gossip_query_channel_range.
We weren't waiting for gossipd to actually process the
dev_set_max_scids_encode_size message, so under Travis it sometimes
split the reply before processing that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell 57794b9285 gossipd: also delay locally-generated disables when peer vanishes.
Note that we mark both directions of the channel disabled immediately,
it's just the broadcast of the update which is delayed, just like the
ones generated when channeld tells us to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell f9b8237d50 gossipd: delay generation of local updates.
We disable the channel every time the peer disconnects; if it reconnects
we get two updates.

The simplest solution: delay all updates by 15 seconds.  Replace any
pending delayed update.  If update is redundant after 15 seconds,
discard.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 55a09d79b9 pytest: make test_gossip_no_empty_announcements robust when update delayed.
In this test we tell l3 to disconnect on sending WIRE_CHANNEL_ANNOUNCEMENT.
This is hit by gossipd (to disconnect from l2) but *also* channeld to
disconnect from l4.  That's OK, because normally by this point l4 has
sent its real channel_update.

However, the next patch introduces a delay in sending channel_updates,
meaning l4 hasn't sent it yet.  If l3 doesn't reconnect to l4, we
never get the channel_update and the test which expects l1 to eventually
see both sides of the channel fails.

So we manually reconnect then.  Note that we remove the redundant
'dev-no-reconnect' option from l2: it's added automatically as it
doesn't set 'may_reconnect'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell ef59a8f4aa gossipd: suppress redundant local updates which we would generate.
This doesn't do anything for us now, since we actually tend to produce
DISABLE/ENABLE update pairs.  But the infrastructure is useful for the
next patch.

We also add more details to the trace message in the core update code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 2d533dc82e channeld: don't manually disable channel.
gossipd will do it when peer dies anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 8e571ba688 listnodes: expose global features.
Since nobody sets these yet, it's a bit moot, but it will be great in
future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 9fa738a741 listpeers: expose peer features as 'local_features' and 'global_features'
For now, just the connected peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell 7b735fbeee gossipd: fix json_listpeers printing node information.
json_listpeers returns an array of peers, and an array of nodes: the latter
is a subset of the former, and is used for printing alias/color information.

This changes it so there is a 1:1 correspondance between the peer information
and nodes, meaning no more O(n^2) search.

If there is no node_announce for a peer, we use a negative timestamp
(already used to indicate that the rest of the gossip_getnodes_entry
is not valid).

Other fixes:
1. Use get_node instead of iterating through the node map.
2. A node without addresses is perfectly valid: we have to use the timestamp
   to see if the alias/color are set.  Previously we wouldn't print that
   if it didn't also advertize an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
William Casarin a9cf73380a cli: launch a manpage when help is passed an argument
example

  lightning-cli help pay
  lightning-cli help cli

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-07 05:01:31 +00:00
Christian Decker 13796f0d0f travis: Add xmllint to the builder image
Seems that it is required for the manpage conversion.
2018-07-05 00:24:33 +00:00
Mark Beckwith fc2d955b01 Fixed spacing in run-params
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith 7d9ad89010 params: removed the param_opt_tok macro
There doesn't seeem to be a need for this anymore (unless I'm missing something).
I added the sendpay_nulltok() unit test to confirm.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith 1b50ea2abd params: removed tal context.
@rustyrussell showed we don't need temporary objects for params.

This means params no longer need a tal context.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Rusty Russell 32ccfa5b29 test/run-params: suppress stderr for expected failures.
And use err() instead of perror/exit(0) (we should have exit(1) there anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 6ff901d7b0 params: simplify lifetimes of params.
@wythe points out we don't need to keep the around now param_is_set()
is removed.  We can in fact go further and avoid marshalling them into
temporary objects at the caller altogether.

This means internally we have an array of struct param, rather than an
array of 'struct param *', which causes most of the noise in this
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00