Commit Graph

4884 Commits

Author SHA1 Message Date
Rusty Russell dfaaa09bc6 common/cryptomsg: remove unused 'reading_body' flag and 'peer_in_started'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 74ab2125af connectd: cleamn up broken resolver detection code.
We only call it once, so don't free the "old" one.  And fix some indenting.
And make hostname const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 6dbf1ef2c4 connectd: remove unused timers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell c9001f653a connectd: reorder functions for clarity.
And remove the unused TESTING ifdef, otherwise MOVEONLY.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell a90c200fcb pytest: wait_channel_quiescent helper to wait for resolved htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell 9fa7f5e30e listpeers: include current htlc information.
This enables the next patch, which allows us to wait until all HTLCs are
completely resolved.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell 704f8b5b0e pytest: make test_autocleaninvoice more reliable.
We're about to slow down the invoice rpc (esp. under valgrind), which
breaks the delicate timing of the autocleaninvoice test.

Change that so the autocleaner (and timestamp) starts after the invoices
are added.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell 96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Christian Decker d590302523 pytest: Fix benchmarks after the fixture migration 2018-09-27 00:18:24 +00:00
Rusty Russell 59bdba6bac pytest: don't test crashing under valgrind at all.
Travis failures:

valgrind: m_scheduler/sema.c:104 (vgModuleLocal_sema_down): Assertion 'sema->owner_lwpid != lwpid' failed.
host stacktrace:
==1296==    at 0x38083F48: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38084064: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380841F1: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38135DAE: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380D328D: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809A4AC: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809AE43: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380988CF: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
sched status:
  running_tid=0
Thread 1: status = VgTs_WaitSys (lwpid 1296)
==1296==    at 0x5729730: __poll_nocancel (syscall-template.S:84)
==1296==    by 0x4348DF: daemon_poll (daemon.c:78)
==1296==    by 0x4169E7: io_poll_lightningd (lightningd.c:543)
==1296==    by 0x471ECD: io_loop (poll.c:282)
==1296==    by 0x416E06: main (lightningd.c:744)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 16:32:08 -07:00
lisa neigut dfe0378464 git: ignore vim's c-tags 'tags' file 2018-09-26 06:29:14 +00:00
Rusty Russell e450c6bbdb gossipd: remove time-delayed local channel_update, produce DISABLE on-demand.
We have a lot of infrastructure to delay local channel_updates to
avoid spamming on each peer reconnect; we had to keep tracking of
pending ones though, in case we needed the very latest for sending an
error when failing an HTLC.

Instead, it's far simpler to set the local_disabled flag on a channel
when we disconnect, but only send a disabling channel_update if we
actually fail an HTLC.

Note: handle_channel_update() TAKES update (due to tal_arr_dup), but we
didn't use that before.  Now we do, add annotation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell 16e16a725e gossipd: apply private updates to announce channel.
We trade channel_update before channel_announce makes the channel
public, and currently forget them when we finally get the
channel_announce.  We should instead apply them, and not rely on
retransmission (which we remove in the next patch!).

This earlier channel_update means test_gossip_jsonrpc triggers too
early, so have that wait for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell 66105e83ea gossipd: simplify "broadcast channel_announcement now we have channel_update" logic
It's simpler and more robust to just check that it's not yet announced
(the broadcast index will be 0).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Mark Beckwith eee31a1519 git: ignore all current and future unit tests
I wanted to add run-secret_eq_consttime and run-find_my_abspath to the
.gitignore file, but instead I replaced all the names of tests with a simple
regex.

This works because git doesn't ignore files that are already in git. So
source and header file are safe. It doesn't ignore *.c files that haven't been
added to git either.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 18:36:26 -07:00
Mark Beckwith cbde3e20f7 cli: help command now also prints usage
The help command now adds command usage to its output by calling each
command handler in CMD_USAGE mode.

Instead of seeing, for example:

	decodepay
	    Decode {bolt11}, using {description} if necessary

we see:

	decodepay bolt11 [description]
	    Decode {bolt11}, using {description} if necessary

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Mark Beckwith 1a4f355a7b param: add ok flag to struct command
Callers to param() can now optionally set a flag to see if command_fail was
called.

This is necessary because the `cmd` is freed in case of failure.

I spent a bit of time trying to extend the lifetime of the `cmd` to the end
of parse_request(), but the destructors still needed to be called when they
were, and it was getting ugly.  So I took this minimal approach.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Mark Beckwith 30b67c0334 param: call param() all the time
Now call param() even for commands that don't accept any parameters.

This is a bugfix of sorts.  For example, before you could call:

	bitcoin-cli getinfo blah

and the blah parameter would be ignored.

Now you will get an error: "too many parameters: got 1, expected 0"

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Mark Beckwith d91b94a812 param: add command mode
Added the concept of a "command mode".  The
behavior of param() changes based on the mode.

Added and tested the command mode of CMD_USAGE for
setting the usage of a command without running it.

Only infrastructure and test.  No functional changes.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Christian Decker c3f433ec66 json: Support streaming JSON messages
It turns out we were heavily relying on the fact that after each message from
the client there'd be a flush, and that there would not be anything after the
JSON object we read. This will no longer be the case once we start streaming
things or we are very quick in issuing the JSON-RPC requests.

This just takes one of the error paths (incomplete read) and makes it into a
successful path if we have indeed read a full root element.
2018-09-25 05:47:23 +00:00
Rene Pickhardt fc12f65a3d Invoiceapidoc patch
Added the fallback address to the API documentation of the invoice command
2018-09-25 05:21:00 +00:00
lisa neigut 3ce53ab9ed tools/gen-wire: 3th -> 3rd
Small grammar fix in wire gen tool
2018-09-22 19:06:06 +02:00
Rusty Russell 8455b12781 Revert "gossipd: handle premature node_announcements in the store."
This reverts commit e2f426903d.

With the new store version, this can't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 48de77d56e gossipd: invalidate old gossip_stores.
Incrementing version number means stores which were prior to the previous
commit will be removed, and refreshed.  The simplest fix, if not the most
efficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 0855422110 gossip_control: when searching for a txout, make sure it's not spent!
There's no reason for the db to ever return non-NULL if it's spent.  And there's
only one caller, for which that is definitely true.

Suggested-by: @cdecker
Fixes: #1934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 24c386c086 test_gossip: gossip retransmit on spent UTXO test.
Aka #1934.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 6228c25643 test_gossip: basic 'node notices close' test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell d16c3dcdc7 devtools/decodemsg: take series of msgs from stdin.
Useful in combination with gossipwith.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 47f5bc4deb gossipwith: add ability to send message.
Just cmdline for now, rather than a proper stdin io loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 0925daa087 gossipwith: simple tool to snarf gossip from a node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 1b80cb4269 Makefile: don't rebuild all the time
The code to regenerate the local BOLT copy was causing eternal rebuild.

So only build if it's wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:55:51 +02:00
Rusty Russell edb74260f4 pytest: more reliable onchain tests.
We extract the tx from the logs, and then we wait until that hits
the mempool.  This is more reliable than 'sendrawtx' in the logs,
which might catch a previous sendrawtx; it's also more explicit
that we expect that tx exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 00:28:44 +00:00
lisa neigut b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
lisa neigut b1f15c2605 BOLT updates: broken link fixes
See a9195a84d0
2018-09-21 00:24:12 +00:00
lisa neigut 73ea6d0038 BOLT 2 updates for fix placment of chain_hash req
See 4b62d26af9
2018-09-21 00:24:12 +00:00
lisa neigut b287f2f007 BOLT 11 human-readable formatting changes 2018-09-21 00:24:12 +00:00
lisa neigut 2327f04b54 HACKING.md: Update rec'd test cmd to `full-check`
Otherwise you won't run `check-source` and your Travis build will
fail on you.

Also remove comment about how it's currently disabled cuz that's a lie.
2018-09-21 00:21:20 +00:00
Rusty Russell 5b2e829b4f hsmd: remove master pointer.
We used to use it to complain about bad requests, but we use the status conn
now, so it's unused except for tests and asserts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 019bc4fcd7 hsmd: reorder functions (MOVEONLY).
We don't need to pre-declare any more, but I left it in the previous
patch for review simplicity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell b0769d9c0c hsmd: don't use daemon_conn for clients.
It offers them a DoS vector, if they don't read the replies.  We really want
to use raw ccan/io so we can avoid buffering for this.

It makes the handing of fds for new clients a bit more complex
(callback based), but it's not too bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 573f2f065a hsmd: document as part II of our journey.
Thanks greatly to the four people who I *know* have read this:

	@wythe, @ZmnSCPxj, @SimonVrouwe, and @cdecker

Your feedback will help future developers seeking enlightenment!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 04c77f4853 lightningd: use hsm_get_client_fd() helper for global daemons too.
We couldn't use it before because it asserted dbid was non-zero.  Remove
assert and save some code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>



Header from folded patch 'fixup!_lightningd__use_hsm_get_client_fd()_helper_for_global_daemons_too.patch':

fixup! lightningd: use hsm_get_client_fd() helper for global daemons too.

Suggested-by: @ZmnSCPxj
2018-09-20 09:49:39 +02:00
Rusty Russell cc48e794c9 hsmd: extract and use common sign_all_inputs() helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 8f1f1784b3 hsmd: remove hsmd/client.c
It was only used by handshake.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell f8df069536 hsmd: move HTLC TX signing next to commitment TX signing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 6b6b7eac61 hsmd: use status_conn to send bad_request messages, make handlers uniform.
The current code sends hsmstatus_client_bad_request via the req fd;
this won't work, since lightningd uses that synchronously and only
expects a reply to its commands.  So send it via status_conn.

We also enhance hsmstatus_client_bad_request to include details, and
create convenience functions for it.  Our previous handling was ad-hoc;
we sometimes just closed on the client without telling lightningd,
and sometimes we didn't tell lightningd *which* client was broken.

Also make every handler the exact same prototype, so they now use the
exact same patterns (hsmd *only* handles requests, makes replies).

I tested this manually by corrupting a request to hsmd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell da9d92960d lightningd: accept hsmstatus_client_bad_request messages (and log!)
We currently just ignore them.  This is one reason the hsm (in some places)
explicitly calls log_broken so we get some idea.

This was the only subdaemon which had a NULL msgcb and msgname, so eliminate
those checks in subd.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 1e4e476c9e hsmd: implement bitcoin_key() to subsume bitcoin_pubkey and bitcoin_keypair.
This mirrors the node_key() interface we already have.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 3e63d88ad1 hsmd: rename per-peer to per-channel.
And remove cut&paste of derive_peer_seed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00