Commit Graph

643 Commits

Author SHA1 Message Date
Christian Decker 6b7546b94d json-rpc: Rename `getroutestats` and move stats to getinfo
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker facd7d16aa json-rpc: Add `listforwardings` command 2018-10-19 21:58:27 +00:00
Christian Decker 6efdd5275c pytest: Test that we record forwardings correctly
Adapts the `test_forward_stats` test to include checks for the
`forwarded_payments` table. Will add checks for the `listforwardings`
RPC call next.

Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Rusty Russell 09b33015c4 connectd: give user a hint when wrong key is used.
When the wrong key is used, the remote end simply hangs up.

We used to get a random errno, which tends to be "Operation now in progress."
Now it's defined to be 0, detect and provide a better error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 00:53:00 +00:00
lisa neigut 0ae1d03513 BOLT7: broadcast `htlc_maximum_msat` in `channel_update s
Have c-lightning nodes send out the largest value for
`htlc_maximum_msat` that makes sense, ie the lesser of
the peer's max_inflight_htlc value or the total channel
capacity minus the total channel reserve.
2018-10-16 03:32:27 +00:00
Rusty Russell 030fe1ce53 gossipd: don't expose private channels for routeboost.
We don't create unannouncable channels, but other implementations can.
Not only is it rude to expose these via invoices, it's probably not
useable anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-15 23:04:17 +00:00
Christian Decker a44491fff0 benchmark: Add two benchmarks for forwarded payments
We weren't benchmarking them, so do that now.
2018-10-13 10:58:53 +00:00
Christian Decker 81f5b89fe3 wallet: Actually load wallet stats when asked to
The call to `sqlite3_step` is actually needed, otherwise we'll always
get the default values for all types.
2018-10-13 10:58:53 +00:00
Christian Decker 2354b3e706 json-rpc: Create test showcasing the null-accounting issue
There is an issue with the way we retrieve the channel accounting info
that will result in always showing 0 for all stats. This tests for it
and the next commit will fix it.
2018-10-13 10:58:53 +00:00
William Casarin b3f7236201 python: add cheroot to requirements.txt
It looks like this is a new dependency.

Sort the list while we're at it.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-10-10 18:00:22 +02:00
Rusty Russell fcb5310873 pytest: make wait_for do exponential backoff, start at 0.25 seconds.
This doesn't alter runtime very much, but does reduce log spam.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-10 06:10:42 +00:00
Rusty Russell 84b9e3e72b lightningd: reduce log spam from bitcoin-cli invocations.
During tests, this is half our log!  And Travis truncates it if we get
a failure in test_restart_many_payments.

Interestingly, test_logging had a bug which relied on this spam :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-10 06:10:42 +00:00
Rusty Russell a4730089e8 pytest: make test_restart_many_payments more Pythony!
IIUC, namedtuple is like tuple for grown-ups: Pythonify!

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell d85251ac6c db: fix up HTLCs which are missing failure information.
We don't save them to the database, so fix things up as we load them.

Next patch will actually save them into the db, and this will become
COMPAT code.

Also: call htlc_in_check() with NULL on db load, as otherwise it aborts
internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 77be009354 pytest: add restart-during-n-way payment test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 65f6813706 lightningd: handle the case where the db contains a resolved HTLC without a preimage.
We need to handle this case (old db) before the next commit, which actually
fixes it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell c94ab7370c pytest: extend the test_fulfill_incoming_first case to cover reconnect.
Which we don't handle, due to a separate bug, so it's xfail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 4040c53258 lightningd: handle case where incoming HTLC vanished before fulfilled outgoing.
We now need an explicit 'local' flag, rather than relying on the existence
of the 'in' pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 3643e1bd90 pytest: add test for when an incoming fulfilled HTLC expires before outgoing.
Usually, we only close an incoming HTLC once the outgoing HTLC is completely
resolved.  However, we short-cut this in the FULFILL case: we have the
preimage, so might as well use it immediately (in fact, we wait for it to
be committed, but we don't need to in theory).

As a side-effect of this, our assumption that every outgoing HTLC has
a corresponding incoming HTLC is incorrect, and this test (xfail) tickles
that corner case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell b3fb23f921 pytest: make line_graph wait for gossip propagation if announce param is True
This is what the callers want; generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
lisa neigut a9bf1f5573 tests: quiet down DeprecationWarnings for escape sequences
Nuke all the `DeprecationWarning: invalid escape sequence
\[` messages that show up when you run python tests.
2018-10-08 13:18:31 +02:00
Christian Decker d6fcfe00c7 pytest: Stabilize test_feerates 2018-09-28 22:45:30 +00:00
Rusty Russell bb5e2ffafb gossipd: don't create redundant node_announcements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 6c54a22d63 pytest: make test_node_reannounce check for redundant announce (xfail).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell b0d9182fc3 pytest: allow log_all_io through get_nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 0baa5f7071 gossipd: send node announcement on startup.
I suspect this fixes #1660 too, but checking would be good.

Fixes: #1781
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell cbd1d1d0f2 pytest: test that we reannounce node after restart.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell c8c3294a23 json-invoice: add routeboost, warnings.
We split json_invoice(), as it now needs to round-trip to the gossipd,
and uniqueness checks need to happen *after* gossipd replies to avoid
a race.

For every candidate channel gossipd gives us, we check that it's in
state NORMAL (not shutting down, not still waiting for lockin), that
it's connected, and that it has capacity.  We then choose one with
probability weighted by excess capacity, so larger channels are more
likely.

As a side effect of this, we can tell if an invoice is unpayble (no
channels have sufficient incoming capacity) or difficuly (no *online*
channels have sufficient capacity), so we add those warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell 9455331575 json: use bolt naming for features arrays in listnodes, listpeers.
Deprecate the old names.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 3358437062 connectd: don't log every time a peer disconnects.
Great for a few of our tests, but generally spammy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell a90c200fcb pytest: wait_channel_quiescent helper to wait for resolved htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell 704f8b5b0e pytest: make test_autocleaninvoice more reliable.
We're about to slow down the invoice rpc (esp. under valgrind), which
breaks the delicate timing of the autocleaninvoice test.

Change that so the autocleaner (and timestamp) starts after the invoices
are added.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Christian Decker d590302523 pytest: Fix benchmarks after the fixture migration 2018-09-27 00:18:24 +00:00
Rusty Russell 59bdba6bac pytest: don't test crashing under valgrind at all.
Travis failures:

valgrind: m_scheduler/sema.c:104 (vgModuleLocal_sema_down): Assertion 'sema->owner_lwpid != lwpid' failed.
host stacktrace:
==1296==    at 0x38083F48: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38084064: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380841F1: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38135DAE: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380D328D: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809A4AC: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809AE43: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380988CF: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
sched status:
  running_tid=0
Thread 1: status = VgTs_WaitSys (lwpid 1296)
==1296==    at 0x5729730: __poll_nocancel (syscall-template.S:84)
==1296==    by 0x4348DF: daemon_poll (daemon.c:78)
==1296==    by 0x4169E7: io_poll_lightningd (lightningd.c:543)
==1296==    by 0x471ECD: io_loop (poll.c:282)
==1296==    by 0x416E06: main (lightningd.c:744)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 16:32:08 -07:00
Rusty Russell e450c6bbdb gossipd: remove time-delayed local channel_update, produce DISABLE on-demand.
We have a lot of infrastructure to delay local channel_updates to
avoid spamming on each peer reconnect; we had to keep tracking of
pending ones though, in case we needed the very latest for sending an
error when failing an HTLC.

Instead, it's far simpler to set the local_disabled flag on a channel
when we disconnect, but only send a disabling channel_update if we
actually fail an HTLC.

Note: handle_channel_update() TAKES update (due to tal_arr_dup), but we
didn't use that before.  Now we do, add annotation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell 16e16a725e gossipd: apply private updates to announce channel.
We trade channel_update before channel_announce makes the channel
public, and currently forget them when we finally get the
channel_announce.  We should instead apply them, and not rely on
retransmission (which we remove in the next patch!).

This earlier channel_update means test_gossip_jsonrpc triggers too
early, so have that wait for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Mark Beckwith cbde3e20f7 cli: help command now also prints usage
The help command now adds command usage to its output by calling each
command handler in CMD_USAGE mode.

Instead of seeing, for example:

	decodepay
	    Decode {bolt11}, using {description} if necessary

we see:

	decodepay bolt11 [description]
	    Decode {bolt11}, using {description} if necessary

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Rusty Russell 8455b12781 Revert "gossipd: handle premature node_announcements in the store."
This reverts commit e2f426903d.

With the new store version, this can't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 48de77d56e gossipd: invalidate old gossip_stores.
Incrementing version number means stores which were prior to the previous
commit will be removed, and refreshed.  The simplest fix, if not the most
efficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 0855422110 gossip_control: when searching for a txout, make sure it's not spent!
There's no reason for the db to ever return non-NULL if it's spent.  And there's
only one caller, for which that is definitely true.

Suggested-by: @cdecker
Fixes: #1934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 24c386c086 test_gossip: gossip retransmit on spent UTXO test.
Aka #1934.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 6228c25643 test_gossip: basic 'node notices close' test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell 0925daa087 gossipwith: simple tool to snarf gossip from a node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell edb74260f4 pytest: more reliable onchain tests.
We extract the tx from the logs, and then we wait until that hits
the mempool.  This is more reliable than 'sendrawtx' in the logs,
which might catch a previous sendrawtx; it's also more explicit
that we expect that tx exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 00:28:44 +00:00
lisa neigut b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
Rusty Russell da9d92960d lightningd: accept hsmstatus_client_bad_request messages (and log!)
We currently just ignore them.  This is one reason the hsm (in some places)
explicitly calls log_broken so we get some idea.

This was the only subdaemon which had a NULL msgcb and msgname, so eliminate
those checks in subd.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell e41e1a177e pytest: wait until mock is called for set_feerates.
Got a spurious failure in test_no_fee_estimate; we fired too soon from the logs (presumably
we raced in on the first response, but estimatesmartfee gets called 3 times).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:12:28 +02:00
Rusty Russell 252bbe1d2d pytest: don't wait for sendrawtx, wait for expected tx.
In particular, test_no_fee_estimate was flaky due to seeing the funding
tx being sent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:04:01 +02:00
Christian Decker f29f92a5fe pytest: Clean up bitcoind_cmd_override, it's no longer used 2018-09-16 00:05:34 +02:00
Christian Decker 9e5d7dacb0 pytest: Use the mock bitcoind everywhere 2018-09-16 00:05:34 +02:00