Commit Graph

4691 Commits

Author SHA1 Message Date
Rusty Russell 4146950496 lightningd: don't access htlc_in's failoutchannel on db restore.
failoutchannel tells us which channel to send an update for (specifically
for temporary_channel_failure); but we don't save it into the db.  It's
not even clear we should, since it's a corner case and the channel might
not even exist when we come back.

So on db restore, change such errors to WIRE_TEMPORARY_NODE_FAILURE
which doesn't need an update.

We also don't memset it to 0 in the normal case (we only access if it
failcode has the UPDATE bit set) so valgrind will trigger if we're
wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 79ebb8a92e db: save the failcode / failuremsg into db.
Now we can finally move the fixup code under COMPAT_V061, so it's only
for old nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 2293f4ce9e db: restore failuremsg/failcode from db.
We don't write it in there yet, so this change currently has no effect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell d85251ac6c db: fix up HTLCs which are missing failure information.
We don't save them to the database, so fix things up as we load them.

Next patch will actually save them into the db, and this will become
COMPAT code.

Also: call htlc_in_check() with NULL on db load, as otherwise it aborts
internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 77be009354 pytest: add restart-during-n-way payment test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 1526f024fb channeld: disable check for my_current_per_commitment_point.
Under stress, it fails (test_restart_many_payments, the next test).

I suspect a deep misunderstanding in the comparison code, will chase
separately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell ec74aadce2 lightningd: save outgoing HTLC's preimage to db.
We can now wrap the 'missing preimage' hack in COMPAT_V061.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 65f6813706 lightningd: handle the case where the db contains a resolved HTLC without a preimage.
We need to handle this case (old db) before the next commit, which actually
fixes it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell c94ab7370c pytest: extend the test_fulfill_incoming_first case to cover reconnect.
Which we don't handle, due to a separate bug, so it's xfail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 9ef67e50ff lightningd: don't leave htlc_out's in pointer dangling when htlc_in freed.
Now we know this can happen (see previous patch), we need to handle it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 4040c53258 lightningd: handle case where incoming HTLC vanished before fulfilled outgoing.
We now need an explicit 'local' flag, rather than relying on the existence
of the 'in' pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 3643e1bd90 pytest: add test for when an incoming fulfilled HTLC expires before outgoing.
Usually, we only close an incoming HTLC once the outgoing HTLC is completely
resolved.  However, we short-cut this in the FULFILL case: we have the
preimage, so might as well use it immediately (in fact, we wait for it to
be committed, but we don't need to in theory).

As a side-effect of this, our assumption that every outgoing HTLC has
a corresponding incoming HTLC is incorrect, and this test (xfail) tickles
that corner case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell b77906634e lightningd: even more HTLC consistency checking: check states.
This means we need to check when we've altered the state, so the checks
are moved to the callers of htlc_in_update_state and htlc_out_update_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell a516e26331 lightningd: beef up HTLC consistency checking.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 123713fbf5 lightningd: fix outstanding taken pointer.
lightningd: Outstanding taken pointers: lightningd/pay.c:243:channel_update

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell 0b1870908a Makefile: our first db-from-0.6.1 bugfix is coming.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell b3fb23f921 pytest: make line_graph wait for gossip propagation if announce param is True
This is what the callers want; generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell aa9e15edb0 options: don't end default name with '-' in non-DEVELOPER mode.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell de37586a97 gossipd: use riskfactor in getroute, not "1".
AFAICT, this was there in the original commit by @cdecker.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell d946e965a6 gossipd: test that fromwire from lightningd messages succeeds.
Also tiny drive-by cleanup for gossip_disable_local_channels to modern form.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell 864812019f gossipd: use tal_arr_expand instead of open-coding it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell 915ffe35ed gossipd: clean up getnodes handling.
globalfeatures should not be accessed if we haven't received a
channel_update.  Treat it like the other fields which are only
initialized and marshalled/unmarshalled if the timestamp is positive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell df27fc55af More renaming of gfeatures to globalfeatures.
Use the BOLT #1 naming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
William Casarin f2488fbe14 make: fix broken error message in bolt-check
It looks like a printf was missing for the bolt-check error message

Signed-off-by: William Casarin <jb55@jb55.com>
2018-10-09 08:39:53 +00:00
William Casarin 7c4b9c8a0d build: fix compile error on gcc ~7.3.0
It seems to be having a bit of trouble understanding the control flow to realize
it's not actually uninitialized.

Add an error handler after the switch in case we miss a real uninitialized error
in the future.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-10-09 05:56:03 +00:00
lisa neigut 45b07e7e89 jsonrpc: fix cppcheck warning for potential null pointer deref
cppcheck was failing with a warning for a null pointer deref at this
line, this makes it stop complaining (and lets make check-source
succeed)
2018-10-09 00:08:41 +02:00
lisa neigut 0ecbdf3421 makefile: don't print shellcheck to stdout 2018-10-09 00:08:41 +02:00
lisa neigut 9d496f9a74 hacking: install `cppcheck` and `shellcheck`
You need `cppcheck` and `shellcheck` installed to run `check-source`
successfully.
2018-10-09 00:08:41 +02:00
lisa neigut a9bf1f5573 tests: quiet down DeprecationWarnings for escape sequences
Nuke all the `DeprecationWarning: invalid escape sequence
\[` messages that show up when you run python tests.
2018-10-08 13:18:31 +02:00
Christian Decker d6fcfe00c7 pytest: Stabilize test_feerates 2018-09-28 22:45:30 +00:00
Rusty Russell bb5e2ffafb gossipd: don't create redundant node_announcements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell afc92dd757 gossipd: use array[32] not pointer for alias.
And use ARRAY_SIZE() everywhere which will break compile if it's not a
literal array, plus assertions that it's the same length.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 6c54a22d63 pytest: make test_node_reannounce check for redundant announce (xfail).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell b0d9182fc3 pytest: allow log_all_io through get_nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 76e7f8aa5c wireaddr: add wireaddr_eq.
We can't STRUCTEQ_DEF() it since addrlen is variable, so open-code it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 0baa5f7071 gossipd: send node announcement on startup.
I suspect this fixes #1660 too, but checking would be good.

Fixes: #1781
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell cbd1d1d0f2 pytest: test that we reannounce node after restart.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell f9fba1d7c6 doc/lightning-invoice: document warnings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell c8c3294a23 json-invoice: add routeboost, warnings.
We split json_invoice(), as it now needs to round-trip to the gossipd,
and uniqueness checks need to happen *after* gossipd replies to avoid
a race.

For every candidate channel gossipd gives us, we check that it's in
state NORMAL (not shutting down, not still waiting for lockin), that
it's connected, and that it has capacity.  We then choose one with
probability weighted by excess capacity, so larger channels are more
likely.

As a side effect of this, we can tell if an invoice is unpayble (no
channels have sufficient incoming capacity) or difficuly (no *online*
channels have sufficient capacity), so we add those warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell 2f667c5227 gossipd: routine to get route_info for known incoming channels.
For routeboost, we want to select from all our enabled channels with
sufficient incoming capacity.  Gossipd knows which are enabled (ie. we
have received a `channel_update` from the peer), but doesn't know the
current incoming capacity.

So we get gossipd to give us all the candidates, and lightningd
selects from those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell f64eee717d gossipd: make helpers const-correct.
Always be const if you can.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell 89fbae4198 gossip_msg: marshal functions for struct route_info.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell 95c9a73fbb gossipd: set sent flag when sending reply_short_channel_ids_end
Otherwise, if we don't announce the last node, we'll not flush this
out; it will be delayed until the next time we send gossip!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 14:39:25 +02:00
Rusty Russell fbb7bafc3b gossipd: don't include channel in query_short_channel_ids reply if no channel_update.
This is consistent: we don't broadcast a channel_announce until we've seen
a channel_update, so we probably shouldn't advertise it here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 14:39:25 +02:00
Rusty Russell 9455331575 json: use bolt naming for features arrays in listnodes, listpeers.
Deprecate the old names.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 41b0872f58 Use localfeatures and globalfeatures consistently.
That's what BOLT #1 calls them; make it easier for people to grep.

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 15e8801285 connectd: fixes as suggested by @niftynei.
I split the peer_connected() function into the peer_reconnected(),
which is basically an entire separate path from the rest of
peer_connected().

Also, removed unused TAKEN annotation from `id` parameter.  Nobody actually
hands us take() there, and just as well, since we don't take it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell c3ffa6b9aa connectd: Code documentation part III.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 3358437062 connectd: don't log every time a peer disconnects.
Great for a few of our tests, but generally spammy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 89bac0d516 connectd: don't look for broken resolver if DNS disabled.
It does leak some information, in theory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00