Commit Graph

7153 Commits

Author SHA1 Message Date
Rusty Russell 4270031d75 wallet: add per-channel feerate_state to the database.
The upgrade here is a bit tricky: we map the two values into the
feerate_state.  This is trivial if they're both the same, but if
they're different we don't know exactly what state they're in (this
being the source of the bug!).

So, we assume that the have received the update and not acked it,
as that would be the normal case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell d2673a4e6f channeld: remove changes_pending flags.
These used to be necessary as we could have feerate changes which
we couldn't track: now we do, we don't need these flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 24d54f98ad channeld: use fee_states internally.
This is an intermediary step: we still don't save it to the database,
but we do use the fee_states struct to track it internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell acb89debdd channeld: generate revoke_and_ack before telling lightningd about commitment.
The `channel_got_commitsig` we send the lightningd also implies we sent
the revoke_and_ack, as an optimization.  It doesn't currently matter,
since channel_sending_revoke_and_ack doesn't do anything important to the
state, but that changes once we start uploading the entire fee_states.

So now we move our state machine *before* sending to lightningd, in
preparation for sending fee_states too.

Unfortunately, we need to marshall the info to send before we
increment the state, as lightningd expects that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 8ffd9d570b channeld: cleanup: use the channel_feerate() accessor everywhere.
And also move it to initial_channel, so we can use it there.

This saves churn in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell b9d6e8bbaa common: add fee_states state machine for update_fee.
This uses the same state machine as HTLCs, but they're only
ever added, not removed.  Since we can only have one in each
state, we use a simple array; mostly NULL.

We could make this more space-efficient by folding everything into the
first 5 states, but that would be more complex than just using the
identical state machine.

One subtlety: we don't send uncommitted fee_states over the wire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell ec1505199e lightningd: fix failure message in waitsendpay with multi-part payments.
Bastien TEINTURIER <bastien@acinq.fr> writes:
> One thing I noticed but didn't investigate much: after sending the two
> payments, I tried using `waitsendpay` and it reported an error *208*
> (*"Never attempted payment for
> '98ee736d29d860948e436546a88b0cc84f267de8818531b0fdbe6ce3d080f22a'"*).
> 
> I was expecting the result to be something like: "payment succeeded for
> that payment hash" (the HTLCs were correctly settled).

Indeed, if you waitsendpay without specifying a partid, you are waiting
for 0, which may not exist.  Clarify the error msg.

Reported-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell edab0df611 lightningd: fix spurious "more than twice final" error.
Bastien TEINTURIER <bastien@acinq.fr> writes:
> It looks like the split on c-lightning side is quite limited at the moment:
> the only option is to split a payment in exactly its two halves,
> otherwise I get rejected because of the rule of overpaying more than
> twice the amount?

We only tested exactly two equal-size payments; indeed, our finalhop
test was backwards.  We only complain if the final hop pays more than
twice msat (technically, this test is still too loose for mpp: the
spec says we should sum to the exact amount).

Reported-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 7fb4efd98a lightningd: require payment_secret for MPP.
It makes sense, and it's been proposed for addition to the spec to
broad agreement:

	https://github.com/lightningnetwork/lightning-rfc/pull/712

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 326ceab8ad pytest: add more multi-part-payment tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 5d413e0f79 common: offer option_basic_mpp for EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 207689c274 plugins: listpays will now consolidate multi-part payments.
This won't usually be visible to the end-user, since the pay plugin doesn't
do multi-part yet (and mpp requires EXPERIMENTAL_FEATURES), but we're ready
once it does.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell f1bc0b21f1 plugins: listpays ignores pre-0.7.0 or manual sendpay payments w/ no bolt11.
The pay plugin has been supplying the bolt11 string since 0.7.0, so only
ancient "pay" commands would be omitted by this change.

You can create a no-bolt11 "sendpay" manually, but then you'll find it
in 'listsendpays'.

Changelog-Removed: JSON: `listpays` won't shown payments made via sendpay without a bolt11 string, or before 0.7.0.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 8e3234e67a lightningd: sew in htlc set.
The invoice_try_pay code now takes a set, rather than a single htlc, but
it's basically the same thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell c2b357b916 lightningd: wrap htlc replay in a database transaction.
AFAICT this only "worked" previously because replay htlc simply failed
them all (no peers are currently connected).  With upcoming changes
(foreshadowed by the comment) this is no longer true:

Attempting to prepare a db_stmt outside of a transaction: wallet/invoices.c:373
lightningd: FATAL SIGNAL 6 (version v0.7.3-188-g45b0af4-modded)
0x55b475590a73 send_backtrace
	common/daemon.c:41
0x55b475590b1d crashdump
	common/daemon.c:54
0x7f16c557b46f ???
	???:0
0x7f16c557b3eb ???
	???:0
0x7f16c555a898 ???
	???:0
0x55b475564c8f fatal
	lightningd/log.c:814
0x55b4755c3ed5 db_prepare_v2_
	wallet/db.c:605
0x55b4755c76b5 invoices_find_unpaid
	wallet/invoices.c:373
0x55b4755ce91c wallet_invoice_find_unpaid
	wallet/wallet.c:1990
0x55b47555861f invoice_check_payment
	lightningd/invoice.c:257
0x55b475557a7c htlc_add_set
	lightningd/htlc_set.c:112
0x55b47557b294 handle_localpay
	lightningd/peer_htlcs.c:332
0x55b47557c63c htlc_accepted_hook_callback
	lightningd/peer_htlcs.c:857
0x55b475585573 plugin_hook_call_
	lightningd/plugin_hook.c:118
0x55b47557c747 plugin_hook_call_htlc_accepted
	lightningd/peer_htlcs.c:882
0x55b47557ca3e peer_accepted_htlc
	lightningd/peer_htlcs.c:991
0x55b47557ffb9 htlcs_resubmit
	lightningd/peer_htlcs.c:2131
0x55b4755620f7 main
	lightningd/lightningd.c:801

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 36b1e884dd lightningd: implement htlc sets.
This isn't plumbed in yet, but the idea is that every htlc gets put
into a "set" and then we process them once the set is satisfied.  For
the !EXPERIMENTAL_FEATURES, the set is simply always size 1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 7f629e545e lightningd: split invoice check into separate function.
We now return the same error for various "does not match this
invoice", so it makes sense to encapsulate these checks.  We'll also
want to expose this for multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 8b129b439b lightningd: cleanup redundant args from handle_localpay
The cltv_expiry and payment_hash are in hin, so no need to hand them
in here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 0e4a30c635 doc: update experimental bolt version quotes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 280523e72a pytest: Add tests to make sure received onion is as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell b670b51fa2 waitsendpay: add partid arg.
We need to be able to wait for a unique payment, now payment_hash is not
always unique.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 63fffd41c8 configure: make partid payments only available with EXPERIMENTAL_FEATURES and payment_secret
Explicit #if EXPERIMENTAL_FEATURES check in case we enable them at different
times, but it requires a payment_secret since we put them in the same field.

This incidently stops it working on legacy nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell cd35835c5a sendpay/sendonion: add optional partid arg, finesse msatoshi argument.
msatoshi was used to indicate the amount the invoice asked for, but
for parallel sendpay it's required, as it allows our sanity check of
limiting the total payments in flight, ie. it becomes
'total_msat'.

There's a special case for sendonion, which always tells us the value is 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell ce4403d638 lightningd: change amount-in-flight check to be more nuanced.
We currently refuse a payment if one is already in flight.  For parallel
payments, it's a bit more subtle: we want to refuse if it we already have
the total-amount-of-invoice in flight.

So we get all the current payments, and sum the pending ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 618c390475 lightningd: share more code between sendpay and sendonion.
In particular, we're about to do surgery on the detection-of-previous-payments
logic, and we should not do this in two places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 12985331f7 htlcs: remove origin_htlc_id from htlc_out.
This is a transient field, so rework things so we don't leave it in
struct htlc_out.  Instead, load htlc_in first and connect htlc_out to
them as we go.

This also changes one place where we use it instead of the am_origin
flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 345ca9b122 db: add partid field to htlc_out.
This is in preparation for partial payments.  For existing payments,
partid is 0 (to match the corresponding payment).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 2d18c3a209 db: add partid, total_msat fields to payment entries.
This is in preparation for partial payments.  For existing payments,
partid is 0 (arbitrarity) and total_msat is msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Michael Schmoock 727d6a0415 doc: correct wording in changelog 2019-12-12 13:56:16 +01:00
Christian Decker 9660549b0c changelog: Update changelog for the 0.7.4 release 2019-12-12 00:15:23 +01:00
Christian Decker 5ca938015a tools: Fix changelog script to be case insensitive and support auth 2019-12-12 00:15:23 +01:00
Christian Decker d449423983 pyln: Bump the version of pyln-client 2019-12-12 00:15:23 +01:00
lisa neigut dd34cccbea hsmtool: fixup miscount on params
assuming argv is zero indexed, we should check that the argc count
is at least as big as the desired args
2019-12-11 20:41:09 +01:00
Rusty Russell d56513362a lightningd: don't always defer commitment_signed if we're not synced.
Because my node runs under valgrind, it can take quite a while to
sync; nodes tend to disconnect and reconnect if you block too long.

This is particularly problematic since we often update fees: when the
other side sends its commitment_signed we block.

In particular, this triggers the corner case we have where we
update_fee twice, disconnecting each time, and our state machine gets
confused (which is why we never saw this exact corner case before this
change in 0.7.3!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-11 16:20:50 +01:00
Vasil Dimov 38d88f1aff doc: fix commit-time formatting 2019-12-11 16:19:43 +01:00
darosior b4c18a10cf doc/lightningd-config: precise alias are 32 __bytes__ long.
And not 32 characters long, as a character (for example 🐨) can be 4
bytes long.

Reported-By: Vasil Dimov <@vasild>
2019-12-11 16:19:43 +01:00
darosior 2986854c88 doc/lightningd-config: correct misaligned RGB option paragraph 2019-12-11 16:19:43 +01:00
Christian Decker 3def5393d8 pytest: Stabilize the hsm_encryption test
Tailing the log when we expect the process to exit can be racy (in this case
we were waiting for the last line that'd be printed).
2019-12-11 16:18:34 +01:00
Christian Decker 6cc60f57a4 pytest: Parametrize test_deprecated_closing_compat for elements 2019-12-11 16:18:34 +01:00
Christian Decker 43dfb4dae7 pytest: Stabilize and parameterize closing_specified_destination
This test was doing a few contortions, so I disentangled it a bit and
parametrized it
2019-12-11 16:18:34 +01:00
Christian Decker 301b634699 pytest: Skip some tests in elements if paratrization is convoluted
These mainly use hard-coded addresses
2019-12-11 16:18:34 +01:00
Christian Decker 843846f4d5 pytest: Parametrize test_config_in_subdir for elements
Same as previous
2019-12-11 16:18:34 +01:00
Christian Decker d6a5b87395 pytest: Parametrize test_query_short_channel_id for elements
Same as the previous commit.
2019-12-11 16:18:34 +01:00
Christian Decker fc93787db9 pytest: Parametrize test_gossip_query_channel_range for elements
We were hardcoding the chainparams->chain_hash which caused the query to
return an empty result. By parametrizing the test we can make it work on
elements.
2019-12-11 16:18:34 +01:00
Christian Decker ff5f7b194f sphinx: Return the error in parse_onionpacket
As suggested by @niftynei here: https://github.com/ElementsProject/lightning/pull/3260#discussion_r347543999

Suggested-by: Lisa Neigut <@niftynei>
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2019-12-11 16:18:34 +01:00
Christian Decker e1b1f47c53 pytest: Avoid starting the btcproxy twice 2019-12-11 16:18:34 +01:00
Christian Decker 15f04d5fff pytest: Give details about which node exited with a return code != 0 2019-12-11 16:18:34 +01:00
Christian Decker 518c43a9c2 pytest: Remember a node's return code so we can check it later 2019-12-11 16:18:34 +01:00
Vasil Dimov cd30f76539 build: fix ./configure options' descriptions
The description of

--enable/disable-experimental-features was a bogus "Developer mode, good
for testing" and

--enable/disable-valgrind was "Valgrind binary to use for tests" which
gave the false impression that it should be set to something like
/usr/local/bin/valgrind whereas it is a boolean option.
2019-12-11 00:50:20 +01:00
darosior fcbd11f0c5 plugins/libplugin: hook support
Changelog-Added: plugins: libplugin now supports writing plugins which register to hooks
2019-12-09 16:18:28 +01:00