Commit Graph

152 Commits

Author SHA1 Message Date
Christian Decker c8dbcac38e pay: Use gossmap for reachability check 2021-04-06 11:03:47 +09:30
Christian Decker ea92466025 pay: Abort the payment if destination is unreachable
We were not aborting if we had routehints, even though all routehints
may have been filtered out.

Changelog-Fixed: pay: `pay` will now abort early if the destination is not reachable directly nor via routehints.
2021-03-11 14:09:51 +10:30
Christian Decker cb21a5384b pay: Remove routehints with unreachable entrypoints
Changelog-Added: pay: `pay` will now remove routehints that are unusable due to the entrypoint being unknown or unreachable.
2021-03-11 14:09:51 +10:30
Christian Decker 1e1d7b387c pay: Filter out routehints whose entrypoint is unknown
We would happily spin on attempts that are doomed to fail because we
don't know the entrypoint. Next up: remove routehints whose
entrypoints are known but unreachable.
2021-03-11 14:09:51 +10:30
Christian Decker 60af8be5ba libplugin: Introduce `payment_abort` to terminate a payment
The main responsibility of this new function is to mark a payment
process as terminated and set a reasonable error message, that will be
displayed to the caller. We also skip the remaining modifiers since
they might end up clobbering the message.
2021-03-11 14:09:51 +10:30
Christian Decker 9869083a79 plugin: Check that we have a non-empty route in libplugin
Changelog-Fixed: pay: Payments with an empty route (self-payment) are now aborted.
2021-02-10 15:17:38 +01:00
Rusty Russell 1a85edd207 lightningd: no longer forward failures to gossipd, let caller do it.
We fix up the test by using pay, instead of sendpay (and making pay log
the expected message).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: sendpay no longer extracts updates from errors, the caller should do it from the `raw_message`.
2021-02-02 13:44:01 +01:00
Rusty Russell 780fc25413 pay: inject channel updates from errors ourselves.
Cut & paste from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-02 13:44:01 +01:00
Christian Decker 4b2efd69d7 pay: Fix a crash when waitblockheight times out
Fixes #4309
Changelog-Fixed: pay: Fixed an issue where waiting for the blockchain height to sync could time out.
2021-01-08 09:19:24 +01:00
Rusty Russell 723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell a33e39b7e8 pay, wallet: rename internal `bolt11` vars to `invstring`.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell c5a2e64fd4 plugins/libplugin-pay: expose gossmap for plugin special effects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell 54c57e7495 libplugin-pay: don't expose bolt11 details.
When we support bolt12, this won't exist.  We only need min_final_cltv_expiry,
routes and features, so put them into struct payment explicitly.

We move the default final ctlv out to the caller, too, which is clearer.
e.g. keysend was using this value, but it was hard to tell.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Christian Decker 313976e2f4 pay: Cleanup the route applicability checks for channel hints
I previously mistyped the rather lengthy conditions for failures, so
let's dissect it into its smaller components and add rationale behind
the individual parts of the decision.
2020-11-17 14:04:19 +10:30
Christian Decker 4d6b4a0445 pay: Retry the route computation if we could not apply the chanhints
This adds a new state `PAYMENT_STEP_RETRY_GETROUTE` which is used to
retry just that one step, without spawning a completely new
attempt. It's a new state so that modifiers do not act on it twice.

Changelog-Fixed: pay: Improved the performance of the `pay` command considerably by avoiding conflicting changes to our local network view.
2020-11-17 14:04:19 +10:30
Christian Decker 544e110c96 pay: Add a pre-apply check to channel_hint updates
This allows us to atomically update all channel_hints and determine if
we had a collision and therefore should retry.
2020-11-17 14:04:19 +10:30
Christian Decker 83f57ac300 pay: Move the chanhint update up to payment_getroute
We were delaying the channel_hint update till after the `createonion`
call which gave us the same situation we had with concurrent
`getroute` calls. Now we update the hints as soon as the plugins have
had their say in the route construction. If we still fail, either
because a modifier changed the route causing the failure, or because
we interleaved the route computation for multiple parts, we reset the
attempt and retry inline (i.e., without creating a new sub-payment).

Notice that interleaved route computations now only happen if the
modifier makes an async call to some RPC or similar.
2020-11-17 14:04:19 +10:30
Rusty Russell cd5a93d0bd gossmap: fix reutrn of gossmap_xxx_has_feature, rename.
1. One place returned false instead of -1.
2. The names implied it returned a bool, and it doesn't.

Fix both, and curse C's loose typing a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 15:32:12 +10:30
Rusty Russell b470ae2c73 plugins/libplugin-pay: use gossmap.
This is a fairly direct translation.  Even so, it should be faster in
most cases, and and we can do more sophisticated things if we want.

This also handles disabled channels better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.
2020-10-21 08:58:34 +10:30
Rusty Russell eadf2c91fe libplugin-pay: incorporate gossip store.
So we can use this for routing determinations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
ZmnSCPxj jxPCSnmZ 0eb1e7e0ca plugins/libplugin-pay.c: Add new payee_incoming_limit to limit number of HTLCs based on payee connectivity.
Fixes: #3926

(probably)

Changelog-Fixed: pay: Also limit the number of splits if the payee seems to have a low number of channels that can enter it, given the max-concurrent-htlcs limit.
2020-09-10 16:50:52 +09:30
ZmnSCPxj jxPCSnmZ deced56344 plugins/libplugin-pay.c: Add facility to have paymods request lowering of the estimated max HTLCs. 2020-09-10 16:50:52 +09:30
ZmnSCPxj jxPCSnmZ d15717b576 plugins/libplugin-pay.c: Keep p->invoice->routes valid when the routehints paymod mutates it.
The routehints paymod shares the storage of the array d->routehints and
p->invoice->routes, but once it operates, it possibly leaves it as a stale
pointer to memory it used to have.

Since other paymods may be interested in the invoice details, including
the routehints in the invoice, we should ensure the p->invoice->routes
remains valid whenever we try mutating that array.
2020-09-10 16:50:52 +09:30
Rusty Russell 191355e0e7 pay: fix handling of legacy vs tlv encoding.
As revealed by the failure of tests in #3936, where we ended up trying
to send a partial payment using legacy style, we are not handling
style properly.

1. BOLT9 has features, so we can *know* that the destination supports
   MPP.  We may not have seen a node_announcement.
2. We can't assume that nodes inside routehints support TLV.
3. We can't assume direct peers support TLV.

The keysend code tried to fix this up, so I'm not sure that this caused
the issue in #3968, though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: `pay` will now make reliable multi-part payments to nodes it doesn't have a node_announcement for.
2020-09-10 16:50:32 +09:30
Rusty Russell 0c7d04bd98 libplugin-pay: fix default CLTV.
This was changed recently, but without a bolt quote, we didn't find
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 06:59:13 +09:30
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
ZmnSCPxj jxPCSnmZ 932709cad9 plugins/libplugin-pay.c: Micro-optimize start_block sampling.
Using `waitblockheight 0` is a very slightly faster query than `getinfo`.
Also, avoid querying blockheight for child payments (allow `waitblockheight`
paymod to provide the blockheight returned from the `waitblockheight`, and
just resample the starting blockheight from the parent).

Changelog-None: pointless micro-optimization
2020-08-28 16:40:27 +02:00
ZmnSCPxj jxPCSnmZ 05daa8e5f3 plugins/libplugin-pay.c: Micro-optimization of plugin_is_finished.
This was checked with `gcc -S -O2` to see how an optimized build
would compile the function.
The original code completed calls into each child (and the `.s`
file showed that GCC 9.x was not smart enough to do early-out).

This modification explicitly does early-out, and avoids call-return
stack overhead for the common case where a payment is an ancestor
of a long line of single-child payments due to retrying.

Changelog-None: pointless micro-optimization
2020-08-28 16:40:27 +02:00
Rusty Russell 496c0dd1e6 common/random_select: central place for reservoir sampling.
Turns out we can make quite a simple API out of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell 12d0d5c185 amount: cleanup usage.
We've got some recently-added primitives which help.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 18:16:28 +02:00
Christian Decker 21d87f7075 pay: Implement simple presplit fix for ludicrous amounts
This is the simplest possible fix: increase the target amount until we get
the desired number of parts, while still bucketizing payments together that
are in approximately the same size.

The current logic puts all payments that are in the range x < amount <= 16*x
in the same bucket, making them harder to distinguish.

Changelog-Fixed: pay: The `presplit` modifier now supports large payments without exhausting the available HTLCs.
2020-08-27 10:19:21 +09:30
ZmnSCPxj jxPCSnmZ 128adf0938 plugins/libplugin-pay.c: Round-robin routehints when splitting.
This improves the success rate of `test_mpp_interference_2`, though
still not quite up to the level that we can remove `@flaky` from it.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ c27d7a3110 plugins/libplugin-pay.c: Store the route description, and re-report on failure. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 6468616c02 plugins/libplugin-pay.c: Propagate local_id from parent to child payment object. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ d8678467fa plugins/libplugin-pay.c: Show routes being tried, also print updates to channel hints not just initial creations. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ d89c77c0ce plugins/libplugin-pay.c: Describe the bits of unrecognized failure codes. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 0d2d85ab5f plugins/libplugin-pay.c: Also print events that create new sub-payments. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 98583e84b5 plugins/libplugin-pay.c: Give cmd id and partid for each log message.
Changelog-None: internal debugging

Makes it easier to debug payments with tons of splits.
2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ f81611e551 plugins/libplugin-pay.c: Make sure blockheight disagreement does not prevent all future progress.
Blockheight disagreement is signalled with a permanent failure at the
end node, but is actually a transient failure.
2020-08-13 12:50:16 +02:00
Christian Decker 8769f9ed93 pay: Fix final TLV payload if not going through MPP modifiers
Reported-by: ZmnSCPxj
Signed-off-by: Christian Decker <@cdecker>

Changelog-Fixed: pay: Correct a case where we put the sub-payment value instead of the *total* value in the `total_msat` field of a multi-part payment.
2020-08-13 12:50:16 +02:00
ZmnSCPxj jxPCSnmZ 0279be1d13 plugins/libplugin-pay.c: Be less aggressive with advancing through routehints.
Only advance through routehints if no route was found at all, or if the
estimated capacity at the routehint is lower than the amount that we
have to send through the routehint.

Changelog-Fixed: pay: Be less aggressive with forgetting routehints.
2020-08-13 12:50:16 +02:00
Christian Decker 894c886bdd pay: Inherit payment label to all children 2020-08-12 19:10:48 +02:00
Christian Decker 0dcd974d97 pytest: Reproduce #3915 2020-08-12 19:10:48 +02:00
Vincenzo Palazzo 1521c29fcf listpays mod 1: add destination inside the response when bolt11 is null
Changelog-Added: JSON-RPC: `listpays` now lists the `destination` if it was provided (e.g., via the `pay` plugin or `keysend` plugin)
2020-08-09 16:03:03 +02:00
Rusty Russell fa829f23db amount: add amount_msat_scale, amount_msat_ratio, amount_{msat,sat}_div
It's not all that rare to do these operations, and requiring annotations
for it is a little painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
ZmnSCPxj jxPCSnmZ 094eac4e95 plugins/libplugin-pay.c: Properly handle exclusions for routehints with two hops or more.
Arguably a low-priority bug since no current node ever generates routehints longer
than one hop.

However, it is possible as an edge case, if the destination is directly accessible
*and* supports multiple channels, that we route through the destination, one of the
*other* channels it has not in the routehint, to the entry point, and then through
the routehint.

This change removes the risk of the above edge case.

Changelog-None: arguably a low-priority bug.
2020-08-03 15:20:29 +09:30
Christian Decker a456d08ad0 pay: Be less aggressive when estimating channel capacity
We'd previously take the failed attempt and estimate the failing channel's
capacity at 3/4 of the attempted amount, which is rather aggressive. This
reduces this aggressiveness to use the exact amount tried, but excluding on
equality. This still skips attempting the same route with the same amount, but
also permits attempts that are in the range [3/4, 1] of the failed attempt
amount to still be attempted.
2020-08-03 12:15:08 +09:30
Christian Decker d7cca0781d jsonrpc: Add `msatoshi` argument to `sendonion` to annotate
While not directly necessary, it still feeds the `listpays` result, and so we
should pass it along if we can, so we don't have to rely solely on the
`amount_sent` field, which includes the fees.

Reported-by: Rusty Russell <@rustyrussell>
2020-07-28 16:17:39 +09:30
Vincent 81fd552e84 plugins/pay: hand bolt11 arg to sendonion if we have one (i.e. for `pay`)
[ Extracted into standalone patch and comment added by RR ]
2020-07-27 13:11:14 +02:00
Christian Decker 15d1a190a0 pay: Remove duplicate message field
jsonrpc_stream_fail already adds a message field.
2020-07-24 11:35:49 +02:00