As side-effect, getroute(0) is special too.
Reported-by: MiddleW4y in Discord
Fixes: #6577
Changelog-Fixed: `pay` will still use an invoice routehint if path to it doesn't take 1-msat payments.
We have a report that LND said our (unannounced) channel was disabled, so we didn't
use it for routehints. We're better off ignoring that in this case (if the peer is
actually not connected, the routehint code will check that and ignore anyway).
Fixes: #6555
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pay: use channels in routehints even if peer says they're "disabled" (LND compat)
Compiler can't tell that we always set have_state[PAY_FLOW_FAILED_FINAL]
when we set this:
```
plugins/renepay/payment.c: In function ‘payment_reconsider’:
plugins/renepay/payment.c:287:25: error: ‘final_error’ may be used uninitialized [-Werror=maybe-uninitialized]
287 | payment_fail(payment, final_error, "%s", final_msg);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
plugins/renepay/payment.c:194:30: note: ‘final_error’ was declared here
194 | enum jsonrpc_errcode final_error, ecode;
| ^~~~~~~~~~~
plugins/renepay/payment.c:287:25: error: ‘final_msg’ may be used uninitialized [-Werror=maybe-uninitialized]
287 | payment_fail(payment, final_error, "%s", final_msg);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
plugins/renepay/payment.c:195:21: note: ‘final_msg’ was declared here
195 | const char *final_msg;
| ^~~~~~~~~
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We cannot carry pointers into the gossmap across localmod addition
or removal.
We didn't notice because the map->chan_arr is not normally resized,
but if we change gossmap.c line 689 to only allocate 1 to start, we see this:
```
VALGRIND=1 valgrind -q --error-exitcode=7 --track-origins=yes --leak-check=full --show-reachable=yes --errors-for-leak-kinds=all plugins/renepay/test/run-mcf > /dev/null
==2349744== Invalid read of size 4
==2349744== at 0x1788C2: gossmap_chan_scid (gossmap.c:558)
==2349744== by 0x1872A2: get_chan_extra_half_by_chan (flow.c:346)
==2349744== by 0x187797: remove_completed_flow (flow.c:488)
==2349744== by 0x187927: remove_completed_flow_set (flow.c:518)
==2349744== by 0x18DF4D: main (run-mcf.c:393)
==2349744== Address 0x4b80f38 is 88 bytes inside a block of size 136 free'd
==2349744== at 0x4848C63: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2349744== by 0x173D71: tal_resize_ (tal.c:744)
==2349744== by 0x177E36: next_free_chan (gossmap.c:336)
==2349744== by 0x177ED3: new_channel (gossmap.c:351)
==2349744== by 0x178441: add_channel (gossmap.c:458)
==2349744== by 0x1798D4: gossmap_apply_localmods (gossmap.c:904)
==2349744== by 0x18DEDB: main (run-mcf.c:388)
==2349744== Block was alloc'd at
==2349744== at 0x4848C63: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2349744== by 0x173D71: tal_resize_ (tal.c:744)
==2349744== by 0x177E36: next_free_chan (gossmap.c:336)
==2349744== by 0x177ED3: new_channel (gossmap.c:351)
==2349744== by 0x178441: add_channel (gossmap.c:458)
==2349744== by 0x178B6D: map_catchup (gossmap.c:635)
==2349744== by 0x178F45: load_gossip_store (gossmap.c:697)
==2349744== by 0x179D71: gossmap_load (gossmap.c:978)
==2349744== by 0x18D22F: main (run-mcf.c:295)
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not there if it's a local error:
```
{
"code": 202,
"message": "Parsing '{message:%,data:{erring_index:%,failcode:%,raw_message:': object does not have member raw_message"
}
```
Reported-by: https://github.com/daywalker90Fixes: #6553
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Unifies the pay_flow resolve functions, and moves remove_htlc_payflow
and commit_htlc_payflow to the top.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We want to make sure that on every path, we terminate the flow. The simplest
way to do this is encourage the pattern "return pay_flow_xxx(flow)".
Indeed, this caught a few places I missed!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The main function here is payment_reconsider:
* Each payment has a list of pay_flow.
* This is populated in try_paying(), calling add_payflows & sendpay_new_flows.
* When we get a notification, we resolve a pay_flow using one of the pay_flow_failedxxx
or pay_flow_succeeded functions.
* They call payment_reconsider() which cleans up finished flows decides what to do:
often calling try_paying again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not required, but it should be there so we might as well use it
(though we sometimes don't put one in, esp if it's a private channel).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids a gratuitous "ctx" field, and the simplified declaration
is now understood by `make update-mocks`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Treat it just like "PAY_TRY_OTHER_ROUTE", except it is from the final node:
this means we correctly process that it "succeeded".
Add a test: this crashes sometimes, but it's cleaned up soon...
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As recommended by your TODO, a bit simpler: we also make the hash function
return a ptr rather than the (now rather large) struct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Use json_scan(), and use the new pay_flow_from_notification() routine.
Also, the tal_dup_or_null can be tal_dup, since &preimage is never NULL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There are a few fields in `struct renepay` which are genuinely
transient, but it makes the code much harder to follow than simply
having a single structure.
More cleanups will follow, but this is the minimal set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The general pattern for xxx_new is that it should populate all
fields, for encapsulation and so you never can have a half-formed
object.
This means a fair bit of work for now, but it pays off in the next
patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You cannot refresh the gossmap with localmods applied, nor apply localmods
when others have applied localmods in the same process.
There are optimizations we could do, but for now always apply/unapply before
querying gossmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We've stomped errno, so if exec fails we don't get a reliable result:
```
2023-08-07T17:58:45.713Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
See: https://github.com/bitcoindevkit/bdk/issues/1047#issuecomment-1660645669
In general, futures produced by most libraries in the ecosystem of Rust, and bounds placed
on users of famous runtimes like tokio and its spawn method all lack Sync requirements.
Because of this, anyone who creates a callback using any sort of library that returns a
non-Sync future (which most libraries fit this description) inside of it will get some
cryptic error messages (async error messages still leave a lot to be desired).
Removing these Sync requirements will make the library more useful.
Fixes 32-bit builds:
```
In file included from plugins/renepay/pay.c:5:
./plugins/renepay/pay_flow.h: In function 'fmt_payflow_key':
./plugins/renepay/pay_flow.h:54:17: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'u64' {aka 'long long unsigned int'} [-Werror=format=]
54 | "key: groupid=%ld, partid=%ld, payment_hash=%s",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
55 | k->groupid,k->partid,
| ~~~~~~~~~~
| |
| u64 {aka long long unsigned int}
```
etc
It's expressed in bits, but really it's clearer as a quantity, given
how it's used.
Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.
Implement the channeld splicing protocol leveraging the interactivetx protocol.
Implement lightningd’s channel_control to support channeld in its splicing efforts.
Changelog-Added: Added the features to enable splicing & resizing of active channels.
Update gossip routiens and various other hecks on the channel state to consider AWAITING_SPLICE to be routable and treated similar to CHANNELD_NORMAL.
Small updates to psbt interface
Changelog-None
- adopt "const <type> *"convention
- remove use_shadow option for some pyln tests
- show prob. information of flows into paynotes
- show prob. of success of entire payment flow in paynotes
- minflow: We were not releasing the memory of flow arrays when replacing
them with a new canditate.
- use memleak_scan_obj in memleak_check
- replace u64 with size_t
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
The global is an *internal* hack because dijkstra_item_mover doesn't
take a context arg! It should be used with care.
Easy, since all the accessors exist: we just hand in the struct dijkstra.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- remove internal gheap checks
- add check for arc_t.chanidx overflow
- remove outdated comments
- check the delta flow bounds before augmenting along a path
- get_flow_paths uses a dynamic tal array instead of a list.
- fix a unit test that depended on the order of returned flows
- fix bug: lightnind doesn't like if I reuse the partid of a failed
flow, therefore use a higher partid than any of the previous attempts.
- plugin_err instead of LOG_BROKEN if sendpay fails and we cannot get a
an error code.
- fix wrong comments.
- remove the background timer.
- This is a bugfix. Previous to this the MCF network was built using the
knowledge of the min and max liquidity but it didn't take into account
pending HTLCs.
- Also remove the min_prob_success option but hardcode a 90% value.
Removing some options that are not relevant to the user, they're kept
for developer mode only:
- base_fee_penalty
- min_prob_success
- prob_cost_factor
- remove heap.h, not used
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
This way unreserving the PSBT will work as intended, and we don't have
to keep track of how many times we've called reserved for any one input.
Technically we're supposed to not reserve inputs at *all* while doing
opens, this moves us slightly closer to that.
The alias may not be set for non-alias channels after they
confirm. The other branch is safe because we only consider active
channels.
Changelog-None
Fixes#6450
Caught by leak detection, we just re-assigned this when we retried: sure,
it's temporary, but it's technically a leak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I shut down bitcoind during a test, and bcli leak reports flooded in.
They're all temporary, but this fixes them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is the simplest solution, not the best, but there's significant risk in try to remove the "we have a path" assumption in the code pay code.
Includes removing a `tal_steal` which was incorrect: the buffer has the same lifetime as the plugin, so if we steal it then things get messy when we free the struct payment.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `pay` will now pay your own invoices if you try.
This means (temporarily) that blacklisting won't work (fix later), and
means that old-style (commando.py) master-secret-override doesn't work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugins: `commando` no longer allows datastore ['commando', 'secret'] to override master secret (re-issue runes if you were using that!).
In preparation for going async:
1. Split try_command's tail into a new function called execute_command() after
the rune checks have succeeded.
2. Put all the info execute_command() needs into struct cond_info, to make it
a simple callback style.
So we create new_cond_info() which dynamically allocates `struct cond_info`
and sets the destructor.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We would create a `struct commando` to marshal our incoming messages,
then try_command would create a *new* one. We can simply reuse, but
when I did I noticed a trick: the new one was not in the `incomings`
array, so didn't work towards the ratelimit. So we need to remove it
from `incomings` in `try_command`, but at least it's now explicit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to activate on the first rune creation, but we're no longer in charge
of runes, so we can't make that call.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Debugging a number of payments showed that we sometimes waste a number
of attempts routing through a channel via its alias, rather than its
scid. This is because while we annotate the scid when it has been set,
we do not do so for the alias. The alias then is picked for routing
despite not having enough capacity, failing the attempt locally.
It can also happen that we alternate between scid and alias, doubling
the number of failed attempts before we can make progress.
This patch sets the hint for the alias to a capacity of 0 and disables
it as if the peer were offline. This means when available we'll always
use the scid, which is also far easier to read in the logs.
Changelog-Fixed: pay: We now track spendable amounts when routing on both the local alias as well as the short channel ID
The presplitter modifier would split a payment before trying the first
attempt based on some common sizes. Its goal was to have smaller parts
in flight over different paths, in order to make it more difficult for
a forwarding node to learn payment amount. However it was causing some
issues for direct payments, and estimates on spendable amounts which
considers only the first HTLC being added, but presplitter would
always cause multiple HTLCs to be kicked off, causing the estimate to
be off.
Removing the presplitter fixes this, making draining channels easier,
and worse success rates, due to more HTLCs in flight directly
impacting the changes of getting stuck.
Changelog-Removed: pay: `pay` no longer splits based on common size, as it was causing issues in various scenarios.
This is almost always true already; fix up the few non-standard ones.
This is enforced with an assert, and I ran the entire test suite to
double-check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously, our code checked for the presence of the `lightning:`
prefix while decoding a bolt11 string. Although this prefix is valid
and accepted by the core lightning pay command, it was causing issues
with how we managed invoices. Specifically, we were skipping the prefix
when creating a copy of the invoice string and storing the raw invoice
(including the prefix) in the database, which caused inconsistencies
in the user experience.
To address this issue, we need to strip the `lightning:` prefix before
calling each core lightning command. In addition, we should
modify the invstring inside the db with the canonical one.
This commit fixes the issue by stripping the `lightning:` prefix
from the `listsendpays` function, which will improve the
user experience and ensure consistency in our invoice management (see
next commit).
Reported-by: @johngribbin
Link: ElementsProject#6207
Fixes: debbdc0
Changelog-Fixes: trim the `lightning:` prefix from invoice everywhere.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
We usually have access to `ld`, so avoid the global.
The only place generic code needs it is for the json command struct,
and that already has accessors: add one for libplugin and lightningd
to tell it if deprecated apis are OK.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.
Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we're opening a channel with a peer which support anchors (and
we do), we tell fundpsbt/utxopsbt to enforce the emergency reserve;
this matters, as it doesn't know about the channel yet, and thus
won't (if it's our first anchor channel).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `fundchannel` and `multifundchannel` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (or are opening one).
This was added to fundpsbt/utxopsbt in v0.10, but the spender plugin
didn't take advantage of it, instead calculating its own change amount
and output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was added to fundpsbt/utxopsbt in v0.10, but the txprepare plugin
didn't take advantage of it, instead calculating its own change amount
and output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were marking our inputs very late, which means any early failure
would not know to unreserve them.
This becomes particularly bad when we start enforcing emergency reserves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
mfc->feerate_str is *never* NULL, since we set it in getfeerate; this is
confusing, as many places check for NULL.
Indeed, the logic in perform_fundpsbt() was *wrong* in this case: it used
`normal` (if it was NULL, which it never was) instead of `opening` to fundpsbt.
And the correct thing is for multifundchannel to not use a string here at
all, but to use the exact feerate it is counting on (even the same
string may have different values now if a block has come in).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we didn't hash the descriptions properly (see previous commit), we
cannot immediately deprecate omitting the descriptions (since you'd
have to omit them for backwards compat!).
And move the "must have description or hash" test into bolt11.c core.
Changelog-Deprecated: `pay` has *undeprecated* paying a description-hash invoice without providing the description.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we need to push off requring this for another full deprecation cycle!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `pay` and `decodepay` with description now correctly handle JSON escapes (e.g " inside description)
I added a plugin arg and was surprised that compile didn't break.
This is because typesafe_cb et al are conditional casts: if the type
isn't as expected it has no effect, but we're passing plugin_option() through
varargs, so everything is accepted!
Add a noop inline to check type, and fix up the two cases where we
used `const char *` instead of `char *`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
junderw
ShahanaFarooqui
Dusty Daemon
Lagrang3
niftynei
Christian Decker
Vincenzo Palazzo
Greg Sanders