Commit Graph

305 Commits

Author SHA1 Message Date
Rusty Russell 511e8e6477 common/blindedpay: routines to construct a blinded payment.
Don't shoehorn it into onion_nonfinal_hop() and onion_final_hop(), but
provide an explicit routine "blinded_onion_hops" and an onion helper
"onion_blinded_hop()" for it to call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell f00cc23f67 sphinx: rename confusing functions, ensure valid payloads.
"sphinx_add_hop" takes a literal hop to include,
"sphinx_add_modern_hop" prepends the length.  Now we always prepend a
length, make it clear that the literal version is a shortcut:

* sphinx_add_hop -> sphinx_add_hop_has_length
* sphinx_add_modern_hop -> sphinx_add_hop

In addition, we check that length is actually correct!  This means
`createonion` can no longer create legacy or otherwise-invalid onions:
fix tests and update man page to remove legacy usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `createonion` no longer allows non-TLV-style payloads.
2022-09-28 13:40:57 +02:00
Rusty Russell f52ff07558 lightningd: allow delpay to delete a specific payment.
This is actually what the autoclean plugin wants, especially since
you can't otherwise delete a payment which has failed then succeeded.

But insist on neither or both being specified, at least for now.

Changelog-Added: JSON-RPC: `delpay` takes optional `groupid` and `partid` parameters to specify exactly what payment to delete.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell 2022e4a7a9 wallet: simplify payments lookup so sqlite3 uses index.
Filtering by status is rare, so we can do it in the caller; just let sqlite3
filter by payment_hash.

With ~650,000 payments in db:

Before:
```
129/300000 complete 5.60/sec (33078 invs, 169 pays, 0 retries) in 30 seconds. 19 hours-14 hours remaining.
201/300000 complete 7.20/sec (43519 invs, 241 pays, 0 retries) in 40 seconds. 16 hours-11 hours remaining.
257/300000 complete 5.60/sec (54568 invs, 289 pays, 0 retries) in 50 seconds. 16 hours-14 hours remaining.
305/300000 complete 4.80/sec (65772 invs, 337 pays, 0 retries) in 60 seconds. 16 hours-17 hours remaining.
361/300000 complete 5.60/sec (75875 invs, 401 pays, 0 retries) in 70 seconds. 16 hours-14 hours remaining.
```

After:
```
760/300000 complete 40.00/sec (19955 invs, 824 pays, 0 retries) in 20 seconds. 2 hours-2 hours remaining.
1176/300000 complete 41.60/sec (30082 invs, 1224 pays, 0 retries) in 30 seconds. 2 hours-119 minutes remaining.
1584/300000 complete 40.80/sec (40224 invs, 1640 pays, 0 retries) in 40 seconds. 2 hours-2 hours remaining.
1984/300000 complete 40.00/sec (49938 invs, 2048 pays, 0 retries) in 50 seconds. 2 hours-2 hours remaining.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell 2da5244e83 jsonrpc: make error codes an enum.
This allows GDB to print values, but also allows us to use them in
'case' statements.  This wasn't allowed before because they're not
constant terms.

This also made it clear there's a clash between two error codes,
so move one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: Error code from bcli plugin changed from 400 to 500.
2022-09-19 10:18:55 +09:30
Christian Decker cb3ee0ac2e wallet: Load and value `completed_at` timestamp from DB 2022-09-14 13:14:10 +02:00
Rusty Russell 054339e0cb lightningd: obey first hop channel id.
Reported-by: Warren Togami
Changelog-Changed: `sendpay` and `sendonion` now obey the first hop "channel" short_channel_id, if specified.
Changelog-Deprecated: `sendpay` and `sendonion` could take a bogus first hop "channel" short_channel_id
2022-08-09 16:33:24 -05:00
Rusty Russell 36a29fbfbc lightningd/json.h: remove.
There are hardly any lightningd-specific JSON functions: all that's left
are the feerate ones, and there's already a comment that we should have
a lightningd/feerate.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell 401f1debc5 common: clean up json routine locations.
We have them split over common/param.c, common/json.c,
common/json_helpers.c, common/json_tok.c and common/json_stream.c.

Change that to:
* common/json_parse (all the json_to_xxx routines)
* common/json_parse_simple (simplest the json parsing routines, for cli too)
* common/json_stream (all the json_add_xxx routines)
* common/json_param (all the param and param_xxx routines)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell 1217f479df sendpay: allow route to contain both amount_msat and (deprecated) msatoshi.
Since it's a deprecation, we simply ignore one, rather than properly
checking they match etc.

Fixes: #5386
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Christian Decker 692a001198 ld: Use the local alias when reporting failures with zeroconf
Ran into this with a zeroconf channel, without confs, that was
disconnected.
2022-07-04 22:14:06 +02:00
Rusty Russell 08d5776ebc lightningd: deprecate `msatoshi` in `sendpay` `route`.
We should be using amount_msat always.  Many tests were not.  Plus,
deprecating it simplifies the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSONRPC: `sendpay` `route` elements `msatoshi` (use `amount_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell cd7e784d6f lightningd: change `msatoshi` args to `amount_msat`.
This is consistent with our output changes, and increases consistency.
It also keeps future sanity checks happy, that we only use JSON msat
helpers with '_msat' fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument is now called `amount_msat` to match other fields.
Changelog-Deprecated: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice` `msatoshi` (use `amount_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell e47786da04 pay/sendpay: also store description in case bolt11 uses description_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-04 20:26:04 +09:30
Rusty Russell 7f89763f9e sendpay: add payment_metadata argument.
And document the missing arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 7abc491f4c pay: fix crash on invalid onion in reply in some cases.
fail can be NULL here.

Reported-by: https://github.com/shafemtol
Fixes: #5053
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:59:46 +10:30
Rusty Russell f7aba31448 lightningd: allow 'style' 'legacy' for clboss compat (with deprecated_apis).
We keep the parameter around for future use (e.g. PTLC support?), but clboss
master still sets 'style' to "legacy".  Accept, but ignore it.

Reported-by: grubman on #c-lightning (IRC)
Changelog-Deprecated: JSON-RPC: `sendpay` `route` argument `style` "legacy" (don't use it at all, we ignore it now and always use "tlv" anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 13:19:10 +10:30
Rusty Russell 21e1d68e3b lightningd: remove (most) functions to search channels by status.
This is generally verboten now, since there can be multiple.  There are a
few exceptions:

1. We sometimes want to know if there are *any* active channels.
2. Some dev commands still take peer id when they mean channel_id.
3. We still allow peer id when it's fully determined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `close` by peer id will fail if there is more than one live channel (use `channel_id` or `short_channel_id` as id arg).
2022-03-23 13:20:12 +10:30
Rusty Russell 43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell 5065bd6fc2 lightningd: use our cached channel_update for errors instead of asking gossipd.
We also no longer strip the type off: everyone handles both forms, and
Eclair doesn't strip (and it's easier!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell 967ffbfbcb global: use tal_dup_or_null().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
niftynei bddd3694fa coin_mvt: record fees for an outbound htlc
If we initialized the payment, the fees are the entire fee-chain
(final hop amount - starting hop amount)

If it's a payment we routed, the fees are the diff between the
inbound htlc and the outbound (net gain by this routing)

Added to database so data persists nicely.
2021-12-28 04:42:42 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Christian Decker 6196d2f559 pay: Fail a `sendpay` or `sendonion` that'd produce a DB collision 2021-10-13 13:41:18 +10:30
Christian Decker 818a4d14cb paycore: Default `groupid` to increment from last one
This re-establishes the prior behavior where a `sendpay` or
`sendonion` that'd match a prior payment would cause the prior payment
to be deleted. While we no longer delete prior attempts we now avoid a
primary key collision by incrementing once. This helps us not having
to touch all existing tests, and likely avoids breaking other users
too.
2021-10-13 13:41:18 +10:30
Christian Decker ec9040577e paycore: Prevent multiple concurrent payment groups
One of the fundamental constraints of the payment groups idea is that
there may only ever be one group in flight at any point in time, so if
we find a group that is in flight, any new `sendpay` or `sendonion`
must match its `groupid`.
2021-10-13 13:41:18 +10:30
Christian Decker 39248b5fa7 pay: Do not delete old sendpay attempts if we retry
This was the main cause of the pay states flip-flopping, since we
reset the status on each attempt any final status is not really
final. Let's keep them around, and provide a stable history.
2021-10-13 13:41:18 +10:30
Christian Decker 5f260840ab jsonrpc: Add `groupid` to `waitsendpay` 2021-10-13 13:41:18 +10:30
Christian Decker 817b1b361e db: Add `groupid` to HTLCs 2021-10-13 13:41:18 +10:30
Christian Decker a1f509e4b5 jsonrpc: Add groupid to `sendpay` and `sendonion`
Also add `groupid` to the payment fields so we can retrieve them too.
2021-10-13 13:41:18 +10:30
Vincenzo Palazzo fd33aed4b5 rpc: Integrate the status flow in the listsendpays command
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Rusty Russell 7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 7e7ab4cb3b gossipd: remove routing, listchannels and listnodes infrastructure.
This involves removing some fields from the now-misnamed routing.h
datastructures, and various internal messages.

One non-obvious change is to our "keepalive" logic which refreshes
channels every 13 days: instead of using the 'enabled' flag on the
last channel broadcast to decide whether to refresh it, we use the
local connected status directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell a4df2c7697 lightning/pay: don't crash when trying to send payment down unconfirmed channel.
An "active" channel may still be CHANNELD_AWAITING_LOCKIN, so have ->scid NULL.

You can only trigger this by trying to sendpay to the node using a
manual route, since routing would never use such a channel.

```
lightningd: FATAL SIGNAL 11 (version v0.10.0-319-g81cbc20-modded)
0x55e79d194e17 send_backtrace
	common/daemon.c:39
0x55e79d194ec1 crashdump
	common/daemon.c:52
0x7fce2d79920f ???
	???:0
0x7fce2d8e16f7 ???
	???:0
0x55e79d2019eb tal_dup_
	ccan/ccan/tal/tal.c:801
0x55e79d14e1d9 immediate_routing_failure
	lightningd/pay.c:365
0x55e79d14fe91 send_payment_core
	lightningd/pay.c:1022
0x55e79d150995 send_payment
	lightningd/pay.c:1180
0x55e79d151975 json_sendpay
	lightningd/pay.c:1462
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:09 +09:30
fiatjaf 28953256d0
stop requiring channel and direction on `sendonion` since they're not used.
Changelog-Changed: `sendonion` no longer requires the gratuitous `direction` and `channel` fields in the `firsthop` parameter.
2021-06-04 10:48:10 +09:30
fiatjaf e424180058 remove checking for unused "direction" property in sendpay. 2021-06-03 13:25:59 +09:30
fiatjaf 64199d99fd "createonion" to accept an optional custom onion_size.
Changelog-Added: `createonion` RPC command now accepts an optional `onion_size`.
2021-05-24 12:52:19 +02:00
Rusty Russell e531a38963 gossipd / plugin: clean up names in struct route_hop.
We're going to unify them, but the names are not the normal ones.

Fix that first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 1a85edd207 lightningd: no longer forward failures to gossipd, let caller do it.
We fix up the test by using pay, instead of sendpay (and making pay log
the expected message).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: sendpay no longer extracts updates from errors, the caller should do it from the `raw_message`.
2021-02-02 13:44:01 +01:00
Rusty Russell 406eb37717 listsendpays: remove deprecated "null" amount_msat.
Changelog-Removed: `listsendpays` will no longer add `amount_msat` `null` (deprecated 0.9.1).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-01 09:57:35 +10:30
Rusty Russell 001b5d6416 offers: make it a runtime option.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
2021-01-13 14:45:36 +01:00
Rusty Russell a15878b3a4 sendonion: remove underscores from JSON arg name.
I fixed the "sendpay" one but not this one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
fiatjaf 6af8adde26 fix memset for route_hop json parsing when no channel_id exists.
Changelog-None
2021-01-04 11:24:55 +01:00
Rusty Russell 723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell a33e39b7e8 pay, wallet: rename internal `bolt11` vars to `invstring`.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 3f4683e3f8 sendpay: optional argument to link local offer.
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer.  We mark it used when it's paid, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 32c7c133f4 common/sphinx: make onionpacket.routinginfo a dynamic member.
Still asserts that it's the standard size, but makes it a dynamic
member.  For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell 924cc04bd2 bolt11: have caller supply preferred chain.
This lets us distinguish testnet from signet invoices, since they
have the same prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 09:24:14 +09:30