Commit Graph

255 Commits

Author SHA1 Message Date
niftynei 679a473f9a fundpsbt: add option to filter out wrapped p2sh inputs
We need to be able to only use non-wrapped inputs for v2/interactive tx
protocol.

Changelog-Added: JSONRPC: `fundpsbt` option `nonwrapped` filters out p2sh wrapped inputs
2023-02-04 15:31:16 +10:30
Rusty Russell 611795beee listtransactions: get rid of per-tx type annotations.
We didn't actually populate them properly, and the real annotations
are on inputs and outputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `listtransactions` `channel` and `type` field removed at top level.
2023-01-30 15:15:41 -06:00
Rusty Russell 02d7454226 db: add invoicerequests table.
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly.  Create a new table for them, and
associated functions.

The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
2022-11-09 15:08:03 +01:00
Rusty Russell 8720bbedae common/onion: split into decode and encode routines.
Some places (e.g. the pay plugin) only need to construct onions,
not decode them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Christian Decker adf14151fa wallet: Use boolean to determine whether an output is coinbase 2022-11-09 11:55:25 +01:00
niftynei 26f5dcd2a5 wallet: mark coinbase outputs as 'immature' until spendable
Changelog-Changed: JSON-RPC: `listfunds` now lists coinbase outputs as 'immature' until they're spendable
Changelog-Changed: JSON-RPC: UTXOs aren't spendable while immature
2022-11-09 11:55:25 +01:00
Rusty Russell 8771c86379 common/onion: expunge all trace of different onion styles.
In particular, remove special routines to pull length: it's there,
take it and check it yourself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell c8ad9e18a9 common/onion: remove all trace of legacy parsing.
We still have an "enum forward_style" for the database, where old-style
forwards can still exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer forward HTLCs with legacy onions.
2022-09-28 13:40:57 +02:00
Rusty Russell 68f15f17bb delforward: allow deletion of "unknown in_htlc_id" and fix autoclean to use it.
Note the caveats: we will delete *all* of them at once!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-27 14:42:03 +09:30
Rusty Russell f52ff07558 lightningd: allow delpay to delete a specific payment.
This is actually what the autoclean plugin wants, especially since
you can't otherwise delete a payment which has failed then succeeded.

But insist on neither or both being specified, at least for now.

Changelog-Added: JSON-RPC: `delpay` takes optional `groupid` and `partid` parameters to specify exactly what payment to delete.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell 3079afb024 lightningd: add `delforward` command.
Changelog-Added: JSON-RPC: `delforward` command to delete listforwards entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell 7420a7021f lightningd: add `listhtlcs` to list all the HTLCs we know about.
Using `listfowards` for this wrong; expose this directly if people
care (and unlike listforwards, which could be deleted, we have to
remember these while the channel is still open!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listhtlcs` new command to list all known HTLCS.
2022-09-22 15:19:46 +02:00
Rusty Russell 311807ff1f lightningd: add in_htlc_id / out_htlc_id to listforwards.
And document that we never know payment_hash.

Changelog-Added: JSON-RPC: `listforwards` now shows `in_htlc_id` and `out_htlc_id`
Changelog-Changed: JSON-RPC: `listforwards` now never shows `payment_hash`; use `listhtlcs`.
2022-09-22 15:19:46 +02:00
Rusty Russell 2022e4a7a9 wallet: simplify payments lookup so sqlite3 uses index.
Filtering by status is rare, so we can do it in the caller; just let sqlite3
filter by payment_hash.

With ~650,000 payments in db:

Before:
```
129/300000 complete 5.60/sec (33078 invs, 169 pays, 0 retries) in 30 seconds. 19 hours-14 hours remaining.
201/300000 complete 7.20/sec (43519 invs, 241 pays, 0 retries) in 40 seconds. 16 hours-11 hours remaining.
257/300000 complete 5.60/sec (54568 invs, 289 pays, 0 retries) in 50 seconds. 16 hours-14 hours remaining.
305/300000 complete 4.80/sec (65772 invs, 337 pays, 0 retries) in 60 seconds. 16 hours-17 hours remaining.
361/300000 complete 5.60/sec (75875 invs, 401 pays, 0 retries) in 70 seconds. 16 hours-14 hours remaining.
```

After:
```
760/300000 complete 40.00/sec (19955 invs, 824 pays, 0 retries) in 20 seconds. 2 hours-2 hours remaining.
1176/300000 complete 41.60/sec (30082 invs, 1224 pays, 0 retries) in 30 seconds. 2 hours-119 minutes remaining.
1584/300000 complete 40.80/sec (40224 invs, 1640 pays, 0 retries) in 40 seconds. 2 hours-2 hours remaining.
1984/300000 complete 40.00/sec (49938 invs, 2048 pays, 0 retries) in 50 seconds. 2 hours-2 hours remaining.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Christian Decker cb3ee0ac2e wallet: Load and value `completed_at` timestamp from DB 2022-09-14 13:14:10 +02:00
Rusty Russell 15751ea1b8 lightningd: do inline parsing for listforwards status parameter
We can do this now the function is cleaned up.

Always better to do the work inside param() since then `check`
gets the benefit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell e96eb07ef4 lightningd: test that hsm_secret is as expected, at startup.
If you get the wrong hsm_secret, your node_id will change, and
peers won't know who you are, bitcoind will reject your transaction
signatures, and other madness.

Catch this as soon as it happens, by storing our node_id in the db.

Suggested-by: @cdecker, @fiatjaf
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `lightningd` will refuse to start with the wrong node_id (i.e. hsm_secret changes).
2022-07-20 19:28:33 +09:30
Rusty Russell e47786da04 pay/sendpay: also store description in case bolt11 uses description_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-04 20:26:04 +09:30
Rusty Russell 2f7f7ec255 lightningd: add "style" to listforwards.
Suggested-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` has new entry `style`, currently "legacy" or "tlv".
2022-03-31 19:38:05 +10:30
Rusty Russell aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Ken Sedgwick e8f43ef6ca wallet: make wallet_can_spend non-static because needed 2021-12-14 11:24:52 +10:30
Rusty Russell 484222b0a1 daemons: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Simon Vrouwe 63bd569bf6 lightningd: cleanup, freeing jsonrpc in shutdown cannot trigger db write's anymore
since PR #3867 utxos are unreserved by height, destroy_utxos and
related functions are not used anymore so clean them up also

However free(ld->jsonrpc) still needs to happen before free(ld) because its
destructors need list_head pointers from ld
2021-11-30 13:34:44 +10:30
Rusty Russell da03985996 wallet: only hand onchaind the HTLCs it needs to know.
This will make closing long-lived channels more efficient, and it's
just nicer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell c503232cde common: use bitcoin_outpoint.
I started pulling this thread, and the entire codebase got unravelled.

Oh well, it's done now!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell 8a85bf6880 lightningd: populate min/max commit fields in db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell bdaec48400 wallet: wrap htlc_state enum in db function.
All enums in the db should be wrapped this way on reading/writing them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Christian Decker 5f260840ab jsonrpc: Add `groupid` to `waitsendpay` 2021-10-13 13:41:18 +10:30
Christian Decker 817b1b361e db: Add `groupid` to HTLCs 2021-10-13 13:41:18 +10:30
Christian Decker 0bb0d8a707 wallet: Add function to retrieve the latest groupid for a payment
When doing things like `waitsendpay` without specifying the `groupid`
we likely want to use the latest `groupid` we created, since that's
the one in flight. This adds a function to quickly retrieve that.
2021-10-13 13:41:18 +10:30
Christian Decker ba4be0313b db: Add groupid to the payments table 2021-10-13 13:41:18 +10:30
Vincenzo Palazzo fd33aed4b5 rpc: Integrate the status flow in the listsendpays command
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Rusty Russell ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell fe86c117d9 datastore: turn keys into arrays
After some discussion with @shesek, and my own usage, we agreed that
a more comprehensive interface, which explicitly supports grouping,
is desirable.

Thus keys are now arrays, with the semantic that a key is either a
parent or has a value, never both.

For convenience in the JSON schema, we always return them as arrays,
though we accept simple strings as arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 533571a655 datastore: add generation, simple atomicity.
We add a generation counter, and allow update or del conditional
on a given generation.

Formalizes error codes, too, since we have more now.

Suggested-by: @shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 432508e65e datastore: allow replace/append.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 2fb8735f04 wallet: add datastore access routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 66af5f8a28 chaintopology: tell gossipd that channels no longer exist on reorg.
This actually caused the flake in test_funding_reorg_private, where
l1 and l2 might not mark the original channel disabled.  In fact, they
should *remove* it as it gets reorged out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:38 +09:30
Rusty Russell 44c469d52b disableoffer: fix disabling of already-used offers.
Turns out we didn't actually test this at all, and next commit does :(

    offer_status_in_db: 4 is invalid
    lightningd: FATAL SIGNAL 6 (version v0.10.0-459-g48fbd45-modded)
    0x5608cd360855 send_backtrace
	common/daemon.c:39
    0x5608cd3608ff crashdump
	common/daemon.c:52
    0x7f9af1dae20f ???
	???:0
    0x7f9af1dae18b ???
	???:0
    0x7f9af1d8d858 ???
	???:0
    0x5608cd30a47e fatal
	lightningd/log.c:819
    0x5608cd3430c5 offer_status_in_db
	wallet/wallet.h:1424
    0x5608cd34f1f3 wallet_offer_disable
	wallet/wallet.c:4494
    0x5608cd33ae2e json_disableoffer
	lightningd/offer.c:256
    0x5608cd3038fc command_exec
	lightningd/jsonrpc.c:643

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
niftynei 86f7a179a9 channel utxo: persist the 'csv' lock value to database
Channel leases modify the CSV height that an output is eligible for
being spent at,  persist this to the database
2021-07-20 13:28:38 -04:00
Rusty Russell efe29c9db8 sendpsbt, fundpsbt, utxopsbt, reserveinputs, unreserveinputs: allow custom number of blocks to reserve.
Not an API break: reserve=true|false still works for fundpsbt and utxopsbt,
but we also allow a raw number in there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-26 15:08:01 +09:30
niftynei d04c373283 rbf: when a channel is open, remove all the inflights
The channel's open has been mined, we don't need to keep all of these around
now.
2021-05-24 12:17:48 +09:30
niftynei 36f3b13279 inflight: add a 'channel-inflight' concept, in-progress channel stuffs
"inflights" are all potential channel funding transactions that we
currently have commitment transactions for.
2021-03-03 12:28:22 +10:30
Karol Hosiawa 8a8f81175d Added in_channel,out_channel,state params to listforwards 2021-03-03 09:19:20 +10:30
Rusty Russell 874ca99c32 offers: make 'used' flag more useful.
We used to only set it for single-use offers (where it's required),
but it's still interesting for multi-use offers, so let's keep it
there.

We also put this field in the documentation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 14:44:03 +10:30
Christian Decker 8e908ff652 Revert "wallet: Remove in-memory utxoset filter"
This reverts commit c239a7161b.

The goal of c239a716 was to reduce the memory footprint of our
internal UTXO set tracking, and testing against the sqlite3 backend
showed no performance impact. We have since found that the added
roundtrips to the DB server with postgres was having a considerable
performance impact, and backups would also bloat due to the increased
number of queries.

Undoing this change skips the noop updates that were causing this
regression.

Changelog-Fixed: db: Fixed a performance regression during block sync, resulting in many more queries against the DB than necessary.
2021-01-08 12:08:32 +10:30
Rusty Russell 723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell a33e39b7e8 pay, wallet: rename internal `bolt11` vars to `invstring`.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 4c4288e3e5 invoice: handle bolt12 strings if EXPERIMENTAL_FEATURES.
The database still calls them `bolt11`, but we treat them depending on
prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 3f4683e3f8 sendpay: optional argument to link local offer.
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer.  We mark it used when it's paid, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00