Commit Graph

4150 Commits

Author SHA1 Message Date
Rusty Russell af971fd025 wait forwards: add in_htlc_id
Without this, we have no unique identifier for which forward happened.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell f2162bf202 lightningd: add ordering and pagination to listforwards.
Changelog-Added: JSON-RPC: `listforwards` new parameters `index`, `start` and `limit`.
2023-10-28 15:48:26 +10:30
Rusty Russell 1d8af90b56 listforwards: add created_index and updated_index fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` fields `created_index` (old: `id`) and `updated_index`.
2023-10-28 15:48:26 +10:30
Rusty Russell de1e459196 forwards: hook into the wait subsystem.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `wait` now works for `forwards` infrastructure.
2023-10-28 15:48:26 +10:30
Rusty Russell e58ae31947 lightningd: hook forwards into the wait system.
This table doesn't have `id`, except as the implicit one in Sqlite3,
so we need to add it for postgres.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell c4f2ada2ff lightningd: split up peer_htlcs a little, create forwards.c
peer_htlcs has become a bit of a dumping ground: move listforwards
etc to its own file.

Also move `struct channel_info` from peer_htlcs.h to channel.h where
it more logically belongs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell c70a326f1f lightningd: add ordering and pagination to listsendpays.
Changelog-Added: JSON-RPC: `listsendpays` new parameters `index`, `start` and `limit`.
2023-10-28 15:48:26 +10:30
Rusty Russell 00ac1a9475 listsendpays: add created_index and updated_index fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sendpay`, `listsendpays`, `delpay` new fields `created_index` (old: `id`) and `updated_index`.
2023-10-28 15:48:26 +10:30
Rusty Russell 9ec6ac9922 lightningd: don't log that we're trying to pay if we're not.
Looking through logs I was surprise to see:

```
lightningd-1 2023-10-26T03:42:36.824Z INFO    lightningd: Sending 200000000msat over 1 hops to deliver 200000000msat
```

On a re-payment where we simply returned from sendpay immediately!  Move that log to later.
2023-10-28 15:48:26 +10:30
Rusty Russell f65c3cab75 wallet: use standard-style iterators for payments.
We used to have "unsaved" payments: now we don't we can use
our normal "iterator" pattern rather than returning arrays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 89cda26164 lightningd: rename struct sendpay_command to waitsendpay_command.
It used to be used for both `sendpay` and `waitsendpay` but now it's
only for the latter, so the name is confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 2689a4ed76 wallet: don't expose wallet_payment constructor, use wallet_add_payment.
Have it construct and return.  No need to expose details about dbid...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 52b1ba2992 wallet: remove premature optimization when making payment.
We didn't write to db immediately, but waited until it the actual HTLC got
added (or failed).  That way we didn't have a separate transaction to
write the payment into the db, but the complexity is not worth it: it
makes the next refactors harder, since we can't use the normal
iterator patterns like we do with the rest of the db (as we have to add
the unstored ones).

We might as well also make sendpay return immediately: we used to return
once the HTLC had been confirmed sent, since we entered it in the db
at that point, but we can keep it simple now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell c2d89e4624 sendpays: hook into the wait subsystem.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `wait` now works for `sendpays` infrastructure.
2023-10-28 15:48:26 +10:30
Rusty Russell ccb8d4b687 sendpays: add to wait subsystem.
Adding an index means:

1. Add the new subsystem, and new updated_index field to the db, and
   create xxx_index_deleted/created/updated APIs.
2. Hook up these functions to the points they need to be called.
3. Add index, start and limit fields to the list command.
4. Add created_index and updated_index into the list command.

This does #1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell f8e036b22d channeld: use anchors on peer's commitment(s) if we can't broadcast our own.
This means refactoring out some of the generic anchor info, from the
per-commitment-tx info (we can have at least two, perhaps more with
splicing!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell 484deb94fa lightningd: save peer's commitment anchor info into db.
We actually only need to remember the last two, worst case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell e609bc934e channeld: tell lightningd about local anchor for each commitment tx.
It's going to want to remember these, in case it encounters peers'
commitment tx and needs to boost it with CPFP on the anchor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell 1d4ae91d2c channeld: get rid of unused commit_sig and htlc_sigs in channeld_sending_commitsig
We don't actually use this at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Chris Guida 54ef84e891 display paid_outpoint on invoices when present
Changelog-Added: JSON-RPC: `listinvoices` new field `paid_outpoint` if an invoice is paid onchain.
2023-10-26 19:11:17 +10:30
Chris Guida 8c03dd8161 mark invoice as paid when we detect an on-chain payment to a fallback 2023-10-26 19:11:17 +10:30
Chris Guida 8bbf2342f7 lightningd: adapt invoice_try_pay declaration for onchain payment.
i.e. `set` may be NULL, so we need to explicitly hand the amount.
We also add an outpoint.
2023-10-26 19:11:17 +10:30
Chris Guida e3e0abc73b add autogenerated fallbacks (not custom fallbacks) to invoice_fallbacks table 2023-10-26 19:11:17 +10:30
Chris Guida cdc0bd47ff track fallbacks for invoices, pt 1 2023-10-26 19:11:17 +10:30
Chris Guida fc4b2f864e add invoices-onchain-fallback config option and warnings
Changelog-Added: Config: `invoices-onchain-fallback` to automatically add an onchain p2tr address to invoices, and allow that for payment.
2023-10-26 19:11:17 +10:30
Ken Sedgwick 44798e298c hsmd: implement the hsmd outpoint check
Tihis commit is implementing a 2-phase commit between
the signer the node and the peer.

The main reason for this is that everybody must agree on the lock,
otherwise one of them will want N signatures (on the splice candidates),
and another will produce only 1 signature.

check_outpoint is the "prepare" for the signer, and lock_outpoint is the
"commit". if check_outpoint returns true, lock_outpoint must not fail.

Link: https://github.com/ElementsProject/lightning/issues/6722
Suggested-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-26 16:42:47 +10:30
Rusty Russell a50b795bc2 lightningd: implement `recover` command.
Changelog-Added: JSON-RPC: `recover` command to force (unused) lightningd node to restart with `--recover` flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell 7f18ed7743 lightningd: allow --recover to take a 64-char hex string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `--recover` can take a 32-byte hex string, as well as codex32.
2023-10-26 12:59:55 +10:30
Rusty Russell 3b622f06c3 lightningd: use param_check on all commands which do extra checks.
This makes `check` much more thorough, and useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` now does much more checking on every command (not just basic parameter types).
2023-10-26 12:59:55 +10:30
Rusty Russell c490be62f0 lightningd: preapproveinvoice doesn't need to string lightning: prefix.
We already do this in param_invstring now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell 95f20a3978 lightningd, pyln-testing: do extra checks to make sure check *cannot* write to db.
Put an assertion inside db.c, and run every command we do (in testing) through
a `check` variant.

I inserted a deliberate bug (made addpsbtoutput call wallet_get_newindex()
before returning when running `check`, and indeed, backtrace as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell a01216505f common: add new param_check() which doesn't abort if we're simply checking.
We often want to do more parameter checks after param(), so allow a
new param_check(), with the proviso that the caller needs to also return
command_check_done() after other checks if command_check_only(cmd) is true.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell 35a47c57e3 lightningd: add direct close outputs to listfunds (mutual close).
We had a complaint that you can't CPFP a mutual close, which you
should be able to do.

Fixes: #6692
Changelog-Fixed: wallet: close change outputs show up immediately in `listfunds` so you can CPFP.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:58:34 +10:30
Peter Neuroth aab948e538 datastore: Add datastoreusage command
datastoreusage returns the total_bytes that are stored under a given
{Key} or from root. {Key} is the entry point from which we begin to
traverse the datastore.

Changelog-Added: JSON-RPC: `datastoreusage`: returns the total bytes that are stored under a given key.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-10-26 12:58:04 +10:30
Rusty Russell 985d0db143 lightningd: fix crash when blocks come it fast.
This is from the recent rework, as revealed by CI:

```
2023-10-25T00:57:29.1394358Z lightningd-2 2023-10-25T00:27:22.912Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 4ba8a31-modded)
2023-10-25T00:57:29.1395035Z lightningd-2 2023-10-25T00:27:22.912Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x5574aa3c29dd
2023-10-25T00:57:29.1395670Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x5574aa3c2b6f
2023-10-25T00:57:29.1396230Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce44251f
2023-10-25T00:57:29.1396768Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce4969fc
2023-10-25T00:57:29.1397297Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce442475
2023-10-25T00:57:29.1397945Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce4287f2
2023-10-25T00:57:29.1398616Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:95 (call_error) 0x5574aa56d36a
2023-10-25T00:57:29.1399308Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:169 (check_bounds) 0x5574aa56d565
2023-10-25T00:57:29.1399968Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:179 (to_tal_hdr) 0x5574aa56d5b7
2023-10-25T00:57:29.1400613Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:514 (tal_free) 0x5574aa56e058
2023-10-25T00:57:29.1401364Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:204 (rebroadcast_txs) 0x5574aa313ccc
2023-10-25T00:57:29.1402128Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:843 (updates_complete) 0x5574aa315a6b
2023-10-25T00:57:29.1402884Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:1055 (get_new_block) 0x5574aa31659f
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 23:11:37 +02:00
Rusty Russell 9c5e364f16 lightningd: don't re-enter transaction if we have to call plugin_exclusive_loop.
```
Already in transaction from lightningd/plugin.c:727
```

There are two callers, and one didn't disable transactions, so do it in plugin_exclusive_loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 10:53:48 +02:00
Rusty Russell eeee5d6249 lightningd: really fill in our own details when channeld says to make channel_update.
Now we've asserted that channeld would tell lightningd the same thing it
would do anyway, we can simply have channeld say "enable=True|False" and
lightningd fill in the other fields.

This means there's a pile of things channeld doesn't need to know any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 5dbe218bd4 channeld/gossipd/lightningd: reverse polarity of channel_update 'disable_flag'
Rename it to `enable` and invert it everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 9c04d0b02d lightningd: make sure that channeld_local_channel_update fields are redundant.
channeld used to talk directly to gossipd, so it made sense for it to
tell gossipd directly when it wanted it to make a new channel_update.

When that changed with v0.11, we simply directed the message via
lightningd.

But much of the information is actually told to channeld by lightningd!

So I applied this assertion and ran the test suite, before the next patch makes it redundant.

We got one assertion: test_setchannel_zero deliberately drives the
advertized htlc_max over the real htlc max in test_setchannel_zero for
testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 222da7f185 channeld: don't attach channel_update to errors, let lightningd do it.
This is far simpler: lightningd no longer needs to tell channeld when updates change, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 6294b617a3 lightningd: call finished callback *every* time we re-xmit a transaction.
We have to work quite hard to do this, since we don't want to call
finish if the broadcast has been freed in the meantime.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 4922b2ea04 lightningd: have broadcast_tx take a ctx.
Previously, every broadcast was attached to a channel, but we can
make it explicit, so when the context is freed, the re-broadcast stops
(if rebroadcast is set).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 1bdbb9261e lightningd: add context arg to bitcoind_sendrawtx()
If the context is freed, the callback isn't called.  This doesn't matter
yet, since our callbacks tend to be such that the callback itself is
required to free things, but it's clearer this way and allows more
flexible usage in following patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 6f1bb6fa41 lightningd: don't process request twice if plugin dies.
We remove it from the pending_requests strmap before calling it,
so it doesn't get called again by destroy_plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 7c1d07a94f lightningd: handle properly if our own request to plugin is freed.
We should really unify the cases of a local request, vs a forwarded
request, but for now, don't steal the request onto the plugin, and
if we return from the plugin and the request is gone, don't get upset.

This uncovered a case where we weren't inside a transaction, in
test_hook_crash, so fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell db7f59eeba lightningd: simplify plugin hook handling.
Now the internal code will generate a "PLUGIN_TERMINATED" response
when the plugin dies, we can handle it in plugin_hook.

But we can also simplify it by turning the snapshot of hooks into
a simple array: this means we are robust against any combination of plugins
exiting at any time.

Note: this reveals an issue with test_rpc_command_hook where we run
the request hook again (unexpectedly), so we disable that for the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 22ea202ae3 lightningd: generically fail requests when plugin dies.
We had special code to fail a forwarded request, but not for an
internally-generated request.  Instead, we should pretend the (dead)
plugin responded with a PLUGIN_TERMINATED error, and handle the
request through the normal paths.

This breaks the case where a plugin crashes (or stops itself) in a
hook, so we handle that next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 8170aba75f lightingnd: wrap all JSON reply callbacks in db transactions.
It was always a bit weird they weren't, and it seems a premature
optimization to make the callbacks to this themselves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 524eff0e93 lightningd: make pending_request map inside plugin struct.
It's per-plugin, so why is there a single map for all plugins?  It
works because we always make unique ids, but it's weird.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 83e69cdfc8 lightningd: fix --dev-debugger use with listconfigs
When using DEBUG_SUBD with pytest:

```
lightningd: Unknown decode for --dev-debugger=<subprocess>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30