Commit Graph

4255 Commits

Author SHA1 Message Date
Alex Myers 4265699fcd lightningd: add a feerate offset when updating feerates as opener
Adding a fee offset as the channel opener reduces the likelihood of a
disconnect by the peer do to slight variation in feerate calculation
between nodes.

Changelog-Fixed: Some peer disconnects due to update_fee disagreements are avoided.
2023-11-02 09:49:59 +10:30
Rusty Russell 28fd70a3d8 lightningd: rewrite anchor spend to use multiple UTXOs if needed.
Closes: #6747
Changelog-EXPERIMENTAL: Fixed anchor spending to be able to use more than one UTXO.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Rusty Russell ebf6f2e344 lightningd: use wallet_utxo_boost for zero-fee htlc_tx.
The previous logic looked wrong anyway!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Rusty Russell e4d7266fff common: add amount_feerate helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Rusty Russell ed034d9deb wallet: specialize get_utxos interfaces.
Turns out we really only want two:
1. wallet_get_all_utxos()
2. wallet_get_unspent_utxos()

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-01 14:11:28 +10:30
Rusty Russell af971fd025 wait forwards: add in_htlc_id
Without this, we have no unique identifier for which forward happened.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell f2162bf202 lightningd: add ordering and pagination to listforwards.
Changelog-Added: JSON-RPC: `listforwards` new parameters `index`, `start` and `limit`.
2023-10-28 15:48:26 +10:30
Rusty Russell 1d8af90b56 listforwards: add created_index and updated_index fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` fields `created_index` (old: `id`) and `updated_index`.
2023-10-28 15:48:26 +10:30
Rusty Russell de1e459196 forwards: hook into the wait subsystem.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `wait` now works for `forwards` infrastructure.
2023-10-28 15:48:26 +10:30
Rusty Russell e58ae31947 lightningd: hook forwards into the wait system.
This table doesn't have `id`, except as the implicit one in Sqlite3,
so we need to add it for postgres.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell c4f2ada2ff lightningd: split up peer_htlcs a little, create forwards.c
peer_htlcs has become a bit of a dumping ground: move listforwards
etc to its own file.

Also move `struct channel_info` from peer_htlcs.h to channel.h where
it more logically belongs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell c70a326f1f lightningd: add ordering and pagination to listsendpays.
Changelog-Added: JSON-RPC: `listsendpays` new parameters `index`, `start` and `limit`.
2023-10-28 15:48:26 +10:30
Rusty Russell 00ac1a9475 listsendpays: add created_index and updated_index fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sendpay`, `listsendpays`, `delpay` new fields `created_index` (old: `id`) and `updated_index`.
2023-10-28 15:48:26 +10:30
Rusty Russell 9ec6ac9922 lightningd: don't log that we're trying to pay if we're not.
Looking through logs I was surprise to see:

```
lightningd-1 2023-10-26T03:42:36.824Z INFO    lightningd: Sending 200000000msat over 1 hops to deliver 200000000msat
```

On a re-payment where we simply returned from sendpay immediately!  Move that log to later.
2023-10-28 15:48:26 +10:30
Rusty Russell f65c3cab75 wallet: use standard-style iterators for payments.
We used to have "unsaved" payments: now we don't we can use
our normal "iterator" pattern rather than returning arrays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 89cda26164 lightningd: rename struct sendpay_command to waitsendpay_command.
It used to be used for both `sendpay` and `waitsendpay` but now it's
only for the latter, so the name is confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 2689a4ed76 wallet: don't expose wallet_payment constructor, use wallet_add_payment.
Have it construct and return.  No need to expose details about dbid...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell 52b1ba2992 wallet: remove premature optimization when making payment.
We didn't write to db immediately, but waited until it the actual HTLC got
added (or failed).  That way we didn't have a separate transaction to
write the payment into the db, but the complexity is not worth it: it
makes the next refactors harder, since we can't use the normal
iterator patterns like we do with the rest of the db (as we have to add
the unstored ones).

We might as well also make sendpay return immediately: we used to return
once the HTLC had been confirmed sent, since we entered it in the db
at that point, but we can keep it simple now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell c2d89e4624 sendpays: hook into the wait subsystem.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `wait` now works for `sendpays` infrastructure.
2023-10-28 15:48:26 +10:30
Rusty Russell ccb8d4b687 sendpays: add to wait subsystem.
Adding an index means:

1. Add the new subsystem, and new updated_index field to the db, and
   create xxx_index_deleted/created/updated APIs.
2. Hook up these functions to the points they need to be called.
3. Add index, start and limit fields to the list command.
4. Add created_index and updated_index into the list command.

This does #1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell f8e036b22d channeld: use anchors on peer's commitment(s) if we can't broadcast our own.
This means refactoring out some of the generic anchor info, from the
per-commitment-tx info (we can have at least two, perhaps more with
splicing!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell 484deb94fa lightningd: save peer's commitment anchor info into db.
We actually only need to remember the last two, worst case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell e609bc934e channeld: tell lightningd about local anchor for each commitment tx.
It's going to want to remember these, in case it encounters peers'
commitment tx and needs to boost it with CPFP on the anchor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell 1d4ae91d2c channeld: get rid of unused commit_sig and htlc_sigs in channeld_sending_commitsig
We don't actually use this at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Chris Guida 54ef84e891 display paid_outpoint on invoices when present
Changelog-Added: JSON-RPC: `listinvoices` new field `paid_outpoint` if an invoice is paid onchain.
2023-10-26 19:11:17 +10:30
Chris Guida 8c03dd8161 mark invoice as paid when we detect an on-chain payment to a fallback 2023-10-26 19:11:17 +10:30
Chris Guida 8bbf2342f7 lightningd: adapt invoice_try_pay declaration for onchain payment.
i.e. `set` may be NULL, so we need to explicitly hand the amount.
We also add an outpoint.
2023-10-26 19:11:17 +10:30
Chris Guida e3e0abc73b add autogenerated fallbacks (not custom fallbacks) to invoice_fallbacks table 2023-10-26 19:11:17 +10:30
Chris Guida cdc0bd47ff track fallbacks for invoices, pt 1 2023-10-26 19:11:17 +10:30
Chris Guida fc4b2f864e add invoices-onchain-fallback config option and warnings
Changelog-Added: Config: `invoices-onchain-fallback` to automatically add an onchain p2tr address to invoices, and allow that for payment.
2023-10-26 19:11:17 +10:30
Ken Sedgwick 44798e298c hsmd: implement the hsmd outpoint check
Tihis commit is implementing a 2-phase commit between
the signer the node and the peer.

The main reason for this is that everybody must agree on the lock,
otherwise one of them will want N signatures (on the splice candidates),
and another will produce only 1 signature.

check_outpoint is the "prepare" for the signer, and lock_outpoint is the
"commit". if check_outpoint returns true, lock_outpoint must not fail.

Link: https://github.com/ElementsProject/lightning/issues/6722
Suggested-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-26 16:42:47 +10:30
Rusty Russell a50b795bc2 lightningd: implement `recover` command.
Changelog-Added: JSON-RPC: `recover` command to force (unused) lightningd node to restart with `--recover` flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell 7f18ed7743 lightningd: allow --recover to take a 64-char hex string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `--recover` can take a 32-byte hex string, as well as codex32.
2023-10-26 12:59:55 +10:30
Rusty Russell 3b622f06c3 lightningd: use param_check on all commands which do extra checks.
This makes `check` much more thorough, and useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` now does much more checking on every command (not just basic parameter types).
2023-10-26 12:59:55 +10:30
Rusty Russell c490be62f0 lightningd: preapproveinvoice doesn't need to string lightning: prefix.
We already do this in param_invstring now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell 95f20a3978 lightningd, pyln-testing: do extra checks to make sure check *cannot* write to db.
Put an assertion inside db.c, and run every command we do (in testing) through
a `check` variant.

I inserted a deliberate bug (made addpsbtoutput call wallet_get_newindex()
before returning when running `check`, and indeed, backtrace as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell a01216505f common: add new param_check() which doesn't abort if we're simply checking.
We often want to do more parameter checks after param(), so allow a
new param_check(), with the proviso that the caller needs to also return
command_check_done() after other checks if command_check_only(cmd) is true.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell 35a47c57e3 lightningd: add direct close outputs to listfunds (mutual close).
We had a complaint that you can't CPFP a mutual close, which you
should be able to do.

Fixes: #6692
Changelog-Fixed: wallet: close change outputs show up immediately in `listfunds` so you can CPFP.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:58:34 +10:30
Peter Neuroth aab948e538 datastore: Add datastoreusage command
datastoreusage returns the total_bytes that are stored under a given
{Key} or from root. {Key} is the entry point from which we begin to
traverse the datastore.

Changelog-Added: JSON-RPC: `datastoreusage`: returns the total bytes that are stored under a given key.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-10-26 12:58:04 +10:30
Rusty Russell 985d0db143 lightningd: fix crash when blocks come it fast.
This is from the recent rework, as revealed by CI:

```
2023-10-25T00:57:29.1394358Z lightningd-2 2023-10-25T00:27:22.912Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 4ba8a31-modded)
2023-10-25T00:57:29.1395035Z lightningd-2 2023-10-25T00:27:22.912Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x5574aa3c29dd
2023-10-25T00:57:29.1395670Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x5574aa3c2b6f
2023-10-25T00:57:29.1396230Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce44251f
2023-10-25T00:57:29.1396768Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce4969fc
2023-10-25T00:57:29.1397297Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce442475
2023-10-25T00:57:29.1397945Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f7cce4287f2
2023-10-25T00:57:29.1398616Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:95 (call_error) 0x5574aa56d36a
2023-10-25T00:57:29.1399308Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:169 (check_bounds) 0x5574aa56d565
2023-10-25T00:57:29.1399968Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:179 (to_tal_hdr) 0x5574aa56d5b7
2023-10-25T00:57:29.1400613Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:514 (tal_free) 0x5574aa56e058
2023-10-25T00:57:29.1401364Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:204 (rebroadcast_txs) 0x5574aa313ccc
2023-10-25T00:57:29.1402128Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:843 (updates_complete) 0x5574aa315a6b
2023-10-25T00:57:29.1402884Z lightningd-2 2023-10-25T00:27:22.913Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:1055 (get_new_block) 0x5574aa31659f
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 23:11:37 +02:00
Rusty Russell 9c5e364f16 lightningd: don't re-enter transaction if we have to call plugin_exclusive_loop.
```
Already in transaction from lightningd/plugin.c:727
```

There are two callers, and one didn't disable transactions, so do it in plugin_exclusive_loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 10:53:48 +02:00
Rusty Russell eeee5d6249 lightningd: really fill in our own details when channeld says to make channel_update.
Now we've asserted that channeld would tell lightningd the same thing it
would do anyway, we can simply have channeld say "enable=True|False" and
lightningd fill in the other fields.

This means there's a pile of things channeld doesn't need to know any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 5dbe218bd4 channeld/gossipd/lightningd: reverse polarity of channel_update 'disable_flag'
Rename it to `enable` and invert it everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 9c04d0b02d lightningd: make sure that channeld_local_channel_update fields are redundant.
channeld used to talk directly to gossipd, so it made sense for it to
tell gossipd directly when it wanted it to make a new channel_update.

When that changed with v0.11, we simply directed the message via
lightningd.

But much of the information is actually told to channeld by lightningd!

So I applied this assertion and ran the test suite, before the next patch makes it redundant.

We got one assertion: test_setchannel_zero deliberately drives the
advertized htlc_max over the real htlc max in test_setchannel_zero for
testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 222da7f185 channeld: don't attach channel_update to errors, let lightningd do it.
This is far simpler: lightningd no longer needs to tell channeld when updates change, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-25 07:00:05 +10:30
Rusty Russell 6294b617a3 lightningd: call finished callback *every* time we re-xmit a transaction.
We have to work quite hard to do this, since we don't want to call
finish if the broadcast has been freed in the meantime.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 4922b2ea04 lightningd: have broadcast_tx take a ctx.
Previously, every broadcast was attached to a channel, but we can
make it explicit, so when the context is freed, the re-broadcast stops
(if rebroadcast is set).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 1bdbb9261e lightningd: add context arg to bitcoind_sendrawtx()
If the context is freed, the callback isn't called.  This doesn't matter
yet, since our callbacks tend to be such that the callback itself is
required to free things, but it's clearer this way and allows more
flexible usage in following patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 6f1bb6fa41 lightningd: don't process request twice if plugin dies.
We remove it from the pending_requests strmap before calling it,
so it doesn't get called again by destroy_plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 7c1d07a94f lightningd: handle properly if our own request to plugin is freed.
We should really unify the cases of a local request, vs a forwarded
request, but for now, don't steal the request onto the plugin, and
if we return from the plugin and the request is gone, don't get upset.

This uncovered a case where we weren't inside a transaction, in
test_hook_crash, so fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell db7f59eeba lightningd: simplify plugin hook handling.
Now the internal code will generate a "PLUGIN_TERMINATED" response
when the plugin dies, we can handle it in plugin_hook.

But we can also simplify it by turning the snapshot of hooks into
a simple array: this means we are robust against any combination of plugins
exiting at any time.

Note: this reveals an issue with test_rpc_command_hook where we run
the request hook again (unexpectedly), so we disable that for the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 22ea202ae3 lightningd: generically fail requests when plugin dies.
We had special code to fail a forwarded request, but not for an
internally-generated request.  Instead, we should pretend the (dead)
plugin responded with a PLUGIN_TERMINATED error, and handle the
request through the normal paths.

This breaks the case where a plugin crashes (or stops itself) in a
hook, so we handle that next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 8170aba75f lightingnd: wrap all JSON reply callbacks in db transactions.
It was always a bit weird they weren't, and it seems a premature
optimization to make the callbacks to this themselves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 524eff0e93 lightningd: make pending_request map inside plugin struct.
It's per-plugin, so why is there a single map for all plugins?  It
works because we always make unique ids, but it's weird.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Rusty Russell 83e69cdfc8 lightningd: fix --dev-debugger use with listconfigs
When using DEBUG_SUBD with pytest:

```
lightningd: Unknown decode for --dev-debugger=<subprocess>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 15:07:08 +10:30
Vincenzo Palazzo 843fae7caf psbt: fix PSBT mutation in the changeset
During the changeset calculation after the `openchannel2_sign`
hook.

So this commit patch the problem with the following change:

- Addressed an issue where `psbt_get_changeset` was modifying the original PSBT unnecessarily.
- This modification led to problems with a different hsmd, as referenced in [Issue #6672](https://github.com/ElementsProject/lightning/issues/6672).
- Noted a potential optimization where only a subpart of the PSBT
needs to be cloned, as the mutation is specific to inputs.

Link: https://github.com/ElementsProject/lightning/issues/6672
Reported-by: @devrandom
Suggested-by: Ken Sedgwick <ken@bonsai.com>
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-24 12:57:45 +10:30
Rusty Russell bd80af5295 lightningd: allow sending of even messages.
It's your funeral!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 11:50:57 +10:30
Rusty Russell ad7dcf381e lightningd: tell connectd about the custom messages.
We re-send whenever a plugin which allows them starts/finishes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 11:50:57 +10:30
Rusty Russell 774719530e plugins: allow plugins to specify even messages they accept.
Changelog-Added: Plugins: plugins can now specify (unknown) even messages we should accept from peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 11:50:57 +10:30
Rusty Russell 93189724dd lightningd: let channeld/dual_openingd send error to peer.
We do it here, but it's not necessary, and we also deprive them of the
chance to do so (since we kill them).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell cf2ebed567 lightningd: close channel ourselves, if we receive an error.
Previously, we would forward the message to a subd, but now we have
the case where the subd is gone, but we're still connected.  If the
peer anything but a reestablish in that state, we drop the connection.

Instead, an error should always make us fail the channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell 9abf9c2924 lightningd: pass disconnect flag to subd's errcb.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell 443082ba98 common: add peer_failed_warn_nodisconnect routine for non-disconnecting warnings
We generalize the current df-only "aborted" flag (and invert it) to a
"disconnected" flag in the peer status message.

We convert it back to the aborted flag for now inside subd.c, but that's
next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell a32b3b68e2 lightningd: stop all subds when we want to disconnect.
This prepares us for connectd disconnecting more gently: it can
wait for these to all disconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell 19c581c33f lightningd: simplify lease update blockheight code.
Move the "no lease, return" to the top, to avoid testing twice.  Also,
we won't spam now for most channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:59:54 +10:30
Rusty Russell 91699d8483 lightningd: fix crash caused by sending channeld message to closingd
```
Sending closingd an invalid message 03f40000006d
...
0x564b55bb7378 fatal
	lightningd/log.c:1035
0x564b55bf7841 subd_send_msg
	lightningd/subd.c:841
0x564b55b87d08 try_update_blockheight
	lightningd/channel_control.c:143
0x564b55b8c497 channel_notify_new_block
	lightningd/channel_control.c:1687
0x564b55bb26bb notify_new_block
	lightningd/lightningd.c:743
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:59:54 +10:30
Rusty Russell 03e51ae879 Revert "runes: fix `checkrune` when `method` parameter is the empty string."
This reverts commit ae94be4ce2.
2023-10-17 14:58:40 +10:30
Vincenzo Palazzo 9b69c56cfb lightningd: print the version of cln early
As a node matures and is no longer new, it can take some time
to determine which version of `cln` it's running.

To address this, we now display the version earlier. This ensures
that even in the event of a crash, we're aware of the running version.

(cln-meta-project-py3.11) ➜  lightning git:(macros/log-version) ✗ ./lightningd/lightningd
2023-10-12T19:21:00.899Z INFO    lightningd: v23.08.1-209-gae94be4-modded
2023-10-12T19:21:00.994Z INFO    lightningd: Creating configuration directory /home/vincent/.lightning/bitcoin
2023-10-12T19:21:01.235Z INFO    lightningd: Creating database
2023-10-12T19:21:01.279Z UNUSUAL hsmd: HSM: created new hsm_secret file

Could not connect to bitcoind using bitcoin-cli. Is bitcoind running?

Make sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.

You can verify that your Bitcoin Core installation is ready for use by running:

    $ bitcoin-cli echo 'hello world'
2023-10-12T19:21:01.349Z **BROKEN** plugin-bcli: \nCould not connect to bitcoind using bitcoin-cli. Is bitcoind running?\n\nMake sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.\n\nYou can verify that your Bitcoin Core installation is ready for use by running:\n\n    $ bitcoin-cli echo 'hello world'\n
2023-10-12T19:21:01.349Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

Link: https://github.com/ElementsProject/lightning/issues/6374
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-14 17:52:15 +02:00
Tony Aldon ae94be4ce2 runes: fix `checkrune` when `method` parameter is the empty string.
This fix #6725.

Changelog-Fixed: fix `checkrune` when `method` parameter is the empty string.
2023-10-12 20:42:43 +02:00
Rusty Russell e11b35cb3a common/memleak: implement callback arg for dump_memleak.
This makes it easier to use outside simple subds, and now lightningd can
simply dump to log rather than returning JSON.

JSON formatting was a lot of work, and we only did it for lightningd, not for
subdaemons.  Easier to use the logs in all cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-03 10:05:55 +02:00
Rusty Russell 0b680c5f40 lightningd: fix assertion when funding depth changes fast.
We didn't apply the inflight to the channel struct before asserting, so
we can break test_rbf_non_last_mined:

```
lightningd: lightningd/dual_open_control.c:981: dualopend_tell_depth: Assertion `bitcoin_txid_eq(&channel->funding.txid, txid)' failed.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-03 11:40:43 +10:30
ShahanaFarooqui c15fc54586 runes: Runes will use unique id returned from `runes` table not `runes_uniqueid` from `vars` 2023-10-03 08:56:53 +10:30
ShahanaFarooqui 9bcabbc912 runes: Reimplement `rate` in terms of `per`
Changelog-Changed: JSON-RPC: `checkrune` `rate` restriction is slightly stricter (exact division of time like `per`)
2023-10-03 06:57:54 +10:30
Rusty Russell 72f914a295 lightningd: simplify funding_depth_cb now it only handles main funding tx.
We make dualopend_tell_depth static, which means we move it
higher in the file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 79d04090fb lightningd: fix dual-funding case where we coop close and an RBF confirms.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 2f394653a8 lightningd: ensure we *always* watch channel spend.
Now we're not always using the same functions to watch during
dual-funding opening, we need to make sure we're watching the close
(in particular, df close before the opening is confirmed).

So, keep a pointer, and if it's not set in drop_to_chain, set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 3ca753e0b4 lightningd: fix bug where we didn't correctly change outpoint of splice scid.
We used the original channel funding output number.  I'm not sure if this
was true in the previous code, or a regression I introduced, but it
caused occasonal failures in test_splice_gossip!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 854bda81ac lightningd: don't share funding_depth_cb for non-funding txs.
We use the *same* callback for the funding tx, as well as for inflight dual-funding txs, as well as inflight splice txs.  This is deeply confusing!

Instead, use explicit cbs for splicing and df.  Once they're locked in, use the normal callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 1cd53ae53e lightningd: fix watch on existing tx.
We never do this, but we're about to (we always watch before
we broadcast a tx).

We use a `depth` member to avoid calling the callback multiple times
for the same event, but we initialize it to 0.  This means if we
register a watch, and the first thing that happens is that it
reorganizes out, we *don't* make the callback.

Use an impossible value at initialization, instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell eb1ef40f96 lightningd: make watch_txid more generic.
Don't assume the arg is a channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 4eb2f95e16 lightningd: remove watch_tx() in favor of watch_txid().
It was a wrapper only used in one place anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 5cf536d4b1 lightningd: make channel-query functions all take state.
It has the information we need, now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell acc30c0b3f lightningd: split DUALOPEND_OPEN_INIT into DUALOPEND_OPEN_INIT and DUALOPEND_OPEN_COMMITTED.
The latter is used when we're put in the db, the former is the uncommitted state.
Currently dbid == 0 is used in addition to the state, which is unwieldy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: JSON-RPC: added new dual-funding state `DUALOPEND_OPEN_COMMITTED`
2023-10-02 11:41:19 +10:30
Rusty Russell 55285629d3 common/json_stream, lightningd/notification: clean up function APIs
We usually hand times by copy, not by pointer (and if we did, they should
be const!).  I noticed this particularly for the state changed code, but
it goes down to to json_add_timeiso, so I fixed that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 3589cf6713 lightningd/channel.h: rename channel_unsaved to the more explicit channel_state_uncommitted.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 36bed08f90 lightningd: remove peer_any_unsaved_channel and use peer_any_channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 0b4622bcbd lightningd/channel.h: clean up channel states.
We should use capability tests for states (can you add htlcs?) rather than vague
descriptions (are you closing?).

And as much as possible, use switch () statements to force us to think
about all the cases, especially when we add new states!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 11ec03c6da lightningd: generalize peer_any_active_channel to peer_any_channel.
Take an optional filter function, so callers can say exactly what they want. 

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 2de7171286 lightningd: fold funding tx depth into a single function.
Currently it's half done in funding_depth_cb, and half in
channeld_tell_depth.  It's very confusing as a result,
with splicing, dual-funding and zeroconf.

This does introduce a behaviour change: if a channel is NORMAL and
it gets reorganized, we force close (unless we were the one who funded
it, or it's zeroconf anyway).  This is safer than continuing to use
the channel in this case!

Some tests are changed to zeroconf to make them work, but v2 doesn't
support zeroconf, so that's removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell c0ffb17661 lightningd: make dualopen_tell_depth match channeld_tell_depth.
Rename slightly, remove first arg, and make it a noop of
there's no owner on channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 328d685a2a lightningd: don't report original depth once splice started.
This is a workaround, the real fix is to use a different
callback for inflight splice attempts, which comes later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell 58d27c4bdc lightningd: clean up channel_tell_depth.
It's a mess right now.

Try to express it as a switch() statement over the states we can be in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell fe0959fd31 lightningd: disconnect on *any* transient error, except abort
Not just if htlc addition is too slow, make this the default.  dual-open's txabort
is excluded, however.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Matt Whitlock 7c582944bd lightningd/test: use tmpdir_mkstemp() to create log file
Changelog-None
2023-09-25 13:59:13 +09:30
Rusty Russell 3d3bf87b34 build: remove all trace of DEVELOPER.
If you previously configured with `--enable-developer` we turn that into `--enable-debugbuild`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: build: `--enable-developer` arg to configure (and DEVELOPER variables): use `./configure --enable-debugbuild` and `developer` setting at runtime.
2023-09-21 20:08:24 +09:30
Rusty Russell 3e124d9ec1 lightningd: remove #if DEVELOPER.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell ef87999f9a common: add option for dev-only parameters.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell de81a59b1e lightningd: mark explicitly what commands are dev-only.
And require --developer to use them.

Also refuse redirection to deprecated APIs if deprecated APIs are disabled! 

Changelog-Removed: `dev-sendcustommsg` (use `sendcustommsg`, which was added in v0.10.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell a9f26b7d07 common/daemon.c: remove #ifdef DEVELOPER in favor of runtime flag.
Also requires us to expose memleak when !DEVELOPER, however we only
ever used the memleak tracking when the LIGHTNINGD_DEV_MEMLEAK
environment variable was set, so keep that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell 67a391f8d0 lightningd: pass --developer down to plugins, subdaemons when it set.
They might need to know this before they do anything, so cmdline seems most
natural.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell 64ab7c0c1d lightningd: add --developer runtime option.
Currently it just defaults to the DEVELOPER compile option, but we'll
move over to this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `--developer` enables developer options and changes default to be "disable deprecated APIs".
2023-09-21 20:08:24 +09:30
Rusty Russell 5882978147 lightningd: fix bogus unused var warning.
We check for list_empty, so it's always actually set.

```
lightningd/peer_control.c: In function ‘drop_to_chain’:
lightningd/peer_control.c:353:17: error: ‘tx’ may be used uninitialized [-Werror=maybe-uninitialized]
  353 |                 resolve_close_command(ld, channel, cooperative, tx);
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lightningd/peer_control.c:341:36: note: ‘tx’ was declared here
  341 |                 struct bitcoin_tx *tx;
      |                                    ^~
cc1: all warnings being treated as errors
make: *** [Makefile:298: lightningd/peer_control.o] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Dusty Daemon 0a5ef7f2e6 splice: Fixes from splice-out test
Added a test for splicing out that exposed some behavior and code glitches that are addressed in this commit.

Added test for splice gossip.

Also added documentation for how to do a splice out.

ChangeLog-Fixed: Added docs, testing, and some fixes related to splicing out, insufficent balance handling, and restarting during a splice.
2023-09-21 13:41:45 +09:30
ShahanaFarooqui 91eb184141 lightningd/runes: Added last_time_check for validating rune for `per` restriction
Changelog-Added: runes: `per=Nsec/min/hour/msec/usec/nsec` for general ratelimiting
2023-09-21 13:31:34 +09:30
ShahanaFarooqui 3f95597a5e common: make json_add_timeabs full precision & renamed json_add_time
json_add_timeabs only printed in milliseconds and json_add_time outputs a string which is weird

Changelog-Changed: JSON-RPC time fields now have full nanosecond precision (i.e. 9 decimals not 3): `listfowards` `received_time` `resolved_time` `listpays`/`listsendpays` `created_at`.
2023-09-21 13:31:34 +09:30
ShahanaFarooqui d745323f74 lightning/runes: added last_used in showrunes
Changelog-Added: JSON-RPC: `showrunes` new field `last_used`
2023-09-21 13:31:34 +09:30
ShahanaFarooqui db55deae2a lightningd/runes: update last_used timestamp when a rune successfully used 2023-09-21 13:31:34 +09:30
ShahanaFarooqui eacf0b502c wallet: add last_used_nsec in runes table
We will use this for better rate limiting
2023-09-21 13:31:34 +09:30
Rusty Russell 68a6084bab lightningd: update to BOLT 7d3ef5a6b20eb84982ea2bfc029497082adf20d8 "Allow unset `onion_hash` in `invalid_onion_blinding` (#1093)"
Explicitly allow all-zero in the onion_hash: we didn't do anything except log if it was unexpected anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell 48522f3e9e common: don't hang up when we receive a warning.
We were allowed to, but the spec removed that.  So we handle warnings
differently from errors now.

This also means the LND "internal error" workaround is done in
lightningd (we still disconnect, but we don't want to close channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we no longer disconnect every time we receive a warning message.
2023-09-20 13:56:46 +09:30
Rusty Russell 1bb83a1ae6 common: don't send channel_id on peer error.
It's unused: they know what channel it is.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell 821d5f48ea doc: update to BOLT 50b2df24a27879e8329712c275db78876fd022fe "Update onion errors since we allow overpaying or under-CLTVing"
We already do this behaviour, we just didn't adjust comments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell 2765939adb runes: fix weird unnecessary zero-test.
This seems to be a cut & paste bug (mine, AFAICT!) from the command code:

```
	rune = rune_derive_start(cmd, master_rune,
				 tal_fmt(tmpctx, "%"PRIu64,
					 rune_counter ? *rune_counter : 0));
```

In that case, rune_counter was a pointer, which could be NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-12 15:19:02 +09:30
Rusty Russell 785fe973a6 runes: ensure that uniqueid is a valid number.
It always is for runes we create, but in theory you can take our secret key
and make our own runes with your own tools.

(We correctly refuse runes without uniqueids if they're *not* ours
anyway: uniqueid is only used for our own runes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-12 15:19:02 +09:30
Rusty Russell ecb09778db lightningd: refuse to create rune with empty fields.
These look like uniqueids, and so can confuse us (I discovered this by
making a typo in a test!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-12 15:19:02 +09:30
Rusty Russell bb38f83b88 checkrune: make nodeid and method optional.
nodeid is only useful when we know the peer we're talking to (e.g. commando).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No-schema-diff-check: We're simply making optional, not deprecating!
2023-09-12 15:19:02 +09:30
Rusty Russell af39424491 lightningd: don't restart subds unnecessarily on reconnect.
During tests we can see that the subdaemon can be restarted unnecessarily if we're slow enough; we don't need to do so if it's still running.

Reported-by: Matt Morehouse <mattmorehouse@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:14:01 +09:30
Christian Decker 71a8e59acd openingd: Reject zeroconf if the peer is not allowed
We determine whether they are allowed or not based on the hook return
value of `mindepth`. To do so we need to pass that value down to
`openingd` and verify that the `channel_type` and our permissions
match up.
2023-08-18 20:30:03 +09:30
Rusty Russell b263c7d2e4 lightningd: documentation updates.
1. announce-addr-discovered-port takes a port option.
2. accept-htlc-tlv-types was deprecated in favor of multiple accept-htlc-tlv-type.
3. Document clnrest.py options.
4. Don't list --version twice in lightningd --help (initial_config_opts calls
   opt_register_version() already).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell d5e86bf330 lightningd: don't print plugin deprecated options in --help.
We don't publish core deprecated options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell f4e2d9a0ae lightningd: clean up properly if we fail to exec plugin.
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: we clean up properly if a plugin fails to start, and we don't kill all processes if it's from `plugin startdir`.
2023-08-18 13:18:49 +09:30
Lisa Neigut 55168e66ad Don't restart my machine if I add code to kill a failed plugin. 2023-08-18 13:18:49 +09:30
Dusty Daemon c67f1f92a8
splice: prevent splice going to onchaind & race prevention
Don’t send the funding spend to onchaind if we detect it in inflights (aka. a splice). While we already prevented onchaind_funding_spent from being called directly, the call to wallet_channeltxs_add meant onchaind_funding_spent would be called *anyway* on restart. This is now fixed.

Additionally there was a potential for a race problem depending on the firing order of the channel depth and and funding spent events.

Instead of requiring these events fire in a specific order, we make a special “memory only” inflight object to prevent the race regardless of firing order.

Changelog-Fixed: Splice: bugfix for restart related race condition interacting with adversarial close detection.
2023-08-16 12:28:53 +09:30
Rusty Russell 9239278cdd plugins: fix compile warning with some gcc versions and -Og
Indeed, we can fall through this if it's not a valid enum value.

gcc-12 (Ubuntu 12.2.0-17ubuntu1) 12.2.0

```
In file included from plugins/commando.c:10:
ccan/ccan/tal/str/str.h: In function ‘rune_altern_to_english’:
ccan/ccan/tal/str/str.h:43:9: error: ‘cond_str’ may be used uninitialized [-Werror=maybe-uninitialized]
   43 |         tal_fmt_(ctx, TAL_LABEL(char, "[]"), __VA_ARGS__)
      |         ^~~~~~~~
plugins/commando.c:97:21: note: ‘cond_str’ was declared here
   97 |         const char *cond_str;
      |                     ^~~~~~~~
cc1: all warnings being treated as errors
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-15 06:21:23 +09:30
Rusty Russell 6a16a6fe25 closingd: fix case where we we can pass under min-relay-fee for mutual close.
In spec commit 498f104fd399488c77f449d05cb21c0b604636a2 (August 2021),
Bastien Teinturier removed the requirement that the mutual close fee be
less than or equal the final commitment tx.

We adopted that change in v0.10.2, but we made sure to never offer a fee
under the final commitment tx's fee, so we didn't break older nodes.

However, the closing tx can actually be larger than the final commitment tx!
The final commit tx has a 22-byte P2WKH output and a 34-byte P2WSH output;
the closing can have two 34-byte outputs, making it 4*8 = 32 Sipa heavier.
Previously this would only happen if both sides asked for P2WSH outputs,
but now it happens with P2TR, which we now do.

The result is that we create a tx which is below the finally commitment
tx fee, and may be below minrelayfee (as it was in regtest).

So it's time to remove that backwards-compatibility hack.

Changelog-Fixed: Protocol: We may propose mutual close transaction which has a slightly higher fee than the final commitment tx (depending on the outputs, e.g. two taproot outputs).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6545
2023-08-12 13:17:46 +09:30
Rusty Russell 8f16b0593c lightingd: fix minfee when we're ignoring limits.
Presumably we still want to insist that this have *some* chance to propagate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 13:17:46 +09:30
Rusty Russell 15c30f4018 renepay: make pay_plugin a tal object.
Avoids a gratuitous "ctx" field, and the simplified declaration
is now understood by `make update-mocks`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Dusty Daemon c50e93d9fb splice: Move splice to experimental feature bit
This was recommended by @t-bast: if the final spec commits to something
compatible, we can simply advertize and accept both features, but if it
does change in incompatible ways we won't cause problems for nodes
who implement the official spec.

(I split this, so first, we remove the OPT_SPLICE entirely, to make
sure we caught them all. --RR)

Suggested-by: @t-bast
Changelog-None
2023-08-10 15:52:53 +09:30
Rusty Russell ac092c0bf9 hsmd: fix capability check for signing splices.
The nomenclature confusion mean that we were ANDING a capability
with a message number (29) which always returned non-zero.  We really
do need a new capability which we can hand to channeld to make these
splice txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell d95cfc0b64 hsmd: rename "capabilities" flags for hsm fds to "permissions"
I obviously like the word "capabilities" since I reused it to refer
to the HSM's overall features :(

Suggested-by: @ksedgwic
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell 1fbe87f6e4 lightningd: use fsync not fdatasync.
Apparently MacOS doesn't always have fdatasync, so use fsync.  Even more importantly
check whether it succeeds!

Fixes: #6516
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 09:45:10 +09:30
Rusty Russell 54bcb10227 lightningd: fix bolt11 parsing in preapproveinvocie, sendonion, listsendpays and renepay
Since bolt11_decode now insists that any `lightning:` prefix be removed, we need
to make sure to use param_invstring not param_string for all bolt11 parameters:

```
2023-08-07T05:55:32.515Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v23.08rc1-21-g0bf5ee6)
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x55dd94934154
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x55dd949342e6
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7f5cf5a3bcef
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7f5cf5a3bc45
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5cf5a227fb
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5cf5a2271a
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5cf5a33595
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:734 (bolt11_decode_nosig) 0x55dd94929967
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:953 (bolt11_decode) 0x55dd9492a44f
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/pay.c:1730 (json_listsendpays) 0x55dd948d7d72
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:658 (command_exec) 0x55dd948b525b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:786 (rpc_command_hook_final) 0x55dd948b5876
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:285 (plugin_hook_call_) 0x55dd948f6446
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:874 (plugin_hook_call_rpc_command) 0x55dd948b5c77
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:984 (parse_request) 0x55dd948b6234
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:1090 (read_json) 0x55dd948b670f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55dd94ac9bf4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55dd94aca823
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55dd94aca865
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x55dd94accbff
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x55dd948b33c4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1332 (main) 0x55dd948ba429
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) 0x7f5cf5a2350f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../csu/libc-start.c:381 (__libc_start_main_impl) 0x7f5cf5a235c8
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55dd94881e74
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Fixes: #6524
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: broken in master since last release.
2023-08-07 18:46:48 +09:30
Rusty Russell 91ea85be36 lightningd: close connection when HTLC addition times out.
I noticed this while debugging an issue with ACINQ, that we got upset,
but didn't trigger a reconnect cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We now close connection with a peer if adding an HTLC times out (which may be a TCP connectivity issue).
2023-08-07 18:45:29 +09:30
Rusty Russell f556be5d82 renepay: allow it to die gracefully without crashing lightningd.
Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 17:13:29 +09:30
Rusty Russell 34b6731b82 lightningd: don't return to a default filter level if there are no per-file filters.
In this case, the user's default was info, but they specifically asked for debug
from one plugin.  Since there were no per-file filters, it set filtering to the
default level, info, and rejected it.  Since it's been explicitly filtered in,
we need to pass it at this point.

Reported-by: @wtogami
Fixes: #6503
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Dusty Daemon fb179f2d5e
splicing: Remove dependency on experimental_dual_fund
Splicing should work automatically on v1 or v2 channels so this requirement isn’t needed.

Changelog-None
[ Squashed fixup into a single commit --RR ]
2023-08-06 12:19:42 +09:30
Rusty Russell 0f6687ec7b codex32: use "cl" instead of "ms" as our HRP.
This was strongly recommended by Russell O'Connor: the "ms" implies that
it's a BIP-32 master secret, and this is CLN specific.

If we changed the hrp to "cln" it would be better, but apparently that
means we no longer fit in a "standard billfold metal wallet" (and
our code assumes a 2-byte prefix anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 16:12:32 +09:30
Rusty Russell e981a4f14a lightningd: pass signed tx through to close callback.
Thread the signed tx through so close's JSON return contains that,
rather than the unsigned channel->last_tx.

We have to split the "get cmd_id" from "resolve the close commands" though;
and of course, as before, we don't actually print the txids of multiple
transactions even though we may have multi in flight due to splice!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `close` returns a `tx` field with witness data populated (i.e. signed).
Fixes: #6440
2023-08-02 07:09:28 +09:30
Rusty Russell d90a185188 lightningd: clean up signing of last_tx.
Return a copy, don't sign in place then remove the signature after.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 07:09:28 +09:30
niftynei dd871d9e26 inflights: use ctx for making new ones
Also convert everything to an array thingy
2023-07-31 21:00:22 +09:30
Dusty Daemon 4628e3ace8 channeld: Code to implement splicing
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.

Implement the channeld splicing protocol leveraging the interactivetx protocol.

Implement lightningd’s channel_control to support channeld in its splicing efforts.

Changelog-Added: Added the features to enable splicing & resizing of active channels.
2023-07-31 21:00:22 +09:30
Dusty Daemon 50fe819f47 splicing: Add channel state AWAITING_SPLICE
Update gossip routiens and various other hecks on the channel state to consider AWAITING_SPLICE to be routable and treated similar to CHANNELD_NORMAL.

Small updates to psbt interface

Changelog-None
2023-07-31 21:00:22 +09:30
Rusty Russell 2c2592fe36 lightningd: fix crash on startup expirations.
The wait code assumes ld->wallet is populated, but it's not.  Start the
expiration cycle later.

```
0x7f271a18d08f ??? /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
0x5581a27dc082 wait_index_increment lightningd/wait.c:112
0x5581a27e331a invoice_index_inc wallet/invoices.c:738
0x5581a27e3dfe invoice_index_update_status wallet/invoices.c:775
0x5581a27e3ea3 trigger_expiration wallet/invoices.c:185
0x5581a27e3f47 invoices_new wallet/invoices.c:134
0x5581a27e8a2c wallet_new wallet/wallet.c:121
0x5581a27b08b5 main lightningd/lightningd.c:1082
```

Fixes: #6457
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-31 20:42:31 +09:30
Rusty Russell 58327a594f codex32: rework.
Firstly, I wanted the results easier to use:
1. Make them always lower case, even if the string was UPPER.
2. Decode the payload for them.
3. Don't give the user any fields they don't need, and make
   the field sizes explicit.

Secondly, I wanted to avoid the pattern of "check in one place, assume
in another", in favour of "check on use".

So, I changed the code to lower the string if it needs to at the start,
and then changed the pull functions so we always use them to get data:
this way we should fail clearly and gracefully if we don't have enough data.

I made all the checks explicit, where we assign the fields.

I also addressed the FIXME: I think the array is *often* one shorter,
but not always, so I trim the last byte at the end if needed.

[ Aditya modified the tests to work ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-31 11:29:59 +09:30
adi2011 51f72c1d1d lightningd: Added plugin hook to notify whenever started in rocover mode. 2023-07-31 11:29:59 +09:30
adi2011 88a2c0d17e lightningd/options: Always start in offline mode when run using --recover flag. 2023-07-31 11:29:59 +09:30
adi2011 581677691d lightningd/common: Added --recover flag to populate HSM_secret with appropriate secret through a valid codex32 secret. 2023-07-31 11:29:59 +09:30
niftynei 9b8909e507 dual-fund: keep track of aborted requests, seamlessly restart daemon
Clean restart of daemon after a tx-abort is a nice way to work around
the 'persistent' disconnect that we t-bast noticed.

Changelog-Fixed: `dualopend`: Fix behavior for tx-aborts. No longer hangs, appropriately continues re-init of RBF requests without reconnction msg exchange.
2023-07-30 15:20:04 +09:30
Rusty Russell 4b4937b9bd gossipd: tell lightningd about all our previous channel_updates at startup.
This will at least *help* the case where these were not populated, causing us
to send errors without channel_updated appended.

It's not perfect: we can still send such errors if the gossip store is
corrupted, and we still send them for private channels, but it should
help.

(The much better fix is far more invasive, so slips to next release!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-29 09:44:20 +09:30
Rusty Russell c98711ad28 lightningd: don't crash if peer manages to spend onchain HTLC after we've abandoned upstream.
Since we now abandon HTLCs to avoid breaking a channel (in limited circumstances), the
peer can take the funds if they beat us to spend the HTLC before we timeout.  This is extremely
unusual, but finally happend in CI.

First it fails our internal sanity checks, but then it would fail when we tried to fulfill
an already-failed HTLC:

```
**BROKEN** lightningd: fulfill_our_htlc_out:Output preimage, input failonion?
**BROKEN** lightningd: FATAL SIGNAL 6 (version f82fedb-modded)
**BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x55c09b04f0b5
**BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x55c09b04f247
**BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7f607463551f
**BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7f6074689a7c
**BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7f6074689a7c
**BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7f6074689a7c
**BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7f6074635475
**BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f607461b7f2
**BROKEN** lightningd: backtrace: lightningd/log.c:1016 (fatal_vfmt) 0x55c09afdb7cb
**BROKEN** lightningd: backtrace: lightningd/log.c:1026 (fatal) 0x55c09afdb880
**BROKEN** lightningd: backtrace: lightningd/htlc_end.c:87 (corrupt) 0x55c09afc9472
**BROKEN** lightningd: backtrace: lightningd/htlc_end.c:207 (htlc_out_check) 0x55c09afc9c6b
**BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1451 (fulfill_our_htlc_out) 0x55c09b004dd7
**BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1526 (onchain_fulfilled_htlc) 0x55c09b0050fe
**BROKEN** lightningd: backtrace: lightningd/onchain_control.c:313 (handle_extracted_preimage) 0x55c09afdf9f8
**BROKEN** lightningd: backtrace: lightningd/onchain_control.c:1423 (onchain_msg) 0x55c09afe2da9
**BROKEN** lightningd: backtrace: lightningd/subd.c:557 (sd_msg_read) 0x55c09b019ac8
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-28 10:18:32 +09:30