We usually have access to `ld`, so avoid the global.
The only place generic code needs it is for the json command struct,
and that already has accessors: add one for libplugin and lightningd
to tell it if deprecated apis are OK.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.
Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is needed when we know we're *opening* an anchor channel, to
override the "do we already have an anchor channel open?" logic.
Also, document the nonwrapped arg added in v23.02.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundpsbt` and `utxopsbt` new parameter `opening_anchor_channel` so lightningd knowns it needs emergency reserve for anchors.
This is the simple version which always tries to keep some sats if we
have an anchor channel. Turns out that we need something more
sophisticated for multifundchannel, so that's next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (and `all` will be reduced appropriately).
Changelog-Changed: JSON-RPC: `fundpsbt` and `utxopsbt` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels.
If you did call fundpsbt with amount 'all' and `excess_as_change`
true, you would get everything going to the change output. That's
obviously not the intention, and we'd like to use this to add change
outputs even for "all" when have keep emergency reserves.
And change the finish_psbt() API to take an explicit change amount:
at the moment it's either all or nothing, but that will change with
emergency-sat reserves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In most cases, it's the same as option_anchor_outputs, but for
fees it's different. This transformation is the simplest:
pass it as a pair, and test it explicitly.
In future we could rationalize some paths, but this was nice
and mechanical.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't actually use it anywhere, but we actually want to now for
CPFP. So give it more parameters and make it return bool so it can
be set without necessarily suppressing rexmit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were setting the wrong input number: don't assume it's the
same as the UTXO number, but simply the last-appended input.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It'll wrap, probably be in the past, and infinite loop. This was caused by an invoice
with expiry set at 2076. This wrap caused us to think the expiry has already
passed, and keep looping!
Reported-by: @telelvis
Fixes: #6339
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightnind: don't infinite loop on 32 bit platforms if only invoices are expiring after 2038.
1. Make it the standard "return the error" pattern.
2. Rather than flags to indicate what types are allowed, have the callers
check the return explicitly.
3. Document the APIs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We removed the (experimental-only!) annotation output in 611795beee
but we still loaded them from the db. Turns out that we were putting bogus
annotations into the db, and accessing out of range when loading them.
Consider the following db entry in transaction_annotations:
```
CREATE TABLE transaction_annotations ( txid BLOB, idx INTEGER, location INTEGER, type INTEGER, channel INTEGER REFERENCES channels(id), UNIQUE(txid, idx));
...
INSERT INTO transaction_annotations VALUES(X'19706f9af2875508a06c7db1754ef7ecb3da745ead005992e626441e4e83465f',18,1,129,53699);
```
Here is the corresponding entry in txs:
```
INSERT INTO transactions VALUES(X'19706f9af2875508a06c7db1754ef7ecb3da745ead005992e626441e4e83465f',710327,966,X'02000000000101f2add69112a1d557317826120e1f4ea3bc1cbe4674d720325695b26ecfe8355d120000000000000000013634000000000000160014dca21f104359bbb81e88ed7da985549f2cd0cbc30347304402201cdc854b76c4c7523e3ca09f38a81539d3b2f7fbd9a0de6ae10b7ceaa65ed9d402205a1770058cd1ef081c77c2fe957c07a334cb3a11bc0cc502834a29c59424fe010120589da1f809d955c7af150bf53123c27ffc0a741489b5291f6be811189863ec838576a9142f188d0d973c4ad1865a619d3748340b30746e088763ac672103b7bbcd592197ba6501e7176aabd3f908d94b126ae82ab1e7a4c58f5a789782e57c820120876475527c21029bcf62114eb36758fcb1aead7e67b6f707d32f34e67816894d5211ac9f2d6ce752ae67a9144483a115219ba65c63a3844be8445f739703bea988ac686800000000',NULL,NULL);
```
The annotation refers to output 18 of the tx, but it only has one output!
However, decoding the tx shows that it spent output 18 of a previous tx, so
that's probably where the `18` came from.
Remove this logic: we can remove the remaining (clearly broken!) annotation
adding code in another cleanup commit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was always the intent, but now we have to reconstruct from the
disparate fields.
This means `option_anchor_outputs` is now redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I couldn't figure out why my new SQL query was returning 0 rows,
and it was because we were ignoring errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fun story. We're changing onchaind to hand txs to us, and we will
construct them and do the broadcast for it. lightningd tells onchaind
the witness it used (with flags to indicate which fields were
signatures so should be ignored) so onchaind can recognize the tx
when/if it is mined.
And when onchaind was waiting for a CLTV delay, it wouldn't tell
lightningd yet, but wait until the parent was sufficiently deep
But this caused bugs!
In particular, on replay, onchaind would see transactions which it
hasn't sent yet. This was not a problem before, as onchaind had
created the tx, even if it hadn't told lightningd to broadcast it, so
recognized the variant when it came in. When we're relying on
lightningd to tell us what the tx will look like, this doesn't work
any more.
The cause of this is that we fire off txowatches ("this output was
spent!") while we process blocks, and only fire off txwatches ("this
tx increased depth") once all the current blocks are processed. Often
this didn't matter, since we replay messages to onchaind from the
database, *but* we trim the last few blocks on restart (or, if there's
a small reorg while we're stopped), and we can hit this misordering.
Changing our topology code to only ever process one block at a time
would be a solution, but slows down catchup (and tests, where we often
mine a run of blocks).
So, this seems like a premature optimization, but it's really
required! And in future, lightningd can use this knowledge of pending
transactions to combine them in more clever ways.
Note that if a tx is valid at block N, we broadcast it once we see
block N-1, to get it in the mempool for block N.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There are cases (difficult to reproduce with a test) where
a payment will fail one time and succeed later.
As far I understand in this case the groupid field of the payment
is the same, and the only thing that change is the status, so
our logic inside the delpay is ambiguous where it is not
possible to delete a payment as described in https://github.com/ElementsProject/lightning/issues/6114
A sequence of commands that explain the problem is
```
$ lc -k listpays payment_hash=H
{
"pays": [
{
"bolt11": "I",
"destination": "redacted",
"payment_hash": "H",
"status": "complete",
"created_at": redacted,
"completed_at": redacted,
"preimage": "P",
"amount_msat": "redacted",
"amount_sent_msat": "redacted"
}
]
}
$ lc delpay H complete
{
"code": 211,
"message": "Payment with hash H has failed status but it should be complete"
}
```
In this case, the delpay is not able to delete a payment because the
listpays is returning only the succeeded one, so by running the
listsendpays we may see the following result where our delpay logic
will be stuck because it works to ensure that all the payments stored
in the database has the status specified by the user
```
➜ VincentSSD clightning --testnet listsendpays -k payment_hash=7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4
{
"payments": [
{
"id": 322,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 1,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 1664,
"created_at": 1679510203,
"completed_at": 1679510205,
"status": "failed",
"bolt11": "lntb1pjpkj4xsp52trda39rfpe7qtqahx8jjplhnj3tatxy8rh6sc6afgvmdz7n0llspp50lr5hmdm0re0xvcp2hv3nf2wwvx0r8q3h3e7jmqz0awdfg6w206qdp0w3jhxarfdenjqargv5sxgetvwpshjgrzw4njqun9wphhyaqxqyjw5qcqp2rzjqtp28uqy77te96ylt7ek703h4ayldljsf8rnlztgf3p8mg7pd0qzwf8a3yqqpdqqqyqqqqt2qqqqqqgqqc9qxpqysgqgeya2lguaj6sflc4hx2d89jvah8mw9uax4j77d8rzkut3rkm0554x37fc7gy92ws9l76yprdva2lalrs7fqjp9lcx40zuty8gca0g5spme3dup"
},
{
"id": 323,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 2,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 3663,
"created_at": 1679510205,
"completed_at": 1679510207,
"status": "failed"
},
{
"id": 324,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 3,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 3663,
"created_at": 1679510207,
"completed_at": 1679510209,
"status": "failed"
},
{
"id": 325,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 4,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 4663,
"created_at": 1679510209,
"completed_at": 1679510221,
"status": "complete",
"payment_preimage": "43f746f2d28d4902489cbde9b3b8f3d04db5db7e973f8a55b7229ce774bf33a7"
}
]
}
```
This commit solves the problem by forcing the delete query in the
database to specify status too, and work around this kind of
ambiguous case.
Fixes: f52ff07558 (lightningd: allow delpay to delete a specific payment.)
Reported-by: Antoine Poinsot <darosior@protonmail.com>
Link: https://github.com/ElementsProject/lightning/issues/6114
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Co-Developed-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: delpay be more pedantic about delete logic by allowing
delete payments by status directly on the database.
We don't cover three common patterns:
1. Optional integers (db_col_u64 has different form from structs)
2. Optional strings.
3. Optional array fields.
But it does neaten and reduce the scope for cut&paste errors in the
common "if not-NULL, tal and assign".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We actually have an assertion that there are no channels remaining when
we delete peers, so this is confusing!
Actually removing the constraint is db-specific and deeply non-trivial.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
PSBTv2 support is quite low in the ecosystem, so having a call to convert
log messages and the like should be useful since they'll often be in v2.
Changelog-Added: Added setpsbtversion RPC to aid debugging and compatibility
Libwally update breaks compatibility, so
we do this in one large step.
Changelog-Changed: JSON-RPC: elements network PSET now only supports PSETv2.
Changelog-Added: JSON-RPC: PSBTv2 supported for fundchannel_complete, openchannel_update, reserveinputs, sendpsbt, signpsbt, withdraw and unreserveinputs parameter psbt, openchannel_init and openchannel_bump parameter initialpsbt, openchannel_signed parameter signed_psbt and utxopsbt parameter utxopsbt
`struct lightningd` is not completely initialized, so we added a
"migration_context" which only had some of the fields. But we ended
up handing in `struct lightningd` anyway, so I don't think this
complexity is worthwhile.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
At the moment only lightingd needs it, and this avoids missing any
places where we do bip32 derivation.
This uses a hsm capability to mean we're backwards compatible with older
hsmds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now always double-check bitcoin addresses are correct (no memory errors!) before issuing them.
It's needed as the db and wallet is being set up (db migrations), so
it's simpler this way to always use ld->bip32_base for the next patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Importantly, adds the version number at the *front* to help future
parsing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fix-hsm-check-pubkey.patch':
fixup! hsmd: capability addition: ability to check pubkeys.
e778ebb9af ("wallet: only log broken if we
have duplicate scids in channels.") downgraded the fatal() to a broken
log message, but the user reports it still won't start up.
Perhaps they're hitting the fatal() outside the loop? (And we're
not getting that output).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Missed a DEFAULT in the db clause.
Feb 15 16:02:12 citrine lightningd[902093]: Accessing a null column lease_satoshi/15 in query SELECT funding_tx_id, funding_tx_outnum, funding_feerate, funding_satoshi, our_funding_satoshi, funding_psbt, last_tx, last_sig, funding_tx_remote_sigs_received, lease_expiry, lease_commit_sig, lease_chan_max_msat, lease_chan_max_ppt, lease_blockheight_start, lease_fee, lease_satoshi FROM channel_funding_inflights WHERE channel_id = ? ORDER BY funding_feerate
Fixes#6016
People are upgrading to 22.11.1 not, and in some configurations like the one
mentioned in the issue, we should
put some info information in the log when we are not able to upgrade.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
It's not likely but possible that the node's settings will shift btw a
start and an RBF; we persist the setting to the database so we don't
lose it.
Right now holding onto it forever is kind of extra but maybe we'll
reuse the setting for splices? idk.
Should this be a channel type??
technically we don't need this info after the channel opens, but for any
subsequent RBF (and maybe splice?) we need to remember what the
open/accept peer signaled
We need to be able to only use non-wrapped inputs for v2/interactive tx
protocol.
Changelog-Added: JSONRPC: `fundpsbt` option `nonwrapped` filters out p2sh wrapped inputs
Rusty Russell
Greg Sanders
Vincenzo Palazzo
Kristaps Kaupe
niftynei