The latter is used when we're put in the db, the former is the uncommitted state.
Currently dbid == 0 is used in addition to the state, which is unwieldy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: JSON-RPC: added new dual-funding state `DUALOPEND_OPEN_COMMITTED`
We usually hand times by copy, not by pointer (and if we did, they should
be const!). I noticed this particularly for the state changed code, but
it goes down to to json_add_timeiso, so I fixed that too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently it's half done in funding_depth_cb, and half in
channeld_tell_depth. It's very confusing as a result,
with splicing, dual-funding and zeroconf.
This does introduce a behaviour change: if a channel is NORMAL and
it gets reorganized, we force close (unless we were the one who funded
it, or it's zeroconf anyway). This is safer than continuing to use
the channel in this case!
Some tests are changed to zeroconf to make them work, but v2 doesn't
support zeroconf, so that's removed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And require --developer to use them.
Also refuse redirection to deprecated APIs if deprecated APIs are disabled!
Changelog-Removed: `dev-sendcustommsg` (use `sendcustommsg`, which was added in v0.10.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Some functions have vanished in master, and it's confusing to
see this change when we run `make update-mocks` later in the
series.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can keep a single array of 'already considered' utxos, with the same
result as Tony's patch prior.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If a node has an onchain balance with at least one uneconomical UTXO, the fundchannel RPC call will lock up the node and will eventually crash it with OOM issues if the economical UTXO(s) do not add up to the fundchannel amount. This is because the while loop never exits because it keeps pulling in the same uneconomical UTXOs forever.
Changelog-Fixed: wallet: fundchannel no longer loops forever if the wallet contains insufficient funds, but an uneconomical UTXO.
It always is for runes we create, but in theory you can take our secret key
and make our own runes with your own tools.
(We correctly refuse runes without uniqueids if they're *not* ours
anyway: uniqueid is only used for our own runes).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
"id" is a magic name, so it was being populated by sqlite3
automatically, starting at 0. Fortunately, we only fetched by id in
one place: to indicate the `stored` flag when asked about an explicit
rune in `showrunes`.
Reported-by: @ShahanaFarooqui
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `showrunes` on a specific rune would always say `stored`: false.
Don’t send the funding spend to onchaind if we detect it in inflights (aka. a splice). While we already prevented onchaind_funding_spent from being called directly, the call to wallet_channeltxs_add meant onchaind_funding_spent would be called *anyway* on restart. This is now fixed.
Additionally there was a potential for a race problem depending on the firing order of the channel depth and and funding spent events.
Instead of requiring these events fire in a specific order, we make a special “memory only” inflight object to prevent the race regardless of firing order.
Changelog-Fixed: Splice: bugfix for restart related race condition interacting with adversarial close detection.
Avoids a gratuitous "ctx" field, and the simplified declaration
is now understood by `make update-mocks`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was changed by mistake in 23fafe98e3b4f4e15cdf764e1346d813d1eb8039: if
it's null we turn it into 0 (which is what the default call does, but it
does log BROKEN about it!):
```
2023-08-03T14:10:49.001Z **BROKEN** lightningd: Accessing a null column total_msat/15 in query SELECT id, status, destination, msatoshi, payment_hash, timestamp, payment_preimage, path_secrets, route_nodes, route_channels, msatoshi_sent, description, bolt11, paydescription, failonionreply, total_msat, partid, local_invreq_id, groupid, completed_at FROM payments ORDER BY id;
```
Fixes: #6501
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Thread the signed tx through so close's JSON return contains that,
rather than the unsigned channel->last_tx.
We have to split the "get cmd_id" from "resolve the close commands" though;
and of course, as before, we don't actually print the txids of multiple
transactions even though we may have multi in flight due to splice!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `close` returns a `tx` field with witness data populated (i.e. signed).
Fixes: #6440
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.
Implement the channeld splicing protocol leveraging the interactivetx protocol.
Implement lightningd’s channel_control to support channeld in its splicing efforts.
Changelog-Added: Added the features to enable splicing & resizing of active channels.
It's confusing: we can (and should) load this before other operations, though we don't actually need to yet. But more importantly, don't put it under the "outpointfilters_init" trace span.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Clean restart of daemon after a tx-abort is a nice way to work around
the 'persistent' disconnect that we t-bast noticed.
Changelog-Fixed: `dualopend`: Fix behavior for tx-aborts. No longer hangs, appropriately continues re-init of RBF requests without reconnction msg exchange.
Pointed out by @ShahanaFarooqui, we leave a single unused entry in the datastore,
so we should clean that up too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If they have invalid runes, we bail, but if they have runes which used
a different master secret (old commando.py allowed you to override
secret), we just complain and delete them.
Note that this requires more mocks in wallet/test/run-db.c...
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The wallet_datastore_first() SELECT statement only iterates from the
given key (if any), relying on the caller to notice when the key no
longer applies. (e.g. startkey = ["foo", "bar"] will return key
["foo", "bar"] then ["foo", "bar", "child" ], then ["foo", "baz"]).
The only caller (listdatastore) would notice the keychange and stop
looping, but reallly wallet_datastore_next() should do this. When I
tried to use it for migrations, I got very confused!
Also, several places want a simple "wallet_datastore_get()" function,
so provide that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
During migrations, wallet doesn't exist yet, so we use raw db. Split
functions into lower-level ones and make public API a simple wrapper.
Unfortunately, this means db_datastore_next needs to proceed db_datastore_first
since they're now static (and first calls next), plus, fix some weird indents,
so diff is bigger than expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If you miss a wait event, you can catch up by doing listinvoices and
getting the max of these fields. It's also a good debugging clue.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we have defined ordering, we can add a start param.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listinvoices` has `index` and `start` parameters for listing control.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `setchannel` adds a new `ignorefeelimits` parameter to allow peer to set arbitrary commitment transaction fees on a per-channel basis.
`struct log` becomes `struct logger`, and the member which points to the
`struct log_book` becomes `->log_book` not `->lr`.
Also, we don't need to keep the log_book in struct plugin, since it has
access to ld's log_book.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently our CI is not able to complete the compilation because
there is the following compilation error introduced in `0bcff1e76d`
```
cc wallet/db.c
wallet/db.c: In function 'migrate_normalize_invstr':
wallet/db.c:1734:3: error: too many arguments to function 'db_bind_text'
1734 | db_bind_text(update_stmt, 0, invstr);
| ^~~~~~~~~~~~
In file included from wallet/db.c:10:
./db/bindings.h:25:6: note: declared here
25 | void db_bind_text(struct db_stmt *stmt, const char *val);
| ^~~~~~~~~~~~
wallet/db.c:1735:3: error: too many arguments to function 'db_bind_u64'
1735 | db_bind_u64(update_stmt, 1, id);
| ^~~~~~~~~~~
In file included from wallet/db.c:10:
./db/bindings.h:23:6: note: declared here
23 | void db_bind_u64(struct db_stmt *stmt, u64 val);
| ^~~~~~~~~~~
wallet/db.c:1758:3: error: too many arguments to function 'db_bind_text'
1758 | db_bind_text(update_stmt, 0, invstr);
| ^~~~~~~~~~~~
In file included from wallet/db.c:10:
./db/bindings.h:25:6: note: declared here
25 | void db_bind_text(struct db_stmt *stmt, const char *val);
| ^~~~~~~~~~~~
wallet/db.c:1759:3: error: too many arguments to function 'db_bind_u64'
1759 | db_bind_u64(update_stmt, 1, id);
| ^~~~~~~~~~~
In file included from wallet/db.c:10:
./db/bindings.h:23:6: note: declared here
23 | void db_bind_u64(struct db_stmt *stmt, u64 val);
| ^~~~~~~~~~~
make: *** [Makefile:299: wallet/db.o] Error 1
make: *** Waiting for unfinished jobs....
rm external/build-x86_64-linux-gnu/libwally-core-build/src/secp256k1/libsecp256k1.la
```
Fixes: 0bcff1e76d
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This is almost always true already; fix up the few non-standard ones.
This is enforced with an assert, and I ran the entire test suite to
double-check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can expose the dbid, rather than pretending we have some "struct
invoice" which is actually just the dbid. And don't have a pile of
"wallet_" wrappers for redirection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit make a db migration to canonicalize all the
invoice string stored inside the database.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
We usually have access to `ld`, so avoid the global.
The only place generic code needs it is for the json command struct,
and that already has accessors: add one for libplugin and lightningd
to tell it if deprecated apis are OK.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.
Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is needed when we know we're *opening* an anchor channel, to
override the "do we already have an anchor channel open?" logic.
Also, document the nonwrapped arg added in v23.02.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundpsbt` and `utxopsbt` new parameter `opening_anchor_channel` so lightningd knowns it needs emergency reserve for anchors.
This is the simple version which always tries to keep some sats if we
have an anchor channel. Turns out that we need something more
sophisticated for multifundchannel, so that's next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (and `all` will be reduced appropriately).
Changelog-Changed: JSON-RPC: `fundpsbt` and `utxopsbt` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels.
If you did call fundpsbt with amount 'all' and `excess_as_change`
true, you would get everything going to the change output. That's
obviously not the intention, and we'd like to use this to add change
outputs even for "all" when have keep emergency reserves.
And change the finish_psbt() API to take an explicit change amount:
at the moment it's either all or nothing, but that will change with
emergency-sat reserves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In most cases, it's the same as option_anchor_outputs, but for
fees it's different. This transformation is the simplest:
pass it as a pair, and test it explicitly.
In future we could rationalize some paths, but this was nice
and mechanical.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't actually use it anywhere, but we actually want to now for
CPFP. So give it more parameters and make it return bool so it can
be set without necessarily suppressing rexmit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were setting the wrong input number: don't assume it's the
same as the UTXO number, but simply the last-appended input.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It'll wrap, probably be in the past, and infinite loop. This was caused by an invoice
with expiry set at 2076. This wrap caused us to think the expiry has already
passed, and keep looping!
Reported-by: @telelvis
Fixes: #6339
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightnind: don't infinite loop on 32 bit platforms if only invoices are expiring after 2038.
1. Make it the standard "return the error" pattern.
2. Rather than flags to indicate what types are allowed, have the callers
check the return explicitly.
3. Document the APIs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We removed the (experimental-only!) annotation output in 611795beee
but we still loaded them from the db. Turns out that we were putting bogus
annotations into the db, and accessing out of range when loading them.
Consider the following db entry in transaction_annotations:
```
CREATE TABLE transaction_annotations ( txid BLOB, idx INTEGER, location INTEGER, type INTEGER, channel INTEGER REFERENCES channels(id), UNIQUE(txid, idx));
...
INSERT INTO transaction_annotations VALUES(X'19706f9af2875508a06c7db1754ef7ecb3da745ead005992e626441e4e83465f',18,1,129,53699);
```
Here is the corresponding entry in txs:
```
INSERT INTO transactions VALUES(X'19706f9af2875508a06c7db1754ef7ecb3da745ead005992e626441e4e83465f',710327,966,X'02000000000101f2add69112a1d557317826120e1f4ea3bc1cbe4674d720325695b26ecfe8355d120000000000000000013634000000000000160014dca21f104359bbb81e88ed7da985549f2cd0cbc30347304402201cdc854b76c4c7523e3ca09f38a81539d3b2f7fbd9a0de6ae10b7ceaa65ed9d402205a1770058cd1ef081c77c2fe957c07a334cb3a11bc0cc502834a29c59424fe010120589da1f809d955c7af150bf53123c27ffc0a741489b5291f6be811189863ec838576a9142f188d0d973c4ad1865a619d3748340b30746e088763ac672103b7bbcd592197ba6501e7176aabd3f908d94b126ae82ab1e7a4c58f5a789782e57c820120876475527c21029bcf62114eb36758fcb1aead7e67b6f707d32f34e67816894d5211ac9f2d6ce752ae67a9144483a115219ba65c63a3844be8445f739703bea988ac686800000000',NULL,NULL);
```
The annotation refers to output 18 of the tx, but it only has one output!
However, decoding the tx shows that it spent output 18 of a previous tx, so
that's probably where the `18` came from.
Remove this logic: we can remove the remaining (clearly broken!) annotation
adding code in another cleanup commit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was always the intent, but now we have to reconstruct from the
disparate fields.
This means `option_anchor_outputs` is now redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I couldn't figure out why my new SQL query was returning 0 rows,
and it was because we were ignoring errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fun story. We're changing onchaind to hand txs to us, and we will
construct them and do the broadcast for it. lightningd tells onchaind
the witness it used (with flags to indicate which fields were
signatures so should be ignored) so onchaind can recognize the tx
when/if it is mined.
And when onchaind was waiting for a CLTV delay, it wouldn't tell
lightningd yet, but wait until the parent was sufficiently deep
But this caused bugs!
In particular, on replay, onchaind would see transactions which it
hasn't sent yet. This was not a problem before, as onchaind had
created the tx, even if it hadn't told lightningd to broadcast it, so
recognized the variant when it came in. When we're relying on
lightningd to tell us what the tx will look like, this doesn't work
any more.
The cause of this is that we fire off txowatches ("this output was
spent!") while we process blocks, and only fire off txwatches ("this
tx increased depth") once all the current blocks are processed. Often
this didn't matter, since we replay messages to onchaind from the
database, *but* we trim the last few blocks on restart (or, if there's
a small reorg while we're stopped), and we can hit this misordering.
Changing our topology code to only ever process one block at a time
would be a solution, but slows down catchup (and tests, where we often
mine a run of blocks).
So, this seems like a premature optimization, but it's really
required! And in future, lightningd can use this knowledge of pending
transactions to combine them in more clever ways.
Note that if a tx is valid at block N, we broadcast it once we see
block N-1, to get it in the mempool for block N.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There are cases (difficult to reproduce with a test) where
a payment will fail one time and succeed later.
As far I understand in this case the groupid field of the payment
is the same, and the only thing that change is the status, so
our logic inside the delpay is ambiguous where it is not
possible to delete a payment as described in https://github.com/ElementsProject/lightning/issues/6114
A sequence of commands that explain the problem is
```
$ lc -k listpays payment_hash=H
{
"pays": [
{
"bolt11": "I",
"destination": "redacted",
"payment_hash": "H",
"status": "complete",
"created_at": redacted,
"completed_at": redacted,
"preimage": "P",
"amount_msat": "redacted",
"amount_sent_msat": "redacted"
}
]
}
$ lc delpay H complete
{
"code": 211,
"message": "Payment with hash H has failed status but it should be complete"
}
```
In this case, the delpay is not able to delete a payment because the
listpays is returning only the succeeded one, so by running the
listsendpays we may see the following result where our delpay logic
will be stuck because it works to ensure that all the payments stored
in the database has the status specified by the user
```
➜ VincentSSD clightning --testnet listsendpays -k payment_hash=7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4
{
"payments": [
{
"id": 322,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 1,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 1664,
"created_at": 1679510203,
"completed_at": 1679510205,
"status": "failed",
"bolt11": "lntb1pjpkj4xsp52trda39rfpe7qtqahx8jjplhnj3tatxy8rh6sc6afgvmdz7n0llspp50lr5hmdm0re0xvcp2hv3nf2wwvx0r8q3h3e7jmqz0awdfg6w206qdp0w3jhxarfdenjqargv5sxgetvwpshjgrzw4njqun9wphhyaqxqyjw5qcqp2rzjqtp28uqy77te96ylt7ek703h4ayldljsf8rnlztgf3p8mg7pd0qzwf8a3yqqpdqqqyqqqqt2qqqqqqgqqc9qxpqysgqgeya2lguaj6sflc4hx2d89jvah8mw9uax4j77d8rzkut3rkm0554x37fc7gy92ws9l76yprdva2lalrs7fqjp9lcx40zuty8gca0g5spme3dup"
},
{
"id": 323,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 2,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 3663,
"created_at": 1679510205,
"completed_at": 1679510207,
"status": "failed"
},
{
"id": 324,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 3,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 3663,
"created_at": 1679510207,
"completed_at": 1679510209,
"status": "failed"
},
{
"id": 325,
"payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
"groupid": 1,
"partid": 4,
"destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
"amount_msat": 300,
"amount_sent_msat": 4663,
"created_at": 1679510209,
"completed_at": 1679510221,
"status": "complete",
"payment_preimage": "43f746f2d28d4902489cbde9b3b8f3d04db5db7e973f8a55b7229ce774bf33a7"
}
]
}
```
This commit solves the problem by forcing the delete query in the
database to specify status too, and work around this kind of
ambiguous case.
Fixes: f52ff07558 (lightningd: allow delpay to delete a specific payment.)
Reported-by: Antoine Poinsot <darosior@protonmail.com>
Link: https://github.com/ElementsProject/lightning/issues/6114
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Co-Developed-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: delpay be more pedantic about delete logic by allowing
delete payments by status directly on the database.
Rusty Russell
hMsats
ShahanaFarooqui
Tony Giorgio
Dusty Daemon
niftynei
Christian Decker
Vincenzo Palazzo
Greg Sanders