Commit Graph

522 Commits

Author SHA1 Message Date
Christian Decker cb3ee0ac2e wallet: Load and value `completed_at` timestamp from DB 2022-09-14 13:14:10 +02:00
Rusty Russell 1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell 15751ea1b8 lightningd: do inline parsing for listforwards status parameter
We can do this now the function is cleaned up.

Always better to do the work inside param() since then `check`
gets the benefit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 6c33f7db65 common: remove unused parameter "allow_deprecated" from parse_wireaddr_internal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell eb006dcadd wallet: fix incorrect column-width access.
Postgresql actually checks, and fails.

It's unclear why this field is an INTEGER (and u32) when it's a BIGINT 
in db here (it's an INTEGER in the channels table, just a BIGINT in the
channel_funding_inflights table).

Changelog-Fixed: db: postgresql crash on startup when dual-funding lease open is pending with "s32 field doesn't match size: expected 4, actual 8"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-10 10:34:12 -05:00
niftynei aa7ffb78bd wallet: resolve crash when blockheight is null
We weren't ignoring the 'txindex' field
2022-07-28 12:08:18 +09:30
Rusty Russell e96eb07ef4 lightningd: test that hsm_secret is as expected, at startup.
If you get the wrong hsm_secret, your node_id will change, and
peers won't know who you are, bitcoind will reject your transaction
signatures, and other madness.

Catch this as soon as it happens, by storing our node_id in the db.

Suggested-by: @cdecker, @fiatjaf
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `lightningd` will refuse to start with the wrong node_id (i.e. hsm_secret changes).
2022-07-20 19:28:33 +09:30
Rusty Russell 912ac25270 lightningd: remove 'connected' flag from channel structure.
It's directly a product of "does it have a current owner subdaemon"
and "does that subdaemon talk to peers", so create a helper function
which just evaluates that instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
adi2011 6ba8abb0de lightningd: Add RPC for populating DB with stub channels and set an error on reconnecting 2022-07-14 12:24:48 -05:00
adi2011 eca844eb36 channel: Add struct scb_chan in channel and making last tx optional. 2022-07-14 12:24:48 -05:00
Rusty Russell 3a1a7eb93f wallet: allow saving forwarding scid even if we don't have amount.
They're not logically connected: we can know where they wanted to
go, but we didn't send it.

Where possible, it's the scid *they asked for*; otherwise, it's the
scid or fallback to the alias, but do this in the *caller*, not by
overriding inside wallet_forwarded_payment_add.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Christian Decker 252ccfa7ab db: Store the local alias for forwarded incoming payments
Not only can the outgoing edge be a zeroconf channel, it can also be
the incoming channel. So we revert to the usual trick of using the
local alias if the short_channel_id isn't known yet.

We use the LOCAL alias instead of the REMOTE alias even though the
sender likely used the REMOTE alias to refer to the channel. This is
because we control the LOCAL alias, and we keep it stable during the
lifetime of the channel, whereas the REMOTE one could change or not be
there yet.
2022-07-04 22:14:06 +02:00
Christian Decker 2dc86bf29b db: Store the alias if that's all we got in a forward 2022-07-04 22:14:06 +02:00
Christian Decker de1c0b51f0 zeroconf: Add alias_remote and alias_local to channel and DB
`alias_local` is generated locally and sent to the peer so it knows
what we're calling the channel, while `alias_remote` is received by
the peer so we know what to include in routehints when generating
invoices.
2022-07-04 22:14:06 +02:00
Vincenzo Palazzo 7ff62b4a00 lightnind: remove`DEFAULT_PORT` global definition
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-06-28 06:09:01 +09:30
Rusty Russell 37403e471c pytest: add (failing) test for db with old Tor v2 address.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-24 10:24:39 +09:30
Rusty Russell d18817a525 wallet: don't get so upset on orphaned HTLC.
User grubman on IRC reported a crash due to new HTLC checks:

```
2022-04-05T01:15:17.707Z **BROKEN** wallet: Missing preimage for orphaned HTLC; replacing with zeros
...
2022-04-05T01:15:44.950Z **BROKEN** lightningd: check_already_failed:Both failed and succeeded?
2022-04-05T01:15:45.020Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.11.0rc1-modded)
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0xaaaab8f7f8bb
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (crashdump) 0xaaaab8f7f91b
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa4a315bf
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa45c6cd8
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa45b3a27
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/log.c:821 (fatal_vfmt) 0xaaaab8f3e1b7
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/log.c:829 (fatal) 0xaaaab8f3e24b
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/htlc_end.c:87 (corrupt) 0xaaaab8f325d3
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/htlc_end.c:175 (htlc_out_check) 0xaaaab8f32ee3
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1471 (check_already_failed) 0xaaaab8f53ea7
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1575 (onchain_failed_our_htlc) 0xaaaab8f54aab
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:411 (handle_missing_htlc_output) 0xaaaab8f4037b
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:544 (onchain_msg) 0xaaaab8f409bf
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/subd.c:556 (sd_msg_read) 0xaaaab8f5f8ef
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0xaaaab8fda197
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0xaaaab8fda76f
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0xaaaab8fda82f
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0xaaaab8fdc77f
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0xaaaab8f37857
2022-04-05T01:15:45.021Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1181 (main) 0xaaaab8f3bcff
2022-04-05T01:15:45.021Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa45b3d4f
2022-04-05T01:15:45.021Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xaaaab8f1ed33
```

Since no HTLCs are likely to still be pending since v0.6.1, we can remove that compat
code.  Indeed, this happens on my node: we've not loaded the incoming HTLC because
it's resolved.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-05 15:39:38 +09:30
Rusty Russell e47786da04 pay/sendpay: also store description in case bolt11 uses description_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-04 20:26:04 +09:30
Rusty Russell 7e789be0ea doc: update BOLTs to latest master.
Just typo fixes and the like.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 2f7f7ec255 lightningd: add "style" to listforwards.
Suggested-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` has new entry `style`, currently "legacy" or "tlv".
2022-03-31 19:38:05 +10:30
Rusty Russell aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Rusty Russell 999c734bb5 setchannel: add minhtlc
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
2022-03-22 18:45:41 +10:30
Rusty Russell 6fdcc86f9d lightningd: store htlc_maximum_msat for channel in the db.
We currently don't allow setting it, but it's been requested.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
niftynei 453ef7f0ce db: mark that column is unused/ignored
2022-01-25T23:41:45.2994844Z ----------------------------- Captured stderr call -----------------------------
2022-01-25T23:41:45.2995230Z lightningd: Never accessed column 1 in query SELECT 1  FROM offers WHERE offer_id = ?;
2022-03-05 15:03:34 +10:30
niftynei ce12d2b8a9 database: pull out database code into a new module
We're going to reuse the database controllers for the accounting plugin
2022-03-05 15:03:34 +10:30
niftynei 03c950bae8 db: decouple `fatal` reliance, have as impl defined function
`fatal` is defined in lightningd and has logfile dependencies etc.

Make it more generic by allowing declaration in the use file (wallet.c)
2022-03-05 15:03:34 +10:30
niftynei 36ca175ec7 wallet: was erroring out, saving to null field
Add missing field to first write
2022-02-07 13:02:09 +10:30
niftynei 8225a9decf coin_mvt: log events for pushes/lease_fees for leased channels
We need to stash/save the amount of the lease fees on a leased channel,
we do this by re-using the 'push' amount field on channel (which is
technically correct, since we're essentially pushing the fee amount to
the peer).

Also updates a bit of how the pushes are accounted for (pushed to now
has an event; their channel will open at zero but then they'll
immediately register a push event).

Leases fees are treated exactly the same as pushes, except labeled
differently.

Required adding a 'lease_fee' field to the inflights so we keep track of
the fee for the lease until the open happens.
2021-12-28 04:42:42 +10:30
niftynei bddd3694fa coin_mvt: record fees for an outbound htlc
If we initialized the payment, the fees are the entire fee-chain
(final hop amount - starting hop amount)

If it's a payment we routed, the fees are the diff between the
inbound htlc and the outbound (net gain by this routing)

Added to database so data persists nicely.
2021-12-28 04:42:42 +10:30
niftynei d2c4d4aec2 coin_mvts: rewrite how onchain events are recorded, update tests
The old model of coin movements attempted to compute fees etc and log
amounts, not utxos. This is not as robust, as multi-party opens and dual
funded channels make it hard to account for fees etc correctly.

Instead, we move towards a 'utxo' view of the onchain events. Every
event is either the creation or 'destruction' of a utxo. For cases where
the value of the utxo is not (fully) debited/credited to our account, we
also record the output_value. E.g. channel closings spend a utxo who's
entire value we may not own.

Since we're now tracking UTXOs onchain, we can now do more complex
assertions about the onchain footprint of them. The integration tests
have been updated to now use more 'chain aware' assertions about the
ending state.
2021-12-28 04:42:42 +10:30
Ken Sedgwick e8f43ef6ca wallet: make wallet_can_spend non-static because needed 2021-12-14 11:24:52 +10:30
Rusty Russell 484222b0a1 daemons: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
ZmnSCPxj jxPCSnmZ 3433ff5e15 wallet/db.c, wallet/wallet.c: Add a partial index to speed up startup.
Closes: #4901

Tested by `EXPLAIN QUERY PLAN` on sqlite3; #4901 shows the result from
@whitslack doing a similar partial index on PostgreSQL on his ~1000 chan
node.

ChangeLog-Added: db: Speed up loading of pending HTLCs during startup by using a partial index.
2021-12-02 11:54:57 +01:00
Simon Vrouwe 63bd569bf6 lightningd: cleanup, freeing jsonrpc in shutdown cannot trigger db write's anymore
since PR #3867 utxos are unreserved by height, destroy_utxos and
related functions are not used anymore so clean them up also

However free(ld->jsonrpc) still needs to happen before free(ld) because its
destructors need list_head pointers from ld
2021-11-30 13:34:44 +10:30
Rusty Russell 2e6df89274 wallet: remove redundant field counters, ignored fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell c462ccae1a wallet: have db track what columns are accessed in DEVELOPER mode.
And add db_col_ignore helper for cases where it's deliberate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell b8a240bd15 wallet: convert wallet.c to db_col_ APIs
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell 787fbb1228 db: create simple hashtable of fields in SELECT.
This simplistically maps names to numbers, eg:

	SELECT foo, bar FROM tbl;

'foo' -> 0
'bar' -> 1

If a statement is too complex for our simple parsing, we treat it as a
single field (which currently it always is).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
niftynei 42e40c1ced htlcs: add flag to 'fail immediately'
If we're over the dust limit, we fail it immediatey *after* commiting
it, but we need a way to signal this throughout the lifecycle, so we add
it to htlc_in struct and persist it through to the database.

If it's supposed to be failed, we fail after the commit cycle is
completed.
2021-10-23 12:59:13 +02:00
niftynei 1fe829c546 lightningd: new option for htlc dust limit
To reduce the surface area of amount of a channel balance that can be
eaten up as htlc dust, we introduce a new config
'--max-dust-htlc-exposure-msat', which sets the max amount that any
channel's balance can be added as dust

Changelog-Added: config: new option --max-dust-htlc-exposure-msat, which limits the total amount of sats to be allowed as dust on a channel
2021-10-23 12:59:13 +02:00
Rusty Russell da03985996 wallet: only hand onchaind the HTLCs it needs to know.
This will make closing long-lived channels more efficient, and it's
just nicer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell c503232cde common: use bitcoin_outpoint.
I started pulling this thread, and the entire codebase got unravelled.

Oh well, it's done now!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell 433e61fcb9 wallet: remove unnecessary data from channel_htlcs when htlc is dead.
In particular, the onion and errors can be large, but now we'll never
need to retransmit them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell 8a85bf6880 lightningd: populate min/max commit fields in db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell bdaec48400 wallet: wrap htlc_state enum in db function.
All enums in the db should be wrapped this way on reading/writing them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Christian Decker 5f260840ab jsonrpc: Add `groupid` to `waitsendpay` 2021-10-13 13:41:18 +10:30
Christian Decker 817b1b361e db: Add `groupid` to HTLCs 2021-10-13 13:41:18 +10:30
Christian Decker 0bb0d8a707 wallet: Add function to retrieve the latest groupid for a payment
When doing things like `waitsendpay` without specifying the `groupid`
we likely want to use the latest `groupid` we created, since that's
the one in flight. This adds a function to quickly retrieve that.
2021-10-13 13:41:18 +10:30
Christian Decker ba4be0313b db: Add groupid to the payments table 2021-10-13 13:41:18 +10:30