Commit Graph

11866 Commits

Author SHA1 Message Date
Alex Myers 4167fe8dd9 gossip_store: fix offset error
The gossip_store version byte was unaccounted for in the initial traversal
of gossip_store_end. This lead to an offset and a bogus message length
field. As a result, an early portion of the gossip_store could have been
skipped, potentially leading to gossip propagation issues downstream.

Fixes #5572 #5565

Changelog-fixed: proper gossip_store operation may resolve some previous gossip propagation issues
2022-09-14 13:14:32 +02:00
Christian Decker f64d755e43 pay: Aggregate `completed_at` in `listpays` and `pay`
The completion of a successful payment is defined as the first
completion of a successful part, hence we just collect the minimum
completion time of successes.

Changelog-Added: JSON-RPC: `pay` and `listpays` now lists the completion time.
2022-09-14 13:14:10 +02:00
Christian Decker 246e1fb0b3 wallet: Set the `completed_at` timestamp when updating the status 2022-09-14 13:14:10 +02:00
Christian Decker a80211e960 doc: Update generated artifacts to match master 2022-09-14 13:14:10 +02:00
Christian Decker cb3ee0ac2e wallet: Load and value `completed_at` timestamp from DB 2022-09-14 13:14:10 +02:00
Christian Decker daeec66bd7 db: Add completed_at field to payments
We'll want to use these to track durations for `sendpays` and `pays`.
2022-09-14 13:14:10 +02:00
niftynei 3ad8347969 bkpr-test: maybe fix race in test_bookkeeping_closing_trimmed_htlcs
test_bookkeeping_closing_trimmed_htlcs fails to find 'all outputs
resolved' occassionally, seems like it's because the
OUR_DELAYED_TO_WALLET doesn't make it into the mempool before we start
mining blocks?

So here make sure there's something in the mempool before before we
start making new blocks.
2022-09-14 15:59:37 +09:30
niftynei efad09f966 bkpr: confirm that replaying the open+lock-in txs at start is ok
Make sure that we're not issuing duplicate lease_fee events!
2022-09-14 15:59:37 +09:30
niftynei c143914ebf bkpr: migration to delete any duplicate lease_fee entries
Clean up for #5557.

If you've got duplicate 'lease_fee' entries, we delete them!
2022-09-14 15:59:37 +09:30
niftynei 1980ba420b notif: dont send balance snapshot for not yet opened channel
We were double counting channel lease fees because we were double firing
the channel open event sequence (so to speak). If we don't report
balances for unopened channels, we don't have this problem?

Changelog-Changed: Plugins: `balance_snapshot` notification does not send balances for channels that aren't locked-in/opened yet
2022-09-14 15:59:37 +09:30
niftynei 8452d903b4 bkpr: failing test for bookkeeper crash
Reproduce crash for #5557!

If we record the channel open because bookkeeper was added after the
channel open request started but the channel confirms later, we end up
with re-recording any associated push or leased fees (paid or rcvd).

In the case where you've paid for these fees, your channel balance goes
negative and the node crashes the next time you call `listbalances`.

Reported-by: @chrisguida
2022-09-14 15:59:37 +09:30
Michael Schmoock e0d6f3ceb1 connectd: DNS Bolt7 #911 no longer EXPERIMENTAL
Changelog-Changed: Bolt7 #911 DNS annoucenent support is no longer EXPERIMENTAL
2022-09-13 06:42:20 +09:30
Rusty Russell 375215a141 lightningd: more graceful shutdown.
Be more graceful in shutting down: this should fix the issue where
bookkeeper gets upset that its commands are rejected during shutdown,
and generally make things more graceful.

1. Stop any new RPC connections.
2. Stop any per-peer daemons (channeld, etc).
3. Shut down plugins.
4. Stop all existing RPC connections.
5. Stop global daemons.
6. Free up peer, chanen HTLC datastructures.
7. Close database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: RPC operations are now still available during shutdown.
2022-09-12 14:00:41 +02:00
Rusty Russell e853cdc3ff db: fix sqlite3 code which manipulates columns.
Because it used internal routines, it didn't pass operations through the
db hook!  So make it use the generic routines, with the twist that they
are not translated.

And when we use this in a migration hook, we're actually in a
transaction.

This, in turn, introduces an issue: we need to be outside a transaction
to "PRAGMA foreign_keys = OFF", but completing the transaction when
there is a db hook actually enters the io loop, freeing the tmpctx!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell d7aa2749c3 db: fix migrations which write to db.
valgrind noticed that this was uninitialized when I tried a complex
migration.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell 6438ee4126 doc: disallow additional properties in sendcustommsg.
We have to scatter this everywhere in our schemas, as there's no way
to make it the default :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell b3b5b74069 libplugin: allow NULL calllbacks for jsonrpc_set_datastore.
You often don't care about the reply, so this is quite convenient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell fb433a70f8 doc: escape output types (esp `short_channel_id`).
We can also remove the listpeers closer hack, which was removed from
the schema already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell bef2a47ab7 db: fix renaming/deleting cols of DBs when there are UNIQUE(x, b, c) constraints.
Get stricter with recognizing real column defs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell 2c3d4e46fc tools/test: fix very confused code.
This broke with COPTFLAGS="-flto -O3", and so I took a look (it
complains more than normal because main isn't there).  We should never
be running update-mocks except on programs expected to compile: in
this case, that's tools/test/run-test-wire.c.

Remove the code which tries to run this, which also means
non-developers won't be running update-mocks!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell 2ac775f9f4 lightningd: fix crash with -O3 -flto.
It's foolish to ban passing NULL, 0 to memcpy, memset et al, but
it's been done.  At high level of optimization, GCC assumes this doesn't
happen, and yep, assumes "if (ctx)" inside tal_free() must be true.

So when a psbt is NULL, and psbt_get_bytes returns NULL, a crash ensues:

```
lightningd: FATAL SIGNAL 6 (version v0.12.0rc2-6-g47efa5d-modded)
0x5557dfc42fef send_backtrace
	common/daemon.c:33
0x5557dfc42fef crashdump
	common/daemon.c:46
0x7fe93ef5851f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7fe93efaca7c __pthread_kill_implementation
	./nptl/pthread_kill.c:44
0x7fe93efaca7c __pthread_kill_internal
	./nptl/pthread_kill.c:78
0x7fe93efaca7c __GI___pthread_kill
	./nptl/pthread_kill.c:89
0x7fe93ef58475 __GI_raise
	../sysdeps/posix/raise.c:26
0x7fe93ef3e7f2 __GI_abort
	./stdlib/abort.c:79
0x5557dfbb0c28 call_error
	ccan/ccan/tal/tal.c:93
0x5557dfbb0c34 check_bounds
	ccan/ccan/tal/tal.c:165
0x5557dfbb0c34 to_tal_hdr
	ccan/ccan/tal/tal.c:178
0x5557dfc7a1d3 tal_free
	ccan/ccan/tal/tal.c:482
0x5557dfc609d3 tal_free
	ccan/ccan/tal/tal.c:477
0x5557dfc609d3 towire_wally_psbt
	bitcoin/psbt.c:743
0x5557dfbc5dfc towire_dualopend_got_offer_reply
	openingd/dualopend_wiregen.c:358
0x5557dfbc5dfc openchannel2_hook_cb
	lightningd/dual_open_control.c:671
0x5557dfc22f4f plugin_hook_callback
	lightningd/plugin_hook.c:210
0x5557dfc1dfbe plugin_response_handle
	lightningd/plugin.c:591
0x5557dfc1dfbe plugin_read_json_one
	lightningd/plugin.c:702
0x5557dfc1dfbe plugin_read_json
	lightningd/plugin.c:747
0x5557dfc71756 next_plan
	ccan/ccan/io/io.c:59
0x5557dfc775d5 io_ready
	ccan/ccan/io/io.c:417
0x5557dfc775d5 io_loop
	ccan/ccan/io/poll.c:453
0x5557dfbdb1ce io_loop
	ccan/ccan/io/poll.c:380
0x5557dfbdb1ce io_loop_with_timers
	lightningd/io_loop_with_timers.c:22
0x5557dfbb37d1 main
	lightningd/lightningd.c:1195
0x7fe93ef3fd8f __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7fe93ef3fe3f __libc_start_main_impl
	../csu/libc-start.c:392
0x5557dfbb6e84 ???
	???:0
0xffffffffffffffff ???
	???:0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell 4ca1203eb8 doc: include recent BOLT recommendation on grace period.
This includes the recommendation that we use 10 minute grace period,
so add quotes to where we use that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 3cc6d0ec2c doc: upgrade to BOLTs 341ec844f13c0c0abc4fe849059fbb98173f9766
This is a slightly looser behavior, so no change needed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell b208c0d8dd doc: upgrade to BOLTs 2ecc091f3484f7a3450e7f5543ae851edd1e0761
I disagree with this change, so I've commented and added a FIXME.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 5b7f14a7cb channeld/dualopend/lightningd: use channel_ready everywhere.
This alters the billboard, but that's a human-readable thing so not
noted in CHANGELOG.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listpeers` `status` now refers to "channel ready" rather than "funding locked" (BOLT language change for zeroconf channels)
Changelog-Added: JSON-RPC: `channel_opened` notification `channel_ready` flag.
Changelog-Deprecated: JSON-RPC: `channel_opened` notification `funding_locked` flag (use `channel_ready`: BOLTs namechange).
2022-09-12 09:34:52 +09:30
Rusty Russell 1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell 341bbdfcbe doc: increase BOLT level to 03468e17563650fb9bfe58b2da4d1e5d28e92009
`flags` in `channel_disabled` gets renamed.  We don't use it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell fbcdf2c565 devtools/bolt-catchup.sh: a tool to update the specs, one commit at a time.
This would be more effective if we didn't *merge* in the specs repo,
but still.

Usage: ./devtools/bolt-catchup.sh

It goes through one commit at a time, up to current HEAD, and stops when there
are changes, or quotes change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 1a0f7ddb0d hsmtool: remove hsm_secret passwords on cmdline support in `dumponchaindescriptors`.
Changelog-Removed: `hsmtool`: hsm_secret (ignored) on cmdline for dumponchaindescriptors (deprecated in v0.9.3)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 6cf3d47505 offers: remove backwards-compatiblity invoice_request signatures.
We changed the field name in v0.11.0, so this breaks compat with
v0.10.2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 136d0c8005 offers: update to remove "vendor" and "timestamp" fields.
Changelog-EXPERIMENTAL: remove "vendor" (use "issuer") and "timestamp" (use "created_at") fields (deprecated v0.10.2).
2022-09-12 09:34:52 +09:30
Rusty Russell 318650a627 listchannels: don't show "htlc_maximum_msat" if channel_update didn't set it.
It was deprecated in v0.10.1, but only one channel on the network
doesn't set it now anyway, and we'll be ignoring that soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 29264e83fb lightningd: remove `use_proxy_always` parameter to plugin init.
Changelog-Removed: Plugins: plugin init `use_proxy_always` (deprecated v0.10.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 733ce81bd4 plugins: require usage for plugin APIs.
Changelog-Removed: JSON-RPC: plugins must supply `usage` parameter (deprecated v0.7)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 15751ea1b8 lightningd: do inline parsing for listforwards status parameter
We can do this now the function is cleaned up.

Always better to do the work inside param() since then `check`
gets the benefit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell a45ec78c36 lightningd: don't allow old listforwards arg order.
Changelog-Removed: Old order of the `status` parameter in the `listforwards` rpc command (deprecated in v0.10.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 43b037ab0b lightningd: always require "jsonrpc": "2.0" in request.
Changelog-Removed: JSONRPC: RPC framework now requires the `"jsonrpc"` property inside the request (deprecated in v0.10.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell bfe342c64b lightningd: remove double-wrapped rpc_command hook.
Somehow we missed this deprecation, found by grep.

Changelog-Removed: JSON API: Removed double wrapping of `rpc_command` payload in `rpc_command` JSON field (deprecated v0.8.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 6c33f7db65 common: remove unused parameter "allow_deprecated" from parse_wireaddr_internal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell a56b17c759 wire/test: neaten and complete tlv checks.
In particular, we didn't check the remote_addr in the init msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell e0259b246e test: fix tlvs test in funding_locked tlv.
This FIXME caught my eye, as it's wrong: TLVs are canonical, so they cannot
differ in bits and be equal.

The equality function needs to be written correctly, however, otherwise it
will crash!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Michael Schmoock c6858748bb cleanup: fix mixed indentation of json_getinfo
This one got badly messed up over time. I know we usually don't fix
these to have easier git-bisect. I can remove this commit if required.
2022-09-11 18:09:13 +02:00
Michael Schmoock ec95c7c18c peer_control: fix getinfo showing unannounced addr
Currently discovered IPs are only announced when we don't have any
usable addresses detected or configured already. However, the cli
command `getinfo` still showed theses unannounced addr as if they
were announced.

Changelog-Fixed: peer_control: getinfo showing unannounced addresses.
2022-09-11 18:09:13 +02:00
niftynei 7df530d184 builds: cleanup duplicate and unused code, fix spelling
Few extremely minor updates to the ubuntu dockerfile and ci builds
2022-09-10 11:24:31 +09:30
niftynei 34a0d7083a build: use ubuntu 22.04 LTS
Requires us to update to latest lnproto which is now using the most up
to date python-bitcoinlib, as well as updating our python lock files
(which pin the grpcio deps, because of locking problems h/t @cdecker)
2022-09-10 11:24:31 +09:30
Dustin Dettmer 9de458b23b docs: Clear up Ubutu documentation
People who copy paste the commands might not realize this chunk is not copy paste-able.
2022-09-08 00:38:06 +03:00
Rusty Russell cde93ab703 doc: document that we can build lowdown, remove from Alpine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell 50d1043a91 external: build lowdown if not already found.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell 5112329a6b external/lowdown: local import of lowdown source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell 04b59d991a doc: always escape underscores in property names
If there's only a single underscore, lowdown ignores it, but if there are multiple
(see min_final_cltv_expiry) it decides we're trying to highlight part of the word.

Reported-by: @wtogami
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00