Commit Graph

13864 Commits

Author SHA1 Message Date
Rusty Russell dabd6c6b70 pytest: run splicing test on every run.
EXPERIMENTAL_SPLICING=1 turns it on for *all* tests, to make sure we don't
accidentally break those.  But we can (and should!) run the splice test
under every possible CI scenario.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell ac092c0bf9 hsmd: fix capability check for signing splices.
The nomenclature confusion mean that we were ANDING a capability
with a message number (29) which always returned non-zero.  We really
do need a new capability which we can hand to channeld to make these
splice txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell d95cfc0b64 hsmd: rename "capabilities" flags for hsm fds to "permissions"
I obviously like the word "capabilities" since I reused it to refer
to the HSM's overall features :(

Suggested-by: @ksedgwic
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
junderw 32b88a2340 Fix: Remove Sync requirements on Futures returned in the Rust plugin library.
See: https://github.com/bitcoindevkit/bdk/issues/1047#issuecomment-1660645669

In general, futures produced by most libraries in the ecosystem of Rust, and bounds placed
on users of famous runtimes like tokio and its spawn method all lack Sync requirements.

Because of this, anyone who creates a callback using any sort of library that returns a
non-Sync future (which most libraries fit this description) inside of it will get some
cryptic error messages (async error messages still leave a lot to be desired).

Removing these Sync requirements will make the library more useful.
2023-08-08 09:45:42 +09:30
Peter Neuroth 29fea55980 doc: Fix typo in the description of fields `private`
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
[ Regenerated man pages --RR ]
2023-08-08 09:45:20 +09:30
Rusty Russell 1fbe87f6e4 lightningd: use fsync not fdatasync.
Apparently MacOS doesn't always have fdatasync, so use fsync.  Even more importantly
check whether it succeeds!

Fixes: #6516
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 09:45:10 +09:30
Adi Shankara 98c805e267
fix flake8 errors in the script file (#6528)
* fix flake8 errors

* fix E126 error

* fix E123 error
2023-08-07 21:45:44 +05:30
Rusty Russell 54bcb10227 lightningd: fix bolt11 parsing in preapproveinvocie, sendonion, listsendpays and renepay
Since bolt11_decode now insists that any `lightning:` prefix be removed, we need
to make sure to use param_invstring not param_string for all bolt11 parameters:

```
2023-08-07T05:55:32.515Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v23.08rc1-21-g0bf5ee6)
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x55dd94934154
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x55dd949342e6
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7f5cf5a3bcef
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7f5cf5a3bc45
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5cf5a227fb
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5cf5a2271a
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5cf5a33595
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:734 (bolt11_decode_nosig) 0x55dd94929967
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:953 (bolt11_decode) 0x55dd9492a44f
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/pay.c:1730 (json_listsendpays) 0x55dd948d7d72
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:658 (command_exec) 0x55dd948b525b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:786 (rpc_command_hook_final) 0x55dd948b5876
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:285 (plugin_hook_call_) 0x55dd948f6446
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:874 (plugin_hook_call_rpc_command) 0x55dd948b5c77
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:984 (parse_request) 0x55dd948b6234
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:1090 (read_json) 0x55dd948b670f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55dd94ac9bf4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55dd94aca823
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55dd94aca865
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x55dd94accbff
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x55dd948b33c4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1332 (main) 0x55dd948ba429
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) 0x7f5cf5a2350f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../csu/libc-start.c:381 (__libc_start_main_impl) 0x7f5cf5a235c8
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55dd94881e74
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Fixes: #6524
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: broken in master since last release.
2023-08-07 18:46:48 +09:30
Rusty Russell 7a88900476 pytest: test for listsendpays and lightning: prefix crash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 18:46:48 +09:30
Rusty Russell 91ea85be36 lightningd: close connection when HTLC addition times out.
I noticed this while debugging an issue with ACINQ, that we got upset,
but didn't trigger a reconnect cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We now close connection with a peer if adding an HTLC times out (which may be a TCP connectivity issue).
2023-08-07 18:45:29 +09:30
Rusty Russell d4ed1c7f6f pytest: test that we close connection if adding an HTLC times out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 18:45:29 +09:30
Rusty Russell f556be5d82 renepay: allow it to die gracefully without crashing lightningd.
Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 17:13:29 +09:30
Rusty Russell 2042b50978 plugins/bcli: update minimum required bitcoind version.
Less than 22 is obsolete anyway, so we should increment this from 16.0 at least!

Closes: #6234
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 16:04:14 +09:30
ShahanaFarooqui 248b34acec docker: bitcoin and elements version update
Updated bitcoin and elements versions to 22.0.
2023-08-07 16:04:14 +09:30
Rusty Russell 91a58a0bdc channeld: don't send splice TLV fields unless negotiated.
This make ACINQ seize up, and not send revoke_and_ack.  Eventually,
this can cause a bad signature error, should payments go in both
directions, which is a separate bug, but this is the trigger.

See: #6500
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 15:22:38 +09:30
Christian Decker 0bf5ee6bba meta: Add the VLS team as codeowners of hsmd/hsmd_wire.csv
We use this file as a proxy for breaking changes in the signer
protocol. It may not catch all the breaking changes, but it's a
good first approximation.
2023-08-07 13:10:56 +09:30
Christian Decker f56ba464b8 meta: Remove ZmnSCPxj from codeowners 2023-08-07 13:10:56 +09:30
Dusty Daemon aa2df28faf splicing: Update documentation
Added documentation for splice_update & splice_signed and tweaked splice_init.

Added corresponding schemas for splice_* commands

Changelog-None
2023-08-07 12:38:36 +09:30
Rusty Russell 5faaa39773 docs: try to fix up readthedocs.io, so it has the latest man pages, but points to docs.corelighting.org for the rest
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:56:28 +09:30
Adi Shankara 49acafe653 random doc changes to test.
[Moved into a separate commit --RR]
2023-08-07 11:55:57 +09:30
Adi Shankara be829a1475 add workflow to sync rpc commands 2023-08-07 11:55:57 +09:30
Adi Shankara f64aaaf943 add script to sync rpc commands 2023-08-07 11:55:57 +09:30
Rusty Russell 84fe3dc940 build: more 32-bit printf fixes.
Reported-by: Shahana
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:52:40 +09:30
Rusty Russell 34b6731b82 lightningd: don't return to a default filter level if there are no per-file filters.
In this case, the user's default was info, but they specifically asked for debug
from one plugin.  Since there were no per-file filters, it set filtering to the
default level, info, and rejected it.  Since it's been explicitly filtered in,
we need to pass it at this point.

Reported-by: @wtogami
Fixes: #6503
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell be2cbe784f pytest: test for filtering bug.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell 3191ff7e32 doc: make staticbackup output autogenerated in man page.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 12:34:36 +09:30
Christian Decker 9074341ef6 msggen: Add `staticbackup` call to list of managed methods
Changelog-Added: cln-grpc: Added `staticbackup` support to cln-grpc
2023-08-06 12:34:36 +09:30
Christian Decker 3e14056ce0 doc: The SCB fields are hex-encoded binary TLVs, not strings
This makes them smaller when passing them over a binary-capable encoding.
2023-08-06 12:34:36 +09:30
ShahanaFarooqui 33ab1ce78b docker: Install clnrest dependencies from rquirements.txt
Changelog-None
2023-08-06 12:34:19 +09:30
ShahanaFarooqui 18b928046b plugin/clnrest: Downgrading pyln-proto versino from 23.08 to 23.5.2
Bug fix: pip install from plugins/clnrest/requirements.txt was unable to find "not yet released" pyln-proto version 23.08.
2023-08-06 12:34:19 +09:30
Dusty Daemon fb179f2d5e
splicing: Remove dependency on experimental_dual_fund
Splicing should work automatically on v1 or v2 channels so this requirement isn’t needed.

Changelog-None
[ Squashed fixup into a single commit --RR ]
2023-08-06 12:19:42 +09:30
ShahanaFarooqui 620a61ee09 makefile: py plugins installed twice with `make install`
Steps to Reproduce: Run `sudo make install` then again `sudo make install` without uninstalling plugins in between.
First command will install the plugin at `/usr/local/libexec/c-lightning/plugins/clnrest` as expected but the
second command will install it inside the first folder like `/usr/local/libexec/c-lightning/plugins/clnrest/clnrest`.

Fix: Check and delete if the folder already exists in `install-program`.

Changelog-None
2023-08-05 17:12:17 +09:30
Dusty Daemon 7c9f5e9847 renepay: Use macros instead of manual format types
Fixes 32-bit builds:

```
In file included from plugins/renepay/pay.c:5:
./plugins/renepay/pay_flow.h: In function 'fmt_payflow_key':
./plugins/renepay/pay_flow.h:54:17: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'u64' {aka 'long long unsigned int'} [-Werror=format=]
   54 |                 "key: groupid=%ld, partid=%ld, payment_hash=%s",
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   55 |                 k->groupid,k->partid,
      |                 ~~~~~~~~~~
      |                  |
      |                  u64 {aka long long unsigned int}
```

etc
2023-08-04 20:33:20 +09:30
Rusty Russell 881d7efd6e wallet: don't BROKEN log id payments.total_msat is null.
This was changed by mistake in 23fafe98e3b4f4e15cdf764e1346d813d1eb8039: if
it's null we turn it into 0 (which is what the default call does, but it
does log BROKEN about it!):

```
2023-08-03T14:10:49.001Z **BROKEN** lightningd: Accessing a null column total_msat/15 in query SELECT  id, status, destination, msatoshi, payment_hash, timestamp, payment_preimage, path_secrets, route_nodes, route_channels, msatoshi_sent, description, bolt11, paydescription, failonionreply, total_msat, partid, local_invreq_id, groupid, completed_at FROM payments ORDER BY id;
```

Fixes: #6501
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-04 20:31:14 +09:30
Rusty Russell 86d63a16ca doc: give an example for multifundchannel
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Closes: #4642
2023-08-04 13:01:49 +09:30
Rusty Russell 568f277835 channeld: replace `struct splice` with `struct splicing` for OpenBSD.
Since it's only for transitory splicing info, the new name makes sense.

```
cc channeld/channeld.c
In file included from channeld/channeld.c:23:
./channeld/splice.h:37:8: error: redefinition of 'splice'
struct splice {
       ^
/usr/include/sys/socket.h:140:8: note: previous definition is here
struct  splice {
        ^
```

Reported-by: @grubles
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6486
2023-08-03 13:43:35 +09:30
Rusty Russell a2830e22d2 CHANGELOG: Update.
Co-authored-by: Alex Myers <95372134+endothermicdev@users.noreply.github.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-03 09:05:59 +09:30
Rusty Russell 0c20cc70ab pyln-client, pyln-proto, pyln-testing: update to new versions.
I actually did this manually to send 23.08 to Pypi a few days ago, as it's
required for clnrest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-03 09:05:59 +09:30
Rusty Russell 0f6687ec7b codex32: use "cl" instead of "ms" as our HRP.
This was strongly recommended by Russell O'Connor: the "ms" implies that
it's a BIP-32 master secret, and this is CLN specific.

If we changed the hrp to "cln" it would be better, but apparently that
means we no longer fit in a "standard billfold metal wallet" (and
our code assumes a 2-byte prefix anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 16:12:32 +09:30
Rusty Russell 67e43ea868 renepay: clean up ARC_ADDITIONAL_BITS.
It's expressed in bits, but really it's clearer as a quantity, given
how it's used.

Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Rusty Russell 15c8a6f6fe renepay: switch from arc_t to struct arc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Rusty Russell b793dc9224 renepay: implement an alternative to a union with bitsets.
This was actually relying on little-endian; if we want this, we need to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Rusty Russell e8b294d338 CCAN: update for base64 compile fix on ARM.
```
ccan/ccan/base64/base64.c:34:10: error: result of comparison of constant 255 with expression of type 'int8_t' (aka 'signed char') is always false [-Werror,-Wtautological-constant-out-of-range-compare]
        if (ret == (char)0xff) {
            ~~~ ^  ~~~~~~~~~~
ccan/ccan/base64/base64.c:44:57: error: result of comparison of constant 255 with expression of type 'const signed char' is always true [-Werror,-Wtautological-constant-out-of-range-compare]
        return (maps->decode_map[(const unsigned char)b64char] != (char)0xff);
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~
```

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:33 +09:30
Rusty Russell 1526b4e7e1 docs: document hsmtool getcodexsecret, refer to it for recover.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 08:42:50 +09:30
Rusty Russell 5900742b3a hsmtool: remove threshold arg.
It really has to be 0, since it's the complete secret.  And we didn't handle
it well, (`a` would be treated as 0, for example!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 08:42:50 +09:30
Rusty Russell 4dcd712d6e hsmtool: tighten checks for validity on getcodexsecret
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 08:42:50 +09:30
Rusty Russell e981a4f14a lightningd: pass signed tx through to close callback.
Thread the signed tx through so close's JSON return contains that,
rather than the unsigned channel->last_tx.

We have to split the "get cmd_id" from "resolve the close commands" though;
and of course, as before, we don't actually print the txids of multiple
transactions even though we may have multi in flight due to splice!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `close` returns a `tx` field with witness data populated (i.e. signed).
Fixes: #6440
2023-08-02 07:09:28 +09:30
Rusty Russell d90a185188 lightningd: clean up signing of last_tx.
Return a copy, don't sign in place then remove the signature after.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 07:09:28 +09:30
Rusty Russell 673d81adde pytest: test for bad 'tx' result from close.
We print it without the witness!

Reported-by: https://github.com/wtogami
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 07:09:28 +09:30
Adi Shankara aa2f4a1fa0
Fixing env reference for API key and path to doc (#6478)
* fix the env reference for the API key

* fix path to directory in the workflow file
2023-08-01 17:48:00 +05:30