Commit Graph

13705 Commits

Author SHA1 Message Date
Rusty Russell 574716dfd5 plugins/renepay: convenience routing to log when we disable a chan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 00e9af57f5 plugins/renepay: neaten the command notifications.
It now looks like (for test_hardmpp):

```
# we have computed a set of 1 flows with probability 0.328, fees 0msat and delay 23
#   Flow 1: amount=1800000000msat prob=0.328 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0->-103x3x0/1->
#   Flow 1: Failed at node #1 (WIRE_TEMPORARY_CHANNEL_FAILURE): failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
#   Flow 1: Failure of 1800000000msat for 103x5x0/0 capacity [0msat,3000000000msat] -> [0msat,1799999999msat]
# we have computed a set of 2 flows with probability 0.115, fees 0msat and delay 23
#   Flow 2: amount=500000000msat prob=0.475 fees=0msat delay=12 path=-103x6x0/0(min=max=4294967295msat)->-103x1x0/1->-103x4x0/1->
#   Flow 3: amount=1300000000msat prob=0.242 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0(max=1799999999msat)->-103x3x0/1->
#   Flow 3: Failed at node #1 (WIRE_TEMPORARY_CHANNEL_FAILURE): failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
#   Flow 3: Failure of 1300000000msat for 103x5x0/0 capacity [0msat,1799999999msat] -> [0msat,1299999999msat]
# we have computed a set of 2 flows with probability 0.084, fees 0msat and delay 23
#   Flow 4: amount=260000000msat prob=0.467 fees=0msat delay=12 path=-103x6x0/0(500000000msat in 1 htlcs,min=max=4294967295msat)->-103x1x0/1(500000000msat in 1 htlcs)->-103x4x0/1(500000000msat in 1 htlcs)->
#   Flow 5: amount=1040000000msat prob=0.179 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0(max=1299999999msat)->-103x3x0/1->
#   Flow 5: Failed at node #1 (WIRE_TEMPORARY_CHANNEL_FAILURE): failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
#   Flow 5: Failure of 1040000000msat for 103x5x0/0 capacity [0msat,1299999999msat] -> [0msat,1039999999msat]
# we have computed a set of 2 flows with probability 0.052, fees 0msat and delay 23
#   Flow 6: amount=120000000msat prob=0.494 fees=0msat delay=12 path=-103x6x0/0(760000000msat in 2 htlcs,min=max=4294967295msat)->-103x1x0/1(760000000msat in 2 htlcs)->-103x4x0/1(760000000msat in 2 htlcs)->
#   Flow 7: amount=920000000msat prob=0.105 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0(max=1039999999msat)->-103x3x0/1->
#   Flow 7: Success
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 7db6aaa523 renepay: clarify chan_extra_can_send.
I am doing to add more more debugging, but sent here is 0.
Document that clearly, and put a real value in sent.

Also: since we already sub 1 msat from x, amount_msat_less_eq should
be amount_msat_less (it may be equal to our min, in theory).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 7287ade705 plugins/renepay: use struct short_channel_id_dir.
No changes, just use this convenience type for handing
around, and arrays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell a6c2cd77a1 plugins/renepay: clean up unused fmt_payflows.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell eef57488f9 pay_flow: downgrade messages about shadow route delays.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 4196f97571 plugins/renepay: Add infra for per-flow notes.
And set logging levels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
sumBTC a28e0afaef announce-addr not announce only 2023-08-23 10:45:52 +09:30
Aditya Sharma 6eb7e81201 docs: update the details of emergency.recover and --recovery for users. 2023-08-22 09:37:31 +09:30
ShahanaFarooqui 6ee495b1dc docker: bookworm image for arm32v7 2023-08-22 09:36:03 +09:30
Rusty Russell 138d178dff pytest: fix plugin_connected_hook_chaining flake.
The code to workaround the intermittant error didn't work,
and we finally hit it again:

```
        # If reject happens fast enough, connect fails with "disconnected
        # during connection"
        try:
            l3.connect(l1)
        except RpcError as err:
>           assert "disconnected during connection" in err.error
E           assert 'disconnected during connection' in {'code': 402, 'message': 'disconnected during connection'}
E            +  where {'code': 402, 'message': 'disconnected during connection'} = RpcError("RPC call failed: method: connect, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'host': '127.0.0.1', 'port': 41865}, error: {'code': 402, 'message': 'disconnected during connection'}").error
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:14:01 +09:30
Rusty Russell af39424491 lightningd: don't restart subds unnecessarily on reconnect.
During tests we can see that the subdaemon can be restarted unnecessarily if we're slow enough; we don't need to do so if it's still running.

Reported-by: Matt Morehouse <mattmorehouse@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:14:01 +09:30
Rusty Russell dec8ad3b8d pytest: fix autoclean test flake
```
        l3.rpc.setconfig('autoclean-cycle', 10)
    
        # First it expires.
>       wait_for(lambda: only_one(l3.rpc.listinvoices('inv1')['invoices'])['status'] == 'expired')

```

If we're slow enough, the invoice is cleaned before we see it expire!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:14:01 +09:30
Rusty Russell 763ec6ad43 plugins/renepay: don't free waiting-for-addgossip flow!
```
Aug 18 13:45:13  lightningd: 0x7fa921f8ffcf ???
Aug 18 13:45:13  lightningd:         ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
Aug 18 13:45:13  lightningd: 0x55b3bb54e6d3 pay_flow_finished_adding_gossip
Aug 18 13:45:13  lightningd:         plugins/renepay/pay_flow.c:675
Aug 18 13:45:13  lightningd: 0x55b3bb54af25 addgossip_done
Aug 18 13:45:13  lightningd:         plugins/renepay/pay.c:171
```

The assert we fail is almost certainly due to the flow being freed:

```
struct pf_result *pay_flow_finished_adding_gossip(struct pay_flow *pf)
{
	assert(pf->state == PAY_FLOW_FAILED_GOSSIP_PENDING);
```

Reported-by: https://github.com/daywalker90
Fixes: #6567
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:13:48 +09:30
Vincenzo Palazzo d2e39f7af8 meta: ignore renepay binary
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-08-21 10:35:41 +09:30
Rusty Russell fa58cd9364 CHANGELOG.md: Update for -rc3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:54:38 +09:30
Christian Decker fe6707f952 openingd: Clean up channel_type_accept
The `accept_zeroconf` parameter is pointless, since it's too early to
check that anyway. Keeping it in there would suggest otherwise, so
remove it.
2023-08-18 20:30:03 +09:30
Christian Decker 71a8e59acd openingd: Reject zeroconf if the peer is not allowed
We determine whether they are allowed or not based on the hook return
value of `mindepth`. To do so we need to pass that value down to
`openingd` and verify that the `channel_type` and our permissions
match up.
2023-08-18 20:30:03 +09:30
Christian Decker 6bc71796c8 openingd: Tentatively accept zeroconf when proposed
We then still use our own minimum_depth to enforce our security. There
is no harm in accepting theirs
2023-08-18 20:30:03 +09:30
Rusty Russell 685c413b61 renepay: reduce minimum "known" if it proves incorrect.
There's a fascinating bug report which suggests this happens on local channels,
implying spendable_msat is wrong?

See-also: #6567
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:55 +09:30
Rusty Russell 40bb22b9ed CI: add clnrest requirements so --help lists rest params.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell b263c7d2e4 lightningd: documentation updates.
1. announce-addr-discovered-port takes a port option.
2. accept-htlc-tlv-types was deprecated in favor of multiple accept-htlc-tlv-type.
3. Document clnrest.py options.
4. Don't list --version twice in lightningd --help (initial_config_opts calls
   opt_register_version() already).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell b5d1708900 plugins/sql: make testing-only sqlfilename option `dev-sqlfilename`.
dev options don't need to be documented, and this is only for debugging
the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell d5e86bf330 lightningd: don't print plugin deprecated options in --help.
We don't publish core deprecated options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell d135c5f3d0 doc: document lightning-cli options.
We don't accept --allow-deprecated-apis any more, and we allow --regtest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell 93d8e10786 tools/check-manpage.sh: update undoc to a simple list.
It didn't handle the case where an undocumented option was
was a flag (i.e. didn't end in =), so rework it to be
a simple list and use grep.

Add some more options we don't document, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell b03358e933 build: have check-manpage.sh ignore options documented as deprecated.
These days we sometimes keep docs for deprecated options, so ignore
those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell 9085293622 build: fix check-manpage.sh
Our formats changed, so this didn't work any more!

Add a sanity check that in future if we get no options from a command,
we complain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell bcea0a9580 listchannels: also list splicing channels as active: true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 17:09:54 +09:30
Rusty Russell 0147b3ba94 pay: don't filter out local channels which are splicing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 17:09:54 +09:30
Rusty Russell 71a09138b6 renepay: splicing channels are also usable for payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 17:09:54 +09:30
Rusty Russell be3a59c7c3 gossmap: fix false valgrind uninitialized error on arm64, ppc.
Doesn't happen on x86, but struct gossmap_chan defines:

```
	u32 private: 1;
	u32 plus_scid_off: 31;
```

And complains when we initialize plus_scid_off and access it later:

```
VALGRIND=1 valgrind -q --error-exitcode=7 --track-origins=yes --leak-check=full --show-reachable=yes --errors-for-leak-kinds=all plugins/renepay/test/run-mcf > /dev/null
==186886== Conditional jump or move depends on uninitialised value(s)
==186886==    at 0x10076388: chan_iter (gossmap.c:1098)
==186886==    by 0x100797F3: gossmap_next_chan (gossmap.c:1112)
==186886==    by 0x1008C5AF: main (run-mcf.c:309)
==186886==  Uninitialised value was created by a heap allocation
==186886==    at 0x40F0A44: malloc (vg_replace_malloc.c:431)
==186886==    by 0x10072BAF: allocate (tal.c:256)
==186886==    by 0x100737A7: tal_alloc_ (tal.c:463)
==186886==    by 0x100738DF: tal_alloc_arr_ (tal.c:506)
==186886==    by 0x10079507: load_gossip_store (gossmap.c:690)
==186886==    by 0x10079667: gossmap_load (gossmap.c:978)
==186886==    by 0x1008C4AF: main (run-mcf.c:295)
```

Reported-by: @grubles
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6557
2023-08-18 16:21:57 +09:30
Rusty Russell d3c7d48acb pay: don't discard high-htlc_min channels when searching for routehint starts.
As side-effect, getroute(0) is special too.

Reported-by: MiddleW4y in Discord
Fixes: #6577
Changelog-Fixed: `pay` will still use an invoice routehint if path to it doesn't take 1-msat payments.
2023-08-18 14:55:32 +09:30
Rusty Russell 2c3f8b8a04 pytest: test that we can pay via routehints even if a 1000-min-htlc channel is in path.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 14:55:32 +09:30
Rusty Russell f4e2d9a0ae lightningd: clean up properly if we fail to exec plugin.
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: we clean up properly if a plugin fails to start, and we don't kill all processes if it's from `plugin startdir`.
2023-08-18 13:18:49 +09:30
Rusty Russell 04ddb4af8d pytest: test that failing to start a plugin due to failure to exec does not put it in plugins list.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 13:18:49 +09:30
Rusty Russell bc36a4580c pytest: test (failing) for not crashing if plugin fails to exec via `plugin startdir`
It hits the new assert(): before that (tested!) it actually kills everything and logs you out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 13:18:49 +09:30
Lisa Neigut 55168e66ad Don't restart my machine if I add code to kill a failed plugin. 2023-08-18 13:18:49 +09:30
Dusty Daemon c67f1f92a8
splice: prevent splice going to onchaind & race prevention
Don’t send the funding spend to onchaind if we detect it in inflights (aka. a splice). While we already prevented onchaind_funding_spent from being called directly, the call to wallet_channeltxs_add meant onchaind_funding_spent would be called *anyway* on restart. This is now fixed.

Additionally there was a potential for a race problem depending on the firing order of the channel depth and and funding spent events.

Instead of requiring these events fire in a specific order, we make a special “memory only” inflight object to prevent the race regardless of firing order.

Changelog-Fixed: Splice: bugfix for restart related race condition interacting with adversarial close detection.
2023-08-16 12:28:53 +09:30
niftynei a9ffa370c4 nixos install: tweaks to get CLN building from fresh start
Needed to change a few things to get CLN building on a fresh NixOS
box.

Ideally we'd move this over to a flake.nix for devshell etc.
2023-08-16 12:27:14 +09:30
Christian Decker e0f97cab1b doc: Drive-by fixes of JSON examples in hooks.md 2023-08-16 12:26:43 +09:30
Alex Myers 553369896b pytest: add no argument cli test 2023-08-16 12:25:25 +09:30
Alex Myers bd4a001279 cli: don't crash when there's no argument
This should provide the default help message and exit, but was
resulting in a segmentation fault from freeing pointers passed to
the default config.

Changelog-Fixed: lightning-cli properly returns help without argument
2023-08-16 12:25:25 +09:30
Dusty Daemon 5531c9d460 splice: quieter in more cases
We also shouldn’t be announcing channels or commitments when we have requested for `stfu` mode but are waiting for our apeer.

Changelog-None
2023-08-15 06:25:30 +09:30
Rusty Russell af6b535efd pay: still use channels for routehints even if peer says it's disabled.
We have a report that LND said our (unannounced) channel was disabled, so we didn't
use it for routehints.  We're better off ignoring that in this case (if the peer is
actually not connected, the routehint code will check that and ignore anyway).

Fixes: #6555
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pay: use channels in routehints even if peer says they're "disabled" (LND compat)
2023-08-15 06:22:53 +09:30
Rusty Russell be082cdff4 common/test: fix gcc-12.3.0 -O3 warning.
```
common/test/run-bolt12_decode.c:206:17: error: ‘valid’ may be used uninitialized [-Werror=maybe-uninitialized]
  206 |                 printf("%s %s\n", str, valid ? "OK": "INVALID");
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
common/test/run-bolt12_decode.c:191:22: note: ‘valid’ was declared here
  191 |                 bool valid, actual;
      |                      ^~~~~
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-15 06:21:23 +09:30
Rusty Russell c889976d03 common: fix gcc-12.3.0 -O3 warning.
This is actually a valid complaint (though this is a sanity check for
things we make ourselves, still!).

```
In file included from common/test/run-blindedpath_onion.c:9:
common/test/../sphinx.c: In function ‘sphinx_add_hop_has_length’:
common/test/../sphinx.c:117:12: error: ‘prepended_len’ may be used uninitialized [-Werror=maybe-uninitialized]
  117 |         if (lenlen + prepended_len != tal_bytelen(payload))
      |            ^
common/test/../sphinx.c:109:27: note: ‘prepended_len’ was declared here
  109 |         bigsize_t lenlen, prepended_len;
      |                           ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-15 06:21:23 +09:30
Rusty Russell 8dc8416ed7 plugins/renepay: fix gcc-12.3.0 -O3 warning.
Compiler can't tell that we always set have_state[PAY_FLOW_FAILED_FINAL]
when we set this:

```
plugins/renepay/payment.c: In function ‘payment_reconsider’:
plugins/renepay/payment.c:287:25: error: ‘final_error’ may be used uninitialized [-Werror=maybe-uninitialized]
  287 |                         payment_fail(payment, final_error, "%s", final_msg);
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
plugins/renepay/payment.c:194:30: note: ‘final_error’ was declared here
  194 |         enum jsonrpc_errcode final_error, ecode;
      |                              ^~~~~~~~~~~
plugins/renepay/payment.c:287:25: error: ‘final_msg’ may be used uninitialized [-Werror=maybe-uninitialized]
  287 |                         payment_fail(payment, final_error, "%s", final_msg);
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
plugins/renepay/payment.c:195:21: note: ‘final_msg’ was declared here
  195 |         const char *final_msg;
      |                     ^~~~~~~~~
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-15 06:21:23 +09:30
Rusty Russell 9239278cdd plugins: fix compile warning with some gcc versions and -Og
Indeed, we can fall through this if it's not a valid enum value.

gcc-12 (Ubuntu 12.2.0-17ubuntu1) 12.2.0

```
In file included from plugins/commando.c:10:
ccan/ccan/tal/str/str.h: In function ‘rune_altern_to_english’:
ccan/ccan/tal/str/str.h:43:9: error: ‘cond_str’ may be used uninitialized [-Werror=maybe-uninitialized]
   43 |         tal_fmt_(ctx, TAL_LABEL(char, "[]"), __VA_ARGS__)
      |         ^~~~~~~~
plugins/commando.c:97:21: note: ‘cond_str’ was declared here
   97 |         const char *cond_str;
      |                     ^~~~~~~~
cc1: all warnings being treated as errors
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-15 06:21:23 +09:30
Rusty Russell 028dcb875b channeld: fix gcc-12.3.0 -O3 warning.
```
In function ‘peer_reconnect’,
    inlined from ‘init_channel’ at channeld/channeld.c:5890:3,
    inlined from ‘main’ at channeld/channeld.c:5951:2:
channeld/channeld.c:5028:21: error: ‘next_matches_inflight’ may be used uninitialized [-Werror=maybe-uninitialized]
 5027 |                 if (remote_next_funding && !next_matches_current
      |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 5028 |                     && !next_matches_inflight) {
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~
channeld/channeld.c: In function ‘main’:
channeld/channeld.c:4595:36: note: ‘next_matches_inflight’ was declared here
 4595 |         bool next_matches_current, next_matches_inflight;
      |                                    ^~~~~~~~~~~~~~~~~~~~~
channeld/channeld.c:5042:57: error: ‘inflight’ may be used uninitialized [-Werror=maybe-uninitialized]
 5042 |                                                         &inflight->outpoint.txid),
      |                                                         ^
channeld/channeld.c:4594:26: note: ‘inflight’ was declared here
 4594 |         struct inflight *inflight;
      |                          ^~~~~~~~
cc1: all warnings being treated as errors
make: *** [Makefile:300: channeld/channeld.o] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-15 06:21:23 +09:30