Commit Graph

2074 Commits

Author SHA1 Message Date
Rusty Russell 7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell 65f5bb2638 pytest: test for compat code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 5aa108f176 pytest: fix flake in test_pay.py::test_setchannel_state
The second disconnect could fail, if slow enough:

```
>       l1.rpc.disconnect(l2.info['id'], force=True)

tests/test_pay.py:2067:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:690: in disconnect
    return self.call("disconnect", payload)
contrib/pyln-testing/pyln/testing/utils.py:639: in call
    res = LightningRpc.call(self, method, payload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pyln.testing.utils.PrettyPrintingLightningRpc object at 0x7fb139ce87d0>
method = 'disconnect'
payload = {'force': True, 'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d'}

...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: disconnect, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'force': True}, error: {'code': -1, 'message': 'Peer not connected'}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 09:53:38 +10:30
Vincenzo Palazzo 53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Rusty Russell be75518559 pytest: fix bad gossip flake in test_multifunding_v1_v2_mixed
This was missed in e8d2176e6b.

```
>           raise ValueError(str(errors))
E           ValueError: 
E           Node errors:
E            - lightningd-2: had bad gossip messages
E            - lightningd-3: had bad gossip messages
E           Global errors:

contrib/pyln-testing/pyln/testing/fixtures.py:201: ValueError

...

0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 105x1x2 (current block 104)
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Bad gossip order: WIRE_CHANNEL_UPDATE before announcement 105x1x2/0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell bcf3cef96c pytest: fix flake in test_connection.py::test_opener_feerate_reconnect
Make sure it sees disconnect before reconnect, otherwise the next command
fails since we're now disconnected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 2a80400a0f pytest: fix test_connection.py::test_funding_close_upfront fake
Breaks when there are no peers:

```
>       _fundchannel(l1, l2, amt_normal, None)

tests/test_connection.py:1564: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_connection.py:1535: in _fundchannel
    wait_for(lambda: not has_normal_channels(l2, l1))
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
tests/test_connection.py:1535: in <lambda>
    wait_for(lambda: not has_normal_channels(l2, l1))
tests/test_connection.py:1527: in has_normal_channels
    for c in only_one(l1.rpc.listpeers(l2.info['id'])['peers'])['channels']])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = []

    def only_one(arr):
        """Many JSON RPC calls return an array; often we only expect a single entry
        """
>       assert len(arr) == 1
E       AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 486b1b2481 pytest: fix flake in test_funding_fail
We may not see a disconnect instantly:

```
>       assert len(l2.rpc.listpeers()['peers']) == 0
E       assert 1 == 0
E         +1
E         -0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell aeeaa14430 pytest: add reconnect and restart to test_multichan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 8458994a7f pytest: test closing one multichannel works as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 75596b3e0f lightningd: use a better channel if available to next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell debc1b90d3 lightningd: remove checks which prevent us from opening multiple channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now support opening multiple channels with the same peer.
2022-03-23 13:20:12 +10:30
Rusty Russell 21e1d68e3b lightningd: remove (most) functions to search channels by status.
This is generally verboten now, since there can be multiple.  There are a
few exceptions:

1. We sometimes want to know if there are *any* active channels.
2. Some dev commands still take peer id when they mean channel_id.
3. We still allow peer id when it's fully determined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `close` by peer id will fail if there is more than one live channel (use `channel_id` or `short_channel_id` as id arg).
2022-03-23 13:20:12 +10:30
Rusty Russell cb5dc48cab lightningd: make setchannelfee handle multiple channels per peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 90be2cc104 lightningd: remove some "single active channel" assumptions.
Generally this means converting a lazy "peer_active_channel(peer)" call
into an explicit iteration.

1. notify_feerate_change: call all channels (ignores non-active ones anyway).
2. peer_get_owning_subd remove unused function.
3. peer_connected hook: don't save channel, do lookup and iterate channels.
4. In json_setchannelfee "all" remove useless call to peer_active_channel
   since we check state anyway, and iterate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 7de7b7be61 lightningd: use channel_id when a peer is activated.
Rather than intuiting whether this is a new channel / active channel,
use the channel_id.  This simplifies things and makes them explicit,
and prepares for multiple live channels per peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 2424b7dea8 connectd: hold peer until we're interested.
Either because lightningd tells us it wants to talk, or because the peer
says something about a channel.

We also introduce a behavior change: we disconnect after a failed open.
We might want to modify this later, but we it's a side-effect of openingd
not holding onto idle connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell deecedb033 connectd: tell lightningd when disconnect is complete.
This avoids races in our tests where we assume it's sync (and is kind
of nicer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 10e36e073c openingd: disconnect from peer when an error occurs.
openingd currently holds the connection to idle peers, but we're about
to change that: it will only look after peers which are actively
opening a connection.  We can start this process by disconnecting
whenever we have a negotiation failure.

We could stay connected if we wanted to, but that would be up to
connectd to decide.  Right now it's easier if we disconnect from any
idle peer once it's been active.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 0cba062318 pytest: add test to check we notice remote disconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell fcd0b2eb42 connectd: prepare for multiple subd connections.
We still always have 1, but the infrastructure is now in place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell 5704653d4c setchannel: don't let them advertize htlc_maximum_msat larger than capacity.
And check for the obvious setting min > max.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell 999c734bb5 setchannel: add minhtlc
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
2022-03-22 18:45:41 +10:30
Rusty Russell f29890ed66 lightningd: check htlc_maximum_msat of channels for routehints.
We still use the channel hint here (as it's the only option), we just
warn about lack of capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell 1751b1becc pytest: add checks that pay and getroute respect htlc_maximum_msat.
We need to add some, since our internal representations of
htlc_maximum_msat round up, and we need to disable mpp which succeeds
in getting a payment through by splitting.

We also allow dev_routes to suppress invoice routehints altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell 42f91ff2fa lightningd: deprecate setchannelfee, use setchannel in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `setchannelfee` (use `setchannel`).
2022-03-22 18:45:41 +10:30
Rusty Russell 43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell 072c4711ec pytest: convert test_sendonion_rpc to modern TLV onion.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell 45143cc731 pytest: Remove onion test vectors containing legacy onions.
I thought about fixing them up, but really these should be in
lnprototest anyway.  Turns out they're from the spec, so we should
actually fix them up there.

I moved the vector files into contrib/pyln-proto, since that still
needs them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell 0db05f6e9c lightningd: opt_var_onion is now a compulsory feature.
We're about to drop support for legacy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Michael Schmoock ef84d6eec5 chore: remove EXPERIMENTAL for rfc #917 remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock 07fbc7ef13 pytest: IP address discovery updates node_announcement 2022-03-11 16:42:45 +10:30
Michael Schmoock f1981461ef connectd: ignore private remote_addr on non-DEVELOPER builds
When compiled without DEVELOPER this will now filter out `remote_addr` that
come from localhost. The testcase checks for DEVELOPER to test for correct
function of `remote_addr`.

Also, I renamed "test_connect" to "test_connect_basic" so it can be started
without all the other tests in that file that start with "test_connect..."
2022-03-11 16:42:45 +10:30
Michael Schmoock e92176248e chore: fix typo announcable -> announceable
"announcable" is a common misspelling of "announceable", see:

https://en.wiktionary.org/wiki/announcable
2022-03-11 16:42:45 +10:30
Christian Decker af4eed3787 cln-plugin: Make hooks asynchronous 2022-03-10 10:21:41 +10:30
Christian Decker a7ef38732f cln-plugin: Make rpcmethod handlers async 2022-03-10 10:21:41 +10:30
Christian Decker 60e773239c cln-plugin: Add notification subscriptions and hooks to the plugins
For now hooks are treated identically to rpcmethods, with the
exception of not being returned in the `getmanifest` call. Later on we
can add typed handlers as well.
2022-03-10 10:21:41 +10:30
Christian Decker 8c6af21169 cln-plugin: Add support for synchronous RPC methods
Changelog-Experimental: cln-plugin: Added support for non-async RPC method passthrough (async support coming soon)
2022-03-10 10:21:41 +10:30
Christian Decker 4aba119733 pytest: Use valgrind target suppressions instead of skipping tests
Having a list of very targeted suppressions allows us to still run the
majority of tests with valgrind checking, and not fail when Rust does
some trickery. This is for example the case with `std::sync::Once`
which uses `num_procs` calling out to the cgroups subsystem, sometimes
with a null path.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-10 10:21:41 +10:30
Christian Decker a1555623bc pytest: Mark Rust-dependent tests as skipped with VALGRIND
`valgrind` reports seems to flag some memory accesses that are ok in
the Rust standard library, which we can consider false positives for
our purposes:

```Valgrind error file: valgrind-errors.69147
==69147== Syscall param statx(file_name) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==
==69147== Syscall param statx(buf) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==

```
2022-03-10 10:21:41 +10:30
Christian Decker fbcb4c33ad cln-plugin: Populate the options when we get an `init` call 2022-03-10 10:21:41 +10:30
Christian Decker 249fa8675a cln-plugin: Add options to the `getmanifest` call 2022-03-10 10:21:41 +10:30
Christian Decker fe21b89b56 pytest: Add a test for the cln-plugin logging integration 2022-03-10 10:21:41 +10:30
niftynei ecb19ba6f2 coin_mvt: report mutual close outputs also
It's better to report every single utxo on close so we know when
to mark a channel account as definitively closed.
2022-03-05 15:12:12 +10:30
niftynei 7a00277b43 coin_mvt: mark coins destined for wallet w/ originating acct
Only shows up on delayed to us outputs, but nice to have anyway.

It's missing for channel index destined deposits, maybe nice to add at
some point in the future; right now you can figure out which close a
wallet deposit comes from via the channel close txid
2022-03-05 15:12:12 +10:30
niftynei 7bfbe1492a test_close: sync l1 to the blockchain, so that the events are there
Flake on CI coming from `penalty_in/outhtlc` due to the fact that all
the events haven't arrived for the check.

If we wait to sync `l1` as well as `l2`, this should resolve the flake.

        # We use a subset of tags in expected_2 that are used in expected_1
>       tags = check_utxos_channel(l1, [channel_id], expected_1)

tests/test_closing.py:726:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/utils.py:321: in check_utxos_channel
    txid = matchup_events(u_set, evs, chans, tag_list)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

u_set = [[{'account_id': 'external', 'blockheight': 104, 'coin_type': 'bcrt', 'credit': '100000000msat', ...}, None]]
evs = [('external', ['penalty'], None, None), ('external', ['penalty'], None, None), ('external', ['penalty'], None, None)]
chans = ['2722a5fe49a8b5fa4004c19828f3e903632ac02712c6fe78ebea418daad2691f']
tag_list = {'0': '892c64c7d8c8f15d7bdcdcde34b615817d273d2e33d9d775cc9ff38e8e3deeb2', 'A': '1e69d2aa8d41eaeb78fec61227c02a6303e9f32898c10440fab5a849fea52227', 'B': '0e11e2ca01bf5f30b4c54522af172af055dcff8e3810f80a5069a2394cad74b5'}

    def matchup_events(u_set, evs, chans, tag_list):
>       assert len(u_set) == len(evs) and len(u_set) > 0
E       AssertionError
2022-03-01 16:19:34 -06:00
Simon Vrouwe 175e0b54b3 testing: fix flake in test_ping_timeout
[ Folded 2 fixups --RR ]
2022-03-01 09:38:40 +10:30
Simon Vrouwe 84bead9396 pyln-testing: in LightningNode.openchannel, make wait_for_announce more reliable
it now waits for 'alias' in node_announcement, not just block confirms.
more cleanup
2022-03-01 09:38:40 +10:30
Simon Vrouwe 2d06c38997 plugin-funder: fix typos in option lease-fee-base-msat and funder-fuzz-percent
Changelog-Experimental: option `--lease-fee-base-msat` renamed to `--lease-fee-base-sat`

Changelog-Experimental: option `--lease-fee-base-msat` deprecated and will be removed next release
2022-03-01 09:38:40 +10:30
Simon Vrouwe 9bbb52097e lightningd: fix parsing of option dev-force-features
It did not pick-up the last (bolt11) bits set.
2022-03-01 09:38:40 +10:30