Commit Graph

12252 Commits

Author SHA1 Message Date
Christian Decker 1f5e579f1b docker: Clean up dependencies for alpine build 2022-10-15 13:42:58 +02:00
briancolecoinmetrics f0d81f46f0 Clean up Dockerfile.alpine
bump Dockerfile.alpine from Alpine 3.14 to 3.16 (current stable) and remove apparently redundant dependencies.
2022-10-15 13:42:58 +02:00
briancolecoinmetrics 74652f7cf4 move alpine build dependencies to virtual package
move alpine build dependencies to virtual package so it can be easily removed all at once
2022-10-15 13:42:58 +02:00
briancolecoinmetrics 2136d5912f update package dependencies for Alpine Linux
Alpine no longer has a `python` (2) package, which is fine because it doesn't seem to be needed. Also, the listed commands didn't result in all needed dependencies being installed for runtime, so I've added that in an additional step.
2022-10-15 13:42:58 +02:00
Christian Decker a99509db36 py: Update protobuf dependency to silence dependabot 2022-10-14 19:31:39 +02:00
Rusty Russell e855ac2f9e keysend: just strip even unknown fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `keysend` now removes unknown even (technically illegal!) fields, to try to accept more payments.
2022-10-04 17:48:08 +02:00
adi2011 e7e7a7186f tests/test_misc.py: Check if funds are getting recovered on reconnecting... Changelog-None: Increasing test scope 2022-10-01 14:01:19 +02:00
okjodom 335f52d1a8 cln-rpc: implement from Secret to slice conversion 2022-09-30 10:18:12 +02:00
Dustin Dettmer 6aca9f665b devtools: Make fund_nodes compatible w/zsh
Mac is updating to using zsh in general.

The “for i in” reads strings with spaces as a single entry instead of multiple entries as sh did.

Using “while read” … “<<< $var” makes it treat each space as a new entry.

Changelog-None
2022-09-30 10:17:46 +02:00
joemphilips 52be59587c msggen: generate deprecated fields in rust.py 2022-09-29 11:18:06 +02:00
Rusty Russell 22c8cfc374 channeld: remove onion objects.
We don't actually process onion messages here any more (they moved to
connectd), but the flag and object files were still linked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 2749b1f61e common/onion: remove old blinded payment handling.
It's been obsoleted and needs replacing; less confusing if we remove
it first.

Also, these fields are now present even without an expermintal build
(we'll control at runtime).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell c9d42d8279 bitcoin: add routine to check a Schnorr sig given a 33-byte pubkey.
There's no secp256k1 routine to do this, but we're going to need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 0195b41461 pytest: test that we don't change our payer_key calculation.
If we do, an upgrade would mean we can no longer get refunds on old
invoices.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 125b17b7fc devtools: enhance bolt12-cli to convert to/from hex
This makes it easier to add/delete fields manually to a bolt12 encoding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 2fbe0f59f1 plugins/fetchinvoice: remove obsolete string-based API.
Generate the payload in the callers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 41ef85318d onionmessages: remove obsolete onion message parsing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 342e330b56 doc: update references to old BOLTs repo.
This reveals that common/test/run-bolt12_merkle-json.c was broken!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 49fe1c8ed7 lightningd: have `makesecret` take `hex` or `string` (just like `datastore`)
Changelog-Added: JSON-RPC: `makesecret` can take a string argument instead of hex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Christian Decker 6adb1e0b4b pytest: Bypass schema verification for some RPC calls
The goal here is to test the node validation, not whether we can
trigger the schema validation with bogus values. So we bypass the
verifying RPC wrapper.
2022-09-28 15:13:07 +02:00
Christian Decker 93b315756f schema: Add `enforcedelay` to `setchannel`
This was missing in the last version
2022-09-28 15:13:07 +02:00
Justin Litchfield 51e2433087 Setchannel request is provided 2022-09-28 15:13:07 +02:00
Matt Whitlock 6324980484 channeld: do not enforce max_accepted_htlcs on LOCAL in channel reinit
If a channel goes offline while the count of outstanding outgoing HTLCs
exceeds the limit that we enforce against the peer, then the channel
could never be brought online again because `add_htlc` called by
`channel_force_htlcs` in `channeld/full_channel.c` would return
`CHANNEL_ERR_TOO_MANY_HTLCS`. The protocol specification actually does
allow us to exceed the limits that we are enforcing against the peer;
we are only prohibited from exceeding the limits that the peer is
enforcing against us. `add_htlc` takes an `enforce_aggregate_limits`
parameter that appears to have been intended for `channel_force_htlcs`
to exempt the local node from obeying the limits that it is enforcing
against the peer, but this parameter was only being respected for the
total HTLC value-in-flight check but not for the HTLC count check. This
commit respects the parameter for the HTLC count check as well and
resolves the problem of "Could not restore HTLCs".

Fixes: #5636
Changelog-Fixed: channeld: Channel reinitialization no longer fails when the number of outstanding outgoing HTLCs exceeds `max_accepted_htlcs`.
2022-09-28 14:00:20 +02:00
Rusty Russell f00cc23f67 sphinx: rename confusing functions, ensure valid payloads.
"sphinx_add_hop" takes a literal hop to include,
"sphinx_add_modern_hop" prepends the length.  Now we always prepend a
length, make it clear that the literal version is a shortcut:

* sphinx_add_hop -> sphinx_add_hop_has_length
* sphinx_add_modern_hop -> sphinx_add_hop

In addition, we check that length is actually correct!  This means
`createonion` can no longer create legacy or otherwise-invalid onions:
fix tests and update man page to remove legacy usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `createonion` no longer allows non-TLV-style payloads.
2022-09-28 13:40:57 +02:00
Rusty Russell 8771c86379 common/onion: expunge all trace of different onion styles.
In particular, remove special routines to pull length: it's there,
take it and check it yourself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell c8ad9e18a9 common/onion: remove all trace of legacy parsing.
We still have an "enum forward_style" for the database, where old-style
forwards can still exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer forward HTLCs with legacy onions.
2022-09-28 13:40:57 +02:00
Rusty Russell 1de4e46276 tests: add onion-test-vector from "BOLT 4: Remove legacy format, make var_onion_optin compulsory."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
joemphilips 836a2aa261 use msat_or_all for fundpsbt request amount 2022-09-28 10:52:55 +02:00
Rusty Russell d4ef20d54a pytest: fix flake in test_gossip_persistence.
We used to ensure the l3<->l4 channel was private by
simply not mining enough blocks to announce.  Then
we started mining 13 blocks to close the channel, and
it will get announced, causing a failure:

```
        assert non_public(l2) == []
>       wait_for(lambda: non_public(l3) == [scid34, scid34])
...
>               raise ValueError("Timeout while waiting for {}", success)
E               ValueError: ('Timeout while waiting for {}', <function test_gossip_persistence.<locals>.<lambda> at 0x7f6cc69b4170>)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-27 13:14:20 +02:00
Rusty Russell 695f001161 pytest: fix flake in test_zeroconf_forward
Second pay can fail if first is not completely settled:

```
    def test_zeroconf_forward(node_factory, bitcoind):
        """Ensure that we can use zeroconf channels in forwards.

...
        # Make sure (esp in non-dev-mode) blockheights agree so we don't WIRE_EXPIRY_TOO_SOON...
        sync_blockheight(bitcoind, [l1, l2, l3])
        inv = l3.rpc.invoice(42 * 10**6, 'inv1', 'desc')['bolt11']
        l1.rpc.pay(inv)

        # And now try the other way around: zeroconf channel first
        # followed by a public one.
        wait_for(lambda: len(l3.rpc.listchannels()['channels']) == 4)
        inv = l1.rpc.invoice(42, 'back1', 'desc')['bolt11']
>       l3.rpc.pay(inv)

...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: pay, payload: {'bolt11': 'lnbcrt420p1p3junrssp588gtjzmlrr4pfj7ssmdlulzhhushrpq3rdqxjuz2m33scsvzdjlspp5fk5yhu6netc0d0sgp8es52vjk6akhd3uayr08u8max4d8rwzpjuqdq8v3jhxccxqyjw5qcqp99qyysgqcpyyugejv4nya97v6gw8fhtr0ru3vq87jjlltav99wlat436a95n0z8yzdp699p9md0zz9tmnsjpvfj622n9g9fh7r6ldhpgh9wmr4qpcru3rk'}, error: {'code': 210, 'message': 'Destination 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 is not reachable directly and all routehints were unusable.', 'attempts': [{'status': 'failed', 'failreason': 'Destination 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 is not reachable directly and all routehints were unusable.', 'partid': 0, 'amount_msat': 42msat}]}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-27 13:14:20 +02:00
Rusty Russell 68f15f17bb delforward: allow deletion of "unknown in_htlc_id" and fix autoclean to use it.
Note the caveats: we will delete *all* of them at once!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-27 14:42:03 +09:30
Rusty Russell 6eac8dfe3c delforward: tally up deleted forwards so that getinfo's `fees_collected_msat` doesn't change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5627
2022-09-27 14:42:03 +09:30
Rusty Russell cafa1a8c65 db: correctly migrate forwards for closed incoming channels.
We have to allow them (as otherwise `fees_collected_msat` in getinfo breaks),
but it means that actually, in_htlc_id might be missing in listforwards
(also, out_htlc_id might be missing, which we didn't catch before).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5628
2022-09-27 14:42:03 +09:30
Rusty Russell 9023bd9334 pytest: add test for migrations upgrade which breaks 'fees_collected_msat'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-27 14:42:03 +09:30
Christian Decker 657b315f1c pyln: Bump versions to v0.12.1
This is just before the introduction of `get_json_id`, but has the
correct dependency constraints such that all packages can be updated
to >=v0.12 and we don't mix minor versions.
2022-09-26 22:06:05 +02:00
elsirion 6abcb18145 Add basic arithmetic to `Amount` type 2022-09-26 22:05:30 +02:00
elsirion 09dfe3931d Make eligible types `Copy` 2022-09-26 22:05:30 +02:00
elsirion b4b0b479ac Use `secp256k1` for public key type 2022-09-26 22:05:30 +02:00
elsirion e272c93a88 Use `bitcoin_hashes` for `Sha256` 2022-09-26 22:05:30 +02:00
elsirion 10917743fe Implement a typed version of `call` to avoid useless matching 2022-09-26 22:05:30 +02:00
elsirion 7046252f96 Impl `std::error::Error` for `RpcError` to make it anyhow compatible 2022-09-26 22:05:30 +02:00
Melroy van den Berg 87bab2b851 Add ConfigurationDirectory
Add ConfigurationDirectory for `/etc/lightningd ` as well.
2022-09-26 17:48:11 +02:00
Melroy van den Berg a7ce03bae6 The project is called Core Lightning
Rename project in the description of the service file.
2022-09-26 17:48:11 +02:00
Rusty Russell 6e86fa9220 lightningd: figure out optimal channel *before* forward_htlc hook.
Otherwise what the hook sees is actually a lie, and if it sets it
we might override it.

The side effect is that we add an explicit "forward_to" field, and
allow hooks to override it.  This lets a *hook* control channel
choice explicitly.

Changelod-Added: Plugins: `htlc_accepted_hook` return can specify what channel to forward htlc to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-26 13:52:04 +02:00
Rusty Russell b698a5a5ef channeld: send error, not warning, if peer has old commitment number.
This is the minimal change to meet the desired outcome of https://github.com/lightning/bolts/issues/934
which wants to give obsolete-db nodes a chance to fix things up, before we
close the channel.

We need to dance around a bit here, since we *will* close the channel if
we receive an ERROR, so we suppress that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-26 11:36:09 +02:00
Christian Decker 437ae11610 pytest: Configure the plugin logging to debug 2022-09-25 15:13:12 +02:00
Christian Decker b13ab8de3a msggen: Use owned versions to convert from cln-rpc to cln-grpc
We needed to clone a lot since we were using a non-mutable borrow of
the cln-rpc version to then create the cln-grpc version. This alone
reduces the number of allocations of the `listpeers` test from 52
allocations to 38.
2022-09-25 15:13:12 +02:00
Christian Decker e1fc88ff70 cln-plugin: Prep the logging payload in a let
I wanted to debug why we weren't getting some log messages, and this
makes that a bit easier in future.
2022-09-25 15:13:12 +02:00
Christian Decker 064a5a6940 cln-plugin: Add log filtering support
We filter based on the environment variable `CLN_PLUGIN_LOG`,
defaulting to `info` as that is not as noisy as `debug` or `trace`, at
least libraries will not spam us too heavily.

Changelog-Added cln-plugin: The logs level from cln-plugins can be configured by the `CLN_PLUGIN_LOG` environment variable.
2022-09-25 15:13:12 +02:00
Christian Decker 8898511cf6 cln-plugin: Defer binding the plugin state until after configuring
We had a bit of a chicken-and-egg problem, where we instantiated the
`state` to be managed by the `Plugin` during the very first step when
creating the `Builder`, but then the state might depend on the
configuration we only get later. This would force developers to add
placeholders in the form of `Option` into the state, when really
they'd never be none after configuring.

This defers the binding until after we get the configuration and
cleans up the semantics:

 - `Builder`: declare options, hooks, etc
 - `ConfiguredPlugin`: we have exchanged the handshake with
   `lightningd`, now we can construct the `state` accordingly
 - `Plugin`: Running instance of the plugin

Changelog-Changed: cln-plugin: Moved the state binding to the plugin until after the configuration step
2022-09-25 15:13:12 +02:00