Commit Graph

11501 Commits

Author SHA1 Message Date
Christian Decker 09ee28cb51 cln-grpc: Do not start unless a `grpc-port` is specified
For now we don't want to autostart.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-30 12:15:55 +10:30
Christian Decker bf7ad86ef2 cln-grpc: Rename the grpc plugin and binary
Suggested-by: Rusty Russell <@rustyrussell>
Changelog-Added: plugins: `cln-grpc` first class GRPC interface for remotely controlling nodes over mTLS authentication
2022-03-30 12:15:55 +10:30
Christian Decker 461a65ba16 py: Add grpcio-tools for the cln-grpc-plugin 2022-03-30 12:15:55 +10:30
Christian Decker dd66c85fcb grpc-plugin: Make the grpc port to listen on configurable
Changelog-Added: cln-grpc-plugin: The plugin can be configured to listen on a specific port using the `grpc-port` option
2022-03-30 12:15:55 +10:30
Christian Decker 647ed6a8c8 cln: Add grpc port as configurable option 2022-03-30 12:15:55 +10:30
Christian Decker 6098386182 cln-grpc: Add the `connect` method 2022-03-30 12:15:55 +10:30
Christian Decker f5147bbf1e pytest: Add a test for incorrect credentials
If we aren't using the correct certificates we should reject the
connections during the mTLS connection setup. This test tries to
connect with the wrong client cert to the node, and the server will
reject it.
2022-03-30 12:15:55 +10:30
Christian Decker 27e468d2ae grpc-plugin: Generate mTLS certificates and use them in grpc 2022-03-30 12:15:55 +10:30
Christian Decker d221c9b491 pytest: Add a test for the grpc plugin
Currently still unencrypted, but will get its mTLS authentication in
the next commits.
2022-03-30 12:15:55 +10:30
Christian Decker a17edeb839 cln-grpc-plugin: Add basic grpc-plugin 2022-03-30 12:15:55 +10:30
Christian Decker f3d95530f4 make: Generate grpc bindings if we want to test with rust enabled 2022-03-30 12:15:55 +10:30
Christian Decker 8bc4f0137a make: Add a hook for us to depend on generated files for tests
We are about to generate the python grpc bindings, but only when we
have Rust enabled.
2022-03-30 12:15:55 +10:30
Christian Decker b4c7ba2d8f cln-grpc-plugin: Add scaffolding for the cln-grpc-plugin 2022-03-30 12:15:55 +10:30
Rusty Russell 5c949e3116 subd: make channel/peer own the subd.
We get some memleak reports because ld owns the subd, but once
the peer/channel is freed, there's no reference for the brief time
until the subd exits.

This happens for both opening and closingd.  For openingd, the
peer owns it, for others (including dualopend) the channel owns it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-30 06:27:52 +10:30
Christian Decker c673c092dc py: Update the PyPI job to use poetry and correct versions
Let's use poetry when we already use it for everything else.

Changelog-None
2022-03-30 06:12:40 +10:30
Rusty Russell 20392ae526 connectd: restore obs2 onion support.
I removed these prematurely: we *haven't* had a release since
introducing them!

This consists of reverting d15d629b8b
"plugins/fetchinvoice: remove obsolete string-based API." and
plugins/fetchinvoice: remove obsolete string-based
API. "onion_messages: remove obs2 support."

Some minor changes due to updated fromwire_tlv API since they
were removed, but not much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: REVERT: Removed backwards compat with onion messages from v0.10.1.
2022-03-29 10:55:12 +10:30
Rusty Russell 9e11ae1a0b plugins/topology: don't get upset if we hit deleted channel.
There's a race under CI, where a channel is deleted then we see the
channel_update in the gossip store.  We assumed this wouldn't happen,
but it can!

```
[gw1] [ 95%] FAILED tests/test_connection.py::test_multichan
[gw1] [ 95%] ERROR tests/test_connection.py::test_multichan
...
>           raise ValueError(str(errors))
E           ValueError:
E           Node errors:
E            - lightningd-3: had BROKEN messages
E            - lightningd-3: Node exited with return code 1
E           Global errors:
...
lightningd-3: 2022-03-28T00:11:42.160Z DEBUG   wallet: Owning output 0 100000sat (SEGWIT) txid 30616903feba1839a3834e2b3b6123759ce1fe0d76414ca77e2dbc17414772e0 CONFIRMED
lightningd-3: 2022-03-28T00:11:42.392Z DEBUG   hsmd: Client: Received message 5 from client
lightningd-3: 2022-03-28T00:11:42.393Z DEBUG   hsmd: new_client: 2
lightningd-3: 2022-03-28T00:11:42.398Z INFO    plugin-topology: Killing plugin: exited during normal operation
lightningd-3: 2022-03-28T00:11:42.400Z **BROKEN** plugin-topology: Plugin marked as important, shutting down lightningd!
...
----------------------------- Captured stderr call -----------------------------
topology: update for channel 105x1x1 not found!
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:50:01 +10:30
Vincenzo Palazzo 7e6893af9e rust: fixed compiler warning in the example
Changelog-None: rust: fixed compiler warning in the example

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-29 10:07:25 +10:30
Rusty Russell aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Rusty Russell ccaf04d268 invoice: add deschashonly parameter.
LNURL wants this so they can include images etc in descriptions.

Replaces: #4892
Changelog-Added: JSON-RPC: `invoice` has a new parameter `deschashonly` to put hash of description in bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:04:16 +10:30
Rusty Russell 290dfd2b81 doc/schemas: handle delinvoice of paid invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:04:16 +10:30
Rusty Russell 7abc491f4c pay: fix crash on invalid onion in reply in some cases.
fail can be NULL here.

Reported-by: https://github.com/shafemtol
Fixes: #5053
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:59:46 +10:30
Rusty Russell 5c20b8b1a3 devtools/decodemsg: add --list-tlvs
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell b19f3a5e7f devtools/decodemsg: don't require --onion for onion tlvs, fail if unknown tlvname.
Generate a table, let decodemsg sort it out.  Do more up-front work in
argparsing too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 7147fea2ff common: add const to accepted_extra_tlvs arg.
It was tlv_fields_valid that wanted a non-const: now that's gone, we
can make this correctly const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 93caef3584 common/tlvstream: allow fromwire_tlv to accept *any* unknown type.
This is used for keysend if EXPERIMENTAL_FEATURES is set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 83ee68ab06 common/tlvstream: put TLV checking back in the generic function.
Callers were supposed to call "tlv_fields_valid" after fromwire_tlv,
but few did.  Make this the default, and call the underlying function
directly where we want to be more flexible (one place).

This loses the ability to allow misordered fields, or to pass through
*any* even fields.  We restore that for special cases in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell a770f51d0e tools/generate_wire.py: make functions allocate the TLV.
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 2b422272e5 wire: generate printwire_ routines for bolt12 CSV.
Good exercise, and we need it when bolt12 messages appear in onion
messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell d15d629b8b plugins/fetchinvoice: remove obsolete string-based API.
Generate the payload in the callers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 04a152fbcf tools/generate-wire.py: make (and expose) individual TLV print functions.
When we actually put bolt12 fields (.e.g tlv_invoice) in onion messages,
that code will try to call printwire_tlv_invoice(), so expose it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell b3a2666a3e decodemsg: if decode fails, status 1.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 8f39002715 tools/generate-wire.py: have printwire routines do demarshal.
This works better in general: let printwire_x do the work of figuring
out how to demarshal x.  This is particularly important for TLVs, which
require a call to tlv_x_new() first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 88de64a580 tools/generate-wire.py: simplify printwire routines, fix ... handling.
We make them return bool, and always use names `cursor` and `plen` in
callers, for simplicity.

Also, `...` means "loop until finished" not "loop this many bytes".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 12c7b156c5 tools/generate_wire.py: allow generated C files to have includes added.
We previously ignored --include= for these, but onion is about to start
needing bolt12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell fa0c29f959 tools/generate_wire.py: tlvs should start with tlv_
No more "towire_offer", but "towire_tlv_offer".

This means we double-up on the unfortunately-named `tlv_payload` inside
the onion, but we should rename that in the spec when we remove
old payloads.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell f447e39d2d offers: neater response on malformed invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell 1cb93ffc27 devtools: remove blindedpath tool.
It only made obsolete onions anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 65f5bb2638 pytest: test for compat code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 4d0a97e1bd offers: import latest variant from draft, part 2.
Notably, the latest draft uses the correct tlv types inside the onion,
but we don't want to (we'd prefer to demarshal those as a separate step,
for better diagnostics), so we change it, then add a spec patch to change
it back.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 726b6878d1 offers: import latest variant from draft.
In particular, this changes the name of a field in invoice_request:
`payer_signature` becomes simply `signature`.  So we allow both for
now, and send the old one unless deprecated_apis is disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell e36d4d1143 devtools/decodemsg: fix printing of wireaddr.
printwire_ routines are supposed to print!  And they're only needed
inside devtools/.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 6f7d51ee68 common/onion: always fill in failtlvtype and failtlvpos parameters.
When we did fill them in, we filled them in wrong: the offset should be
the offset in the message, not the field number!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 613a21693d test: avoid testing for secret_eq under CI.
It's failing under CI, but loadavg is under 1.  Simply remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:40:55 +10:30
Sjors Provoost 560b090bd8 Add more warnings to fundchannel_start
[ Slight modifications to fix schema -- RR ]
2022-03-24 14:11:40 +10:30
Rusty Russell f95c9522fd plugins/bcli: fix false memleak detection.
Well, it is leaking a bool for the command duration, but that's probably OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 13:19:10 +10:30
Rusty Russell f7aba31448 lightningd: allow 'style' 'legacy' for clboss compat (with deprecated_apis).
We keep the parameter around for future use (e.g. PTLC support?), but clboss
master still sets 'style' to "legacy".  Accept, but ignore it.

Reported-by: grubman on #c-lightning (IRC)
Changelog-Deprecated: JSON-RPC: `sendpay` `route` argument `style` "legacy" (don't use it at all, we ignore it now and always use "tlv" anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 13:19:10 +10:30
Rusty Russell 5aa108f176 pytest: fix flake in test_pay.py::test_setchannel_state
The second disconnect could fail, if slow enough:

```
>       l1.rpc.disconnect(l2.info['id'], force=True)

tests/test_pay.py:2067:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:690: in disconnect
    return self.call("disconnect", payload)
contrib/pyln-testing/pyln/testing/utils.py:639: in call
    res = LightningRpc.call(self, method, payload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pyln.testing.utils.PrettyPrintingLightningRpc object at 0x7fb139ce87d0>
method = 'disconnect'
payload = {'force': True, 'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d'}

...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: disconnect, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'force': True}, error: {'code': -1, 'message': 'Peer not connected'}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 09:53:38 +10:30
Vincenzo Palazzo 53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30