Commit Graph

11385 Commits

Author SHA1 Message Date
Rusty Russell 072c4711ec pytest: convert test_sendonion_rpc to modern TLV onion.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell 45143cc731 pytest: Remove onion test vectors containing legacy onions.
I thought about fixing them up, but really these should be in
lnprototest anyway.  Turns out they're from the spec, so we should
actually fix them up there.

I moved the vector files into contrib/pyln-proto, since that still
needs them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell 0db05f6e9c lightningd: opt_var_onion is now a compulsory feature.
We're about to drop support for legacy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Ken Sedgwick 36466af3eb hsmd: Add fields to hsmd_sign_{,remote_}commitment_tx for validating signers 2022-03-16 12:06:21 +10:30
Christian Decker 704162f24a docker: Build the docker image with psql support
Changelog-Fixed: docker: The docker image is now built with postgresql support
2022-03-16 11:25:41 +10:30
Rusty Russell 508b8cb54a doc: fix sendonionmessage documentation.
It's a low-level interface now, expecting you to build your own TLVs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Reported-by: @valentinewallace
2022-03-16 11:24:42 +10:30
Christian Decker a8aa9bd5ae cln-rpc: Fixed two minor compiler warnings 2022-03-16 11:11:58 +10:30
Christian Decker 487b5e6169 msggen: Add meta file to keep the field numbers stable
We are inferring the field numbers on the fly, which isn't really
compatible with the way GRPC field numbers work, i.e., they must be
stable while the IDL file evolves. So far when a field was added in
the middle of a struct or removed all subsequent fields would get
renumbered, essentially breaking any client that was using the old
scheme.

We now add a meta file `.msggen.json` that keeps track of the numbers
assigned so far, so they can be reused, and new ones can be generated
not to conflict with existing ones. This file is intentionally kept
generic, so other generators can add more information that has to be
managed across runs.

Changelog-None
2022-03-16 11:11:58 +10:30
Christian Decker e99b11565c pyln: Change the default argument to `exclude` in `pay`
We must use `None` for default arguments since otherwise they aren't
filtered out when serializing the request. In addition default
arguments to functions are initialized once and, if mutable, could
persist internal changes across function calls.

Changelog-None
2022-03-15 12:20:35 -05:00
William Casarin f72a08c802 websocketd: fix random failures by blocking stdin reads
Example request that is dying:

NEW REQUEST! lightning_websocketd:main [1955685] <-- bad request from safari

   read 507
   write_all 1
-> websocket_to_lightningd
-> read_payload_header
   read 2
   read_all 1
   read -11      <--- This tried to read a part of the header, is this -EAGAIN?
   read_all 0         should we be blocking on these reads?
   *dies*

Fixes #5089

Changelog-Fixed: `experimental-websocket` intermittent read errors fixed
Signed-off-by: William Casarin <jb55@jb55.com>
2022-03-14 12:26:46 -05:00
GoofyAF a2a6b8c3ff
Goofy af tor.md systemd syntax update (#5059)
Update TOR.md

Update tor install to represent current systemd syntax for enabling and starting a new system service
Corrected a typo. After tor is installed to refresh /etc/tor/torrc config changes the command should be 'sudo systemctl restart tor' not 'sudo systemctl start tor'
2022-03-13 10:30:11 +10:30
Michael Schmoock 9703ee05bf doc: update faq and docs for IP discovery 2022-03-12 16:35:09 +10:30
Vincenzo Palazzo 6a146bed26 db: small code cleanup
Changelog-None: db: small code cleanup

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-11 16:43:15 +10:30
Michael Schmoock db95893aec lightningd: do not use remote_addr for always_use_proxy 2022-03-11 16:42:45 +10:30
Michael Schmoock ef84d6eec5 chore: remove EXPERIMENTAL for rfc #917 remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock 57fb34ed06 test: connectd netaddress
Increases test coverage by adding a testcase for connectd/netaddress.c

Changelog-None
2022-03-11 16:42:45 +10:30
Michael Schmoock 07fbc7ef13 pytest: IP address discovery updates node_announcement 2022-03-11 16:42:45 +10:30
Michael Schmoock 67fdc6f8ad gossipd: send updated node_annoucement remote_addr
This is the cheapest algo I came up with that simply checks that the
same `remote_addr` has been report by two different peers. Can be
improved in many ways:

 - Check by connecting to a radonm peers in the network
 - Check for more than two confirmations or a certain fraction
 - ...

Changelog-Added: Send updated node_annoucement when two peers report the same remote_addr.
2022-03-11 16:42:45 +10:30
Michael Schmoock 28b4e57974 lightningd: store recently reported remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock b930b8c548 wireaddr: adds wireaddr_eq_without_port and wireaddr_cmp_type
Adds wireaddr_eq_without_port so it can be used later.
Moves wireaddr_cmp_type from connectd.c to this file, so it can be reused later.
2022-03-11 16:42:45 +10:30
Michael Schmoock f1981461ef connectd: ignore private remote_addr on non-DEVELOPER builds
When compiled without DEVELOPER this will now filter out `remote_addr` that
come from localhost. The testcase checks for DEVELOPER to test for correct
function of `remote_addr`.

Also, I renamed "test_connect" to "test_connect_basic" so it can be started
without all the other tests in that file that start with "test_connect..."
2022-03-11 16:42:45 +10:30
Michael Schmoock a01e2740ef pyln-proto: fix port typo in example script
Changelog-None
2022-03-11 16:42:45 +10:30
Michael Schmoock e92176248e chore: fix typo announcable -> announceable
"announcable" is a common misspelling of "announceable", see:

https://en.wiktionary.org/wiki/announcable
2022-03-11 16:42:45 +10:30
Vincenzo Palazzo 2c55c27ad4 doc: remove ambiguity for functionality enabled after a release version
Changelog-None: remove ambiguity for functionality enabled after a release version

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-10 10:24:34 +10:30
Marnix c357411337 Update LICENSE
Update LICENSE to 2022
2022-03-10 10:23:40 +10:30
Ken Sedgwick 8f56f96801 hsmd: Add wallet index metadata to existing messages 2022-03-10 10:22:29 +10:30
Ken Sedgwick 3abe22213c hsmd: Augment call to hsmd_sign_withdrawal_tx with wallet index metadata 2022-03-10 10:22:29 +10:30
Ken Sedgwick 8876c03791 hsmd: Add PSBT keypath utility functions 2022-03-10 10:22:29 +10:30
Ken Sedgwick 1e1948bfd2 hsmd: Add redeemscripts to anchor outputs 2022-03-10 10:22:29 +10:30
Christian Decker 6706384c50 cln-plugin: Ensure we cleanly shut down when we lose the master conn 2022-03-10 10:21:41 +10:30
Christian Decker 1bd2b8c9f7 cln-plugin: Make notification handlers asynchronous 2022-03-10 10:21:41 +10:30
Christian Decker af4eed3787 cln-plugin: Make hooks asynchronous 2022-03-10 10:21:41 +10:30
Christian Decker a7ef38732f cln-plugin: Make rpcmethod handlers async 2022-03-10 10:21:41 +10:30
Christian Decker 60e773239c cln-plugin: Add notification subscriptions and hooks to the plugins
For now hooks are treated identically to rpcmethods, with the
exception of not being returned in the `getmanifest` call. Later on we
can add typed handlers as well.
2022-03-10 10:21:41 +10:30
Christian Decker 8c6af21169 cln-plugin: Add support for synchronous RPC methods
Changelog-Experimental: cln-plugin: Added support for non-async RPC method passthrough (async support coming soon)
2022-03-10 10:21:41 +10:30
Christian Decker 22618a2f94 cln-plugin: Rework the plugin library using a Builder 2022-03-10 10:21:41 +10:30
Christian Decker 4aba119733 pytest: Use valgrind target suppressions instead of skipping tests
Having a list of very targeted suppressions allows us to still run the
majority of tests with valgrind checking, and not fail when Rust does
some trickery. This is for example the case with `std::sync::Once`
which uses `num_procs` calling out to the cgroups subsystem, sometimes
with a null path.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-10 10:21:41 +10:30
Christian Decker a1555623bc pytest: Mark Rust-dependent tests as skipped with VALGRIND
`valgrind` reports seems to flag some memory accesses that are ok in
the Rust standard library, which we can consider false positives for
our purposes:

```Valgrind error file: valgrind-errors.69147
==69147== Syscall param statx(file_name) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==
==69147== Syscall param statx(buf) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==

```
2022-03-10 10:21:41 +10:30
Christian Decker fbcb4c33ad cln-plugin: Populate the options when we get an `init` call 2022-03-10 10:21:41 +10:30
Christian Decker 249fa8675a cln-plugin: Add options to the `getmanifest` call 2022-03-10 10:21:41 +10:30
Christian Decker fe21b89b56 pytest: Add a test for the cln-plugin logging integration 2022-03-10 10:21:41 +10:30
Christian Decker f5e1829117 cln-plugin: Implement logging facade adapter for cln plugins
We wrap emitted messages into a JSON-RPC notification envelope and
write them to stdout. We use an indirection over an mpsc channel in
order to avoid deadlocks if we emit logs while holding the writer lock
on stdout.
2022-03-10 10:21:41 +10:30
Christian Decker 9ae1f33992 cln-plugin: Get started with the plugin interface 2022-03-10 10:21:41 +10:30
Vincenzo Palazzo c0c826d2ee ci: introduce in ci the compilation testing on different os
Changelog-None: introduce in ci the compilation testing on different os

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-10 09:44:16 +10:30
Rusty Russell d7ffb712e5 dualopend: restore memleak calls.
And implement a timeout (20 seconds) just in case it's not listening.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-10 09:40:09 +10:30
Rusty Russell 84e0e743eb dualopend: fix memleak reports.
Next patch re-enables runtime leak checking for dualopend, so fix those
leak reports.

In some cases, this menas allocating off tmpctx or state->channel
(which gets reset on failure), not state.  The problem with tmpctx is
that there are event loops in the *middle* of some functions, which
free it.  So for RBF functions we use a rbf_ctx temporary (with leak
detection suppressed, like it is for tmpctx), then be careful to free
it on all exits!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-10 09:40:09 +10:30
Rusty Russell 00bb6f07d7 lightningd: simplify memleak code.
Instead of doing this weird chaining, just call them all at once and
use a reference counter.

To make it simpler, we return the subd_req so we can hang a destructor
off it which decrements after the request is complete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-10 09:40:09 +10:30
Christian Decker 0c4cc782df docker: Clean up the dockerfile to use poetry 2022-03-08 05:00:36 +10:30
Christian Decker 81259bbd67 py: Update mrkd 2022-03-08 05:00:36 +10:30
Christian Decker ad22becb26 pyln: Switch to binary psycopg2 2022-03-08 05:00:36 +10:30