Commit Graph

10482 Commits

Author SHA1 Message Date
Rusty Russell c9b82bf1d2 channeld: restore ping command, but only for channeld.
It's probably not worth fixing for the other daemons.

Changelog-Changed: JSON-RPC: `ping` now only works if we have a channel with the peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell be2622a4ff channeld: perform regular keepalive pings.
Send a ping every 15-45 seconds.  If we try to send another one and we
haven't got a reply, hang up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Send regular pings to detect dead connections (particularly for Tor).
2021-10-10 15:32:57 +02:00
Rusty Russell 140b2deec0 channeld: fix halting gossip when a timer is active.
We would sleep until the next timer, even if that's long past when we would
send gossip.  Normally we use very short timers, so we didn't notice, but
we will in the next patch, where we use continuous timers for pings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell 9138ebf807 channeld: remove liveness logic pre-commitment.
We're going to continuously ping, so this is redundant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell 1c85b27b4c gossipd: remove ping/pong handling
To minimize the diffs, we #if 0 the code.  We'll reenable it once
channeld is ready.

We also temporarily disable the ping tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell c394fd5db2 common/read_peer_msg: handle pings in handle_peer_gossip_or_error().
This is a noop for now, since gossipd handles them.  But that
will change in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Dustin Dettmer 6dc7fc6aee Fix memory leak abort in debug mode 2021-10-10 13:22:29 +02:00
Dustin Dettmer 3aa31d7cfd On Mac OS X a different error is raised on failed connection
[ Typo fixed --RR ]
2021-10-10 13:22:29 +02:00
Christian Decker c963e6b03f db: Don't switch between text and blob fields
The database will yell at you if you try.
2021-10-10 11:05:01 +10:30
Christian Decker b9c602c7e4 db: Fix a syntax error with the optional parameters
SQL doesn't really allow `a OR 1` as a clause since `1` is not a
boolean expression. Moving it into `a OR 1=1` however is valid again.
2021-10-10 11:05:01 +10:30
Christian Decker 8d3861ee29 ci: Add missing envvar forward for the DB configuration
We were not forwarding the matrix DB configuration, so we were only
ever testing `sqlite3`.
2021-10-10 11:05:01 +10:30
Rusty Russell 1534216aad channeld: handle reestablish from previous release with EXPERIMENTAL_FEATURES.
We switched channel_types from optional to compulsory bits in
cb22015b2a.

The result is infinite reconnects against older nodes; we reject what
they send, and they reject what we send.

The simplest fix is to neither send nor receive the (optional!) tlvs
unless we both advertize option_quiesce, which we now do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: channel_upgrade draft upgraded: cannot upgrade channels until peers also upgrade.
2021-10-08 16:07:21 +02:00
Rusty Russell 7157a92ea6 channeld: import updated channel_upgrade spec.
It now uses raw bitfields instead of a subtype, and only allows a single
option for any upgrade.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell 55dbe82162 features: EXPERIMENTAL_FEATURES: advertize option_quiesce
The latest draft has a feature bit here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell 1d14c7ee3d lnprototest: update to latest master.
Importantly, this version no longer assumes features 34/35 are unused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell c92ce59892 BOLT 12: switch invoice_request/invoice to singular `chain` field.
We keep the now-removed chains field, and in deprecated mode, we set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
2021-10-08 13:47:30 +02:00
Rusty Russell 45bf7a3974 bolt12: update to latest spec.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
2021-10-08 13:47:30 +02:00
Rusty Russell 8f582e770c BOLT12: use point32 instead of pubkey32.
That's the modern BOLT12 term.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
Rusty Russell 51ed7557a1 bolt12: upgrade to handle latest test vectors.
The latest ones use lno, not lni (this unit tests loads from
../lightning-rfc, silently exiting if it doesn't have the test
vector).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
Rusty Russell 947cc4730c Makefile: be sure to build generated sources before tests.
In particular, they are allowed to include .c files!

Here's `make check-units` on a maintainer-clean tree:

```
onchaind/test/run-onchainstress.c:4:10: fatal error: ../../hsmd/hsmd_wiregen.c: No such file or directory
    4 | #include "../../hsmd/hsmd_wiregen.c"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:276: onchaind/test/run-onchainstress.o] Error 1
make: *** Waiting for unfinished jobs....
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
ZmnSCPxj jxPCSnmZ d4690358d9 plugins/libplugin.c: Allow freeing notification `struct command *`.
We always allocate a new `struct command` when we get a full JSON
object from stdin:

b2df01dc73/plugins/libplugin.c (L1229-L1233)

If it happens to be a notification, we pass the `struct command` to
the handler, and not free it ourselves:

b2df01dc73/plugins/libplugin.c (L1270-L1275)

There are only nine points in `plugins/libplugin.c` where we `tal_free`
anything, and only one of them frees a `struct command`:

b2df01dc73/plugins/libplugin.c (L224-L234)

The above function `command_complete` is not appropriate for
notification handlers; the above function sends out a response
to our stdout, which a notification handler should not do.

However, as-is, it does mean that notification handling leaks
`struct command` objects, which can be problematic if we ever
have future built-in plugins which are significantly more
dependent on notifications.

This commit changes notification handlers to return
`struct command_result *`, because possibly in the future
notification handlers may want to perform `send_outreq`, so we
might as well use our standard convention for callbacks, and
to encourage future developers to check how to properly
terminate notification handlers (and free up the
`struct command`).

We also now provide a `notification_handled` function which a
notification handler must eventually call, as well as a
`notification_handler_pending` which is just a snowclone of
`command_still_pending`.
2021-10-08 14:40:04 +10:30
Devrandom e393791d4b Length prefix bitcoin tx, to fix layer violation 2021-10-07 14:52:18 +02:00
Christian Decker 69a47a3e1e ci: Add BSD configuration 2021-10-07 12:11:24 +02:00
Michael Folkson 40f8f180af docs: Update README to link to pyln-client 2021-10-06 16:20:13 +02:00
Christian Decker 24e60055bc gha: Fix the trigger in the PyPI publication action 2021-10-06 16:18:21 +02:00
jerzybrzoska 284a6a9024 correcting: 'needs' instead of 'need' 2021-10-06 16:17:18 +02:00
ZmnSCPxj jxPCSnmZ 20fe5c6ab1 common/json.c: Have `json_add_literal` *use* `len` argument.
The current `json_add_literal` does not use its `len` argument, at all:

09c2fef4a4/common/json.c (L1076-L1083)

Which is something of a WTF, why even require the `len` argument if it
is not even used in the function?
This would have been acceptable if it were a callback and the
callback-requiring function passed it in always, but this function is
not intended to be a callback but instead to be called directly.

Existing callers always pass in `strlen(literal)`, so this bug was not
noticed before:

09c2fef4a4/lightningd/jsonrpc.c (L511)
09c2fef4a4/lightningd/jsonrpc.c (L583)
09c2fef4a4/lightningd/options.c (L1408-L1409)
2021-10-06 16:09:20 +02:00
Michael Folkson a418cf5654 pylightning: Make it clear pyln-client should be used instead 2021-10-06 13:44:42 +02:00
Rusty Russell 09c2fef4a4 onion_message: dev options to ignore obsolete/modern onions.
This lets us test that both work, as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 368fc07d05 offers: send a modern onion reply in response to a modern request.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: Protocol: Updated onion_message support to match updated draft specification (with backwards compat for old version)
2021-10-04 11:58:31 +02:00
Rusty Russell af7e0a1183 offers: save replytok properly.
We carefully copied the buffer, but the tok is inside an array.  We get away
with it for now, but with coming changes it gets freed.  We need to copy
the token and all the tokens within it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell a71f2475d5 fetchinvoice: handle modern onion_message reply.
This comes in via the onion_message_ourpath hook, and we identify the
path by checking the node alias it came to (vs the obsolete version
which used the blinding).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 8dd038c106 fetchinvoice: send modern as well as obsolete messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 6c8c6a7c7d gossipd: handle modern onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell f9a21d9fc9 lightningd: handle modern onion termination.
This adds a new hook: onion_message_ourpath for when we know a message
came in via a blinded path we created.  The onion_message_blinded hook
is now called for all other messages, since all messages are now
blinded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 89d143bc63 lightningd: fix use-after-free during shutdown.
When we are calling hooks, we track them via a linked list.  As they
execute, we pop them off the list in plugin_hook_killed().

When we kill a plugin, we have a destructor which remove its entry from the linked list: plugin_hook_killed.

If it's at the head of the list, that means the plugin died while
processing the hook, so instead of just deleting it, we call
plugin_hook_killed() which behaves as if it said "result: continue".

But plugin_hook_killed() just returns if we're shutting down; this
leaves the link (then freed) on the list, and the *next* plugin tries
to unlink from the list, accessing the previous free entry.

The fix is simple: unlink from the list in plugin_hook_killed() even
if we're shutting down.

```
Valgrind error file: valgrind-errors.78570
==78570== Invalid write of size 8
==78570==    at 0x174B55: list_del_ (list.h:328)
==78570==    by 0x174FCC: plugin_hook_killed (plugin_hook.c:135)
==78570==    by 0x21DC3F: notify (tal.c:240)
==78570==    by 0x21E156: del_tree (tal.c:402)
==78570==    by 0x21E1A8: del_tree (tal.c:412)
==78570==    by 0x21E4F2: tal_free (tal.c:486)
==78570==    by 0x16EBD1: plugin_kill (plugin.c:345)
==78570==    by 0x16F9C4: plugin_conn_finish (plugin.c:724)
==78570==    by 0x20F1A5: destroy_conn (poll.c:244)
==78570==    by 0x20F1C9: destroy_conn_close_fd (poll.c:250)
==78570==    by 0x21DC3F: notify (tal.c:240)
==78570==    by 0x21E156: del_tree (tal.c:402)
==78570==  Address 0x6aee688 is 40 bytes inside a block of size 72 free'd
==78570==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==78570==    by 0x21E224: del_tree (tal.c:421)
==78570==    by 0x21E1A8: del_tree (tal.c:412)
==78570==    by 0x21E4F2: tal_free (tal.c:486)
==78570==    by 0x16EBD1: plugin_kill (plugin.c:345)
==78570==    by 0x16F9C4: plugin_conn_finish (plugin.c:724)
==78570==    by 0x20F1A5: destroy_conn (poll.c:244)
==78570==    by 0x20F1C9: destroy_conn_close_fd (poll.c:250)
==78570==    by 0x21DC3F: notify (tal.c:240)
==78570==    by 0x21E156: del_tree (tal.c:402)
==78570==    by 0x21E4F2: tal_free (tal.c:486)
==78570==    by 0x20D7B6: io_close (io.c:450)
==78570==  Block was alloc'd at
==78570==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==78570==    by 0x21DCAD: allocate (tal.c:250)
==78570==    by 0x21E26E: tal_alloc_ (tal.c:428)
==78570==    by 0x175599: plugin_hook_call_ (plugin_hook.c:259)
==78570==    by 0x13616F: plugin_hook_call_onion_message_blinded (onion_message.c:126)
==78570==    by 0x13643B: handle_obs_onionmsg_to_us (onion_message.c:187)
==78570==    by 0x138BBD: gossip_msg (gossip_control.c:140)
==78570==    by 0x178AEC: sd_msg_read (subd.c:495)
==78570==    by 0x20CA00: next_plan (io.c:59)
==78570==    by 0x20D608: do_plan (io.c:407)
==78570==    by 0x20D64A: io_ready (io.c:417)
==78570==    by 0x20F8F1: io_loop (poll.c:445)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
jerzybrzoska d54594c3dc Correcting typo 2021-10-02 14:02:55 +02:00
Christian Decker b2df01dc73 gci: Checkout tags in CI to get auto-versioning working 2021-09-28 18:34:43 +02:00
Christian Decker b8f79d3e44 gci: Remove push trigger 2021-09-28 18:34:43 +02:00
Christian Decker 058484b8b1 gci: Use stripped down bitcoind for Mac OS 2021-09-28 18:34:43 +02:00
Christian Decker ef579e7e9f gci: Pin down a couple more dependencies 2021-09-28 18:34:43 +02:00
Christian Decker 0e987e5ef5 gci: Limit PyPI publication to pull request merged and version tags 2021-09-28 18:34:43 +02:00
Christian Decker 478c43cd9c pyln: Derive version from git for pyln-proto 2021-09-28 18:34:43 +02:00
Christian Decker 5efa7659f9 pyln: Derive version from git for pyln-testing 2021-09-28 18:34:43 +02:00
Christian Decker 808f582638 gci: Checkout entire history in PyPI build to ensure tags are there
setuptools_scm requires the ability to look up the latest tag.
2021-09-28 18:34:43 +02:00
Christian Decker c4af904342 pyln: Add production publication on tag push 2021-09-28 18:34:43 +02:00
Christian Decker ec75090890 doc: Add the in-tree-build option to install docs
This is required for the setuptools_scm package to correctly identify
the root of the git repository when installing from the
requirements.txt file in the root. It'd otherwise copy the source
directory, which doesn't yet include the version metadata, into a
separate directory and the building from there, which breaks the
git lookup.

`in-tree-build` is the future default, so we'll eventually be able to
strip that option again. See [1] for details.

[1] https://github.com/pypa/pip/issues/7555
2021-09-28 18:34:43 +02:00
Christian Decker 71740283e4 gci: Add missing tooling 2021-09-28 18:34:43 +02:00
Christian Decker fe9949ce0b pyln: Derive version from git for pyln-client 2021-09-28 18:34:43 +02:00
Christian Decker c8a360b81a gci: Give PyPI packages a reasonable name 2021-09-28 18:34:43 +02:00