Commit Graph

10501 Commits

Author SHA1 Message Date
Rusty Russell a71f2475d5 fetchinvoice: handle modern onion_message reply.
This comes in via the onion_message_ourpath hook, and we identify the
path by checking the node alias it came to (vs the obsolete version
which used the blinding).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 8dd038c106 fetchinvoice: send modern as well as obsolete messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 6c8c6a7c7d gossipd: handle modern onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell f9a21d9fc9 lightningd: handle modern onion termination.
This adds a new hook: onion_message_ourpath for when we know a message
came in via a blinded path we created.  The onion_message_blinded hook
is now called for all other messages, since all messages are now
blinded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell 89d143bc63 lightningd: fix use-after-free during shutdown.
When we are calling hooks, we track them via a linked list.  As they
execute, we pop them off the list in plugin_hook_killed().

When we kill a plugin, we have a destructor which remove its entry from the linked list: plugin_hook_killed.

If it's at the head of the list, that means the plugin died while
processing the hook, so instead of just deleting it, we call
plugin_hook_killed() which behaves as if it said "result: continue".

But plugin_hook_killed() just returns if we're shutting down; this
leaves the link (then freed) on the list, and the *next* plugin tries
to unlink from the list, accessing the previous free entry.

The fix is simple: unlink from the list in plugin_hook_killed() even
if we're shutting down.

```
Valgrind error file: valgrind-errors.78570
==78570== Invalid write of size 8
==78570==    at 0x174B55: list_del_ (list.h:328)
==78570==    by 0x174FCC: plugin_hook_killed (plugin_hook.c:135)
==78570==    by 0x21DC3F: notify (tal.c:240)
==78570==    by 0x21E156: del_tree (tal.c:402)
==78570==    by 0x21E1A8: del_tree (tal.c:412)
==78570==    by 0x21E4F2: tal_free (tal.c:486)
==78570==    by 0x16EBD1: plugin_kill (plugin.c:345)
==78570==    by 0x16F9C4: plugin_conn_finish (plugin.c:724)
==78570==    by 0x20F1A5: destroy_conn (poll.c:244)
==78570==    by 0x20F1C9: destroy_conn_close_fd (poll.c:250)
==78570==    by 0x21DC3F: notify (tal.c:240)
==78570==    by 0x21E156: del_tree (tal.c:402)
==78570==  Address 0x6aee688 is 40 bytes inside a block of size 72 free'd
==78570==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==78570==    by 0x21E224: del_tree (tal.c:421)
==78570==    by 0x21E1A8: del_tree (tal.c:412)
==78570==    by 0x21E4F2: tal_free (tal.c:486)
==78570==    by 0x16EBD1: plugin_kill (plugin.c:345)
==78570==    by 0x16F9C4: plugin_conn_finish (plugin.c:724)
==78570==    by 0x20F1A5: destroy_conn (poll.c:244)
==78570==    by 0x20F1C9: destroy_conn_close_fd (poll.c:250)
==78570==    by 0x21DC3F: notify (tal.c:240)
==78570==    by 0x21E156: del_tree (tal.c:402)
==78570==    by 0x21E4F2: tal_free (tal.c:486)
==78570==    by 0x20D7B6: io_close (io.c:450)
==78570==  Block was alloc'd at
==78570==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==78570==    by 0x21DCAD: allocate (tal.c:250)
==78570==    by 0x21E26E: tal_alloc_ (tal.c:428)
==78570==    by 0x175599: plugin_hook_call_ (plugin_hook.c:259)
==78570==    by 0x13616F: plugin_hook_call_onion_message_blinded (onion_message.c:126)
==78570==    by 0x13643B: handle_obs_onionmsg_to_us (onion_message.c:187)
==78570==    by 0x138BBD: gossip_msg (gossip_control.c:140)
==78570==    by 0x178AEC: sd_msg_read (subd.c:495)
==78570==    by 0x20CA00: next_plan (io.c:59)
==78570==    by 0x20D608: do_plan (io.c:407)
==78570==    by 0x20D64A: io_ready (io.c:417)
==78570==    by 0x20F8F1: io_loop (poll.c:445)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
jerzybrzoska d54594c3dc Correcting typo 2021-10-02 14:02:55 +02:00
Christian Decker b2df01dc73 gci: Checkout tags in CI to get auto-versioning working 2021-09-28 18:34:43 +02:00
Christian Decker b8f79d3e44 gci: Remove push trigger 2021-09-28 18:34:43 +02:00
Christian Decker 058484b8b1 gci: Use stripped down bitcoind for Mac OS 2021-09-28 18:34:43 +02:00
Christian Decker ef579e7e9f gci: Pin down a couple more dependencies 2021-09-28 18:34:43 +02:00
Christian Decker 0e987e5ef5 gci: Limit PyPI publication to pull request merged and version tags 2021-09-28 18:34:43 +02:00
Christian Decker 478c43cd9c pyln: Derive version from git for pyln-proto 2021-09-28 18:34:43 +02:00
Christian Decker 5efa7659f9 pyln: Derive version from git for pyln-testing 2021-09-28 18:34:43 +02:00
Christian Decker 808f582638 gci: Checkout entire history in PyPI build to ensure tags are there
setuptools_scm requires the ability to look up the latest tag.
2021-09-28 18:34:43 +02:00
Christian Decker c4af904342 pyln: Add production publication on tag push 2021-09-28 18:34:43 +02:00
Christian Decker ec75090890 doc: Add the in-tree-build option to install docs
This is required for the setuptools_scm package to correctly identify
the root of the git repository when installing from the
requirements.txt file in the root. It'd otherwise copy the source
directory, which doesn't yet include the version metadata, into a
separate directory and the building from there, which breaks the
git lookup.

`in-tree-build` is the future default, so we'll eventually be able to
strip that option again. See [1] for details.

[1] https://github.com/pypa/pip/issues/7555
2021-09-28 18:34:43 +02:00
Christian Decker 71740283e4 gci: Add missing tooling 2021-09-28 18:34:43 +02:00
Christian Decker fe9949ce0b pyln: Derive version from git for pyln-client 2021-09-28 18:34:43 +02:00
Christian Decker c8a360b81a gci: Give PyPI packages a reasonable name 2021-09-28 18:34:43 +02:00
Christian Decker 8a22cbfce7 pyln: Materialize setup.py and requirements.txt in pyln-spec
When downloading a python package from the PyPI repository the links
where pointing to a non-existent parent directory, thus breaking the
packages. The files don't ever change, and are really simple, so let's
just materialize them.
2021-09-28 18:34:43 +02:00
Christian Decker 4fe99d459d gci: Add workflow to deploy PyPI artifacts 2021-09-28 18:34:43 +02:00
Christian Decker accaa07dde pytest: Stabilize `test_addgossip`
It was incredibly flaky due to the potential for l2 announcing the
channel before l1 could get to it, thus suppressing the outgoing
announcement which we were looking for. This now checks either
direction.

Before this fix the failure rate was 24% (out of 100 runs), afterwards
it's 0%.

Changelog-None
2021-09-27 17:54:50 +02:00
Vincenzo Palazzo 3283d05303 misc: ignored last autogenerated file from the github tree.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-27 16:28:41 +02:00
Christian Decker bb307fa59c misc: Add generated files to .gitignore 2021-09-25 14:55:31 +02:00
Christian Decker d35043a1c6 gci: Add macOS configuration 2021-09-24 17:19:16 +02:00
W. J. van der Laan dd93f5dd29 Add missing headers for compiling on FreeBSD 13
- `netinet/in.h`: struct sockaddr_in, struct sockaddr_in6
- `unistd.h`: close()
2021-09-24 12:40:09 +09:30
Rusty Russell 33168fc733 lightningd: provide 10 minutes for channel fee increases to propagate.
This was measured as a 95th percentile in our rough testing, thanks to
all the volunteers who monitored my channels.

Fixes: #4761
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `setchannelfee` gives a grace period (`enforcedelay`) before rejecting old-fee payments: default 10 minutes.
2021-09-23 15:05:09 +02:00
Rusty Russell 8fe0ac8d37 lightningd: refactor forward feecheck.
Make it do the feerate calc internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:05:09 +02:00
Rusty Russell 09e0794b06 common/features: try to add all proposed feature names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:02:05 +02:00
Rusty Russell 7ef2f4d7fb devtools/features: tool to convert feature bitmap to names.
For example:

```
$ ./devtools/features 80008008226aa2
option_data_loss_protect/odd (optional)
option_upfront_shutdown_script/odd (optional)
option_gossip_queries/odd (optional)
option_var_onion_optin/odd (optional)
option_gossip_queries_ex/odd (optional)
option_static_remotekey/odd (optional)
option_payment_secret/even (compulsory)
option_basic_mpp/odd (optional)
option_anchor_outputs/odd (optional)
option_shutdown_anysegwit/odd (optional)
option_onion_messages/odd (optional)
option_unknown_54/odd (optional)
```


Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:02:05 +02:00
Rusty Russell be8e45b16d pytest: fix flake in test_gossip.py::test_addgossip
We can miss it in both logs, so wait for it instead:

```
2021-09-22T07:25:59.1582950Z >       l3.rpc.addgossip(ann.split()[3])
2021-09-22T07:25:59.1583911Z E       AttributeError: 'NoneType' object has no attribute 'split'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 12:57:07 +02:00
Rusty Russell 79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Rusty Russell 3dbf3b057a Makefile: remove NO_PYTHON flag in anticipation of making it compulsory.
And note the EXPERIMENTAL_DUAL_FUND env var in HACKING.md.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 15:25:14 +02:00
Vincenzo Palazzo eb103c15df review 2/2: Use generic query to make the code more readable.
Suggested by @cdecker

P.S: Also this include an API refactoring from my previous solution, also this it is suggested by @cdecker.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo bea6ea27e8 review 1/2: Fixed the doc missing details about formatting.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo 9d5c992fc8 plugin: Adding status to the pay plugin
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

Rebase

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo 62ef403d20 doc: Update doc with the new parameter supported
Changelog-Added: Support to listpays the status parameter to filter the payments by status.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo 98623aaff8 pyln-client: Adding parameter to the listpays rpc method.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo ee8c7252a3 wallet db: Support the query on database with status and payment hash.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo fd33aed4b5 rpc: Integrate the status flow in the listsendpays command
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Rusty Russell eaec226f99 common: json_to_reply_path to helper to handle blindedpath/onion_message hook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell b8498b60f2 lightningd: `blindedpath` helper to create a blinded path to ourselves.
Currently it will be used for onion replies, but we can use it for offers
and invoices in future, if we want to avoid revealing our node_id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell b87e0eb4bf lightningd: new sendonionmessage routine.
This expects the caller to create the TLVs to put in each hop; it
simply creates the onion and sends it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell 01161aac68 hsmd: derive an onion_reply secret.
We put this in reply paths, so we can tell if they are used.  This lets us
avoid responding unless the correct reply path is used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell 3a966191b8 common/blindedpath: enctlv unwrapping primitives.
And we check them on our test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell b98e8a099d common/blindedpath: enctlv creation primitives.
And test vector generation!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell f2a4bd6ad8 wire: import new onion message spec.
One change from the obsolete version handling, gossipd will no longer send
forwarding onion msgs to lightningd, but will forward it directly.
That was the effect before, anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Christian Decker 5c38e5a08f invoice: Fix mismatch between ">=" and "greated than" in error msg 2021-09-22 09:08:48 +09:30
Christian Decker 910e79ddb5 bcli: Accept "already in chain" errors as success 2021-09-22 09:08:48 +09:30
Christian Decker 39b71ef4a1 doc: Add links to mandelbit's recovery walkthrough 2021-09-22 09:08:48 +09:30