Commit Graph

10899 Commits

Author SHA1 Message Date
Rusty Russell 405239e353 doc/Makefile: speed up schema check, only do on check-doc.
Don't do this on every build, it takes 4 seconds.  And split it
into individual targets, so make can parallelize (1.6 seconds here).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-14 09:37:52 +10:30
Simon Vrouwe 426ff0abff lightningd: cleanup obsolete plugins->shutdown flag
After leaving the main event loop, the only path to destroy_plugin
goes via shutdown_plugins.
2021-12-14 09:33:10 +10:30
Simon Vrouwe 28816c387c testing: remove test_htlc_accepted_hook_shutdown, move relevant parts
into test_plugin_shutdown

The fact that plugins are kept alive untill the very end also ensures
their hooks are not silently unregistered.
2021-12-14 09:33:10 +10:30
Simon Vrouwe 9b70c9d63b testing: fix flake in test_fetchinvoice_3hop 2021-12-14 09:33:10 +10:30
Simon Vrouwe 605fda7214 common: cleanup unsused parameter in timer_expired() 2021-12-14 09:33:10 +10:30
Simon Vrouwe f936fa926f plugins: simplify shutdown loop, simply close the db
The only thing that needs ld->wallet after this is destroy_invoices_waiter (off jsonrpc)
Could not find any other destructors (destroy_*) that need wallet or db access after this.
Any db access would now segfault.
2021-12-14 09:33:10 +10:30
benthecarman be196b4219 Update .gitignore for tmp files
Changelog-None
2021-12-11 01:56:57 +00:00
Michael Schmoock c2d2cc1274 connectd: fix empty error message
1. Adds the missing DNS error massages so they can be handled by
   connect_control.
2. Prepends a 'All addresses failed' to code 401 message, so we
   always have at least some error message to the user.

Changelog-None
2021-12-08 13:52:24 +01:00
nicolas.dorier d5f2b1126c Fix dockerfile for arm32/64 2021-12-08 12:46:58 +01:00
Michael Schmoock bad09887e0 pytest: fix test_pluginconnected_hook_chaining
The last line of the testcase was checking on the wrong node l3
instead of l1. l3 didn't had the plugins configured that would
produce the log entry we were looking for not to be present.

Changelog-None
2021-12-06 14:53:37 +01:00
JohnOnGit 8898af1ea7 Upgrade bitcoin-cli to 22.0 2021-12-06 14:45:23 +01:00
Jan Sarenik 4274080d81 doc: lightningd-config.5.md - default ports
Also nit: s/possible/possibly suggested by michaelfolkson.
2021-12-06 17:10:08 +10:30
Jan Sarenik 96f28323bd Set default port according to network
The idea is to have different default ports for different networks.

Current default port is `9735` for everything. Let's use it for
the mainnet and reuse the difference added to the default port
from `rpc_port` values in `bitcoin/chainstate.c`.

Testnet would be `19735` (adding rpc_port - 8332 = `10000`).

Signet would be `39735` (adding rpc_port - 8332 = `30000`).

Regtest would be `19846` (adding rpc_port - 8332 = `10111`).

With Vincenzo's kind pair-programming help over tmate.

Two other commits were squashed into this one so that bisecting
never ends up in half-baked state:

1. chainparams: Fix regtest default rpc_port

   bitcoind -help says this:

    -rpcport=<port>
         Listen for JSON-RPC connections on <port> (default: 8332, testnet:
         18332, signet: 38332, regtest: 18443)

2. test_gossip: Default port for regtest

   hex: 2607 is now .... (could be 4d86 but Elements uses another port)
   dec: 9735 is now any port (could be 19846 ^^ but now is for any port)

   The lines which were binding to default port were removed as the
   default port is different on each network.

NOTE: Remember not to modify gossip_store tests which loads everything raw
      including the checksums.

Changelog-Changed: If the port is unspecified, the default port is chosen according to used network similarly to Bitcoin Core.
2021-12-06 17:10:08 +10:30
Andrew Toth 78fb78478b make: remove generated files when running clean 2021-12-06 10:06:25 +10:30
Rusty Russell d22fd59997 gossipd: remove gossip_msg.[ch]
This was a remnant from when we used to get routing from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 484222b0a1 daemons: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 786732601c common: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell d9968bbc0c bitcoin: remove unused functions, or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell f64df6fbb5 devtools: simple script to look for functions which don't seem to be used.
Gives a rough first-pass, at least.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 186b895440 tools/check-includes.sh: make shellcheck happy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell c3899806ee common: remove unused io_lock.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell bdabef9a9b pytest: fix occasional flake in test_announce_address
We make sure the gossip msg was sent, but the other node might not
have digested it yet:

```
        # Check other node can parse these
>       addresses = l2.rpc.listnodes(l1.info['id'])['nodes'][0]['addresses']
E       KeyError: 'addresses'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:18 +10:30
Matt Whitlock 1f79aad830 common/utils: introduce tmpdir_mkstemp; use it
Various unit tests were creating temporary files unconditionally in /tmp
and were not cleaning up after themselves. Introduce a new variant of
mkstemp(3p) that respects the TMPDIR environment variable, and use it in
the offending unit tests. This allows each test run to use a dedicated
TMPDIR that can be cleaned up after the run.

Changelog-None

Signed-off-by: Matt Whitlock <c-lightning@mattwhitlock.name>
2021-12-06 09:20:27 +10:30
Rusty Russell 5284ee4dae connectd: don't advertize websocket support if we have no other (non-Tor) addrs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Rusty Russell b45544c659 options: fix handling of wildcard (allproto) address.
We treated ':' as an empty DNS name in EXPERIMENTAL, which is wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Rusty Russell ff556fefc6 connectd: fix websocket binding when we're doing both IPv4 and IPv6 on same port.
We would fail connectd when listening on the IPv6 version failed; instead we should
allow that.

Changelog-Experimental: experimental-websocket-port fixed to work with default addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Rusty Russell cf40e585c3 pytest: test to show conflict between websocket and wildcard addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Michael Schmoock 94cb42bee6 status: do not log ping and pong packets
Debug logging of ping and pong packets turned out to be
too excessive since this is now done on all channels
and without config option.

Changelog-None
2021-12-05 14:57:53 +10:30
Jan Sarenik 84d83bd716 .github: Fix FreeBSD workflow
The docs https://github.com/marketplace/actions/freebsd-vm
say that `runs-on:` must be `macos-10.15`.
2021-12-05 14:51:17 +10:30
Andrew Toth 5483465fb5 doc: add exclude arg doc for pay rpc 2021-12-04 21:37:42 +10:30
Andrew Toth 69bc1191cb tests: add pay test for exclude arg 2021-12-04 21:37:42 +10:30
Andrew Toth fa6f01d5b1 Add route exclusion payment modifier and pay rpc exclude arg
Changelog-Added: Add `exclude` option for `pay` command to manually exclude channels or nodes when finding a route.
2021-12-04 21:37:42 +10:30
Andrew Toth 384c359c79 refactor: move exclude parsing to json_tok 2021-12-04 21:37:42 +10:30
ZmnSCPxj jxPCSnmZ 80661f920b doc/lightning-datastore.7.md: Fix typos.
Changelog-None
2021-12-02 12:27:32 +00:00
ZmnSCPxj jxPCSnmZ 3433ff5e15 wallet/db.c, wallet/wallet.c: Add a partial index to speed up startup.
Closes: #4901

Tested by `EXPLAIN QUERY PLAN` on sqlite3; #4901 shows the result from
@whitslack doing a similar partial index on PostgreSQL on his ~1000 chan
node.

ChangeLog-Added: db: Speed up loading of pending HTLCs during startup by using a partial index.
2021-12-02 11:54:57 +01:00
Rusty Russell 5a5cf8c696 pytest: fix flake in testing.
As noted in 0a406230d0 (diff-5871d4c569454db5e625383975462132da0bd03d32df145d8d72d8fafd86d952R3544-R3546)

Turns out we sometimes hang up before l2 sees the previous tx revoked,
so we get a normal unilateral close, not a cheat.

Reported-by: Simon Vrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 09:48:00 +10:30
Rusty Russell e2698c5fc3 lightningd: fix leak report for self_id
This has been here for a while: self_id hangs around while we're
calling the hook, but now it triggers sometimes.

```
E           ValueError: 
E           Node errors:
E           Global errors:
E            - Node /tmp/ltests-3mcyp67u/test_dev_rawrequest_1/lightning-1/ has memory leaks: [
E               {
E                   "backtrace": [
E                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E                       "gossipd/gossipd_wiregen.c:528 (fromwire_gossipd_got_onionmsg_to_us)",
E                       "lightningd/onion_message.c:152 (handle_onionmsg_to_us)",
E                       "lightningd/gossip_control.c:137 (gossip_msg)",
E                       "lightningd/subd.c:548 (sd_msg_read)",
E                       "ccan/ccan/io/io.c:59 (next_plan)",
E                       "ccan/ccan/io/io.c:407 (do_plan)",
E                       "ccan/ccan/io/io.c:417 (io_ready)",
E                       "ccan/ccan/io/poll.c:453 (io_loop)",
E                       "lightningd/io_loop_with_timers.c:21 (io_loop_with_timers)",
E                       "lightningd/lightningd.c:1164 (main)"
E                   ],
E                   "label": "gossipd/gossipd_wiregen.c:528:struct secret",
E                   "parents": [
E                       "lightningd/onion_message.c:149:struct onion_message_hook_payload",
E                       "lightningd/plugin_hook.c:81:struct hook_instance *[]"
E                   ],
E                   "value": "0x55cf3cbc9458"
E               }
E           ]

```
2021-12-01 05:44:28 +10:30
Rusty Russell 132ac0b1b4 blinding: check against two of the test vectors in the PR.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 296437c655 bolt12: remove deprecated `chains` fields
We also move recurrence fields into a separate spec patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 163d3a16f4 doc: clean up offers bolt quotes
As of 2b923a0367c5f9154fcec706e3302cc4658dd889.

Recurrence quotes need to be marked separately, since they're no longer
in offers main bolt.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 894f1841b4 features: don't announce onion_messages in bolt11 invoices.
As noted by Joost.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 4c63fedce9 test: make an onionmessage test vector.
This builds on the enctlv vectors, but actually goes all the way
to creating a modern onionmessage.

Thanks to Thomas H for corrections!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell e02e0a197d common/test/run-blindedpath_enctlv: use modern functions to create test vector.
And include the final destination enctlv.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 1c230eee86 test: hack in test for (now-obsolete!) test vector Thomas emailed me.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 605b3bf985 pytest: re-enable modern/obsolete fetchonion tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 5361107c9c gossipd: handle receipt of modern onion message.
And wire it through to the hook; update the plugins to recognize
modern vs obs2 onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell e7b263304e lightningd: Send updated onion spec messages.
It's very similar to the previous, but there are a few changes:

1. The enctlv fields are numbered differently.
2. The message itself is a different number.

The onionmsg_path type is the same, however, so we keep that constant
at least.

The result is a lot of cut & paste, but we will delete the old one
next release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell 5cf2c2fbd7 lightningd: return both obsolete and modern blindedpaths from "blindedpath" RPC.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30