Commit Graph

96 Commits

Author SHA1 Message Date
Christian Decker 23149c3daa plugin: Actually wait the 20 seconds promised in the docs
We promised we'd be waiting up to 20 seconds, but were only waiting for
10. Fix that by bumping to the documented 20.
2020-02-27 09:21:44 +10:30
Christian Decker 4a21883553 plugin: Fix hanging hook calls if the plugin dies
Changelog-Fixed: plugin: A crashing plugin will no longer cause a hook call to be delayed indefinitely
2020-02-27 09:21:44 +10:30
Christian Decker 72757933f0 pytest: Test a plugin crash while handling a hook call 2020-02-27 09:21:44 +10:30
darosior a47fd8cf3e pytest: test Bitcoin plugin registration and the bcli plugin 2020-02-12 11:45:07 +10:30
darosior e6a6ef23fd pytest: make the libplugin test non flaky
Rusty fixed the underlying bug in 8b8e4bb5fd
2020-02-12 09:30:02 +10:30
Christian Decker 490550d508 plugin: Allow multiple plugins to register the `htlc_accepted` hook
Make the `htlc_accepted` hook the first chained hook in our repertoire. The
plugins are called one after the other in order until we have no more plugins
or the HTLC was handled by one of the plugins. If no plugins handles the HTLC
we continue to handle it internally like always.

Handling in this case means the plugin returns either `{"result": "resolve",
...}` or `{"result": "fail", ...}`.

Changelog-Changed: plugin: Multiple plugins can now register for the htlc_accepted hook.
2020-02-11 15:57:22 +10:30
Rusty Russell 30580731a6 Minor fixups on PR #3477
Feedback from @niftynei and me; nothing major, but avoids
another round-trip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 13:53:31 +10:30
Christian Decker ebd3a8d7f2 plugin: Add featurebits registered by plugins to node_announcements
This is the last venue we need to add custom featurebits to, so we also unmark
the test as xfail.

Changelog-Added: plugin: Plugins can now signal support for experimental protocol extensions by registering featurebits for `node_announcement`s, the connection handshake, and for invoices. For now this is limited to non-dynamic plugins only
2020-02-11 13:53:31 +10:30
Christian Decker 966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
darosior 972b4def57 lightningd/plugin: unregister a plugin's options when stopping it
This also remove the now duplicate plugin_hook_unregister_all(), added
in the tal destructor of the struct plugin.
2020-02-10 09:49:15 +10:30
darosior 84e3fa39b3 pytest: test libplugin's send_outreq 2020-02-10 09:49:15 +10:30
darosior 75becf7227 pytest: test hooks and notifications with libplugin 2020-02-10 09:49:15 +10:30
darosior ceeb5503cc libplugin: fix 'dynamic' field in getmanifest
As a separated commit because it was pre-existent (changelog + xfail test).

This also fix a logical problem in lightningd/plugin_control: we were
assuming a plugin started with 'plugin start' but which did not comport
a 'dynamic' entry in its manifest to be dynamic, though it should have
been treated as static.

Changelog-fixed: plugins: Dynamic C plugins can now be managed when lightningd is up
2020-02-10 09:49:15 +10:30
darosior da476848d1 pytest: add a C testing plugin to test libplugin
We mark the test as xfail() as it exposes that libplugin's PLUGIN_RESTARTABLE
was not taken into account !
2020-02-10 09:49:15 +10:30
Rusty Russell 295ca2a436 lightningd: remove things we deprecated 6 months ago.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Relative plugin paths are not relative to startup (deprecated v0.7.2.1)
Changelog-Removed: Dummy fields in listforwards (deprecated v0.7.2.1)
2020-01-31 03:48:01 +00:00
darosior dc3aa33927 pytest: use pyln.client for functional tests 2020-01-21 22:38:14 +01:00
Christian Decker b800904409 pay: Move notify_sendpay_* calls out of the waiter loop
Changelog-Changed: plugin: `notify_sendpay_success` and `notify_sendpay_failure` are now always called, even if there is no command waiting on the result.
2020-01-13 23:34:46 +01:00
Christian Decker cc37862273 pytest: Reproduce issue #3403, sendpay notifications with no waiter
We clone the test above, but this time we don't attach waiters (they'd be racy
anyway), and we wait for the notification to be called. This fails, but is
fixed in the next two commits.
2020-01-13 23:34:46 +01:00
darosior 1cfb8425f5 plugin_control: more descriptive key for 'plugin stop' result
Changelog-Changed: JSONRPC: 'plugin stop' result is now accessible using the 'result' key instead of the empty ('') key.
2019-12-29 06:44:22 +08:00
Rusty Russell 839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Rusty Russell 30c8db148d lightningd: make sure rpc_command replacement is well-formed.
In particular:
1. It must redirect to an existing command.
2. It must contain method, params and id.

And update the docs to show the id, which is vital.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell e572b487d3 pytest: demonstrate failing rpc_command hook rewrite to plugin.
lightningd: lightningd/plugin.c:648: plugin_rpcmethod_dispatch: Assertion `idtok != NULL' failed.
lightningd: FATAL SIGNAL 6 (version v0.7.3-234-gf3aeb51-modded)
0x557652363eb1 send_backtrace
	common/daemon.c:41
0x557652363f5b crashdump
	common/daemon.c:54
0x7f5b065a346f ???
	???:0
0x7f5b065a33eb ???
	???:0
0x7f5b06582898 ???
	???:0
0x7f5b06582768 ???
	???:0
0x7f5b06594005 ???
	???:0
0x557652355901 plugin_rpcmethod_dispatch
	lightningd/plugin.c:648
0x557652331836 command_exec
	lightningd/jsonrpc.c:588
0x557652331bad rpc_command_hook_callback
	lightningd/jsonrpc.c:655
0x557652358745 plugin_hook_callback
	lightningd/plugin_hook.c:90
0x55765235486a plugin_response_handle
	lightningd/plugin.c:258
0x557652354a16 plugin_read_json_one
	lightningd/plugin.c:356
0x557652354b6d plugin_read_json
	lightningd/plugin.c:388
0x5576523b5e6c next_plan
	ccan/ccan/io/io.c:59
0x5576523b6a51 do_plan
	ccan/ccan/io/io.c:407
0x5576523b6a93 io_ready
	ccan/ccan/io/io.c:417
0x5576523b8d2b io_loop
	ccan/ccan/io/poll.c:445
0x55765232dc6f io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x557652334a6e main
	lightningd/lightningd.c:848
0x7f5b065841e2 ???
	???:0
0x5576523193ed ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20191206093336

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell f6ed7f2e89 plugin: handle corner case where rpc_command is to stop the plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
Rusty Russell d8fc5332c3 pytest: test rpc_command hook when it handles command to stop itself.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
Rusty Russell aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Christian Decker 5a233a1b27 pytest: Fix TLV hook tests in experimental mode
I missed this test breaking with `EXPERIMENTAL_FEATURES` in #3261
2019-11-22 03:48:54 +00:00
Christian Decker dda792c766 plugin: Pass the full raw_payload including realm to htlc_accepted
So far we've only handled legacy payloads, which meant we could drop the realm
byte since it was always 0x00. Once we start handling TLV payloads the first
byte, i.e., the former realm byte, is important since it gives us the length
of the payload. This is a breaking change, however I don't think there's
anyone using the `raw_payload` as of yet.

Changelog-Changed: JSON-RPC: the `raw_payload` now includes the first byte, i.e., the realm byte, of the payload as well. This allows correct decoding of a TLV payload in the plugins.
2019-11-18 12:00:15 +01:00
Rusty Russell 709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell 2a2259083a lightningd: handle tlv-style payloads.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: JSON API: `htlc_accepted` hook has `type` (currently `legacy` or `tlv`) and other fields directly inside `onion`.
Changelog-deprecated: JSON API: `htlc_accepted` hook `per_hop_v0` object deprecated, as is `short_channel_id` for the final hop.
2019-11-14 10:15:33 +01:00
darosior 95c6513ff3 pytest: test the 'rpc_command' hook 2019-11-11 22:30:01 +01:00
darosior 7dac7a8c52 pytest: deactivate shadow routing for some tests that use 'pay'
So that we can assert payment values
2019-11-08 03:27:58 +00:00
darosior f3d304901b pay: add a dev-only parameter to deactivate shadow routing
Had to make a special pylightning function to avoid rewriting all calls
to 'pay()' with 'rpc.call()' in the next commit..
2019-11-08 03:27:58 +00:00
darosior c0ae3cb8c4 pytest: make test_plugin_slowinit a test again 2019-10-11 10:05:08 -05:00
darosior 32d98f0a87 plugins: refactor the tests for dynamic plugins
This adapts the test to the new 'plugin' command: no more sleeping,
since we are synchronous !

This tests the timeout by increasing the 'slowinit' plugin sleep
duration at init reception.

This adds a broken plugin to make sure we won't crash because of a
misbehaving plugin (unmet dependency is the most common case).
2019-09-30 00:20:16 +00:00
Christian Decker efc4aa94a3 pytest: Skip some tests that assume we have a sqlite3 db on postgres
These will not work since they touch the DB file itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
trueptolemy a3a0f651e3 pytest: Let `fail_htlcs.py` plugin generates `TEMPORARY_NODE_FAILURE` error 2019-09-16 12:22:06 +08:00
trueptolemy 619362ae19 pytest: Add a test for 'sendpay_success' and 'sendpay_failure' 2019-09-11 00:57:39 +00:00
Christian Decker 492d77f213 db: Add setup and teardown function to DB
These are used to do one-time initializations and wait for pending statements
before closing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Rusty Russell 049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell f0ab4d72da pytest: add test that htlc_accepted hook only gets called after init.
We fail this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell 20a2bf9547 pytest: make test_forward_event_notification more explicit.
We seem to be getting intermittant failures, but it's hard
to disgnose.  Simplify it by moving all the test logic into
the test itself, and making the plugin dumber.  This means we'll
see exactly what the differences are if it fails again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Simon Vrouwe 5b5d70d640 lightningd/plugin: extend plugins->startup window to include plugins_config
Fixes incorrect configuration[`startup`] in plugin `init`, modified tests
to test this.
2019-08-27 00:02:20 +00:00
Rusty Russell 63103db3f3 pytest: disable some v. slow tests when !DEVELOPER.
We've gone over 50 minutes with valgrind, and my measurements show
these are the slowest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-14 05:53:36 +00:00
Rusty Russell 118150227e cli: restore 0.7.0-style whitespace printing.
@renepickhardt has a shell script we broke.  While we still produce
perfectly valid JSON, we should not gratuitously change tool output.

Plus, I prefer the missing space before the ':'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:16:48 +08:00
Christian Decker e2f9a410f4 pytest: Run plugin --help tests in the test directory
This is a followup to #2892. Since we now attempt to lock the PID file before
starting plugins we need to make sure that we actually use a unique lightning
directory for anything that attempts to call `--help`. If not we may be
conflicting with a `lightningd` that is running against that directory.

Notice that this still means that we will be unable to call `--help` on
`lightningd` if we have a running instance, but isolation in this case is
good, otherwise we'd be reading the default config anyway.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
Rusty Russell b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell 4fcd94a091 pytest: use absolute paths for plugin arguments.
We're going to change this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00