Commit Graph

207 Commits

Author SHA1 Message Date
Conor Scott 3d270fcca6 Skip README.md when reading in plugins dir 2019-01-15 19:40:12 +00:00
Rusty Russell 26dda57cc0 utils: make tal_arr_expand safer.
Christian and I both unwittingly used it in form:

	*tal_arr_expand(&x) = tal(x, ...)

Since '=' isn't a sequence point, the compiler can (and does!) cache
the value of x, handing it to tal *after* tal_arr_expand() moves it
due to tal_resize().

The new version is somewhat less convenient to use, but doesn't have
this problem, since the assignment is always evaluated after the
resize.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Christian Decker 5813567856 plugin: Dispatch notifications to subscribed plugins
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-30 14:36:02 +01:00
Christian Decker 37b2f907d4 plugin: Make plugin_send a more generic function
This used to be request-specific, but we now want to send
notifications and requests. As a drive-by we also clarify the
ownership of the json_stream instance that is being sent.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-30 14:36:02 +01:00
Christian Decker 7355e62964 plugin: Add subscriptions when processing the plugin manifest
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-30 14:36:02 +01:00
Richard Bondi 9ca07728ad fix crash on check with plugin command 2018-12-22 14:20:15 +01:00
Rusty Russell 68bb36b210 json-rpc: make commands return 'struct command_result *'.
Usually, this means they return 'command_param_failed()' if param()
fails, and changing 'command_success(); return;' to 'return
command_success()'.

Occasionally, it's more complex: there's a command_its_complicated()
for the case where we can't exactly determine what the status is,
but it should be considered a last resort.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell 12731c4a60 json_tok_len, json_tok_contents: rename to json_tok_full_len and json_tok_full
These are only supposed to be used when you want the token contents including
surrounding "".  We should use this when reporting errors, but usually
we just want to access the tok members directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell 22ca896b54 json: add and use a json_strdup() helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Christian Decker 2c53572798 plugin: Add missing context to tal_fmt call on error message
Seems the context parameter got lost somewhere.

Reported-by: Ulmo <@ulmo>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-16 04:40:06 +00:00
Christian Decker 968aeac908 plugin: Set LIGHTNINGD_PLUGIN env var inform plugins
It might be useful to take special precautions inside a plugin when
being run as a plugin (and not as a standalone executable). This env
var is just set so plugins can differentiate correctly. I don't unset
the variable since it shouldn't have any effect on `lightningd`
itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-15 15:04:32 +01:00
Christian Decker a304db9be2 plugin: Handle log notifications from plugins
Logs are parsed and injected into the main daemon's logs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-13 02:36:43 +00:00
Christian Decker dc25c43945 plugin: Split the parsing from the request handling in the plugin
This is a preparatory step for the next commit which adds notification
parsing as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-13 02:36:43 +00:00
Christian Decker 2834053457 plugin: Fix an issue with string IDs ending up quoted twice
The transparent passthrough that was recently introduced would end up
causing phantom quotes to appear around IDs when one of them was a
string. This happened for example when using `lightning-cli`, the code
would copy the quotes from the original request, insert our u64 ID,
and then re-add them on the way back as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-10 18:56:49 +00:00
Rusty Russell d7e233e47d Move json and param core functionality into common, for plugins.
json_escaped.[ch], param.[ch] and jsonrpc_errors.h move from lightningd/
to common/.  Tests moved too.

We add a new 'common/json_tok.[ch]' for the common parameter parsing
routines which a plugin might want, taking them out of
lightningd/json.c (which now only contains the lightningd-specific
ones).

The rest is mainly fixing up includes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 86c517ac9b common/json: add context arg to json_parse_input.
All callers currently just hand the same arg twice, but plugins might
want this different.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell b3d30095cb lightningd/plugin: simply patch requests through and don't interpret them (much).
We simply look for the id token, and substitute it on the way in/out.
We also need to make sure output is '\n\n' terminated.

I started this because we weren't forwarding complex errors properly
(we treated them as a string), but it's also a huge simplification.

`struct plugin_rpc_request` is eliminated entirely: the information we need
is actually inside `struct command` already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 8015e7dcfb jsonrpc: add the obj token to the callback.
This (will) avoid the plugin having to walk back from the params object
as it currently does.

No code changes; I removed UNUSED and UNNEEDED labels from the other
parameters though (as *every* json_rpc callback needs to call param()
these days, they're *always* used).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell f653723a85 plugin: log response given by plugin if it's invalid JSON.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell fbeef504f5 plugin: preserve stderr for plugins.
Now we've updated ccan/pipecmd, we can use pipecmd_preserve to
preserve stderr for plugins so we see their error spew.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 6da213be31 ccan: update to get updated pipecmd.
Note that this changes the order of arguments to pipecmd to match the
documentation, so we fix all the callers!

Also make configure re-run when configurator changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 6323cc1898 plugins: allow --dev-debugger=<pluginname>.
This currently just invokes GDB, but we could generalize it (though
pdb doesn't allow attaching to a running process, other python
debuggers seem to).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell b484933b40 plugin: simplify plugin dir test a little.
Check if it's a normal file first, then check permissions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Christian Decker be7674ed6c plugin: Added .params.configuration to init call
This tells the plugin both the `lightning-dir` as well as the
`rpc-filename` to use to talk to `lightningd`. Prior to this they'd
had to guess.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-07 17:17:53 +01:00
Christian Decker bd6ce102e6 plugin: Better cleanup when a plugin fails
This used to be a use-after-free bug in which we'd free the plugin and
then still have two connections that expect to be able to operate on
the plugin. This now signals the connections to exit and cleans up
once they do.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-05 23:15:59 +00:00
Christian Decker b23a33ec7a jsonrpc: Use tal_arr_remove instead of leaving NULL in the commands
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-05 23:15:59 +00:00
Christian Decker 230730eca4 plugin: Migrate request creation to json_stream
We can use the internal buffering of the json_stream instead of
manually building JSON-RPC calls. This makes it a lot easier to handle
these requests.

Notice that we do not flush concurrently and still buffer all the
things, but it avoids double-buffering things.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-05 23:15:59 +00:00
Christian Decker dc4fb650dc plugin: Add a timeout to the `getmanifest` call
If the plugin fails to respond to we may end up hanging indefinitely,
so we limit the time we're willing to wait to 10 seconds.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-05 23:15:59 +00:00
Christian Decker 83ecb61890 plugin: Ignore directories in the plugin-directory
They pass the executable test, but aren't really executable.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-05 23:15:59 +00:00
Rusty Russell 111d6df442 plugins: make log prefix the basename.
It's not perfect if they have multiple with same name, but better than number.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-05 01:22:55 +01:00
Rusty Russell eb03b33655 plugins: add and install built-in plugin dir, add clear and disable options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-05 01:22:55 +01:00
Rusty Russell a4287f99fd lightningd: add --plugin-dir option to load directory full of plugins.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-05 01:22:55 +01:00
Christian Decker d7e94a9386 plugin: Iterate over the options from a plugin using the tok->size
I had this really contorted way of iterating over options that could
cause valgrind to choke. This is the much more intuitive way to
iterate.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-02 22:55:47 +00:00
Christian Decker b8a1445317 plugin: Make plugin_kill a printf-like function
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker e625fd7e82 plugin: Map results back to the incoming JSON-RPC request
The final step in the JSON-RPC passthrough: map the result we got from
the plugin back to the original request we got from the client.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-02 22:55:47 +00:00
Christian Decker 7c93cf00bb plugin: Dispatch incoming RPC calls to appropriate plugin
Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker a0f6a82a0b plugin: Make memleak happy
List element structs must have the list_node as their first element.

Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker bf89d86ba8 plugin: Plugins need a list of methods they registered
This will be used in the next commit to dispatch calls to the correct
plugin.

Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker a71208b2a0 plugin: Remove added JSON-RPC methods if a plugin gets killed
Removes the method from the dispatch table, leaving a NULL entry
currently.

Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker 7de4c40b77 plugin: Add plugin rpcmethods to the JSON-RPC interface
Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker 3e1138951b plugin: Add pointer to jsonrpc so we can add new methods
Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker a4ded47d34 plugin: Fix memory leak when requests are done
We weren't cleaning the requests we fulfilled, so this does that :-)
2018-12-02 22:55:47 +00:00
arowser 3ba751797b add needed include file 2018-11-29 23:42:50 +00:00
Christian Decker da465f0fdd plugin: Exit if we fail to start a plugin
Signed-off-by: Christian Decker <@cdecker>
2018-11-26 22:53:37 +00:00
Christian Decker b02861bfe1 plugin: Give each plugin their own log-prefix
There is no good naming just yet, so we just number them 1 to n.
2018-11-13 00:44:50 +01:00
Christian Decker 74c58e9f25 docs: Add an initial draft of the plugin documentation
Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker 805a76a97f plugin: Make the plugins a list
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker 775e4b784d plugin: Get rid of redundant stdin and stdout members
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker e6ef675ea1 plugin: Send the configure request once we collected all options
This is the final step to get the plugins working. After parsing the
early options (including `--plugin`), then starting and asking the
plugins for options, and finally reading in the options we just
registered, we just need to assemble the options and send them over.

Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker 3abc32cbbc plugin: Terminate objects with an empty line to signal end
This is just meant as a hint to the plugin that the message is done,
and it can try to parse the buffer.

Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker 404352fc7e plugin: Configure plugins once we've collected all cli options 2018-11-13 00:44:50 +01:00
Christian Decker 55d6d6b0e7 plugin: Register plugin cli options
We also make `--help` a non-early arg so it allows for the plugins to
register their options before printing the help message. The options
themselves are stored in a separate struct inbetween them being
registered and them being forwarded to the plugin. Currently only
supports string options.

Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker 10338983a6 plugin: Add logs to plugin and add method to kill a plugin
Also includes some sanity checks for the results returned by the
plugin, such as ensuring that the ID is as expected and that we have
either an error or a real result.
2018-11-13 00:44:50 +01:00
Christian Decker 084224f134 plugin: Add request muxing to the plugin subsystem 2018-11-13 00:44:50 +01:00
Christian Decker d0de6e59c6 plugin: Start each plugin and setup the connection to it
Mostly copied from bitcoind.c
2018-11-13 00:44:50 +01:00
Christian Decker c71f4f3bd9 plugin: Add listconfigs stub for the --plugin option 2018-11-13 00:44:50 +01:00
Christian Decker b6a1735929 plugin: Basic scaffolding for the plugin subsystem 2018-11-13 00:44:50 +01:00