This avoids some very ugly switch() statements which mixed the two,
but we also take the chance to rename 'towire_gossip_' to
'towire_gossipd_' for those inter-daemon messages; they're messages to
gossipd, not gossip messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We now keep multiple commands for a json_connection, and an array of
json_streams.
When a command wants to write something, we allocate a new json_stream
at the end of the array.
We always output from the first available json_stream; once that
command has finished, we free that and move to the next. Once all are
done, we wake the reader.
This means we won't read a new command if output is still pending, but
as most commands don't start writing until they're ready to write
everything, we still get command parallelism.
In particular, you can now 'waitinvoice' and 'delinvoice' and it will
work even though the 'waitinvoice' blocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
json_stream_success / json_stream_fail belong in jsonrpc.c, and the
json_tok helpers for special types belong in json.x
json_add_object() isn't used, remove it rather than moving it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We promote 'struct json_stream' to contain the membuf; we only attach
the json_stream to the command when we actually call
json_stream_success / json_stream_fail.
This means we are closer to 'struct json_stream' being an independent
layer; the tests are already modified to use it directly to create
JSON.
This is also the first step toward re-enabling non-serial command
execution.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is the final step to get the plugins working. After parsing the
early options (including `--plugin`), then starting and asking the
plugins for options, and finally reading in the options we just
registered, we just need to assemble the options and send them over.
Signed-off-by: Christian Decker <@cdecker>
We also make `--help` a non-early arg so it allows for the plugins to
register their options before printing the help message. The options
themselves are stored in a separate struct inbetween them being
registered and them being forwarded to the plugin. Currently only
supports string options.
Signed-off-by: Christian Decker <@cdecker>
Also includes some sanity checks for the results returned by the
plugin, such as ensuring that the ID is as expected and that we have
either an error or a real result.
The idea is that `plugin` is an early arg that is parsed (from command
line or the config file). We can then start the plugins and have them
tell us about the options they'd like to add to the mix, before we
actually parse them.
Signed-off-by: Christian Decker <@cdecker>
The spec says so, and it's right: with the right pattern of packet loss
(thanks Travis!) the other end can still be in channeld, waiting for our
`shutdown` message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This way there's no need for a context pointer, and freeing a msg_queue
frees its contents, as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When developing in regtest or testnet it is really inconvenient to
have to fake traffic and generate blocks just to get estimatesmartfee
to return a valid estimate. This just sets the minfee if bitcoind
doesn't return a valid estimate.
Reported-by: Rene Pickhardt <@renepickhardt>
Signed-off-by: Christian Decker <@cdecker>
The only change is that the final_incorrect_htlc_amount field is now 64
bit. Since no implementation yet parses that field, we just updated it
quietly in the spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We always have an addr entry in the db (though it may be an ephemeral socket
the peer connected in from). We don't have to handle a NULL address.
While we're there, simplify new_peer not to take the features args;
the caller who cares immediately updates the features anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In e46ce0fc84 I accidentally removed the
actual code which fails the command. As a result, if we retry and it
succeeds later, we can end up "succeeding" the started-failing
command, causing us to hit the 'assert(!cmd->have_json_stream);' in
new_json_stream.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If there are two HTLCs with the same preimage, lightningd would always
find the first one. By including the id in the `struct htlc_stub`
it's both faster (normal HTLC lookup) and allows lightningd to detect
that onchaind wants to fail both of them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We spend quite a bit of time in libsecp256k1 moving them to and from
DER encoding. With a bit of care, we can transfer the raw bytes from
gossipd and manually decode them so a malformed one can't make us
abort().
Before:
real 0m0.629000-0.695000(0.64985+/-0.019)s
After:
real 0m0.359000-0.433000(0.37645+/-0.023)s
At this point, the main issues are 11% of time spent in ccan/io's
backend_wake (I tried using a hash table there, but that actually makes
the small-number-of-fds case slower), and 65% of gossipd's time is
in marshalling the response (all those tal_resize add up!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
My test case is a mainnet gossip store with 22107 channels, and
time to do `lightning-cli listchannels`:
Before: `lightning-cli listchannels` DEVELOPER=0
real 0m1.303000-1.324000(1.3114+/-0.0091)s
After:
real 0m0.629000-0.695000(0.64985+/-0.019)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's a very ugly one-liner; really ccan/io should have an io_replan
for this, but it would have to be written carefully as it makes
assumptions currently about plans not changing. In this case, we know
it's in io_write, and we're just moving a pointer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Such an API is required for when we stream it directly. Almost all our
handlers fit this pattern already, or nearly do.
We remove new_json_result() in favor of explicit json_stream_success()
and json_stream_fail(), but still allowing command_fail() if you just
want a simple all-in-one fail wrapper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This isn't a big change, since we basically dump the entire JSON
resuly string into the membuf then write it out, but it's prep for the
next changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
arowser
Conor Scott
Christian Decker