Commit Graph

4825 Commits

Author SHA1 Message Date
Rusty Russell b522b35e01 connectd: rename peer_reconnected local var for clarity.
We call it 'pr' in the retry_peer_connected callback, so call it that
here too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 5b7ef7ab49 connectd: don't leak init message.
It's a very bounded leak, since we can only have one and it's
connected to the peer lifetime, but we don't need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell ab735dcbe6 gossipd: wire up memleak detection.
For simplicity we dump leaks to logs, and just return a bool to master.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 78771ca371 gossipd: mark timers as not being leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 2debbb0a00 common/subdaemon: helpers for dumping memleaks in subdaemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 5a81dbd783 common/daemon: enable/cleanup memleak in daemon_setup / daemon_shutdown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 22526911cb tools/generate-wire.py: allocate array members off array in fromwire.
If we have an array of varlen structures (which require a ctx arg), we
should make that arg the array itself (which was tal_arr()), not the
root context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 29b672b117 gossipd: hear no wumbo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell bed7986650 channeld: speak no wumbo.
We cap our htlc_maximum_msat at 2^32-1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell 5a12af817a channeld: don't keep local copy of config.
We keep it in struct channel already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell 22858f35f9 struct channel: keep a copy of configs, not just pointers.
This simplifies lifetime assumptions.  Currently all callers keep the
original around, but everything broke when I changed that in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell b5a96765d8 struct channel: remove most helpers.
They were not universally used, and most are trivial accessors anyway.

The exception is getting the channel reserve: we have to multiply by 1000
as well as flip direction, so keep that one.

The BOLT quotes move to `struct channel_config`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell 9620393109 gossipd: store chainparams internally.
We keep a chain_hash in struct daemon, becayse otherwise we end up with
`&peer->daemon->rstate->chainparams->genesis_blockhash` which is a bit
ridiculous.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 21:43:37 +00:00
Rusty Russell 9088b19ed9 CHANGELOG: update to mention new getinfo fields.
Fixes: 3f420dc408
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 12:57:11 +01:00
Rusty Russell 56fd2541c3 doc: make it clearer that specifying addr etc disables autolisten.
Reported-by: Greg Sanders
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 03:09:03 +00:00
Jon Griffiths 47743a287c Avoid a sha256/ripemd160 call for each pubkey we derive
We only take the pubkey and ignore all other fields, so we might as well
save the cycles used computing the hash for something else.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2018-11-21 01:18:24 +00:00
Jon Griffiths 087ef43a0d Call wally_init(0) on startup to enabled assembly-optimised SHA-256
We probably also want to call secp_randomise/wally_secp_randomize here
too, and since these calls all call setup_tmpctx, it probably makes
sense to have a helper function to do all that. Until thats done, I
modified the tests so grepping will show the places where the sequence
of calls is repeated.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2018-11-21 01:18:24 +00:00
Jon Griffiths d3bfc9f260 libwally: Update to latest master
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2018-11-21 01:18:24 +00:00
Rusty Russell 2c582cbcfb pytest: test that we don't revert config handling again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 01:08:42 +00:00
Rusty Russell 6daa599c74 options: plug leak (triggered by next test)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 01:08:42 +00:00
arowser 56a3b3f3df read config file before change working dir 2018-11-21 01:08:42 +00:00
Conor Scott 795bc1b3a7 [test] update test mocks for getinfo rpc update 2018-11-21 00:37:04 +00:00
Conor Scott 3f420dc408 [rpc] Add peer stats to getinfo rpc 2018-11-21 00:37:04 +00:00
Conor Scott dd27205ba2 [rpc] move getinfo implementation to peer_control.c 2018-11-21 00:37:04 +00:00
Rusty Russell 5312ec1e34 gossipd: add documentation comments now it's relatively understandable.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell ea2c03e2e2 gossipd: don't have code to exit final loop; we always leave via master_gone.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 4038061d0f gossipd: use take() in getroute_req.
Trivial optimization.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 5c60d7ffb2 gossipd: split wire types into msgs from lightningd and msgs from per-peer daemons
This avoids some very ugly switch() statements which mixed the two,
but we also take the chance to rename 'towire_gossip_' to
'towire_gossipd_' for those inter-daemon messages; they're messages to
gossipd, not gossip messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 07b16e37d0 daemon_conn: don't rely on outq_empty callback telling us to retry queue.
We had at least one bug caused by it not returning true when it had
queued something.  Instead, just re-check thq queue after it's called.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 4e9eba1965 gossipd: rework query_channel_range to accept overlapping range.
We shouldn't insist on an exact reponse match: they can batch it and send
a whole batch, as long as it overlaps what we ask.

We also change to a bitmap to save some memory.

This isn't note in the CHANGELOG since we don't actually send gossip
range queries except for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell fac68630b8 ccan: add ccan/bitmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 363564301f gossipd: be more rigorous in handling peer messages vs. daemon requests.
Messages from a peer may be invalid in many ways: we send an error
packet in that case.  Rather than internally calling peer_error,
however, we make it explicit by having the handle_ functions return
NULL or an error packet.

Messages from the daemon itself should not be invalid: we log an error
and close the fd to them if it is.  Previously we logged an error but
didn't kill them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 1bd76861fd gossipd: reorder functions into related groups (MOVEONLY)
It's MOVEONLY but for the removal of the '#ifndef TESTING' which was
needed for old test code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell debbdc0781 bolt11: accept lightning: prefix.
The Blockstream store produces these, for example, so let's ignore them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 21:02:31 +01:00
Rusty Russell 8a246e2c0a jsonrpc: provide overview of how this all connects together.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell 6f7650e471 pytest: test using malformed JSONRPC commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell 7799368144 pytest: add test for multiplexed RPC output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell 3f5487e247 jsonrpc: dev_slowcmd, a command which starts output then delays.
This lets us explicitly test that our JSON outputs don't intermingle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell e0d14bddb9 jsonrpc: allow multiple commands at once.
We now keep multiple commands for a json_connection, and an array of
json_streams.

When a command wants to write something, we allocate a new json_stream
at the end of the array.

We always output from the first available json_stream; once that
command has finished, we free that and move to the next.  Once all are
done, we wake the reader.

This means we won't read a new command if output is still pending, but
as most commands don't start writing until they're ready to write
everything, we still get command parallelism.

In particular, you can now 'waitinvoice' and 'delinvoice' and it will
work even though the 'waitinvoice' blocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell f9fd802147 jsonrpc: make struct json_connection definition private.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell 47d2a71ef0 json.c and jsonrpc.c: move functions between them.
json_stream_success / json_stream_fail belong in jsonrpc.c, and the
json_tok helpers for special types belong in json.x

json_add_object() isn't used, remove it rather than moving it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell e17f69ce2d json_stream: disentangle JSON handling from command.
We promote 'struct json_stream' to contain the membuf; we only attach
the json_stream to the command when we actually call
json_stream_success / json_stream_fail.

This means we are closer to 'struct json_stream' being an independent
layer; the tests are already modified to use it directly to create
JSON.

This is also the first step toward re-enabling non-serial command
execution.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell db3f0ba965 CHANGELOG: note that pylightning uses double-newline.
Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell b9b7411d88 lightning.py: use double-\n as marker for RPC parsing.
This doesn't make a performance difference, but even better, it
simplifies the code.

We hacked test_multirpc to send 200x as many commands, and timed the
pytest over 20 runs:

Before:
=================== 1 passed, 136 deselected in 8.550000-9.400000(9.0045+/-0.2) seconds ===================

After:
=================== 1 passed, 136 deselected in 8.540000-9.370000(8.97286+/-0.16) seconds ===================

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-19 21:36:40 +01:00
Rusty Russell b2378654d7 jsonrpc: add double '\n' to end of JSON RPC commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-19 21:36:40 +01:00
Rusty Russell 0c3f85d931 lightning.py: parse multiple JSON RPC commands accurately.
We need to keep the remaining buffer, and we need to try to parse it
before we read the next.  I first tried keeping it in the object, but
its lifetime is that of the *socket*, which we actually reopen for
every command.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-19 21:36:40 +01:00
Rusty Russell fe11ee5406 pytest: make test_multirpc check that RPC commands complete.
This was hanging sometimes in travis, but actually checking the result
of the commands makes it *always* hang.  We remove the waitinvoice
which will not return.

ZmnSCPxj points out that this behavior, introduced in
ce0bd7abd3, is a regression: it would be
nice to be able to cancel a waitinvoice.  But that fix is more complex,
and will have to be another PR.

This test will now hang, but it's OK: we're about to fix it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-19 21:36:40 +01:00
Conor Scott 9a0f3a764c [pylightning] Fix lightning-pay script 2018-11-19 21:24:03 +01:00
arowser d5aaa11373 update match network log 2018-11-15 20:19:34 +01:00
Conor Scott 189fca4039 [doc] Remove old GOSSIPING state from docs 2018-11-15 11:29:04 +01:00