Commit Graph

5422 Commits

Author SHA1 Message Date
Rusty Russell 4122f955c1 plugins/pay: simplify listpeers_done code a little.
Avoid the unnecessary extra var, and don't use "capacity" since
that usually refers to static capacity.

Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell a393272ebf plugins/pay: clarify field names
Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 02436a8e6d libplugin: mention error field in error message.
Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell c83823a525 pytest: fix spurious valgrind output.
Had a couple of tests randomly fail because a valgrind error file was
not empty.  It contained:

   lightning_channeld: Writing out status 65520: Broken pipe

This shouldn't happen, but the simplest workaround is not to print
that (useless) error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 29b106720e plugins/pay: paystatus should explicitly describe why it is making each attempt.
So add a new 'strategy' field.  This makes it clearer what is going
on, currently one of:

* "Initial attempt"
* "Excluded channel <scid>"
* "Removed route hint"
* "Excluded expensive channel <scid>"
* "Excluded delaying channel <scid>"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 9d33a3d752 plugins/pay: eliminate worst channel if we go over fee / delay threshold.
But keep the error in this case, so we don't always report "no route".

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell e2777642c0 getroute: add direction to route returned.
We also ignore it in sendpay.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 0a8b4f8935 pay: remove inbuilt command in favor of plugin.
This doesn't actually remove some of the now-unnecessary infrastructure
though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 3f8dd7a95f plugins/pay: add paystatus command to get gory details of payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 5201c75d9d plugins/pay: store history of payment attempts (non-persisent).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 9403df8d0d plugins/pay: add shadow CLTV calculation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell b17c344b71 plugins/pay: retry when routehint fails.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell e3f893444d pytest: test more-than-one-hop route hints.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 2b72ebb3b6 plugins/pay: implement routeboost hints, naive version.
We sanitize the routes: firstly, we assume appending so eliminate the
first hop if the route points straight to us.  Secondly, eliminate empty
hints.  Thirdly, trim overlong hints.

Then we just use the first route hint.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell b787200752 pytest: test that we correctly use routeboost information from bolt11 invoice.
This is implemented in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 802b772cad plugins/pay: use final_cltc from bolt11 invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 6974c58b47 plugins/pay: get max-locktime-blocks at init for setting default.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell f85d7d03b8 plugins/pay: get our own id during init phase.
We'll want this for routeboost filtering.
2019-01-17 13:02:24 +01:00
Rusty Russell a8f2f28c72 plugins/pay: implement maxfeepercent, maxdelay and exemptfee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell c6fa993203 plugins/pay: exclude local channels with insufficient current capacity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 139604a618 pytest: test that pay command retries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell d9fa8a3536 plugins/pay: retry on failure in a loop.
We use the 'exclude' option to getroute for successive attempts.  This
is more robust than having gossipd disable for some limited time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 40637d0017 contrib/pylightning: temporarily convert to use plugin/pay for tests.
That this simply pay plugin passes the tests is a poor reflection on our
test cases, really.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell ad1e1bd528 plugins: minimal 'pay' plugin.
I wrote this sync first, then rewrote async, then developed libplugin.
But committing all that just wastes reviewer time, so I present it as
if it was always asnc and using the library helper.

Currently the command it registers is 'pay2', but when it's complete
we'll remove the internal 'pay' and rename it. This does a single
'getroute/sendpay' call.  No retries, no options.

Shockingly, this by itself is almost sufficient to pass our current test
suite with `pay`->`pay2`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell de4043a32a plugin/libplugin: API for C plugins.
Doesn't do logging yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Christian Decker ed356dae62 json: Add a helper to decode a hex-encoded value from JSON
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker 05ec56a968 plugin: Plugin hook callbacks need to be wrapped in a DB transaction
We therefore keep a reference to the DB and will wrap and unwrap when
a hook returns.

Notice that this might cause behavior changes when moving logic into a
hook callback, since the continuation runs in a different transaction
than the event that triggered the hook in the first place. Should not
matter too much, since we don't use DB rollbacks at the moment, but
it's something to keep in mind.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker a281f4b692 plugin: Send jsonrpc_request when calling hooks and dispatch results
This ties all the things together, using the serializer to transform
the payload into a valid `jsonrpc_request`, sending it to the plugin,
and then using the deserializer on the way back before calling the
hook callback with the appropriate information.

Notice that the serializer and deserializer is skipped if we don't
have a plugin that registered for this hook.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker a2fa0788fc plugin: Remove plugin_request_new and expose plugin_request_send
plugin_request_new did nothing special aside from registering the
request ID with the dispatch code. This duty has now been moved to
plugin_request_send instead, which is also exposed so we can use that
code in plugin_hook.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker b8584a744b plugin: Add hooks that a plugin might register
This is the first use of the `hooks` autodata field, and it required a
dummy element in order for the section not to be dropped, it'll be
removed once we have actual hooks.
2019-01-17 05:42:49 +00:00
Christian Decker ff897f8788 jsonrpc: Generalize plugin_request to jsonrpc_request
There is very little that is plugin specific in the jsonrpc_request so
this just extracts the common parts so we can reuse them outside of
the plugin compilation unit as well.
2019-01-17 05:42:49 +00:00
Christian Decker 1a952667ee plugin: Remove plugin_request argument from callbacks
None of the existing callbacks was making use of it and we will be
exposing the method callback interface to outside compilation unit
where the struct definition is not visible. So just remove it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker f8f76e3d31 plugin: Add hook registration
I might have gone a bit overboard with the type-checking, but
typesafe_cb_cast is quite nice to use, so why not. The macro to
register a new hook encapsulates the entire flow from param
serialization, to dispatch, parsing and callback dispatch in one
bundle. I was tempted to have the callback outside of the
registration, but it's unlikely that we'll have two calls to the same
hook with different callbacks.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker 025fbbc9ea pylightning: Always assign the `init` method in plugin
Also fixes a crash if the doc was empty.
2019-01-17 05:42:49 +00:00
Christian Decker a707ae091d pylightning: Add hooks as a new type of method
Instead of creating a new map I opted to re-use the Plugin.methods
map, since the semantics are really similar and we don't allow
duplicates. The only difference is in how they are announced to
lightningd, so we use an enum to differentiate rpcmethods from hooks,
since only the former will get added to the JSON-RPC dispatch table in
lightningd.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker 71934f983a notifications: Fix iteration over notification topics
Turns out that checking boundaries is important...

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Conor Scott 5fd17602d0 Update CHANGELOG to reflect update to listpeers RPC 2019-01-16 11:50:52 -08:00
Conor Scott 0535cbbc96 [rpc] Add funding allocation to listpeers command 2019-01-16 11:50:52 -08:00
Conor Scott 3d270fcca6 Skip README.md when reading in plugins dir 2019-01-15 19:40:12 +00:00
nicolas.dorier a565915d08 bitcoind: allow "getblock" to fail for txout lookup.
Apparently on pruned nodes it sometimes gives exit status 1?
2019-01-15 19:39:15 +00:00
Rusty Russell 0ba547ee10 gossipd: handle overflowing query properly (avoid slow 100% CPU reports)
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  #14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  #15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  #16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  #17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  #18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  #19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  #20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  #21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  #22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  #23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  #24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  #25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  #26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  #27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  #28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  #29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  #30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  #31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  #32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  #33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  #34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  #35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  #36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell ba41d6e3df pytest: failing test for overflow in query_channel_range
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell 52750f2dcc pytest: tighten the query_channel_range test.
Make the two channels adjacent, and specify exactly the number of
divide-and-conquer steps there are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell 4c57bf002e pytest: neaten fund_channel scid extraction.
We used to have a bug where decoderawtransaction would fail, fixed in
fedcfd661 (pytest: hand 'True' to decoderawtransaction so it doesn't
get confused.).

So we can remove the fallback decode, and might as well extract the
ugliness into a helper function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell a00c357854 JSON: remove redundant word "channel" from direction fields.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell 9f1f79587e short_channel_id_dir: new primitive for one direction of short_channel_id
Currently only used by gossipd for channel elimination.

Also print them in canonical form (/[01]), so tests need to be
changed.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell 80753bfbd5 Feedback from @niftynei.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell dc2ee9639b listchannels: allow source arg to list channels by their source node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell de682f5806 json_getroute: don't return generic error.
We use the PAY error code here, but it's appropriate (otherwise the
pay command simply has to substitute it, which seems silly).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell 5b3abd80b1 pytest: don't skip valgrinding plugins.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00