Commit Graph

5241 Commits

Author SHA1 Message Date
Rusty Russell 390117c9bb docs: document changes to waitsendpay command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell f8ecd08721 pay: don't list dummy channel if error is from final hop.
List the final one instead; if there's an error from the node it
may actually make sense to blame that channel (ie. previous node
did something wrong).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell 4e6b8e13a4 lightningd/pay: simplify code significantly.
We no longer need a 'sendpay_result' structure, we can pass
appropriate parameter directly now they're simple calls.

Every waitsendpay command ends in tell_waiters_failed or
tell_waiters_success, which call sendpay_success or sendpay_fail on
all matching waiters.  These all return 'struct command_result *'.

In cases where the result is immediately known, we call
sendpay_success/sendpay_fail directly for the command.

This also adds a helpful 'failcodename' field to the JSON output.

[ This was four separate cleanup patches, but that contained much
redundancy and was even worse to review ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell 1c58351551 lightningd: hardcode callbacks again.
With only one caller, we don't need a callback pointer any more; we can simply
call the function.

This required some code shuffling, and I changed the callback function
arguments to be in a more natural order, now they're not used as
callbacks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell a45a62aff6 lightningd: move pay internals back into pay.c
Now we don't have a second caller for these routines, we can move
them back into pay.c and make the functions static.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell afab1f7b3c gossipd: handle onion errors internally.
As a general rule, lightningd shouldn't parse user packets.  We move the
parsing into gossipd, and have it respond only to permanent failures.

Note that we should *not* unconditionally remove a channel on
WIRE_INVALID_ONION_HMAC, as this can be triggered (and we do!) by
feeding sendpay a route with an incorrect pubkey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell 4eddf57fd9 gossipd: don't mark channels unroutable.
For transient failures, the pay plugin should simply exclude those
from route considerations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell 018a3f1d58 short_channel_id: make mk_short_channel_id return a failure.
We had a bug 0ba547ee10 caused by
short_channel_id overflow.  If we'd caught this, we'd have terminated
the peer instead of crashing, so add appropriate checks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-21 12:31:06 +01:00
Rusty Russell d69680934e short_channel_id: catch parsing errors.
I upgraded my node with --disable-compat, and a heap of channels closed like:

	CHANNELD_NORMAL:We disagree on short_channel_ids: I have 557653x0x1351, you say 557653x2373x1", 

This is because the scids are strings in the databases, and it failed to parse
them properly.

Now we'll not start if that happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-21 12:31:06 +01:00
Rusty Russell ba8a9d1fde libplugin: quick fix for bad JSON produced by plugins on bad paramters.
Internally libplugin turns ' into ", which causes these messages to produce
bad JSON.

The real fix is to remove the '->" convenience substitution and port the
JSON creation APIs into common/ from lightningd/

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-21 11:20:11 +01:00
lisa neigut 28699f0eca option_data_loss_protect: reenable by default 2019-01-21 00:48:25 +00:00
Rusty Russell 4a45caae32 plugins: install pay plugin when 'make install'
Reported-by: ctrlbreak on IRC
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-20 14:57:29 +01:00
lisa neigut c45d034bc0 option_data_loss_protect: fixup commitment point check
Spurious errors were occuring around checking the provided
current commitment point from the peer on reconnect when
option_data_loss_protect is enabled. The problem was that
we were using an inaccurate measure to screen for which
commitment point to compare the peer's provided one to.

This fixes the problem with screening, plus makes our
data_loss test a teensy bit more robust.
2019-01-20 03:09:48 +00:00
Christian Decker c78d7e0f95 plugin: Increase manifest timeout to 60 seconds
Valgrind seems to be slowing the pay-plugin down enough for the 10
seconds timeout to get triggered on a semi-regular basis.

Reported-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-20 03:06:03 +00:00
Rusty Russell 82ff580a66 json: add more efficient iterators for objects and arrays.
Christian points out that we can iterate by ->size rather than calling
json_next() to find the end (which traverses the entire object!).

Now ->size is reliable (since previous patch), this is OK.

Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 16:22:32 +01:00
Rusty Russell 7b59e26dd7 jsmn: upgrade so JSMN_OBJECT's size parameter is usable.
jsmn would accept invalid JSON objects.  This is bad because it would
set ->size incorrectly: we expect to have at least size * 2 tokens (in
pairs).  We want to rely on ->size, but this would create an exploitable
buffer overflow!

Fortunately, this is fixed upstream, so we add a test and upgrade to v1.0.0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 16:22:32 +01:00
Rusty Russell be8006a4fc common/test/run-param: fix parsing test.
Wasn't using valid JSON, but worked anyway.  This is actually OK
because we don't rely on tok->size, but we want to, so another fix
coming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 16:22:32 +01:00
Rusty Russell 774973263c common/test/run-json: check tok->size is as expected.
The external/jsmn/README.md only says:
		int size;        // Number of child (nested) tokens

But it only counts *direct* children, or *direct* members for an object.

This test verifies this (the bug proved to be elsewhere: see next patch!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 16:22:32 +01:00
Rusty Russell 7c0863f785 plugins/pay: add comment on why we don't use an empty string
plugins/pay.c:879:7: error: zero-length gnu_printf format string [-Werror=format-zero-length]

Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 4122f955c1 plugins/pay: simplify listpeers_done code a little.
Avoid the unnecessary extra var, and don't use "capacity" since
that usually refers to static capacity.

Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell a393272ebf plugins/pay: clarify field names
Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 02436a8e6d libplugin: mention error field in error message.
Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell c83823a525 pytest: fix spurious valgrind output.
Had a couple of tests randomly fail because a valgrind error file was
not empty.  It contained:

   lightning_channeld: Writing out status 65520: Broken pipe

This shouldn't happen, but the simplest workaround is not to print
that (useless) error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 29b106720e plugins/pay: paystatus should explicitly describe why it is making each attempt.
So add a new 'strategy' field.  This makes it clearer what is going
on, currently one of:

* "Initial attempt"
* "Excluded channel <scid>"
* "Removed route hint"
* "Excluded expensive channel <scid>"
* "Excluded delaying channel <scid>"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 9d33a3d752 plugins/pay: eliminate worst channel if we go over fee / delay threshold.
But keep the error in this case, so we don't always report "no route".

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell e2777642c0 getroute: add direction to route returned.
We also ignore it in sendpay.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 0a8b4f8935 pay: remove inbuilt command in favor of plugin.
This doesn't actually remove some of the now-unnecessary infrastructure
though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 3f8dd7a95f plugins/pay: add paystatus command to get gory details of payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 5201c75d9d plugins/pay: store history of payment attempts (non-persisent).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 9403df8d0d plugins/pay: add shadow CLTV calculation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell b17c344b71 plugins/pay: retry when routehint fails.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell e3f893444d pytest: test more-than-one-hop route hints.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 2b72ebb3b6 plugins/pay: implement routeboost hints, naive version.
We sanitize the routes: firstly, we assume appending so eliminate the
first hop if the route points straight to us.  Secondly, eliminate empty
hints.  Thirdly, trim overlong hints.

Then we just use the first route hint.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell b787200752 pytest: test that we correctly use routeboost information from bolt11 invoice.
This is implemented in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 802b772cad plugins/pay: use final_cltc from bolt11 invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 6974c58b47 plugins/pay: get max-locktime-blocks at init for setting default.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell f85d7d03b8 plugins/pay: get our own id during init phase.
We'll want this for routeboost filtering.
2019-01-17 13:02:24 +01:00
Rusty Russell a8f2f28c72 plugins/pay: implement maxfeepercent, maxdelay and exemptfee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell c6fa993203 plugins/pay: exclude local channels with insufficient current capacity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 139604a618 pytest: test that pay command retries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell d9fa8a3536 plugins/pay: retry on failure in a loop.
We use the 'exclude' option to getroute for successive attempts.  This
is more robust than having gossipd disable for some limited time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 40637d0017 contrib/pylightning: temporarily convert to use plugin/pay for tests.
That this simply pay plugin passes the tests is a poor reflection on our
test cases, really.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell ad1e1bd528 plugins: minimal 'pay' plugin.
I wrote this sync first, then rewrote async, then developed libplugin.
But committing all that just wastes reviewer time, so I present it as
if it was always asnc and using the library helper.

Currently the command it registers is 'pay2', but when it's complete
we'll remove the internal 'pay' and rename it. This does a single
'getroute/sendpay' call.  No retries, no options.

Shockingly, this by itself is almost sufficient to pass our current test
suite with `pay`->`pay2`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell de4043a32a plugin/libplugin: API for C plugins.
Doesn't do logging yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Christian Decker ed356dae62 json: Add a helper to decode a hex-encoded value from JSON
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker 05ec56a968 plugin: Plugin hook callbacks need to be wrapped in a DB transaction
We therefore keep a reference to the DB and will wrap and unwrap when
a hook returns.

Notice that this might cause behavior changes when moving logic into a
hook callback, since the continuation runs in a different transaction
than the event that triggered the hook in the first place. Should not
matter too much, since we don't use DB rollbacks at the moment, but
it's something to keep in mind.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker a281f4b692 plugin: Send jsonrpc_request when calling hooks and dispatch results
This ties all the things together, using the serializer to transform
the payload into a valid `jsonrpc_request`, sending it to the plugin,
and then using the deserializer on the way back before calling the
hook callback with the appropriate information.

Notice that the serializer and deserializer is skipped if we don't
have a plugin that registered for this hook.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker a2fa0788fc plugin: Remove plugin_request_new and expose plugin_request_send
plugin_request_new did nothing special aside from registering the
request ID with the dispatch code. This duty has now been moved to
plugin_request_send instead, which is also exposed so we can use that
code in plugin_hook.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-17 05:42:49 +00:00
Christian Decker b8584a744b plugin: Add hooks that a plugin might register
This is the first use of the `hooks` autodata field, and it required a
dummy element in order for the section not to be dropped, it'll be
removed once we have actual hooks.
2019-01-17 05:42:49 +00:00
Christian Decker ff897f8788 jsonrpc: Generalize plugin_request to jsonrpc_request
There is very little that is plugin specific in the jsonrpc_request so
this just extracts the common parts so we can reuse them outside of
the plugin compilation unit as well.
2019-01-17 05:42:49 +00:00