Commit Graph

7815 Commits

Author SHA1 Message Date
Christian Decker 264b2d1975 pyln-testing: Add more meaningful error for mock_estimatefees
We only support a very limited number of argument combinations, and apparently
sometimes we trigger a case we aren't handling. This adds a more useful error
message, including the params we didn't match.
2020-04-20 07:48:32 +09:30
Christian Decker f1633de6e8 plugin: Add the keysend plugin to the list of installable plugins
Suggested-by: Matt Whitlock <@whitslack>
Signed-off-by: Christian Decker <@cdecker>
2020-04-20 07:29:07 +09:30
lisa neigut 9b42d9d8ba build: add mkrd to docker install? 2020-04-19 08:07:47 +09:30
lisa neigut 251bae55d7 build: Update build-release.sh to remove i386 build
fixup! docker: Remove Dockerfile for i386 builder
2020-04-19 08:07:47 +09:30
lisa neigut 9663d110d1 build: make non-parallel
There's a race condition with loading the submodules that's causes a
build failure on my machine, since the libwally 'includes' aren't on
disk yet when the gcc build step starts.
2020-04-19 08:07:47 +09:30
lisa neigut 1b2f4990ab tests: on reconnect, we need to send any fulfills before adds
The peer will fail the channel on reconnect if we send an 'add' we
don't have balance for yet; we can avoid this issue by always sending
fulfills (+) before sending adds (-)
2020-04-19 08:07:47 +09:30
lisa neigut 8189e9a64d v0.8.2-rc1 Changelog updates
Changelog-None
2020-04-19 08:07:47 +09:30
Rusty Russell bf053dcf7c plugins: simplify htlc_accepted hook payload-setting API.
As discussed with Christian, prepending the length to the payload returned
is awkward, but it's the only way to set a legacy payload.  As this will
be soon deprecated, simplify the external API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 18:03:35 +09:30
Christian Decker de0fd1f274 keysend: Change the plugin to add an invoice and then continue
So far we were relying on `lightningd` to create an ad-hoc invoice when
telling it to `resolve` with a given preimage. We now switch to having the
plugin create the invoice, remove the mandatory `keysend_preimage`
field (which would upset `lightningd` otherwise), and then return the modified
payload with the instructions to `continue` instead of resolving.

This ties back in with the existing payment/invoice handling code. Invoices
are created only if we don't have a label clash (unlikely since we have the
nano-time in the label), or the `payment_hash` was already used for another
invoice (at which point `lightningd` will automatically reject the payment and
we're a bit poorer for it, but meh :-)
2020-04-16 18:03:35 +09:30
Rusty Russell 63441075b5 lightningd: allow htlc_accepted hook to replace onion payload.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-added: `htlc_accepted` hook can now offer a replacement onion `payload`.
2020-04-16 18:03:35 +09:30
Christian Decker f3315ca110 pyln-proto: Add Tu32Field and Tu64Field helpers
These are necessary for the interim keysend plugin
2020-04-16 18:03:35 +09:30
Christian Decker 97fb27650d keysend: Announce keysend featurebit in node_announcement
The featurebit isn't quite settled.
2020-04-16 18:03:35 +09:30
Christian Decker e03acd9663 libplugin: Add features to plugin_main and getmanifest 2020-04-16 18:03:35 +09:30
Christian Decker 59567dc04b plugin: Move feature_place enum to features.h
Better not duplicate these, we might end up mixing them.
2020-04-16 18:03:35 +09:30
Christian Decker b5b11a3f67 keysend: Ignore keysends if the payload contains unknown even fields
We must really make sure that we understand the entire payload, not just the
fields we are interested in.
2020-04-16 18:03:35 +09:30
Christian Decker 568773daad pytest: Add a test for keysend
It currently uses a borrowed sending implementation from the noise plugin, but
we'll implement that functionality in the native keysend plugin next.
2020-04-16 18:03:35 +09:30
Christian Decker 1b32cc1c73 wire: Add a function to serialize a raw set of TLV fields
The generated wrappers will ignore the raw fields and will only consider the
shortcut fields. This function takes the raw fields and serializes them
instead.
2020-04-16 18:03:35 +09:30
Christian Decker ef0f28cd50 plugin: Implement the ability to receive keysend payments
This still uses the experimental TLV-type, but once the type is standardized
we can add detection for the new type quite easily.

Changelog-Added: pay: The `keysend` plugin implements the ability to receive spontaneous payments (keysend)
2020-04-16 18:03:35 +09:30
Christian Decker 5be59f4759 plugin: Add keysend skeleton 2020-04-16 18:03:35 +09:30
Christian Decker 5b11bab876 plugin: Check that the preimage returned as resolution matches hash
The plugin can basically return whatever it thinks the preimage is, but we
weren't handling the case in which it doesn't actually match the hash. If it
doesn't match now we just return an error claiming we don't have any matching
invoice.
2020-04-16 18:03:35 +09:30
Christian Decker 27ea47ae37 plugins: Fix undefined deallocation order in `struct plugins`
We use the new function `plugins_free` to define the correct deallocation
order on shutdown, since under normal operation the allocation tree is
organized to allow plugins to terminate and automatically free all dependent
resources. During shutdown the deallocation order is under-defined since
siblings may get freed in any order, but we implicitly rely on them staying
around.
2020-04-16 18:03:35 +09:30
Rusty Russell 284bd7de0c pytest: fix flaky test_excluded_adjacent_routehint
We weren't actually waiting until l3 got the channel_update from l2,
so it might not be able to create the routehint.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-15 20:52:13 -05:00
Rusty Russell deac09950a plugins: make chained hooks have two different callbacks.
One is called on every plugin return, and tells us whether to continue;
the other is only called if every plugin says ok.

This works for things like payload replacement, where we need to process
the results from each plugin, not just the final one!

We should probably turn everything into a chained callback next
release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 9aedb0c61f plugin: simplify hooks calling methods, and make lifetime requirements explicit.
They callback must take ownership of the payload (almost all do, but
now it's explicit).

And since the payload and cb_arg arguments to plugin_hook_call_() are
always identical, make them a single parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 1e34d8989d utils: add marker for functions which take ownership of pointers.
We have several of these, and they're not always called obvious things like
"delete" or "free".  `STEALS` provides a strong hint here.

I only added it to a couple I knew about off the top of my head.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 2c3543e42d lightningd: fix crash when failing htlc once channeld dies.
We were reaching through it to get `ld`, but channeld is already dead.
Caught this on test_onchaind_replay (without valgrind) on my test machine:

    INFO 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
    **BROKEN** lightningd: FATAL SIGNAL 11 (version f6e1735)
    **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x5634dc83dc55
    **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x5634dc83dca9
    **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fd4b7c0b46f
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:285 (failmsg_incorrect_or_unknown) 0x5634dc82625a
    **BROKEN** lightningd: backtrace: lightningd/htlc_set.c:109 (htlc_set_add) 0x5634dc801e5c
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:496 (handle_localpay) 0x5634dc826997
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1008 (htlc_accepted_hook_callback) 0x5634dc827e60
    **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:197 (plugin_hook_callback) 0x5634dc831ea1
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:261 (plugin_response_handle) 0x5634dc82d2c3
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:359 (plugin_read_json_one) 0x5634dc82d46f
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:391 (plugin_read_json) 0x5634dc82d5c6
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x5634dc896319
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x5634dc896efe
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x5634dc896f40
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:39:49 +09:30
Rusty Russell 1dc281c111 pytest: test for pay adjacent routehint crash.
When route returns a result which is too expensive, we try to figure out which
hop is most expensive to exclude it for next time.

If it's a single-hop route, we don't count it, since the first hop is free.
That's not usually a problem, since single-hop routes can't exceed our limits
(they're always "free"!).

But if we are using a routehint, the total cost could exceed our limits,
even if the start of the routehint is a single hop away.

This reproduces that test case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 14:13:26 -05:00
Rusty Russell df31af5221 invoice: ignore dead-end heuristic on explicitly specified channels.
This makes testing easier, and makes sense: lightningd might not
*know* about other connected channels, depending on gossip, but if the
user specifies it we should obey it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON: `invoice` `exposeprivatechannels` now includes explicitly named channels even if they seem like dead-ends.
2020-04-14 14:13:26 -05:00
Rusty Russell 1d29228136 lightningd: make sendpay support `blinding` and `enctlv` fields (EXPERIMENTAL)
This is what actually lets us pay blinded invoices.

Unfortunately, our internal logic assumes every hop in a path has a
next `short_channel_id`, so we have to use a dummy.  This is
sufficient for testing, however.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell 23f3a3c840 devtools/blindedpath: implement scid support.
This is what we expect in an enctlv used for htlcs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell a85d40fc5e common/onion: add blinding and enctlv encoding.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell 490a819402 lightningd: add `blinding` and `enctlv` field to `struct route_hop`.
This will be used when we want to specify these in a route.  But for now, they
only alter gossipd, which always sets them to NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell b29d1ed3ff channeld: support HTLCs with blinding (EXPERIMENTAL_FEATURES)
Note that it's channeld which calculates the shared secret, too.  This
minimizes the work that lightningd has to do, at cost of passing this
through.

We also don't yet save the blinding field(s) to the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell 539a73e1ec common/onion: EXPERIMENTAL handling of enctlv field to override next_short_channel_id.
This requires us to call ecdh() in the corner case where the blinding seed
is in the TLV itself (which is the case for the start of a blinded route).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell ae80787d1e EXPERIMENTAL: new error code for use when HTLC is blinded.
Everyone should return this for any HTLC error when they're handed a blinding
alongside the HTLC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell f33b39075b EXPERIMENTAL: onion TLV field for enctlv for blinding.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell 990a3ec92e EXPERIMENTAL: protocol TLV field for update_add_htlc to support blinding.
All NULL for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell a9fe1a3c08 features: add EXPERIMENTAL option_onion_messages from draft.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell bf2a42ada5 channeld: defer first update_fee until we have an HTLC to send.
Sending update_fee immediately after channel establishment seems to
upset LND, so work around it by deferring it.  The reason we increase
the fee after establishment is because now we might need to close the
channel in a hurry due to htlcs, but until there are htlcs that's
unnecessary.

Fixes: #3596
Changelog-Changed: Added workaround for lnd rejecting our commitment_signed when we send an update_fee after channel confirmed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-13 20:58:06 -05:00
Christian Decker 197a144505 plugin: Ensure RPC passthrough calls are terminated when plugin dies
We now track all pending RPC passthrough calls, and terminate them with an
error if the plugin dies.

Changelog-Fixed: JSON-RPC: Pending RPC method calls are now terminated if the handling plugin exits prematurely.
2020-04-14 11:07:55 +09:30
darosior 7ae8e21247 connectd: add darosior's seed node 2020-04-14 10:09:38 +09:30
darosior 11acd902ff pyln-testing: specify fallbackfee for newer versions of bitcoind
Changelog-None
2020-04-13 19:50:05 +09:30
darosior af76bce0fa pyln-client: send proper JSONRPC2 errors on failed call
Changelog-Added: pyln now sends proper error on bad calls to plugin methods

Co-Authored-By: Sergi Delgado Segura <sergi.delgado.s@gmail.com>
2020-04-11 12:17:51 +09:30
Christian Decker 813356b88c docs: Correct the key names for estimatefees 2020-04-11 08:43:06 +09:30
Christian Decker f00e9d6ea8 doc: Give an example of the supported DSN formats
I was told that DSNs are not trivial to format, so this adds an example to
help setup the postgres wallet backend.
2020-04-11 08:43:06 +09:30
Christian Decker 8370a627bb invoice: Parse amounts without a multiplier as BTC not msats
The spec states that invoices with an amount, but lacking a multiplier, should
be interpreted as integer Bitcoin amounts:

   `amount`: optional number in that currency, followed by an optional
   `multiplier` letter. The unit encoded here is the 'social' convention of a
   payment unit -- in the case of Bitcoin the unit is 'bitcoin' NOT satoshis.

Suggested-by: Stefano Pellegrini <@St333p>
Signed-off-by: Christian Decker <@cdecker>
Changelog-Fixed: invoice: The invoice parser assumed that an amount without a multiplier was denominated in msatoshi instead of bitcoins.
2020-04-11 08:43:06 +09:30
Christian Decker a696c8417e fundchannel: Remove the now disabled deprecated `satoshi` param
While we removed the `satoshi` param in #3603 it appears that the
`fundchannel` plugin was still passing it to the `fundchannel_start`
call. This fixes up the help text. Notice that technically the help text
changes the param name, but since it was internally always called `amount`
this change doesn't break the API, the help was just wrong.
2020-04-11 08:43:06 +09:30
Christian Decker 3b09662768 pyln: Allow non-empty directory when creating node in node_factory
So far we've always cleared the node directory when provisioning the node, but
while testing some plugins we noticed that pre-generating some files in that
directory is useful. This just adds yet another flag to `get_node` that
disables deleting any existing node directory.
2020-04-11 08:43:06 +09:30
Christian Decker b68066e8e8 python: Consolidate requirements.txt files in a single place
We had them split according the separate use-cases:

 - testing
 - doc-gen
 - wire-gen

But that was causing new contributors to miss some dependencies when they
first got hacking. So this consolidates all of our own dependencies in a root
requirements.txt, with the notable exception of `pyln-client`, `pyln-testing`
and `pyln-proto` which are distributed as PyPI modules and therefore have
their own dependencies that need to be tracked in the module root.

Closes #3518
2020-04-09 15:14:06 +02:00
darosior 25405191fb doc: document the plugin interface to gather Bitcoin data
Changelog-Added: plugins: the interface to gather data from Bitcoin backend plugins is now final.
2020-04-09 14:21:15 +09:30