Commit Graph

6003 Commits

Author SHA1 Message Date
lisa neigut b0b813a171 test: add initial tests for starting an external fundchannel
Test for getting through the address generation portion.
2019-06-12 02:22:43 +00:00
lisa neigut dd11d3bd81 lightningd: add start for fundchannel_continue
Add an RPC method (not working at the moment) called
`fundchannel_continue` that takes as its parameters a
node_id and a txid for a transaction (that ostensibly has an output
for a channel)
2019-06-12 02:22:43 +00:00
lisa neigut 52042bde80 opening: stash amount outside of the wtx
Some channels won't be opened with a wtx struct, so keep
the total funding amount separate from it so we can
show some stats for listpeers.

Note that we're going to need to update/confirm this once
the transaction gets confirmed.
2019-06-12 02:22:43 +00:00
lisa neigut 6e5928aa1e openingd: update billboard with funding info
Make it easier for a user to know what's going on with
a channel in `listpeers`.
2019-06-12 02:22:43 +00:00
lisa neigut 23e7846053 pylightning: add fundchannel_start
Add method to rpc handler
2019-06-12 02:22:43 +00:00
lisa neigut 5920e656cf opening: wire up walking through open channel up thru accept
Fill in details to make fundchannel_start work.
2019-06-12 02:22:43 +00:00
lisa neigut 685aa47b65 openingd: pull out setup funder checks into separate method
Useful for adding the funder_start stuff
2019-06-12 02:22:43 +00:00
lisa neigut 8103acd9b3 opening: add entry point for `funding_start rpc command
Beginnings of wiring up the funding_start rpc command. missing
the part that actually starts the funding channel dance.
2019-06-12 02:22:43 +00:00
lisa neigut 846bc9cbc4 openingd: allow funding_failed to complete successfully
For the `fundchannel_cancel` we're going to want
to 'successfully' fail a funding channel operation. This allows
us to report it a failure back as an RPC success, instead of
automatically failing the RPC request.
2019-06-12 02:22:43 +00:00
lisa neigut 1b2a593b05 tests: default all addresses to bech32
Needed for composing a transaction externally to c-lightning, using
bitcoind util.
2019-06-12 02:22:43 +00:00
lisa neigut eb0b3d7b09 json: correct error message for param parser
Looks like copy-paste from another commit didn't update
the field for this
2019-06-12 02:22:43 +00:00
lisa neigut a9d0550cf4 common: pull out scriptPubkey address method
We're going to need this for P2WSH scripts. pull it out into
a common file plus adopt the sanity checks so that it will allow for
either P2WSH or P2WPKH (previously only encoded P2WPKH scripts)
2019-06-12 02:22:43 +00:00
Rusty Russell 5a520f4a07 plugin: don't call notification after free.
This is an old bug, where a plugin can get called while we're shutting
down (and have freed plugins), but it's triggered more reliably by the
new warning notification hook.

For good measure, we also make freeing a plugin self-delete.

Valgrind error file: valgrind-errors.16763
==16886== Invalid read of size 8
==16886==    at 0x422919: plugins_notify (plugin.c:1096)
==16886==    by 0x413919: notify_warning (notification.c:61)
==16886==    by 0x412BDE: logv (log.c:251)
==16886==    by 0x412A98: log_ (log.c:311)
==16886==    by 0x4044BE: bcli_finished (bitcoind.c:178)
==16886==    by 0x459480: destroy_conn (poll.c:244)
==16886==    by 0x459499: destroy_conn_close_fd (poll.c:250)
==16886==    by 0x4619E1: notify (tal.c:235)
==16886==    by 0x461A7E: del_tree (tal.c:397)
==16886==    by 0x461AB5: del_tree (tal.c:407)
==16886==    by 0x461AB5: del_tree (tal.c:407)
==16886==    by 0x461AB5: del_tree (tal.c:407)
==16886==  Address 0x634a578 is 40 bytes inside a block of size 352 free'd
==16886==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16886==    by 0x461AFD: del_tree (tal.c:416)
==16886==    by 0x461FB7: tal_free (tal.c:481)
==16886==    by 0x411E0A: main (lightningd.c:841)
==16886==  Block was alloc'd at
==16886==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16886==    by 0x4617CE: allocate (tal.c:245)
==16886==    by 0x461E4C: tal_alloc_ (tal.c:423)
==16886==    by 0x42255E: plugins_new (plugin.c:106)
==16886==    by 0x41133D: new_lightningd (lightningd.c:218)
==16886==    by 0x411AD4: main (lightningd.c:649)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell c6ec9443b9 jsonrpc: don't return "stop" until we actually have freed resources.
This is a painpoint with testing, that there's a noticable delay between
"Shutting down" from lightning-cli and being able to restart lightningd.

This fixes that by creating a canned response for this case, which is
simply written out immediately before exit.  At this point, the pidfile
has been deleted, the sockets have been closed, and the database
has been closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell d817735dc2 lightningd: use ccan/json_out.
This is now a fairly simple transition, which only effects the internals
of json_stream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 7f75043ab2 lightningd: tighten interal json_stream API.
Move it closer to ccan/json_out, in preparation for using that as a
replacement.

In particular:

1. Add a 'quote' field in json_add_member.
2. json_add_member now always escapes if 'quote' is true.
3. json_member_direct is exposed to allow avoiding of escaping.
4. json_add_hex can use this, so no longer needs to be in json_stream.c.
5. We don't make JSON manually, but always use helpers.
6. We now flush the stream (wake reader) only when we close it, or mark
   command as pending.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell adc4bf1817 pylightning: remove 0.6.2 JSON compat wedge.
The next commit breaks it: `if b' }\n' not in buff:` is always true since
we're about to clean up our JSON so there won't be a space.  I could have
hacked the space in our JSON, but 6 months is long enough anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell bb7bbd03c5 lightningd: have json_stream_success start the "result" object.
"result" should always be an object (so that we can add new fields),
so make that implicit in json_stream_success.

This makes our primitives well-formed: we previously used NULL as our
fieldname when calling the first json_object_start, which is a hack
since we're actually in an object and the fieldname is 'result' (which
was already written by json_object_start).

There were only two cases which didn't do this:
1. dev-memdump returned an array.  No API guarantees on this.
2. shutdown returned a string.

I temporarily made shutdown return an empty object, which shouldn't
break anything, but I want to fix that later anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 0e336ac252 plugins/pay: use proper JSON construction for failure paths.
The string cut & paste hack was nasty; make ->failure a json_out
object so we can splice it in properly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell c585f22711 libplugin: update API to use json_out.
We now hand around struct json_out members, rather than using formatted
strings, so plugins need to construct them properly.

There's no automatic conversion between ' and " any more, so those
are eliminated too.  pay still uses some manual construction of elements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell c77e29c403 libplugin: use json_out internally.
Since this handles escaping for us, this automatically fixes our previous
escaping issued.

Fixes: #2612
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 220449e1cd ccan: import ccan/json_out and ccan/json_escape.
These are generalized from our internal implementations.

The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.

The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 26cdf9d3dc plugins/pay: don't retry routehint if it contains already-eliminated channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 260febd88b plugins/pay: fix attempt counter on failure message.
An "attempt" is when we actually try to send, not every route lookup
we do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 1e32b4ab29 gossipd: adjust gossip filters if we discover we're missing gossip.
We pick up to three random peers and ask them to gossip more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell 6830233d0b gossipd: control gossip level so we don't get flooded by peers.
We seek a certain number of peers at each level of gossip; 3 "flood"
if we're missing gossip, 2 at 24 hours past to catch recent gossip, and
8 with current gossip.  The rest are given a filter which causes them
not to gossip to us at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell f5ea57d4c0 gossipd: reset gossip_missing if no reports for 10 minutes.
An arbitrary timeout.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell b9053767e7 gossipd: query unknown short_channel_ids, note if they were really missing.
The first sign that we're missing gossip is that we get a channel_update
for an unknown channel.  The peer might be wrong (or lying), but if it turns
out to be a real channel, we were definitely missing something.

This patch does two things: queries when we get an unknown channel_update,
and then notes that a channel_announcement was from such an update when
it's finally processed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell 18069ab3da gossipd: APIs return more information about routing message handling.
In particular, we'll need to know the short_channel_id if a
channel_update is unknown (implies we're missing a channel), and whether
processing a pending channel_announcement was successful (implies that
the channel was real).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell 5ef7aa70d2 gossipd: prepare for internally-generated short-channel-id queries.
Up until now we only generated these in dev mode for testing.  Hoist
into common code, turn counter into a flag (we're only allowed one!)
and note if query is internal or not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell 21c920a8e8 gossipd: note if loaded store seems reasonably up-to-date.
If not, we can ask peers for full gossip (for now we just set a flag).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 00:37:46 +00:00
Rusty Russell 0d2a4830ed ccan: update to faster and correct crc32c implementation.
I decided to try a faster implementation, only to find our crc32c was
not correct!  Ouch.

I removed the crc32c functions from ccan/crc, and added a new crc32c
module which has the Mark Adler x86-64-optimized variants.

We bump gossip_store version again, since csums have changed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:40:10 +00:00
darosior 7ce4fcd983 Added a CHANGELOG brag line about 'fundchannel' accepting an utxo list 2019-06-11 23:24:07 +00:00
darosior 2436214145 fundchannel: Add some tests to fund channels by utxos 2019-06-11 23:24:07 +00:00
darosior 1b29b063b2 fundchannel: Update the manpage and Pylightning's method
Add the new 'utxo' parameter to both
2019-06-11 23:24:07 +00:00
darosior ac038b340b fundchannel: Add a new 'utxo' parameter
This new parameter takes a list of outpoints (as txid:vout) and fund a channel from the corresponding utxos.
Example : fundchannel <id> 10000 normal 1 [10767f0db0e568127fffd7f70a154d4599f42d62babf63230a7c3378bfce3cb0:0, c9e040e0b5fc8c59d5e7834108fbc5583001f414dd83faf0a05cff9d1a92d32c:0]
2019-06-11 23:24:07 +00:00
darosior ca4b07c529 json: add a helper to split a json token given a specific character
credits @rustyrussell
2019-06-11 23:24:07 +00:00
Rusty Russell db22d2ba9a pytest: simplify and annotate test_channel_drainage.
Remove gratuitous prints, add explanations of what's going on,
and demonstrate that we can add a final trimmed HTLC but not
a non-trimmed one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell 1275928fa2 channeld: don't add HTLCs if that would drive us negative.
We track whether each change is affordable as we go;
test_channel_drainage got us so close that the difference mattered; we
hit an assert when we tried to commit the tx and realized we couldn't
afford it.

We should not be trying to add an HTLC if it will result in the funder
being unable to afford it on either the local *or remote* commitments.

Note the test still "fails" because it refuses to send the final
payment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell 431401ad35 channeld: don't subtract both reserves from advertized htlc_max.
Subtracting both arbitrarily reduces our capacity, even for ourselves
since the routing logic uses this maximum.

I also changed 'advertise' to 'advertize', since we use american
spelling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell 044860881e lightningd: don't let spendable_msat go wumbo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell bb6c34fb13 lightningd: take into account outstanding HTLCs for 'spendable_msat'
The current calculation ignores them, which is unrealistic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell ab31f40aa2 gossipd: don't charge ourselves fees when calculating route.
This means there's now a semantic difference between the default `fromid`
and setting `fromid` explicitly to our own node_id.  In the default case,
it means we don't charge ourselves fees on the route.

This means we can spend the full channel balance.

We still want to consider the pricing of local channels, however:
there's a *reason* to discount one over another, and that is to bias
things.  So we add the first-hop fee to the *risk* value instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell ad24d94c7b lightningd: make 'spendable_msatoshi' more realistic.
Take into account the fee we'd have to pay if we're the funder, and
also drop to 0 if the amount is less than the smallest HTLC the peer
will accept.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell e7778a0494 channeld: extract HTLC trim logic into common.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell 7b7d46c166 pytest: extract separate tests that spendable_msat is accurate.
Turns out we needed more comprehensive testing; we ended up with three
separate tests.  To avoid changing test_channel_drainage as we fix
spendable_msat, I substituted raw numbers there.

The first is a variation of the existing tests, testing we can't
exceed spendable_msat, and we can pay it, both ways.

The second is with a larger amount, which triggers a different problem.

The final is with a giant channel, which tests our 2^32-1 msat cap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell 409368e058 pytest: move test_channel_drainage to test_pay.py
This is where payment tests should go.  Also mark it xfail for the moment,
and remove developer-only tag (propagating gossip is only 60 seconds, which
is OK).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell 380c044668 tests: add plugin for simple invoice delay.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Rusty Russell b48c644e7a listchannels: add `htlc_minimum_msat` and `htlc_maximum_msat` fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Michael Schmoock 4a242edc1f test: drains a channel to crash the daemon 2019-06-11 23:19:11 +00:00