Commit Graph

5937 Commits

Author SHA1 Message Date
Rusty Russell fc27250f80 gossipd: be more verbose and less assert()ive on bad node_announcement.
We hit the timestamp assert on #2750; it shouldn't happen, but crashing
doesn't leave much information.

Reported-by: @m-schmook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 22:03:35 +00:00
Rusty Russell 9bf0467967 pytest: fix test_gossip_store_load_no_channel_update
It wasn't invalid due to a missing channel_update, but in fact was a
bad checksum due to a cut & paste bug.  Fix that, and assert it's not
actually truncating.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 22:03:35 +00:00
Rusty Russell adcb6641a7 common/gossip_store: handle short reads.
This happened on Travis, and the gossip_store was a suspicious 4096
bytes long.  This implies they're using some non-atomic filesystem
(gossipd always does atomic writes to gossip_store), but if they are,
others surely are too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 04:54:29 +00:00
Rusty Russell f1b57063f7 bitcoin/tx: use fromwire_fail in pull_bitcoin_tx.
This is the correct way to mark failure: it also sets *max to 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 03:56:59 +00:00
Rusty Russell 711762ec67 bitcoin/tx: separate assertions.
This might help diagnose what's happening with #2759

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 03:56:59 +00:00
Rusty Russell 47b5f2e837 gossipd: truncate gossip_store.tmp for compaction.
If something went wrong and there was an old one, we were
appending to it!

Reported-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-20 02:53:52 +00:00
lisa neigut 92bd3a3587 connect test: use wait_for_log in test_funding_external_wallet
test_funding_external_wallet is flakey because we don't wait for
the channel state to update before asserting that it's changed.
2019-06-18 06:54:21 +00:00
Rusty Russell 77018e6200 CHANGELOG.md: note the db_write hook and listtransactions command.
Reported-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-18 04:47:38 +00:00
Rusty Russell 841fba7295 lightningd: restore ' ' before '}' for older pylightning.
It needs this in compat mode to detect old (pre-0.6.3) end of JSON.
But it always does the first command in compat mode.

This was never really reliable, since the first command could be to
a plugin for which we simply pass through the JSON (though, carefully
appending the expected '\n\n' if not already there).

Reported-by: @laanwj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-17 21:46:25 +02:00
darosior da676baac4 plugins doc: remove the 'only string options' warning
int and bool are now also accepted ! :)
2019-06-17 10:25:30 +00:00
Rusty Russell 5e3690b3c5 gossipd: delete channel_amount from the store when we delete channel_announcement.
Otherwise we slowly build up cruft: compaction simply moves them since
they're not deleted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell 10c503b4b4 gossip_store: clean up a truncated store.
We might have channel_announcements which have no channel_update: normally
these don't get written into the store until there is one, but if the
store was truncated it can happen.  We then get upset on compaction, since
we don't have an in-memory representation of the channel_announcement.

Similarly, we leave the node_announcement pending until after that
channel_announcement, leading to a similar case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell adc52b6ee8 pytest: add test for dangling channel_announcement/node_announcement after gossip_store.
This can happen if the store was truncated.

Reported-by: @jb55
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell a35ab51a06 pytest: gossip_store test for channel_amount truncated.
We pass, but this test should have been added a while ago with the
original code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell 909f22f117 pytest: gossip_store test for node_announcement before update.
We pass, but this test should have been added a while ago with the fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
darosior 8edf55b34a doc: Correct preimage field name in pay return value 2019-06-15 10:49:29 +02:00
Rusty Russell 57405d7b68 CHANGELOG.md: v0.7.1rc2
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-14 07:37:48 +00:00
Rusty Russell 24cc371cdf gossipd: gossip_store errors after rewrite are fatal.
We can't continue, since we've moved the indexes.  We'll just crash
anyway, as seen from bugs #2742 and #2743.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-14 02:17:32 +00:00
Rusty Russell eb5cc47bdd gossipd: count deleted records correctly when loading gossip_store.
The result of an incorrect count was that we failed on next compaction.

Fixes: #2743
Fixes: #2742
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-14 02:17:32 +00:00
Rusty Russell 12a523f7c5 pytest: add (xfail) test for store load miscount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-14 02:17:32 +00:00
Rusty Russell ea1c52f380 CHANGELOG: Unreleased -> rc1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-13 05:58:35 +00:00
Rusty Russell bbbe2f5437 CHANGELOG.md: reorder with most impactful changes at top.
And reword a few for consistency.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-13 05:58:35 +00:00
Rusty Russell 745634d9b9 gossipd: don't catch pending node_announcements more than once.
We catch node_announcements for nodes where we haven't finished
analyzing the channel_announcement yet (either because we're still
checking UTXO, or in this case, because we're waiting for a channel_update).

But we reference count the pending_node_announce, so if we have
multiple channels pending, we might try to insert it twice.  Clear it
so this doesn't happen.

There's a second bug where we continue to catch node_announcements
until *all* the channel_announcements are no longer pending; this is fixed
by removing it from the map.

Fixes: #2735
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-13 05:58:09 +00:00
Rusty Russell 67eb26c5e6 lightningd: fix bogus channel iteration on setchannelfee
Setting channel to NULL then iterating doesn't work!

Fixes: #2733
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-13 01:28:15 +00:00
Rusty Russell e9ff8e4b4e openingd_control: fix merge error with json_stream_success.
That was changed to start the response object, which broke the openingd
code once we merged.

Of course, I should have *renamed it* when I changed the semantic!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 06:21:53 +00:00
Rusty Russell 4c118bcdba openingd: fix compile error caused by merge.
Compile broke because we were using low-level JSON primitives here
(which, incidentally, would produce bad JSON now, since we can't just
put a raw string inside an object!).

Use json_add_string, which also has the benefit of escaping JSON
for us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 06:21:53 +00:00
lisa neigut c0475d0d9d doc: add manpages for fundchannel_[start,complete,cancel]
Manpages for the manpage-god!!
2019-06-12 02:22:43 +00:00
lisa neigut 388f31a151 funding: update CHANGELOG with new RPC calls
Let people know what's changed
2019-06-12 02:22:43 +00:00
lisa neigut c00e0d2936 funding: rename fundchannel_continue -> _complete
Renaming. "complete" more accurately describes what we're doing here.
2019-06-12 02:22:43 +00:00
lisa neigut 3ae78a61fa tests: finish up test for external funding flow
Add to test for fundchannel with composing and broadcasting
an external transaction.
2019-06-12 02:22:43 +00:00
lisa neigut 5aad642c59 opening: add fundchannel_cancel command
Provide the option to cancel a funding-opening with a peer.
Must either call `fundchannel_cancel` or `fundchannel_continue`
2019-06-12 02:22:43 +00:00
lisa neigut 478762dbd2 funding: wire up funding_continue
Big wiring re-org for funding-continue

In openingd, we move the 'persistent' state (their basepoints,
pubkey, and the minimum_depth requirement for the opening tx) into
the state object. We also look to keep code-reuse between
'continue' and normal 'fundchannel' as high as possible. Both
of these call the same 'fundchannel_reply' at the end.

In opening_control.c, we remap fundchannel_reply such that it is
now aware of the difference between an external/internally funded
channel open. It's the same return path, with the difference that
one finishes making and broadcasting the funding transaction; the
other is skips this.
2019-06-12 02:22:43 +00:00
lisa neigut 8a60caaed9 funding: add a 'inflight' marker
We need a way to gate allowing continue to proceed or not
2019-06-12 02:22:43 +00:00
lisa neigut 7ea21c36b1 fundchannel: add txout field to RPC/API
We'll need the outpoint for the funding output.
2019-06-12 02:22:43 +00:00
lisa neigut b0b813a171 test: add initial tests for starting an external fundchannel
Test for getting through the address generation portion.
2019-06-12 02:22:43 +00:00
lisa neigut dd11d3bd81 lightningd: add start for fundchannel_continue
Add an RPC method (not working at the moment) called
`fundchannel_continue` that takes as its parameters a
node_id and a txid for a transaction (that ostensibly has an output
for a channel)
2019-06-12 02:22:43 +00:00
lisa neigut 52042bde80 opening: stash amount outside of the wtx
Some channels won't be opened with a wtx struct, so keep
the total funding amount separate from it so we can
show some stats for listpeers.

Note that we're going to need to update/confirm this once
the transaction gets confirmed.
2019-06-12 02:22:43 +00:00
lisa neigut 6e5928aa1e openingd: update billboard with funding info
Make it easier for a user to know what's going on with
a channel in `listpeers`.
2019-06-12 02:22:43 +00:00
lisa neigut 23e7846053 pylightning: add fundchannel_start
Add method to rpc handler
2019-06-12 02:22:43 +00:00
lisa neigut 5920e656cf opening: wire up walking through open channel up thru accept
Fill in details to make fundchannel_start work.
2019-06-12 02:22:43 +00:00
lisa neigut 685aa47b65 openingd: pull out setup funder checks into separate method
Useful for adding the funder_start stuff
2019-06-12 02:22:43 +00:00
lisa neigut 8103acd9b3 opening: add entry point for `funding_start rpc command
Beginnings of wiring up the funding_start rpc command. missing
the part that actually starts the funding channel dance.
2019-06-12 02:22:43 +00:00
lisa neigut 846bc9cbc4 openingd: allow funding_failed to complete successfully
For the `fundchannel_cancel` we're going to want
to 'successfully' fail a funding channel operation. This allows
us to report it a failure back as an RPC success, instead of
automatically failing the RPC request.
2019-06-12 02:22:43 +00:00
lisa neigut 1b2a593b05 tests: default all addresses to bech32
Needed for composing a transaction externally to c-lightning, using
bitcoind util.
2019-06-12 02:22:43 +00:00
lisa neigut eb0b3d7b09 json: correct error message for param parser
Looks like copy-paste from another commit didn't update
the field for this
2019-06-12 02:22:43 +00:00
lisa neigut a9d0550cf4 common: pull out scriptPubkey address method
We're going to need this for P2WSH scripts. pull it out into
a common file plus adopt the sanity checks so that it will allow for
either P2WSH or P2WPKH (previously only encoded P2WPKH scripts)
2019-06-12 02:22:43 +00:00
Rusty Russell 5a520f4a07 plugin: don't call notification after free.
This is an old bug, where a plugin can get called while we're shutting
down (and have freed plugins), but it's triggered more reliably by the
new warning notification hook.

For good measure, we also make freeing a plugin self-delete.

Valgrind error file: valgrind-errors.16763
==16886== Invalid read of size 8
==16886==    at 0x422919: plugins_notify (plugin.c:1096)
==16886==    by 0x413919: notify_warning (notification.c:61)
==16886==    by 0x412BDE: logv (log.c:251)
==16886==    by 0x412A98: log_ (log.c:311)
==16886==    by 0x4044BE: bcli_finished (bitcoind.c:178)
==16886==    by 0x459480: destroy_conn (poll.c:244)
==16886==    by 0x459499: destroy_conn_close_fd (poll.c:250)
==16886==    by 0x4619E1: notify (tal.c:235)
==16886==    by 0x461A7E: del_tree (tal.c:397)
==16886==    by 0x461AB5: del_tree (tal.c:407)
==16886==    by 0x461AB5: del_tree (tal.c:407)
==16886==    by 0x461AB5: del_tree (tal.c:407)
==16886==  Address 0x634a578 is 40 bytes inside a block of size 352 free'd
==16886==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16886==    by 0x461AFD: del_tree (tal.c:416)
==16886==    by 0x461FB7: tal_free (tal.c:481)
==16886==    by 0x411E0A: main (lightningd.c:841)
==16886==  Block was alloc'd at
==16886==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16886==    by 0x4617CE: allocate (tal.c:245)
==16886==    by 0x461E4C: tal_alloc_ (tal.c:423)
==16886==    by 0x42255E: plugins_new (plugin.c:106)
==16886==    by 0x41133D: new_lightningd (lightningd.c:218)
==16886==    by 0x411AD4: main (lightningd.c:649)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell c6ec9443b9 jsonrpc: don't return "stop" until we actually have freed resources.
This is a painpoint with testing, that there's a noticable delay between
"Shutting down" from lightning-cli and being able to restart lightningd.

This fixes that by creating a canned response for this case, which is
simply written out immediately before exit.  At this point, the pidfile
has been deleted, the sockets have been closed, and the database
has been closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell d817735dc2 lightningd: use ccan/json_out.
This is now a fairly simple transition, which only effects the internals
of json_stream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 7f75043ab2 lightningd: tighten interal json_stream API.
Move it closer to ccan/json_out, in preparation for using that as a
replacement.

In particular:

1. Add a 'quote' field in json_add_member.
2. json_add_member now always escapes if 'quote' is true.
3. json_member_direct is exposed to allow avoiding of escaping.
4. json_add_hex can use this, so no longer needs to be in json_stream.c.
5. We don't make JSON manually, but always use helpers.
6. We now flush the stream (wake reader) only when we close it, or mark
   command as pending.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00