Commit Graph

7525 Commits

Author SHA1 Message Date
Michael Schmoock 3e2607bf2d chore: updates coincurve requirement to v13
Update to v13 as v12 started to raise undefined symbol exceptions agains latest
libsecp256k1.

Note: any version of `pip install coincurve` fails if no libsecp256k1 headers
are installed, should we point this out somewhere/somehow?

Changelog-None
2020-02-18 10:10:17 +10:30
Rusty Russell 9d9480f17e ccan: update to latest version which allows destructors to self-delete.
Also, a fix for compiling with NDEBUG.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-17 19:55:13 +10:30
Rusty Russell 7de03e990f CHANGELOG.md: 0.8.1: "Channel to the Moon"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-17 19:07:09 +10:30
Rusty Russell 73f760342b doc: remove two more references to the '[Unreleased]' in CHANGELOG.md
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-17 19:07:09 +10:30
Rusty Russell 0b123a3cec CHANGELOG.md: 0.8.1rc3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-15 09:04:50 +10:30
Rusty Russell df50faba6a lightningd: fix crash when plugin has been unloaded and we abort cmd.
I reproduced this by putting a sleep(60) in the pay plugin, then
'lightning-cli pay', 'lightning-cli plugin stop pay' and then ^C
the `lightning-cli pay`:

2020-02-14T00:33:11.217Z INFO plugin-pay: Killing plugin: pay stopped by lightningd via RPC
2020-02-14T00:33:15.250Z DEBUG lightningd: Still waiting for initial block download
==5157== Invalid read of size 8
==5157==    at 0x12A29C: destroy_jcon (jsonrpc.c:149)
==5157==    by 0x1C6F2A: notify (tal.c:235)
==5157==    by 0x1C7441: del_tree (tal.c:397)
==5157==    by 0x1C7493: del_tree (tal.c:407)
==5157==    by 0x1C77DD: tal_free (tal.c:481)
==5157==    by 0x1B7380: io_close (io.c:450)
==5157==    by 0x1B71A7: do_plan (io.c:401)
==5157==    by 0x1B7214: io_ready (io.c:417)
==5157==    by 0x1B94AC: io_loop (poll.c:445)
==5157==    by 0x1291C9: io_loop_with_timers (io_loop_with_timers.c:24)
==5157==    by 0x12EC7E: main (lightningd.c:928)
==5157==  Address 0x4ebab98 is 40 bytes inside a block of size 88 free'd
==5157==    at 0x483BA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==5157==    by 0x1C750F: del_tree (tal.c:416)
==5157==    by 0x1C7493: del_tree (tal.c:407)
==5157==    by 0x1C77DD: tal_free (tal.c:481)
==5157==    by 0x153856: clear_plugin (plugin_control.c:209)
==5157==    by 0x1538FF: plugin_dynamic_stop (plugin_control.c:225)
==5157==    by 0x153C51: json_plugin_control (plugin_control.c:295)
==5157==    by 0x12B4EC: command_exec (jsonrpc.c:588)
==5157==    by 0x12B8AB: rpc_command_hook_callback (jsonrpc.c:679)
==5157==    by 0x154575: plugin_hook_call_ (plugin_hook.c:170)
==5157==    by 0x12BCD3: plugin_hook_call_rpc_command (jsonrpc.c:756)
==5157==    by 0x12BD04: call_rpc_command_hook (jsonrpc.c:764)
==5157==  Block was alloc'd at
==5157==    at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==5157==    by 0x1C6F98: allocate (tal.c:245)
==5157==    by 0x1C7559: tal_alloc_ (tal.c:423)
==5157==    by 0x15135A: plugin_rpcmethod_add (plugin.c:706)
==5157==    by 0x151600: plugin_rpcmethods_add (plugin.c:756)
==5157==    by 0x151BDD: plugin_parse_getmanifest_response (plugin.c:893)
==5157==    by 0x151C9C: plugin_manifest_cb (plugin.c:915)
==5157==    by 0x14FFB9: plugin_response_handle (plugin.c:258)
==5157==    by 0x150165: plugin_read_json_one (plugin.c:356)
==5157==    by 0x1502BC: plugin_read_json (plugin.c:388)
==5157==    by 0x1B65ED: next_plan (io.c:59)
==5157==    by 0x1B71D2: do_plan (io.c:407)

Fixes: #3509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-15 08:49:18 +10:30
Rusty Russell 77e3df0a29 gossipd: remove assert which can trigger.
We can actually fail to find a shorter route, but it's a fairly obscure case.

Fixes: #3517
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-15 08:48:04 +10:30
Rusty Russell 243078ac0c CHANGELOG.md: fix extra 'v' in version.
Breaks `tools/build-release.sh`

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-15 08:48:04 +10:30
Rusty Russell e7a1cf137a CHANGELOG.md: v0.8.1rc2
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-13 16:10:26 +10:30
Rusty Russell fc4d01cd9d db: put scids in forwards even if we didn't actually send.
If the peer is not connected, or other error which means we don't
actually create an outgoing HTLC, we don't record the
short_channel_id.  This is unhelpful!

Pass the scid down to the wallet code, and explicitly hand the
scid and amount down to the notification code rather than handing it
the htlc_out (which it doesn't need).

Changelog-Changed: JSON API: `listforwards` now shows `out_channel` even if we couldn't forward.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-12 22:27:57 -06:00
Rusty Russell 5e2053feea pay: fix crash when we paystatus too soon.
If the attempt hasn't started yet, we can have an empty 'attempts' array
(while it's in listpeers, for example):

```
pay: plugins/pay.c:1457: json_paystatus: Assertion `tal_count(ps->attempts)' failed.
pay: FATAL SIGNAL 6 (version v0.8.1rc1-1-g946af3f)
0x55e895110543 send_backtrace
	common/daemon.c:39
0x55e8951105e9 crashdump
	common/daemon.c:52
0x7f92b0928f1f ???
	???:0
0x7f92b0928e97 ???
	???:0
0x7f92b092a800 ???
	???:0
0x7f92b091a399 ???
	???:0
0x7f92b091a411 ???
	???:0
0x55e895100012 json_paystatus
	plugins/pay.c:1457
0x55e895103aaa ld_command_handle
	plugins/libplugin.c:968
0x55e895103bd4 ld_read_json_one
	plugins/libplugin.c:1011
0x55e895103cd2 ld_read_json
	plugins/libplugin.c:1030
0x55e895124cbd next_plan
	ccan/ccan/io/io.c:59
0x55e89512583a do_plan
	ccan/ccan/io/io.c:407
0x55e895125878 io_ready
	ccan/ccan/io/io.c:417
0x55e895127a3e io_loop
	ccan/ccan/io/poll.c:445
0x55e895104593 plugin_main
	plugins/libplugin.c:1194
0x55e895100cbc main
	plugins/pay.c:1697
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-13 14:20:50 +10:30
Rusty Russell 7a178ebb7a plugins: libplugin don't insist on 'jsonrpc' field from lightningd.
Spark does this, for example:

{"method":"pay","params":["lnbc..."],"id":22}

Which doesn't have a jsonrpc field.  The result is that the command
doesn't terminate, there is nothing in the logs, stderr contains
"pay: JSON-RPC message does not contain "jsonrpc" field", and
from then on "Unknown command 'pay'".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-13 12:34:31 +10:30
Rusty Russell ff968e7730 CHANGELOG.md: Update for 0.8.1-rc1.
Normalized lines; split some where we've deprecated something (needs a
line each in Deprecated section).

Also, removed unused [Unreleased] footnote in favor of [0.8.0] footnote.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-12 16:23:28 +10:30
Rusty Russell 2c88ab53d4 doc: update MAKING-RELEASES.
1. GH_TOKEN is probably required to run the changelog script.
2. Note the footnotes need updating; we no longer have [Unreleased].

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-12 16:23:28 +10:30
darosior a47fd8cf3e pytest: test Bitcoin plugin registration and the bcli plugin 2020-02-12 11:45:07 +10:30
darosior 4c5862eab1 pytest: fixup flaky test_closing_specified_destination 2020-02-12 11:45:07 +10:30
darosior bbc5c10919 pytest: adjust mocks and logs for bitcoind
For bitcoind_fail_first:
We only ever send `getblock` if we got a successful block hash from
`getblockhash`, and if we can't get the block in that case it means
our Bitcoin backend is faulty and we shouldnt continue.

So, mock `getblockhash` instead, which is authorized to spuriously fail.

For both bitcoind_fail_first and bitcoind_failure:
Adapt the logs.
2020-02-12 11:45:07 +10:30
darosior 06d598efc1 lightningd: clean utxos after the topology is setup
We need our Bitcoin backend to be ready to get infos about some utxos
2020-02-12 11:45:07 +10:30
darosior 580556b1d0 lightningd/bitcoind: remove all bitcoin-cli specific code
Changelog-Added: pluggable backends for Bitcoin data queries, default still bitcoind (using bitcoin-cli).
2020-02-12 11:45:07 +10:30
darosior ced444a605 lightningd/bitcoind: use the Bitcoin plugin for fee estimates
And remove bitcoin-cli interaction code, now unused.
2020-02-12 11:45:07 +10:30
darosior c79ab0f1b0 lightningd/bitcoind: use the Bitcoin plugin for getutxout 2020-02-12 11:45:07 +10:30
darosior 78cb921b80 lightningd/bitcoind: use getrawblockatheight for getfilteredblock
This avoids the getblockhash+getblock, and more importantly that was the
last functionality making use of bitcoind_getrawblock() and bitcoin_getblockhash(),
so we can also get rid of them.
2020-02-12 11:45:07 +10:30
darosior 947f5ddde1 lightningd/bitcoind: use the Bitcoin plugin to send transactions
This restrains the informations we get about how the sending went to
an errmsg as we cant rely on bitcoin-cli specific output nor its exit code.
2020-02-12 11:45:07 +10:30
darosior ca7c1f8bc8 lightningd/chaintopology: use plugin for tip polling 2020-02-12 11:45:07 +10:30
darosior 7b18b24e39 lightningd/chaintopology: use plugin backend to setup topology
This adds `getchaininfo` and `getrawblockbyheight` handling lightningd-side,
and use them in setup_topology().

We then remove legacy bitcoind_getblockcount() (we already get the count in
`getchaininfo`), bitcoind_getblockchaininfo() (it was only used in setup_topology()),
and wait_for_bitcoind() (this was specific to bitcoin-core and we assume our Bitcoin
backend to be functional if the plugin responds to `init`).
2020-02-12 11:45:07 +10:30
darosior 2b75a46a6c chaintopology: dont check bitcoin-core version at startup
The Bitcoin backend is generalized through the Bitcoin plugin and this
was specific to core.
2020-02-12 11:45:07 +10:30
darosior 3e8a782da7 bitcoind: initialize Bitcoin-backend plugin early
We need our Bitcoin backend to be initialized, but the plugins have not yet been
started at this point.
2020-02-12 11:45:07 +10:30
darosior c4ff960b74 bitcoind: allow commands to be registered by different plugins
An strmap is convenient to get a plugin given a method. Populate it
while checking commands.
2020-02-12 11:45:07 +10:30
darosior ae249a2294 chaintopology: check bitcoin plugin commands at startup
Exit early if we won't be able to fully communicate with our Bitcoin
backend.
2020-02-12 11:45:07 +10:30
darosior 5840e90ceb plugins/libplugin: don't crash if 'lightning-rpc' doesnt exist (yet)
We are going to initialize a plugin before its creation, so log as
UNUSUAL instead.

Also, `pay` and `fundchannel` inits are using rpc_delve(), so we need to
io_new_conn() (which sets the socket as non blocking) after calling the
plugin's init.
2020-02-12 11:45:07 +10:30
darosior c1009635ed plugins/bcli: register Bitcoin-related options 2020-02-12 11:45:07 +10:30
darosior f81cc9f552 plugins/bcli: wait for bitcoind to be warmed up at init
This is also taken and adapted from lightningd/bitcoind.

The call to 'getblockchaininfo' is replaced by 'echo' as we don't
make use of the result and the former can sometimes be slow (e.g. on
IBD).
2020-02-12 11:45:07 +10:30
darosior 70a79e3998 plugins/bcli: a new plugin for gathering Bitcoin data
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
  whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
  `getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
  `getfilteredblock` in `lightningd/bitcoind`.
2020-02-12 11:45:07 +10:30
darosior 3eb0f56f87 libplugin: generalize the plugin_timer callback type
We don't take the callback result into account, so it can better be void.
Having a general callback parameter is handy, because for bcli we want
to pass it the struct bcli.
2020-02-12 11:45:07 +10:30
darosior b0b55d36ef libplugin: add a 'still_pending' helper 2020-02-12 11:45:07 +10:30
ZmnSCPxj jxPCSnmZ 5c7f9b542a doc/lightning-listpeers.7.md: Expand and correct.
ChangeLog-Fixed: Corrected and expanded `lightning-listpeers.7` documentation.
2020-02-12 11:06:17 +10:30
darosior e6a6ef23fd pytest: make the libplugin test non flaky
Rusty fixed the underlying bug in 8b8e4bb5fd
2020-02-12 09:30:02 +10:30
Rusty Russell 86c28b2272 channeld: channel drain mitigation.
Add new check if we're funder trying to add HTLC, keeping us
with enough extra funds to pay for another HTLC the peer might add.

We also need to adjust the spendable_msat calculation, and update
various tests which try to unbalance channels.  We eliminate
the now-redundant test_channel_drainage entirely.

Changelog-Fixed: Corner case where channel could become unusable (https://github.com/lightningnetwork/lightning-rfc/issues/728)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 15:34:05 +01:00
Rusty Russell 02565bcbea channeld: trivial refactor of fee_for_htlcs().
Extract out num_untrimmed_htlcs() from inside fee_for_htlcs(), and
remove unused view arg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 15:34:05 +01:00
Rusty Russell 6efde31ade pytest: simplified drain test (xfail)
This is inspired by @m-schmook's https://github.com/ElementsProject/lightning/pull/3498
except this is simply a two-channel version which probes for the amount.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 15:34:05 +01:00
Rusty Russell be8e261da6 ccan: update to include opt_unregister fix.
This was caught by valgrind on Travis, and really confused me.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 16:37:21 +10:30
Christian Decker c223932e40 doc: Update plugins.md to describe the chain-mode for hooks 2020-02-11 15:57:22 +10:30
Christian Decker bdbbfaeb40 doc: Reduce useless nesting in plugins.md
Triple nesting seems a bit excessive, I can't even read the titles in the
sidebar of http://lightning.readthedocs.org anymore :-)
2020-02-11 15:57:22 +10:30
Christian Decker 490550d508 plugin: Allow multiple plugins to register the `htlc_accepted` hook
Make the `htlc_accepted` hook the first chained hook in our repertoire. The
plugins are called one after the other in order until we have no more plugins
or the HTLC was handled by one of the plugins. If no plugins handles the HTLC
we continue to handle it internally like always.

Handling in this case means the plugin returns either `{"result": "resolve",
...}` or `{"result": "fail", ...}`.

Changelog-Changed: plugin: Multiple plugins can now register for the htlc_accepted hook.
2020-02-11 15:57:22 +10:30
Christian Decker d639bdd416 plugin: Remove special case for plugin stopping while handling hooks
This used to be necessary because we allocated the `plugin_hook_request` off
of the plugin instance (only tal allocated object we could grab at that
time. Now the plugin was replaced by a list, which itself is tal-allocated,
making that workaround pointless, or even wrong once we have multiple plugins
registering for that hook.
2020-02-11 15:57:22 +10:30
Christian Decker a3ab3d2990 plugin: Call next plugin that registered hook if result is continue 2020-02-11 15:57:22 +10:30
Christian Decker 71e67ba47f plugin: Split plugin_hook_call_ into initialization and call_next
We will be using `plugin_hook_call_next` as part of the loop to traverse all
plugins that registered the hook, so group initialization in the init function
and move per-plugin logic into `plugin_hook_call_next`
2020-02-11 15:57:22 +10:30
Christian Decker dc2f9a9088 plugin: Internalize plugin_hook call payload in the request struct
We are about to call multiple plugins, and we'll have to pass the payload into
each call. Sadly the serialized stream gets consumed during the call, so keep
the unserialized payload around.
2020-02-11 15:57:22 +10:30
Christian Decker b25e195c2c plugin: Multiple plugins can register a singl hook
Switch from having a single plugin to a list of plugins. If the hook is of
type single we will enforce that constraint on the number of registered
plugins when attempting to add.
2020-02-11 15:57:22 +10:30
Christian Decker 9a2a09efd6 plugin: Introduce plugin type to allow singleton and chaining
The newly introduced type is used to determine what the call semantics of the
hook are. We have `single` corresponding to the old behavior, as well as
`chain` which allows multiple plugins to register for the hook, and they are
then called sequentially (if all plugins return `{"result": "continue"}`) or
exit the chain if the hook event was handled.
2020-02-11 15:57:22 +10:30