Commit Graph

7563 Commits

Author SHA1 Message Date
Rusty Russell ff968e7730 CHANGELOG.md: Update for 0.8.1-rc1.
Normalized lines; split some where we've deprecated something (needs a
line each in Deprecated section).

Also, removed unused [Unreleased] footnote in favor of [0.8.0] footnote.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-12 16:23:28 +10:30
Rusty Russell 2c88ab53d4 doc: update MAKING-RELEASES.
1. GH_TOKEN is probably required to run the changelog script.
2. Note the footnotes need updating; we no longer have [Unreleased].

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-12 16:23:28 +10:30
darosior a47fd8cf3e pytest: test Bitcoin plugin registration and the bcli plugin 2020-02-12 11:45:07 +10:30
darosior 4c5862eab1 pytest: fixup flaky test_closing_specified_destination 2020-02-12 11:45:07 +10:30
darosior bbc5c10919 pytest: adjust mocks and logs for bitcoind
For bitcoind_fail_first:
We only ever send `getblock` if we got a successful block hash from
`getblockhash`, and if we can't get the block in that case it means
our Bitcoin backend is faulty and we shouldnt continue.

So, mock `getblockhash` instead, which is authorized to spuriously fail.

For both bitcoind_fail_first and bitcoind_failure:
Adapt the logs.
2020-02-12 11:45:07 +10:30
darosior 06d598efc1 lightningd: clean utxos after the topology is setup
We need our Bitcoin backend to be ready to get infos about some utxos
2020-02-12 11:45:07 +10:30
darosior 580556b1d0 lightningd/bitcoind: remove all bitcoin-cli specific code
Changelog-Added: pluggable backends for Bitcoin data queries, default still bitcoind (using bitcoin-cli).
2020-02-12 11:45:07 +10:30
darosior ced444a605 lightningd/bitcoind: use the Bitcoin plugin for fee estimates
And remove bitcoin-cli interaction code, now unused.
2020-02-12 11:45:07 +10:30
darosior c79ab0f1b0 lightningd/bitcoind: use the Bitcoin plugin for getutxout 2020-02-12 11:45:07 +10:30
darosior 78cb921b80 lightningd/bitcoind: use getrawblockatheight for getfilteredblock
This avoids the getblockhash+getblock, and more importantly that was the
last functionality making use of bitcoind_getrawblock() and bitcoin_getblockhash(),
so we can also get rid of them.
2020-02-12 11:45:07 +10:30
darosior 947f5ddde1 lightningd/bitcoind: use the Bitcoin plugin to send transactions
This restrains the informations we get about how the sending went to
an errmsg as we cant rely on bitcoin-cli specific output nor its exit code.
2020-02-12 11:45:07 +10:30
darosior ca7c1f8bc8 lightningd/chaintopology: use plugin for tip polling 2020-02-12 11:45:07 +10:30
darosior 7b18b24e39 lightningd/chaintopology: use plugin backend to setup topology
This adds `getchaininfo` and `getrawblockbyheight` handling lightningd-side,
and use them in setup_topology().

We then remove legacy bitcoind_getblockcount() (we already get the count in
`getchaininfo`), bitcoind_getblockchaininfo() (it was only used in setup_topology()),
and wait_for_bitcoind() (this was specific to bitcoin-core and we assume our Bitcoin
backend to be functional if the plugin responds to `init`).
2020-02-12 11:45:07 +10:30
darosior 2b75a46a6c chaintopology: dont check bitcoin-core version at startup
The Bitcoin backend is generalized through the Bitcoin plugin and this
was specific to core.
2020-02-12 11:45:07 +10:30
darosior 3e8a782da7 bitcoind: initialize Bitcoin-backend plugin early
We need our Bitcoin backend to be initialized, but the plugins have not yet been
started at this point.
2020-02-12 11:45:07 +10:30
darosior c4ff960b74 bitcoind: allow commands to be registered by different plugins
An strmap is convenient to get a plugin given a method. Populate it
while checking commands.
2020-02-12 11:45:07 +10:30
darosior ae249a2294 chaintopology: check bitcoin plugin commands at startup
Exit early if we won't be able to fully communicate with our Bitcoin
backend.
2020-02-12 11:45:07 +10:30
darosior 5840e90ceb plugins/libplugin: don't crash if 'lightning-rpc' doesnt exist (yet)
We are going to initialize a plugin before its creation, so log as
UNUSUAL instead.

Also, `pay` and `fundchannel` inits are using rpc_delve(), so we need to
io_new_conn() (which sets the socket as non blocking) after calling the
plugin's init.
2020-02-12 11:45:07 +10:30
darosior c1009635ed plugins/bcli: register Bitcoin-related options 2020-02-12 11:45:07 +10:30
darosior f81cc9f552 plugins/bcli: wait for bitcoind to be warmed up at init
This is also taken and adapted from lightningd/bitcoind.

The call to 'getblockchaininfo' is replaced by 'echo' as we don't
make use of the result and the former can sometimes be slow (e.g. on
IBD).
2020-02-12 11:45:07 +10:30
darosior 70a79e3998 plugins/bcli: a new plugin for gathering Bitcoin data
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
  whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
  `getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
  `getfilteredblock` in `lightningd/bitcoind`.
2020-02-12 11:45:07 +10:30
darosior 3eb0f56f87 libplugin: generalize the plugin_timer callback type
We don't take the callback result into account, so it can better be void.
Having a general callback parameter is handy, because for bcli we want
to pass it the struct bcli.
2020-02-12 11:45:07 +10:30
darosior b0b55d36ef libplugin: add a 'still_pending' helper 2020-02-12 11:45:07 +10:30
ZmnSCPxj jxPCSnmZ 5c7f9b542a doc/lightning-listpeers.7.md: Expand and correct.
ChangeLog-Fixed: Corrected and expanded `lightning-listpeers.7` documentation.
2020-02-12 11:06:17 +10:30
darosior e6a6ef23fd pytest: make the libplugin test non flaky
Rusty fixed the underlying bug in 8b8e4bb5fd
2020-02-12 09:30:02 +10:30
Rusty Russell 86c28b2272 channeld: channel drain mitigation.
Add new check if we're funder trying to add HTLC, keeping us
with enough extra funds to pay for another HTLC the peer might add.

We also need to adjust the spendable_msat calculation, and update
various tests which try to unbalance channels.  We eliminate
the now-redundant test_channel_drainage entirely.

Changelog-Fixed: Corner case where channel could become unusable (https://github.com/lightningnetwork/lightning-rfc/issues/728)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 15:34:05 +01:00
Rusty Russell 02565bcbea channeld: trivial refactor of fee_for_htlcs().
Extract out num_untrimmed_htlcs() from inside fee_for_htlcs(), and
remove unused view arg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 15:34:05 +01:00
Rusty Russell 6efde31ade pytest: simplified drain test (xfail)
This is inspired by @m-schmook's https://github.com/ElementsProject/lightning/pull/3498
except this is simply a two-channel version which probes for the amount.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 15:34:05 +01:00
Rusty Russell be8e261da6 ccan: update to include opt_unregister fix.
This was caught by valgrind on Travis, and really confused me.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 16:37:21 +10:30
Christian Decker c223932e40 doc: Update plugins.md to describe the chain-mode for hooks 2020-02-11 15:57:22 +10:30
Christian Decker bdbbfaeb40 doc: Reduce useless nesting in plugins.md
Triple nesting seems a bit excessive, I can't even read the titles in the
sidebar of http://lightning.readthedocs.org anymore :-)
2020-02-11 15:57:22 +10:30
Christian Decker 490550d508 plugin: Allow multiple plugins to register the `htlc_accepted` hook
Make the `htlc_accepted` hook the first chained hook in our repertoire. The
plugins are called one after the other in order until we have no more plugins
or the HTLC was handled by one of the plugins. If no plugins handles the HTLC
we continue to handle it internally like always.

Handling in this case means the plugin returns either `{"result": "resolve",
...}` or `{"result": "fail", ...}`.

Changelog-Changed: plugin: Multiple plugins can now register for the htlc_accepted hook.
2020-02-11 15:57:22 +10:30
Christian Decker d639bdd416 plugin: Remove special case for plugin stopping while handling hooks
This used to be necessary because we allocated the `plugin_hook_request` off
of the plugin instance (only tal allocated object we could grab at that
time. Now the plugin was replaced by a list, which itself is tal-allocated,
making that workaround pointless, or even wrong once we have multiple plugins
registering for that hook.
2020-02-11 15:57:22 +10:30
Christian Decker a3ab3d2990 plugin: Call next plugin that registered hook if result is continue 2020-02-11 15:57:22 +10:30
Christian Decker 71e67ba47f plugin: Split plugin_hook_call_ into initialization and call_next
We will be using `plugin_hook_call_next` as part of the loop to traverse all
plugins that registered the hook, so group initialization in the init function
and move per-plugin logic into `plugin_hook_call_next`
2020-02-11 15:57:22 +10:30
Christian Decker dc2f9a9088 plugin: Internalize plugin_hook call payload in the request struct
We are about to call multiple plugins, and we'll have to pass the payload into
each call. Sadly the serialized stream gets consumed during the call, so keep
the unserialized payload around.
2020-02-11 15:57:22 +10:30
Christian Decker b25e195c2c plugin: Multiple plugins can register a singl hook
Switch from having a single plugin to a list of plugins. If the hook is of
type single we will enforce that constraint on the number of registered
plugins when attempting to add.
2020-02-11 15:57:22 +10:30
Christian Decker 9a2a09efd6 plugin: Introduce plugin type to allow singleton and chaining
The newly introduced type is used to determine what the call semantics of the
hook are. We have `single` corresponding to the old behavior, as well as
`chain` which allows multiple plugins to register for the hook, and they are
then called sequentially (if all plugins return `{"result": "continue"}`) or
exit the chain if the hook event was handled.
2020-02-11 15:57:22 +10:30
Rusty Russell 30580731a6 Minor fixups on PR #3477
Feedback from @niftynei and me; nothing major, but avoids
another round-trip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 13:53:31 +10:30
Christian Decker 4737977128 plugin: Allow custom features only if the plugin is not dynamic
This is in order to avoid having to update featurebits as plugins get
activated and deactivated.
2020-02-11 13:53:31 +10:30
Christian Decker 5bd419d9fd doc: Document the featurebits functionality for plugins 2020-02-11 13:53:31 +10:30
Christian Decker ebd3a8d7f2 plugin: Add featurebits registered by plugins to node_announcements
This is the last venue we need to add custom featurebits to, so we also unmark
the test as xfail.

Changelog-Added: plugin: Plugins can now signal support for experimental protocol extensions by registering featurebits for `node_announcement`s, the connection handshake, and for invoices. For now this is limited to non-dynamic plugins only
2020-02-11 13:53:31 +10:30
Christian Decker 8d6c8c3cd1 connectd: Pass the init_featurebits down to connectd and use in init
The `init_featurebits` are computed at startup, and then cached
indefinitely. They are then used whenever a new `init` handshake is performed.

We could add a new message to push updates to `connectd` whenever a plugin is
added or removed, but that's up for discussion.
2020-02-11 13:53:31 +10:30
Christian Decker 58c9a6a004 plugin: Include featurebits registered by plugins in invoices 2020-02-11 13:53:31 +10:30
Christian Decker 532bf1730f plugin: Add function to collect featurebits that plugins registered 2020-02-11 13:53:31 +10:30
Christian Decker 9b976da3bc features: Add featurebits_or helper to combine two featurebitstrings
We will be doing this when collecting featurebits from the plugins, so make
this a reusable function.
2020-02-11 13:53:31 +10:30
Christian Decker ea62d97879 plugin: Store a plugin's featurebits in the plugin struct
We'll collect the featurebits on-demand from all currently active plugins when
needed.
2020-02-11 13:53:31 +10:30
Christian Decker 966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
Christian Decker a13591400a pyln-client: Add facility to register featurebits from plugins
we have 4 venues in which we can add features, 3 of which are unilaterally
controlled (`init`, `node_announcement`, and `invoices`) the
`channel_announcement` is co-signed by both parties, so we can't add
featurebits without additional coordination overhead.

Each location is encoded as a key-value pair in a dict called `featurebits` in
the manifest (omitted if no custom featurebits are set).
2020-02-11 13:53:31 +10:30
lisa neigut 27c7707874 Add note about rationale for check 2020-02-10 15:59:26 +10:30