Commit Graph

9193 Commits

Author SHA1 Message Date
niftynei a31b078721 mfc: consolidate to a single FAILED state 2020-11-23 12:41:05 -06:00
niftynei 991ae65e9e mfc-df: track destination's openchannel version
Plus method to help count/tabulate how many of each we've got
around still.
2020-11-23 12:41:05 -06:00
niftynei e0c4865eea mfc: add a 'fail_destination' helper
Caches state at which we failed + sets error
2020-11-23 12:41:05 -06:00
niftynei 5b6b012af9 mfc: add happy path-way for v2 in multifundchannel
Tested and works with both sides funding! Yay.

Doesn't do any amount of reasonable cleanup or handling of errors.
2020-11-23 12:41:05 -06:00
niftynei 78d32b12d0 nit,df test plugin: change up how feerate is formatted
Suggested-by: @cdecker
2020-11-23 12:41:05 -06:00
ZmnSCPxj jxPCSnmZ ff090ecfe6 doc/TOR.md: Add missing instructions to add user to Tor group.
Changelog-None

Fixes: #4208
2020-11-23 12:22:10 -06:00
niftynei e4cd5eac28 htlc.h: move NUM_SIDES to define, not enum member 2020-11-23 12:21:33 -06:00
YOSHIDA Masanori ff2535651e lightningd: remove unused pid_fd member in struct lightningd
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-11-23 12:13:28 -06:00
ZmnSCPxj jxPCSnmZ dc83862fc2 doc/PLUGINS.md: Fix typo. 2020-11-23 12:13:09 -06:00
niftynei 66b2bb9a90 Update to v0.9.2 2020-11-20 15:32:38 -06:00
niftynei ce0888d7d6 release: bump to v0.9.2rc2 2020-11-17 11:22:31 -06:00
Rusty Russell 5ae18ecb32 Revert accidental experimental libwally update from f56266c1c1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-17 11:21:58 -06:00
Rusty Russell f56266c1c1 plugins: undeprecate old form of hooks.
This effectively reverts ac93b780d5.

Christian points out that plugins need time before we deprecate
the old options (probably 6 months) as they need to work with
both old and new.

Changelog-Deprecated: **UNDO** plugins: hooks should now be specified using objects, not raw names.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-17 14:42:58 +10:30
Christian Decker 313976e2f4 pay: Cleanup the route applicability checks for channel hints
I previously mistyped the rather lengthy conditions for failures, so
let's dissect it into its smaller components and add rationale behind
the individual parts of the decision.
2020-11-17 14:04:19 +10:30
Christian Decker 4d6b4a0445 pay: Retry the route computation if we could not apply the chanhints
This adds a new state `PAYMENT_STEP_RETRY_GETROUTE` which is used to
retry just that one step, without spawning a completely new
attempt. It's a new state so that modifiers do not act on it twice.

Changelog-Fixed: pay: Improved the performance of the `pay` command considerably by avoiding conflicting changes to our local network view.
2020-11-17 14:04:19 +10:30
Christian Decker 544e110c96 pay: Add a pre-apply check to channel_hint updates
This allows us to atomically update all channel_hints and determine if
we had a collision and therefore should retry.
2020-11-17 14:04:19 +10:30
Christian Decker 83f57ac300 pay: Move the chanhint update up to payment_getroute
We were delaying the channel_hint update till after the `createonion`
call which gave us the same situation we had with concurrent
`getroute` calls. Now we update the hints as soon as the plugins have
had their say in the route construction. If we still fail, either
because a modifier changed the route causing the failure, or because
we interleaved the route computation for multiple parts, we reset the
attempt and retry inline (i.e., without creating a new sub-payment).

Notice that interleaved route computations now only happen if the
modifier makes an async call to some RPC or similar.
2020-11-17 14:04:19 +10:30
niftynei abad494fcf connectd: properly cleanup 'competing' outgoing connections
Coauthored-By: Rusty Russell @rustyrussell
2020-11-16 20:00:51 -06:00
niftynei 7e60ef6974 tests: make connection details correct 2020-11-16 20:00:51 -06:00
niftynei 4a3ee19a22 connectd: Update connection list with new address
If we're already attempting to connect to a peer, we would ignore
new connection requests. This is problematic if your node has bad
connection details for the node -- you can't update it while inflight.

This patch appends new connection suggestions to the list of connections
to try.

Fixes #4154
2020-11-16 20:00:51 -06:00
niftynei cd7d5cdff9 pytest: use the 'contrib' plugin set, not the 'test' set
They're clobbering each other, and the test hangs
2020-11-13 14:28:29 -06:00
Michael Schmoock 183f7698c3 chore: reduce CI load merging a testcase 2020-11-13 14:25:02 -06:00
Michael Schmoock 4134c24cc9 pytest: fix a probable flake on test_channel_opener 2020-11-13 14:25:02 -06:00
Michael Schmoock 90a4476f16 pytest: tests closer on unilateral close 2020-11-13 14:25:02 -06:00
Michael Schmoock 36499e6779 fix: closer set to remote for onchain settlement
Incase we have been offline while a channel was force closed on us we
now set the 'closer' to 'remote' instead of null because this is by far
the most probable reason.

Changelog-None
2020-11-13 14:25:02 -06:00
Michael Schmoock 3e42d08c89 fix: hangup in plugin rescan
This adds a missing plugins_send_getmanifest() call in the rescan function
that lead to a RPC hangup. Not sure though if this is the proper fix.

Changelog-None
2020-11-12 15:27:04 -06:00
niftynei 0f2b3c9179 v0.9.2rc1 - Changelog 2020-11-11 13:18:55 -06:00
niftynei cefb64cd68 htlc_accepted: log BROKEN if both 'failure_msg' + 'failure_onion'
Log an error for incorrect use of API

Suggested-By: @cdecker
2020-11-10 19:03:23 -06:00
fiatjaf 65cdb78bb4 refactor htlc_accepted_hook_deserialize. 2020-11-10 19:03:23 -06:00
fiatjaf c5aa2ea911 failure_onion: a test that only ensures lightningd doesn't crash. 2020-11-10 19:03:23 -06:00
fiatjaf 9e4bed73d9 optional "failure_onion" in reply to htlc_accepted hook.
Changelog-Added: `htlc_accepted` hook can now return custom `failure_onion`.
2020-11-10 19:03:23 -06:00
niftynei 72dc97c0c1 pytest: silence pytest warnings about unregistered 'slow_test' marker
We've been getting a bunch of 'warnings' about an unknown mark. This
silences the warning, by registering the mark as expected.

=========================================================== warnings summary ===========================================================
tests/test_closing.py:152
  /home/niftynei/dev/lightning/tests/test_closing.py:152: PytestUnknownMarkWarning: Unknown pytest.mark.slow_test - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    @pytest.mark.slow_test

tests/test_closing.py:214
  /home/niftynei/dev/lightning/tests/test_closing.py:214: PytestUnknownMarkWarning: Unknown pytest.mark.slow_test - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    @pytest.mark.slow_test

tests/test_closing.py:704
2020-11-10 16:17:04 -06:00
Rusty Russell 9f5f9e15d0 devtools/decodemsg: handle truncated TLVs properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-10 10:52:18 -06:00
Rusty Russell 50f8da85ac gossipd: fix response for large replies.
We tried to send an absurdly-long message, because our limit
was really large, as we were subtracting the tlv length, not
the tlv overhead.

In addition, we were treating the array as a tal object, which
it isn't if the offset is non-zero.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-10 10:52:18 -06:00
Antoine Poinsot 6b347e1895 doc: document output descriptors in hsmtool
Touch a bit about it as a backup/recovery mechanism in the FAQ

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot fc964edc06 pytest: test onchain wallet descriptor import on bitcoind
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot a561360334 pytest: create a wallet for bitcoind if none is present
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot 09fb34c6f3 hsmtool: add a 'testnet' option to 'dumponchaindescriptors'
Actually, it's more complex to translate the xpub descriptor to
testnet because of the descriptor checksum.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot 9f8e2e0502 tools: add a new 'dumponchaindescriptors' command to hsmtool
This adds a command which outputs the two output descriptors
corresponding to our onchain wallet.

This can be useful for an external service to monitor / send fund to our
wallet.

Further, an "xpriv" version of such descriptors could be used to import
onchain funds on a new wallet.

Changelog-Added: lightning-hsmtool: a new command was added to hsmtool for dumping descriptors of the onchain wallet
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot 4f2ae48c54 fuzz: add a fuzz target for common/descriptor_checksum
A small one just to check that we don't crash nor go out of bounds!

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot bf0a2f2b85 fuzz: add a 'to_string' function to libfuzz
We want to use it outside of fuzz-amount

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot 419612c4bd common: add a descriptor checksum computation module
This is stolen from William's clightning-dumpkeys (https://github.com/jb55/clightning-dumpkeys),
itself adapted from 42b66a6b81/src/script/descriptor.cpp (L25)

Co-authored-by: William Casarin <jb55@jb55.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Rusty Russell fa006fd2f7 pyln-testing: require bitcoin v0.20 for PSBT handling, and create wallet for v0.21
With older bitcoind, PSBTs fail:

```
    def test_utxopsbt(node_factory, bitcoind, chainparams):
...
>       psbt = bitcoind.rpc.decodepsbt(funding['psbt'])

tests/test_wallet.py:561:
...
self = <bitcoin.rpc.RawProxy object at 0x7f4ec602e100>, service_name = 'decodepsbt'
args = ('cHNidP8BADMCAAAAAaoMihSVXlpdBHGcJePiroqtwq/b1zu09j8IkTG4OKs7AQAAAAD9////AGYAAAAAAQDeAgAAAAABAefqB6BkZE1/AqXaf36T02a7.../7Stf971PEgvUXgvASECXPTIO6tIVxDih6tfKy6suj6WJhhjycwoaTeuso/AQ8llAAAAAQEfQEIPAAAAAAAWABQB+tkKvNZml+JZIWRyLeSpXr7hZQA=',)
postdata = '{"version": "1.1", "method": "decodepsbt", "params": ["cHNidP8BADMCAAAAAaoMihSVXlpdBHGcJePiroqtwq/b1zu09j8IkTG4OKs7AQ...gvUXgvASECXPTIO6tIVxDih6tfKy6suj6WJhhjycwoaTeuso/AQ8llAAAAAQEfQEIPAAAAAAAWABQB+tkKvNZml+JZIWRyLeSpXr7hZQA="], "id": 1}'
headers = {'Authorization': b'Basic cnBjdXNlcjpycGNwYXNz', 'Content-type': 'application/json', 'Host': 'localhost', 'User-Agent': 'AuthServiceProxy/0.1'}
response = {'error': {'code': -22, 'message': 'TX decode failed PSBT is not sane.: iostream error'}, 'id': 1, 'result': None}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: *** Requires bitcoind v0.20.1 or above ***
2020-11-09 20:05:30 -06:00
positiveblue fa1483a00d `hsm_secret` generation from a seed-phrase
tools: Add `generatehsm` method to hsmtool to derivate BIP32 seeds from a
mnemonic using the BIP39 standard.

The new method uses libwally for the BIP39 to BIP32 derivation. It also
fails if an hsm_secret file already exists, so we do not overwrite
someone else's wallet without noticing.

It allows the use of passphrases, the ECHO mode in the terminal is
disable for higher security.

It currently supports "en", "es", "fr", "it", "jp", "zhs", "zht".

Changelog-Added: hsmtool: `hsm_secret` generation from a seed-phrase following BIP39.
2020-11-09 17:49:24 -06:00
Rusty Russell 9b0af9f046 gossipd: minor cleanups.
Thanks to m-schmook's feedback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 17:37:10 -06:00
Rusty Russell 9a575a98a0 gossipd: simplify large reply code.
We used to create the entire reply, the if it was too big, split in
half and retry.

Now that the main network is larger, this always happens with a full
request, which is inefficient.

Instead, produce a reply assuming no compression, then compress as a
bonus.  This is simpler and more efficient, at cost of sending more
packets.

I also renamed an internal dev var to make it clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 17:37:10 -06:00
Rusty Russell 8db5fb7345 gossipd: new struct to hold scids and timestamps together.
It's not (yet?) compulsory to have the timestamps, but handing them around
together makes sense (a missing timestamp has the same effect as a zero
timestamp).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 17:37:10 -06:00
Rusty Russell 59f23bf23c gossipd: use straight counter instead of bitmap for query_channel_range replies.
The spec (since d4bafcb67dcf1e4de4d16224ea4de6b543ae73bf in March
2020) requires that reply_channel_range be in order (and all
implementations did this anyway).

But when I tried this, I found that LND doesn't (always) obey this,
since don't divide on block boundaries.  So we have to loosen the
constraints here a little.

We got rid of the old LND compat handling though, since everyone should
now be upgraded (there are CVEs out for older LNDs).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Support for receiving full gossip from ancient LND nodes.
2020-11-09 17:37:10 -06:00
Rusty Russell 4745e7e3d0 pytest: make test_hook_dep tests more stable.
I think this is what Travis is having an issue with, but it work
fine locally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Michael Schmoock f3804daabb pytest: simplify test_openchannel_hook_chaining now order is deterministic. 2020-11-09 15:22:33 -06:00