Commit Graph

9358 Commits

Author SHA1 Message Date
Christian Decker 91e406ed6f doc: Document existence of `erring_direction` in `sendpay` result
Fixes #3537
2021-01-08 19:28:30 +01:00
Christian Decker 16801e31e9 configure: Prefer pytest over pytest3
When looking for a pytest executable we should be looking for the ones
that `pip` installs in a virtualenv (`pytest` and `py.test`) before we
look for the ones that `apt` installs system-wide (`pytest3` and
`py.test3`) because these may not be part of the virtualenv that all
other packages are installed in.
2021-01-08 19:28:30 +01:00
niftynei d0dcb7cb42 tests: wait for drop to chain test_onchain_middleman_their_unilateral_in
this was catching the wrong 'sendrawtx exit 0' and consequently
not mining the close tx into a block
2021-01-08 19:28:30 +01:00
Christian Decker a91254de11 pyln: Add ammag key to onion keyset
This was missing, and is required to wrap error responses.
2021-01-08 19:28:30 +01:00
Rusty Russell 1a8978100e invoice: add ctlv option.
This is required if we want to create a "bouncer" plugin (in my copious free time!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
2021-01-08 10:45:56 +01:00
Christian Decker 4b2efd69d7 pay: Fix a crash when waitblockheight times out
Fixes #4309
Changelog-Fixed: pay: Fixed an issue where waiting for the blockchain height to sync could time out.
2021-01-08 09:19:24 +01:00
Christian Decker ca204eacfa pytest: Reproduce waitblockheight timeout issue #4309 2021-01-08 09:19:24 +01:00
Christian Decker 0d1b45dfcc json-rpc: Give `waitblockheight` an error code for timeouts
We need to differentiate timeouts from other results, so make it recognizable.
2021-01-08 09:19:24 +01:00
Christian Decker 8e908ff652 Revert "wallet: Remove in-memory utxoset filter"
This reverts commit c239a7161b.

The goal of c239a716 was to reduce the memory footprint of our
internal UTXO set tracking, and testing against the sqlite3 backend
showed no performance impact. We have since found that the added
roundtrips to the DB server with postgres was having a considerable
performance impact, and backups would also bloat due to the increased
number of queries.

Undoing this change skips the noop updates that were causing this
regression.

Changelog-Fixed: db: Fixed a performance regression during block sync, resulting in many more queries against the DB than necessary.
2021-01-08 12:08:32 +10:30
Rusty Russell 3b7d0e7a62 common/json: make json_scan return an error string.
This makes for more useful errors.  It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 53582a0f81 lightningd/bitcoind: use json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 11a4d46c44 chaintopology: fix crash if bcli doesn't include errmsg.
Interestingly, we required that "sendrawtx" include "errmsg" field even
on success, otherwise we crashed in broadcast_remainder.

We only actually insist on an "errmsg" if success is false.  And this
logic here is weird (the !success) was added by darosior in
947f5ddde1, which makes the msg checks redundant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell fd7d1a1cc0 common: remove now-unused json_delve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 09b18bf64f libplugin: replace rpc_delve with rpc_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell b61da8c5a9 plugins: use json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 35e8949df3 plugins/bcli: convert to json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell a5befb0072 common: enhance json_scan with simple array helpers.
In several places we want to access the first element of an array.
This uses a '[indexnum:xxx]' form which is a bit weird, but works similarly
to the way we specify member matches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 553daf17e0 common/json: new helper to scan JSON: json_scan.
This takes a JSON-style format string, and does intelligent parsing,
removing a lot of boilerplate from code which needs to deal with JSON.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 24b6132ee8 common: add length-limited versions of json_streq and json_get_member.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 2b6cdfeb5a json_to_psbt: fix API to match other allocating json_to functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Christian Decker 70410b8ee8 pytest: Test LightningRpc and plugin command notification support 2021-01-07 08:32:40 +01:00
Christian Decker b6650425b9 pyln: Add notifications support to LightningRpc
Changelog-Added: pyln-client: Added support for command notifications to LightningRpc via the `notify` context-manager.
2021-01-07 08:32:40 +01:00
Christian Decker 84b3653606 pyln: Add command notification support for plugins
Changelog-Added: pyln-client: Plugin methods can now report progress or status via the `Request.notify` function
2021-01-07 08:32:40 +01:00
Christian Decker a1d33c3781 pytest: Add a test for `listinvoices`
We now have some more logic in the query, so let's test it
exhaustively.
2021-01-06 18:48:06 +01:00
Christian Decker 2981184fac docs: Document `bolt11` and `payment_hash` arguments of listinvoices 2021-01-06 18:48:06 +01:00
Christian Decker 30abda2750 jsonrpc: Allow filtering via bolt11 or payment_hash in listinvoices
A user reported that it is sometimes cumbersome to search an invoice
based on the payment hash or the bolt11 string in the full list, which
may be required when we don't have the label available.

This adds support for querying / filtering based on the `payment_hash`
or `bolt11` string.

Changelog-Added: JSON-RPC: `listinvoices` can now query for an invoice matching a `payment_hash` or a `bolt11` string, in addition to `label`
2021-01-06 18:48:06 +01:00
Rusty Russell 106afd005f channeld: stop taken() leak.
I saw this message:

```
lightning_channeld: outstanding taken(): channeld/channeld.c:3087:blinding
lightning_channeld: outstanding taken(): channeld/channeld.c:3087:blinding
lightning_channeld: outstanding taken(): channeld/channeld.c:3087:blinding
```

The caller does take(blinding), but blinding can be NULL.  We should
move the code around to do the take() earlier anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-06 15:34:02 +01:00
niftynei 0726d2912e df-mfc: remove guard-rails for failures
This will cause blow ups for v2 multifundchannel attempts with failures,
but allows us to return the expected errors for single-shot
fundchannel attempts.

Error handling is coming, i promise
2021-01-06 14:50:02 +01:00
niftynei b040a893ed mfc: avoid off-by-one rounding errors when finding change fee
Since we round down in `amount_tx_fee`, find the change fee as the
difference between what we've already paid and what the combined/total
fee would be if the change weight were also added.
2021-01-06 14:50:02 +01:00
niftynei 3793687550 mfc-df: if the psbt we've got has inputs from our peer, remove them
We only want to attempt to unreserve inputs that are ours (otherwise we
log a broken error)
2021-01-06 14:50:02 +01:00
niftynei 6e3c517611 mfc-df: if there's an error with a openchannel_signed, fail
Ideally we'd 'cure' the error and re-attempt, except that if this was a
bitcoin-backend 'failure to broadcast' then it really needs user
intervention to figure out what's wrong -- it's possible that the
peer successfully broadcast the transaction
2021-01-06 14:50:02 +01:00
Antoine Poinsot a4f07a3e78 tests/fuzz: add a fuzz target for hsm_encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 506d524508 libfuzz: add a NULL-termination in to_string
It's more useful if we actually want to use the output as, well, a
string..

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 45bb1bfa3c hsmd: cleanup encrypted hsm_secret detection
This makes use of the constant defined in the previous commits to more
accurately detect plaintext, encrypted, and invalid seeds. We now error
on invalid seeds.

Changelog-changed: hsmd: we now error at startup on invalid hsm_secret
Changelog-changed: hsmtool: all commands now error on invalid hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot d2a903992c hsmd: regroup hsm_secret decryption logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot c6bc22b0f5 hsmd: group hsm_secret encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot a441485a35 lightningd: regroup hsm_secret password input logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 917f78a4f8 lightningd: group hsm_secret encryption key derivation
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 38204f9b6d hsmtool: use errx() instead of err() everywhere
errx() was printing the confusing errno as well ("Error could not [...] :Success")

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Rusty Russell d41b383cf7 pyln-testing: suppress mypy on FileLock
3.0.12 here:
```
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-06 12:15:39 +01:00
niftynei 0c9072272f df-plugin: add `signonly` to signing in df_accepter
We're the accepter, so we look up which inputs we need to sign and only
sign those.
2021-01-05 19:12:00 +01:00
niftynei d98bd39d60 dual_open: neaten spacing for macro 2021-01-05 19:12:00 +01:00
niftynei 55bbdf1866 df: update an error message to give more info 2021-01-05 19:12:00 +01:00
niftynei 713bb7491b json: allow helper to take/free psbt
Makes things a bit cleaner elsewhere
2021-01-05 19:12:00 +01:00
niftynei e3a1d1a7f3 pyln-client: to_whole_satoshi returns the rounded up satoshi value
A fractional satoshi value isn't really useful; rounding up loses
precision but that's why you called "whole satoshi", wasn't it?

Changelog-Changed: pyln-client: Millisatoshi has new method, `to_whole_satoshi`; *rounds value up* to the nearest whole satoshi
2021-01-05 19:12:00 +01:00
niftynei cce0020568 pyln-client: add missing `min_witness_weight` to fundpsbt/utxopsbt
We need to use it for the 'df_accepter' plugin, so we get the feerate
correct.

Changelog-Added: pyln-client: `fundpsbt`/`utxopsbt` now support `min_witness_weight` param
2021-01-05 19:12:00 +01:00
Antoine Poinsot 19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot dd1def335c hsmtool: reuse read_stdin_pass for `generatehsm` passphrase
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00