Commit Graph

89 Commits

Author SHA1 Message Date
Rusty Russell c1ee32027d pyln-testing: check the request schemas.
This means suppressing schemas in some places too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell bf4d9e30d2 fundpsbt: deprecate reserve=true/false usage.
Make it always a number; this makes the JSON request specification
simpler.  We allowed a number since v0.10.1.

(reserve=True is the default anyway, so usually it can be omitted:
reserve=False becomes reserve=0).

Changelog-Deprecated: JSON-RPC: `fundpsbt`/`utxopsbt` `reserve` must be a number, not bool (for `true` use 72/don't specify, for `false` use 0).  Numbers have been allowed since v0.10.1.
2022-04-02 09:46:01 +10:30
Vincenzo Palazzo 53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Rusty Russell d0c7e18995 bitcoind: importmulti fails (bitcoin master), use importdescriptors
But this requires a watch-only wallet, and python-bitcoinlib doesn't support
multiple wallets, so we need to unload the original one, but then we need
to generate a block, so that can't generate a new address, so we need
an address arg to generate_block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30
niftynei 4dafeede5c coin moves: notify when we make deposits to external accounts
The blockheight is zero though, since these aren't included in a block
yet.

We also don't issue an 'external' deposit event if we can tell that the
address you're sending to actually belongs to our wallet (we'll issue a
deposit event when it gets included in a block)
2022-01-26 13:34:45 +10:30
niftynei 29c6718297 coin_mvt: record new 'fees' field on htlc channel moves
We record the amount of fees collected for a routed payment. For
simplicity's sake on the data agg side, we record the fee payment on
*BOTH* the incoming htlc and the outgoing htlc. Note that this results
in double counting if you add up the fees from both an in-routed and
out-routed payment.
2021-12-28 04:42:42 +10:30
niftynei d2c4d4aec2 coin_mvts: rewrite how onchain events are recorded, update tests
The old model of coin movements attempted to compute fees etc and log
amounts, not utxos. This is not as robust, as multi-party opens and dual
funded channels make it hard to account for fees etc correctly.

Instead, we move towards a 'utxo' view of the onchain events. Every
event is either the creation or 'destruction' of a utxo. For cases where
the value of the utxo is not (fully) debited/credited to our account, we
also record the output_value. E.g. channel closings spend a utxo who's
entire value we may not own.

Since we're now tracking UTXOs onchain, we can now do more complex
assertions about the onchain footprint of them. The integration tests
have been updated to now use more 'chain aware' assertions about the
ending state.
2021-12-28 04:42:42 +10:30
Vincenzo Palazzo 43ff949ea7 lightningd: support hsm error code
Suggested-by: Rusty Russell
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Changed: Support hsm specific error error code in lightning-cli
2021-12-15 12:24:54 +10:30
Rusty Russell 5b644a2319 pytest: ensure hsmtool tests do complete writes.
This may be causing the test flakiness we sometimes see in CI.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 20:19:03 +09:30
Rusty Russell 0a99b8c64c listtransactions: don't create a msat field called "satoshis".
That's a terrible, terrible idea.  (Documentation comes in later patch
which has the schema).

Also, blockheight is a u32, so simplify.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listtransactions` `outputs` `satoshis` field (use `msat` instead).
2021-06-25 09:49:33 +09:30
Antoine Poinsot e19b67f6da qa: test hsm encryption password provided from stdin
Test that it roundtrips with the non-stdin way, in order to make sure we
don't introduce a discrepancy between the two.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-06-18 10:40:07 +09:30
Rusty Russell 3cf98085d4 tests: test future segwit versions for withdraw / close.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:20 +09:30
Antoine Poinsot b5aaee6679 Revert "pytest: Skip hsm encryption test if we don't have a TTY"
This reverts commit 2b12cac31e. There is
no need to skip the test in this case, and it seems to simply be an artifact
of CI-debugging hell :)

Changelog-None
2021-06-03 11:27:16 +09:30
Rusty Russell f24dc9173d wallet: add "reserved_to_block" field to listfunds.
We already have this field in reserveinputs and unreserveinputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listfunds` has a new `reserved_to_block` field.
2021-05-26 15:08:01 +09:30
Rusty Russell b352df4179 pytest: make it work with latest bitcoind master branch.
They seem to have changed the JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-24 13:57:16 +09:30
niftynei ea95ad9c12 fund/utxopsbt: flag 'excess_as_change' to add a change output for excess
In the case where you want a PSBT and also want the output to be added
as a change address, use `excess_as_change` = true.

Generates a change address to use. If you want to pay the excess
elsewhere, you will have to add separately.

Changelog-Added: JSON-RPC: Add new parameter `excess_as_change` to fundpsbt+utxopsbt
2021-02-04 13:25:34 -06:00
Christian Decker 5ecaff65ee pytest: Give each run of the hsmtool its own pty 2021-01-29 10:29:09 +10:30
Christian Decker 542f3225e3 pytest: Parameterize process waits for hsmtool calls
We were sometimes waiting only 5 seconds, which is way too short on a
heavily loaded machine such as CI. Making it 30 seconds and collecting
it in a single place so we can adjust more easily.
2021-01-29 10:29:09 +10:30
Christian Decker 2b12cac31e pytest: Skip hsm encryption test if we don't have a TTY 2021-01-29 10:29:09 +10:30
Antoine Poinsot 56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot 1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker 71fafd23fd pytest: Add a test to reproduce #4258 2020-12-09 06:56:21 +10:30
Antoine Poinsot fc964edc06 pytest: test onchain wallet descriptor import on bitcoind
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
niftynei e2a6fd7112 common: pull up `param_psbt`
Usable other places, not just in wallet
2020-10-20 12:50:31 +10:30
Rusty Russell 2d897be10a pytest: fix experimental test.
Broken back in 3b8c0a7397 which
changed the behavior of funding not to mark as immediately spent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-08 11:00:23 +10:30
niftynei eab7fcf8e5 test_utxopsbt: account for fee output diff when elements 2020-09-10 12:31:36 +09:30
niftynei 4311c658d0 test_txprepare: skip over fee outputs (elements) 2020-09-10 12:31:36 +09:30
niftynei 250af67908 elements: run sign + send psbt check for elements tests
Now that we're using an 'elements' witness_utxo and issue 3998 has been
resolved, this should work as expected.
2020-09-10 12:31:36 +09:30
ZmnSCPxj jxPCSnmZ c6186829df tests/test_wallet.py: Simple multiwithdraw tests. 2020-09-09 20:36:08 +09:30
Christian Decker c7f5889a32 pytest: Skip test_sign_and_send_psbt for elements
This test is flaky because the generated PSBT seems to not have the change
output adjusted, or it is missing.

Tracking-Issue: ElementsProject/lightning#3998
2020-09-09 20:17:26 +09:30
niftynei 437ca83fa5 elementsd: add a fee output to a fund/utxopsbt for elements transactions
fundpsbt / utxopsbt create a (typically) output-less PSBT,
however for elements we require the fees to be encapsulated in an
output.

this patch updates fundpsbt / utxopsbt to add a fee output for elements
transactions. includes test updates.

Fixes #3998
2020-09-09 16:50:14 +09:30
Rusty Russell 83298c030a wallet: switch over to withdraw in module, remove lots of unused code.
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.

This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined.  Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
2020-09-08 10:14:42 +09:30
Rusty Russell 6b2a3f4dfb txprepare: remove old code, switch to plugin.
Some minor phrasing differences cause test changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: txprepare reservations stay across restarts: use fundpsbt/reservepsbt/unreservepsbt
Changelog-Removed: txprepare `destination` `satoshi` argument form removed (deprecated v0.7.3)
2020-09-08 10:14:42 +09:30
Rusty Russell 843d3d6745 pytest: remove test_txprepare_restart
This tested that we unreserved everything on restart, which we no
longer do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 7435d50970 signpsbt: add signonly parameter to restrict/enforce what inputs to sign.
This is an extra safety check for dual funding, where we only want to sign
the inputs we provided!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `signpsbt` takes an optional `signonly` array to limit what inputs to sign.
2020-08-18 11:38:25 -05:00
Rusty Russell aab3808668 utxopsbt: let caller specify locktime, add tests and python binding.
Changelog-Added: JSON-RPC: `utxopsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 11:38:25 -05:00
Rusty Russell 14baaaa8ba fundpsbt: let caller specify locktime.
Required for dual funding where the opener sets it.

Changelog-Added: JSON-RPC: `fundpsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 11:38:25 -05:00
Rusty Russell 0fa5cde782 pytest: fix occasional error in test_sign_and_send_psbt:
The modded_psbt is not necessarily invalid:

       # Try a modified (invalid) PSBT string
        modded_psbt = psbt[:-3] + 'A' + psbt[-3:]
        with pytest.raises(RpcError, match=r"should be a PSBT, not"):
>           l1.rpc.signpsbt(modded_psbt)
E           AssertionError: Pattern 'should be a PSBT, not' not found in 'RPC call failed: method: signpsbt, payload: {'psbt': 'cHNidP8BAM0CAAAABJ9446mTRp/ml8OxSLC1hEvrcxG1L02AG7YZ4syHon2sAQAAAAD9////JFJH/NjKwjwrP9myuU68G7t8Q4VIChH0KUkZ5hSAyqcAAAAAAP3///8Uhrj0XDRhGRno8V7qEe4hHvZcmEjt3LQSIXWc+QU2tAEAAAAA/f///wstLikuBrgZJI83VPaY8aM7aPe5U6TMb06+jvGYzQLEAQAAAAD9////AcDGLQAAAAAAFgAUyQltQ/QI6lJgICYsza18hRa5KoEAAAAAAAEBH0BCDwAAAAAAFgAUqc1Qh7Q5kY1noDksmj7cJmHaIbQAAQEfQEIPAAAAAAAWABS3bdYeQbXvBSryHNoyYIiMBwu5rwABASBAQg8AAAAAABepFD1r0NuqAA+R7zDiXrlP7J+/PcNZhwEEFgAUKvGgVL/ThjWE/P1oORVXh/ObucYAAQEgQEIPAAAAAAAXqRRsrE5ugA1VJnAith5+msRMUTwl8ocBBBYAFMrfGCiLi0ZnOCY83ERKJ1sLYMY8AAAA='}, error: {'code': -1, 'message': 'No wallet inputs to sign'}'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-13 10:39:56 +09:30
Rusty Russell 73a5f5b313 fundpsbt: make parameters more usable.
fundpsbt forces the caller to manually add their weight * feerate
to the satoshis they ask for.  That means no named feerates.

Instead, create a startweight parameter and do the calc for them
internally, and return the feerate we used (and, while we're at it,
the estimated final weight).

This API change is best done now, as it would otherwise have to
be appended as a parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-17 13:53:46 +02:00
Rusty Russell 371cabf976 txprepare: revert 1fb9a078b6 (`psbt` field)
We're actually going to deprecate this, so don't add new features!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: ***REMOVE*** JSON-API: `txprepare` returns a psbt version of the created transaction
2020-07-15 18:49:02 +09:30
Rusty Russell e85c895c5b pytest: restore test_sign_and_send_psbt.
It uses reservations heavily, and assumed we generated change, etc.
It's now a simpler test, in many ways.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell 31d7e013bd pytest: test fundpsbt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell 4ee527a59c pytest: test reserve and unreserve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell 56ea215ba0 wallet: new JSON commands reserveinputs and unreserveinputs.
reserveinputs marks UTXOs reserved for 12 hours, so we won't select them
for spending: unreserveinputs marks them available again.

Exposes param_psbt() for wider use.

Disabled the test_sign_and_send_psbt since we're altering the API;
the final patch restores it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
niftynei bea05b1250 elements,tests: skip over fee type outputs
They don't contain addresses
2020-07-13 11:37:24 +09:30
niftynei eec5dfe6aa elements: update output amounts for elements test
The amount we expect back, post change, is different for an elements tx
2020-07-13 11:37:24 +09:30
Rusty Russell 7aa8ffa2a0 bitcoin: add weight calculation helpers.
These are pulled from wallet/wallet.c, with the fix now that we grind sigs.

This reduces the fees we pay slightly, as you can see in the coinmoves changes.

I now print out all the coin moves in suitable format before we match:
you only see this if the test fails, but it's really helpful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00