Commit Graph

901 Commits

Author SHA1 Message Date
Rusty Russell bf5e99403e utxopsbt: make default to only allow unreserved utxos.
This more closely mirrors fundpsbt (which will only select unreserved ones)
but you can turn it off if you want to e.g. rbf in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: New low-level command `utxopsbt` to create PSBT from existing utxos.
2020-08-18 12:52:41 +09:30
Rusty Russell b07327a687 utxopsbt: new command to create PSBT from given utxos.
It's *possible* to do this using various RPC calls, but it's
unfriendly:

1. Call getinfo to get the current block height.
2. Call listfunds to map the UTXOs.
3. Create the PSBT and hope you get all the fields correct.

Instead, this presents an interface just like `fundpsbt`, with identical
returns.

I think it's different enough to justify a new command (though it
shares much internally, of course).

In particular, it's now quite simple to create a command which uses
specified utxos, and then adds more to meet any shortfall.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 12:52:41 +09:30
Rusty Russell 754765c139 utxo: keep flag to recognize to-remote option_anchor_outputs closes.
We need to remember this in the db (it's a P2WSH for option_anchor_outputs),
and we need to set nSequence to 1 to spend it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell d8d1014ca4 channeld: implement htlc sig checking changes for option_anchor_outputs.
This is best done by passing `struct bitcoin_signature` around instead
of raw signatures.  We still save raw sigs to the db, and of course the
wire protocol uses them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell 85e3b43176 channeld, openingd: take into account option_anchor_outputs for fees.
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell edf60b4f9e openingd: remember if we negotiated option_anchor_outputs, to put in the db.
And hand it through to channeld just like option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell b093856d0c db: add option_anchor_outputs entry to channel table.
This is the same way we handle option_static_remotekey, which
is also sticky (if negotiated at opening time, it always applies).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell bcd632f2ae listfunds: add "redeemscript" field.
You'd need this if you ever wanted to make your own PSBT.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `listfunds` now has a `redeemscript` field for p2sh-wrapped outputs.
2020-08-13 12:37:02 -05:00
Rusty Russell cf25ca11b8 sendpsbt: annotate transactions in wallet.
This is what txsend does, only we have a psbt so we have
to change the db interface to take a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-13 12:37:02 -05:00
Rusty Russell 601fc805d7 fundpsbt: simplify the logic a little.
It keeps multiple different variables around the loop, but a simple
"are we done yet?" helper makes this clearer and reduces special
cases or all-vs-target.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:39:24 +09:30
Rusty Russell 600d0a4a1d psbt: make psbt_from_b64 more conventional.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:39:24 +09:30
Rusty Russell c1df8d586d utxo: remove unused scriptSig field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:39:24 +09:30
Jon Griffiths 95d3d65c62 wally: update to the latest wally version
Includes:
psbt: Use renamed functions for new wally version
psbt: Set the transaction directly to avoid script workarounds
psbt: Use low-S grinding when computing signatures
tx: Use wally_tx_clone from libwally now that its exported

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
niftynei d87f31f9a8 utxo: clean up NULL handling of scriptpubkey
Now that we're *guaranteed* to have a scriptpubkey entry in the
database, we remove the NULL handling for it.
2020-07-29 13:13:46 +02:00
niftynei 90b393ca1a hsmd/db: backfill pubkey information so that psbts signing works
the way we use PSBTs to sign things requires that we have the
scriptpubkey available on the utxo so we can populate the witness-utxo
field with it.

this causes problems if we don't already have the scriptpubkey cached in
the database, as in *some* cases we require a round trip to the HSM to
populate them

to get over this hump, we backfill any and all missing scriptpubkey
information for the utxo's that we hold in our wallet.

this will allow us to clean up the NULL handling of missing
scriptpubkeys.
2020-07-29 13:13:46 +02:00
niftynei 65c2bac2f3 hsmd/wallet: pass the bip32_key down into migrations
we're about to add a migration that requires access to the bip32_key
in order to calculate missing scriptpubkeys.

prior to this patch, we don't have access to the bip32 key in the db
migration, as it's set on the wallet but after the db migrations are
run.

here we patch it through so that every migration can access it
2020-07-29 13:13:46 +02:00
Christian Decker 1da977a04c db: Guard against accessing NULL partid and total_msat
These were spamming my logs and could result in misleading results being
returned on `listpays` and `listsendpays`.
2020-07-28 16:17:39 +09:30
Rusty Russell 73a5f5b313 fundpsbt: make parameters more usable.
fundpsbt forces the caller to manually add their weight * feerate
to the satoshis they ask for.  That means no named feerates.

Instead, create a startweight parameter and do the calc for them
internally, and return the feerate we used (and, while we're at it,
the estimated final weight).

This API change is best done now, as it would otherwise have to
be appended as a parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-17 13:53:46 +02:00
Rusty Russell 371cabf976 txprepare: revert 1fb9a078b6 (`psbt` field)
We're actually going to deprecate this, so don't add new features!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: ***REMOVE*** JSON-API: `txprepare` returns a psbt version of the created transaction
2020-07-15 18:49:02 +09:30
Rusty Russell 10e62af7c7 fundpsbt: add reserve arg.
It's easier for us to call it atomically than have the user loop and
retry!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell bd19ec2292 fundpsbt: new JSON API to gather UTXOs.
Technically, they could do this themselves, but it's much nicer to have one
place to do it (and it makes sure we get the required information into the
PSBT, which is actually not entirely accessible through listfunds, as that
doesn't want to consult with the HSM for close outputs).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON RPC: new low-level coin selection `fundpsbt` routine.
2020-07-15 18:49:02 +09:30
Rusty Russell be17a9392f reserveinputs: add exclusive flag.
This is the normal case: you only want to reserve inputs which
are not already reserved.  This saves you iterating through the
results and unreserving some if you weren't exclusive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell 56ea215ba0 wallet: new JSON commands reserveinputs and unreserveinputs.
reserveinputs marks UTXOs reserved for 12 hours, so we won't select them
for spending: unreserveinputs marks them available again.

Exposes param_psbt() for wider use.

Disabled the test_sign_and_send_psbt since we're altering the API;
the final patch restores it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell 27b48959d7 wallet: add wallet_find_utxo().
This is a new fundamental routine to obtain UTXOs from the database.

It's not the most efficient approach, as it returns a single UTXO at a
time, but it can consolidate all our UTXO handling (becoming more
complex by the reservation timeout logic).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell 50ff0b26fd wallet: explicit routines to reserve/unreserve a UTXO.
These keep the struct utxo in sync with the database, explicitly:
these will be the only places where utxo->status is set.

The old routines will be removed at the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
niftynei 7ebdc14397 utxos: add a 'reserved_til' marker for utxos
Allow a utxo to be reserved until explicitly unreserved or until a timer
runs out. Currently unused.

We explicitly do not unreserve these at startup.
2020-07-15 18:49:02 +09:30
niftynei f5f85b389d elements,pset: populate elements specific data for PSBTs
PSETs have a bit different requirements. The witness_utxo needs
the asset tag + values, and these should also be added to the PSET
struct separately as well. To do this, we create a new 'init' method for
elements inputs, which takes care of the elements specific things.
2020-07-13 11:37:24 +09:30
niftynei 14de198bd1 wally-tx: add type-to-string for a wally-tx
and then use it to print out things
2020-07-13 11:37:24 +09:30
Rusty Russell 94de18ace7 listfunds: add scriptpubkey if it's known.
(Which it is, since 0.7.3).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON: `listfunds` now has a 'scriptpubkey' field.
2020-07-08 21:07:20 +02:00
Rusty Russell a9427f1a8d bitcoin/feerate: new exposure for feerate parsing outside lightningd.
This exposes the numeric part of param_feerate() as param_feerate_val().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell e12defa7fb wallet: no longer expose wallet_add_utxo function.
All users are now internal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell 4271fc8652 wallet: add explicit API for onchaind to register UTXOs.
This is the only place outside the wallet code where we create
a 'struct utxo', so it makes sense for us to move that logic inside
the wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell 1708fb91e5 wallet: clean up json output creation, part 2.
It looked like we weren't printing the address on closing outputs.

But we are, because the 'scriptPubkey' field is in the 'outputs' db
table since 0.7.3 (66a47d2761).

So make the logic clearer, and remove a completely bogus comment (UTXOs
with closing_info are definitely spendable!).

We export the json_add_utxos() for future use, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell 5b82271187 wallet: clean up json output creation, part 1.
We're not allowed to command_fail() once we've started json_success.

That's OK, because encoding a known output can only fail if something is
badly, badly wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell 7aa8ffa2a0 bitcoin: add weight calculation helpers.
These are pulled from wallet/wallet.c, with the fix now that we grind sigs.

This reduces the fees we pay slightly, as you can see in the coinmoves changes.

I now print out all the coin moves in suitable format before we match:
you only see this if the test fails, but it's really helpful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
ZmnSCPxj jxPCSnmZ d0c85033d2 wallet/walletrpc.c: `txprepare`d transactions now use current tip blockheight by default.
Changelog-Changed: `txprepare` now prepares transactions whose `nLockTime` is set to the tip blockheight, instead of using 0. `fundchannel` will use `nLockTime` set to the tip blockheight as well.
2020-07-01 15:17:33 +00:00
Christian Decker 7b899da801 db: Retrieve peer ID if it exists or create the peer if not
We were assuming `wallet_channel_insert` that there cannot be a matching peer
if our in-memory representation isn't bound to it (`dbid == 0`). If we then
attempt to create the peer, and we already had one it'd cause a unique
constraint violation. As far as I can tell this could end up happening if we
have an uncommitted channel, and then exited without cleanup (`tal_destructor`
on the uncommitted channel not running). This could then leave the peer in the
DB. This is because the constraint that every peer has at least one channel is
not enforce at DB level, but rather in destructors that may or may not run.

Changelog-Fixed: Fixed a failing assertion if we reconnect to a peer that we had a channel with before, and then attempt to insert the peer into the DB twice.
2020-07-01 09:22:29 +09:30
Rusty Russell ae0cccb293 wallet: fix memleak if we get shut down before finishing utxo cleanup.
"backtrace": [
                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
                       "wallet/wallet.c:154 (wallet_stmt2output)",
                       "wallet/wallet.c:275 (wallet_get_utxos)",
                       "wallet/wallet.c:3792 (wallet_clean_utxos)",
                       "lightningd/lightningd.c:914 (main)"
                   ],
                   "label": "wallet/wallet.c:154:struct utxo",
                   "parents": [
                       "wallet/wallet.c:273:struct utxo*[]"
                   ],
                   "value": "0x24c1be8"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-01 09:19:39 +09:30
ZmnSCPxj jxPCSnmZ d14460ef3d wallet/walletrpc.c: Show input annotations for inputs.
Changelog-None
2020-06-30 17:59:17 +02:00
niftynei 9830c94778 rpc: new signpsbt + sendpsbt rpcs
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
2020-06-29 16:10:05 +02:00
niftynei fd8a716695 wallet: have wallet_extract_outputs take wally_tx, not bitcoin_tx
With the incursion of PSBTs, we're moving away from bitcoin_tx
2020-06-29 16:10:05 +02:00
niftynei 103dce63ef reserve/unreserve input: new RPC commands for reserving inputs/outputs
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.

Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.

Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
2020-06-29 16:10:05 +02:00
niftynei b90be4f6c8 prepare-tx: pass back the feerate, as json_tx_prepare sometimes sets it
Unused here, but we'll use it in the next commit so that we can always
pass back the effective / used feerate to the caller of `reserveinputs`

This makes opening a channel much easier if we've internally determined
the feerate
2020-06-29 16:10:05 +02:00
niftynei 0bd0de54fa psbt: have withdraw_tx use psbt's to create signed txs
this will allow us to add inputs that aren't ours to a tx that we sign
and finalize
2020-06-23 14:49:32 +02:00
niftynei 85dca84327 psbt-json: remove reliance on bitcoin_tx, use straight wally_psbt struct 2020-06-23 14:49:32 +02:00
niftynei c3ae44e296 psbt: don't crash if we can't add a partial sig
instead return a boolean indicating the success/failure of a sig set
2020-06-23 14:49:32 +02:00
lisa neigut 16656a85cf withdraw: refactor change output handling
We're not using the change_outnum for withdraw tx's (and the way
we were calculating it was broken as of the addition of 'multiple
outputs'). This removes the change output knowhow from withdraw_tx
entirely, and pushes the responsibility up to the caller to
include the change output in the output set if desired.

Consequently, we also remove the change output knowhow from hsmd.
2020-06-23 14:49:32 +02:00
niftynei 2900da6112 migrations: ignore channels that don't have a peer_id
We erase peer data after the last channel close transaction for that
peer is 100 blocks deep. We were failing to finish the migration because
the peer_id lookup on these was failing.

Now we ignore any channel with a null peer_id.

Fixes #3768
2020-06-17 14:30:41 +02:00
niftynei 431463b57a listfunds: also list reserved outputs
Currently 'listfunds' lies, a teensy eeinsy bit, in that it doesn't list
all of the funds in a wallet (it omits reserved wallet UTXOs). This
change makes the reserved outputs visible by listing them in the
'outputs' section along with a new field, 'reserved', which denotes the
UTXO's state

Changelog-Changed: JSON-RPC: `listfunds` 'outputs' now includes reserved outputs, designated as 'reserved' = true
2020-06-16 15:58:58 +02:00
niftynei bb589e0eaf psbt: database migration for converting last_tx to a psbt
We update the `last_tx` in `channels` to be psbt format, instead
of a linearized transaction.

We need the amount of the input populated, which we have since
this is the 'funding' amount. Ideally we'd also populate the funding
scriptPubkey, but to do that we'd need to access the HSM module to fetch
our local funding pubkey, which isn't initialized at the time that the
database migrations are run.

Since the only field the HSM uses currently when signing these is the
amount field, it's ok to just leave it out.

needs a test!
2020-06-11 13:13:13 +02:00