Commit Graph

384 Commits

Author SHA1 Message Date
niftynei f8c4cc73ae mfc: use declared constants
reduce, reuse, recycle for a greener world
2020-10-21 11:24:16 +10:30
niftynei e66937e012 mfc: Add a `commitment_feerate` optional parameter to multifundchannel
Technically there *are* two feerates that we need to know:
  - the feerate to use for the funding transaction, and
  - the feerate to tell our peer to use for our commitment txs/htlc txs

As written, `multifundchannel` uses the same feerate for both. This
optional parameter will allow us to differentiate between the two, which
will be exceedingly handy for anchor output worlds. ;)

FIXME: test this

Changelog-Added: JSON API: `multifundchannel` has a new optional argument, 'commitment_feerate', which can be used to differentiate between the funding feerate and the channel's initial commitment feerate
2020-10-21 11:24:16 +10:30
Rusty Russell b470ae2c73 plugins/libplugin-pay: use gossmap.
This is a fairly direct translation.  Even so, it should be faster in
most cases, and and we can do more sophisticated things if we want.

This also handles disabled channels better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.
2020-10-21 08:58:34 +10:30
Rusty Russell eadf2c91fe libplugin-pay: incorporate gossip store.
So we can use this for routing determinations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 92f2461b5d plugins/pay: fix leak on failed new payments.
Start with attaching the payment to cmd (in case of failure), then steal
onto the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
niftynei 4926c25bb5 fundchannel: take a 'close_to' address
Finally, extends the 'close_to' functionality up to the flagship 'open a
channel' command.

Changelog-Added: JSON-API `fundchannel` now accepts an optional 'close_to' param, a bitcoin address that the channel funding should be sent to on close. Requires `opt_upfront_shutdownscript`
2020-10-16 13:51:57 +10:30
niftynei 628df74e26 mfc: allow a 'close_to' address to be passed in
This will allow us to let `fundchannel` handle a close to address, also
2020-10-16 13:51:57 +10:30
Rusty Russell 6195d953cc plugins: use "slow" feerate for mutual close negotiation.
We're rarely in a hurry here, and bitcoind is aggressive with fees.
You can always spend this output if you really have to, using CPFP.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: mutual closing feerate reduced to "slow" to avoid overpaying.
2020-10-13 20:53:34 +02:00
Rusty Russell 7260d9ea3d plugins: generate list of plugins more atomically, respect V=1 and --quiet.
I got a corrupt file, which looked like multiple concurrent attempts
to build it.  So instead, build it in one command, but also use
VERBOSE so we print correctly with V=1 (and --quiet).

Also move into plugins/ where it logically belongs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 13:19:59 +02:00
Rusty Russell 924cc04bd2 bolt11: have caller supply preferred chain.
This lets us distinguish testnet from signet invoices, since they
have the same prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 09:24:14 +09:30
Antoine Poinsot 15adcc915f Remove varint typedef for bigsize
It's not part of the spec anymore

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-23 16:30:36 +09:30
Rusty Russell 607075a3d4 bitcoin/psbt: wallt_tx_output needs a tal ctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 480f671e91 bitcoin/psbt: psbt_txid needs a tal ctx.
It returns a wally_tx; it's an anti-pattern not to hand in a tal context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 1deba1c8e4 plugins/pay: handle case where amount_msat is 'null'.
Deprecated, but this can happen:

```
==1578== Conditional jump or move depends on uninitialised value(s)
==1578== at 0x12B30E: amount_msat_add (amount.c:224)
==1578== by 0x11270B: add_amount_sent (pay.c:1734)
==1578== by 0x112D89: listsendpays_done (pay.c:1831)
==1578== by 0x114F4B: handle_rpc_reply (libplugin.c:555)
==1578== by 0x115704: rpc_read_response_one (libplugin.c:685)
==1578== by 0x115821: rpc_conn_read_response (libplugin.c:705)
==1578== by 0x148E40: next_plan (io.c:59)
==1578== by 0x1499BD: do_plan (io.c:407)
==1578== by 0x1499FB: io_ready (io.c:417)
==1578== by 0x14BBC1: io_loop (poll.c:445)
==1578== by 0x117A82: plugin_main (libplugin.c:1322)
==1578== by 0x113ABC: main (pay.c:2096)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-11 12:48:48 +09:30
ZmnSCPxj jxPCSnmZ 0eb1e7e0ca plugins/libplugin-pay.c: Add new payee_incoming_limit to limit number of HTLCs based on payee connectivity.
Fixes: #3926

(probably)

Changelog-Fixed: pay: Also limit the number of splits if the payee seems to have a low number of channels that can enter it, given the max-concurrent-htlcs limit.
2020-09-10 16:50:52 +09:30
ZmnSCPxj jxPCSnmZ deced56344 plugins/libplugin-pay.c: Add facility to have paymods request lowering of the estimated max HTLCs. 2020-09-10 16:50:52 +09:30
ZmnSCPxj jxPCSnmZ d15717b576 plugins/libplugin-pay.c: Keep p->invoice->routes valid when the routehints paymod mutates it.
The routehints paymod shares the storage of the array d->routehints and
p->invoice->routes, but once it operates, it possibly leaves it as a stale
pointer to memory it used to have.

Since other paymods may be interested in the invoice details, including
the routehints in the invoice, we should ensure the p->invoice->routes
remains valid whenever we try mutating that array.
2020-09-10 16:50:52 +09:30
Rusty Russell 191355e0e7 pay: fix handling of legacy vs tlv encoding.
As revealed by the failure of tests in #3936, where we ended up trying
to send a partial payment using legacy style, we are not handling
style properly.

1. BOLT9 has features, so we can *know* that the destination supports
   MPP.  We may not have seen a node_announcement.
2. We can't assume that nodes inside routehints support TLV.
3. We can't assume direct peers support TLV.

The keysend code tried to fix this up, so I'm not sure that this caused
the issue in #3968, though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: `pay` will now make reliable multi-part payments to nodes it doesn't have a node_announcement for.
2020-09-10 16:50:32 +09:30
ZmnSCPxj jxPCSnmZ 9587425f2d plugins/spender/fundchannel.c: Make fundchannel a thin layer around multifundchannel.
Changelog-Changed: protocol: `fundchannel` now shuffles inputs and outputs, and no longer follows BIP69.
2020-09-10 15:28:50 +09:30
ZmnSCPxj jxPCSnmZ f6c145d2df plugins/spender/multifundchannel.c: Move multifundchannel to the spenderp plugin. 2020-09-10 15:28:50 +09:30
niftynei 81a7a6742a elements: use normalization for elements fee output
This will update the fee output if it exists, rather than unilaterally
adding a new one.

Also, if the fee output already exists, we should make sure that it
doesn't interfere with the outnums of the other outputs
2020-09-10 12:31:36 +09:30
niftynei 4c28e7b362 txprepare: elements requires inclusion of an accurate fee output
so we add an accurate one
2020-09-10 12:31:36 +09:30
ZmnSCPxj jxPCSnmZ 277ff0f44c plugins/spender/multiwithdraw.c: Implement multiwithdraw command.
Fixes: #2679

Changelog-Added: JSON-RPC: New `multiwithdraw` command to batch multiple onchain sends in a single transaction.  Note it shuffles inputs and outputs, does not use BIP69.
2020-09-09 20:36:08 +09:30
ZmnSCPxj jxPCSnmZ e04febfe0c plugins/spender/: New plugin that will eventually absorb all onchain-spending commands. 2020-09-09 20:36:08 +09:30
ZmnSCPxj jxPCSnmZ 45e1b3828a plugins/multifundchannel.c: Add minchannels flag to multifundchannel.
[ changed from best_effort binary to minchannels counter -- RR]
2020-09-09 16:45:56 +09:30
ZmnSCPxj jxPCSnmZ 0d3a3b6d48 plugins/multifundchannel.c: Implementation of `multifundchannel`.
Changelog-Added: We now have `multifundchannel` as a builtin plugin command to fund multiple channels to different peers all in a single onchain transaction.


Header from folded patch 'fixup-use-json_add_psbt.patch':

fixup!


Header from folded patch 'use-goto-no-ok-chain.patch':

fixup!


Header from folded patch 'destinations-at-parse-time.patch':

fixup!


Header from folded patch 'multifundchannel__use_jsmntoks_to_pass_through_json_string,_not_strings.patch':

multifundchannel: use jsmntoks to pass through json string, not strings

Passing in "" for utxos would crash lightningd on the command-line
otherwise. Now returns an error.


Header from folded patch 'update_plugins-multifundchannel.c.patch':

Update plugins/multifundchannel.c

Co-authored-by: Darosior <darosior@protonmail.com>
2020-09-09 16:45:56 +09:30
ZmnSCPxj jxPCSnmZ ee276bcb86 plugins/bcli.c: `sendrawtransaction` now has a required `allowhighfees` argument.
Changelog-Deprecated: plugin: `bcli` replacements should note that `sendrawtransaction` now has a second required Boolean argument, `allowhighfees`, which if `true`, means ignore any fee limits and just broadcast the transaction. Use `--deprecated-apis` to use older `bcli` replacement plugins that only support a single argument.
2020-09-09 12:38:19 +09:30
Rusty Russell 83298c030a wallet: switch over to withdraw in module, remove lots of unused code.
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.

This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined.  Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
2020-09-08 10:14:42 +09:30
Rusty Russell f34d0b1cf5 plugins/txprepare: create simple variant for "withdraw".
This is a little lazy, but simpler than extracting the common parts
or making withdraw a plugin which calls txprepare (which should be
deprecated soon in favor of fundpsbt etc).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 8d4557938b plugins/txprepare: move functions higher.
Simple moveonly change for next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 6b2a3f4dfb txprepare: remove old code, switch to plugin.
Some minor phrasing differences cause test changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: txprepare reservations stay across restarts: use fundpsbt/reservepsbt/unreservepsbt
Changelog-Removed: txprepare `destination` `satoshi` argument form removed (deprecated v0.7.3)
2020-09-08 10:14:42 +09:30
Rusty Russell 3b1a226f67 txprepare: add txsend functionality.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 522459ba6e txprepare: add txdiscard functionality.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 74c6307c54 plugins/txprepare: use utxopsbt to handle utxos arg.
Very similar to fundpsbt, but takes 'utxos' instead of 'minconf'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell 8d5ff210bf txprepare: a plugin to replace `txprepare` (not yet active).
This uses `fundpsbt` and similar to simulate the txprepare command.
It has one difference (when complete), in that it those reservations
are now timed and don't get reset on restart.

It also doesn't have the restriction that `all` can only be used with
no other output, as I didn't realize that when I implemented it!

Note that change is now inserted in a random position, not sorted
into BIP69 order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Christian Decker 31c67f1392 bcli: Avoid tal_fmt when handling a raw hex block
We were using `tal_fmt` to truncate off the last byte of the
response (newline), which required an allocation, a call to `vsnprintf` and a
copy of the block contents. This being >2MB in size on mainnet was rather
expensive.

We now just signal the end of the string by overwriting the trailing newline,
and stealing directly onto the result.
2020-09-02 13:21:32 +02:00
Rusty Russell 0c7d04bd98 libplugin-pay: fix default CLTV.
This was changed recently, but without a bolt quote, we didn't find
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 06:59:13 +09:30
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
ZmnSCPxj jxPCSnmZ 932709cad9 plugins/libplugin-pay.c: Micro-optimize start_block sampling.
Using `waitblockheight 0` is a very slightly faster query than `getinfo`.
Also, avoid querying blockheight for child payments (allow `waitblockheight`
paymod to provide the blockheight returned from the `waitblockheight`, and
just resample the starting blockheight from the parent).

Changelog-None: pointless micro-optimization
2020-08-28 16:40:27 +02:00
ZmnSCPxj jxPCSnmZ 05daa8e5f3 plugins/libplugin-pay.c: Micro-optimization of plugin_is_finished.
This was checked with `gcc -S -O2` to see how an optimized build
would compile the function.
The original code completed calls into each child (and the `.s`
file showed that GCC 9.x was not smart enough to do early-out).

This modification explicitly does early-out, and avoids call-return
stack overhead for the common case where a payment is an ancestor
of a long line of single-child payments due to retrying.

Changelog-None: pointless micro-optimization
2020-08-28 16:40:27 +02:00
Rusty Russell 496c0dd1e6 common/random_select: central place for reservoir sampling.
Turns out we can make quite a simple API out of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell 12d0d5c185 amount: cleanup usage.
We've got some recently-added primitives which help.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 18:16:28 +02:00
Christian Decker 21d87f7075 pay: Implement simple presplit fix for ludicrous amounts
This is the simplest possible fix: increase the target amount until we get
the desired number of parts, while still bucketizing payments together that
are in approximately the same size.

The current logic puts all payments that are in the range x < amount <= 16*x
in the same bucket, making them harder to distinguish.

Changelog-Fixed: pay: The `presplit` modifier now supports large payments without exhausting the available HTLCs.
2020-08-27 10:19:21 +09:30
ZmnSCPxj jxPCSnmZ 128adf0938 plugins/libplugin-pay.c: Round-robin routehints when splitting.
This improves the success rate of `test_mpp_interference_2`, though
still not quite up to the level that we can remove `@flaky` from it.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ c27d7a3110 plugins/libplugin-pay.c: Store the route description, and re-report on failure. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 6468616c02 plugins/libplugin-pay.c: Propagate local_id from parent to child payment object. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ d8678467fa plugins/libplugin-pay.c: Show routes being tried, also print updates to channel hints not just initial creations. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ d89c77c0ce plugins/libplugin-pay.c: Describe the bits of unrecognized failure codes. 2020-08-25 12:17:18 +02:00