We have to work quite hard to do this, since we don't want to call
finish if the broadcast has been freed in the meantime.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously, every broadcast was attached to a channel, but we can
make it explicit, so when the context is freed, the re-broadcast stops
(if rebroadcast is set).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
`struct log` becomes `struct logger`, and the member which points to the
`struct log_book` becomes `->log_book` not `->lr`.
Also, we don't need to keep the log_book in struct plugin, since it has
access to ld's log_book.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we can CPFP, we don't have to track the feerate as closely. But
it still needs to get in the mempool, so we use 10 sat/byte, or the
100 block estimate if that is higher.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates` has new fields `unilateral_anchor_close` to show the feerate used for anchor channels (currently experimental), and `unilateral_close_nonanchor_satoshis`.
Changelog-Changed: JSON-RPC: `feerates` `unilateral_close_satoshis` now assumes anchor channels if enabled (currently experimental).
We don't actually use it anywhere, but we actually want to now for
CPFP. So give it more parameters and make it return bool so it can
be set without necessarily suppressing rexmit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We also do it on every block, but since bitcoind can't always be counted
to rebroadcast for us, we might as well be aggressive!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We would only set it the first time, which was OK for how we were using it
before. Now we want to also set it for rebroadcast.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Drop try_get_feerate() in favor of explicit feerate_for_deadline() and
smoothed_feerate_for_deadline().
This shows us everywhere we deal with old-style feerates by names.
`delayed_to_us` and `htlc_resolution` will be moving to dynamic fees,
so deprecate those.
Note that "penalty" is still used for generating penalty txs for
watchtowers, and "unilateral_close" still used until we get zero-fee
anchors.
Changelog-Added: JSON-RPC: `feerates` `estimates` array shows fee estimates by blockcount from underlying plugin (usually *bcli*).
Changelog-Changed: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) value *slow* is now 100 block-estimate, not half of 100-block estimate.
Changelog-Deprecated: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) expressed as, "delayed_to_us", "htlc_resolution", "max_acceptable" or "min_acceptable". Use explicit block counts or *slow*/*normal*/*urgent*/*minimum*.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than have specific-purpose levels, have an array of
[blockcount, feerate], and rebuild the specific-purpose levels
for now on top.
We also keep a *separate* smoothed feerate, so you can ask for that
explicitly.
Since all the plugins used the same formula to derive the different
named fee levels, we apply the reverse to return to the underlying
estimates: updating the interface comes next.
This is ugly for now, but various specific-purpose levels will be
going away, as we shift to deadline-driven fees.
This temporarily breaks the floor calculation, so that test is
disabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have the FEERATE_FLOOR constant if you don't care, but usually you want
to use the current bitcoind lower limit, so call get_feerate_floor()
(which is currently the same, but coming!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fun story. We're changing onchaind to hand txs to us, and we will
construct them and do the broadcast for it. lightningd tells onchaind
the witness it used (with flags to indicate which fields were
signatures so should be ignored) so onchaind can recognize the tx
when/if it is mined.
And when onchaind was waiting for a CLTV delay, it wouldn't tell
lightningd yet, but wait until the parent was sufficiently deep
But this caused bugs!
In particular, on replay, onchaind would see transactions which it
hasn't sent yet. This was not a problem before, as onchaind had
created the tx, even if it hadn't told lightningd to broadcast it, so
recognized the variant when it came in. When we're relying on
lightningd to tell us what the tx will look like, this doesn't work
any more.
The cause of this is that we fire off txowatches ("this output was
spent!") while we process blocks, and only fire off txwatches ("this
tx increased depth") once all the current blocks are processed. Often
this didn't matter, since we replay messages to onchaind from the
database, *but* we trim the last few blocks on restart (or, if there's
a small reorg while we're stopped), and we can hit this misordering.
Changing our topology code to only ever process one block at a time
would be a solution, but slows down catchup (and tests, where we often
mine a run of blocks).
So, this seems like a premature optimization, but it's really
required! And in future, lightningd can use this knowledge of pending
transactions to combine them in more clever ways.
Note that if a tx is valid at block N, we broadcast it once we see
block N-1, to get it in the mempool for block N.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was once only called on failure, now it's always called (if set).
It was called different things in different places, so unify it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had a quadratic check when processing a block, and this just
reduces it to be linear. Combined with the previous fix of adding txs
multiple times, this should speed up block processing considerably.
We were using a quadratic lookup to locate the transactions we
broadcast, so let's use an `htable` instead, allowing for constant
lookups during block processing.
First, merge the _ahf_ and non-ahf interfaces.
Second, remove the always-NULL txs->cmd field.
Then, add optional id_prefix for bitcoind_sendrawx, so if it's
triggered by a command (e.g. "withdraw") it's shown correctly in logs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We annotate them with UNNEEDED, which is legal but weird, but it
makes gcc (at least 11.2.0) complain about shadowing:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106424
I simply removed the names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There are hardly any lightningd-specific JSON functions: all that's left
are the feerate ones, and there's already a comment that we should have
a lightningd/feerate.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Not necessary yet, but it will be once shutdown starts waiting for
plugins to respond: we don't want these to try to access the bcli
plugin once it's freed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's a legacy from when it didn't have an ld pointer to access ld's
timer structure. Now it's just confusing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Useful for regtest and testnet. Sure, you shouldn't use this on mainnet,
but I haven't restricted it because our users are usually pretty clever.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #1806
Changelog-Added: config: `force_feerates` option to allow overriding feerate estimates (mainly for regtest).
Changelog-Deprecated: plugin: `bcli` replacements should note that `sendrawtransaction` now has a second required Boolean argument, `allowhighfees`, which if `true`, means ignore any fee limits and just broadcast the transaction. Use `--deprecated-apis` to use older `bcli` replacement plugins that only support a single argument.
We're not going to mutate transactions in a block, so computing the txids
every time we need them is a waste, let's compute them upfront and use them
afterwards.
We were basing the 2016 block timeout on the blockchain height that we had
processed at the time completed the funding, which could lag considerably
behind the network-wide blockheight. For example this could happen if we
started with `--rescan` from a low height, taking some time to catch up.
This means that we could end up forgetting channels even before reaching the
network blockheight. This patch instead uses the headercount reported by the
backend plugin if we aren't caught up yet. While the chances of this happening
are still there, the window it might happen are much reduced, since headers
can be synced in a couple of minutes.
Reported-by: Riccardo Masutti
Changelog-Changed: Funding timeout is now based on the header count reported by the bitcoin backend instead of our current blockheight which might be lower.
We kept track of an URGENT, a NORMAL, and a SLOW feerate. They were used
for opening (NORMAL), mutual (NORMAL), UNILATERAL (URGENT) transactions
as well as minimum and maximum estimations, and onchain resolution.
We now keep track of more fine-grained feerates:
- `opening` used for funding and also misc transactions
- `mutual_close` used for the mutual close transaction
- `unilateral_close` used for unilateral close (commitment transactions)
- `delayed_to_us` used for resolving our output from our unilateral close
- `htlc_resolution` used for resolving onchain HTLCs
- `penalty` used for resolving revoked transactions
We don't modify our requests to our Bitcoin backend, as the next commit
will batch them !
Changelog-deprecated: The "urgent", "slow", and "normal" field of the `feerates` command are now deprecated.
Changelog-added: The fields "opening", "mutual_close", "unilateral_close", "delayed_to_us", "htlc_resolution" and "penalty" have been added to the `feerates` command.
If the same memory gets reallocated, our "has the tip changed?" test
gets a false negative. This happened for me about one time in 10,
causing tests/test_misc.py::test_funding_reorg_remote_lags to fail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- result fundchannel command now depends on successful or failed broadcast of the funding tx
- failure returns error code FUNDING_BROADCAST_FAIL
- don't fail the channel when broadcast failed, but keep in CHANNELD_AWAITING_LOCKIN
- after fixing the initial broadcast failure, the user could manually rebroadcast the tx and
keep the channel
openingd/opening_funder_finished:
- broadcast_tx callback function now handles both success and failure
jsonrpc: added error code FUNDING_BROADCAST_FAIL
manpage: added error code returned by fundchannel command
This makes the user more aware of broadcast failure, so it hopefully doesn't
try to broadcast new tx's that depend on its change_outputs. Some users have reported (see
issue #2171) a whole sequence of fundings failing, because each funding was using the change
output of the previous one, which would not confirm.
Handers of a specific form are both designed to be used as callbacks
for param(), and also dispose of the command if something goes wrong.
Make them return the 'struct command_result *' from command_failed(),
or NULL.
Renaming them just makes sense: json_tok_XXX is used for non-command-freeing
parsers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And, reluctantly, default to bitcoind style.
"It's wrong to be right too soon."
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We could refine this later (based on existing wallet, for example), but
this gives some estimate.
[ Rename onchain_estimates -> onchain_fee_estimates Suggested-by: @SimonVrouwe ]
[ Factor of 1000 fix Reported-by: @SimonVrouwe ]
Suggested-by: @molxyz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't know what our peer is doing, but if we see those values, maybe
they did too, and for longer. And add the min/max acceptable values
into our JSON API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And no more filtering out messages, as we should no longer spam the
logs with them (the 'Connected json input' one was removed some time
ago).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
memleak can't see into htables, as it overloads unused pointer bits.
And it can't see into intmap, since they use malloc (it only looks for tal
pointers).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
Christian Decker
ZmnSCPxj jxPCSnmZ
darosior
Simon Vrouwe