Greg Sanders helped debug this:
```
# Payment should succeed.
> l1.bitcoin.generate_block(1, wait_for_mempool=txid1)
tests/test_closing.py:2145:
...
> raise ValueError("Timeout while waiting for {}".format(success))
E ValueError: Timeout while waiting for <function BitcoinD.generate_block.<locals>.<lambda> at 0x7f7cd7271560>
```
The lgos show the HTLC tx doesn't go through because it double-spent an input but didn't spend enough:
```
2023-07-06T03:05:54.3424456Z lightningd-2 2023-07-06T02:57:37.490Z DEBUG plugin-bcli: sendrawtx exit 26 (bitcoin-cli -regtest -datadir=/tmp/ltests-yihsd7f4/test_onchain_middleman_simple_1/lightning-2/ -rpcport=39033 -rpcuser=... -stdinrpcpass sendrawtransaction 0200000000010220f632933db174def3b4bd879ad892e28f4422ee6e844bda27c4e5bdbc754fda030000000001000000975e48aeaced953f7c2b85f20e85b5c241258cb9dbd2ba13de3038daba6316330100000000fdffffff02a1860100000000002200202df545ea882889846c52fc5e111ac07ce07e0c09418ac15743a6f6284c2a4fa739391e000000000016001461ed06c0632af284ec9e192e97758fc04692c8290500473044022064446978d7f15d923237d44d7701e4a09a2d03ebb0a7e2c42e22c67435ad2fcc02201c51002fb72920978c79872e427acd90a13423e841b4198717c1771e7355ba4683473044022059ed9e2c536a71fac3cd63c7349c64a4445f0f936270295518a8aa03607d1e9d022064d318669a7602f585c84fe80aa95487816920c2a8ac26836601fbb369068ff38320478171dbde0e1c243e5c1ae23bcce446ab361197ca10d57c1d8f030cc9ff52158c76a914f5fb7361abefe39af0c3ab31d5be71096deeb4198763ac6721034bbdd4e5a933a3a83f6c3d22714cf23c452cb0c5ac8e429eea14992262787e687c8201208763a9147a3d3592e3d93a525beed57acce3eb70ba1b514288527c21039a62150fd6808d6c68aabd1e9d144c93e84a8f54f4e0f9ec5b3c37eee0a051c752ae6775017eb175ac6851b275680247304402200fde6a943a2f7f0287af2f5c5872326a4b8ad7020a098c4e05c28d2c2a0f2018022053a1263f982b98bb8dfcbb3a66641fe9f298e7ab432a78d21a2e79190d74753f012102397b0449a60d9d35634401bceaf3beb6118fc229b8552bd7122f735808b28aee00000000) error code: -26\nerror message:\ninsufficient fee, rejecting replacement 76f438f176d8f9beabb286f53c81aa7dcb4948d12f034f51753f4dd9071d6a74; new feerate 0.00029576 BTC/kvB <= old feerate 0.00054659 BTC/kvB
```
This is because sometimes we reuse the same UTXO for the anchor push spend as we do for the HTLC. That would be fine, except that we can have bitcoind mine the commitment tx and not the anchor push, and then we fail to replace it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
With the exception of fuzzing, make all builds in the `compile` job,
and simply download them in the other steps.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to have to use environment variables to pass through to
github/scripts/build.sh, but now we run ./configure directly it's
clearer to use explicit flags (though some matrixes still use env vars
for simplicity).
We also don't need to set COMPAT, as it's the default (MacOS tests
that we build without it, but otherwise we assume it's on).
And we make `gather` actually depend on all the other steps!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This controls debug flags for the build, rather than --developer,
which is going away.
I thought about making this flag control the RUST_PROFILE too, but
it seems that we want that set to "release" for CI, whereas for the
C code we want --enable-debugbuild.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We usually have access to `ld`, so avoid the global.
The only place generic code needs it is for the json command struct,
and that already has accessors: add one for libplugin and lightningd
to tell it if deprecated apis are OK.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We keep several peer pointers, but we just add a hook to NULL them
manually when a peer dies, rather than using voodoo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use a "softref" which is a magic pointer which gets NULL'ed when
the object is freed. But it's heavy, and a bit tricky to use, and we
only use it in gossipd.
Instead, keep the nodeid, and do a lookup (now that's fast) if we want
to credit the sender for valid gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We will access the freed connection to gossipd. This is weird to track
down when the *actual* issue is that gossipd died!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.
Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When the `--daemon` flag is used, `lightningd` requires that `--log-file`
is used as well. By consequence, the `lightningd.service` didn't work
out-of-the-box for me.
This changes also sends the logs to `journald`.
The new approach is consistent with the `bitcoind.service` file in the bitcoin-core repository.
I prefer this approach because it comes with automatic log-rotation.
If we're opening a channel with a peer which support anchors (and
we do), we tell fundpsbt/utxopsbt to enforce the emergency reserve;
this matters, as it doesn't know about the channel yet, and thus
won't (if it's our first anchor channel).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `fundchannel` and `multifundchannel` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (or are opening one).
This is needed when we know we're *opening* an anchor channel, to
override the "do we already have an anchor channel open?" logic.
Also, document the nonwrapped arg added in v23.02.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundpsbt` and `utxopsbt` new parameter `opening_anchor_channel` so lightningd knowns it needs emergency reserve for anchors.
This is the simple version which always tries to keep some sats if we
have an anchor channel. Turns out that we need something more
sophisticated for multifundchannel, so that's next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (and `all` will be reduced appropriately).
Changelog-Changed: JSON-RPC: `fundpsbt` and `utxopsbt` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels.
For anchors, we need some sats sitting around in case we need to CPFP
a close.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `min-emergency-msat` setting for (currently experimental!) anchor channels, to keep funds in reserve for forced closes.
This was added to fundpsbt/utxopsbt in v0.10, but the spender plugin
didn't take advantage of it, instead calculating its own change amount
and output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was added to fundpsbt/utxopsbt in v0.10, but the txprepare plugin
didn't take advantage of it, instead calculating its own change amount
and output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If you did call fundpsbt with amount 'all' and `excess_as_change`
true, you would get everything going to the change output. That's
obviously not the intention, and we'd like to use this to add change
outputs even for "all" when have keep emergency reserves.
And change the finish_psbt() API to take an explicit change amount:
at the moment it's either all or nothing, but that will change with
emergency-sat reserves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were marking our inputs very late, which means any early failure
would not know to unreserve them.
This becomes particularly bad when we start enforcing emergency reserves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use parameterization here. The old `anchor_expected()` was for
non-zero-fee anchors, and have bitrotted so there are some other
changes as well.
Unfortunately, all the anchor accounting seems to be broken, but I
cannot understand these tests at all. I had to simply disable them
for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We disabled experimental support for opening non-zero-fee anchor
channels (though old nodes may still have such channels if they turned
that on!).
So we simply call this `experimental-anchors`, since this is the variant
which we expect to be used widely.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: protocol: added support for zero-fee-htlc anchors (`option_anchors_zero_fee_htlc_tx`), using `--experimental-anchors`.
In most cases, it's the same as option_anchor_outputs, but for
fees it's different. This transformation is the simplest:
pass it as a pair, and test it explicitly.
In future we could rationalize some paths, but this was nice
and mechanical.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since HTLC txs when using anchors are
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY, we can attach other inputs to
give it a higher feerate. But we need the HSMd to actually sign the
combo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we can CPFP, we don't have to track the feerate as closely. But
it still needs to get in the mempool, so we use 10 sat/byte, or the
100 block estimate if that is higher.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates` has new fields `unilateral_anchor_close` to show the feerate used for anchor channels (currently experimental), and `unilateral_close_nonanchor_satoshis`.
Changelog-Changed: JSON-RPC: `feerates` `unilateral_close_satoshis` now assumes anchor channels if enabled (currently experimental).
mfc->feerate_str is *never* NULL, since we set it in getfeerate; this is
confusing, as many places check for NULL.
Indeed, the logic in perform_fundpsbt() was *wrong* in this case: it used
`normal` (if it was NULL, which it never was) instead of `opening` to fundpsbt.
And the correct thing is for multifundchannel to not use a string here at
all, but to use the exact feerate it is counting on (even the same
string may have different values now if a block has come in).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't actually use it anywhere, but we actually want to now for
CPFP. So give it more parameters and make it return bool so it can
be set without necessarily suppressing rexmit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out it's a single sig, identical to the already-handled
case where we spend a to_remote output.
We also close a temporary memleak: stack was unused, but
tallocated off the psbt, so it lives as long as the PSBT.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It depends on whether we negotiated anchors: just document that this
field doesn't apply for anchors (it becomes zero by the end of this
patch series, which is weird).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were setting the wrong input number: don't assume it's the
same as the UTXO number, but simply the last-appended input.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Shahana Farooqui
Rusty Russell
Erik De Smedt
Tony Aldon