```
2023-07-14T05:32:54.3688763Z # Set ECONOMICAL/6 feerate, for unilateral_close and htlc_resolution
2023-07-14T05:32:54.3689123Z l1.set_feerates((15000, 11000, 0, 0), True)
2023-07-14T05:32:54.3689484Z feerates = l1.rpc.feerates('perkw')
2023-07-14T05:32:54.3689919Z > assert feerates['perkw']['unilateral_close'] == 11000
2023-07-14T05:32:54.3690226Z E assert 15000 == 11000
2023-07-14T05:32:54.3690391Z
2023-07-14T05:32:54.3690514Z tests/test_misc.py:1572: AssertionError
```
The rough checks in set_feerates don't actually ensure that we've digested
the changes, so copy the check from elsewhere that makes sure
feerates['estimates'] has indeed been updated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We assume that RBFs will happen in order (txid1, txid2) but that doesn't always happen.
```
for depth in range(2, 10):
bitcoind.generate_block(1)
# l2 should RBF, twice even, one for the l1 main output,
# one for the l1 HTLC output.
# Don't assume a specific order!
start = l2.daemon.logsearch_start
> txid1 = get_rbf_txid(l2, txid1)
tests/test_closing.py:1671:
...
print("({} was previously in logs!)".format(r))
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('RBF onchain .*1fe38fe22852baaedccc3a9fd9d897e46bae5b7ca31daf23e0aa456fb235475e')]" in logs.
contrib/pyln-testing/pyln/testing/utils.py:328: TimeoutError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously, our code checked for the presence of the `lightning:`
prefix while decoding a bolt11 string. Although this prefix is valid
and accepted by the core lightning pay command, it was causing issues
with how we managed invoices. Specifically, we were skipping the prefix
when creating a copy of the invoice string and storing the raw invoice
(including the prefix) in the database, which caused inconsistencies
in the user experience.
To address this issue, we need to strip the `lightning:` prefix before
calling each core lightning command. In addition, we should
modify the invstring inside the db with the canonical one.
This commit fixes the issue by stripping the `lightning:` prefix
from the `listsendpays` function, which will improve the
user experience and ensure consistency in our invoice management (see
next commit).
Reported-by: @johngribbin
Link: ElementsProject#6207
Fixes: debbdc0
Changelog-Fixes: trim the `lightning:` prefix from invoice everywhere.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
We fixed the others. There are no fields, but this keeps it consistent.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `shutdown` notification contains `shutdown` object (notification consistency)
This is just housekeeping that allows up
to do not spam the logs of people with not
useful information.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Turns out we resubmit two txs (the commitment tx, and the anchor spend), but only wait
for one of them: if we mine a block before the anchor spend, it doesn't go in:
```
@unittest.skipIf(TEST_NETWORK != 'regtest', 'elementsd anchors unsupported')
@pytest.mark.developer("needs dev_disconnect")
def test_closing_anchorspend_htlc_tx_rbf(node_factory, bitcoind):
...
l1.daemon.wait_for_log('Peer permanent failure in CHANNELD_NORMAL: Offered HTLC 0 SENT_ADD_ACK_REVOCATION cltv 116 hit deadline')
l1.daemon.wait_for_log('Creating anchor spend for CPFP')
wait_for(lambda: len(bitcoind.rpc.getrawmempool()) == 2)
# But we don't mine it! And fees go up again!
l1.set_feerates((3000, 3000, 3000, 3000))
bitcoind.generate_block(1, needfeerate=5000)
l1.daemon.wait_for_log('RBF anchor spend')
l1.daemon.wait_for_log('sendrawtx exit 0')
# And now we'll get it in (there's some rounding, so feerate a bit lower!)
bitcoind.generate_block(1, needfeerate=2990)
> wait_for(lambda: 'ONCHAIN:Tracking our own unilateral close' in only_one(l1.rpc.listpeerchannels()['channels'])['status'])
```
If we mine too fast, gossip can reach a node which considers it too far in the guture. Break it up.
```
@pytest.mark.developer("Too slow without --dev-fast-gossip")
def test_routing_gossip(node_factory, bitcoind):
nodes = node_factory.get_nodes(5)
for i in range(len(nodes) - 1):
src, dst = nodes[i], nodes[i + 1]
src.rpc.connect(dst.info['id'], 'localhost', dst.port)
src.openchannel(dst, CHANNEL_SIZE, confirm=False, wait_for_announce=False)
# openchannel calls fundwallet which mines a block; so first channel
# is 4 deep, last is unconfirmed.
# Allow announce messages.
mine_funding_to_announce(bitcoind, nodes, num_blocks=6, wait_for_mempool=1)
# Deep check that all channels are in there
comb = []
for i in range(len(nodes) - 1):
comb.append((nodes[i].info['id'], nodes[i + 1].info['id']))
comb.append((nodes[i + 1].info['id'], nodes[i].info['id']))
def check_gossip(n):
seen = []
channels = n.rpc.listchannels()['channels']
for c in channels:
seen.append((c['source'], c['destination']))
missing = set(comb) - set(seen)
logging.debug("Node {id} is missing channels {chans}".format(
id=n.info['id'],
chans=missing)
)
return len(missing) == 0
for n in nodes:
> wait_for(lambda: check_gossip(n))
tests/test_gossip.py:721:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
success = <function test_routing_gossip.<locals>.<lambda> at 0x7f3200534ef0>
timeout = 180
def wait_for(success, timeout=TIMEOUT):
start_time = time.time()
interval = 0.25
while not success():
time_left = start_time + timeout - time.time()
if time_left <= 0:
> raise ValueError("Timeout while waiting for {}".format(success))
E ValueError: Timeout while waiting for <function test_routing_gossip.<locals>.<lambda> at 0x7f3200534ef0>
Having the grpc bindings in the pyln-testing package was always a bit
strange, however it came with additional issues. Due to the way that
protos are handled by protobuf, any name clash, independently of where
we import the protos, would cause an import error. This usually
happens in diamond-pattern dependencies, and so pull out the generated
files into their own package that everyone else can rely on.
Changelog-None
Greg Sanders helped debug this:
```
# Payment should succeed.
> l1.bitcoin.generate_block(1, wait_for_mempool=txid1)
tests/test_closing.py:2145:
...
> raise ValueError("Timeout while waiting for {}".format(success))
E ValueError: Timeout while waiting for <function BitcoinD.generate_block.<locals>.<lambda> at 0x7f7cd7271560>
```
The lgos show the HTLC tx doesn't go through because it double-spent an input but didn't spend enough:
```
2023-07-06T03:05:54.3424456Z lightningd-2 2023-07-06T02:57:37.490Z DEBUG plugin-bcli: sendrawtx exit 26 (bitcoin-cli -regtest -datadir=/tmp/ltests-yihsd7f4/test_onchain_middleman_simple_1/lightning-2/ -rpcport=39033 -rpcuser=... -stdinrpcpass sendrawtransaction 0200000000010220f632933db174def3b4bd879ad892e28f4422ee6e844bda27c4e5bdbc754fda030000000001000000975e48aeaced953f7c2b85f20e85b5c241258cb9dbd2ba13de3038daba6316330100000000fdffffff02a1860100000000002200202df545ea882889846c52fc5e111ac07ce07e0c09418ac15743a6f6284c2a4fa739391e000000000016001461ed06c0632af284ec9e192e97758fc04692c8290500473044022064446978d7f15d923237d44d7701e4a09a2d03ebb0a7e2c42e22c67435ad2fcc02201c51002fb72920978c79872e427acd90a13423e841b4198717c1771e7355ba4683473044022059ed9e2c536a71fac3cd63c7349c64a4445f0f936270295518a8aa03607d1e9d022064d318669a7602f585c84fe80aa95487816920c2a8ac26836601fbb369068ff38320478171dbde0e1c243e5c1ae23bcce446ab361197ca10d57c1d8f030cc9ff52158c76a914f5fb7361abefe39af0c3ab31d5be71096deeb4198763ac6721034bbdd4e5a933a3a83f6c3d22714cf23c452cb0c5ac8e429eea14992262787e687c8201208763a9147a3d3592e3d93a525beed57acce3eb70ba1b514288527c21039a62150fd6808d6c68aabd1e9d144c93e84a8f54f4e0f9ec5b3c37eee0a051c752ae6775017eb175ac6851b275680247304402200fde6a943a2f7f0287af2f5c5872326a4b8ad7020a098c4e05c28d2c2a0f2018022053a1263f982b98bb8dfcbb3a66641fe9f298e7ab432a78d21a2e79190d74753f012102397b0449a60d9d35634401bceaf3beb6118fc229b8552bd7122f735808b28aee00000000) error code: -26\nerror message:\ninsufficient fee, rejecting replacement 76f438f176d8f9beabb286f53c81aa7dcb4948d12f034f51753f4dd9071d6a74; new feerate 0.00029576 BTC/kvB <= old feerate 0.00054659 BTC/kvB
```
This is because sometimes we reuse the same UTXO for the anchor push spend as we do for the HTLC. That would be fine, except that we can have bitcoind mine the commitment tx and not the anchor push, and then we fail to replace it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was added to fundpsbt/utxopsbt in v0.10, but the spender plugin
didn't take advantage of it, instead calculating its own change amount
and output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use parameterization here. The old `anchor_expected()` was for
non-zero-fee anchors, and have bitrotted so there are some other
changes as well.
Unfortunately, all the anchor accounting seems to be broken, but I
cannot understand these tests at all. I had to simply disable them
for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In most cases, it's the same as option_anchor_outputs, but for
fees it's different. This transformation is the simplest:
pass it as a pair, and test it explicitly.
In future we could rationalize some paths, but this was nice
and mechanical.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we can CPFP, we don't have to track the feerate as closely. But
it still needs to get in the mempool, so we use 10 sat/byte, or the
100 block estimate if that is higher.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates` has new fields `unilateral_anchor_close` to show the feerate used for anchor channels (currently experimental), and `unilateral_close_nonanchor_satoshis`.
Changelog-Changed: JSON-RPC: `feerates` `unilateral_close_satoshis` now assumes anchor channels if enabled (currently experimental).
Add a fuzz test for BOLT 8 message encryption and decryption. The fuzz
test is based on the unit test at common/test/run-cryptomsg.c and uses a
static initial state with fuzzer-generated messages to encrypt or
decrypt.
This header will be used by multiple fuzz targets to fuzz Acts 1, 2, and
3 of the BOLT 8 handshake.
We could make this header into a full library, but considering its
narrow use let's try not to over-engineer things.
Since we didn't hash the descriptions properly (see previous commit), we
cannot immediately deprecate omitting the descriptions (since you'd
have to omit them for backwards compat!).
And move the "must have description or hash" test into bolt11.c core.
Changelog-Deprecated: `pay` has *undeprecated* paying a description-hash invoice without providing the description.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we need to push off requring this for another full deprecation cycle!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `pay` and `decodepay` with description now correctly handle JSON escapes (e.g " inside description)
This is almost certainly because the HTLCs are not fully settled, so wait for that:
```
2023-06-20T11:37:56.2332158Z assert apys_2[0]['our_start_balance_msat'] == Millisatoshi(0)
2023-06-20T11:37:56.2332443Z > assert apys_1[0]['routed_out_msat'] == apys_2[0]['routed_in_msat']
2023-06-20T11:37:56.2332571Z E assert 1892216msat == 2810170msat
2023-06-20T11:37:56.2332580Z
2023-06-20T11:37:56.2332717Z tests/test_pay.py:81: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we reconnect before the channel is completely closed, we might get
a "reconnected" message, so mine a block after and make sure it's
processed.
```
2023-06-20T11:37:56.1302058Z if errors.has_errors():
2023-06-20T11:37:56.1302648Z # Format a nice list of everything that went wrong and raise an exception
2023-06-20T11:37:56.1303781Z request.node.has_errors = True
2023-06-20T11:37:56.1304091Z > raise ValueError(str(errors))
2023-06-20T11:37:56.1304370Z E ValueError:
2023-06-20T11:37:56.1304624Z E Node errors:
2023-06-20T11:37:56.1305042Z E - lightningd-2: had unexpected reconnections
2023-06-20T11:37:56.1305340Z E Global errors:
```
...
```
2023-06-20T11:37:56.1960525Z lightningd-2 2023-06-20T11:21:28.638Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#2: Peer has reconnected, state CLOSINGD_SIGEXCHANGE: connecting subd
```
I added a plugin arg and was surprised that compile didn't break.
This is because typesafe_cb et al are conditional casts: if the type
isn't as expected it has no effect, but we're passing plugin_option() through
varargs, so everything is accepted!
Add a noop inline to check type, and fix up the two cases where we
used `const char *` instead of `char *`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Do it slightly intelligently, so if we had set previously using setconfig
we don't keep appending new ones, but replace it in-place.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently only implemented for min-capacity-sat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: new command `setconfig` allows a limited number of configuration settings to be changed without restart.
We get "disconnected during connection" if we haven't finished processing
the connection when the peer sends error and hangs up.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We re-enable sendrawtransaction then mine a block to kick off RBF, but there's
a race where it can get a tx in that block, and then we timeout waiting for
both txs to get into the next block.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Shahana Farooqui
Rusty Russell
Vincenzo Palazzo
Greg Sanders
Matt Morehouse
Christian Decker