Tihis commit is implementing a 2-phase commit between
the signer the node and the peer.
The main reason for this is that everybody must agree on the lock,
otherwise one of them will want N signatures (on the splice candidates),
and another will produce only 1 signature.
check_outpoint is the "prepare" for the signer, and lock_outpoint is the
"commit". if check_outpoint returns true, lock_outpoint must not fail.
Link: https://github.com/ElementsProject/lightning/issues/6722
Suggested-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
In general, a validating signer may be under a different operational
environment than the node, and therefore may have a different
source of on-chain data. The signer may therefore temporarily disagree
on whether a funding or splice transaction is locked (buried).
We would like to ensure agreement between the signer and the
node on how to progress a channel's state.
The following message are added to provide a solution:
- `check_outpoint(outpoint) -> bool` - check if the signer agrees that a funding candidate outpoint is buried
- `lock_outpoint(outpoint)` - change the funding/splice state to locked
Link: https://github.com/ElementsProject/lightning/issues/6722
Suggested-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Changelog-Added: hsmd protocol: Added hsmd_check_outpoint and hsmd_lock_outpoint
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Added: JSON-RPC: `recover` command to force (unused) lightningd node to restart with `--recover` flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes `check` much more thorough, and useful.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` now does much more checking on every command (not just basic parameter types).
Put an assertion inside db.c, and run every command we do (in testing) through
a `check` variant.
I inserted a deliberate bug (made addpsbtoutput call wallet_get_newindex()
before returning when running `check`, and indeed, backtrace as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We often want to do more parameter checks after param(), so allow a
new param_check(), with the proviso that the caller needs to also return
command_check_done() after other checks if command_check_only(cmd) is true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had a complaint that you can't CPFP a mutual close, which you
should be able to do.
Fixes: #6692
Changelog-Fixed: wallet: close change outputs show up immediately in `listfunds` so you can CPFP.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
datastoreusage returns the total_bytes that are stored under a given
{Key} or from root. {Key} is the entry point from which we begin to
traverse the datastore.
Changelog-Added: JSON-RPC: `datastoreusage`: returns the total bytes that are stored under a given key.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
Under certain conditions, when splicing a new channel quickly enough, an old channel announcement would emit *after* `mutual_splice_lock` and *before* announcement signature exchange.
Since the original channeld wouldn’t start the announcement timer until signatures were exchagned, this wasn’t an issue before.
Now splicing enables us to go from having announcement sigs to losing them, so we have to be prepared for this case.
Changelog-None
```
FAILED tests/test_connection.py::test_remote_addr_port - TimeoutError: Unable to find "[re.compile('Update our node_announcement for discovered address')]" in logs.
```
Because it can happen before the "Already have funding locked in" message:
```
lightningd-2 2023-10-24T22:07:02.018Z DEBUG gossipd: Update our node_announcement for discovered address: 127.0.0.1:1234
lightningd-2 2023-10-24T22:07:02.019Z DEBUG lightningd: Plugin chanbackup returned from peer_connected hook call
lightningd-2 2023-10-24T22:07:02.019Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-chan#2: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
lightningd-2 2023-10-24T22:07:02.036Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-channeld-chan#2: pid 63142, msgfd 67
lightningd-2 2023-10-24T22:07:02.037Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-chan#2: Already have funding locked in (and ready to announce)
```
Also, wait_for_log() asserts itself, no need to assert on result.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
Already in transaction from lightningd/plugin.c:727
```
There are two callers, and one didn't disable transactions, so do it in plugin_exclusive_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we've asserted that channeld would tell lightningd the same thing it
would do anyway, we can simply have channeld say "enable=True|False" and
lightningd fill in the other fields.
This means there's a pile of things channeld doesn't need to know any more!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
channeld used to talk directly to gossipd, so it made sense for it to
tell gossipd directly when it wanted it to make a new channel_update.
When that changed with v0.11, we simply directed the message via
lightningd.
But much of the information is actually told to channeld by lightningd!
So I applied this assertion and ran the test suite, before the next patch makes it redundant.
We got one assertion: test_setchannel_zero deliberately drives the
advertized htlc_max over the real htlc max in test_setchannel_zero for
testing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
`[False]` is considered True in Python, so this flake fix didn't work.
```
>>> if [False]:
... print('x')
...
x
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Wait to be sure l1->l2 is ready. And use the same pattern for l2->l3.
```
def test_excluded_adjacent_routehint(node_factory, bitcoind):
"""Test case where we try have a routehint which leads to an adjacent
node, but the result exceeds our maxfee; we crashed trying to find
what part of the path was most expensive in that case
"""
l1, l2, l3 = node_factory.line_graph(3)
# We'll be forced to use routehint, since we don't know about l3.
wait_for(lambda: len(l3.rpc.listchannels(source=l2.info['id'])['channels']) == 1)
inv = l3.rpc.invoice(10**3, "lbl", "desc", exposeprivatechannels=l2.get_channel_scid(l3))
# This will make it reject the routehint.
err = r'Fee exceeds our fee budget: 1msat > 0msat, discarding route'
with pytest.raises(RpcError, match=err):
> l1.rpc.pay(bolt11=inv['bolt11'], maxfeepercent=0, exemptfee=0)
tests/test_pay.py:3420:
...
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: pay, payload: {'bolt11': 'lnbcrt10n1pjntczasp59x0weqkg4u9amd364yaeyw6rmgnmf9qtra6epylcntvt65yalpzspp5x8wgtmjhq33qruk6mmhutyr7w74xxjhct7v9tppel0t9p4rtautsdq8v3jhxccxqyjw5qcqp9rzjqgkjyd3q5dv6gllh77kygly9c3kfy0d9xwyjyxsq2nq3c83u5vw4jqqqvuqqqqgqqqqqqqqpqqqqqzsqqc9qxpqysgq4euy2qyzl2nufpxv6ahf0s5zry0h5dgrpa5adwu4swrdvjw7qe48cj8kp5fl7k20ex0x3dnk6e8xk5jp82snrdcr6he7eyqd0wrmvlgqwe5nma', 'maxfeepercent': 0, 'exemptfee': 0}, error: {'code': 210, 'message': 'Destination 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d is not reachable directly and all routehints were unusable.', 'attempts': [{'status': 'failed', 'failreason': 'Destination 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d is not reachable directly and all routehints were unusable.', 'partid': 0, 'amount_msat': 1000msat}]}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Make sure plugin has got message to connectd before sending!
```
def test_even_sendcustommsg(node_factory):
l1, l2 = node_factory.get_nodes(2, opts={'log-level': 'io',
'allow_warning': True})
l1.connect(l2)
# Even-numbered message
msg = hex(43690)[2:] + ('ff' * 30) + 'bb'
# l2 will hang up when it gets this.
l1.rpc.sendcustommsg(l2.info['id'], msg)
l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
l1.daemon.wait_for_log('Invalid unknown even msg')
wait_for(lambda: l1.rpc.listpeers(l2.info['id'])['peers'] == [])
# Now with a plugin which allows it
l1.connect(l2)
l2.rpc.plugin_start(os.path.join(os.getcwd(), "tests/plugins/allow_even_msgs.py"))
l1.rpc.sendcustommsg(l2.info['id'], msg)
l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
> l2.daemon.wait_for_log(r'allow_even_msgs.*Got message 43690')
tests/test_misc.py:3623:
...
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('allow_even_msgs.*Got message 43690')]" in logs.
contrib/pyln-testing/pyln/testing/utils.py:327: TimeoutError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have to work quite hard to do this, since we don't want to call
finish if the broadcast has been freed in the meantime.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously, every broadcast was attached to a channel, but we can
make it explicit, so when the context is freed, the re-broadcast stops
(if rebroadcast is set).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If the context is freed, the callback isn't called. This doesn't matter
yet, since our callbacks tend to be such that the callback itself is
required to free things, but it's clearer this way and allows more
flexible usage in following patches.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We remove it from the pending_requests strmap before calling it,
so it doesn't get called again by destroy_plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We should really unify the cases of a local request, vs a forwarded
request, but for now, don't steal the request onto the plugin, and
if we return from the plugin and the request is gone, don't get upset.
This uncovered a case where we weren't inside a transaction, in
test_hook_crash, so fix that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now the internal code will generate a "PLUGIN_TERMINATED" response
when the plugin dies, we can handle it in plugin_hook.
But we can also simplify it by turning the snapshot of hooks into
a simple array: this means we are robust against any combination of plugins
exiting at any time.
Note: this reveals an issue with test_rpc_command_hook where we run
the request hook again (unexpectedly), so we disable that for the next
patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had special code to fail a forwarded request, but not for an
internally-generated request. Instead, we should pretend the (dead)
plugin responded with a PLUGIN_TERMINATED error, and handle the
request through the normal paths.
This breaks the case where a plugin crashes (or stops itself) in a
hook, so we handle that next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was always a bit weird they weren't, and it seems a premature
optimization to make the callbacks to this themselves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's per-plugin, so why is there a single map for all plugins? It
works because we always make unique ids, but it's weird.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When using DEBUG_SUBD with pytest:
```
lightningd: Unknown decode for --dev-debugger=<subprocess>
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
During the changeset calculation after the `openchannel2_sign`
hook.
So this commit patch the problem with the following change:
- Addressed an issue where `psbt_get_changeset` was modifying the original PSBT unnecessarily.
- This modification led to problems with a different hsmd, as referenced in [Issue #6672](https://github.com/ElementsProject/lightning/issues/6672).
- Noted a potential optimization where only a subpart of the PSBT
needs to be cloned, as the mutation is specific to inputs.
Link: https://github.com/ElementsProject/lightning/issues/6672
Reported-by: @devrandom
Suggested-by: Ken Sedgwick <ken@bonsai.com>
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
niftynei
Ken Sedgwick
Dusty Daemon
Rusty Russell
Peter Neuroth
daywalker90
Vincenzo Palazzo
Matt Morehouse