For example, lnprototest got the error 'You gave bad parameters: Did not support channel_type ' which doesn't make it clear that it's rejecting the empty channel type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Let's tell the caller what channel_type they got!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel`, `fundchannel_start` and `openchannel_init`: new field `channel_type`.
As suggested in https://github.com/lightning/bolts/pull/1092.
We still support channels opened without it, but you can no longer open new ones without it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
We're about to make static_remotekey compulsory, but we still want to
do tests for pre-existing channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And add a request schema for multifundchannel.
Changelog-Added: JSON-RPC: `fundchannel` and `multifundchannel` now take an optional `channel_type` parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And add request schemas for openchannel_init and fundchannel_start.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_start` and `openchannel_init` now take an optional `channel_type` parameter.
As suggested in https://github.com/lightning/bolts/pull/1092.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 11 nodes)
As suggested in https://github.com/lightning/bolts/pull/1092.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_data_loss_protect` is now required (advertized by all but 11 nodes)
Min. Cost Flow does not take into account fees when computing a flow
with liquidity constraints.
This is a work-around solution that reduces the amount on every route to
respect the liquidity bound. The deficity in the delivered amount is
solved by running MCF once again.
Changes:
1. the function `flow_complete` allocates amounts to send over the set of routes
computed by the MCF algorithm, but it does not allocate more than liquidity
bound of the route. For this reason `minflow` returns a set of routes that
satisfy the liquidity bounds but it is not guaranteed that the total payment
reaches the destination therefore there could a deficit in the delivery:
`deficit = amount_to_deliver - delivering`.
2. in the function `add_payflows` after `minflow` returns a set of routes we
call `flows_fit_amount` that tries to a allocate the `deficit` in the routes
that the MCF have computed.
3. if the resulting flows pass all payment constraints then we update
`amount_to_deliver = amount_to_deliver - delivering`, and the loop
repeats as long as `amount_to_deliver` is not zero.
In other words, the excess amount, beyond the liquidity bound,
in the routes is removed and then we try to allocate it
into known routes, otherwise we do a whole MCF again just for the
remaining amount.
Fixes issue #6599
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.
Should make bookkeeper less chatty and remove noisy logs
Changelog-None
We were putting out a lot of empty journal entries. Let's
stop doing that.
Now the wallet balance stays uninitialized until/unless you have
funds in it.
Fixes#5672
Now _msat fields are all integers (last conversion 23.08) we can simply
leave them alone, rather than trying to convert them.
And for turning Millisatoshi into JSON, we simply globally replace the
default encoding function to try ".to_json()" on items, which allows
anything to be marshalled.
The global replacement was interfering with other uses of JSON, such
as the clnrest plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-client: no longer autoconverts _msat field to Millisatoshi class (leaves as ints).
Since the class interacts with int very well now, we don't have to
make explicit accesses, so it's easy to write code which works with
Millisatoshi or int.
Also, don't access rpc.decoder in one test, it's unnecessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This alters a few remaining tests, as well.
[ Inclused even more test fixes from Alex Myers <alex@endothermic.dev>! ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listchannels` listing private channels: use listpeerchannels
This breaks our tests a bit, which assumed we can always see ourselves
even if we don't have a proper channel.
Usually we would deprecate this first, but it's unlikely to break
anyone since it's a bit obscure that this worked at all.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listnodes` no longer shows private (peer) nodes: use listpeers
l2 gave us a routehint, but it should have seen l1 as a dead-end. It
didn't due to the presence of a channel alias, which was a bug. We're
about to fix this, which breaks the test.
Add a dummy node off l1.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is redundant if it's a public channel, but vital if it's not. Publishing unconditionally makes
it easier for gossmap: we create a local modification all the time, even if redundant (and we can
have the actual capacity ceiling accurate in this case, since we know it for local channels).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listpeerchannels` now shows gossip update contents (even if channel unannounced).
Reported-by: Shahana Farooqui
Changelog-Fixed: JSON-RPC: Plugin notification `msat` fields in `invoice_payment` and `invoice_created` hooks now a number, not a string with "msat" suffix.
Changelog-Fixed: JSON-RPC: Plugin hook `payment` `msat` field is now a number, not a string with "msat" suffix.
Adds tests for when the connection fails during
1) splice tx_signature
2) splice commitment_signed
Fleshed out the reestablish flow for these two cases and implemented the fixes to make these reestablish flows work.
Part of this work required changing commit process for splices: Now we send a single commit_part for the splice where previously we sent all commits, and accordingly, we no longer revoke in response.
Changelog-Fixed: Implemented splicing restart logic for tx_signature and commitment_signed. Splice commitments are reworked in a manner incompatible with the last version.
Since these worked in v23.08, we can't just rename them. So if they are
used and unclaimed, we should rename them internally (if they're claimed,
it's probably clightning-rest, and we should *NOT* touch them!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `clnrest` parameters `rest-port`, `rest-protocol`, `rest-host` and `rest-certs`: prefix `cln` to them
This will allow users to use clnrest with c-lightning-REST without conflicts.
It was required for applications to have enough time for migrating from c-lightning-REST to clnrest.
Changelog-Changed:
config option `rest-certs` changed to `clnrest-certs`
config option `rest-protocol` changed to `clnrest-protocol`
config option `rest-host` changed to `clnrest-host`
config option `rest-port` changed to `clnrest-port`
config option `rest-cors-origins` changed to `clnrest-cors-origins`
config option `rest-csp` changed to `clnrest-csp`
We have a global JSON encoder hack, which means that any field ending in msat gets special treatment (so we can safely talk to lightningd, even if a field expects satoshi amounts, we are explicit). However, requests uses the JSON parser and DOES NOT want this conversion when sending it out as an HTTP response!
The simplest local fix we could find was Shahana's suggestion to iterate and covert away from Millisatoshi(): the reverse of what our JSON encoder does.
Fixes: https://github.com/ElementsProject/lightning/issues/6848
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It already throws an exception or error, or decodes the JSON response:
we can simply pass it through.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- cln-grpc certificate reuse
- new certificate generation
- `GET` and `POST` requests
- websocket server
- config options and HTTP headers
Link to #6436.
Here, we split up what was "commit_received" into two phases:
- commit-ready, where we're about to send our commitment tx to
peer
- commit-received, when we've gotten the commitment tx from our
peer
This lets us do the right thing (as far as the spec is concerned) with
returning the correct 'next_funding_txid' on reconnect (later commits).
"Patrick, I'm sorry I doubted you."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `large-channels` is now the default, wumbology for all.
Adding a fee offset as the channel opener reduces the likelihood of a
disconnect by the peer do to slight variation in feerate calculation
between nodes.
Changelog-Fixed: Some peer disconnects due to update_fee disagreements are avoided.
Looking through logs I was surprise to see:
```
lightningd-1 2023-10-26T03:42:36.824Z INFO lightningd: Sending 200000000msat over 1 hops to deliver 200000000msat
```
On a re-payment where we simply returned from sendpay immediately! Move that log to later.
We used to have "unsaved" payments: now we don't we can use
our normal "iterator" pattern rather than returning arrays.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The log messages were changed but the test fields weren’t updated, resulting in some test flakiness. Being more explicit with the log message we’re looking for should help.
Changelog-None
This means refactoring out some of the generic anchor info, from the
per-commitment-tx info (we can have at least two, perhaps more with
splicing!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We try to use anchors to CPFP our own commitment, but what if they
get there first? We also need to use anchors on the commitment
txs they broadcast.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `recover` command to force (unused) lightningd node to restart with `--recover` flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes `check` much more thorough, and useful.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` now does much more checking on every command (not just basic parameter types).
We had a complaint that you can't CPFP a mutual close, which you
should be able to do.
Fixes: #6692
Changelog-Fixed: wallet: close change outputs show up immediately in `listfunds` so you can CPFP.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
FAILED tests/test_connection.py::test_remote_addr_port - TimeoutError: Unable to find "[re.compile('Update our node_announcement for discovered address')]" in logs.
```
Because it can happen before the "Already have funding locked in" message:
```
lightningd-2 2023-10-24T22:07:02.018Z DEBUG gossipd: Update our node_announcement for discovered address: 127.0.0.1:1234
lightningd-2 2023-10-24T22:07:02.019Z DEBUG lightningd: Plugin chanbackup returned from peer_connected hook call
lightningd-2 2023-10-24T22:07:02.019Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-chan#2: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
lightningd-2 2023-10-24T22:07:02.036Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-channeld-chan#2: pid 63142, msgfd 67
lightningd-2 2023-10-24T22:07:02.037Z DEBUG 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-chan#2: Already have funding locked in (and ready to announce)
```
Also, wait_for_log() asserts itself, no need to assert on result.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Wait to be sure l1->l2 is ready. And use the same pattern for l2->l3.
```
def test_excluded_adjacent_routehint(node_factory, bitcoind):
"""Test case where we try have a routehint which leads to an adjacent
node, but the result exceeds our maxfee; we crashed trying to find
what part of the path was most expensive in that case
"""
l1, l2, l3 = node_factory.line_graph(3)
# We'll be forced to use routehint, since we don't know about l3.
wait_for(lambda: len(l3.rpc.listchannels(source=l2.info['id'])['channels']) == 1)
inv = l3.rpc.invoice(10**3, "lbl", "desc", exposeprivatechannels=l2.get_channel_scid(l3))
# This will make it reject the routehint.
err = r'Fee exceeds our fee budget: 1msat > 0msat, discarding route'
with pytest.raises(RpcError, match=err):
> l1.rpc.pay(bolt11=inv['bolt11'], maxfeepercent=0, exemptfee=0)
tests/test_pay.py:3420:
...
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: pay, payload: {'bolt11': 'lnbcrt10n1pjntczasp59x0weqkg4u9amd364yaeyw6rmgnmf9qtra6epylcntvt65yalpzspp5x8wgtmjhq33qruk6mmhutyr7w74xxjhct7v9tppel0t9p4rtautsdq8v3jhxccxqyjw5qcqp9rzjqgkjyd3q5dv6gllh77kygly9c3kfy0d9xwyjyxsq2nq3c83u5vw4jqqqvuqqqqgqqqqqqqqpqqqqqzsqqc9qxpqysgq4euy2qyzl2nufpxv6ahf0s5zry0h5dgrpa5adwu4swrdvjw7qe48cj8kp5fl7k20ex0x3dnk6e8xk5jp82snrdcr6he7eyqd0wrmvlgqwe5nma', 'maxfeepercent': 0, 'exemptfee': 0}, error: {'code': 210, 'message': 'Destination 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d is not reachable directly and all routehints were unusable.', 'attempts': [{'status': 'failed', 'failreason': 'Destination 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d is not reachable directly and all routehints were unusable.', 'partid': 0, 'amount_msat': 1000msat}]}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Make sure plugin has got message to connectd before sending!
```
def test_even_sendcustommsg(node_factory):
l1, l2 = node_factory.get_nodes(2, opts={'log-level': 'io',
'allow_warning': True})
l1.connect(l2)
# Even-numbered message
msg = hex(43690)[2:] + ('ff' * 30) + 'bb'
# l2 will hang up when it gets this.
l1.rpc.sendcustommsg(l2.info['id'], msg)
l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
l1.daemon.wait_for_log('Invalid unknown even msg')
wait_for(lambda: l1.rpc.listpeers(l2.info['id'])['peers'] == [])
# Now with a plugin which allows it
l1.connect(l2)
l2.rpc.plugin_start(os.path.join(os.getcwd(), "tests/plugins/allow_even_msgs.py"))
l1.rpc.sendcustommsg(l2.info['id'], msg)
l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
> l2.daemon.wait_for_log(r'allow_even_msgs.*Got message 43690')
tests/test_misc.py:3623:
...
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('allow_even_msgs.*Got message 43690')]" in logs.
contrib/pyln-testing/pyln/testing/utils.py:327: TimeoutError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We remove it from the pending_requests strmap before calling it,
so it doesn't get called again by destroy_plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now the internal code will generate a "PLUGIN_TERMINATED" response
when the plugin dies, we can handle it in plugin_hook.
But we can also simplify it by turning the snapshot of hooks into
a simple array: this means we are robust against any combination of plugins
exiting at any time.
Note: this reveals an issue with test_rpc_command_hook where we run
the request hook again (unexpectedly), so we disable that for the next
patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had special code to fail a forwarded request, but not for an
internally-generated request. Instead, we should pretend the (dead)
plugin responded with a PLUGIN_TERMINATED error, and handle the
request through the normal paths.
This breaks the case where a plugin crashes (or stops itself) in a
hook, so we handle that next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Make sure l1 can see l2's channel and it's in the gossip_store.
```
def test_excluded_adjacent_routehint(node_factory, bitcoind):
"""Test case where we try have a routehint which leads to an adjacent
node, but the result exceeds our maxfee; we crashed trying to find
what part of the path was most expensive in that case
"""
l1, l2, l3 = node_factory.line_graph(3)
# We'll be forced to use routehint, since we don't know about l3.
wait_for(lambda: len(l3.rpc.listchannels(source=l2.info['id'])['channels']) == 1)
inv = l3.rpc.invoice(10**3, "lbl", "desc", exposeprivatechannels=l2.get_channel_scid(l3))
# This will make it reject the routehint.
err = r'Fee exceeds our fee budget: 1msat > 0msat, discarding route'
with pytest.raises(RpcError, match=err):
> l1.rpc.pay(bolt11=inv['bolt11'], maxfeepercent=0, exemptfee=0)
tests/test_pay.py:3420:
...
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: pay, payload: {'bolt11': 'lnbcrt10n1pjjm8hesp5kp4nfgrj2ev6dz6xuqgxg29hz7263ltlafylhw7nglhtjxeqpn7spp5w92tjq8a354psfhdzmeuytfc6eye4f5egl7tj7s0f5ftz0k4pmcqdq8v3jhxccxqyjw5qcqp9rzjqgkjyd3q5dv6gllh77kygly9c3kfy0d9xwyjyxsq2nq3c83u5vw4jqqqvuqqqqgqqqqqqqqpqqqqqzsqqc9qxpqysgqetjwr6ql24jrz02qhj7pdw3kqynw6j3sgj2h32ufeyzasjyp2j6yc5durewjjpjy5yqtfgdxmdj52n7jk0ylzk5wudk4ffmjyyw6jmsqkvjex9', 'maxfeepercent': 0, 'exemptfee': 0}, error: {'code': 210, 'message': 'Destination 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d is not reachable directly and all routehints were unusable.', 'attempts': [{'status': 'failed', 'failreason': 'Destination 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d is not reachable directly and all routehints were unusable.', 'partid': 0, 'amount_msat': 1000msat}]}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We do it here, but it's not necessary, and we also deprive them of the
chance to do so (since we kill them).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was intermittant before: I added a sleep(1) in the code before
sending the error (temporarily) to make it always triggers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We truncate the file on stop(), but don't re-created it on start().
We didn't notice it before, but the net
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
Lagrang3
niftynei
Lagrang3
Matt Morehouse
Erik De Smedt
Dusty Daemon
Alex Myers
ShahanaFarooqui
Tony Aldon
Chris Guida
Aditya Sharma
Christian Decker
Peter Neuroth