I need to update gprcio-tools, but it needs protobuf v4. Christian
added this restriction in a99509db36 to
"Update protobuf dependency to silence dependabot", but perhaps it's
time to actually update?
```
$ poetry update
Updating dependencies
Resolving dependencies... (1.0s)
Because pyln-testing (23.05rc2) @ file:///home/rusty/devel/cvs/lightning/contrib/pyln-testing depends on protobuf (>=3.20.3,<4)
and grpcio-tools (1.54.0) depends on protobuf (>=4.21.6,<5.0dev), pyln-testing (23.05rc2) @ file:///home/rusty/devel/cvs/lightning/contrib/pyln-testing is incompatible with grpcio-tools (1.54.0).
So, because cln-meta-project depends on both grpcio-tools (1.54.0) and pyln-testing (23.05rc2) @ file:///home/rusty/devel/cvs/lightning/contrib/pyln-testing, version solving failed.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were using per-type overrides which caused some asymmetries, where
conversions could end up dropping fields as we went along. Essentially
each conversion would need to override a superset of the previous one,
which then caused issues when attempting to close the loop. By
overriding on the model level we ensure that all representations are
equivalent and convertible into one another, at the expense of
overriding a bit more aggressively, which should be fine anyway.
We use overrides that omit fields in some cases, which makes the
conversion lossy. This also means that until we complete the mapping
we can't reconvert back.
Now we've set everything up, the replacement code is quite simple.
Some tests now have to deal with RBF though, and our rbf tests need work
since they look for the old onchaind messages.
In particular, when we can't afford the fee we want, we back off to
the next blockcount estimate, rather than spending all on fees
(necessarily). So test_penalty_rbf_burn no longer applies.
Changelog-Changed: Protocol: spending unilateral close transactions now use dynamic fees based on deadlines (and RBF), instead of fixed fees.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates`: added `floor` field for current minimum feerate bitcoind will accept
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And consolidate descriptions into lightning-feerates().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` now allow "minimum" and NN"blocks" as `feerate` (`feerange` for `close`).
Drop try_get_feerate() in favor of explicit feerate_for_deadline() and
smoothed_feerate_for_deadline().
This shows us everywhere we deal with old-style feerates by names.
`delayed_to_us` and `htlc_resolution` will be moving to dynamic fees,
so deprecate those.
Note that "penalty" is still used for generating penalty txs for
watchtowers, and "unilateral_close" still used until we get zero-fee
anchors.
Changelog-Added: JSON-RPC: `feerates` `estimates` array shows fee estimates by blockcount from underlying plugin (usually *bcli*).
Changelog-Changed: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) value *slow* is now 100 block-estimate, not half of 100-block estimate.
Changelog-Deprecated: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) expressed as, "delayed_to_us", "htlc_resolution", "max_acceptable" or "min_acceptable". Use explicit block counts or *slow*/*normal*/*urgent*/*minimum*.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than have specific-purpose levels, have an array of
[blockcount, feerate], and rebuild the specific-purpose levels
for now on top.
We also keep a *separate* smoothed feerate, so you can ask for that
explicitly.
Since all the plugins used the same formula to derive the different
named fee levels, we apply the reverse to return to the underlying
estimates: updating the interface comes next.
This is ugly for now, but various specific-purpose levels will be
going away, as we shift to deadline-driven fees.
This temporarily breaks the floor calculation, so that test is
disabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Adding a new field with `added` fails:
```
AssertionError: Field Feerates.perkb.estimates[] does not have an `added` annotation
```
Looks like this assertion is wrong: we should get an added from the field itself or
from the .msggen.json file.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Using single tuples in Python is ugly, so:
1. Rename wait_for_onchaind_tx to wait_for_onchaind_txs.
2. Make it take tuples explicitly.
3. Make wait_for_onchaind_tx a simpler wrapper/unwrapper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can no longer grab the tx in one line as we did with
wait_for_onchaind_broadcast, we need to track the broadcast from
lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a visitor that ensures every new field has at least an `added`
field, and that we don't change the `added` or `deprecated` annotation
after the fact.
Changelog-Changed: msggen: The generated interfaces `cln-rpc` anc `cln-grpc` can now work with a range of versions rather than having to match the CLN version
This patch annotates the fields with a new `optional` attribute which
determines whether the field should be considered an inferred optional
due to being added or deprecated.
The patching system allows us to enrich the raw schema with some
additional information. In this specific case we want to backfill the
`added` and `deprecated` fields for the multiversion support.
CI seems to block; Christian suggests the throttler may be to blame somehow?
Since trying to fix it made it worse, let's just remove it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- moves offset into GossipHeader hdr which is passed to all constuctors
- reads .flags as u16 instead of extracting it from the .length, see 0274d88ba
- adds zombie and ratelimit flag to GossipHeader
- bytes_read start at 0 instead of 1 which is more correct,
the one byte is then corrected for when setting the offset of new header.
- bytes_read is increased in pull_bytes as this is the only place where
something is read
- use new style for various format-strings
This adds the `cln_parse_rpcversion` helper that is already used in
various plugins to pyln-client, so it does not need to be copied
around anymore.
Changelog-None
When doing the updates on the plugin repo, I discovered that
this helper function got broken by the `listpeerchannels` upgrade.
Its rarely used in the main repo, just at the end of the pyln-testing
'pay' helper.
If unfixed, this bug may result in test flakes when using the `pay`
helper, because its not correctly waiting for all HTLCs to be resolved
before returning.
Changelog-None
It's a core concept in the spec which isn't directly exposed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listchannels` added a `direction` field (0 or 1) as per gossip specification.
We didn't actually populate them properly, and the real annotations
are on inputs and outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `listtransactions` `channel` and `type` field removed at top level.
We removed the command for v22.11.
Also, we removed the `refund_for` offer parameter, so remove its description
from the manpage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rename is necessary to roundtrip, otherwise the rust name is used.
This also remove the rename if they are not necessary.
Note that:
```
#[serde(rename="foo", skip_serializing_if=="bar")]
pub field: bool,
```
is equivalent to:
```
#[serde(rename="foo")]
#[serde(skip_serializing_if=="bar")]
pub field: bool,
```
and for simplicity of construction the latter is used
Changelog-Added: JSON-RPC: new command `listpeerchannels` now contains information on direct channels with our peers.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This often helps the msat purge and pyln msat replacement mischmasch issues.
I changed it in a way that the `AttributeError: 'int' object has no attribute 'millisatoshis'`
Error will still be raised whever a `Millisatoshi` is compared to
something else then a `Millisatoshi` or `int`.
Changelog-None
This commit updates outdated dependencies and hangs all
bitcoin-related dependencies off of the `bitcoin` crate, using its
re-exports. This means that as long as the bitcoin crate matches, all
of its dependents will also match.
Changelog-None
We may end up with a short read, that is not empty, when a message
gets fragmented during transport, so we need to loop until we either
reach the full size or we have an empty read indicating a dropped
connection.
Changelog-None
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly. Create a new table for them, and
associated functions.
The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
Changelog-Changed: JSON-RPC: `listfunds` now lists coinbase outputs as 'immature' until they're spendable
Changelog-Changed: JSON-RPC: UTXOs aren't spendable while immature
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: LightningRpc has new `reply_filter` context manager for reducing output of RPC commands.
We suppress schema reply checking when filter is set: we could just
remove all the `required` fields in the JSON schema.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was causing some issues because it was picking up pre-built
artefacts from the host machine. By cloning first we ensure it matches
the latest commit and compiles from scratch.
Mac is updating to using zsh in general.
The “for i in” reads strings with spaces as a single entry instead of multiple entries as sh did.
Using “while read” … “<<< $var” makes it treat each space as a new entry.
Changelog-None
The goal here is to test the node validation, not whether we can
trigger the schema validation with bogus values. So we bypass the
verifying RPC wrapper.
We have to allow them (as otherwise `fees_collected_msat` in getinfo breaks),
but it means that actually, in_htlc_id might be missing in listforwards
(also, out_htlc_id might be missing, which we didn't catch before).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5628
This is just before the introduction of `get_json_id`, but has the
correct dependency constraints such that all packages can be updated
to >=v0.12 and we don't mix minor versions.
We needed to clone a lot since we were using a non-mutable borrow of
the cln-rpc version to then create the cln-grpc version. This alone
reduces the number of allocations of the `listpeers` test from 52
allocations to 38.
Many changes to gossmap (including the pending ones!) don't actually
concern readers, as long as they obey certain rules:
1. Ignore unknown messages.
2. Treat all 16 upper bits of length as flags, ignore unknown ones.
So now we split the version byte into MAJOR and MINOR, and you can
ignore MINOR changes.
We don't expose the internal version (for creating the map)
programmatically: you should really hardcode what major version you
understand!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, some field names have changed. lnprototest in master
has already upgraded names, so this brings us into compatibility again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-spec: package updated to latest spec version.
1. Update default specdir to the new modern name.
2. Don't use python to extract version, use sed.
3. Use poetry publish.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And document that we never know payment_hash.
Changelog-Added: JSON-RPC: `listforwards` now shows `in_htlc_id` and `out_htlc_id`
Changelog-Changed: JSON-RPC: `listforwards` now never shows `payment_hash`; use `listhtlcs`.
Normally, we'd use the delete_columns function to remove the old
`short_channel_id` string field, *but* we can't do that for sqlite, as
there are other tables with references to it. So add a FIXME to do
it once everyone has upgraded to an sqlite3 which has native support
for column deletion.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We didn't have optional Outpoints as arguments so far, so let's
backfill that.
Changelog-Changed: cln-rpc: The `wrong_funding` argument for `close` was changed from `bytes` to `outpoint`
We don't really support using pyln-client with older Core Lightning versions,
but this is neater anyway. I checked: f-strings go back to python 3.6, so
we can use them (I think this may be the first!).
Suggested-by: @MiWCryptAnalytics
Fixes: #5609
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel` and `fundchannel_start` now accept a `reserve` parameter to indicate the absolute reserve to impose on the peer.