Commit Graph

14365 Commits

Author SHA1 Message Date
Christian Decker 60e1532dd8 cln-plugin: Switch from `env_logger` to `tracing-subscriber`
This switches the logging implementation from using the `log`-facade
to using the `tracing-subscriber` instead. This allows us to also tap
into the tracing instrumentation if desired, which was not possible
with `log`.

Changelog-Changed cln-plugin: The logging adapter now uses tracing-subscriber allowing the `tracing` ecosystem to be used. No format changes.
2024-01-16 18:36:12 +01:00
Erik De Smedt 0d877ffb71 cln-plugin: Allow setting `usage` of RPC-method
The `cln-plugin` can be used to create a plugin that registers
additional rpc-methods. However, it doesn't allow to specify the `usage`
of the command.

This change makes it possible to specify the `usage`. It should not
contain any breaking changes.

I've opted to add a new method called `rpcmethod_from_builder` to the
`Builder` struct. This approach allows us to add new parameters in the
future.
2024-01-16 16:55:06 +01:00
Jon Griffiths 43e302cd94 chaintopology: fetch item counts just once when iterating
Removes the tal_count checking overhead when iterating constant arrays.
Separated from the previous commit to make review easier.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths d5808d921a chaintopology: do less work when iterating blocks
- Avoid overhead from tal checks when iterating block txs
- Skip pegin txs as well as coinbase txs while iterating
- Early-exit if the txout cannot possibly be p2wsh
- Don't re-calculate the txid when we already have it
- Don't allocate a script for non-policy asset outputs
- Don't copy txids for non-interesting UTXOs

Note the below -Changed line covers the previous wally and PSBT commits
which also provide general block processing speedups.

Changelog-Changed: core: Processing blocks should now be faster

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 40dd780ea7 bitcoin_block_from_hex: avoid creating PSBT wrappers for finalized block txs
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths cdb5de0bce fromwire_bitcoin_tx: avoid creating a PSBT wrapper just to throw it away again
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 1b37515528 dualfund: avoid creating PSBT wrappers for input UTXOs
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths d4fdf8b94e interactive-tx: avoid creating PSBT wrappers for input UTXOs
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 2137343e7a tx: add pull_bitcoin_tx_only to pull without creating a psbt wrapper
There are many contexts in which it doesn't make sense to create a PSBT
wrapping the tx. There are also tx features that are not supported in
wallys PSBT implementation yet (such as pegins), which fail if a PSBT
wrapper is created, even though such features are unlikely to ever be
used in txs that cln will create/manipulate.

Allow these cases to be explicit that they only want the tx. This avoids
hitting such errors, is clearer on the caller side, and is more efficient.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 2e670a2a62 tx_parts: use wally to clone tx outputs
Input cloning has not been exposed yet; I'll add that to wally in a
future release.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 23a04d1bba base58: use _n api variant to avoid duplicating our input string
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 5e63e55b99 psbt: use wally_map_get instead of wally_map_find
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 16ac5cf085 psbt: use wally_map_replace instead of coding our own
Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths ffeb5553a5 wally: build debug or release according to the top-level configuration
Wally release builds are significantly faster than debug builds. Plus we
pass down our build flags to libsecp, which means release builds have
been disabling the asm optimisations for both libraries.

Changelog-Changed: Enable optimizations for libwally/libsecp256k1-zkp

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 1b2d81f7ce wally: remove now-redundant BUILD_ELEMENTS define
As of wally v1.0.0, Elements/Liquid support is enabled and part of the
library ABI by default.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 84ee3d5701 wally: update libwally to 1.0.0
Also removes incorrect/redundant configure flags when building.

Changelog-Changed: Update libwally to 1.0.0

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Jon Griffiths 1f45dd1acb wally: rename functions that clash with wally v1.0.0
Rename the offending functions from wally_foo to cln_wally_foo.

For the sake of a minimal diff, only calls which conflict with wally
v1.0.0 have been changed. However it is bad form to use the wally_
function namespace; the remaining such calls should also be renamed.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Peter Neuroth 3cbe35e7da devtools: Deduplicate links from linkify
The method linkify just added links for an entry to a list and produced
duplicates if we got more than one changelog line in a PR. We now only
produce a link once per PR.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-01-15 18:36:01 +01:00
Erik De Smedt 0da26933ac `cln_plugin` : Add basic test to `cln_plugin` 2024-01-15 15:32:16 +01:00
Erik De Smedt 000fceb215 Rust cln-rpc crate: Fix `call_enum` and `call`
I've broken the error-handling for `call` in a previous commit.
The key problem is that it failed to parse Json-RPC error's
that were returned by the server. It always returned a parse-
error instead.

To fix it I've adapted `call_raw_request`.
In the previous implementation local errors (e.g: Failing to find the
socket-file) where returned in a Result::Err. However, when the
rpc-server returned an error the data was encoded in the Result::Ok.

In this commit every error is returned as a `Result::Err` and
various methods and other calls have been edited to match this behavior.

I've also added additonal testing
2024-01-09 14:12:46 +01:00
Erik De Smedt dba977dfa2 Some basic documentation for `cln_rpc` 2024-01-09 14:12:46 +01:00
Erik De Smedt 4b5dade87f cln_rpc : use `call_typed` with external structs
One limitation of the `call_typed`-method was that it could not be used
with types defined outside of this crate. (See dependency on
`IntoRequest`)

This is useful for types that are not (yet) defined in this crate. A
possible case is an rpc-method that is defined in a plug-in that is
external to core-lightning or any method which isn't yet a part of the
`msggen`-script.

I've implemented a `TypedRequest` trait to make it work.

PS: This change is breaking. Users of `call_typed` must import
`cln_rpc::models::TypedRequest` instead of
`cln_rpc::models::IntoRequest`
2024-01-09 14:12:46 +01:00
Erik De Smedt ce41aa4ccc cln_rpc: Split low- from high-level API calls.
The `cln::ClnRpc` plugin has a `call` and a `call`-typed method
which worked only on structs that are mentioned in
`src::primitives::Request`.

The consequence is that any rpc-method that is not (yet) defined in this
crate could not be used.

I've adapted the `ClnRpc`-method and create a low-level binding named
`call_raw`. All changes in this commit should be backward compatible.
2024-01-09 14:12:46 +01:00
Christian Decker a59dbbdae5 configure: Use the system `PKG_CONFIG_PATH`
I noticed this while working on a nix devShell to work on CLN. We are
blanked overriding the `pkg-config` search path, which can cause some
trouble. Specifically `nix` uses content addressable locations, and
macOS arm64 and x84_64 use separate `pkg-config` search paths, and by
overwriting it we can cause a mix of different architectures failing
the compilation

Changelog-Fixed: configure: We now respect the `PKG_CONFIG_PATH` environment variable
2024-01-03 12:54:12 +01:00
bitcoin-lightning 2369086fd7 doc:correct listrunes's link 2024-01-02 18:45:18 +01:00
Christian Decker d8f410f2e5 gci: Add build step to PyPI publication 2023-12-29 00:35:45 +01:00
Christian Decker 44b7fe7cc6 meta: Fix the `tools/build-release.sh` script to match repro instructions 2023-12-29 00:35:45 +01:00
Christian Decker d30a2c9b68 docker: Use `git clone` as the build basis for reprobuilds
We used to use the zip archive, which comes with some baggage,
especially for some of the submodule-based dependencies. Using `git
clone` ensures that we have a clean snapshot, based on the latest
commit, and we can skip some of the wildcard operations on zip files.

Changelog-None
2023-12-29 00:35:45 +01:00
Christian Decker 5af9d19e94 misc: Improve generated traces from common/trace.c
The integration with opentelemetry was sub-optimal: it was generating
jaeger-style traces, with short traceIds and we were considering the
entire lifetime as a single trace. This PR changes that to a trace for
startup and then a trace for any event that doesn't already have a
parent.

We also allow using the `CLN_TRACEPARENT` envvar to attach the startup
to a remote / external trace, potentially by whatever started the main
process. This is useful to see the startup trace in the wider context
of whatever tooling is built around it.

Changelog-Added: tracing: It is now possible to inject a parent for the startup trace by setting the `CLN_TRACEPARENT` envvar
2023-12-23 15:19:31 +01:00
royalpinto007 b89e4dc51b fix: load script 2023-12-22 14:06:52 +01:00
Alex Myers 8d0ea8a94a invoice: force inclusion of min_final_cltv_expiry for compatibility
As reported by @wtogami, LND nodes are using a default
min_final_cltv_expiry_delta of 9, which makes them unable to pay invoices
using the modern spec default of 18.  Forcing inclusion of the c field
allows interoperability until broader support of the 18 block default.
Fixes: #6956

Changelog-Fixed: Default bolt11 invoices are payable by LND nodes.
2023-12-22 13:10:10 +01:00
daywalker90 56d42ea767 rust: use "map_or" instead of "if let"
Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-12-16 12:11:14 +01:00
Rusty Russell 09c1cfde39 pyln-client: don't automatically turn JSON into Millisatoshi class.
Now _msat fields are all integers (last conversion 23.08) we can simply
leave them alone, rather than trying to convert them.

And for turning Millisatoshi into JSON, we simply globally replace the
default encoding function to try ".to_json()" on items, which allows
anything to be marshalled.

The global replacement was interfering with other uses of JSON, such
as the clnrest plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-client: no longer autoconverts _msat field to Millisatoshi class (leaves as ints).
2023-12-16 10:22:01 +01:00
Rusty Russell 8b224b6ec2 pytest: don't assume that msat fields get turned into Millisatoshi.
Since the class interacts with int very well now, we don't have to
make explicit accesses, so it's easy to write code which works with
Millisatoshi or int.

Also, don't access rpc.decoder in one test, it's unnecessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-16 10:22:01 +01:00
Erik De Smedt 4f089accba Update pyln-testing cheroot dependency
Using `pyln-testing` with `python3.11` results in the following warning.

```
.../venv/lib/python3.11/site-packages/cheroot/__init__.py:7: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources
```

I've updated the `cheroot`-dependency.

However, the changes will not take effect immediately because we are
facing an internal conflict.

This conflict comes from the following requirements
- `//contrib/pyln-testing/pyproject.toml` requires `cheroot`. We can pick
  the requried version
- `./external/lnprototest/pyproject.toml` requires `pyln="^0.12"` requires
  `cheroot="^8"

It appears we have to do a multi-stage upgrade here.

**Step 1**
This commit configures `./contrib/pyln-testing/pyproject.toml`
to require `cheroot=">=8 <=10`. This allows users of `pyln-testing`
to start using a new release of `cheroot` immediately.

However, we still require `cheroot="^8"` because of `lnprototests`.
Even after running `poetry install` in the project root the warning
will remain.

**Step 2**
Not a part of this commit.

Publish a new release of `pyln-testing` on PyPI.

Once this release is finished we can update
`./external/lnprototests/pyproject.toml` to use the new
version of `pyln-testing` and the warning will disappear.
2023-12-16 12:28:55 +10:30
Adi Shankara 28671c14ac Update and rename backup-and-recovery.md to backup.md
Update title and filename since we moved out recovery to a new page.
2023-12-16 12:28:08 +10:30
Adi Shankara 549b3b96aa Update recovery.md
Using sentence case in title
2023-12-16 12:28:08 +10:30
ShahanaFarooqui ab77030cbe doc: Recovery using database
- Adds instructions to recover the wallet with database
- Breaks the recovery subtitle into separate page

Changelog-None.
2023-12-16 12:28:08 +10:30
Erik De Smedt 36f69a1a98 Docs: Add `custommsg` notification 2023-12-16 11:36:42 +10:30
Erik De Smedt 375985bf6c Implement tests for notifications on `custommsg` 2023-12-16 11:36:42 +10:30
Erik De Smedt 1ae16bfaa5 Create notification on `customssg`
Create a notification that is triggered when a `costummsg` is received.

Changelog-Added: Plugins: notification custommsg for receiving an unknown protocol message
2023-12-16 11:36:42 +10:30
Adi Shankara 3391bd1a72
Publish code-generation.md (#6948)
Set the hidden tag to false and make this article public.
2023-12-15 15:10:16 +00:00
Christian Decker 328863e6be tracing: Add an example exporter to zipkin and document it 2023-12-15 13:23:51 +10:30
Christian Decker 3fa2ec7e7e test: Add a `run-trace` test and document tracing overhead 2023-12-15 13:23:51 +10:30
Christian Decker 7ce0942ee2 common: Add trace_cleanup() function for unit tests 2023-12-15 13:23:51 +10:30
niftynei 7717f6e4df psbt-fix: set to NULL, which will cleanup dangling pointer
On July 18th, @jgriffiths wrote:

> You need to set this to NULL after freeing it, otherwise if line 72 returns you have a dangling pointer and potential later use-after-free here. Alternately use wally_psbt_set_input_final_witness(NULL) which will free any existing witness and set the value to NULL.

Reported-By: @jgriffiths
2023-12-15 10:20:19 +10:30
ShahanaFarooqui b250bca429 doc: documentation update for hooks and testing
Changelog-None.

Based upon suggestions:
https://dash.readme.com/project/blockstream-core-lightning/v1/suggested/update/655c70ab0077d2003b3b657b
https://dash.readme.com/project/blockstream-core-lightning/v1/suggested/update/655c8b912adff80063e82be8
2023-12-15 10:19:14 +10:30
Erik De Smedt 151a431628 Docs: Consistent whitespace in developers guide
The example used a mix of whitespace and tabs to format the JSON
which made it appear poorly on my device.
2023-12-15 10:18:04 +10:30
evansmj 7d52d4135e Sync bind-addr docs
In the original [$6173] pr `bind` was a typo for `bind-addr`.
`bind` never existed, so this commit updates uses of `bind` to `bind-addr`.
This links lightningd-config.5.md to configuration.md since `bind-addr` has options of interest such as ipv4/ipv6.
2023-12-15 10:03:10 +10:30
Christian Decker 05105d404c rs: Bump versions of `cln-plugin` and `cln-rpc` 2023-12-15 10:01:57 +10:30