This PR:
- adds all the guides (in markdown format) that is published at https://docs.corelightning.org/docs
- adds a github workflow to sync any future changes made to files inside the guides folder
- does not include API reference (json-rpc commands). Those will be handled in a separate PR since they're used as manpages and will require a different github workflow
Note that the guides do not exactly map to their related files in doc/, since we reorganized the overall documentation structure on readme for better readability and developer experience. For example, doc/FUZZING.md and doc/HACKING.md#Testing are merged into testing.md in the new docs. As on the creation date of this PR, content from each of the legacy documents has been synced with the new docs. Until this PR gets merged, I will continue to push any updates made to the legacy documents into the new docs.
If this looks reasonable, I will add a separate PR to clean up the legacy documents from doc/ (or mark them deprecated) to avoid redundant upkeep and maintenance.
Changelog-None
This is a simplification for our build system
that allows for managing and cleaning the external
build artefacts in an easy way.
We have a hard time with the suffix `-modded` inside the
version with some architecture when building a tagged version.
Link: https://discord.com/channels/899980449231814676/899989729183940629/1110957992158965770
Reported-by: @ctrlbreak-
Suggested-by: Rusty Russell <rusty@rustcorp.com.au>
Co-developed-by: @jsarenik
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Memory leak detected by ASan:
==880002==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 32816 byte(s) in 1 object(s) allocated from:
#0 0x5039e7 in malloc (lightningd/lightningd+0x5039e7)
#1 0x7f2e8c203884 in __alloc_dir (/lib64/libc.so.6+0xd2884)
Fixes nullability errors detected by UBSan:
wire/fromwire.c:173:46: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:432:3: note: nonnull attribute specified here
#0 0x65214a in fromwire_secp256k1_ecdsa_signature wire/fromwire.c:173:6
#1 0x659500 in printwire_secp256k1_ecdsa_signature devtools/print_wire.c:331:1
#2 0x646ba2 in printwire_channel_update wire/peer_printgen.c:1900:7
#3 0x637182 in printpeer_wire_message wire/peer_printgen.c:128:11
#4 0x65a097 in main devtools/decodemsg.c:85:10
Otherwise we later copy the uninitialized memory to descendants,
triggering undefined behavior:
plugins/libplugin-pay.c:2882:34: runtime error: load of value 190, which is not a valid value for type 'bool'
It is possible for db_column_bytes() to return 0 and for
db_column_blob() to return NULL even when db_column_is_null() returns
false. We need to short circuit in this case.
Detected by UBSan:
db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
#0 0x95f117 in db_col_arr_ db/bindings.c:479:2
#1 0x95ef85 in db_col_channel_type db/bindings.c:459:32
#2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9
#3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23
#4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9
#5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7
#6 0x672856 in main lightningd/lightningd.c:1121:25
The function is tiny and was only used in one location. And that one
location was leaking memory.
Detected by ASan:
==2637667==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 7 byte(s) in 1 object(s) allocated from:
#0 0x4cd758 in __interceptor_strdup
#1 0x64c70c in json_stream_log_suppress_for_cmd lightning/lightningd/jsonrpc.c:597:31
#2 0x68a630 in json_getlog lightning/lightningd/log.c:974:2
...
SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
Detected by UBSan:
$ UBSAN_OPTIONS=print_stacktrace=1 ./wallet/test/run-psbt_fixup
bitcoin/psbt.c:733:2: runtime error: applying zero offset to null pointer
#0 0x53c829 in psbt_from_bytes lightning/bitcoin/psbt.c:733:2
#1 0x5adcb0 in main lightning/wallet/test/run-psbt_fixup.c:174:10
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/psbt.c:733:2
We now know the base reference, and we've rebased, so we can do a
simple diff. Also, this means we can use a magic commit message
`No-schema-diff-check` to suppress false positives.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This reflects what we actually do when we apply the commit, and also
means we can easily iterate the commits.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listconfigs` direct fields, use `configs` sub-object and `set`, `value_bool`, `value_str`, `value_int`, or `value_msat` fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This integrates them with configvars properly: they almost "just work"
in listconfigs now, and we don't put them in a special sub-object
under their plugin.
Unfortunately, this means `listconfigs` now has a loose schema: any
plugin can add something to it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: reloaded plugins get passed any vars from configuration files.
Changelog-Deprecated: Config: boolean plugin options set to `1` or `0` (use `true` and `false` like non-plugin options).
We use multi-specifiable options elsewhere, this is just another.
Otherwise you can't add, you can only set them all.
Changelog-Added: Config: `accept-htlc-tlv-type` (replaces awkward-to-use `accept-htlc-tlv-types`)
Changelog-Deprecated: Config: `accept-htlc-tlv-types` (use `accept-htlc-tlv-type` multiple times)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `default` no longer accepted on `flag` type parameters (it was silently ignored, so just don't set it).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
listconfigs is convenient, but it doesn't handle multi-options well: it
outputs an object with duplicate fields in this case (e.g. log-file), nor
is it extensible to show more than raw values.
However, listconfigs doesn't do what other list commands do (use a
sub-object "configs") so we can put the new values under that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listconfigs` now has `configs` subobject with more information about each config option.
It literally contained \" to avoid it being interpreted as a literal;
now we have OPT_SHOWINT we no longer need this hack.
It's obscure, so I'm not bothering with a deprecation cycle.
Changelog-Fixed: JSON-RPC: `listconfigs` `rpc-file-mode` no longer has gratuitous quotes (e.g. "0600" not "\"0600\"").
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have hacky code to show some listconfigs values as literals; instead
explicitly encode the types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Clearly, listconfigs shouldn't list these.
Also, hoist the opt_hidden check since it's independent of whether
there's an arg or not.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's an obscure command, but this we won't see old plugin commands in
listconfigs (once it uses configvars).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Developers, rejoice (we already have --testnet, --signet and --mainnet!).
Changelog-Added: Config: `--regtest` option as alias for `--network=regtest`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we wire in the code which gathers configvars and parses from there;
lightningd keeps the array of configuration variables for future use.
Note that lightning-cli also needs to read the config, but it has its
own options (including short ones!) and doesn't want to use this
configvar mechanism, so we have a different API for that now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a nod towards moving to runtime-developer-mode, and also
quite clear; we currently have all these options prefixed with dev,
but this flags makes handling explicit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This will be used for listconfig, which knows these should be presented
as arrays, not single values.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These are gathered from the config files and the commandline, but the
process is rather complex! We want to remember where the options came
from in future (for a `setconfig` command), and also generalize
and simplify handling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that this actually changes listconfigs output for three msat
fields, which were not changed with the great msat merge. Since
listconfigs isn't actually used by grpc, and the values are always a
little vague, I simply changed this.
Changelog-Fixed: JSON-RPC: `listconfigs` `htlc-minimum-msat`, `htlc-maximum-msat` and `max-dust-htlc-exposure-msat` fields are now numbers, not strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This adds:
1. ability to search for an option by name.
2. allowance to set our own bits when registering options.
3. show callbacks which can say "don't show", and variable length.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We leave the code in contrib/pyln-client/pyln/client/lightning.py to handle
msat null fields for now, though, for a bit more compatibility.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently it fails if a field is missing, but sometimes that's OK. So
allow a fieldname ending in `?` to mean "skip over if it's missing".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Reported in #6270, there was an attempt to delete gossip overrunning
the end of the gossip_store. This logs the gossip type that was attempted to be deleted and avoids an immediate crash (tombstones would be fine to
skip over at least.)
Changelog-None
The `cln-grpc` crate really has a dual purpose: server and
client. Having the server feature be included by default means that we
are pulling in `cln-rpc` which is a Unix only crate, because of the
use of UDS to talk to CLN. We want to use `cln-grpc` on clients too,
and those might not be Unix variants, hence they'd fail when compiling
`cln-rpc`. This PR guards the Unix-related parts behind the `server`
feature flag.
These are only likely to confuse users, by silently changing behavior.
Changelog-Deprecated: Config: bind-addr=xxx.onion and addr=xxx.onion, use announce=xxx.onion (which was always equivalent).
Changelog-Deprecated: Config: addr=/socketpath, use listen=/socketpath (which was always equivalent).
This obsoletes the use of --announce-addr-dns which I know Michael
didn't really like either.
Changelog-Deprecated: Config: `announce-addr-dns`; use `--bind-addr=dns:ADDR` for finer control.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a major cleanup to how we parse addresses.
1. parse_wireaddr now supports the "dns:" prefix to support dns records (type 5).
2. We are less reliant on separate_address_and_port() which gets confused by
that colon.
3. We explicitly test every possible address type we can get back from
parsing, and handle them appropriately.
We update the documentation to use the clearer HOSTNAME vs DNS prefixes now
we also have `dns:` as a prefix.
Changelog-Added: Config: `bind` can now take `dns:` prefix to advertize DNS records.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I never really liked this hack: websockets are useful, advertizing
them not so much.
Note that we never actually documented that we would advertize these!
Changelog-EXPERIMENTAL: Protocol: Removed support for advertizing websocket addresses in gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Make it the standard "return the error" pattern.
2. Rather than flags to indicate what types are allowed, have the callers
check the return explicitly.
3. Document the APIs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Adi Shankara
Vincenzo Palazzo
Matt Morehouse
Shahana Farooqui
Rusty Russell
Christian Decker
Alex Myers