The tester CI job uses absolute paths to ensure it is testing the
correct binaries. That clashes with the transfer between builder and
tester job using the `install` target because that switches things
around. This commit introduces a new target that just collects
artifacts in place, and tars them. Then we can use `tar` to unpack
them on the tester jobs again.
And make errors gcc-style, so emacs can jump through the automatically.
```
In devtools/reduce-includes.sh line 21:
echo -n "-$LINE"
^-- SC3037 (warning): In POSIX sh, echo flags are undefined.
In devtools/reduce-includes.sh line 25:
echo -n "."
^-- SC3037 (warning): In POSIX sh, echo flags are undefined.
In tools/rel.sh line 6:
prefix=$(printf '%s\n' "${from#$common}" | sed 's@[^/][^/]*@..@g')
^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
Did you mean:
prefix=$(printf '%s\n' "${from#"$common"}" | sed 's@[^/][^/]*@..@g')
In tools/rel.sh line 7:
printf '%s\n' "$prefix/${to#$common}"
^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
Did you mean:
printf '%s\n' "$prefix/${to#"$common"}"
For more information:
https://www.shellcheck.net/wiki/SC3037 -- In POSIX sh, echo flags are undef...
https://www.shellcheck.net/wiki/SC2295 -- Expansions inside ${..} need to b...
make: *** [Makefile:553: check-shellcheck] Error 123
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A simple standalone python executable to track plugin repositories, clone
to /tmp, install requirements, test plugin runs, then install and enable in
lightningd and in the config.
Changelog-Added: Reckless - a Core Lightning plugin manager
We have to allow them (as otherwise `fees_collected_msat` in getinfo breaks),
but it means that actually, in_htlc_id might be missing in listforwards
(also, out_htlc_id might be missing, which we didn't catch before).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5628
This adds a new "chan_dying" message to the gossip_store, but since we
already changed the minor version in this PR, we don't bump it again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now delay forgetting funding-spent channels for 12 blocks (as per latest BOLTs, to support splicing in future).
We will now simply reject old-style ones as invalid. Turns out the
only trace we could find is a channel between two nodes unconnected to
the rest of the network.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now require all channel_update messages include htlc_maximum_msat (as per latest BOLTs)
Technically this is a non-conformance with the spec, hence the `dev`
flag to opt-in, however I'm being told that it is also implemented in
other implementations. I'll follow this up with a proposal to the spec
to remove the checks we now bypass.
This is incompatible with the spec as it removes the enforcement for
reserves being above dust, but from what I can see from other
implementations it seems that others have allowed this as well.
This commit just guards the necessary changes with compilation guards, so
we can decide either way quickly. This part of the PR is not intended
to be final, just as a discussion basis.
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready. I change that everywhere, and try to fix up the
comments.
Also the `alias` field is called `short_channel_id`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
1. If the tool changes, you need to regenerate since the output may
change.
2. This didn't just filter that out, ignored all but the first
dependency, which made bisecting the bookkeeper plugin a nightmare:
it didn't regenerate the .po file, causing random crashes.
If we want this, try $(filter-out tools/fromschema.py) instead. But I
don't think we want that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These may eventually end up in pyln-client, as they allow talking to
the GRPC interface exposed by cln-grpc, however for now they are used
for testing only. Once we have sufficient API and test coverage we can
move them and leave imports in their place.
This has been replaced with better rust bindings that can then be
consumed via pyo3, consolidating the C interface in a portable
wrapper.
Changelog-Removed: libhsmd: Removed the `libhsmd_python` wrapper as it was unused
The M1 Macs support both x86_64 and arm64 architectures, which forced
homebrew to use a different path for its storage (`/opt/homebrew/`
instead of `/usr/local`). If we don't adjust the path we'd mix x86_64
and arm64 libraries which can lead to weird compiler and linker
errors.
This patch just introduces `CPATH` and `LIBRARY_PATH` as suggested by
the homebrew team, and detects the current architecture automatically.
Changelog-Added: macos: Added m1 architecture support for macos
Now it's formatted properly, we don't need the patch.
But we need to explicitly marshal/unmarshal into a byte stream,
which involves some code rearrangement.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, it's started complaining about "sudo make install"
and the .git directory being owned by someone else :(
Fixes: #5221Fixes: #5189
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Mostly comments and docs: some places are actually paths, which
I have avoided changing. We may migrate them slowly, particularly
when they're user-visible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We do this (send warnings) in almost all cases anyway, so mainly this
is a textual update, but there are some changes:
1. Send ERROR not WARNING if they send a malformed commitment secret.
2. Send WARNING not ERROR if they get the shutdown_scriptpubkey wrong (vs upfront)
3. Send WARNING not ERROR if they send a bad shutdown_scriptpubkey (e.g. p2pkh in future)
4. Rename some vars 'err' to 'warn' to make it clear we send a warning.
This means test_option_upfront_shutdown_script can be made reliable, too,
and it now warns and doesn't automatically close channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The signatures on the new examples are sometimes different from what we produce though?
They're valid, however.
And one example has an unneeded feature 5-bit; it's not *wrong*, but
it's not optimal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We detect whether we have the rust tooling available (mainly `cargo`)
and enable or disable the rust libraries, plugins and examples when it
is enabled. Since the rest of the Makefiles assumes that executables
have an associated header and C source file, we also needed to add a
target that we can add non-C binaries to.
On some distributions (e.g. Gnu Guix) Python packages are not installed in
some standard directory, rather they are installed in different places and
the `PYTHONPATH` variable is modified to include the different places.
So, we must not use the name `PYTHONPATH` in our `Makefile` since `make`
will replace the `PYTHONPATH` environment variable, preventing e.g.
`tools/generate-wire.py` from finding `python-mako` installed on such
distributions.
WebSocket is a bit weird:
1. It starts like an HTTP connection, but they send special headers.
2. We reply with special headers, one of which involves SHA1 of one of theirs.
3. We are then in WebSocket mode, where each frame starts with a 2-20 byte
header.
We relay data in a simplistic way: if either side sends something, we
read it and relay it synchronously. That avoids any gratuitous
buffering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, they are allowed to include .c files!
Here's `make check-units` on a maintainer-clean tree:
```
onchaind/test/run-onchainstress.c:4:10: fatal error: ../../hsmd/hsmd_wiregen.c: No such file or directory
4 | #include "../../hsmd/hsmd_wiregen.c"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:276: onchaind/test/run-onchainstress.o] Error 1
make: *** Waiting for unfinished jobs....
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It also gets rid of the requirement that close negotiation fee maximum
is the old commitment transaction. We still do that, however, to
avoid surprising old peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This includes the new bolt11 test vectors, and also removes the
requirement that HTLCs be less than 2^32 msat. We keep that for now
because Electrum enforced it on receive: in two releases we will stop
that too.
So no longer warn about needing mpp in that case either.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Protocol: No longer restrict HTLCs to
Enable non-dev builds to send custom messages.
Preserves 'dev-' for compat-enabled builds.
Changelog-Changed: JSON-RPC: moved dev-sendcustommsg to sendcustommsg
For markdown, there's no simple comment prefix: we need a postfix too.
We also need to use "" since we want to use ' in some of the Makefiles
in future when V=1.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This includes anysegwit and the updated HTLC tiebreak test vector. It
also adds explicit wording for invalid per_commitment_secret (which
nicely matches our code already!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If you do update-mocks in a dirty tree, the recursive make that it
uses will try to rebuild things! Suppress that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They are currently not installable due to circular dependencies and
stuff, so we just add their source to the path and lnprototest will
pick them up from there.
Instead of "only check suffix quotes when EXPERIMENTAL_FEATURES", make
it so we only check suffix quotes if you override BOLTVERSION on the
cmdline.
Before this, "make check-source-bolt" was effectively a NOOP with
--enable-experimental-features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The main change which affects us is that 2016 blocks to forget a channel
is a fixed number in the spec; we make this clear by renaming the
(developer-only) max_funding_unconfirmed to dev_max_funding_unconfirmed
and making it compile DEVELOPER only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
We assume if they set this to 0 (which nobody did previously), they're
using it as a modern flag and use it to indicate when they're
finished. Otherwise, we count how many blocks they've sent and use
that to determine whether they've finished.
See: https://github.com/lightningnetwork/lightning-rfc/pull/826
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we use `sync_complete` for gossip range query replies, with detection for older spec nodes.
A new target `check-gen-updated` to verify that all derived/generated
files that were modified were also checked in. This is used on CI to
check, and is not added to `check` since it'll complain on dirty
trees, i.e., before the devs check in their changes.
Suggested-by: Rusty Russell <@rustyrussell>
There's a 60 second delay in one of the contrib tests, and I just want
to run flake8 on my alterations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
== is a bash extension; in shell it's a single =:
```
/bin/sh: 1: [: unexpected operator
sql-rewrite wallet/db_postgres_sqlgen.c
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
$(BIN_PROGRAMS) is defined afterwards, leading to the problem that
lightningd doens't get rebuilt (in fact, it was running my installed
lightningd instead!).
It also needs $(PLUGINS) and the subdaemons, so hoist them all.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.
We also include the plugin headers in ALL_C_HEADERS so they get
checked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.
As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.
A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This adds an environment variable $NO_PYTHON check to Makefiles so that:
- It checks and runs into an defined error instead of some python hickup:
`ModuleNotFoundError: No module named 'mako'`
- makes it possible to manually export this environment variable NO_PYTHON=1
to run the same testcase that travis is running on job 1 which causes
so much pain ;)
Changelog-None
Kinda uses an antipattern of `cd`ing into the directory and calling `make`
there, but this keeps the contrib dirs self-contained so we can split them out
eventually.
Christian Decker
Rusty Russell
Alex Myers
niftynei
Christian Decker
Gregory Sanders
William Casarin
Andrew Toth
ZmnSCPxj jxPCSnmZ
Antoine Poinsot
Michael Schmoock