Commit Graph

575 Commits

Author SHA1 Message Date
Matt Morehouse aa9356b961 make: allow Clang coverage instrumentation
Clang's coverage instrumentation [1] is the best I've seen, with
precision down to the expressions within a line of code. Add an option
to use this instrumentation for better coverage reports.

[1] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
2023-06-15 10:47:56 +09:30
Jon Griffiths 8b4136bad5 build: remove gmp
Core and secp have not used gmp for a very long time now (core disabled
it in 2015).

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2023-05-24 18:21:43 +09:30
Rusty Russell e7d4c3175a build: remove --enable-experimental-features / EXPERIMENTAL_FEATURES
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell 4deb552fe9 build: don't generate experimental variants of wire files.
We no longer have any experimental-only wire definitions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Christian Decker 0687fecf0d make: Use the CLN_PLUGIN_EXAMPLES variable for testbin
Listing things multiple times is error-prone, so use the variable we
already have :-)
2023-04-13 18:13:28 -07:00
Matt Morehouse 14afa6efe7 make: add check-fuzz target
The target builds and runs each fuzz target on its seed corpus.
2023-04-09 12:41:27 +09:30
Rusty Russell d4ffc75691 Makefile: update to latest BOLT text.
In particular:
	- Bolt 4: add route blinding construction
	- Bolt 4: add blinded payments

And this means it's not experimental, so we can turn it on
by default!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: blinded payments are now supported by default (not just with `--experimental-onion-messages`)
2023-04-07 21:22:56 +09:30
Rusty Russell 458a85042b Makefile: update to BOLT 20066dc2aba906f37f3be5a810ae67040f265377
"BOLT 03: fix static-remote same amt and pre-image test vector"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell f26b1166b7 Makefile: update bolts a0bbe47b0278b4f152dbaa4f5fab2562413a217c
"BOLT 04: remove associated data from test vector"

(We actually use merge point).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell dfa6c0ca52 Makefile: bolt version b38156b9510c0562cf50f8758a64602cc0315c19
"Allow nodes to overshoot final htlc amount and expiry (#1032)"

Note that this also renamed `min_final_cltv_expiry` to the more-correct
`min_final_cltv_expiry_delta`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell fdf9b13bdb Makefile: update bolts fc40879995ebc61cc50dfd729512f17afb15b355.
"Allow nodes to overshoot the MPP `total_msat` when paying (#1031)"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Allow slight overpaying, even with MPP, as spec now recommends.
2023-04-07 21:22:56 +09:30
Rusty Russell 15f8e1e63c Makefile: update bolts to 60cfb5972ad4bec4c49ee0f9e729fb3352fcdc6a.
"BOLT 4: Remove legacy format, make var_onion_optin compulsory."

This also renamed the redundant "tlv_payload" to "payload", so we
replace "tlv_tlv_payload" with "tlv_payload" everyhere!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell 1e24d4a0a0 Makefile: fix check-gen-update to diff *all* files.
This would have caught the missing man page change!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-06 09:10:53 +09:30
Rusty Russell 3be36a66e3 configure: support sanitizers properly.
For example, if we use -fsanitize=undefined, we can't do unaligned
integer access, but since we didn't test with the sanitizer flags, we
didn't know this, and set `HAVE_UNALIGNED_ACCESS=1`.

Also, add -fno-sanitize-recover= in developer mode, so we actually
fail binaries if something is detected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-05 06:12:24 +09:30
Christian Decker 826c746568 docs: Use blockreplace.py to include all manpages 2023-03-25 15:40:35 +10:30
Christian Decker 59ed23e6cf make: Add doc/index.rst to generated files
It gets partially regenerated, so include it in the check. This is the
root cause for the v22.11-modded issue some have noticed.
2023-02-06 19:53:44 +01:00
Christian Decker a20540eb15 ci: Add a testpack.tar target to tranfer artifacts between CI jobs
The tester CI job uses absolute paths to ensure it is testing the
correct binaries. That clashes with the transfer between builder and
tester job using the `install` target because that switches things
around. This commit introduces a new target that just collects
artifacts in place, and tars them. Then we can use `tar` to unpack
them on the tester jobs again.
2023-01-16 13:53:48 +10:30
Christian Decker 6d67eb934d py: Ignore missing whitespace after keyword for now 2023-01-16 13:53:48 +10:30
Christian Decker 6518f6f26a make: Make the Makefile make 4.4 compatible
Fixes #5693

Changelog-None
2022-11-18 15:10:32 +01:00
Rusty Russell 2e4a58efac check: fix warnings from shellcheck 0.8.0
And make errors gcc-style, so emacs can jump through the automatically.

```
In devtools/reduce-includes.sh line 21:
	    echo -n "-$LINE"
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In devtools/reduce-includes.sh line 25:
	    echo -n "."
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In tools/rel.sh line 6:
prefix=$(printf '%s\n' "${from#$common}" | sed 's@[^/][^/]*@..@g')
                               ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
prefix=$(printf '%s\n' "${from#"$common"}" | sed 's@[^/][^/]*@..@g')


In tools/rel.sh line 7:
printf '%s\n' "$prefix/${to#$common}"
                            ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
printf '%s\n' "$prefix/${to#"$common"}"

For more information:
  https://www.shellcheck.net/wiki/SC3037 -- In POSIX sh, echo flags are undef...
  https://www.shellcheck.net/wiki/SC2295 -- Expansions inside ${..} need to b...
make: *** [Makefile:553: check-shellcheck] Error 123
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Alex Myers b1b280d10b reckless: new tool to manage lightningd plugins
A simple standalone python executable to track plugin repositories, clone
to /tmp, install requirements, test plugin runs, then install and enable in
lightningd and in the config.
Changelog-Added: Reckless - a Core Lightning plugin manager
2022-11-08 13:19:36 +01:00
Rusty Russell cafa1a8c65 db: correctly migrate forwards for closed incoming channels.
We have to allow them (as otherwise `fees_collected_msat` in getinfo breaks),
but it means that actually, in_htlc_id might be missing in listforwards
(also, out_htlc_id might be missing, which we didn't catch before).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5628
2022-09-27 14:42:03 +09:30
Rusty Russell f53155d93b BOLT: update to clarify HTLC tx amount calculation.
Simple quote update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell a1f62ba0e7 gossipd: don't close non-local channels immediately, add 12 block delay.
This adds a new "chan_dying" message to the gossip_store, but since we
already changed the minor version in this PR, we don't bump it again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now delay forgetting funding-spent channels for 12 blocks (as per latest BOLTs, to support splicing in future).
2022-09-24 15:22:27 +09:30
Rusty Russell 253b25522b BOLT: update to version which requires option_channel_htlc_max.
We will now simply reject old-style ones as invalid.  Turns out the
only trace we could find is a channel between two nodes unconnected to
the rest of the network.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now require all channel_update messages include htlc_maximum_msat (as per latest BOLTs)
2022-09-24 15:22:27 +09:30
Rusty Russell 45cdfd2ff7 BOLT: update to fix gossip pruning quote.
Which I disagreed with, and has been fixed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell 0514269f48 Makefile: add targets to upgrade pyln versions, push releases.
And use those in MAKING-RELEASES.md

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 11:41:11 +02:00
Christian Decker 54b4baabb0 opening: Add `dev-allowdustreserve` option to opt into dust reserves
Technically this is a non-conformance with the spec, hence the `dev`
flag to opt-in, however I'm being told that it is also implemented in
other implementations. I'll follow this up with a proposal to the spec
to remove the checks we now bypass.
2022-09-21 11:25:47 +02:00
Christian Decker c3e9cb7a47 openingd: Add zeroconf-no-really-zero mode
This is incompatible with the spec as it removes the enforcement for
reserves being above dust, but from what I can see from other
implementations it seems that others have allowed this as well.

This commit just guards the necessary changes with compilation guards, so
we can decide either way quickly. This part of the PR is not intended
to be final, just as a discussion basis.
2022-09-21 11:25:47 +02:00
Rusty Russell 3cc6d0ec2c doc: upgrade to BOLTs 341ec844f13c0c0abc4fe849059fbb98173f9766
This is a slightly looser behavior, so no change needed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell b208c0d8dd doc: upgrade to BOLTs 2ecc091f3484f7a3450e7f5543ae851edd1e0761
I disagree with this change, so I've commented and added a FIXME.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell 1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell 341bbdfcbe doc: increase BOLT level to 03468e17563650fb9bfe58b2da4d1e5d28e92009
`flags` in `channel_disabled` gets renamed.  We don't use it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell bcabb3825f Makefile: Revert ba7d4a8f6b (make-schema: don't include tools/fromschema.py in SHASUMS)
1. If the tool changes, you need to regenerate since the output may
   change.

2. This didn't just filter that out, ignored all but the first
   dependency, which made bisecting the bookkeeper plugin a nightmare:
   it didn't regenerate the .po file, causing random crashes.

If we want this, try $(filter-out tools/fromschema.py) instead.  But I
don't think we want that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell 8f6afedafe fuzz: fix fuzzing compilation.
It had bitrotted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-25 08:08:32 -07:00
Christian Decker 20b2f0af85 pyln: Ignore generated files when linting 2022-07-21 14:19:06 +09:30
Christian Decker bac322ccdb pytest: Move generated grpc bindings to pyln-testing
These may eventually end up in pyln-client, as they allow talking to
the GRPC interface exposed by cln-grpc, however for now they are used
for testing only. Once we have sufficient API and test coverage we can
move them and leave imports in their place.
2022-07-21 14:19:06 +09:30
Rusty Russell f65d3bb1fc ccan: upgrade to get ccan/runes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell 3e672b784d Makefile: use a library archive for CCAN
The linker discards whole files in an archive if it doesn't need them,
so saves a bit of space (and time).  Also allows us to add more niche
things to CCAN (e.g. runes support!) without bloating all the binaries.

We also had many places which depended on $(CCAN_FILES), but that was
already a dependent of $(ALL_PROGRAMS) and $(ALL_TEST_PROGRAMS).

Before:

```
$ size lightningd/lightning*d
   text	   data	    bss	    dec	    hex	filename
2247683	   8696	  39008	2295387	 23065b	lightningd/lightning_channeld
2086607	   7432	  38880	2132919	 208bb7	lightningd/lightning_closingd
2227916	   8056	  39200	2275172	 22b764	lightningd/lightning_connectd
3369236	 119288	  39240	3527764	 35d454	lightningd/lightningd
2183551	   8352	  38880	2230783	 2209ff	lightningd/lightning_dualopend
2196389	   8024	  39136	2243549	 223bdd	lightningd/lightning_gossipd
2086216	   7488	  39264	2132968	 208be8	lightningd/lightning_hsmd
2134396	   8136	  39424	2181956	 214b44	lightningd/lightning_onchaind
2133391	   8352	  38880	2180623	 21460f	lightningd/lightning_openingd
1512168	   2136	  34384	1548688	 17a190	lightningd/lightning_websocketd
```

After:
```
text	   data	    bss	    dec	    hex	filename
2192065	   8488	  38912	2239465	 222be9	lightningd/lightning_channeld
2030957	   7224	  38816	2076997	 1fb145	lightningd/lightning_closingd
2179571	   7968	  39104	2226643	 21f9d3	lightningd/lightning_connectd
3354296	 119288	  39208	3512792	 3599d8	lightningd/lightningd
2127933	   8144	  38816	2174893	 212fad	lightningd/lightning_dualopend
2141699	   7856	  39072	2188627	 216553	lightningd/lightning_gossipd
2024482	   7288	   5240	2037010	 1f1512	lightningd/lightning_hsmd
2072074	   7920	   5400	2085394	 1fd212	lightningd/lightning_onchaind
2077773	   8144	  38816	2124733	 206bbd	lightningd/lightning_openingd
1408958	   1752	    344	1411054	 1587ee	lightningd/lightning_websocketd
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Christian Decker 1a9979784f contrib: Remove obsolete libhsmd_python class
This has been replaced with better rust bindings that can then be
consumed via pyo3, consolidating the C interface in a portable
wrapper.

Changelog-Removed: libhsmd: Removed the `libhsmd_python` wrapper as it was unused
2022-07-15 15:27:45 +09:30
Christian Decker 99367dbb32 make: Add msggen generated files to check-gen-updated target
Changelog-None
2022-07-15 15:27:45 +09:30
niftynei ba7d4a8f6b make-schema: don't include tools/fromschema.py in SHASUMS
We were including the entire list of prerequisites when generating a
shastamp, which for schemas includes the `tools/fromschema.py` doc.

This meant all of our shasums were updating anytime this tool file
changed.

Instead, we just include the first prerequisite.

See: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
2022-07-14 12:24:48 -05:00
Christian Decker e3f53e072f make: Add macos M1 support
The M1 Macs support both x86_64 and arm64 architectures, which forced
homebrew to use a different path for its storage (`/opt/homebrew/`
instead of `/usr/local`). If we don't adjust the path we'd mix x86_64
and arm64 libraries which can lead to weird compiler and linker
errors.

This patch just introduces `CPATH` and `LIBRARY_PATH` as suggested by
the homebrew team, and detects the current architecture automatically.

Changelog-Added: macos: Added m1 architecture support for macos
2022-07-11 14:06:39 -05:00
Rusty Russell 4daa1b37ec contrib/pylightning: remove lightning-pay helper.
This example predates the pay plugin!  It's obsolete, unmaintained,
and probably doesn't work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell abd01a1701 Makefile: update to include fix for remote_addr generation.
Now it's formatted properly, we don't need the patch.

But we need to explicitly marshal/unmarshal into a byte stream,
which involves some code rearrangement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-05-19 09:47:32 +09:30
Rusty Russell 685fa25756 Makefile: update bolts to include remote_pubkey change.
Only affects comments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-05-19 09:47:32 +09:30
Rusty Russell bf040c398b Makefile: update to BOLTs without zlib.
This contains a typo fix and a clarification on channel_type, but also
removes ZLIB.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-05-19 09:47:32 +09:30
Gregory Sanders b15cf312e8 Change lightning-rfc to bolts post repo move 2022-05-18 10:19:16 +09:30
Rusty Russell 6ced5558fb Makefile: stumble along if git does not work.
In particular, it's started complaining about "sudo make install"
and the .git directory being owned by someone else :(

Fixes: #5221
Fixes: #5189
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-28 13:05:35 +02:00
Rusty Russell 836c1b805b doc: update c-lightning to Core Lightning almost everywhere.
Mostly comments and docs: some places are actually paths, which
I have avoided changing.  We may migrate them slowly, particularly
when they're user-visible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-07 06:53:26 +09:30