Commit Graph

395 Commits

Author SHA1 Message Date
niftynei 6dfba2468a json-schema: allow 'required' to not be present in if switches 2022-07-28 12:08:18 +09:30
Rusty Russell 8c38302ab8 hsmtool: implement checkhsm.
This gives a nice way to ensure your secret is the correct one.

Also, we don't need to suppress VALGRIND for this test, now the output
races are fixed.

Changelog-Added: `hsmtool`: new command `checkhsm` to check BIP39 passphrase against hsm_secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-22 16:57:27 +02:00
Christian Decker b48ae58b56 repro: Update ubuntu jammy reprobuild 2022-07-21 15:21:12 -05:00
Christian Decker b8bcc7d13f pytest: Add a new RPC interface to talk to grpc
This allows us to re-use existing tests (assuming the call and fields
are covered by `cln-rpc` and `cln-grpc`) to test the full roundtrip
from test over the grpc interface to the json-rpc interface and back
again.

You can switch to the grpc interface by setting the `CLN_TEST_GRPC`
environment variable to 1, but for now only very few shims are
implemented (due to the non-generated nature of LightningRpc).
2022-07-21 14:19:06 +09:30
Rusty Russell 5979a7778f lightningd: expand exit codes for various failures.
Most unexpected ones are still 1, but there are a few recognizable error codes
worth documenting.

Rename the HSM ones to put ERRCODE_ at the front, since we have non-HSM ones
too now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-20 19:28:33 +09:30
Rusty Russell 3e672b784d Makefile: use a library archive for CCAN
The linker discards whole files in an archive if it doesn't need them,
so saves a bit of space (and time).  Also allows us to add more niche
things to CCAN (e.g. runes support!) without bloating all the binaries.

We also had many places which depended on $(CCAN_FILES), but that was
already a dependent of $(ALL_PROGRAMS) and $(ALL_TEST_PROGRAMS).

Before:

```
$ size lightningd/lightning*d
   text	   data	    bss	    dec	    hex	filename
2247683	   8696	  39008	2295387	 23065b	lightningd/lightning_channeld
2086607	   7432	  38880	2132919	 208bb7	lightningd/lightning_closingd
2227916	   8056	  39200	2275172	 22b764	lightningd/lightning_connectd
3369236	 119288	  39240	3527764	 35d454	lightningd/lightningd
2183551	   8352	  38880	2230783	 2209ff	lightningd/lightning_dualopend
2196389	   8024	  39136	2243549	 223bdd	lightningd/lightning_gossipd
2086216	   7488	  39264	2132968	 208be8	lightningd/lightning_hsmd
2134396	   8136	  39424	2181956	 214b44	lightningd/lightning_onchaind
2133391	   8352	  38880	2180623	 21460f	lightningd/lightning_openingd
1512168	   2136	  34384	1548688	 17a190	lightningd/lightning_websocketd
```

After:
```
text	   data	    bss	    dec	    hex	filename
2192065	   8488	  38912	2239465	 222be9	lightningd/lightning_channeld
2030957	   7224	  38816	2076997	 1fb145	lightningd/lightning_closingd
2179571	   7968	  39104	2226643	 21f9d3	lightningd/lightning_connectd
3354296	 119288	  39208	3512792	 3599d8	lightningd/lightningd
2127933	   8144	  38816	2174893	 212fad	lightningd/lightning_dualopend
2141699	   7856	  39072	2188627	 216553	lightningd/lightning_gossipd
2024482	   7288	   5240	2037010	 1f1512	lightningd/lightning_hsmd
2072074	   7920	   5400	2085394	 1fd212	lightningd/lightning_onchaind
2077773	   8144	  38816	2124733	 206bbd	lightningd/lightning_openingd
1408958	   1752	    344	1411054	 1587ee	lightningd/lightning_websocketd
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
adi2011 829fe09c13 doc: Add documentation for new RPCs and a FIXME: in fromschema.py 2022-07-14 12:24:48 -05:00
adi2011 286d6c3165 tools/gen: Always return bool! 2022-07-14 12:24:48 -05:00
adi2011 e42ba8366b common: Add scb_wire for serializing the static_chan_backup 2022-07-14 12:24:48 -05:00
Rusty Russell 32af92145b update-mocks: handle missing deprecated_apis.
This expands update-mocks to be able to handle (simple!) missing
symbols which are not functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-09 09:59:52 +09:30
Christian Decker 9039c9c46e docker: Update name from c-lightning to Core-Lightning 2022-05-14 11:43:46 +09:30
Christian Decker 182c900cea repro: Add reprobuild support for ubuntu:22.04 2022-04-28 12:28:48 +09:30
Rusty Russell f5df69020d v0.11.0.1: final fixes for release build
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-25 20:49:44 +09:30
Christian Decker 09ee28cb51 cln-grpc: Do not start unless a `grpc-port` is specified
For now we don't want to autostart.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-30 12:15:55 +10:30
Rusty Russell b19f3a5e7f devtools/decodemsg: don't require --onion for onion tlvs, fail if unknown tlvname.
Generate a table, let decodemsg sort it out.  Do more up-front work in
argparsing too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 83ee68ab06 common/tlvstream: put TLV checking back in the generic function.
Callers were supposed to call "tlv_fields_valid" after fromwire_tlv,
but few did.  Make this the default, and call the underlying function
directly where we want to be more flexible (one place).

This loses the ability to allow misordered fields, or to pass through
*any* even fields.  We restore that for special cases in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell a770f51d0e tools/generate_wire.py: make functions allocate the TLV.
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 04a152fbcf tools/generate-wire.py: make (and expose) individual TLV print functions.
When we actually put bolt12 fields (.e.g tlv_invoice) in onion messages,
that code will try to call printwire_tlv_invoice(), so expose it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 8f39002715 tools/generate-wire.py: have printwire routines do demarshal.
This works better in general: let printwire_x do the work of figuring
out how to demarshal x.  This is particularly important for TLVs, which
require a call to tlv_x_new() first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 88de64a580 tools/generate-wire.py: simplify printwire routines, fix ... handling.
We make them return bool, and always use names `cursor` and `plen` in
callers, for simplicity.

Also, `...` means "loop until finished" not "loop this many bytes".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell 12c7b156c5 tools/generate_wire.py: allow generated C files to have includes added.
We previously ignored --include= for these, but onion is about to start
needing bolt12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell fa0c29f959 tools/generate_wire.py: tlvs should start with tlv_
No more "towire_offer", but "towire_tlv_offer".

This means we double-up on the unfortunately-named `tlv_payload` inside
the onion, but we should rename that in the spec when we remove
old payloads.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Vincenzo Palazzo 53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Ken Sedgwick 36466af3eb hsmd: Add fields to hsmd_sign_{,remote_}commitment_tx for validating signers 2022-03-16 12:06:21 +10:30
Christian Decker f4fda04fb6 misc: Make the spell checker just a tiny bit less trigger happy 2022-03-08 05:00:36 +10:30
Christian Decker 1ef77504b1 misc: Add build targets for the tarball and debian packages
The tarball needs to materialze all submodules, and git needs to be
removed as a build dependency.
2022-02-22 09:43:56 +10:30
Michael Schmoock 7c8b12db0b wiregen: adds note about what <count> can be 2022-02-22 05:45:47 +10:30
Vincenzo Palazzo 43ff949ea7 lightningd: support hsm error code
Suggested-by: Rusty Russell
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Changed: Support hsm specific error error code in lightning-cli
2021-12-15 12:24:54 +10:30
Rusty Russell 186b895440 tools/check-includes.sh: make shellcheck happy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell 4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Vincenzo Palazzo b933b2fa4d tools: add a tool to formatting schema, and fixed (hack) the Makefile code checking.
This include the following commits:

- review 1/2: move from tab to space, and remove the exp. prop from doc;
- review 2/2: remove experimental features;


Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-11-28 10:55:48 +10:30
Christian Decker 39022028e9 db: Ensure sqlite3 library minimum version and major version match
We were really strict with the version check, requiring an exact
match. We now check that we are running at least with the version we
were compiled with (distro upgrades continue to work, and repro builds
are built off of an unupdated installation matching this minimum
requirement), and a major version match (since major versions can and
will introduce breaking changes).

Changelog-Fixed: sqlite3: Relaxed the version match requirements to be at least a minimum version and a major version match
2021-10-12 09:15:48 +10:30
W. J. van der Laan dd93f5dd29 Add missing headers for compiling on FreeBSD 13
- `netinet/in.h`: struct sockaddr_in, struct sockaddr_in6
- `unistd.h`: close()
2021-09-24 12:40:09 +09:30
Rusty Russell 24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell 7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Dustin Dettmer 6f2e27c768 Shellcheck cleanup
Updates from output of latest version of shellcheck as per SC2268

“Avoid x-prefix in comparisons as it no longer serves a purpose”
https://www.shellcheck.net/wiki/SC2268
2021-09-05 18:55:53 +02:00
Rusty Russell 6d1897b796 doc: fix GH/readthedocs rendering of manual pages.
They render the comment as if it's in the list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:02:17 +02:00
Rusty Russell 225fec065a Build race fixes: create ccan/config.h atomically, make tools/test/test_gen.c depend on it
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:43:54 +09:30
Rusty Russell ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
niftynei 265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
LightningHelper dbb7f31d39 added minimum word list needed
Not every user know the minimum word list that is needed.
2021-07-19 12:47:11 +09:30
Christian Decker b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Nalin Bhardwaj 0fccfc0a9a tools/check-spelling: Fix grep error on macOS
Changelog-Fixed: Fixes `make full-check` errors on macOS
2021-06-25 10:16:41 +09:30
Nalin Bhardwaj bbaca4bb5c tools/mockup: Fix macOS bash 3 syntax error
Changelog-Fixed: Fixes `make` with `--enable-developer` option on macOS.
2021-06-25 10:16:41 +09:30
Rusty Russell 6e636a835f tools/fromschema.py: handle deprecated null field, don't create empty lists.
1. listpeers has a deprecated `"closer": null`, which we need
   to handle in the schema, while trying not to damage our
   documentation too much.

2. Don't print a condition if there are no fields to print.

3. Allow a special "untyped" marker for multifundchannel which returns
   arbitrary JSON in a field.

4. Allow a single field return (for 'stop').

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30