Commit Graph

254 Commits

Author SHA1 Message Date
Rusty Russell 24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell 7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell 183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell fb4edc2938 Makefile: update bolt version to include option_anchors_zero_fee_htlc_tx.
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell a5fee67b91 common/memleak: take over dump_memleak(), allow print pointer.
This will let plugins use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
niftynei 86f7a179a9 channel utxo: persist the 'csv' lock value to database
Channel leases modify the CSV height that an output is eligible for
being spent at,  persist this to the database
2021-07-20 13:28:38 -04:00
niftynei 265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei 5041073a55 onchaind: add recovery scans for option-will-fund
option_will_fund changes the to_remote/to_local commitment tx
outputs by altering the CSV lock for leased channels.

We need to grind/scan for these outputs now, provided the defaults don't
work.
2021-07-20 13:28:38 -04:00
niftynei c9d2748081 lease_rates: add csv lock to modify anchor scripts 2021-07-20 13:28:38 -04:00
niftynei e992b54410 script: add csv_lock to scripts 2021-07-20 13:28:38 -04:00
Christian Decker b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Rusty Russell fa0cd3cc76 onchaind: handle static_remotekey thresholds.
No longer a global "on" or "off", it depends on the commitment number.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell 5473d66993 onchaind: limp along if we cheat.
We don't handle our own cheat txs: rather than crash, we should just
log broken and limp along.

This also makes our upcoming penalty test easier: we don't have to
spin up a new node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell 77ed0c280b onchaind: don't hand redundant commit_num to handle_unknown_commitment.
It's a global.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell 7d666e9bfd onchaind: don't rely on knowing option_static_remotekey for unknown commitments.
Just always handle both cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell 2fea448498 gen/impl_template: fix generation of singleton varsize elements.
And as Lisa requested, add testcases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell 25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 9dbac21d3b doc: remove suffix for included-in-master BOLTs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-07 14:34:39 +09:30
Rusty Russell 36bf037ca7 doc: update BOLT quotes (to include preimage leakage).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 13:23:16 +10:30
Rusty Russell 6dab04ebd5 tools/generate-wire.py: include digits in #ifndef idempotent header.
Otherwise check complains that it's '#ifndef LIGHTNING_WIRE_BOLT_WIREGEN_H':

wire/bolt12_wiregen.h seems to be missing the expected include guard:
  #ifndef LIGHTNING_WIRE_BOLT12_WIREGEN_H
  #define LIGHTNING_WIRE_BOLT12_WIREGEN_H
  ...
  #endif /* LIGHTNING_WIRE_BOLT12_WIREGEN_H */

make: *** [Makefile:458: check-includes] Error 1

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell a11edebb7c utf8: handle UTF-8 arrays.
BOLT 12 introduces this as a new fundamental type, which lets us easily
validate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 9d656464f5 bitcoin/pubkey: add pubkey32 primitive for xonly pubkey types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell e13e3e90ec onchaind: cap max_possible_feerate using commitment transaction.
The commitment tx uses the same feerate as the HTLC txs (which we have
to grind to find), but we can't use it directly since the fee could be
increased by the presence of dust HTLCs.  We can still use it to cap
the maximum though.

Time before: 1m6.984s
Time after:  0m15.794s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: onchaind is much faster when unilaterally closing old channels.
2020-12-07 14:23:59 +01:00
Rusty Russell 5bc61e77cb onchaind: speed up HTLC matching by skipping identical CLTVs.
We try signatures to see which HTLC (we can have many) is the right one;
we can trivially match htlcs against commitment tx outputs, but the CTLV
can vary, and that's inside the htlc tx itself.

By sorting them, it's easy to skip comparing duplicates:

Time before: 2m32.547s
Time after: 1m6.984s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell f22d719d2c onchaind: test stressful input.
In particular, this made valgrind OOM, and chewed much CPU.  I dumped the
input and output into a file to allow easy replay.

This will break as soon as we change onchaind's wire format, but it will
serve its purpose until then!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell ff8830876d wire/tlvstream: add tlv_make_fields helper to populate ->fields array.
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!

Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell 5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire.
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell 5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
niftynei 4508584b21 dualfund: rearrange things so that the wire-dependent calls are separate
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.

This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
2020-10-20 14:27:19 +10:30
niftynei 41ebf71e26 psbt: new methods for generating serial_ids for an input/output
we need to do this elsewhere later, pull it out so we can use it
2020-10-20 12:50:31 +10:30
niftynei 82c0b48215 wires: towire/fromwire for wally_tx
We're eventually moving away from 'bitcoin_tx
2020-10-20 12:50:31 +10:30
Rusty Russell f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Matt Whitlock eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
niftynei 5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei 303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
niftynei 8d429ecd06 df: add needed info to any PSBT we produce
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
2020-09-09 19:54:20 +09:30
Rusty Russell 0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes.
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
ZmnSCPxj jxPCSnmZ 06b44f00a3 onchaind/onchaind.c: Implement scorch-the-earth.
Fixes: #3832

Changelog-Changed: onchaind: We now scorch the earth on theft attempts, RBFing up our penalty transaction as blocks arrive without a penalty transaction getting confirmed.
2020-09-09 12:38:19 +09:30
ZmnSCPxj jxPCSnmZ abc585b7a4 onchaind/onchain_wire.csv: Allow suppression of auto-rebroadcasting for RBF txes created by onchaind. 2020-09-09 12:38:19 +09:30
ZmnSCPxj jxPCSnmZ 34bf0133f2 onchaind/onchaind_wire.csv: Propagate minimum relay fee to onchaind. 2020-09-09 12:38:19 +09:30
Rusty Russell d8e8426b52 Makefile: remove EXPERIMENTAL_FEATURES marker from generated files
We're going to make experimental versions of these completely separate files.

Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn.  We can always force-rebuild when we change a rule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell 82f2f43425 Generated files: more merge fallout.
We changed the generation templates!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 11:10:21 +09:30
Rusty Russell 3c6af3efb4 Makefile: commit and preserve all the wiregenerated files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: we no longer require extra Python modules to build.
2020-08-31 21:33:26 -05:00
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00