Using a 'feestep' is more restrictive than you'd want, instead we
enforce that the next feerate must be at least 1/64th more than the
last, but put no upper limit on it
Includes update to lnprototest changes
Contributed-By: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: Protocol: Replaces init_rbf's `fee_step` for RBF of v2 opens with `funding_feerate_perkw`, breaking change
We don't support it (yet), but update the spec to include it.
We include the previous field (recurrence_signature) as a shim for the
moment, for compat with existing nodes. It's ugly, but next release
we'll stop *sending* it, then finally we'll stop accepting it!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I wasn't regenerating these when I changed dependencies, because I was
configured with --enable-experimental-features. Putting them in
ALL_C_HEADERS and ALL_C_SOURCES means they'll be regenerated, even
though nothing depends on them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
We consolidate to the latest/singular RFC patch for dual-funding, so
there's just a single patchfile for the change. Plus we move back to the
opener setting the desired feerate, the accepter merely declines to
participate if they disagree with the set rate.
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.
Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
Caused by missing common/iso4217.c from common/Makefile:
```
In file included from ./common/iso4217.h:4,
from common/iso4217.c:3:
./wire/wire.h:7:10: fatal error: secp256k1_recovery.h: No such file or directory
7 | #include <secp256k1_recovery.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:265: common/iso4217.o] Error 1
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We assume if they set this to 0 (which nobody did previously), they're
using it as a modern flag and use it to indicate when they're
finished. Otherwise, we count how many blocks they've sent and use
that to determine whether they've finished.
See: https://github.com/lightningnetwork/lightning-rfc/pull/826
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we use `sync_complete` for gossip range query replies, with detection for older spec nodes.
This takes from the draft spec at https://github.com/lightningnetwork/lightning-rfc/pull/834
Note that if this draft does not get included, the peer will simply
ignore the warning message (we always close the connection afterwards
anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now report the new (draft) warning message.
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers. But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.
Weaken that check, and include config.h in C files before assert.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
Don't include exp directly, use an ifdef in common/bolt12
(like we do for peer and onion wiregen files).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For some reason, an old version was checked in. We don't currently
use it (since bolt12 support depends on EXPERIMENTAL_FEATURES, which
uses bolt12_exp_wire.csv, which is generated and not checked in).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And rename the current patches to all be _exp_*.patch. These
are applied after the basic patches.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fix.patch':
The previous onion_message code required a confirmed, not-shutting-down
channel, not just a connection. That's overkill; plus before widespread
adoption we will want to connect directly as a last resort.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database. We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!
Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can use a fixed value and close the channel if they don't cover their
amount; this wasn't really helping with anything other than setting a
floor for an expected feerate
There's a lot of it, and it means we can't `make check-source` on
these files.
Also bring bolt quotes up-to-date.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We didn't rename the _csv to .csv, which patterns expect if you're
regenerating (e.g. if you switch on --enable-experimental-features
then switch it off again!)
Also remove unused experimental targets.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids overwriting the ones in git, and generally makes things neater.
We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to make experimental versions of these completely separate files.
Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn. We can always force-rebuild when we change a rule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS. Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Also, remove fuzz caused by varint->bigsize change.
For some reason my build machine sorts patches into another order, and fails
to patch:
patching file wire/gen_onion_wire_csv.104951
Hunk #1 succeeded at 52 with fuzz 1 (offset -19 lines).
patching file wire/gen_onion_wire_csv.104951
Hunk #1 FAILED at 8.
1 out of 1 hunk FAILED -- saving rejects to file wire/gen_onion_wire_csv.104951.rej
make: *** [wire/Makefile:60: wire/gen_onion_wire_csv] Error 1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
libwally's API requires us to pass in NULL pointers if the array size is
zero, so we update our array from wire-er to comply with this
requirement
[ Added fix to avoid tal_resize() of NULL -- RR ]
Trying to rework the TLV streams to have a more homogenous interface to work
with. This is by no means a complete implementation, just the groundwork that
is going to be used by the wire code generator to generate the specific
accessors, but it's enough so we can manipulate TLV streams in the onion and
later just switch to the generated ones.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway). In fact, the open/accept is a TLV
because it was actually the same format.
For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381
Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case. Use that more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those. Now we do, so
there's less excuse for this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Otherwise, it doesn't depend on anything (it's compiled becasue
plugins/keysend explicitly lists wire/tlvstream.o as a dependency).
This made me miss a compile break.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward. This changes it to directly
access those fields.
This is not only more elegant (60 fewer lines), it would also be
more cache friendly. That's right: cache hot singles!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were using patch's '--silent' flag, but that broke on the busybox
implementation of `patch`, since they don't support it. Instead, we use
the universally supported "pipe to /dev/null" approach
Suggested-By: @rustyrussell
Changelog-None
The generated wrappers will ignore the raw fields and will only consider the
shortcut fields. This function takes the raw fields and serializes them
instead.
Note that it's channeld which calculates the shared secret, too. This
minimizes the work that lightningd has to do, at cost of passing this
through.
We also don't yet save the blinding field(s) to the database.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Does the allocation and copying; this is useful because we can
avoid being fooled into doing giant allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a common thing to do, so create a macro.
Unfortunately, it still needs the type arg, because the paramter may
be const, and the return cannot be, and C doesn't have a general
"(-const)" cast.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Before this patch we used to send `double`s over the wire by just
copying them. This is not portable because the internal represenation
of a `double` is implementation specific.
Instead of this, multiply any floating-point numbers that come from
the outside (e.g. JSONs) by 1 million and round them to integers when
handling them.
* Introduce a new param_millionths() that expects a floating-point
number and returns it multipled by 1000000 as an integer.
* Replace param_double() and param_percent() with param_millionths()
* Previously the riskfactor would be allowed to be negative, which must
have been unintentional. This patch changes that to require a
non-negative number.
Changelog-None
GCC 10 defaults to `-fno-common`. no longer automatically sharing
global variable definitions, which makes it important to define
them in only one place (otherwise there will be duplicate definition
errors). Add `extern` qualifiers where (I think) is the best place for
them.
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.
To resolve this:
* Introduce an error code type with a known size:
`typedef s32 errcode_t`.
* Change all error code macros to constants of type `errcode_t`.
Constants also play better with gdb - it would visualize the name of
the constant instead of the numeric value.
* Change all functions that take error codes to take the new type
`errcode_t` instead of `int`.
* Introduce towire / fromwire functions to send / receive the newly added
type `errcode_t` and use it instead of `towire_int()`.
In addition:
* Remove the now unneeded `towire_int()`.
* Replace a hardcoded error code `-2` with a new constant
`INVOICE_EXPIRED_DURING_WAIT` (903).
Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
These messages may be exchanged between the master and any daemon. For now
these are just the daemons that a peer may be attached to at any time since
the first example of this is the custommsg infrastructure.
Add towire_int() and fromwire_int() functions to "(de)serialize"
"int". This will only work as long as both the caller of towire_int()
and the caller of fromwire_int() use the same in-memory representation
of signed integers and have the same sizeof(int).
Changelog-None
Thanks to @t-bast, who made this possible by interop testing with Eclair!
Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Also pulls in a new onion error (mpp_timeout). We change our
route_step_decode_end() to always return the total_msat and optional
secret.
We check total_amount (to prohibit mpp), but we do nothing with
secret for now other than hand it to the htlc_accepted hook.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently the only source for amount_asset is the value getter on a tx output,
and we don't hand it too far around (mainly ignoring it if it isn't the
chain's main currency). Eventually we could bubble them up to the wallet, use
them to select outputs or actually support assets in the channels.
Since we don't hand them around too widely I thought it was ok for them to be
pass-by-value rather than having to allocate them and pass them around by
reference. They're just 41 bytes currently so the overhead should be ok.
Signed-off-by: Christian Decker <@cdecker>
We now have a pointer to chainparams, that fails valgrind if we do anything
chain-specific before setting it.
Suggested-by: Rusty Russell <@rustyrussell>
We used to append new patches to a single file. This caused
some problems and is a lot harder to cleanup later.
This patch moves the experimental patches to their own, individual
patch files, that are named for the current BOLTVERSION, which they're
taken from.
Also moves the current patchfile over to a 'gossipqueries' one,
as it already exists.
531c8d7d9b
In this one, we always send my_current_per_commitment_point, though it's
ignored. And we have our official feature numbers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This removes the WIRE_FINAL_EXPIRY_TOO_SOON which leaked too much info,
and adds the blockheight to WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Make the TLV element a simple array. This is a bit neater, in fact, and
makes the test vectors in that 557 PR work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Use a pointer, so it's explicit and gcc is happy. We avoid the
allocation by pointing it to another stack var.
./wire/tlvstream.c:81:22: error: ‘prev_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're currently called varint, but there's a proposal to call them all
bigsize. Allow both for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to hand -s to both header and body generation, or neither:
wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here
We also need it for printwire, otherwise we get static unused functions for subtypes:
devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The new TLV spec uses BigSize, like Bitcoin's CompactInt but
*little-endian*. So change our name for clarity, and insist that
decoding be minimal as the spec requires.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.
this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
This means we intercept the peer's gossip_timestamp_filter request
in the per-peer subdaemon itself. The rest of the semantics are fairly
simple however.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out the peer part of the spec no longer uses padding (it's used only
in the onion), and GCC-9 with -O3 warns we're padding NULL to memcpy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
for now we straight copy the `extracted_peer_wire_csv` over into the
file that is used to generate the .c/.h files; in the future
we'll use this destination file as a way to modify the
`gen_peer_wire_csv`s from a patch.
We can save significant space by combining both sides: so much that we
can reduce the WIRE_LEN_LIMIT to something sane again.
MCP results from 5 runs, min-max(mean +/- stddev):
store_load_msec:34467-36764(35517.8+/-7.7e+02)
vsz_kb:2637488
store_rewrite_sec:35.310000-36.580000(35.816+/-0.44)
listnodes_sec:1.140000-2.780000(1.596+/-0.6)
listchannels_sec:55.390000-58.110000(56.998+/-0.99)
routing_sec:30.330000-30.920000(30.642+/-0.19)
peer_write_all_sec:50.640000-53.360000(51.822+/-0.91)
MCP notable changes from previous patch (>1 stddev):
-store_rewrite_sec:34.720000-35.130000(34.94+/-0.14)
+store_rewrite_sec:35.310000-36.580000(35.816+/-0.44)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Don't turn them to/from pubkeys implicitly. This means nodeids in the store
don't get converted, but bitcoin keys still do.
MCP results from 5 runs, min-max(mean +/- stddev):
store_load_msec:33934-35251(34531.4+/-5e+02)
vsz_kb:2637488
store_rewrite_sec:34.720000-35.130000(34.94+/-0.14)
listnodes_sec:1.020000-1.290000(1.146+/-0.086)
listchannels_sec:51.110000-58.240000(54.826+/-2.5)
routing_sec:30.000000-33.320000(30.726+/-1.3)
peer_write_all_sec:50.370000-52.970000(51.646+/-1.1)
MCP notable changes from previous patch (>1 stddev):
-store_load_msec:46184-47474(46673.4+/-4.5e+02)
+store_load_msec:33934-35251(34531.4+/-5e+02)
-vsz_kb:2638880
+vsz_kb:2637488
-store_rewrite_sec:46.750000-48.280000(47.512+/-0.51)
+store_rewrite_sec:34.720000-35.130000(34.94+/-0.14)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
niftynei
Rusty Russell
Christian Decker
fiatjaf
YOSHIDA Masanori
Matt Whitlock
Michael Schmoock
Vasil Dimov
Ken Sedgwick
Wladimir J. van der Laan
darosior
trueptolemy