Commit Graph

243 Commits

Author SHA1 Message Date
Rusty Russell 1b2c6964fd Makefile: update spec version
This includes the new bolt11 test vectors, and also removes the
requirement that HTLCs be less than 2^32 msat.  We keep that for now
because Electrum enforced it on receive: in two releases we will stop
that too.

So no longer warn about needing mpp in that case either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Protocol: No longer restrict HTLCs to
2021-09-09 12:04:48 +09:30
niftynei 0f05238e97 psbt: set the witness script to null, also
Otherwise libwally pushes the psbt-key for 'witness script' onto the
serialized version and we fail the 'is this identical' check.

Relevant line from libwally, where if bytes, we push a psbt_key.

```
static void push_typed_varbuff(unsigned char **cursor, size_t *max,
                               uint64_t type,
                               const unsigned char *bytes, size_t bytes_len)
{
    if (bytes) {
        push_psbt_key(cursor, max, type, NULL, 0);
        push_varbuff(cursor, max, bytes, bytes_len);
    }
}
```

Reported-By: @grubles
Changelog-Fixed: openchannel_signed would fail on PSBT comparison of materially identical PSBTs
2021-09-05 14:24:14 +02:00
Rusty Russell cf1dd779d1 gossmap: don't spew to stderr, include counter for callers.
Fixes: #4722
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: don't drop complaints about silly channels to stderr.
2021-08-24 06:42:32 +09:30
Rusty Russell 17489120e6 common/test/run-bolt12_period.c: fix warnings at -O3.
assertions are useful here anyway, inside unit tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:57 +09:30
Rusty Russell 55e167115b common/tests: these test are no longer experimental-only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:57 +09:30
Rusty Russell 207f1def93 common/test: test the test vector JSON from the spec (if present).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-30 15:45:30 +09:30
Rusty Russell 83d9a0005d common/test: Fix test vector generation.
We were printing out the final merkle root before calculating it,
resulting in the final one being the same as the previous.

Reported-by: Aditya Sharma
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-30 15:45:30 +09:30
niftynei 8819278a88 lease rates: helpers for parsing, serializing + equating lease_rates
We're gonna need them
2021-07-20 13:28:38 -04:00
niftynei 63e2ce01e7 lease-rates: calculate the fee owed to peer for the funds lease 2021-07-20 13:28:38 -04:00
niftynei 9dd0a2c2e5 channel-lease: validate accepter's sig on lease terms 2021-07-20 13:28:38 -04:00
niftynei 5b3c02f401 liquidity-ads: import from spec
Import the wires from spec. Here we go!
2021-07-20 13:28:38 -04:00
Christian Decker 415c2bfe3c jsonrpc: Add parsers for routehint-arrays
We'll start passing routehints manually to keysend to reach non-public
nodes as well.
2021-07-15 12:13:06 -04:00
Rusty Russell 3b40bfa801 bolt12_merkle: change offer merkle to latest spec.
And fix up the mess we'd made:
1. We didn't order merkles by lesser-first.
2. We didn't correctly construct tree with last nodes on shortest path.

Now we have tests!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: protocol: offer signature format changed.
2021-07-14 14:33:18 -05:00
Christian Decker b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Rusty Russell 41767560d6 common/test: test wireaddr parsing.
In case this was the issue (it wasn't)!.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-17 11:46:48 +09:30
Antoine Poinsot fe8074c8c3 Refuse to parse v2 onion addresses without deprecated_apis
Tor v2 hidden services have been deprecated for a while:
https://blog.torproject.org/v2-deprecation-timeline .

This prevents user from being able to set them in the configuration
and to connect to them while still letting us be able to parse them
for gossip.

Changelog-Deprecated: lightningd: v2 Tor addresses.  Use v3.  See https://blog.torproject.org/v2-deprecation-timeline.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-05-24 20:22:45 +09:30
Rusty Russell 2bb365a931 common/route: route_from_dijkstra returns route_hop array.
This is what (most) callers actually want, so unify it into one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 167681c709 gossmap: fix gossmap_chan_get_announce() and gossmap_node_get_announce().
These were never used, nor tested.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell d54abd0e36 gossmap: function to get channel features bitmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 770a328d56 gossmap: save the offset where the channel updates are, function for details.
This takes an extra 8 bytes per channel, but means we can go back and
get more information about them; this is implemented in
gossmap_chan_get_update_details() which is what listchannels will need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell d1f8a5f0be gossmap: add gossmap_chan_get_capacity() helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell eb370ab8ff gossmap: remembers when a channel is private.
Steals one bit from the scid_off, and turns that into a relative
offset.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell cc198748d4 common/json_tok: hoist param_short_channel_id from inside lightningd/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 088cfced18 common/test/run-route and run-route-specific: modern gossmap tests
Equivalent to gossipd/test/run-find_route.c and gossipd/test/run-find_route-specific.c
except they use gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Christian Decker c78e2cc024 test: Fix memory leak in unit test run-gossmap_local
This was likely missed because we don't run the tests under valgrind anymore
due to time constraints. I do run them on a semi-regular basis, which is why
I found this.
2021-03-02 11:56:59 +10:30
Rusty Russell c5bd518d2f gossmap: change local API.
Now we create a separate set of local mods, and apply and unapply it.
This is more efficient than the previous approach, since we can do
some work up-front.  It's also more graceful (and well-defined) when a
local modification overlaps an existing one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:36 +10:30
Rusty Russell 1490ae4cf6 common: support transitory local map additions to gossmap.
This will let us add routehints to the map and use dijkstra etc like
normal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:36 +10:30
Rusty Russell cc4ea9420a common: extract fp16 routines into their own file.
We might want to use them elsewhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:36 +10:30
Rusty Russell 2de467274e common/amount: make fmt_amount_sat / fmt_amount_msat etc take copy.
We pass by copy everywhere else, let's do it here too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell 84dc943cf5 common/bolt11_json: extract bolt11->json code.
Our new "decode" command will also handle bolt11.  We make a few cleanups:

1. Avoid type_to_string() in JSON, instead use format functions directly.
2. Don't need to escape description now that JSON core does that for us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell 3b7d0e7a62 common/json: make json_scan return an error string.
This makes for more useful errors.  It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell fd7d1a1cc0 common: remove now-unused json_delve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell a5befb0072 common: enhance json_scan with simple array helpers.
In several places we want to access the first element of an array.
This uses a '[indexnum:xxx]' form which is a bit weird, but works similarly
to the way we specify member matches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 553daf17e0 common/json: new helper to scan JSON: json_scan.
This takes a JSON-style format string, and does intelligent parsing,
removing a lot of boilerplate from code which needs to deal with JSON.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell 1293852542 common/bolt12: add calculations for offer periods.
It's not trivial, but fortunately gmtime and mktime exist already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Christian Decker 1b8a2aba65 tests: Move ecdh stub out of autogenerated stubs
I ran into an issue when regenerating the stubs without
EXPERIMENTAL_FEATURES which then broke builds that have it set. Moving
the missing stub out.
2020-12-11 16:55:55 -06:00
Christian Decker b9fea7972f tests: Update mocks after breaking them via interleaved merges 2020-12-11 21:20:35 +01:00
Rusty Russell 32c7c133f4 common/sphinx: make onionpacket.routinginfo a dynamic member.
Still asserts that it's the standard size, but makes it a dynamic
member.  For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell b5ab7e3ce3 common/sphinx: don't use temporary to xor in cipher stream.
The chacha API makes this a bit awkward, to we use a helper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell 4d086e9939 common/bolt12: encode/decode for bolt12 offer, invoice_request and invoice
Note the collapse of '+\s*' and the test cases from the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 2734fd274e gossmap: helper to try to map x-only nodeid into 33-byte pubkey id.
This is a bridge for offers which use x-only pubkeys, and the current
code which doesn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 9d656464f5 bitcoin/pubkey: add pubkey32 primitive for xonly pubkey types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Christian Decker 47b8b46d54 cleanup: Remove stdio header from param.c 2020-12-09 06:56:21 +10:30
Christian Decker 32000b6660 json: Add two param functions to parse string arrs and outpoint arrs
In a couple of places we accept arrays of strings and don't validate
them. If we forward them, e.g., call a JSON-RPC method from the
plugin, we end up embedding the unverified string in the JSON-RPC
call without escaping, which then leads to invalid JSON being passed
on.

This at least partially causes #4238
2020-12-09 06:56:21 +10:30
Rusty Russell 5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell ca2bd98082 unittest: use common_setup / common_shutdown almost everywhere.
Avoids much cut & paste.  Some tests don't need any of it, but most
want at least some of this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:55:09 +10:30
Rusty Russell f0621cec0d JSON-RPC: don't allow any strings which aren't valid UTF-8.
We already do some sanity checks, add this one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: invalid UTF-8 strings now rejected.
2020-12-02 10:38:04 +10:30
Rusty Russell 5bdd282c2b common/bolt11: reject bad UTF-8 strings.
We don't have a problem with them, but callers may; easier to reject bad
UTF8 here than let the caller fail when it tries to parse output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 10:38:04 +10:30
Michael Schmoock bde2806290 chore: fixes a cppcheck false positive
Just applied the same suppression as rusty in:
6635fe12e4 (Rusty Russell    2020-05-15 15:57:29 +0930 146)
/* cppcheck-suppress uninitvar - false positive on f1->bits */

My cppcheck was complaining about the same issue in the following functions.
I wonder why travis does not care though.

Changelog-None
2020-11-03 10:18:08 -06:00