Commit Graph

1404 Commits

Author SHA1 Message Date
Rusty Russell 3b4c1968a3 common/test: add unit tests for JSON filtering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell f0731d2ca1 common/json_stream: support filtering don't print fields not allowed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell 508a170598 common/json_filter: routine to turn "filter" JSON into a filter.
Since the "struct command" is different from plugins and lightningd, we
need an accessor for this to work (the plugin one is a dummy for now!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell 3c75770586 common/json_filter: routines for json filtering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell a4c482dc07 common/sphinx: don't use fixed lengths anywhere.
1. Remove the very concept of ONION_REPLY_SIZE, instead make it a
   local variable in create_onionreply().

2. Use the proper fromwire_ primitives in unwrap_onionreply() so we
   don't have to do explicit length checks.

3. Make fromwire_tal_arrn() return NULL if it fails to pull, instead of
   a zero-length allocation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we now correctly decrypt non-256-length onion errors (we always forwarded them fine, now we actually can parse them).
2022-11-08 17:40:57 +01:00
Rusty Russell 75c382fe16 lightningd: --dev-onion-reply-length option.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-08 17:40:57 +01:00
Dustin Dettmer 76623b2ef2 Collaborative transaction building
Takes the dualopen collaborative transaction building and makes it available for other daemons to use

Changelog-Added: Added interactive transaction building routine
2022-11-02 12:00:11 +01:00
Christian Decker 8a4f44a58d keysend: Allow quoted numbers in `extratlvs`
This is because JSON technically does not allow numeric keys in maps.

Changelog-Added: JSON-RPC: The `extratlvs` argument for `keysend` now allows quoting the type numbers in string
2022-11-01 17:05:30 +01:00
Christian Decker 83beaa5396 json: Add helper for quoted numbers
The JSON specification technically disallows maps with numeric keys,
so we'll want to slowly migrate away from using them. This helper
extracts the numeric value from a quoted number, which is a legal
representation of the same in JSON.
2022-11-01 17:05:30 +01:00
Rusty Russell 422e68a4d6 common/blindedpath: create onion mesage test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 159fc7d1a2 common/onion_message_parse: generic routine for parsing onion messages.
Instead of open coding in connectd/onion_message, we move it to common
with a nice API.

This lets us process the BOLT test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 5cf86a1a2e common: update to latest onion message spec.
Mainly, field name changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Protocol: Support for forwarding blinded payments (as per latest draft)
2022-10-26 11:29:06 +10:30
Rusty Russell 67b8fadf02 common/test: check we meet bolt04/onion-route-blinding-test.json
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 511e8e6477 common/blindedpay: routines to construct a blinded payment.
Don't shoehorn it into onion_nonfinal_hop() and onion_final_hop(), but
provide an explicit routine "blinded_onion_hops" and an onion helper
"onion_blinded_hop()" for it to call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 325fe2e04c common/onion: cleanup by removing unnecessary local temporary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 21e7c3432e common/onion: enforce payment constraints.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 077ec99788 common/onion: blinded payment support.
We make it look like a normal payment for the caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell c94c742e58 common/features: understand the route_blinding feature (feature 24)
We don't set it, but we know how to now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell c0ae2394d8 common/blindedpath: generalize construction routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 53e40c4380 common/blindedpath: generalize routines.
We're going to share them for onion messages as well as for blinded
payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 85baca56c6 channeld: don't calculate blinding shared secret, let lightningd do it.
It's a premature optimization, and it make modifications more complex.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 1d4f1a5199 common: remove old route-blinding-override test, update route-blinding test for new vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 662c6931f3 Remove point32.
The x-only dream is dead.  Remove all trace.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell e30ea91908 BOLTs: update to more recent bolt12 spec.
It's 2b7ad577d7a790b302bd1aa044b22c809c76e49d, which reverts the
point32 changes.

It also restores send_invoice in `invoice`, which we had removed
from spec and put into the recurrence patch.

I originally had implemented compatibility, but other changes
which followed this are far too widespread.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: complete rework of spec from other teams (yay!) breaks previous compatibility (boo!)
2022-10-26 11:29:06 +10:30
Rusty Russell 82d98e4b96 gossmap: move gossmap_guess_node_id to pay plugin.
This removes a point32 dependency.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 7745513c51 bolt12: change our payer_key calculation.
It was very tied to x-only keys; we could support it in a backwards
compatibility mode for a while, but getting refunds or proving old
pre-finalization invoices is not worth spending time on.

Changelog-EXPERIMENTAL: offers: old `payer_key` proofs won't work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 4e39b3ff3d hsmd: don't use point32 for bolt12, but use pubkeys (though still always 02)
This is the one place where we hand point32 over the wire internally, so
remove it.

This is also our first hsm version change!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell bed905a394 lightningd: use 33 byte pubkeys internally.
We still use 32 bytes on the wire, but internally don't use x-only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 987adb9718 Makefile: check that hsm_version.h changes if wire/hsmd_wire.csv contents does
Otherwise I know we'll miss it.  Simply check for a mention: we could well
change things multiple times within a single release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell fee9a7ce04 hsmd: introduce a simple API versioning scheme.
With the rise of external HSMs like VLS, this is no longer an
internal-only API.  Fortunately, it doesn't change very fast so
maintenance should not be a huge burden.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 2749b1f61e common/onion: remove old blinded payment handling.
It's been obsoleted and needs replacing; less confusing if we remove
it first.

Also, these fields are now present even without an expermintal build
(we'll control at runtime).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 41ef85318d onionmessages: remove obsolete onion message parsing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 342e330b56 doc: update references to old BOLTs repo.
This reveals that common/test/run-bolt12_merkle-json.c was broken!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell f00cc23f67 sphinx: rename confusing functions, ensure valid payloads.
"sphinx_add_hop" takes a literal hop to include,
"sphinx_add_modern_hop" prepends the length.  Now we always prepend a
length, make it clear that the literal version is a shortcut:

* sphinx_add_hop -> sphinx_add_hop_has_length
* sphinx_add_modern_hop -> sphinx_add_hop

In addition, we check that length is actually correct!  This means
`createonion` can no longer create legacy or otherwise-invalid onions:
fix tests and update man page to remove legacy usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `createonion` no longer allows non-TLV-style payloads.
2022-09-28 13:40:57 +02:00
Rusty Russell 8771c86379 common/onion: expunge all trace of different onion styles.
In particular, remove special routines to pull length: it's there,
take it and check it yourself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell c8ad9e18a9 common/onion: remove all trace of legacy parsing.
We still have an "enum forward_style" for the database, where old-style
forwards can still exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer forward HTLCs with legacy onions.
2022-09-28 13:40:57 +02:00
Rusty Russell 1de4e46276 tests: add onion-test-vector from "BOLT 4: Remove legacy format, make var_onion_optin compulsory."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell f53155d93b BOLT: update to clarify HTLC tx amount calculation.
Simple quote update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell bb49e1bea5 common: assume htlc_maximum_msat, don't check bit any more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell 253b25522b BOLT: update to version which requires option_channel_htlc_max.
We will now simply reject old-style ones as invalid.  Turns out the
only trace we could find is a channel between two nodes unconnected to
the rest of the network.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now require all channel_update messages include htlc_maximum_msat (as per latest BOLTs)
2022-09-24 15:22:27 +09:30
Rusty Russell daa5269ea2 gossipd: bump gossip_store to indicate all channel_update have htlc_max.
And in the next patch, gossipd will no longer put new ones in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell 6338758018 gossmap: make API more robust against future changes.
Many changes to gossmap (including the pending ones!) don't actually
concern readers, as long as they obey certain rules:

1. Ignore unknown messages.
2. Treat all 16 upper bits of length as flags, ignore unknown ones.

So now we split the version byte into MAJOR and MINOR, and you can
ignore MINOR changes.

We don't expose the internal version (for creating the map)
programmatically: you should really hardcode what major version you
understand!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell 45cdfd2ff7 BOLT: update to fix gossip pruning quote.
Which I disagreed with, and has been fixed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell 3079afb024 lightningd: add `delforward` command.
Changelog-Added: JSON-RPC: `delforward` command to delete listforwards entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Christian Decker 75fe1c1a66 common: Add multiplication primitives for amount_msat and amount_sat
The scale variants weren't usable since they use `double` as a scale
factor, which in turn causes imprecisions.
2022-09-21 11:25:47 +02:00
Rusty Russell 5b58eda748 libplugin: mark the cmd notleak() whenever command_still_pending() called.
This is what we do in lightningd, which makes memleak much more forgiving:
you can hang temporaries off cmd without getting reports of leaks (also
when send_outreq called).

We remove all the notleak() calls in plugins which worked around this!
And avoid multiple notleak labels, since both send_outreq() and
command_still_pending() can be called multiple times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell 701dd3dcef memleak: remove exclusions from memleak_start()
Add memleak_ignore_children() so callers can do exclusions themselves.

Having two exclusions was always such a hack!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell 3380f559f9 memleak: simplify API.
Mainly renaming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell fd71dfc7f7 gossmap: optimize asserts().
They are surprisingly expensive!

Running `time ./plugins/renepay/test/run-not_mcf-gossmap gossip_store-sgl.rustcorp.com.au-2022-04-19 024b9a1fa8e006f1e3937f65f66c408e6da8e1ca728ea43222a7381df1cc449605 02ebb3b8a2316b3e876ea3f3d8124a3ab97f30b128f619608eb06b5251235dc2d9 10000000000 0.1`:

Before (-Og):
	real	0m1.495s
Before (no opt):
	real	0m2.552s

After (-Og):
	real	0m0.579s
After (no opt):
	real	0m1.061s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell 4cdb4167d2 gossmap: make local_addchan create private channel_announcement in correct order.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30