Commit Graph

11208 Commits

Author SHA1 Message Date
Christian Decker ecda4f717f cln-rpc: Move tests into separate file
These json structs are gigantic, so let's externalize them a bit.
2022-04-02 09:46:01 +10:30
Christian Decker 1613c44b0a cln-rpc: Make Pubkey and ShortChannelId proper types 2022-04-02 09:46:01 +10:30
Christian Decker ef145c7900 msggen: Add RoutehintList as a primitive 2022-04-02 09:46:01 +10:30
Christian Decker ec5cd92580 msggen: Add model-side overrides
Sometimes we just want to paper over the schema directly. Mostly
useful to sidestep the `oneof` things that are required for
expressiveness.
2022-04-02 09:46:01 +10:30
Christian Decker 1f40db3594 msggen: Add parser for "oneOf" 2022-04-02 09:46:01 +10:30
Rusty Russell c1ee32027d pyln-testing: check the request schemas.
This means suppressing schemas in some places too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell b45b731c55 doc/schemas: fixes for request schemas.
Types are fixed, in particular:

* rename "OutputDesc" to more consistent "outputdesc".
* rename "utxo" to more consistent "outpoint".
* it's "boolean" not "bool".
* "number" means int or float, usually it should be u32.

Specific commands:

* close `id` can be by channel id, scid.
* close `feerange` is a feerate type.
* datastore/deldatastore/listdatastore `key` can be singleton.
* delexpiredinvoice: `maxexpirytime` is not required, is a u64.
* invoice/delinvoice/listinvoice `label` can be an integer
* fundpsbt: many fields are u32 not number (JSON for int or float).
* invoice: `msatoshi` can be "any".
* invoice: `expiry` has a type (now must be numeric).
* invoice: `exposeprivatechannels` can be bool or array of scids.
* invoice: `deschashonly` added
* keysend: there's no "float" type, use "number" or "u32" etc.
* keysend: `routehints` is a valid arg, as is `extratlvs` (EXPERIMENTAL_FEATURES)
* listdatastore: `key` is not required.
* newaddr: `addresstype` can be "all"
* pay: `exemptfee` is "msat", new fields `locaofferid` and `exclude`
* sendonion: was mis-formatted, missed `localofferid` and `groupid` fields.
* sendpay: add `localofferid` and `groupid` params.
* signpsbt: add `signonly` param.
* txprepare "outptus" typo.
* waitsendpay: add `groupid` and  fix `partid` type.
* withdraw: `destination` is a bitcoin address, not a pubkey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell 3b11292e72 pyln-testing: add new schema types.
These are useful for requests:
1. "outpoint": <txid>:<outnum>
2. "feerate": strings or a number
3. "outputdesc": bitcoin-style addresses-as-keys
4. "msat_or_all": amount or "all"
4. "msat_or_any": amount or "any"
5. "short_channel_id_dir": scid with /0 or /1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell 1b6f4e7026 pytest: don't hand a string for an integer value.
lightningd is happy, but our schema won't be.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell 5d8fc84675 pyln-testing: add dev-invoice.
This will override the schema later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell 26bee7a2ab pyln-testing: add dev-pay.
This will override the schema later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell dc924b8859 doc: set additionalProperties to false in all request schemas.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell 9784fa816f lightningd: deprecate invoice expiry suffixes.
Makes types harder, and I've never personally used them.

Changelog-Deprecated: JSON-RPC: `invoice` `expiry` no longer allowed to be a string with suffix, use an integer number of seconds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell bf4d9e30d2 fundpsbt: deprecate reserve=true/false usage.
Make it always a number; this makes the JSON request specification
simpler.  We allowed a number since v0.10.1.

(reserve=True is the default anyway, so usually it can be omitted:
reserve=False becomes reserve=0).

Changelog-Deprecated: JSON-RPC: `fundpsbt`/`utxopsbt` `reserve` must be a number, not bool (for `true` use 72/don't specify, for `false` use 0).  Numbers have been allowed since v0.10.1.
2022-04-02 09:46:01 +10:30
Rusty Russell 3feb634d31 pytest: fix if no rust installed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Christian Decker bba68e2136 cln-grpc: Map AmountOrAll and AmountOrAny 2022-04-02 09:46:01 +10:30
Christian Decker 04e7e285d7 cln-rpc: Map PSBT and TX methods 2022-04-02 09:46:01 +10:30
Christian Decker 0354a7fdb1 cln-rpc: Add `OutputDesc` for `{addr: amt}` style arguments
This is likely inherited from bitcoind, and a bit awkward for us, so
we parse it into a classic struct, but serialize it back into the
bitcoind format when talking to the RPC.
2022-04-02 09:46:01 +10:30
Christian Decker eb2aa8c51c cln-rpc: Map feerates and backfill methods using it 2022-04-02 09:46:01 +10:30
Christian Decker 20704746bc grpc: Map `sendonion` and `createonion` 2022-04-02 09:46:01 +10:30
Christian Decker 565518246a msggen: Add custom mappings for JSON -> grpc conversions 2022-04-02 09:46:01 +10:30
Christian Decker a7f3c54600 cln-rpc: Add invoice RPC methods 2022-04-02 09:46:01 +10:30
Christian Decker d90aafeed6 cln-grpc: Add the `connect` method 2022-04-02 09:46:01 +10:30
Christian Decker 646901f588 msggen: Wrap field numberings in the message type
Otherwise we get less nicely consecutive numbers because request and
response share one domain. This separates them again.
2022-04-02 09:46:01 +10:30
Rusty Russell 01e5f1886e pytest: fix flake in test_multichan.
I have a separate branch which fixes this race properly, but it's not anything
to do with this PR.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell ec72d89975 bolt11: reorder invoice production to match test vectors.
After this, we can exactly reproduce the vectors (in DEVELOPER mode).

1. Move payment_metadata position to match test vector.
2. Create flag to suppress `c` field production.
3. Some vectors put secret before payment_hash, hack that in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 7e789be0ea doc: update BOLTs to latest master.
Just typo fixes and the like.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 1c685ce3e8 plugins/pay: send payment_metadata if provided in invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: `pay` (and decode, etc) supports bolt11 payment_metadata a-la https://github.com/lightning/bolts/pull/912
2022-04-02 09:40:18 +10:30
Rusty Russell 7f89763f9e sendpay: add payment_metadata argument.
And document the missing arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 4718ee076c lightningd: metadata received support (log and decline).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell e01abf0b34 bolt11: support payment_metadata.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 2526e804f7 doc: big BOLT update to incorporate warnings language.
We do this (send warnings) in almost all cases anyway, so mainly this
is a textual update, but there are some changes:

1. Send ERROR not WARNING if they send a malformed commitment secret.
2. Send WARNING not ERROR if they get the shutdown_scriptpubkey wrong (vs upfront)
3. Send WARNING not ERROR if they send a bad shutdown_scriptpubkey (e.g. p2pkh in future)
4. Rename some vars 'err' to 'warn' to make it clear we send a warning.

This means test_option_upfront_shutdown_script can be made reliable, too,
and it now warns and doesn't automatically close channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 9f06a59e3c shutdown: don't allow shutdown to p2pkh or p2sh addresses for anchor outputs.
This doesn't have an effect now (except in experimental mode), but it
will when we support anchors.  So we deprecate the use of those in the
close command too.

For experimental mode we have to avoid using p2pkh; adapt that test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `shutdown` no longer allows p2pkh or p2sh addresses.
2022-04-02 09:40:18 +10:30
Rusty Russell 7491af5495 doc: update BOLT references part1, including bolt11 test vectors with payment_secret.
The signatures on the new examples are sometimes different from what we produce though?
They're valid, however.

And one example has an unneeded feature 5-bit; it's not *wrong*, but
it's not optimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 3b536400f9 specs: update CSV files.
Regenerate from current BOLTS via `make extract-bolt-csv`

1. The remote_addr field was added manually into peer_wire.csv: this
   needs to be a patch otherwise it vanishes on regen.
2. We never brought into the channel_disabled fields, because it was
   too much hassle (we never actually generate this!).  Do it now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 906fb6ca86 wire/tlvstream: const.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell 9bddfc2048 connectd: take dev-suppress-gossip from gossipd.
Gossipd didn't actually suppress all gossip, resulting in a flake!
Doing it in connectd now makes much more sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 19:38:05 +10:30
Rusty Russell 2f7f7ec255 lightningd: add "style" to listforwards.
Suggested-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` has new entry `style`, currently "legacy" or "tlv".
2022-03-31 19:38:05 +10:30
Rusty Russell 116a77f1be lightningd: Restore forwarding of legacy onions.
Partial revert of 43a833e405
"lightningd: remove support for legacy onion format."; we restore the
ability to decode legacy onions for forwarding, but not to generate them.
(We don't accept them properly since making payment_secret compulsory
anyway, so no real change there!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: ... but we still forward legacy HTLC onions for now.
2022-03-31 19:38:05 +10:30
Rusty Russell 141d4ef675 pytest: test legacy onion acceptance.
Using a canned sendonion call from an old version (v0.10.2), where I
forced it to use legacy onion for l2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 19:38:05 +10:30
Rusty Russell 910d594214 doc: add partid to sendonion schema.
We never called this except from plugins, which didn't test schema.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 19:38:05 +10:30
Rusty Russell cd9ce92d28 onchaind: fix htlc_stub organization.
We get sent three corresponding arrays:
1. htlc stubs
2. whether we want to know if they're missing,
3. whether to wait 3 blocks or tell us immediately

We then sorted the htlc stubs by CLTV, *but didn't sort the corresponding arrays*.

This fixes that the simplest way possible, and probably also:

Fixes: #4649

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: onchaind: we sometimes failed to close upstream htlcs if more than one HTLC is in flight during unilateral close.
2022-03-31 13:40:27 +10:30
Rusty Russell 861922fb1c channeld: log more information about restoring HTLCs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell 0b7f789291 lightningd: extra sanity checks and rescue attempts for missing HTLCs.
These trip when anything weird happens; turns out that we tell
onchaind about old htlcs (e.g. for penalties), so in that case we can
actually have it tell us about missing HTLCs which we no longer have
in memory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell e616b4fff7 lightningd: add extra debugging for weird onchain htlc interactions.
This doesn't actually fix anything, but may shed more clues if it
happens again.

The broken() logs are overzealous, see next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell 5033e22835 pytest: make test_onchain_close_upstream more accurately reflect report.
1. Don't use dust HTLCs.
2. Make l3 unresponsive, like report.
3. Make l2-l3 fail because we time out on successive HTLC.

We use sendpay rather than pay, because pay can do multiple attempts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell 1e08c3b882 pytest: simple "does onchaind on missing HTLC cause upstream failure" test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell ea7120a313 lightningd: add --dev-no-ping-timer to avoid ping response timeouts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Vincenzo Palazzo a35e1b23e6 docker-ci: fixed script path inside the entry point
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-31 13:01:04 +10:30
Vincenzo Palazzo 154a391cf7 ci: adding pytest parameter to tracing lnprotetest
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-31 13:01:04 +10:30