Commit Graph

112 Commits

Author SHA1 Message Date
Riccardo Casatta 9c35f9c13a Implement conversion JSON->GRPC also for requests type
This could be useful for existing software with  a JSON interface that want
to mimic the interface
2023-02-08 12:03:27 -06:00
Christian Decker e5d5737927 grpc: Allow conversion code to use deprecated fields
The warning was rather superfluous, we should rather annotate the
downstream structs so the developer gets a warning, not us library
maintainers.
2023-02-07 08:23:17 -06:00
Carl Dong 11227d37ba msggen: Enable SignInvoice 2023-02-06 15:54:32 -06:00
Rusty Russell 4b9cb7eb76 doc: remove unused offerout schema.
We removed the command for v22.11.

Also, we removed the `refund_for` offer parameter, so remove its description
from the manpage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Riccardo Casatta 182a9cdcb6 cln-rpc: use serde rename instead of alias
rename is necessary to roundtrip, otherwise the rust name is used.

This also remove the rename if they are not necessary.

Note that:
```
 #[serde(rename="foo", skip_serializing_if=="bar")]
 pub field: bool,
```

is equivalent to:
```
 #[serde(rename="foo")]
 #[serde(skip_serializing_if=="bar")]
 pub field: bool,
```

and for simplicity of construction the latter is used
2023-01-26 15:03:54 +01:00
Jesse de Wit 241cd8d012 generate composite fields in grpc 2023-01-03 10:56:10 +01:00
Christian Decker 8e75232205 rs: Update outdated dependencies
This commit updates outdated dependencies and hangs all
bitcoin-related dependencies off of the `bitcoin` crate, using its
re-exports. This means that as long as the bitcoin crate matches, all
of its dependents will also match.

Changelog-None
2022-12-22 15:42:12 +01:00
Rusty Russell 5b3746172f lightningd: remove `setchannelfee`.
Replaced by `setchannel`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changlog-Removed: JSON-RPC: `setchannelfee` (deprecated in v0.11.0)
2022-12-13 08:28:12 +10:30
Christian Decker 9481fb8815 msggen: Be less magic in detecting the repo root
We were assuming that we are running in a git repo, which is not true
for the PPA
2022-11-18 15:10:32 +01:00
Christian Decker 1a1c0a38fc msggen: Map the `extratlvs` field of `keysend`
Changelog-Added: cln-rpc: `keysend` now exposes the `extratlvs` field
2022-11-01 17:05:30 +01:00
joemphilips 52be59587c msggen: generate deprecated fields in rust.py 2022-09-29 11:18:06 +02:00
Justin Litchfield 51e2433087 Setchannel request is provided 2022-09-28 15:13:07 +02:00
elsirion b4b0b479ac Use `secp256k1` for public key type 2022-09-26 22:05:30 +02:00
elsirion e272c93a88 Use `bitcoin_hashes` for `Sha256` 2022-09-26 22:05:30 +02:00
elsirion 10917743fe Implement a typed version of `call` to avoid useless matching 2022-09-26 22:05:30 +02:00
Christian Decker b13ab8de3a msggen: Use owned versions to convert from cln-rpc to cln-grpc
We needed to clone a lot since we were using a non-mutable borrow of
the cln-rpc version to then create the cln-grpc version. This alone
reduces the number of allocations of the `listpeers` test from 52
allocations to 38.
2022-09-25 15:13:12 +02:00
Christian Decker b9a7f36ab3 msggen: Add conversion from cln-rpc to cln-grpc for Option<Outpoint>
We didn't have optional Outpoints as arguments so far, so let's
backfill that.

Changelog-Changed: cln-rpc: The `wrong_funding` argument for `close` was changed from `bytes` to `outpoint`
2022-09-22 10:41:05 +02:00
Christian Decker b55df5c626 msggen: Use tempfile + rename to make changes to .msggen.json atomic
This was causing issues when multiple instances of msggen were running
in parallel on CI.

Changelog-None
2022-07-26 09:49:20 -07:00
Christian Decker 3f79536437 msggen: Ignore `state_changes` in grpc2py
It's being skipped in grpc so we don't have it later anyway.
2022-07-21 14:19:06 +09:30
Christian Decker 18a9eb2feb msggen: Add `stop` method to generators
We'll need this when testing the grpc interface in pyln-testing,
otherwise tests just slowly die and wither.
2022-07-21 14:19:06 +09:30
Christian Decker 12275d0bfe cln-grpc: Skip serializing fields when Option<Vec<T>> is empty too
The CLN API is rather strict about the fact that we should skip
providing a field whenever it is empty. Checking for `is_none` would
still include empty arrays.

Changelog-Fixed cln-rpc: Optional empty arrays will no longer be serialized in requests
2022-07-21 14:19:06 +09:30
Christian Decker 77f5eb556b msggen: Add fundchannel request 2022-07-21 14:19:06 +09:30
Christian Decker 5307586d4d msggen: Add a new generator for grpc -> python converter
To test the grpc interface we'll want to emulate the JSON-RPC
interface as best we can, hence when talking to the grpc interface we
want to convert back into a parsed JSON format as LightningRpc would
have returned it. This is just the simplest way of closing the loop
here:

```
  pyln-testing --grpc-> cln-grpc --grpc2json
     ^                                  |
     |                                  v
     |                               JSON-RPC
     |                                  |
    TEST                                v
     ^                                 CLN
     |                                  |
     |                                  v
  pyln-testing <-grpc2py-- cln-grpc <- json2grpc
```
2022-07-21 14:19:06 +09:30
Christian Decker 2d35c9a929 msggen: Do not override method names when loading Schema
We were overriding the name right when loading, which is bad since in
some languages we use the method name as tag in the requests, thus
renaming causes us to call something that isn't defined.

Changelog-Fixed: cln-rpc: Fixed a naming mismatch for `ConnectPeer` causing `connectpeer` to be called on the JSON-RPC
2022-07-21 14:19:06 +09:30
Rusty Russell 9685c1adaf lightningd: remove getsharedsecret.
This was introduced to allow creating a shared secret, but it's better to use
makesecret which creates unique secrets.  getsharedsecret being a generic ECDH
function allows the caller to initiate conversations as if it was us; this
is generally OK, since we don't allow untrusted API access, but the commando
plugin had to blacklist this for read-only runes explicitly.

Since @ZmnSCPxj never ended up using this after introducing it, simply
remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSONRPC: `getsharedsecret` API: use `makesecret`
2022-07-15 22:17:58 +09:30
Vincenzo Palazzo 2ab2061cc9 msggen: adding example and fixes typo
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-05-07 11:11:51 +02:00
Vincenzo Palazzo 4e902fbd88 msggen: introduce chain of responsibility pattern to make msggen extensible
Changelog-Added: msggen: introduce chain of responsibility pattern to make msggen extensible

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-05-07 11:11:51 +02:00
joe.miyamoto 562974acdb Fix miner bug in rust client generator.
This commit fixes a bug in a function `gen_enum`, which
is not caught because so far we have no non-required field in enums
defined in json schema.
2022-04-21 16:43:50 +09:30
Christian Decker be04f25666 msggen: Don't assume we have properties in the schema 2022-04-02 09:46:01 +10:30
Christian Decker 241a891190 cln-rpc: Yet more RPC methods being mapped
- disconnect
 - feerates
 - getroute
 - listforwards
 - listpays
 - ping
 - signmessage
2022-04-02 09:46:01 +10:30
Christian Decker d0f6e8c8a6 cln-rpc: Add Sha256 and Secret types 2022-04-02 09:46:01 +10:30
Christian Decker cd9c00b629 cln-rpc: Log requests and responses 2022-04-02 09:46:01 +10:30
Christian Decker 940f5c350c cln-rpc: Skip the HTLC state for now
It's schema definition is weirdly asymmetric, with variants dependent
on another fields' value. Need to decide if we want to either
hand-code a superset or make a more complex decoding, but definitely
not something we'd want the generator to be able to do.
2022-04-02 09:46:01 +10:30
Christian Decker d5f7548c8f cln-rpc: The JSON `number` type is a float not an int
Got some issues parsing dates for example.
2022-04-02 09:46:01 +10:30
Christian Decker ecda4f717f cln-rpc: Move tests into separate file
These json structs are gigantic, so let's externalize them a bit.
2022-04-02 09:46:01 +10:30
Christian Decker 1613c44b0a cln-rpc: Make Pubkey and ShortChannelId proper types 2022-04-02 09:46:01 +10:30
Christian Decker ef145c7900 msggen: Add RoutehintList as a primitive 2022-04-02 09:46:01 +10:30
Christian Decker ec5cd92580 msggen: Add model-side overrides
Sometimes we just want to paper over the schema directly. Mostly
useful to sidestep the `oneof` things that are required for
expressiveness.
2022-04-02 09:46:01 +10:30
Christian Decker 1f40db3594 msggen: Add parser for "oneOf" 2022-04-02 09:46:01 +10:30
Christian Decker bba68e2136 cln-grpc: Map AmountOrAll and AmountOrAny 2022-04-02 09:46:01 +10:30
Christian Decker 04e7e285d7 cln-rpc: Map PSBT and TX methods 2022-04-02 09:46:01 +10:30
Christian Decker 0354a7fdb1 cln-rpc: Add `OutputDesc` for `{addr: amt}` style arguments
This is likely inherited from bitcoind, and a bit awkward for us, so
we parse it into a classic struct, but serialize it back into the
bitcoind format when talking to the RPC.
2022-04-02 09:46:01 +10:30
Christian Decker eb2aa8c51c cln-rpc: Map feerates and backfill methods using it 2022-04-02 09:46:01 +10:30
Christian Decker 20704746bc grpc: Map `sendonion` and `createonion` 2022-04-02 09:46:01 +10:30
Christian Decker 565518246a msggen: Add custom mappings for JSON -> grpc conversions 2022-04-02 09:46:01 +10:30
Christian Decker a7f3c54600 cln-rpc: Add invoice RPC methods 2022-04-02 09:46:01 +10:30
Christian Decker d90aafeed6 cln-grpc: Add the `connect` method 2022-04-02 09:46:01 +10:30
Christian Decker 646901f588 msggen: Wrap field numberings in the message type
Otherwise we get less nicely consecutive numbers because request and
response share one domain. This separates them again.
2022-04-02 09:46:01 +10:30
Christian Decker 6098386182 cln-grpc: Add the `connect` method 2022-03-30 12:15:55 +10:30
Christian Decker a8aa9bd5ae cln-rpc: Fixed two minor compiler warnings 2022-03-16 11:11:58 +10:30
Christian Decker 487b5e6169 msggen: Add meta file to keep the field numbers stable
We are inferring the field numbers on the fly, which isn't really
compatible with the way GRPC field numbers work, i.e., they must be
stable while the IDL file evolves. So far when a field was added in
the middle of a struct or removed all subsequent fields would get
renumbered, essentially breaking any client that was using the old
scheme.

We now add a meta file `.msggen.json` that keeps track of the numbers
assigned so far, so they can be reused, and new ones can be generated
not to conflict with existing ones. This file is intentionally kept
generic, so other generators can add more information that has to be
managed across runs.

Changelog-None
2022-03-16 11:11:58 +10:30
Christian Decker 75f0b8e916 cln-grpc: Add `listpeers` RPC method
This is pretty much the hardest to map, but we map it correctly, with
the exception of the state_changes[] array we truncated out in the
last commit.
2022-02-28 23:25:37 +00:00
Christian Decker b73405a4c2 cln-grpc: Allow fields to be mapped to None to ignore them
`listpeers` is a rather deeply nested structure which has a couple of
caveats, namely that we use the same enum multiple times, which causes
naming clashes. So we truncate the state_changes[]. We can later map
them if needed, but it'll get much easier once we have an abstract
model description that isn't JSON schema, which unrolls all types,
causing us to generate those enums multiple times.
2022-02-28 23:25:37 +00:00
Christian Decker bb4946a6e6 msggen: Support renaming methods in GRPC
There is at least one clash with a built-in for the grpc server trait,
namely `connect` so we add support for renaming a method when
generating the scaffolding
2022-02-28 23:25:37 +00:00
Christian Decker efed7d8617 msggen: Support enums in requests too
They are sent as i32 over protobuf, so we need to convert them into
their enum representation.
2022-02-28 23:25:37 +00:00
Christian Decker 494243d41c msggen: Handle some more types in request conversions 2022-02-28 23:25:37 +00:00
Christian Decker 62dc078271 cln-grpc: Generate server dispatcher
The server doesn't do much more than unwrapping the request from its
grpc envelope, convert it into the matching JSON-RPC binding struct,
initiate the RPC connection (until we have connection pooling), and
then forwards the converted request. The inverse then happens for the
result.
2022-02-28 23:25:37 +00:00
Christian Decker 5d6e9d6dae cln-grpc: Add generation of request conversion
This is taking protobuf requests on one side and converting them into
the JSON-RPC requests.
2022-02-28 23:25:37 +00:00
Christian Decker 8d3871d791 cln-grpc: Add result conversion generator to `msggen`
This takes the Rust bindings and converts them into the generated
protobuf bindings:

> JSON-RPC -> Rust bindings -> grpc bindings -> protobuf
2022-02-28 23:25:37 +00:00
Christian Decker d01b2c21a7 cln-grpc: Add generation of grpc protobuf file from schema 2022-02-28 23:25:37 +00:00
Christian Decker b0053e2ca2 msggen: Generate the cln-rpc Rust structs
We're generating these structs so we can parse them directly into
native objects.
2022-02-11 16:02:25 +01:00
Christian Decker 0fc0ffc961 msggen: Parse JSON-RPC schemas and build the in-memory model
We build an in-memory model of what the API should look like, which
will later be used to generate a variety of bindings. In this PR we
will use the model to build structs corresponding to the requests and
responses for the various methods.

The JSON-RPC schemas serve as ground-truth, however they are missing a
bit of context: methods, and the request-response matching (as well as
a higher level grouping we'll call a Service). I'm tempted to create a
new document that describes this behavior and we could even generate
the rather repetitive JSON schemas from that document. Furthermore
it'd allow us to add some required metadata such as grpc field
numbering once we generate those bindings.

Changelog-Added: JSON-RPC: A new `msggen` library allows easy generation of language bindings for the JSON-RPC from the JSON schemas
2022-02-11 16:02:25 +01:00