vincenzopalazzo
/
rgb-cln
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity
rgb-cln/contrib/msggen/README.md

43 lines
1.6 KiB
Markdown
Raw Normal View History

msggen: Parse JSON-RPC schemas and build the in-memory model We build an in-memory model of what the API should look like, which will later be used to generate a variety of bindings. In this PR we will use the model to build structs corresponding to the requests and responses for the various methods. The JSON-RPC schemas serve as ground-truth, however they are missing a bit of context: methods, and the request-response matching (as well as a higher level grouping we'll call a Service). I'm tempted to create a new document that describes this behavior and we could even generate the rather repetitive JSON schemas from that document. Furthermore it'd allow us to add some required metadata such as grpc field numbering once we generate those bindings. Changelog-Added: JSON-RPC: A new `msggen` library allows easy generation of language bindings for the JSON-RPC from the JSON schemas
2022-01-14 12:49:56 +00:00
# MsgGen - Generating language bindings and docs from schemas and wire descriptions
MsgGen is a collection of tools that are used to parse schemas and
(eventually) protocol wire CSVs into an intermediate representation in
memory, and then generate language specific bindings and
documentation from it.
The dependency graph looks like this:
```dot
digraph {
"JSON-RPC Schemas" -> "msggen model";
"msggen model" -> "grpc proto file";
"msggen model" -> "Rust From<JsonRpc> Converters";
"grpc proto file" -> "Rust grpc bindings"
"Rust grpc bindings" -> "cln-grpc";
"Rust From<JsonRpc> Converters" -> "cln-grpc";
"msggen model" -> "Rust JSON-RPC structs";
"Rust JSON-RPC structs" -> "cln-rpc";
}
```
msggen: Move overrides into the model itself We were using per-type overrides which caused some asymmetries, where conversions could end up dropping fields as we went along. Essentially each conversion would need to override a superset of the previous one, which then caused issues when attempting to close the loop. By overriding on the model level we ensure that all representations are equivalent and convertible into one another, at the expense of overriding a bit more aggressively, which should be fine anyway.
2023-05-03 12:41:18 +01:00
`msggen` will load the schemas in `doc/schemas` into memory, and then
use `Patches` to enrich the model before using it to generate the
bindings for the various languages as well as the converters from one
format to another. These patches can be found in `msggen/patch.py` and
perform a variety of operations:
- Annotate the model with additional data from external sources, such
as the `.msggen.json` file at the repository root to track details
that can be derived but should remain constant (grpc field
numbering and versioning information)
- Aggregate common types with type overrides and omit fields that we
can't map currently.
- Infer optionality based on the versions a field was added or
deprecated, and the currently supported range of versions.
If there is a field that is currently missing in the model, that is in
the schemas it is most likely because it has been marked as omitted in
the patch.