Commit Graph

12142 Commits

Author SHA1 Message Date
Rusty Russell b6134303d4 pyln: add context manager to simpify filter use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: LightningRpc has new `reply_filter` context manager for reducing output of RPC commands.
2022-11-09 20:25:58 +10:30
Rusty Russell 1436ad334d pytest: add filter tests.
We suppress schema reply checking when filter is set: we could just
remove all the `required` fields in the JSON schema.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell 2a14afbf21 lightningd: set filter when we see 'filter' object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `filter` object allows reduction of JSON response to (most) commands.
2022-11-09 20:25:58 +10:30
Rusty Russell 3b4c1968a3 common/test: add unit tests for JSON filtering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell f0731d2ca1 common/json_stream: support filtering don't print fields not allowed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell 22c42de6f1 tests/fuzz: don't pull in JSON common at all.
No tests currently use it, and if they do we'll want to do some
per-test objects.  Otherwise, we are about it introduce a dependency
on common/json_filter.o, which is a can of worms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell 508a170598 common/json_filter: routine to turn "filter" JSON into a filter.
Since the "struct command" is different from plugins and lightningd, we
need an accessor for this to work (the plugin one is a dummy for now!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell 3c75770586 common/json_filter: routines for json filtering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Christian Decker 15112ae87b gci: Force MacOS CI Job to use python 3.10
The runner version was recently bumped causing our tests to fail
because it couldn't find the `poetry` version it just installed. We
instead install python3.10, and force its use, since otherwise we end
up compiling grpcio and fail to do so.

Changelog-None
2022-11-08 17:41:21 +01:00
Rusty Russell a4c482dc07 common/sphinx: don't use fixed lengths anywhere.
1. Remove the very concept of ONION_REPLY_SIZE, instead make it a
   local variable in create_onionreply().

2. Use the proper fromwire_ primitives in unwrap_onionreply() so we
   don't have to do explicit length checks.

3. Make fromwire_tal_arrn() return NULL if it fails to pull, instead of
   a zero-length allocation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we now correctly decrypt non-256-length onion errors (we always forwarded them fine, now we actually can parse them).
2022-11-08 17:40:57 +01:00
Rusty Russell fe1b285bba pytest: add test for generating non-standard length onion errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-08 17:40:57 +01:00
Rusty Russell 75c382fe16 lightningd: --dev-onion-reply-length option.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-08 17:40:57 +01:00
Alex Myers 341d73fdc2 reckless: fix git clone issue with removed dir
Reckless was failing to install multiple plugins due to git not
appreciating the cwd being a now removed dir after the first plugin
tmp files were cleaned up.
2022-11-08 13:19:36 +01:00
Alex Myers a728b04243 Reckless: add man page 2022-11-08 13:19:36 +01:00
Alex Myers e48fda1ba0 reckless: analyze repositories with urlparse 2022-11-08 13:19:36 +01:00
Alex Myers 2f4e862863 reckless: improve config file handling
While loading the appropriate lightningconfig file, it is now checked
against the active config file in lightningd. Because a deviation from the
default file structure would not be possible, a -conf option is also added
to explicitly pass the lightningd config file into reckless.
2022-11-08 13:19:36 +01:00
Alex Myers 53ad1ee576 reckless: add function for lightning-cli calls
This also simplifies dynamic enable/disable by catching the exception
raised when the cli is unable to connect to RPC (lightningd offline or
misconfigured relative to reckless).
2022-11-08 13:19:36 +01:00
Alex Myers 4a95a4c7da reckless: multiline string style cleanup 2022-11-08 13:19:36 +01:00
Alex Myers 71351ceacf reckless: replace os.path with pathlib operations
This change makes it easier to follow retrieval of parent directories.
Additional os.path operations replaced with their pathlib.Path equivalents
to keep module usage consistent.
2022-11-08 13:19:36 +01:00
Alex Myers df98c8b927 reckless: refactor argument list handling.
The goal was to support passing a list to install, enable, etc. in order
to improve performance. Passing lists to most of the functions was less
practical than iterating through the items from the top level.
2022-11-08 13:19:36 +01:00
Alex Myers 24422e9f7c reckless: add type hints 2022-11-08 13:19:36 +01:00
Alex Myers 5d23c7ab0b reckless: raise exception or early termination instead of returning None
More pythonic than returning mixed types.
2022-11-08 13:19:36 +01:00
Alex Myers 791e521179 reckless: update help alias
`reckless help <cmd>` previously called the function docstring. This could
be updated to use the subparser help, but would require a strict naming
convention or a dictionary. Providing a hint to use the built-in
contextual help via the option flag is hopefully sufficient.
2022-11-08 13:19:36 +01:00
Alex Myers f3934cda50 reckless: use argparse subparsers
A more pythonic approach which should also enable additional help context
for subcommands.
2022-11-08 13:19:36 +01:00
Alex Myers 651c5b6de0 reckless: use config that was explicitly passed to lightningd
Regtest environments commonly use explicit definition of the config
file for lightningd.  This can be queried and utilized by default,
saving redundant definitions between lightning and reckless.
2022-11-08 13:19:36 +01:00
Alex Myers f18c5e320d reckless: detect pip3 or pip 2022-11-08 13:19:36 +01:00
Alex Myers 83dd431cdc reckless: use the lightning path when invoking lightning-cli
This enables compatibility with startup_regtest.sh among other uses.
The lightning-cli network flag is also set in case there is no config file.
2022-11-08 13:19:36 +01:00
Alex Myers 7e8a889d89 reckless: it turns out the warning is a bit much.
The user should be informed that their config now has a new source, but
but any config files created downstream should be automatically populated.
2022-11-08 13:19:36 +01:00
Alex Myers b1b280d10b reckless: new tool to manage lightningd plugins
A simple standalone python executable to track plugin repositories, clone
to /tmp, install requirements, test plugin runs, then install and enable in
lightningd and in the config.
Changelog-Added: Reckless - a Core Lightning plugin manager
2022-11-08 13:19:36 +01:00
Alex Myers 0d756ff017 gossipd: Cleanup channel update replacement logic
Private channel updates can no longer be flagged as spam during handling of
a new channel update (this was a bug.) Also slightly reworked previous
channel_update deletion for clarity.
2022-11-03 11:29:11 +01:00
Alex Myers 8f48406985 pytest: test for gossip store corruption by private channel updates
Adds a simple test covering the gossip_store corruption issue #5656
stemming from failure to delete prior private channel updates in all
cases.
2022-11-03 11:29:11 +01:00
Alex Myers 5bd6c715e5 gossipd: ensure old private channel updates are properly deleted
When private channel updates exceed the gossip ratelimit, the previous
gossip store entry was not deleted even though all private channel updates
are stored. This caused gossip store corruption due to duplicate entries
for the same channel.
Fixes: #5656

Changelog-Fixed: Fixed gossip_store corruption from duplicate private channel updates
2022-11-03 11:29:11 +01:00
Christian Decker 2605e117c9 pytest: Add test for optional options in cln-plugin
Changelog-Added: cln-plugin: Options are no longer required to have a default value
2022-11-03 11:28:22 +01:00
Justin Moon f111d6772d Plugin config options with no defaults 2022-11-03 11:28:22 +01:00
Dustin Dettmer 76623b2ef2 Collaborative transaction building
Takes the dualopen collaborative transaction building and makes it available for other daemons to use

Changelog-Added: Added interactive transaction building routine
2022-11-02 12:00:11 +01:00
Dustin Dettmer 62e44e6bc8 doc: Move M1 instructions install to its own section
Changelog-None
2022-11-01 18:25:21 +01:00
Dread 8f549bfcca doc: add `make install` line for macOS instructions 2022-11-01 18:25:21 +01:00
niftynei 7bd0d7641c mkfunding: no scriptPubKey on utxo causing crash, so we add one
Changelog-Fixed: devtools: `mkfunding` command no longer crashes (abort)

Fixes #5363

Assisted-By: @TKChattoraj
2022-11-01 18:23:00 +01:00
niftynei 87616b7ffc mkfunding: add missing common_setup
Was crashing on allocating wally memory, b/c was missing the wally_init
that's wrapped up in common_setup

Assisted-By: @TKChattoraj
2022-11-01 18:23:00 +01:00
Christian Decker 1a1c0a38fc msggen: Map the `extratlvs` field of `keysend`
Changelog-Added: cln-rpc: `keysend` now exposes the `extratlvs` field
2022-11-01 17:05:30 +01:00
Christian Decker 93c95056a3 rs: Fix two small regressions
These were only in the test targets, but still.
2022-11-01 17:05:30 +01:00
Christian Decker 8a4f44a58d keysend: Allow quoted numbers in `extratlvs`
This is because JSON technically does not allow numeric keys in maps.

Changelog-Added: JSON-RPC: The `extratlvs` argument for `keysend` now allows quoting the type numbers in string
2022-11-01 17:05:30 +01:00
Christian Decker 83beaa5396 json: Add helper for quoted numbers
The JSON specification technically disallows maps with numeric keys,
so we'll want to slowly migrate away from using them. This helper
extracts the numeric value from a quoted number, which is a legal
representation of the same in JSON.
2022-11-01 17:05:30 +01:00
Rusty Russell f158b529d3 wire/Makefile: fix missing wire/bolt12_exp_wiregen.c in ALL_C_SOURCES.
And add it as a requirement to the test programs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 422e68a4d6 common/blindedpath: create onion mesage test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 159fc7d1a2 common/onion_message_parse: generic routine for parsing onion messages.
Instead of open coding in connectd/onion_message, we move it to common
with a nice API.

This lets us process the BOLT test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 5cf86a1a2e common: update to latest onion message spec.
Mainly, field name changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Protocol: Support for forwarding blinded payments (as per latest draft)
2022-10-26 11:29:06 +10:30
Rusty Russell 67b8fadf02 common/test: check we meet bolt04/onion-route-blinding-test.json
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 8eee5dd7fd channeld, lightningd: allow blinded payments with !EXPERIMENTAL_FEATURES.
Gate it (where necessary) by the route-blinding feature bit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell 426886ff9b lightningd: return invalid_onon_blinding for any blinded payment error.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30