Commit Graph

12985 Commits

Author SHA1 Message Date
Christian Decker 178e0b6463 docs: Structure the files in mkdocs
We have 3 personas:

 - Users
 - Developers
 - Maintainers

The first one basically cover the installation documentation. The
latter two are sorted into the "Developer" category, and the reference
category serves as a quick lookup for facts on anything CLN related.
2023-03-25 15:40:35 +10:30
Christian Decker a80c1ae40c docs: Switch to mkdocs for documentation 2023-03-25 15:40:35 +10:30
Rusty Russell b8519a6a1a plugins/sql: add listclosedchannels
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sql` now includes `listclosedchannels`.
2023-03-25 15:28:02 +10:30
Rusty Russell 89f91b9bb4 lightningd: add listclosedchannels command.
Changelog-Added: JSON-RPC: `listclosedchannels` to show old, dead channels we previously had with peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell e75cf2e7fb listpeerchannels: add channel_type, both in hex and as array of names.
Changelog-Added: JSON-RPC: `listpeerchannels` now has `channel_type` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell d818614aa9 plugins/sql: recurse correctly into complex objects during processing.
We didn't handle the case of an array inside a subobject.  But that
happens when we add the next commit!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell c9ddf9d1c3 plugins/sql: handle case of subobject with sub-arrays.
i.e. recurse properly in SQL generation.  This is about to happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell 4549002105 common: expose routine to map channel_type to feature names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell 4b6e9649eb wallet: add accessor for closed channels.
This doesn't restore every bit of information we have, but it does
contain the important ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell 6e1eafbb0b wallet: make it clear that `enum state_change` is in db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell f720e0ff0b wallet: use db_col_optional.
We don't cover three common patterns:
1. Optional integers (db_col_u64 has different form from structs)
2. Optional strings.
3. Optional array fields.

But it does neaten and reduce the scope for cut&paste errors in the
common "if not-NULL, tal and assign".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell aae77802ef db_col_optional: wrapper for case where a field is allowed to be NULL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell d9e274cee2 db_bind_scid: rename to db_bind_short_channel_id
We used to have a text version, so this was named 'scid'.  Fix it now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell ae861d1793 wallet: don't clear reference from channel to peers table when we close channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell 09011177a8 wallet: only delete peer from db if it's unused.
This relaxes the assertion that it won't be used, and renames the
function to be clear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell 98d425f1f4 wallet: add comment on db noting that `ON DELETE CASCADE` is never used.
We actually have an assertion that there are no channels remaining when
we delete peers, so this is confusing!

Actually removing the constraint is db-specific and deeply non-trivial.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Christian Decker 7d7b2abd02 msggen: Allow using deprecated fields in the rpc -> grpc conversion
We should rather hand the annotation through to the user code, and
warn there.
2023-03-24 00:23:43 +00:00
Christian Decker 97de4f8e0f grpc: make the mTLS private keys user-readable only
Fixes #6064
Reported-by: denis2342 <@denis2342>

Changelog-Changed: grpc: The mTLS private keys are no longer group-readable
2023-03-24 00:23:43 +00:00
Vincenzo Palazzo 3424f70585 plugin: autoclean: cleanup the forwards with localfailed
While we are cleaning up the list forwards with the autoclean plugin we are
not taking into count the forward's payments with the status set to
`local_failed`. In this case, the forwards have no resolved
time because it was not resolved by us due to some local error.

So, this commit is fixing the auto clean plugin by allowing to delete
of the forwards with status set to local_failed by taking into count
the received_time, with the assumption that the received_time, in this case,
is equal to the resolved time (?)

Reported-by: @denis2342
Link: https://github.com/ElementsProject/lightning/issues/6058
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Fixed: plugin: autoclean: considerer the forwards with status
set to `local_failed`.
2023-03-23 08:18:27 +00:00
Rusty Russell aa1a0e31fd Docker: run directory for post-start if present.
Also, fix the case where we didn't use --network with EXPOSE_TCP,
as reported by @theborakompanioni:

```
I get Wrong network! Our Bitcoin backend is running on 'regtest', but we expect 'main'. with LIGHTNINGD_NETWORK := regtest when param --network is not provided.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-23 16:43:07 +10:30
Greg Sanders c85bce94be Report failure to sign psbt inputs by hsmd 2023-03-23 16:11:52 +10:30
Greg Sanders e7bf52980b test_closing_different_fees: b vs balance in loop 2023-03-23 16:10:55 +10:30
Greg Sanders cf662e55a7 Make startup_regtest.sh more robust to bitcoind wallet state 2023-03-23 16:10:55 +10:30
Greg Sanders 887c6f71cf Add PSBT version setting RPC to aid with debugging and compatibility
PSBTv2 support is quite low in the ecosystem, so having a call to convert
log messages and the like should be useful since they'll often be in v2.

Changelog-Added: Added setpsbtversion RPC to aid debugging and compatibility
2023-03-23 16:10:55 +10:30
Greg Sanders cb7caa3139 Re-enable PSBT tests for Liquid except test_sign_and_send_psbt 2023-03-23 16:10:55 +10:30
Greg Sanders 908f834d66 Update libwally to 0.8.8, support PSBTv2
Libwally update breaks compatibility, so
we do this in one large step.

Changelog-Changed: JSON-RPC: elements network PSET now only supports PSETv2.
Changelog-Added: JSON-RPC: PSBTv2 supported for fundchannel_complete, openchannel_update, reserveinputs, sendpsbt, signpsbt, withdraw and unreserveinputs parameter psbt, openchannel_init and openchannel_bump parameter initialpsbt, openchannel_signed parameter signed_psbt and utxopsbt parameter utxopsbt
2023-03-23 16:10:55 +10:30
Greg Sanders 5eddf3cd73 test: add PSBT field that doesn't collide with PSBTv2 fields
Which gets libwally upset post-update to 0.8.8
2023-03-23 16:10:55 +10:30
Matt Morehouse 3192be5c23 fuzz: fix UBSan nullability error
The issue is that common_setup() wasn't called by the fuzz target,
leaving secp256k1_ctx as NULL.

UBSan error:

$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \
    ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1153355603
INFO: Loaded 1 modules   (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3),
INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98),
./fuzz-channel_id: Running 1 inputs 1 time(s) each.
Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2
bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here
    #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7
    #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7
    #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3
    #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2
    #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long)
    #5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long)
    #6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))
    #7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25)
    #8 0x7fa7fa113082 in __libc_start_main
    #9 0x563bae1efbdd in _start

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in
2023-03-23 13:34:24 +10:30
Matt Morehouse 07527d9fbb fuzz: avoid buffer overflow in bech32 target
If the fuzzer passes an empty data buffer, the fuzz target currently
attempts to read from it. We should short-circuit instead.
2023-03-23 13:28:28 +10:30
Rusty Russell df085a8a87 wallet/db: don't use migration_context.
`struct lightningd` is not completely initialized, so we added a
"migration_context" which only had some of the fields.  But we ended
up handing in `struct lightningd` anyway, so I don't think this
complexity is worthwhile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
Rusty Russell 3db3dc946f lightningd: move bip32_pubkey here from common/, add hsm check.
At the moment only lightingd needs it, and this avoids missing any
places where we do bip32 derivation.

This uses a hsm capability to mean we're backwards compatible with older
hsmds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now always double-check bitcoin addresses are correct (no memory errors!) before issuing them.
2023-03-22 13:50:32 +10:30
Rusty Russell 3f02797e88 lightningd: move bip32_base pointer into struct lightningd.
It's needed as the db and wallet is being set up (db migrations), so
it's simpler this way to always use ld->bip32_base for the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
Rusty Russell 91a9cf3512 hsmd: capability addition: ability to check pubkeys.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
Rusty Russell e02f5f5bb8 hsmd: new version, which tells us the HSM version, and capabilities.
Importantly, adds the version number at the *front* to help future
parsing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


Header from folded patch 'fix-hsm-check-pubkey.patch':

fixup! hsmd: capability addition: ability to check pubkeys.
2023-03-22 13:50:32 +10:30
Rusty Russell 21a1b4e6aa common: update HSM_MIN_VERSION to reflect reality.
We were handing 3 to hsmd (and Ken added that in 7b2c5617c1,
so I guess he's OK with that being the minimum supported version!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
dependabot[bot] 48c334dc81 build(deps): bump werkzeug from 2.2.2 to 2.2.3
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3.
- [Release notes](https://github.com/pallets/werkzeug/releases)
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/werkzeug/compare/2.2.2...2.2.3)

---
updated-dependencies:
- dependency-name: werkzeug
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-22 10:31:21 +10:30
Rusty Russell 03c153ac0b channeld: don't spin trying to send commitment while waiting.
We would sleep for 10msec (default) and try again, spamming the logs
every second.  But we're waiting for revoke_and_ack, and that handler
already sets off the timer, so there's no need to spin at all!

Fixes: #6077
Changelog-Fixed: `channeld`: no longer spin and spam logs when waiting for revoke_and_ack.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 10:30:48 +10:30
Rusty Russell 2f188622b7 pytest: add timeout to test_feerate_stress.
This seems to be getting stuck in CI, so make sure we time out if it happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 10:30:23 +10:30
Michael Schmoock 9a3f69aecf connectd: log status_failed on TOR problems
This changes connectd to use `status_fail()` on TOR problems during statup
instead of `err()`. Using `err()` did not write to the logfile.

To find out TOR problems during startup, the user needed to stop the system
daemon and call `lightningd` manually in console to see the error.

`status_fail()` logs and exits, but also prints a whole stacktrace,
which is a bit too much imho on config errors. But currently there is
no `status_SOMETHING` method that logs, prints and exists on an error
without stacktrace.

Changelog-None
2023-03-22 09:42:13 +10:30
Kristaps Kaupe 906279a46e Output channel_id in listfunds
Changelog-Added: JSON-RPC: `listfunds` now has a `channel_id` field.
2023-03-21 10:29:33 +00:00
Vincenzo Palazzo fca62113f5 plugin: fetchinvoice: set the quantity in invreq
While the user trying to fetch an invoice by specifing the quantity we do
not work as expected.

Running the command

```
lightning-cli fetchinvoice -k offer='lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqffqszsk2p6hycmgv9ek2grpyphxjcm9ypmkjer8v46pyzmhd9jxwet5wvhxxmmdzsqs593pq0ylsvakdua5h976f4g3eautgjt3udvtyga47eaw7339sjrhpwpwz' quantity=2
```

and we answer back with

```json
{
   "code": -32602,
   "message": "quantity parameter required"
}
```

This is caused because we forget to bind the `quanity` field from the
RPC into the `invrequest`.

Reported-by: @aaronbarnardsound
Link: https://github.com/ElementsProject/lightning/issues/6089
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: fetchinvoice: fix: do not ignore the `quantity` field
into the invreq field.
2023-03-20 20:25:02 +00:00
Greg Sanders 6c641bdbbb test_backfill_scriptpubkeys: stop first cln node before second sub-test 2023-03-20 10:47:29 +10:30
Vincenzo Palazzo ba4f0c8dab ci: add timeout field to 2h for each task
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-20 10:43:35 +10:30
Justin Moon 57d21206db cln_plugin: add `shutdown()` method to `Plugin`
When plugins receive a "shutdown" notification, then can call this
method which will shutdown `cln_plugin`.

Then they can await `plugin.join()` and do any remaining cleanup
there.

This helps avoid a pain-point where plugin authors need to handle
2 separate plugin shutdown mechanisms https://github.com/ElementsProject/lightning/issues/6040
2023-03-18 11:52:29 +00:00
Rusty Russell acf01f4c09 pytest: don't run test_backfill_scriptpubkeys under valgrind in CI.
It seems that bitcoind frequently dies on this test.  I assume running
the multiple nodes under valgrind with the extra 214 blocks is too
memory-hungry?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 658bae30d5 lightningd: require "jsonrpc": "2.0" as per JSONRPC spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: require the `"jsonrpc": "2.0"` property (requests without this deprecated in v0.10.2).
2023-03-18 15:55:49 +10:30
Rusty Russell 9366e6b39f cleanup: rename json_add_amount_msat_only to json_add_amount_msat
Now there's no compat variant, we can rename this function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 983542f2a7 global: remove deprecated "msat" suffix on msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: the "msat" suffix on millisatoshi fields, as deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 780f32dfc6 global: remove deprecated non-msat-named msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: all the non-msat-named millisatoshi fields deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 67f23c19f7 lightningd: remove deprecated local_msat, remote_msat from listpeers.
Changelog-Removed: JSON-RPC: `listpeers`.`local_msat` and `listpeers`.`remote_msat` (deprecated v0.12.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30