Commit Graph

12671 Commits

Author SHA1 Message Date
Rusty Russell 09011177a8 wallet: only delete peer from db if it's unused.
This relaxes the assertion that it won't be used, and renames the
function to be clear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell 98d425f1f4 wallet: add comment on db noting that `ON DELETE CASCADE` is never used.
We actually have an assertion that there are no channels remaining when
we delete peers, so this is confusing!

Actually removing the constraint is db-specific and deeply non-trivial.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Christian Decker 7d7b2abd02 msggen: Allow using deprecated fields in the rpc -> grpc conversion
We should rather hand the annotation through to the user code, and
warn there.
2023-03-24 00:23:43 +00:00
Christian Decker 97de4f8e0f grpc: make the mTLS private keys user-readable only
Fixes #6064
Reported-by: denis2342 <@denis2342>

Changelog-Changed: grpc: The mTLS private keys are no longer group-readable
2023-03-24 00:23:43 +00:00
Vincenzo Palazzo 3424f70585 plugin: autoclean: cleanup the forwards with localfailed
While we are cleaning up the list forwards with the autoclean plugin we are
not taking into count the forward's payments with the status set to
`local_failed`. In this case, the forwards have no resolved
time because it was not resolved by us due to some local error.

So, this commit is fixing the auto clean plugin by allowing to delete
of the forwards with status set to local_failed by taking into count
the received_time, with the assumption that the received_time, in this case,
is equal to the resolved time (?)

Reported-by: @denis2342
Link: https://github.com/ElementsProject/lightning/issues/6058
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Fixed: plugin: autoclean: considerer the forwards with status
set to `local_failed`.
2023-03-23 08:18:27 +00:00
Rusty Russell aa1a0e31fd Docker: run directory for post-start if present.
Also, fix the case where we didn't use --network with EXPOSE_TCP,
as reported by @theborakompanioni:

```
I get Wrong network! Our Bitcoin backend is running on 'regtest', but we expect 'main'. with LIGHTNINGD_NETWORK := regtest when param --network is not provided.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-23 16:43:07 +10:30
Greg Sanders c85bce94be Report failure to sign psbt inputs by hsmd 2023-03-23 16:11:52 +10:30
Greg Sanders e7bf52980b test_closing_different_fees: b vs balance in loop 2023-03-23 16:10:55 +10:30
Greg Sanders cf662e55a7 Make startup_regtest.sh more robust to bitcoind wallet state 2023-03-23 16:10:55 +10:30
Greg Sanders 887c6f71cf Add PSBT version setting RPC to aid with debugging and compatibility
PSBTv2 support is quite low in the ecosystem, so having a call to convert
log messages and the like should be useful since they'll often be in v2.

Changelog-Added: Added setpsbtversion RPC to aid debugging and compatibility
2023-03-23 16:10:55 +10:30
Greg Sanders cb7caa3139 Re-enable PSBT tests for Liquid except test_sign_and_send_psbt 2023-03-23 16:10:55 +10:30
Greg Sanders 908f834d66 Update libwally to 0.8.8, support PSBTv2
Libwally update breaks compatibility, so
we do this in one large step.

Changelog-Changed: JSON-RPC: elements network PSET now only supports PSETv2.
Changelog-Added: JSON-RPC: PSBTv2 supported for fundchannel_complete, openchannel_update, reserveinputs, sendpsbt, signpsbt, withdraw and unreserveinputs parameter psbt, openchannel_init and openchannel_bump parameter initialpsbt, openchannel_signed parameter signed_psbt and utxopsbt parameter utxopsbt
2023-03-23 16:10:55 +10:30
Greg Sanders 5eddf3cd73 test: add PSBT field that doesn't collide with PSBTv2 fields
Which gets libwally upset post-update to 0.8.8
2023-03-23 16:10:55 +10:30
Matt Morehouse 3192be5c23 fuzz: fix UBSan nullability error
The issue is that common_setup() wasn't called by the fuzz target,
leaving secp256k1_ctx as NULL.

UBSan error:

$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \
    ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1153355603
INFO: Loaded 1 modules   (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3),
INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98),
./fuzz-channel_id: Running 1 inputs 1 time(s) each.
Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2
bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here
    #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7
    #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7
    #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3
    #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2
    #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long)
    #5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long)
    #6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))
    #7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25)
    #8 0x7fa7fa113082 in __libc_start_main
    #9 0x563bae1efbdd in _start

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in
2023-03-23 13:34:24 +10:30
Matt Morehouse 07527d9fbb fuzz: avoid buffer overflow in bech32 target
If the fuzzer passes an empty data buffer, the fuzz target currently
attempts to read from it. We should short-circuit instead.
2023-03-23 13:28:28 +10:30
Rusty Russell df085a8a87 wallet/db: don't use migration_context.
`struct lightningd` is not completely initialized, so we added a
"migration_context" which only had some of the fields.  But we ended
up handing in `struct lightningd` anyway, so I don't think this
complexity is worthwhile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
Rusty Russell 3db3dc946f lightningd: move bip32_pubkey here from common/, add hsm check.
At the moment only lightingd needs it, and this avoids missing any
places where we do bip32 derivation.

This uses a hsm capability to mean we're backwards compatible with older
hsmds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now always double-check bitcoin addresses are correct (no memory errors!) before issuing them.
2023-03-22 13:50:32 +10:30
Rusty Russell 3f02797e88 lightningd: move bip32_base pointer into struct lightningd.
It's needed as the db and wallet is being set up (db migrations), so
it's simpler this way to always use ld->bip32_base for the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
Rusty Russell 91a9cf3512 hsmd: capability addition: ability to check pubkeys.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
Rusty Russell e02f5f5bb8 hsmd: new version, which tells us the HSM version, and capabilities.
Importantly, adds the version number at the *front* to help future
parsing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


Header from folded patch 'fix-hsm-check-pubkey.patch':

fixup! hsmd: capability addition: ability to check pubkeys.
2023-03-22 13:50:32 +10:30
Rusty Russell 21a1b4e6aa common: update HSM_MIN_VERSION to reflect reality.
We were handing 3 to hsmd (and Ken added that in 7b2c5617c1,
so I guess he's OK with that being the minimum supported version!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 13:50:32 +10:30
dependabot[bot] 48c334dc81 build(deps): bump werkzeug from 2.2.2 to 2.2.3
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3.
- [Release notes](https://github.com/pallets/werkzeug/releases)
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/werkzeug/compare/2.2.2...2.2.3)

---
updated-dependencies:
- dependency-name: werkzeug
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-22 10:31:21 +10:30
Rusty Russell 03c153ac0b channeld: don't spin trying to send commitment while waiting.
We would sleep for 10msec (default) and try again, spamming the logs
every second.  But we're waiting for revoke_and_ack, and that handler
already sets off the timer, so there's no need to spin at all!

Fixes: #6077
Changelog-Fixed: `channeld`: no longer spin and spam logs when waiting for revoke_and_ack.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 10:30:48 +10:30
Rusty Russell 2f188622b7 pytest: add timeout to test_feerate_stress.
This seems to be getting stuck in CI, so make sure we time out if it happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-22 10:30:23 +10:30
Michael Schmoock 9a3f69aecf connectd: log status_failed on TOR problems
This changes connectd to use `status_fail()` on TOR problems during statup
instead of `err()`. Using `err()` did not write to the logfile.

To find out TOR problems during startup, the user needed to stop the system
daemon and call `lightningd` manually in console to see the error.

`status_fail()` logs and exits, but also prints a whole stacktrace,
which is a bit too much imho on config errors. But currently there is
no `status_SOMETHING` method that logs, prints and exists on an error
without stacktrace.

Changelog-None
2023-03-22 09:42:13 +10:30
Kristaps Kaupe 906279a46e Output channel_id in listfunds
Changelog-Added: JSON-RPC: `listfunds` now has a `channel_id` field.
2023-03-21 10:29:33 +00:00
Vincenzo Palazzo fca62113f5 plugin: fetchinvoice: set the quantity in invreq
While the user trying to fetch an invoice by specifing the quantity we do
not work as expected.

Running the command

```
lightning-cli fetchinvoice -k offer='lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqffqszsk2p6hycmgv9ek2grpyphxjcm9ypmkjer8v46pyzmhd9jxwet5wvhxxmmdzsqs593pq0ylsvakdua5h976f4g3eautgjt3udvtyga47eaw7339sjrhpwpwz' quantity=2
```

and we answer back with

```json
{
   "code": -32602,
   "message": "quantity parameter required"
}
```

This is caused because we forget to bind the `quanity` field from the
RPC into the `invrequest`.

Reported-by: @aaronbarnardsound
Link: https://github.com/ElementsProject/lightning/issues/6089
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: fetchinvoice: fix: do not ignore the `quantity` field
into the invreq field.
2023-03-20 20:25:02 +00:00
Greg Sanders 6c641bdbbb test_backfill_scriptpubkeys: stop first cln node before second sub-test 2023-03-20 10:47:29 +10:30
Vincenzo Palazzo ba4f0c8dab ci: add timeout field to 2h for each task
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-20 10:43:35 +10:30
Justin Moon 57d21206db cln_plugin: add `shutdown()` method to `Plugin`
When plugins receive a "shutdown" notification, then can call this
method which will shutdown `cln_plugin`.

Then they can await `plugin.join()` and do any remaining cleanup
there.

This helps avoid a pain-point where plugin authors need to handle
2 separate plugin shutdown mechanisms https://github.com/ElementsProject/lightning/issues/6040
2023-03-18 11:52:29 +00:00
Rusty Russell acf01f4c09 pytest: don't run test_backfill_scriptpubkeys under valgrind in CI.
It seems that bitcoind frequently dies on this test.  I assume running
the multiple nodes under valgrind with the extra 214 blocks is too
memory-hungry?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 658bae30d5 lightningd: require "jsonrpc": "2.0" as per JSONRPC spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: require the `"jsonrpc": "2.0"` property (requests without this deprecated in v0.10.2).
2023-03-18 15:55:49 +10:30
Rusty Russell 9366e6b39f cleanup: rename json_add_amount_msat_only to json_add_amount_msat
Now there's no compat variant, we can rename this function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 983542f2a7 global: remove deprecated "msat" suffix on msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: the "msat" suffix on millisatoshi fields, as deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 780f32dfc6 global: remove deprecated non-msat-named msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: all the non-msat-named millisatoshi fields deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 67f23c19f7 lightningd: remove deprecated local_msat, remote_msat from listpeers.
Changelog-Removed: JSON-RPC: `listpeers`.`local_msat` and `listpeers`.`remote_msat` (deprecated v0.12.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell 06b9009dd8 lightningd: remove deprecated behavior where checkmessage would fail quietly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `checkmessage` now always returns an error when the pubkey is not specified and it is unknown in the network graph (deprecated v0.12.0)
2023-03-18 15:55:49 +10:30
Rusty Russell 1c4f6ab2c5 hsmd: deprecate reply_v1.
We promised two versions after v0.12, and here we are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell fd04f46a92 common/hsm_version: list sha256 for every known version.
Makes it easier when we remove support for a version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Vincenzo Palazzo 4f3f3deab6 fix: partial fix lnprototest runner
This reintroduce lnprototest after 2 releases,
there was a lot of breaking around it and this
will patch them (most of them)!

However, there are some issue related to channel opening and closing
that need some additional love and are disabled for now, but I think it
is good to introduce lnprototest now again in the CI, to be able to
stress the fix for now and see if there are other problem around.

I will take care of it!

Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-18 15:53:29 +10:30
Alex Myers dcc66d58ab doc: update release procedure
Added clarification for sums signing, file ownership, and pyln publishing
as well as a reminder to update pyln version for the release.

Changelog-None
2023-03-17 14:26:56 +10:30
Rusty Russell bfc6fedfbf CHANGELOG.md: v23.03.2
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-14 15:44:50 +10:30
Rusty Russell cdf803cd6f plugins/pay: revert removal of paying invoice without description.
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT #11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1856.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-14 15:44:50 +10:30
Alex Myers be8ed8c7f0 meta: update changelog for v23.02.1
Changelog-None
2023-03-10 15:27:24 -06:00
Alex Myers 9e2287415f offers: enable label for invoicerequest 2023-03-09 06:56:06 -06:00
Rusty Russell cfbfe5d7ee doc: update documentation for fetchinvoice(7) and offer(7).
1. Don't refer to obsolete send_invoice flag.
2. Don't refer to obsolete quantity_min field.
3. Don't refer to unsigned vs signed offers: they're all unsigned.
4. Add references to invoicerequest(7).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell aea8184e58 doc: fix modern usage of sendinvoice (changed in v22.11)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell 5e394ef53f doc: add documentation for invoicerequest commands.
As reported on Discord, these are undocumented.  And thus, um, hard to find!

Reported-by: Aaron Barnard
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell df10c62508 chanbackup: even if they enable experimental-peer-storage, check peers
Seems like LND is hanging up on receiving these messages, even though
they're odd :(

So, when a peer connects, check if it supplies or wants peer backup
(even if it doesn't support both, it shouldn't hang up, and I didn't
want to separate the two paths).

And when we go to send our own, updated backup, check features before
sending.

Fixes: #6065
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `experimental-peer-storage` caused LND to hang up on us, so only send to peers which support it.
2023-03-08 18:46:21 -06:00
Rusty Russell 6c4a438afd wallet: really allow broken migrations.
e778ebb9af ("wallet: only log broken if we
have duplicate scids in channels.") downgraded the fatal() to a broken
log message, but the user reports it still won't start up.

Perhaps they're hitting the fatal() outside the loop?  (And we're
not getting that output).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-08 16:20:09 -06:00