Commit Graph

12217 Commits

Author SHA1 Message Date
Rusty Russell 626998efce lightningd: don't timeout plugins if init is slow!
This is a minimal fix: we wait until all plugins reply from init before
continuing.  Really large or busy nodes can have other things monopolize
lightningd, then the timer goes off and we blame the plugin (which has
responded, we just haven't read it yet!).

The real answer is to have some timeouts only advance when we're idle,
or have them low-priority so we only activate them when we're idle (this
doesn't apply to all timers: some are probably important!).  But
this is a minimal fix for -rc3.

Fixes: https://github.com/ElementsProject/lightning/issues/5736
Changelog-Fixed: plugins: on large/slow nodes we could blame plugins for failing to answer init in time, when we were just slow.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-27 14:58:42 +01:00
Christian Decker 3d311c96b1 cln-plugin: Adjust visibility of some internals 2022-11-25 16:48:33 +01:00
Christian Decker db62d542e1 cln-plugin: Make the configuration in `init` public
It was set to crate level visibility for some reason, not all that
helpful :-)

Suggested-by: Sergi Delgado Segura <@sr-gi>
2022-11-25 16:48:33 +01:00
Rusty Russell 8ebde4574d CHANGELOG.md: include the v0.12.1 CHANGELOG entries!
This should have been merged into master earlier!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-25 16:30:07 +01:00
Rusty Russell 47e9e3d399 CHANGELOG.md: order into a more user-first ordering.
And internal changes don't get CHANGELOG entries, so removed
"- Added interactive transaction building routine ([#5287])".

We need to come up with some way of marking rust crate changes,
as it's not immediately obvious what "- cln-plugin" means?

Maybe we switch from JSON-RPC to Control as a more general
prefix?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-25 16:30:07 +01:00
Christian Decker 64aa42e85b doc: Add a readme to the `cln-grpc` proxy
These are some common questions that come up from time to time, so
let's make sure we address them.
2022-11-25 16:06:41 +01:00
Christian Decker 02f9c2df24 autoclean: Fix a null-pointer derefence when checking HTLC age
The autoclean plugin would assume we have a `resolved_time` which may
not be true for oldish nodes that predate our annotations.

Changelog-None Unreleased change

Reported-by: <@devastgh>
2022-11-24 12:00:24 +10:30
Rusty Russell ece77840f9 pyln-client, libplugin, rust cln-plugin: explicitly flag that we allow non-numeric JSON ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Rusty Russell d5ce5cbab3 lightningd: only use non-numeric JSON ids if plugin says we can.
We also remember whether the id is a string or not, for replacement in
JSON passthrough.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Rusty Russell 24651f57ad plugins: set non_numeric_ids flag based on getmanifest `nonnumericids` field.
And document support for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `getmanfest` response can contain `nonnumericids` to indicate support for modern string-based JSON request ids.
Changelog-Deprecated: Plugins: numeric JSON request ids: modern ones will be strings (see doc/lightningd-rpc.7.md!)
2022-11-21 11:23:54 +01:00
Rusty Russell b1f50c825f doc: document how to construct JSON ids in modern plugins and utilities.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Christian Decker 94928de479 meta: Update CHANGELOG for release candidate v22.11rc2 2022-11-18 16:14:58 +01:00
Christian Decker 2e270ea7d3 rs: Bump cln crate versions to 0.1.1 2022-11-18 15:10:32 +01:00
Christian Decker d2dae46de9 docker: Fix the dockerfile
Turns out that we were once again mixing non-venv and venv python, so
let's go full venv.

Changelog-None
2022-11-18 15:10:32 +01:00
Christian Decker 9481fb8815 msggen: Be less magic in detecting the repo root
We were assuming that we are running in a git repo, which is not true
for the PPA
2022-11-18 15:10:32 +01:00
Christian Decker 4f5ab638df grpc: Add the experimental optional flag to protoc
The `optional` keyword was first removed, and then re-added again, and
then it was made into an experimental flag. Come on people, decide
already which one it is...
2022-11-18 15:10:32 +01:00
Christian Decker 00dc0f082e make: Fix external/lowdown clean targets 2022-11-18 15:10:32 +01:00
Christian Decker 6518f6f26a make: Make the Makefile make 4.4 compatible
Fixes #5693

Changelog-None
2022-11-18 15:10:32 +01:00
Christian Decker d27591ff54 ld: Replace list of outgoing_txs with a hash table
We had a quadratic check when processing a block, and this just
reduces it to be linear. Combined with the previous fix of adding txs
multiple times, this should speed up block processing considerably.
2022-11-18 13:39:09 +01:00
Christian Decker 351b8999c9 ld: Add an outgoing_txs_map htable to avoid costly lookups
We were using a quadratic lookup to locate the transactions we
broadcast, so let's use an `htable` instead, allowing for constant
lookups during block processing.
2022-11-18 13:39:09 +01:00
Christian Decker c7ff3c4288 ld: Do not blindly add rebroadcasts to outgoing_tx set
The list of outgoing transactions was growing with every
rebroadcast. Now we just check whether it is already in the list and
don't add it anymore

Changelog-Fixed: ld: Reduce identification of own transactions to not slow down over time, reducing block processing time

Suggested-by: Matt Whitlock <@whitslack>
2022-11-18 13:39:09 +01:00
Rusty Russell f2291c44d6 onchaind: cap RBF penalty fee for testnet/regtest
On testnet I noticed if we can't reach bitcoind for some reason, we'll
keep RBFing our penalty tx ("it didn't go in, RBF harder!!").  Makes
no sense to grossly exceed the amount needed for next block, so simply
cap penalty at 2x "estimatesmartfee 2 CONSERVATIVE".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 13:38:42 +01:00
Rusty Russell 9e41754e7d wallet: fix typo in debug message.
"coinbase" test is backwards!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 13:38:42 +01:00
Rusty Russell d94b715bcf wallet/wallet.c: don't declare unused variable.
Ubuntu clang 15.0.2-1 complains:

```
wallet/wallet.c:280:6: error: variable 'i' set but not used
      [-Werror,-Wunused-but-set-variable]
        int i;
            ^
wallet/wallet.c:339:6: error: variable 'i' set but not used
      [-Werror,-Wunused-but-set-variable]
        int i;
            ^
wallet/wallet.c:4768:9: error: variable 'count' set but not used
      [-Werror,-Wunused-but-set-variable]
        size_t count;
               ^
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 12:26:59 +01:00
Rusty Russell 5e76c74622 tools/generate_wire.py: don't declare unused for variable.
Ubuntu clang 15.0.2-1 complains:

```
wire/peer_exp_wiregen.c:257:14: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
        for (size_t i = 0; *plen != 0; i++) {
                    ^
wire/peer_exp_wiregen.c:1373:14: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
        for (size_t i = 0; *plen != 0; i++) {
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 12:26:59 +01:00
Jesse de Wit 08cbdaf84f add some stuff to gitignores 2022-11-18 12:26:32 +01:00
Christian Decker d4ead53652 pyln: Bump versions 2022-11-11 10:37:37 +01:00
Christian Decker 5a08691ffc meta: Add version v22.11rc1 changelog 2022-11-11 10:37:37 +01:00
Rusty Russell d45b13df56 lightningd.service: note that the hardening setting seems to break node.js plugins
See-also: https://github.com/Ride-The-Lightning/c-lightning-REST/issues/116
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-11 08:47:59 +01:00
Christian Decker 832b2e5e2e onchaind: Adjust witness weight estimate to be more conservative
We were missing the OP_PUSH for the pubkeys, and the spec mentions we
should be using 73 bytes to estimate the witness weight. Effectively
this adds 4 bytes which really just matters in case fees hit the
floor, and computing the weight becomes important.

Changelog-Fixed: onchaind: Witness weight estimations could be slightly lower than the VLS signer
2022-11-10 16:30:27 +01:00
Christian Decker 5cbd5220d9 onchain: Document how the expected witness weight for the close tx 2022-11-10 16:30:27 +01:00
Christian Decker 7b12d3eb60 reckless: Replace custom logging with the logging crate 2022-11-10 16:29:56 +01:00
Christian Decker 77b7a74cd6 reckless: Replace urllib3 with urllib
`urllib3` does not ship as built-in with any of the recent python
releases, whereas `urllib` does, and for the uses we have, they are
pretty much identical.
2022-11-10 16:29:56 +01:00
Alex Myers fd52e260f0 reckless: neaten path conversions
Changelog-None
2022-11-10 16:29:56 +01:00
Alex Myers f1c2f811b7 reckless: avoid changing directory during install 2022-11-10 16:29:56 +01:00
Alex Myers 36b4457c04 reckless: Use urllib3 instead of requests
The requests package is preferred, but until installation of python user
dependencies is implemented, sticking with standard modules allows a
frictionless experience.
2022-11-10 16:29:56 +01:00
Rusty Russell 2e4a58efac check: fix warnings from shellcheck 0.8.0
And make errors gcc-style, so emacs can jump through the automatically.

```
In devtools/reduce-includes.sh line 21:
	    echo -n "-$LINE"
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In devtools/reduce-includes.sh line 25:
	    echo -n "."
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In tools/rel.sh line 6:
prefix=$(printf '%s\n' "${from#$common}" | sed 's@[^/][^/]*@..@g')
                               ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
prefix=$(printf '%s\n' "${from#"$common"}" | sed 's@[^/][^/]*@..@g')


In tools/rel.sh line 7:
printf '%s\n' "$prefix/${to#$common}"
                            ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
printf '%s\n' "$prefix/${to#"$common"}"

For more information:
  https://www.shellcheck.net/wiki/SC3037 -- In POSIX sh, echo flags are undef...
  https://www.shellcheck.net/wiki/SC2295 -- Expansions inside ${..} need to b...
make: *** [Makefile:553: check-shellcheck] Error 123
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 8a217f13cf bolt12: update comments to match latest spec.
No code changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 37bc4603b8 lightningd: re-add 'offerout' functionality, as 'invoicerequest'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 7906770489 lightningd: add "savetodb" argument to createinvoicerequest, add listinvoicerequests/disableinvoicerequest
This is how we put new invoice_requests into the db; this will be used
by a new "invoicerequest" command which replaces "offerout".

The API	is now the same as the offers api.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 02d7454226 db: add invoicerequests table.
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly.  Create a new table for them, and
associated functions.

The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
2022-11-09 15:08:03 +01:00
Rusty Russell 891cef7b2b bolt12: routines to hash the invreq parts.
This gives us a unique identifier, by which we can match an invoice to
their invoice_request.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 179f573e45 lightningd/invoice.c, plugins/fetchinvoice.c: use tlv_make_fields() instead of towire/fromwire hack.
I forgot this existed!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 1d1174c286 offers: use existing copied fields.
We no longer have to refer back to the offer for which we're making
the invoice_request, or to the invoice_request we made for an invoice,
as they are all mirrored (and we check!).

It's clearer to simply look at the object directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell bc283cecf2 decode: print unknown fields in bolt12 strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell ef2f4a0648 bolt12: use spec field names, update decode API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell fdb3d9186a devtools/bolt12-cli: fix decode to understand modern fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 505356145b bolt12: update to modern signature scheme.
This changed, by popular demands.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 1e3cb01546 bolt12: import the latest spec, update to fit.
I know this is an unforgivably large diff, but the spec has changed so
much that most of this amounts to a rewrite.

Some points:
* We no longer have "offer_id" fields, we generate that locally, as all
  offer fields are mirrored into invoice_request and then invoice.
* Because of that mirroring, field names all have explicit offer/invreq/invoice
  prefixes.
* The `refund_for` fields have been removed from spec: will re-add locally later.
* quantity_min was removed, max == 0 now mean "must specify a quantity".
* I have put recurrence fields back in locally.

This brings us to 655df03d8729c0918bdacac99eb13fdb0ee93345 ("BOLT 12:
add explicit invoice_node_id.")

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 846a520bc2 offers: remove 'send-invoice' offers support.
This has radically changed in the spec, so remove it now, and we'll
reintroduce / rewrite it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00