Commit Graph

1319 Commits

Author SHA1 Message Date
Adi Shankara b2caa56c27 update new documentation and move to doc/ 2023-07-16 12:59:40 +09:30
Rusty Russell ad592d8b0d plugins: fix shutdown notification to contain object.
We fixed the others.  There are no fields, but this keeps it consistent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `shutdown` notification contains `shutdown` object (notification consistency)
2023-07-13 18:59:37 +09:30
Rusty Russell 46fc14c51d plugins: add support for wildcard subscription.
Requested-by: Shahana Farooqui @Shahana
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: plugins can subscribe to all notifications using "*".
2023-07-13 18:59:37 +09:30
Greg Sanders 4b70736d13 Support p2tr deposit addresses
Changelog-Added: JSON-RPC: newaddr: p2tr option to create taproot addresses.
Changelog-Changed: Wallet: we now use taproot change addresses.
2023-07-11 11:41:15 +09:30
Shahana Farooqui 8e14b3ff8f doc: Typo in balance_snapshot notification object 2023-07-10 14:51:11 +09:30
Shahana Farooqui b9fba1c0b5 notification: block_added json key updated from block to block_added
Changelog-Added: JSON-RPC: `block_added` notification wraps fields in `block_added` object (notification consistency)
2023-07-10 14:51:11 +09:30
Shahana Farooqui 79092c16b1 notification: disconnect with json key disconnect
Changelog-Added: JSON-RPC: `connect` and `disconnect` notifications now wrap `id` field in a `connect`/`disconnect` object (consistency with other notifications)
2023-07-10 14:51:11 +09:30
Shahana Farooqui e247d16536 notification: connect with json key connect 2023-07-10 14:51:11 +09:30
Vincenzo Palazzo b3f72f9eed fix(docs): Fix instructions for installing cln on FreeBSD
This commit is adding the instructions for compiling cln on FreeBSD,
because it looks that we not longer compile with the FreeBSD package
manager, and I have no idea who the mantainer of this package is.

Link: https://github.com/ElementsProject/lightning/issues/6301
Reported-by: @bektar
Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-10 13:48:36 +09:30
Shahana Farooqui 5053999ff3 doc: Updated instructions
Updated release instruction after build-release.sh fixes.
Add Docker buildx and QEMU instruction in reproducible
2023-07-10 13:24:51 +09:30
Rusty Russell 3e57496af9 plugins: don't let multifundchannel open a new anchor channel without reserves.
If we're opening a channel with a peer which support anchors (and
we do), we tell fundpsbt/utxopsbt to enforce the emergency reserve;
this matters, as it doesn't know about the channel yet, and thus
won't (if it's our first anchor channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `fundchannel` and `multifundchannel` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (or are opening one).
2023-06-29 11:28:47 -04:00
Rusty Russell 0402e645f0 lightningd: add option `opening_anchor_channel` to fundpsbt, utxopsbt.
This is needed when we know we're *opening* an anchor channel, to
override the "do we already have an anchor channel open?" logic.

Also, document the nonwrapped arg added in v23.02.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundpsbt` and `utxopsbt` new parameter `opening_anchor_channel` so lightningd knowns it needs emergency reserve for anchors.
2023-06-29 11:28:47 -04:00
Rusty Russell 391da2f440 lightningd: don't let them fundpsbt below emergency reserve.
This is the simple version which always tries to keep some sats if we
have an anchor channel.  Turns out that we need something more
sophisticated for multifundchannel, so that's next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (and `all` will be reduced appropriately).
Changelog-Changed: JSON-RPC: `fundpsbt` and `utxopsbt` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels.
2023-06-29 11:28:47 -04:00
Rusty Russell 75aca3cbb6 config: add min-emergency-msat option.
For anchors, we need some sats sitting around in case we need to CPFP
a close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `min-emergency-msat` setting for (currently experimental!) anchor channels, to keep funds in reserve for forced closes.
2023-06-29 11:28:47 -04:00
Rusty Russell 17821da80b lightningd: turn on experimental support for zero-fee htlc anchors.
We disabled experimental support for opening non-zero-fee anchor
channels (though old nodes may still have such channels if they turned
that on!).

So we simply call this `experimental-anchors`, since this is the variant
which we expect to be used widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: protocol: added support for zero-fee-htlc anchors (`option_anchors_zero_fee_htlc_tx`), using `--experimental-anchors`.
2023-06-29 11:28:47 -04:00
Rusty Russell e1a9e25412 common/channel_type: add zero-fee-anchor type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 6087decec3 dualopend: use a lower feerate for first commitment tx for anchors.
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell 96e198f751 openingd: use a lower feerate for first commitment tx for anchors.
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell e45bf14300 lightningd: use lowball feerate for commit_tx on anchor channels.
Since we can CPFP, we don't have to track the feerate as closely.  But
it still needs to get in the mempool, so we use 10 sat/byte, or the
100 block estimate if that is higher.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates` has new fields `unilateral_anchor_close` to show the feerate used for anchor channels (currently experimental), and `unilateral_close_nonanchor_satoshis`.
Changelog-Changed: JSON-RPC: `feerates` `unilateral_close_satoshis` now assumes anchor channels if enabled (currently experimental).
2023-06-29 11:28:47 -04:00
Rusty Russell e47e51edf2 lightningd: don't make htlc_timeout_satoshis/htlc_success_satoshis zero if we support anchors.
It depends on whether we negotiated anchors: just document that this
field doesn't apply for anchors (it becomes zero by the end of this
patch series, which is weird).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
avatar4d 856808aa2d Correct INSTALL.md requirements for OpenBSD
GNU xgettext is required to compile. It is found in the gettext-tools package in OpenBSD. Tested on OpenBSD 7.3
2023-06-26 08:18:28 +09:30
Rusty Russell 263625872e autoclean: allow dynamic changes to autoclean-cycle.
Slightly less trivial: reset timer unless it's currently running callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 20:08:25 +09:30
Rusty Russell f8f8045a06 autoclean: various configuration options now dynamic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `autoclean` configuration variables now settable with `setconfig`.
2023-06-20 20:08:25 +09:30
Rusty Russell 630dba8840 lightningd: call setconfig on plugins' dynamic options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 20:08:25 +09:30
Rusty Russell 703ffdf59b setconfig: comment out now-overridden lines.
Do it slightly intelligently, so if we had set previously using setconfig
we don't keep appending new ones, but replace it in-place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 20:08:25 +09:30
Rusty Russell 6546be9757 lightningd: setconfig command.
Currently only implemented for min-capacity-sat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: new command `setconfig` allows a limited number of configuration settings to be changed without restart.
2023-06-20 20:08:25 +09:30
Rusty Russell aca893f1eb common: allow configvars to be marked dynamic.
To test, we do min-capacity-sat which is simple.  We also update the
listconfigs man page which contained some obsolete information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 20:08:25 +09:30
Matt Morehouse cc71f75c4b doc: add section about Clang code coverage 2023-06-15 10:47:56 +09:30
Vincenzo Palazzo 85992e6e48 feat(bitcoind): pass the current known block height
When core lightning is asking the information about
the blockchain with `getchaininfo` command lightningd
know already the information about the min and max block height.

the problem is when we have a smarter Bitcoin backend that is able
to switch between different clients in some cases is helpful
give the information about current known height by lightningd and
pass it down to the plugin.

In this way, the plugin knows what is the correct known height from lightnind, and can
try to fix some problems if any exit.

This is particularly useful when you are syncing a new backend from scratch
like https://github.com/cloudhead/nakamoto and we avoid returning the
lower height from the known, and avoid the crash of core lightning.

With this information, the plugin can start to sync the chain and return
the answer back only when the chain is in sync with the current status of
lightningd.

Another reason to add this field and not wait the correct block in core
lightning itself is because Bitcoin Core is extremely slow to sync up,
so the question here is, how long should we wait? The time depends
on various factors.

With this approach of informing the plugin about the height, in some cases,
you can start the syncing but move the execution to another backend until
the previous one is ready.

The problem I want to solve is that I don't want to be left in the dark when
we run `getchaininfo`, and I want to have the opportunity to wait for
the blockchain sync or decide to dispatch the request elsewhere.

Changelog-Added: Pass the current known block height down to the getchaininfo call.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-06-13 16:27:10 +02:00
Rusty Russell 5f6933d1c4 doc: document that we strip punctuation from parameter names.
This lead to confusion about how to specify `amount_msat`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-06 19:47:08 +09:30
Adi Shankara e83782f5de doc: Add guides and GitHub workflow for doc sync
This PR:
- adds all the guides (in markdown format) that is published at https://docs.corelightning.org/docs
- adds a github workflow to sync any future changes made to files inside the guides folder
- does not include API reference (json-rpc commands). Those will be handled in a separate PR since they're used as manpages and will require a different github workflow

Note that the guides do not exactly map to their related files in doc/, since we reorganized the overall documentation structure on readme for better readability and developer experience. For example, doc/FUZZING.md and doc/HACKING.md#Testing are merged into testing.md in the new docs. As on the creation date of this PR, content from each of the legacy documents has been synced with the new docs. Until this PR gets merged, I will continue to push any updates made to the legacy documents into the new docs.

If this looks reasonable, I will add a separate PR to clean up the legacy documents from doc/ (or mark them deprecated) to avoid redundant upkeep and maintenance.

Changelog-None
2023-06-06 12:40:19 +09:30
Shahana Farooqui 73b73483bf doc: making release document update 2023-06-05 14:54:32 +02:00
Rusty Russell 50a7681171 doc/Makefile: restore CI check for schema changes.
We now know the base reference, and we've rebased, so we can do a
simple diff.  Also, this means we can use a magic commit message
`No-schema-diff-check` to suppress false positives.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-05 09:25:57 +09:30
Rusty Russell 3ac949d4c3 listconfigs: add `plugin` field if config is for a plugin.
I chose the full path name, not just the basename.

Suggested-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell 6275dd384c lightningd: deprecate listconfigs direct fields.
Changelog-Deprecated: JSON-RPC: `listconfigs` direct fields, use `configs` sub-object and `set`, `value_bool`, `value_str`, `value_int`, or `value_msat` fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell 1b252f349f config: replace accept-htlc-tlv-types with accept-htlc-tlv-type
We use multi-specifiable options elsewhere, this is just another.
Otherwise you can't add, you can only set them all.

Changelog-Added: Config: `accept-htlc-tlv-type` (replaces awkward-to-use `accept-htlc-tlv-types`)
Changelog-Deprecated: Config: `accept-htlc-tlv-types` (use `accept-htlc-tlv-type` multiple times)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell 0df97547dd lightningd: don't simply ignore defaults on flags, deprecate.
Changelog-Deprecated: Plugins: `default` no longer accepted on `flag` type parameters (it was silently ignored, so just don't set it).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell b9270c564a lightningd: listconfigs update, using configvars code.
listconfigs is convenient, but it doesn't handle multi-options well: it
outputs an object with duplicate fields in this case (e.g. log-file), nor
is it extensible to show more than raw values.

However, listconfigs doesn't do what other list commands do (use a
sub-object "configs") so we can put the new values under that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listconfigs` now has `configs` subobject with more information about each config option.
2023-06-03 10:50:29 +09:30
Rusty Russell 5a0994c2d0 lightningd: allow --regtest.
Developers, rejoice (we already have --testnet, --signet and --mainnet!).

Changelog-Added: Config: `--regtest` option as alias for `--network=regtest`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell ca34931a29 doc/Makefile: suppress stupid check.
I wasted far too much time on this, disable and reenable later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell a6772e9dec common: add new internal type for websockets.
Now it's not a public type, we need a way to refer to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell 21958879cf lightningd: deprecated --announce-addr-dns.
This obsoletes the use of --announce-addr-dns which I know Michael
didn't really like either.

Changelog-Deprecated: Config: `announce-addr-dns`; use `--bind-addr=dns:ADDR` for finer control.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell 6d0b46c9b1 wireaddr: clean up option parsing, support `dns:` prefix directly.
This is a major cleanup to how we parse addresses.

1. parse_wireaddr now supports the "dns:" prefix to support dns records (type 5).
2. We are less reliant on separate_address_and_port() which gets confused by
   that colon.
3. We explicitly test every possible address type we can get back from
   parsing, and handle them appropriately.

We update the documentation to use the clearer HOSTNAME vs DNS prefixes now
we also have `dns:` as a prefix.

Changelog-Added: Config: `bind` can now take `dns:` prefix to advertize DNS records.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell 3f35d48fe4 common: remove websocket type from wireaddr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Christian Decker c029de7d56 msggen: Add preapproveinvoice and preapprovekeysend to msggen 2023-05-29 14:50:13 +09:30
Jon Griffiths 8b4136bad5 build: remove gmp
Core and secp have not used gmp for a very long time now (core disabled
it in 2015).

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2023-05-24 18:21:43 +09:30
Rusty Russell e7d4c3175a build: remove --enable-experimental-features / EXPERIMENTAL_FEATURES
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell b8aa3a579e listtransactions: remove annotations, which were only in EXPERIMENTAL_FEATURES.
I don't know if anyone was using them, they seem half-hearted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell ccf084156d channeld: use explicit --experimental-upgrade flag, not #ifdef EXPERIMENTAL_FEATURES
And no longer insist on opt_quiesce.

Changelog-EXPERIMENTAL: Config: `--experimental-upgrade-protocol` enables simple channel upgrades.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell 6c23349c72 channeld: allow stfu based on peer features, not EXPERIMENTAL_FEATURES.
Changelog-EXPERIMENTAL: Config: `--experimental-quiesce` enables queiescence, for testing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Amin Bashiri 222dafb197 doc: remove duplicate wumbo key from listconfigs 2023-05-21 12:45:21 +02:00
Christian Decker 1e614dfb8a jsonrpc: Add request schemas for decode and decodepay 2023-05-05 11:54:41 +09:30
Christian Decker 2a52e52015 jsonrpc: Add versioning annotation to listpeerchannels 2023-05-05 11:54:41 +09:30
Rusty Russell 3f95b559a3 doc: document that urgent doesn't use the 2-block estimate, but the 6-block.
Turns out this was accidentally changed for v0.10.1 in
d8e68893f5 where we made fee levels less
aggressive.

Oops.  I guess we can fix the docs.  And we now have "2blocks" if you
want it really fast!

Closes: #6129
Changelog-Fixed: JSON-RPC: `feerates` document correctly that urgent means 6 blocks (not 2), and give better feerate examples.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-11 12:01:27 -07:00
Rusty Russell 49b7afe58f doc: give helpful examples for feerate values.
mempool.info gives sat/vB, which is 1000 too low for us!

See-also: #6161 (complains setting feerate to 5 doesn't work)

See-also: #6129
Suggested-by: @lightingorb
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-11 12:01:27 -07:00
Rusty Russell 62d9ecb6d3 hsmtool: makerune command.
You still need to actually make a rune when lightningd starts, as
commando (for safety) won't work unless you actually generate a rune
(that it knows of!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmtool: `makerune` command to make a master rune for a node.
2023-04-11 11:19:13 -07:00
Rusty Russell 7e5146ab0c common/channel_type: routines to set known variants, set scid_alias.
I tested this indeed breaks if we don't accept it, then implemented
the code to accept it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: protocol: We now correctly accept the `option_scid_alias` bit in `open_channel` `channel_type`.
Changelog-Deprecated: protocol: Not setting `option_scid_alias` in `option_channel` `channel_type` for unannounced channels.
2023-04-10 17:26:47 +09:30
Rusty Russell 3a3370f4c1 feerates: add `floor` field for the current minimum feerate bitcoind will accept
Changelog-Added: JSON-RPC: `feerates`: added `floor` field for current minimum feerate bitcoind will accept
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 9e2d4240b1 lightningd: handle bcli plugins returning fee_floor and feerates parameters.
Changelog-Added: Plugins: `estimatefees` can return explicit `fee_floor` and `feerates` by block number.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell c46473e615 lightningd: allow "NNblocks" and "minimum" as feerates.
And consolidate descriptions into lightning-feerates().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` now allow "minimum" and NN"blocks" as `feerate` (`feerange` for `close`).
2023-04-10 07:31:12 +09:30
Rusty Russell 64b1ddd761 lightningd: clean up feerate handling, deprecate old terms.
Drop try_get_feerate() in favor of explicit feerate_for_deadline() and
smoothed_feerate_for_deadline().

This shows us everywhere we deal with old-style feerates by names.

`delayed_to_us` and `htlc_resolution` will be moving to dynamic fees,
so deprecate those.

Note that "penalty" is still used for generating penalty txs for
watchtowers, and "unilateral_close" still used until we get zero-fee
anchors.

Changelog-Added: JSON-RPC: `feerates` `estimates` array shows fee estimates by blockcount from underlying plugin (usually *bcli*).
Changelog-Changed: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) value *slow* is now 100 block-estimate, not half of 100-block estimate.
Changelog-Deprecated: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) expressed as, "delayed_to_us", "htlc_resolution", "max_acceptable" or "min_acceptable".  Use explicit block counts or *slow*/*normal*/*urgent*/*minimum*.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 7aa8c76002 pytest: test parsefeerate explicitly.
Since we're messing with feerates, it's good to test this directly upfront.

Also, fix documentation!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell 6799cd5d0b plugins/bcli: move commit-fee (dev-max-fee-multiplier) and into core.
Turns out the two bcli replacements I checked (`sauron` and
`trustedcoin`) don't even implement this, and the multiplier makes
more sense in lightningd, especially as we move to bcli just providing
raw feerate estimates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Matt Morehouse ca80dee514 doc: add section about improving fuzzing corpora
We want to encourage contributions to the seed corpora that improve
coverage.
2023-04-09 12:41:27 +09:30
Matt Morehouse 6e11a2e416 doc: document "make check-fuzz" 2023-04-09 12:41:27 +09:30
Anmol Agrawal dae92c5830 Update INSTALL.md 2023-04-07 16:01:47 +09:30
Rusty Russell 3c83aed9d1 doc: fix commando-listrunes SHA256SUM line.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-05 09:41:04 +02:00
Vincenzo Palazzo b92b9f074d delpay: delete the payment by status from the db
There are cases (difficult to reproduce with a test) where
a payment will fail one time and succeed later.

As far I understand in this case the groupid field of the payment
is the same, and the only thing that change is the status, so
our logic inside the delpay is ambiguous where it is not
possible to delete a payment as described in https://github.com/ElementsProject/lightning/issues/6114

A sequence of commands that explain the problem is

```
$ lc -k listpays payment_hash=H
{
   "pays": [
      {
         "bolt11": "I",
         "destination": "redacted",
         "payment_hash": "H",
         "status": "complete",
         "created_at": redacted,
         "completed_at": redacted,
         "preimage": "P",
         "amount_msat": "redacted",
         "amount_sent_msat": "redacted"
      }
   ]
}
$ lc delpay H complete
{
   "code": 211,
   "message": "Payment with hash H has failed status but it should be complete"
}
```

In this case, the delpay is not able to delete a payment because the
listpays is returning only the succeeded one, so by running the
listsendpays we may see the following result where our delpay logic
will be stuck because it works to ensure that all the payments stored
in the database has the status specified by the user

```
➜  VincentSSD clightning --testnet listsendpays -k payment_hash=7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4
{
   "payments": [
      {
         "id": 322,
         "payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
         "groupid": 1,
         "partid": 1,
         "destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
         "amount_msat": 300,
         "amount_sent_msat": 1664,
         "created_at": 1679510203,
         "completed_at": 1679510205,
         "status": "failed",
         "bolt11": "lntb1pjpkj4xsp52trda39rfpe7qtqahx8jjplhnj3tatxy8rh6sc6afgvmdz7n0llspp50lr5hmdm0re0xvcp2hv3nf2wwvx0r8q3h3e7jmqz0awdfg6w206qdp0w3jhxarfdenjqargv5sxgetvwpshjgrzw4njqun9wphhyaqxqyjw5qcqp2rzjqtp28uqy77te96ylt7ek703h4ayldljsf8rnlztgf3p8mg7pd0qzwf8a3yqqpdqqqyqqqqt2qqqqqqgqqc9qxpqysgqgeya2lguaj6sflc4hx2d89jvah8mw9uax4j77d8rzkut3rkm0554x37fc7gy92ws9l76yprdva2lalrs7fqjp9lcx40zuty8gca0g5spme3dup"
      },
      {
         "id": 323,
         "payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
         "groupid": 1,
         "partid": 2,
         "destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
         "amount_msat": 300,
         "amount_sent_msat": 3663,
         "created_at": 1679510205,
         "completed_at": 1679510207,
         "status": "failed"
      },
      {
         "id": 324,
         "payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
         "groupid": 1,
         "partid": 3,
         "destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
         "amount_msat": 300,
         "amount_sent_msat": 3663,
         "created_at": 1679510207,
         "completed_at": 1679510209,
         "status": "failed"
      },
      {
         "id": 325,
         "payment_hash": "7fc74bedbb78f2f3330155d919a54e730cf19c11bc73e96c027f5cd4a34e53f4",
         "groupid": 1,
         "partid": 4,
         "destination": "030b686a163aa2bba03cebb8bab7778fac251536498141df0a436d688352d426f6",
         "amount_msat": 300,
         "amount_sent_msat": 4663,
         "created_at": 1679510209,
         "completed_at": 1679510221,
         "status": "complete",
         "payment_preimage": "43f746f2d28d4902489cbde9b3b8f3d04db5db7e973f8a55b7229ce774bf33a7"
      }
   ]
}
```

This commit solves the problem by forcing the delete query in the
database to specify status too, and work around this kind of
ambiguous case.

Fixes: f52ff07558 (lightningd: allow delpay to delete a specific payment.)
Reported-by: Antoine Poinsot <darosior@protonmail.com>
Link: https://github.com/ElementsProject/lightning/issues/6114
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Co-Developed-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: delpay be more pedantic about delete logic by allowing
delete payments by status directly on the database.
2023-04-05 06:15:47 +09:30
Rusty Russell 3be36a66e3 configure: support sanitizers properly.
For example, if we use -fsanitize=undefined, we can't do unaligned
integer access, but since we didn't test with the sanitizer flags, we
didn't know this, and set `HAVE_UNALIGNED_ACCESS=1`.

Also, add -fno-sanitize-recover= in developer mode, so we actually
fail binaries if something is detected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-05 06:12:24 +09:30
Shahana Farooqui 3e310a3d3e doc: commando-listrunes & commando-blacklist 2023-04-01 14:07:23 +10:30
Shahana Farooqui af2c1f1881 doc: schemas for commando-listrunes & commando-blacklist 2023-04-01 14:07:23 +10:30
Vincenzo Palazzo 4c6966d16a docs: update autogenerate file
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-03-26 13:42:29 +10:30
Christian Decker f4efe6c899 docs: Add docs on code generation 2023-03-25 15:40:35 +10:30
Christian Decker 7153beff28 docs: Use admonition markup for warnings and notes 2023-03-25 15:40:35 +10:30
Christian Decker f19792c241 docs: Remove redundant ToC in FAQs 2023-03-25 15:40:35 +10:30
Christian Decker f1293ed0e6 docs: Add LICENSE to the About section 2023-03-25 15:40:35 +10:30
Christian Decker 458195c29f docs: Fix a number of broken links in the generated docs
Changelog-None
2023-03-25 15:40:35 +10:30
Christian Decker 178e0b6463 docs: Structure the files in mkdocs
We have 3 personas:

 - Users
 - Developers
 - Maintainers

The first one basically cover the installation documentation. The
latter two are sorted into the "Developer" category, and the reference
category serves as a quick lookup for facts on anything CLN related.
2023-03-25 15:40:35 +10:30
Christian Decker a80c1ae40c docs: Switch to mkdocs for documentation 2023-03-25 15:40:35 +10:30
Rusty Russell b8519a6a1a plugins/sql: add listclosedchannels
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sql` now includes `listclosedchannels`.
2023-03-25 15:28:02 +10:30
Rusty Russell 89f91b9bb4 lightningd: add listclosedchannels command.
Changelog-Added: JSON-RPC: `listclosedchannels` to show old, dead channels we previously had with peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Rusty Russell e75cf2e7fb listpeerchannels: add channel_type, both in hex and as array of names.
Changelog-Added: JSON-RPC: `listpeerchannels` now has `channel_type` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30
Greg Sanders 887c6f71cf Add PSBT version setting RPC to aid with debugging and compatibility
PSBTv2 support is quite low in the ecosystem, so having a call to convert
log messages and the like should be useful since they'll often be in v2.

Changelog-Added: Added setpsbtversion RPC to aid debugging and compatibility
2023-03-23 16:10:55 +10:30
Kristaps Kaupe 906279a46e Output channel_id in listfunds
Changelog-Added: JSON-RPC: `listfunds` now has a `channel_id` field.
2023-03-21 10:29:33 +00:00
Rusty Russell 780f32dfc6 global: remove deprecated non-msat-named msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: all the non-msat-named millisatoshi fields deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
Rusty Russell 67f23c19f7 lightningd: remove deprecated local_msat, remote_msat from listpeers.
Changelog-Removed: JSON-RPC: `listpeers`.`local_msat` and `listpeers`.`remote_msat` (deprecated v0.12.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Alex Myers dcc66d58ab doc: update release procedure
Added clarification for sums signing, file ownership, and pyln publishing
as well as a reminder to update pyln version for the release.

Changelog-None
2023-03-17 14:26:56 +10:30
Alex Myers 9e2287415f offers: enable label for invoicerequest 2023-03-09 06:56:06 -06:00
Rusty Russell cfbfe5d7ee doc: update documentation for fetchinvoice(7) and offer(7).
1. Don't refer to obsolete send_invoice flag.
2. Don't refer to obsolete quantity_min field.
3. Don't refer to unsigned vs signed offers: they're all unsigned.
4. Add references to invoicerequest(7).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell aea8184e58 doc: fix modern usage of sendinvoice (changed in v22.11)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Rusty Russell 5e394ef53f doc: add documentation for invoicerequest commands.
As reported on Discord, these are undocumented.  And thus, um, hard to find!

Reported-by: Aaron Barnard
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-09 06:56:06 -06:00
Carl Dong ef51ae3c6d msggen: Enable SendCustomMsg 2023-02-08 23:10:53 -06:00
Rusty Russell 49b3459be5 lightningd: don't put old deprecated `local_msat` and `remote_msat` in listpeerchannels.
These were deprecated in v0.12.0, hence scheduled for removal next version anyway
(use local_fund_msat and remote_funds_msat).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 21:17:37 -06:00
Rusty Russell a71bd3ea37 options: create enable/disable option for peer storage.
Since it's not spec-final yet (hell, it's not even properly specified
yet!) we need to put it behind an experimental flag.

Unfortunately, we don't have support for doing this in a plugin; a
plugin must present features before parsing options.  So we need to do
it in core.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
niftynei 442b479d2c df: add new config option for v2 opens `--require_confirmed_inputs`
If set, require peers to only provide confirmed inputs for any v2 open
(both in accepter + opener role)
2023-02-07 21:03:36 -06:00
niftynei 9f53e3c7f5 df: wire up peer's "require-confirmed-inputs"
We push this info out to the various RPCs/hooks.
2023-02-07 21:03:36 -06:00
Michael Schmoock e736c4d5f4 doc: listpeers new attribute num_channels 2023-02-07 14:46:04 -06:00
Carl Dong 1dbc29b8c0 lightningd: Add `signinvoice` to sign a BOLT11 invoice.
Though there's already a `createinvoice` command, there are usecases where a
user may want to sign an invoice that they don't yet have the preimage to. For
example, they may have an htlc_accepted plugin that pays to obtain the preimage
from someone else and returns a `{ "result": "resolve", ... }`.

This RPC command addresses this usecase without overly complicating the
semantics of the existing `createinvoice` command.

Changelog-Added: JSON-RPC: `signinvoice` new command to sign BOLT11
invoices.
2023-02-06 15:54:32 -06:00
Carl Dong f0b8544eba doc: Correct `createinvoice`'s `invstring` description
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT #11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
2023-02-06 15:54:32 -06:00
Rusty Russell bcab3f7e83 Makefile: don't try to build sql plugin if there's no sqlite3 support.
Reported-by: @whitslack
Fixes: #5940
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-06 09:20:28 +01:00
Joel Klabo 0d1ee8d7f5 Need `sudo` for `make install` 2023-02-05 15:41:19 +10:30
Joel Klabo 57874574ae Add protobuf as a Dependency 2023-02-05 15:41:19 +10:30
Joel Klabo 091e8cefd1 Must Specify pip3 on macOS 2023-02-05 15:41:19 +10:30
Joel Klabo 72b83efc4b Use Python 3.7.8 Instead of 3.7.4 for macOS Install 2023-02-05 15:41:19 +10:30
niftynei 35f12b4ca1 upgradewallet: JSONRPC call to update p2sh outputs to a native segwit
v2 opens require you to use native segwit inputs

Changelog-Added: JSONRPC: `upgradewallet` command, sweeps all p2sh-wrapped outputs to a native segwit output
2023-02-04 15:31:16 +10:30
niftynei 679a473f9a fundpsbt: add option to filter out wrapped p2sh inputs
We need to be able to only use non-wrapped inputs for v2/interactive tx
protocol.

Changelog-Added: JSONRPC: `fundpsbt` option `nonwrapped` filters out p2sh wrapped inputs
2023-02-04 15:31:16 +10:30
Rusty Russell f87c7ed439 plugins/sql: fix foreign keys.
I noticed that our subtables were not being cleaned, despite being "ON
DELETE CASCADE".  This is because foreign keys were not enabled, but
then I got foreign key errors: rowid cannot be a foreign key anyway!

So create a real "rowid" column.  We want "ON DELETE CASCADE" for
nodes and channels (and other tables in future) where we update
partially.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Rusty Russell 18397c0b87 doc-schema: make address field in getinfo response required
There were cases where address it's empty, and this cases are not right if the
field is considered optional.
This makes it required and add the field also when `--offline` is set.

Changelog-Changed: JSON-RPC: `getinfo` `address` array is always present (though may be empty)
2023-02-01 13:55:00 +10:30
Michael Schmoock dcb9b4b8d1 make: fix make doc error
This seems to fix issue #5941
Certain make version(?) will parse the ')' as function end when not
inside "" but ''.

Changelog-None
2023-02-01 12:46:17 +10:30
Adi Shankara 959678244c doc: remove sections on litestream, .dump and vacuum into
Changelog-None
The sections on SQLite Litestream, sqlite3 .dump and VACUUM INTO commands
were to be removed six months after 0.10.3, due to issues observed in #4857.

We now recommend using --wallet=sqlite3://${main}:${backup} instead.
2023-01-31 10:19:45 +10:30
Rusty Russell fea73680d7 typo fixes found by @niftynei
Also, put the "added" lines in the request schemas for new commands:
this doesn't do anything (yet?) but it keeps `make schema-added-check` happy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 259dd2a652 doc: add examples for sql plugin.
Prompted by @shahanafarooqui's playing with examples and finding
common errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 0240c24936 plugins/sql: listsqlschemas command to retrieve schemas.
Good for detection of what fields are present.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell d8320f015f plugins/sql: add bkpr-listaccountevents and bkpr-listincome support.
This *would* be a 1-line change (add it to Makefile) except that we
previously assumed a "list" prefix on commands.

These use the default refreshing, but they could be done better using
the time-range parameters.

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 9a591277f5 plugins/sql: allow some simple functions.
And document them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 14aac0769c doc: document the sql command.
In particular, we generate the schema part from the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `sql` plugin command to perform server-side complex queries.
2023-01-30 15:15:41 -06:00
Rusty Russell 40fe893172 doc/schemas: fix old deprecations.
`"deprecated": true` is obsolete; we don't document them anyway.

Where it would have otherwise changed the GRPC wrappers, I actually put the
version number in.

We allow "listchannels" to have "satoshis" since we have some tests
that run in deprecated-api mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 2c41c5d52d doc: use specific types in schema rather than "hex".
We have "secret" and "hash" types which are often more appropriate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell fa127a4071 doc/schemas: remove unnecessary length restrictions.
If we have a specific type (not just "hex") the length is implied.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell eb6b8551d4 tools/fromschema.py: don't try to handle more complex cases.
We only handle top-level objects with an array of objects:
make sure it is one before we call the routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 83c690fe5f doc: fix listsendpays man page.
We actually had a partid allowed (in the oneOf clauses), but didn''t
document it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell d6f46e2373 lightningd: fix type of listhtlcs payment_hash.
`hash` is a tighter requirement than simply `hex`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 9ab488fc41 plugins/topology: add direction field to listchannels.
It's a core concept in the spec which isn't directly exposed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listchannels` added a `direction` field (0 or 1) as per gossip specification.
2023-01-30 15:15:41 -06:00
Rusty Russell 611795beee listtransactions: get rid of per-tx type annotations.
We didn't actually populate them properly, and the real annotations
are on inputs and outputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `listtransactions` `channel` and `type` field removed at top level.
2023-01-30 15:15:41 -06:00
Rusty Russell 4b9cb7eb76 doc: remove unused offerout schema.
We removed the command for v22.11.

Also, we removed the `refund_for` offer parameter, so remove its description
from the manpage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell 2dec805465 decode: fix handling of blinded_payinfo.
Our pay code handles this correctly, but decode was still using an old model
where there was a payinfo per hop, not per path.

Reported-by: @t-bast
See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Ken Sedgwick a4dc714cdc hsmd: add hsmd_preapprove_keysend and check_preapprovekeysend pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_keysend` is added.
Changelog-added: JSON-RPC: A new command `preapprovekeysend` is added.
2023-01-26 21:10:15 -06:00
Ken Sedgwick f29343d740 hsmd: add hsmd_preapprove_invoice and check_preapproveinvoice pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_invoice` is added.
Changelog-added: JSON-RPC: A new command `preapproveinvoice` is added.
2023-01-26 21:10:15 -06:00
Michael Schmoock a62c74be7b doc: usage of --announce-addr-discovered-port option 2023-01-25 17:57:04 +01:00
Michael Schmoock 30dea0a431 opts: deprecate --disable-ip-discovery switch
This switch was not doing anything useful anymore.
We deprecate it anyways to notify the user about the new switch.

Changelog-Deprecated: The old --disable-ip-discovery config switch
2023-01-25 14:37:56 +01:00
Michael Schmoock ee046662a7 doc: announce-addr-discovered config switch 2023-01-25 14:37:56 +01:00
Michael Schmoock 3babbc5201 opts: announce-addr-discovered on/off/auto switch
This adds the option to explicitly enable ip-discovery, which maybe
helpful for example when a user wants TOR announced along with
discovered IPs to improve connectivity and have TOR just as a fallback.

Changelog-Added: Adds config switch 'announce-addr-discovered': on/off/auto
2023-01-25 14:37:56 +01:00
Michael Schmoock e932f05bc8 ci: adds git fetch before doing schema checks
This fixes the CI errors when doing `make check-source`
steps 'schema-added-check' and 'schema-removed-check'.
These errors prevented CI from performing these steps correctly:

```
fatal: ambiguous argument 'main': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
```

I changed it so that CI does a `git fetch origin` at first and do
the `git diff` against 'origin/master' (which then exist).
Also fixed a bug in the script that was missing $$master in the same line.

Also I added that the script shows the actual diff before failing,
so the user quickly sees whats wrong.
2023-01-25 14:37:56 +01:00
Rusty Russell 932ca9e91f lightningd: deprecate p2sh-segwit addresses for `newaddr` `addresstype`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `newaddr`: `addresstype` `p2sh-segwit` (use default, or `bech32`).
2023-01-14 21:17:45 +10:30
Rusty Russell 2b5f4d14d1 CI: fix schema diff check.
GitHub's master branch is called "main" apparently.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 10:13:28 +10:30
Rusty Russell f1373fd98c doc: remove manual field descriptions from listpeerchannels(7).
Some are best copied into the schema, but some are already
out-of-date, so cleanest to remove them and rely on the generated (and
thus, checked!) fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 6c0b9b0c78 lightningd: deprecate listpeers.channels
Changelog-Deprecated: JSON-RPC: `listpeers` `channels` array: use `listpeerchannels`

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo c488561282 plugins/topology: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 6fa904b4fb lightningd: add listpeerchannels command
Changelog-Added: JSON-RPC: new command `listpeerchannels` now contains information on direct channels with our peers.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Rusty Russell 3a39c635b6 CI: rough check that schema changes seem to mention added, don't delete non-deprcated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 717cb03f51 doc: add recent additions, fix annotation on listpeers to actually deprecate.
TODO: It would be great to similarly annotate new/deprecated commands and
their parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: doc: we now annotate what versions JSON field additions and deprecations happenened.
2023-01-11 12:51:18 +10:30
Rusty Russell b2148d0eab docs: handle "added": "version" and "deprecated": "version" from schemas.
This means we will document deprecations and additions, rather than just
pretending they've always been that way!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 3f8199bbfd doc: document that we should annotate added and deprecated schemas.
No longer just delete them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 404e961bad cli: add -c/--commando support.
It's easier to type:

```
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== invoice any "Invoice Label" "Invoice Description"
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== commando amount_msat=100000 label="invoice label" description="invoice description"
```

Than:

```
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '["any", "Invoice Label", "Invoice Description"]' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '{"amount_msat": "100000", "label": "invoice label", "description": "invoice description"}' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: `--commando=peerid:rune` (or `-c peerid:rune`) as convenient shortcut for running commando commands.
2023-01-11 11:13:27 +10:30
Rusty Russell 3f0c5b985b commando: add filtering support.
1. When we receive a commando command from a remote using the `filter`
   field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
   more efficient than using filtering locally.

Of course, older remote nodes will ignore the filter, but that's
harmless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
2023-01-11 11:13:27 +10:30
Rusty Russell 435f8d84dc lightning-cli: fix error code on invalid options, document them.
The top of the file indicates the following errors:

    #define NO_ERROR 0
    #define ERROR_FROM_LIGHTNINGD 1
    #define ERROR_TALKING_TO_LIGHTNINGD 2
    #define ERROR_USAGE 3

But we didn't use the right one for opt_parse failure, and didn't use the
correct constants everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
tony-voltage ddfdab1231 doc: channel_state_changed has a timestamp field 2023-01-09 19:37:48 +01:00
Justin Litchfield 879db694e1 Ping request types are changed from number to u16 2023-01-09 14:35:23 +01:00
Dennis Reimann 81213cafd7 doc: fix accept-htlc-tlv-types description 2023-01-09 11:45:14 +01:00
Simon Vrouwe d49aea376c doc: "--enable-experimental-features" hard-codes experimental-offers + experimental-onion-messages
as enabled, ignoring their command line flags.
Also add some references to BOLT drafts (PR #), plus a few cosmetic changes.
2023-01-05 22:14:22 +01:00
arowser f918a04738 unify the zlib version 2023-01-05 14:42:00 +01:00
arowser 90956fa947 change zlib download path 2023-01-05 14:42:00 +01:00
w0xlt 65170be5b4 Update release tag
It just updates the release tag.
2023-01-01 22:47:09 +01:00
Rusty Russell 141c836b39 doc: note that grpc needs the protobuf-compiler
```
error: failed to run custom build command for `cln-grpc v0.1.2 (/home/rusty/devel/cvs/lightning/cln-grpc)`

Caused by:
  process didn't exit successfully: `/home/rusty/devel/cvs/lightning/target/debug/build/cln-grpc-933c4bf5006f522c/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=proto/node.proto
  cargo:rerun-if-changed=proto

  --- stderr
  thread 'main' panicked at 'Could not find `protoc` installation and this build crate cannot proceed without
      this knowledge. If `protoc` is installed and this crate had trouble finding
      it, you can set the `PROTOC` environment variable with the specific path to your
      installed `protoc` binary.If you're on debian, try `apt-get install protobuf-compiler` or download it from https://github.com/protocolbuffers/protobuf/releases

  For more information: https://docs.rs/prost-build/#sourcing-protoc
  ', /home/rusty/.cargo/registry/src/github.com-1ecc6299db9ec823/prost-build-0.11.4/src/lib.rs:1296:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
make: *** [plugins/Makefile:211: target/debug/examples/cln-plugin-startup] Error 101
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-23 10:57:47 +01:00
Simon Vrouwe 93c966c649 doc: sphinx-build fix external links (urls), language warning and broken footnote 2022-12-19 19:00:39 +01:00
Simon Vrouwe 3c8ec25a6f dox: fix "sphinx-build -b html" warning: reference target not found
when runnning `sphinx-build -b html . build/`:
WARNING: None:any reference target not found: #
2022-12-19 19:00:39 +01:00
Rusty Russell 5b3746172f lightningd: remove `setchannelfee`.
Replaced by `setchannel`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changlog-Removed: JSON-RPC: `setchannelfee` (deprecated in v0.11.0)
2022-12-13 08:28:12 +10:30
Rusty Russell 2afe7a1856 pay: remove description_hash without description.
Deprecated v0.11.0.

Changelog-Removed: JSON-RPC: `pay` for a bolt11 which uses a `description_hash`, without setting `description` (deprecated v0.11.0).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-13 08:28:12 +10:30
Simon Vrouwe 80dacd183e doc: replace deprecated parameter keyword "msatoshi" with "amount_msat"
[ amount\_msat -- RR ]
2022-12-12 22:08:42 +10:30
Jesse de Wit 28b7c704dc add reserve to the fundchannel docs
[ Trivial conflict rebase -- RR ]
2022-12-12 21:07:19 +10:30
Vincenzo Palazzo df29990454 docs: fix typo
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-12-12 16:32:04 +10:30
Vincenzo Palazzo 48cfc437c0 docs: remove the table inside the reoribuild docs
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-12-12 16:32:04 +10:30
Matt Whitlock 4d64374b85 doc: drive-by spelling corrections 2022-12-12 15:34:00 +10:30
Matt Whitlock 31732f7825 fromschema.py: escape underscores in descriptions 2022-12-12 15:34:00 +10:30
Matt Whitlock 09d52b3cb4 doc: escape more naughty underscores
The only time underscores aren't special in Markdown is when they appear
in preformatted text. We have gotten away with not escaping underscores
where an asterisk-enclosed span or the paragraph ends before the next
underscore appears, but this is fragile and bad practice. Conversely,
there are many places where we have not escaped underscores but needed
to.

Escape all underscores that do not appear in preformatted blocks or
preformatted spans and are not themselves delineating emphasized spans.

The changes in this commit are exactly the result of executing the
following Bash code:

```bash
e=':x;'           	# begin loop
e+='s/^'          	# anchor match at beginning of line
e+='('            	# begin capturing subexpression
	e+='('    	# begin list of alternatives
		e+='[^`_\\]|'        	# any mundane character, or
		e+='`([^`\\]|\\.)*`|'	# backtick-enclosed span, or
		e+='\b_|_\b|'        	# underscore at boundary, or
		e+='\\.'             	# backslash-escaped character
	e+=')*'   	# any number of the preceding alternatives
e+=')'            	# end capturing subexpression
e+='\B_\B/\1\\_/;'	# escape non-formatting underscore
e+='tx'           	# repeat loop if we escaped an underscore

escape_underscores=(
	sed

	# use extended regular expressions
	-E

	# skip over indented blocks (following an empty line)
	-e '/^$/{:i;n;/^( {4,}|\t)/bi}'

	# skip over preformatted blocks
	-e '/^\s*```/,/^\s*```/{p;d}'

	# skip over generated sections
	-e '/GENERATE-FROM-SCHEMA-START/,/GENERATE-FROM-SCHEMA-END/{p;d}'

	# escape underscores
	-e "${e}"
)

"${escape_underscores[@]}" -i doc/*.[0-9].md
```

Changelog-None
2022-12-12 15:34:00 +10:30
Matt Whitlock 41987ed379 doc: check-manpages: add check for unescaped underscores 2022-12-12 15:34:00 +10:30
Rusty Russell ec025344cc lightningd: don't announce names as DNS by default.
This broke BTCPayServer, so revert.  I originally (accidentally!)
implemented this such that it broadcast both DNS and IP entries, but
Michael reported earlier that they still don't propagage well, so
simply suppress them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5795
Changelog-Changeed: Config: `announce-addr-dns` needs to be set to *true* to put DNS names into node announcements, otherwise they are suppressed.
Changelog-Deprecated: Config: `announce-addr-dns` (currently defaults to `false`).  This will default to `true` once enough of the network has upgraded to understand DNS entries.
2022-12-08 11:04:53 +01:00
Michael Schmoock f0dd701bc5 doc: document the usage of DNS hostnames
This adds documentation on lightningd-config about the usage of DNS
hostnames for --addr, --bind-addr and --announce-addr

Changelog-None
2022-12-06 20:27:00 +01:00
Rusty Russell 4613b29815 doc: document autoclean-once command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-06 10:42:05 +01:00
Christian Decker a31575ca0b tools: Add multi-language support to blockreplace.py
Suggested-by: Rusty Russell <@rustyrussell>
2022-12-06 11:39:53 +10:30
Christian Decker 744d111cea doc: Create a blockreplace tool to update generated blocks in docs
We introduced a minor issue in #5757 that was causing the manpages to
be added every time we regenerate instead of replacing them. The
script was a bit unscrutable, and we do this block-replacement in
several places I thought it might be a good idea to have a dedicated
tool to do it.

This allows us to have simpler Makefiles whenever we update a
generated block inside another file. It's python, but does not have
dependencies :-)

Changelog-None
2022-12-06 11:39:53 +10:30
Christian Decker dbb38e2c7d docs: Add the `reckless` manpage to the readthedocs generation
Changelog-None
2022-11-30 17:03:08 +01:00
Rusty Russell 24651f57ad plugins: set non_numeric_ids flag based on getmanifest `nonnumericids` field.
And document support for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `getmanfest` response can contain `nonnumericids` to indicate support for modern string-based JSON request ids.
Changelog-Deprecated: Plugins: numeric JSON request ids: modern ones will be strings (see doc/lightningd-rpc.7.md!)
2022-11-21 11:23:54 +01:00
Rusty Russell b1f50c825f doc: document how to construct JSON ids in modern plugins and utilities.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Rusty Russell 02d7454226 db: add invoicerequests table.
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly.  Create a new table for them, and
associated functions.

The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
2022-11-09 15:08:03 +01:00
Rusty Russell bc283cecf2 decode: print unknown fields in bolt12 strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell ef2f4a0648 bolt12: use spec field names, update decode API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 1e3cb01546 bolt12: import the latest spec, update to fit.
I know this is an unforgivably large diff, but the spec has changed so
much that most of this amounts to a rewrite.

Some points:
* We no longer have "offer_id" fields, we generate that locally, as all
  offer fields are mirrored into invoice_request and then invoice.
* Because of that mirroring, field names all have explicit offer/invreq/invoice
  prefixes.
* The `refund_for` fields have been removed from spec: will re-add locally later.
* quantity_min was removed, max == 0 now mean "must specify a quantity".
* I have put recurrence fields back in locally.

This brings us to 655df03d8729c0918bdacac99eb13fdb0ee93345 ("BOLT 12:
add explicit invoice_node_id.")

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 3afa5077fe offers: make them always unsigned.
This is in preparation for the spec update where the signature field
does not even exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 5becfa6ee1 onion_message: don't use general secret, use per-message secret.
We had a scheme where lightningd itself would put a per-node secret in
the blinded path, then we'd tell the caller when it was used.  Then it
simply checks the alias to determine if the correct path was used.

But this doesn't work when we start to offer multiple blinded paths.
So go for a far simpler scheme, where the secret is generated (and
stored) by the caller, and hand it back to them.

We keep the split "with secret" or "without secret" API, since I'm
sure callers who don't care about the secret won't check that it
doesn't exist!  And without that, someone can use a blinded path for a
different message and get a response which may reveal the node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
niftynei 26f5dcd2a5 wallet: mark coinbase outputs as 'immature' until spendable
Changelog-Changed: JSON-RPC: `listfunds` now lists coinbase outputs as 'immature' until they're spendable
Changelog-Changed: JSON-RPC: UTXOs aren't spendable while immature
2022-11-09 11:55:25 +01:00
Rusty Russell ae3550cb00 lightning-cli: support --filter parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: new `--filter` parameter to reduce JSON output.
2022-11-09 20:25:58 +10:30
Rusty Russell c31fb99d2d doc: add lightingd-rpc documentation.
This documents how to communicate with lightningd over RPC, including
use of the `filter` object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Documentation: `lightningd-rpc` manual page describes details of our JSON-RPC interface, including compatibility and filtering.
2022-11-09 20:25:58 +10:30
Alex Myers a728b04243 Reckless: add man page 2022-11-08 13:19:36 +01:00
Dustin Dettmer 62e44e6bc8 doc: Move M1 instructions install to its own section
Changelog-None
2022-11-01 18:25:21 +01:00
Dread 8f549bfcca doc: add `make install` line for macOS instructions 2022-11-01 18:25:21 +01:00
Rusty Russell 1cdf21678e offers: print out more details, fix up schema for decode of blinded paths.
We need to print out first_node_id, and "node_id" is now called
"blinded_node_id" in the spec.

And the schema didn't include the payment fields in the blinded path
for invoices (which broke as soon as we actually tested one!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell e30ea91908 BOLTs: update to more recent bolt12 spec.
It's 2b7ad577d7a790b302bd1aa044b22c809c76e49d, which reverts the
point32 changes.

It also restores send_invoice in `invoice`, which we had removed
from spec and put into the recurrence patch.

I originally had implemented compatibility, but other changes
which followed this are far too widespread.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: complete rework of spec from other teams (yay!) breaks previous compatibility (boo!)
2022-10-26 11:29:06 +10:30
niftynei 85039d4f4e df: pass lease data back to funder for rbfs
let's let RBFs know about our lease info!
2022-10-20 13:42:41 +02:00
briancolecoinmetrics 74652f7cf4 move alpine build dependencies to virtual package
move alpine build dependencies to virtual package so it can be easily removed all at once
2022-10-15 13:42:58 +02:00
briancolecoinmetrics 2136d5912f update package dependencies for Alpine Linux
Alpine no longer has a `python` (2) package, which is fine because it doesn't seem to be needed. Also, the listed commands didn't result in all needed dependencies being installed for runtime, so I've added that in an additional step.
2022-10-15 13:42:58 +02:00
Rusty Russell 342e330b56 doc: update references to old BOLTs repo.
This reveals that common/test/run-bolt12_merkle-json.c was broken!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell 49fe1c8ed7 lightningd: have `makesecret` take `hex` or `string` (just like `datastore`)
Changelog-Added: JSON-RPC: `makesecret` can take a string argument instead of hex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Christian Decker 93b315756f schema: Add `enforcedelay` to `setchannel`
This was missing in the last version
2022-09-28 15:13:07 +02:00
Justin Litchfield 51e2433087 Setchannel request is provided 2022-09-28 15:13:07 +02:00
Rusty Russell f00cc23f67 sphinx: rename confusing functions, ensure valid payloads.
"sphinx_add_hop" takes a literal hop to include,
"sphinx_add_modern_hop" prepends the length.  Now we always prepend a
length, make it clear that the literal version is a shortcut:

* sphinx_add_hop -> sphinx_add_hop_has_length
* sphinx_add_modern_hop -> sphinx_add_hop

In addition, we check that length is actually correct!  This means
`createonion` can no longer create legacy or otherwise-invalid onions:
fix tests and update man page to remove legacy usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `createonion` no longer allows non-TLV-style payloads.
2022-09-28 13:40:57 +02:00
joemphilips 836a2aa261 use msat_or_all for fundpsbt request amount 2022-09-28 10:52:55 +02:00
Rusty Russell 68f15f17bb delforward: allow deletion of "unknown in_htlc_id" and fix autoclean to use it.
Note the caveats: we will delete *all* of them at once!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-27 14:42:03 +09:30
Rusty Russell cafa1a8c65 db: correctly migrate forwards for closed incoming channels.
We have to allow them (as otherwise `fees_collected_msat` in getinfo breaks),
but it means that actually, in_htlc_id might be missing in listforwards
(also, out_htlc_id might be missing, which we didn't catch before).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5628
2022-09-27 14:42:03 +09:30
Rusty Russell 6e86fa9220 lightningd: figure out optimal channel *before* forward_htlc hook.
Otherwise what the hook sees is actually a lie, and if it sets it
we might override it.

The side effect is that we add an explicit "forward_to" field, and
allow hooks to override it.  This lets a *hook* control channel
choice explicitly.

Changelod-Added: Plugins: `htlc_accepted_hook` return can specify what channel to forward htlc to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-26 13:52:04 +02:00