Commit Graph

932 Commits

Author SHA1 Message Date
Rusty Russell 0145e8ecec
Makefile: Revert ba7d4a8f6b (make-schema: don't include tools/fromschema.py in SHASUMS)
1. If the tool changes, you need to regenerate since the output may
   change.

2. This didn't just filter that out, ignored all but the first
   dependency, which made bisecting the bookkeeper plugin a nightmare:
   it didn't regenerate the .po file, causing random crashes.

If we want this, try $(filter-out tools/fromschema.py) instead.  But I
don't think we want that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-13 12:29:46 -05:00
Rusty Russell 9ce6b6ce85
doc: generate correct markdown from schemas.
You can't start a list without a paragraph separator.

```diff
 --- /tmp/before	2022-07-20 22:02:23.485372596 +0930
 +++ /tmp/after	2022-07-20 22:02:33.745528456 +0930
 @@ -21,12 +21,16 @@
         On startup of the daemon, no autoclean is set up.
  
  RETURN VALUE
 -       On success, an object is returned,  containing:  -  enabled  (boolean):
 -       whether invoice autocleaning is active
 +       On success, an object is returned, containing:
  
 -       If enabled is true: - expired_by (u64): how long an invoice must be ex‐
 -       pired (seconds) before we delete it - cycle_seconds (u64): how long  an
 -       invoice must be expired (seconds) before we delete it
 +       • enabled (boolean): whether invoice autocleaning is active
 +
 +       If enabled is true:
 +
 +       • expired_by (u64): how long an invoice must be expired  (seconds)  be‐
 +         fore we delete it
 +       • cycle_seconds  (u64):  how  long an invoice must be expired (seconds)
 +         before we delete it
  
  AUTHOR
         ZmnSCPxj <ZmnSCPxj@protonmail.com> is mainly responsible.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-13 12:29:32 -05:00
Rusty Russell 237d16fdbf
doc: format markdown correctly.
There are no definition lists in Markdown, and lists get mangled if
they follow immediately: they need a line between them.

So use bullets for options, and use an indent so the text gets in the
line below.

Here's a before-and-after example:

```diff
 --- /tmp/after	2022-07-20 21:55:54.355487769 +0930
 +++ /tmp/after2	2022-07-20 21:58:17.305642576 +0930
 @@ -10,38 +10,71 @@
         lightning-cli sends commands to the lightning daemon.

  OPTIONS
 -       --lightning-dir=DIR Set the directory for the lightning daemon we're talking to; defaults to $HOME/.lightning.
 +       • --lightning-dir=DIR

 -       --conf=PATH Sets configuration file (default: lightning-dir/config ).
 +         Set the directory for the lightning daemon we're talking to; defaults to $HOME/.lightning.

 -       --network=network --mainnet --testnet --signet Sets network explicitly.
 +       • --conf=PATH

 -       --rpc-file=FILE Named pipe to use to talk to lightning daemon: default is lightning-rpc in the lightning directory.
 +         Sets configuration file (default: lightning-dir/config ).

 -       --keywords/-k Use format key=value for parameters in any order
 +       • --network=network

 -       --order/-o Follow strictly the order of parameters for the command
 +       • --mainnet

 -       --json/-J Return result in JSON format (default unless help command, or result contains a format-hint field).
 +       • --testnet

 -       --raw/-R Return raw JSON directly as lightningd replies; this can be faster for large requests.
 +       • --signet

 -       --human-readable/-H Return result in human-readable output.
 +         Sets network explicitly.

 -       --flat/-F  Return  JSON result in flattened one-per-line output, e.g. { "help": [ { "command": "check" } ] } would become
 +       • --rpc-file=FILE
 +
 +         Named pipe to use to talk to lightning daemon: default is lightning-rpc in the lightning directory.
 +
 +       • --keywords/-k
 +
 +         Use format key=value for parameters in any order
 +
 +       • --order/-o
 +
 +         Follow strictly the order of parameters for the command
 +
 +       • --json/-J
 +
 +         Return result in JSON format (default unless help command, or result contains a format-hint field).
 +
 +       • --raw/-R
 +
 +         Return raw JSON directly as lightningd replies; this can be faster for large requests.
 +
 +       • --human-readable/-H
 +
 +         Return result in human-readable output.
 +
 +       • --flat/-F
 +
 +         Return  JSON  result  in  flattened  one-per-line  output,  e.g.  {  "help":  [ { "command": "check" } ] } would become
         help[0].command=check.  This is useful for simple scripts which want to find a  specific  output  field  without  parsing
         JSON.

 -       --notifications/-N=LEVEL  If  LEVEL is 'none', then never print out notifications.  Otherwise, print out notifications of
 -       LEVEL or above (one of io, debug, info (the default), unusual or broken: they are prefixed with # .
 +       • --notifications/-N=LEVEL
 +
 +         If  LEVEL  is 'none', then never print out notifications.  Otherwise, print out notifications of LEVEL or above (one of
 +         io, debug, info (the default), unusual or broken: they are prefixed with # .
 +
 +       • --help/-h
 +
 +         Pretty-print summary of options to standard output and exit.  The format can be changed using -F, -R, -J, -H etc.
 +
 +       • --version/-V

 -       --help/-h Pretty-print summary of options to standard output and exit.  The format can be changed using -F,  -R,  -J,  -H
 -       etc.
 +         Print version number to standard output and exit.

 -       --version/-V Print version number to standard output and exit.
 +       • allow-deprecated-apis=BOOL

 -       allow-deprecated-apis=BOOL Enable deprecated options. It defaults to true, but you should set it to false when testing to
 -       ensure that an upgrade won't break your configuration.
 +         Enable deprecated options. It defaults to true, but you should set it to false when testing to ensure that  an  upgrade
 +         won't break your configuration.

  COMMANDS
         lightning-cli simply uses the JSON RPC interface to talk to lightningd, and prints the results. Thus the commands  avail‐
 @@ -60,13 +93,13 @@
         this is not encouraged.

  EXAMPLES
 -       1.     List commands
 +       1.     List commands:

 -       lightning-cli help
 +              • lightning-cli help

 -       2.     Fund a 10k sat channel using uncomfirmed outputs
 +       2.     Fund a 10k sat channel using uncomfirmed outputs:

 -       lightning-cli --keywords fundchannel id=028f...ae7d amount=10000sat minconf=0
 +              • lightning-cli --keywords fundchannel id=028f...ae7d amount=10000sat minconf=0

  BUGS
         This manpage documents how it should work, not how it does work. The pretty printing of results isn't pretty.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-13 12:29:20 -05:00
Rusty Russell 76b57a221d
Makefile: replace mrkd with lowdown(1).
Here's the before-vs-after comparison (ignoring whitespace changes):

```diff
 --- /tmp/before	2022-07-20 21:52:44.336641810 +0930
 +++ /tmp/after	2022-07-20 21:55:54.355487769 +0930
 @@ -1,7 +1,7 @@
 -LIGHTNING-CLI(1)                                          lightning-cli                                         LIGHTNING-CLI(1)
 +LIGHTNING-CLI(1)                                                                                                LIGHTNING-CLI(1)
  
  NAME
 -       lightning-cli - Control lightning daemon
 +       lightning-cli -- Control lightning daemon
  
  SYNOPSIS
         lightning-cli [OPTIONS] command
 @@ -14,10 +14,7 @@
  
          --conf=PATH Sets configuration file (default: lightning-dir/config ).
  
 -        --network=network
 -        --mainnet
 -        --testnet
 -        --signet Sets network explicitly.
 +       --network=network --mainnet --testnet --signet Sets network explicitly.
  
          --rpc-file=FILE Named pipe to use to talk to lightning daemon: default is lightning-rpc in the lightning directory.
  
 @@ -43,8 +40,8 @@
  
          --version/-V Print version number to standard output and exit.
  
 -        allow-deprecated-apis=BOOL  Enable  deprecated options. It defaults to true, but you should set it to false when testing
 -       to ensure that an upgrade won't break your configuration.
 +       allow-deprecated-apis=BOOL Enable deprecated options. It defaults to true, but you should set it to false when testing to
 +       ensure that an upgrade won't break your configuration.
  
  COMMANDS
         lightning-cli simply uses the JSON RPC interface to talk to lightningd, and prints the results. Thus the commands  avail‐
 @@ -67,7 +64,7 @@
  
         lightning-cli help
  
 -              1.     Fund a 10k sat channel using uncomfirmed outputs
 +       2.     Fund a 10k sat channel using uncomfirmed outputs
  
         lightning-cli --keywords fundchannel id=028f...ae7d amount=10000sat minconf=0
  
 @@ -84,4 +81,4 @@
         Note: the modules in the ccan/ directory have their own licenses, but the rest of the code is covered  by  the  BSD-style
         MIT license.
  
 -                                                                                                                LIGHTNING-CLI(1)
 +Core Lightning v0.11.0.1-350-gac2e137                                                                           LIGHTNING-CLI(1)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5437
2022-09-13 12:29:06 -05:00
Orb fd8b667e6a use 'postgres' driver (not postgresql). Include plain url to guide, as link is broken on RTD. 2022-08-22 11:21:59 -04:00
greggzigler ce9232af37 Changelog-None 2022-08-22 11:21:19 -04:00
greggzigler c13dffe980 fix MD link where text and url are swapped 2022-08-22 11:21:19 -04:00
adi2011 0b737fd7e5 doc: Typo 2022-08-19 10:27:09 -04:00
Rusty Russell 9219e778a1 doc: update usage of lightning-invoice.
The new parameter name (this version) is amount_msat: msatoshi is
deprecated.   The doc/schemas/invoice.request.json has this correct
(but we don't generate teh *usage* line from that yet!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-18 15:53:14 -04:00
Rusty Russell 1ff8e1bacb doc: note that setchannel maxhtlc/minhtlc only apply to *forwards*.
Reported-by: Warren Togami
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:38:57 -05:00
Rusty Russell fcba09dc33 doc: document that sendonion doesn't have to specify (but can!) the first_hop channel.
This was always the case, though it was ignored.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:33:24 -05:00
Rusty Russell cfe6b06fb5 lightnind: use aliases in routehints for private channels.
We *should* remember the channel type, since this is only required
if they set the channel_type to include option_scid_alias.

However, since we support channel upgrade, channel_type really needs
a new table.  I have a patch for that, from my abandoned original
"fastopen" branch for aliases, but it's too big a chance for rc2 IMHO.

Meanwhile, we allow exposeprivatechannels's scids to be either real or
the aliases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: invoice routehints will use fake short-channel-ids for private channels if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00
Rusty Russell 25b4249f54 doc: fix decode schema for bolt11 routehints.
I decode a routehint in the next patch, and it barfed:

```
>       assert only_one(l1.rpc.decode(inv['bolt11'])['routes'])['short_channel_id'] == alias23

tests/test_opening.py:1515: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:321: in wrapper
    return self.call(name, payload=args)
contrib/pyln-testing/pyln/testing/utils.py:691: in call
    schemas[1].validate(res)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Validator(schema={'$schema': 'http://json-...ft-07/schema#', 'allOf': [{'if': {'properties': {'type': {'enum': [...], ...iption': 'if this is f... diagnostics!', 'type': 'boolean'}}, 'required': ['type', 'valid'], ...}, format_checker=None)
args = ({'amount_msat': 10msat, 'created_at': 1659923931, 'currency': 'bcrt', 'description': 'desc', ...},), kwargs = {}
error = <ValidationError: "1msat is not of type 'u32'">

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           jsonschema.exceptions.ValidationError: 1msat is not of type 'u32'
E           
E           Failed validating 'type' in schema['allOf'][6]['then']['properties']['routes']['items']['items']['properties']['fee_base_msat']:
E               {'description': 'the base fee for payments', 'type': 'u32'}
E           
E           On instance['routes'][0][0]['fee_base_msat']:
E               1msat

```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:31:36 -05:00
niftynei 3fcf60ab7c bkpr: track channel rebalances, display in listincome
Track rebalances, and report income events for them.

Previously `listincome` would report:
	- invoice event, debit, outgoing channel
	- invoice_fee event, debit, outgoing channel
	- invoice event, credit, inbound channel

Now reports:
	- rebalance_fee, debit, outgoing channel
	(same value as invoice_fee above)

Note: only applies on channel events; if a rebalance falls to chain
we'll use the older style of accounting.

Changelog-None
2022-08-09 11:57:18 +09:30
Vincenzo Palazzo ffb3217ea8 docs: clarify the different way to build cln
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-08-04 11:28:00 -05:00
Seth For Privacy 498f9b75f2 Correct basics of backup plugin usage 2022-08-03 21:22:41 -05:00
niftynei 4e503f7d0a bkpr/listpeeers: add lease_fees back to funds; separate out in listpeers
First, how we record "our_funds" and then apply pushes vs lease_fees
(for liquidity ad buys/sales) was exactly opposite.

For pushes we were reporting the total funded into the channel, with the
push representing how much we'd later moved to the peer.

For lease_fees we were rerporting the total in the channel, with the
push representing how much was already moved to the peer.

We fix this (from a view perspective) by re-adding lease fees to what's
reported in the channel funding totals. Since this is now new behavior
(for leased channel values), we added new fields so we can take the old
field names thru a deprecation cycle.

We also make it possible to differentiate btw a push and a lease_fee
(before they were all the same), by adding to new fields to `listpeers`:
`fee_paid_msat` and `fee_rcvd_msat`.

This allows us to avoid math in the bookkeeper, instead we just pick
the numbers out directly and record them.

Fixes #5472

Changelog-Added: JSON-RPC: `listpeers` now has a few new fields for `funding` (`remote_funds_msat`, `local_funds_msat`, `fee_paid_msat`, `fee_rcvd_msat`).
Changelog-Deprecated: JSON-RPC: `listpeers`.`funded` fields `local_msat` and `remote_msat` are now deprecated.
2022-07-31 21:53:05 +09:30
William Casarin 10d66c25c4 commando-rune: show warning when creating runes with no restrictions
Changelog-Added: Show warning when creating runes with no restrictions
Signed-off-by: William Casarin <jb55@jb55.com>
2022-07-29 08:46:14 -07:00
CC 78804d9ea8 fix doc: deschashonly 2022-07-29 16:33:06 +09:30
Rusty Russell 1b5dc4409a doc: remove two more generated manpages.
Somehow these two were missed in 79e09b92ef.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-28 12:08:18 +09:30
Rusty Russell 6a22411f7e doc: note that bookkeeper-dir and bookkeeper-db are in bookkeeper plugin.
We do this for bcli options, too.  Good to note if people disable/replace the
plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-28 12:08:18 +09:30
niftynei 71c03bc082 bkpr: Add an option to set the database to something else (postgres)
`lightningd` has an option --wallet that lets you supply a database dsn
string to connect to a sqlite3/postgres database that's hosted/stored
elsewhere.

This adds the `--bookkeeper-db` option which does the same, except for
the bookkeeping data for a node!

Note that the default is to go in the `lightning-dir` in a database
called `accounts.sqlite3`
2022-07-28 12:08:18 +09:30
niftynei 352b419755 bkpr: save invoice description data to the database and display it
It'll be really nice to be able to read description data about an
invoice, if we've got it!
2022-07-28 12:08:18 +09:30
niftynei 563910e667 bkpr: add docs, change names to 'bkpr-*'
Adds schema definitions and manpages for bkpr- commands; also renames
the commands to all start with 'bkpr-', so they're easier to identify/
make runes about.
2022-07-28 12:08:18 +09:30
niftynei c05900c676 bkpr: add option --bookkeeper-dir
Allow setting custom directory for bookkeeper's database
and default directory for any csv file creation
2022-07-28 12:08:18 +09:30
Rusty Russell 75c89f0b8e doc: fix bolt 12 link (it's not in master), update bolt 11 to new "bolts" repo.
It's weird that readthedocs.io drops the ".md" suffix though;
let's see if this fixes it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-27 14:51:27 -07:00
Rusty Russell da4e33cd0d decode: fix crash when decoding invalid rune.
If rune contains invalid UTF-8, offers (which implements decode) would
produce JSON with invalid UTF-8, which causes lightningd to complain
and kill it, and then die because it's an important plugin.

So don't decode invalid UTF-8!

Reported-by: @jb55
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-25 15:14:01 -07:00
Rusty Russell 8c38302ab8 hsmtool: implement checkhsm.
This gives a nice way to ensure your secret is the correct one.

Also, we don't need to suppress VALGRIND for this test, now the output
races are fixed.

Changelog-Added: `hsmtool`: new command `checkhsm` to check BIP39 passphrase against hsm_secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-22 16:57:27 +02:00
Christian Decker b3fde87063 doc: Spell out the reprobuild instructions for each distro
We also had to switch around the directories, so now the instructions
assume you're in the repo root directory
2022-07-21 15:21:12 -05:00
Christian Decker 217ce4c03c schema: Add missing `mindepth` argument to `fundchannel`
This was likely missed during the zeroconf PR.

Changelog-None
2022-07-21 14:19:06 +09:30
Christian Decker ca8c46c286 schema: `minconf` should be an integer (u32) not a float (number) 2022-07-21 14:19:06 +09:30
Christian Decker 18a9eb2feb msggen: Add `stop` method to generators
We'll need this when testing the grpc interface in pyln-testing,
otherwise tests just slowly die and wither.
2022-07-21 14:19:06 +09:30
Christian Decker 77f5eb556b msggen: Add fundchannel request 2022-07-21 14:19:06 +09:30
Rusty Russell 5979a7778f lightningd: expand exit codes for various failures.
Most unexpected ones are still 1, but there are a few recognizable error codes
worth documenting.

Rename the HSM ones to put ERRCODE_ at the front, since we have non-HSM ones
too now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-20 19:28:33 +09:30
Vincenzo Palazzo ba4e870a1c test: disable schema check of `checkmessage` with deprecated API
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-07-19 17:55:31 +02:00
Vincenzo Palazzo 7ae616ef60 rpc: improve error format
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-07-19 17:55:31 +02:00
Vincenzo Palazzo 1d671a2380 rpc: checkmessage return an error if pubkey is not found
Returning an warning message when the pub key is not specified and there is no node in the graph.

We try to help people that use core lightning as a signer and nothings else.

Changelog-Deprecated: rpc: checkmessage return an error when the pubkey is not specified and it is unknown in the network graph.
2022-07-19 17:55:31 +02:00
Rusty Russell eff53495db lightningd: make "is peer connected" a tristate.
First, connectd tells us the peer has connected, and we call the connected hook,
and if it says it's fine, we are actually connected and we fire off notifications.

Of course, we could be disconnected while in the connected hook, and that would
mean we tell people about a connection which is no longer current.

Make this clear with a tristate: if we're not marked disconnected by
the time the hooks finish, we're good.  It also gives us a cleaner
"connect" command return when we connected but disconnected before
processing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
grubles af2b863b4a Add instructions for checking out a release tag 2022-07-17 11:45:34 +09:30
zero fee routing 58ae885f48 fix typo in commando documentation 2022-07-17 08:51:02 +09:30
Rusty Russell 8c48eda8c7 decode: support decoding runes.
This is a bit weird since it lives in the offers plugin, but it works
well.  This should make runes much more approachable for people!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell 468dff1723 commando: add rate for maximum successful rune use per minute.
I'm assuming that nobody wants a rate slower than 1 per minute; we can
introduce 'drate' if we want a per-day kind of limit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell 4ab09f7cfb commando: add support for parameters by array, parameter count.
Awkward to filter, but they're really practical for many commands.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell cf28cff398 doc: document commando and commando-rune.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell 9685c1adaf lightningd: remove getsharedsecret.
This was introduced to allow creating a shared secret, but it's better to use
makesecret which creates unique secrets.  getsharedsecret being a generic ECDH
function allows the caller to initiate conversations as if it was us; this
is generally OK, since we don't allow untrusted API access, but the commando
plugin had to blacklist this for read-only runes explicitly.

Since @ZmnSCPxj never ended up using this after introducing it, simply
remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSONRPC: `getsharedsecret` API: use `makesecret`
2022-07-15 22:17:58 +09:30
Rusty Russell c34a0a22ad makesecret: change info_hex arg to simply "hex" to match datastore command.
And fix schema: it wasn't tested as there was no test-by-parameter-name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 22:17:58 +09:30
adi2011 829fe09c13 doc: Add documentation for new RPCs and a FIXME: in fromschema.py 2022-07-14 12:24:48 -05:00
niftynei ba7d4a8f6b make-schema: don't include tools/fromschema.py in SHASUMS
We were including the entire list of prerequisites when generating a
shastamp, which for schemas includes the `tools/fromschema.py` doc.

This meant all of our shasums were updating anytime this tool file
changed.

Instead, we just include the first prerequisite.

See: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
2022-07-14 12:24:48 -05:00
Michael Schmoock b3177e945f doc: mention ip discovery only active when no addresses are announced
Changelog-None
2022-07-12 13:59:46 -05:00
Rusty Russell 312751075c lightningd: save outgoing information for more forwards.
There's one case where we can present more infomation, so do that, and
fix documentation.

Changelog-Added: JSON-RPC: `listforwards` now shows `out_channel` in more cases: even if it couldn't actually send to it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5329
2022-07-12 06:38:11 +09:30