Commit Graph

163 Commits

Author SHA1 Message Date
Rusty Russell 932ca9e91f lightningd: deprecate p2sh-segwit addresses for `newaddr` `addresstype`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `newaddr`: `addresstype` `p2sh-segwit` (use default, or `bech32`).
2023-01-14 21:17:45 +10:30
Rusty Russell f1373fd98c doc: remove manual field descriptions from listpeerchannels(7).
Some are best copied into the schema, but some are already
out-of-date, so cleanest to remove them and rely on the generated (and
thus, checked!) fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 6c0b9b0c78 lightningd: deprecate listpeers.channels
Changelog-Deprecated: JSON-RPC: `listpeers` `channels` array: use `listpeerchannels`

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo 6fa904b4fb lightningd: add listpeerchannels command
Changelog-Added: JSON-RPC: new command `listpeerchannels` now contains information on direct channels with our peers.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Rusty Russell 717cb03f51 doc: add recent additions, fix annotation on listpeers to actually deprecate.
TODO: It would be great to similarly annotate new/deprecated commands and
their parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: doc: we now annotate what versions JSON field additions and deprecations happenened.
2023-01-11 12:51:18 +10:30
Rusty Russell b2148d0eab docs: handle "added": "version" and "deprecated": "version" from schemas.
This means we will document deprecations and additions, rather than just
pretending they've always been that way!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 3f8199bbfd doc: document that we should annotate added and deprecated schemas.
No longer just delete them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell 3f0c5b985b commando: add filtering support.
1. When we receive a commando command from a remote using the `filter`
   field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
   more efficient than using filtering locally.

Of course, older remote nodes will ignore the filter, but that's
harmless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
2023-01-11 11:13:27 +10:30
Justin Litchfield 879db694e1 Ping request types are changed from number to u16 2023-01-09 14:35:23 +01:00
Dennis Reimann 81213cafd7 doc: fix accept-htlc-tlv-types description 2023-01-09 11:45:14 +01:00
Simon Vrouwe 93c966c649 doc: sphinx-build fix external links (urls), language warning and broken footnote 2022-12-19 19:00:39 +01:00
Matt Whitlock 4d64374b85 doc: drive-by spelling corrections 2022-12-12 15:34:00 +10:30
Rusty Russell ec025344cc lightningd: don't announce names as DNS by default.
This broke BTCPayServer, so revert.  I originally (accidentally!)
implemented this such that it broadcast both DNS and IP entries, but
Michael reported earlier that they still don't propagage well, so
simply suppress them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5795
Changelog-Changeed: Config: `announce-addr-dns` needs to be set to *true* to put DNS names into node announcements, otherwise they are suppressed.
Changelog-Deprecated: Config: `announce-addr-dns` (currently defaults to `false`).  This will default to `true` once enough of the network has upgraded to understand DNS entries.
2022-12-08 11:04:53 +01:00
Rusty Russell 4613b29815 doc: document autoclean-once command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-06 10:42:05 +01:00
Rusty Russell 02d7454226 db: add invoicerequests table.
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly.  Create a new table for them, and
associated functions.

The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
2022-11-09 15:08:03 +01:00
Rusty Russell bc283cecf2 decode: print unknown fields in bolt12 strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell ef2f4a0648 bolt12: use spec field names, update decode API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 1e3cb01546 bolt12: import the latest spec, update to fit.
I know this is an unforgivably large diff, but the spec has changed so
much that most of this amounts to a rewrite.

Some points:
* We no longer have "offer_id" fields, we generate that locally, as all
  offer fields are mirrored into invoice_request and then invoice.
* Because of that mirroring, field names all have explicit offer/invreq/invoice
  prefixes.
* The `refund_for` fields have been removed from spec: will re-add locally later.
* quantity_min was removed, max == 0 now mean "must specify a quantity".
* I have put recurrence fields back in locally.

This brings us to 655df03d8729c0918bdacac99eb13fdb0ee93345 ("BOLT 12:
add explicit invoice_node_id.")

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell 3afa5077fe offers: make them always unsigned.
This is in preparation for the spec update where the signature field
does not even exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
niftynei 26f5dcd2a5 wallet: mark coinbase outputs as 'immature' until spendable
Changelog-Changed: JSON-RPC: `listfunds` now lists coinbase outputs as 'immature' until they're spendable
Changelog-Changed: JSON-RPC: UTXOs aren't spendable while immature
2022-11-09 11:55:25 +01:00
Rusty Russell 1cdf21678e offers: print out more details, fix up schema for decode of blinded paths.
We need to print out first_node_id, and "node_id" is now called
"blinded_node_id" in the spec.

And the schema didn't include the payment fields in the blinded path
for invoices (which broke as soon as we actually tested one!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell e30ea91908 BOLTs: update to more recent bolt12 spec.
It's 2b7ad577d7a790b302bd1aa044b22c809c76e49d, which reverts the
point32 changes.

It also restores send_invoice in `invoice`, which we had removed
from spec and put into the recurrence patch.

I originally had implemented compatibility, but other changes
which followed this are far too widespread.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: complete rework of spec from other teams (yay!) breaks previous compatibility (boo!)
2022-10-26 11:29:06 +10:30
Rusty Russell 49fe1c8ed7 lightningd: have `makesecret` take `hex` or `string` (just like `datastore`)
Changelog-Added: JSON-RPC: `makesecret` can take a string argument instead of hex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Christian Decker 93b315756f schema: Add `enforcedelay` to `setchannel`
This was missing in the last version
2022-09-28 15:13:07 +02:00
Justin Litchfield 51e2433087 Setchannel request is provided 2022-09-28 15:13:07 +02:00
joemphilips 836a2aa261 use msat_or_all for fundpsbt request amount 2022-09-28 10:52:55 +02:00
Rusty Russell cafa1a8c65 db: correctly migrate forwards for closed incoming channels.
We have to allow them (as otherwise `fees_collected_msat` in getinfo breaks),
but it means that actually, in_htlc_id might be missing in listforwards
(also, out_htlc_id might be missing, which we didn't catch before).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5628
2022-09-27 14:42:03 +09:30
Rusty Russell f52ff07558 lightningd: allow delpay to delete a specific payment.
This is actually what the autoclean plugin wants, especially since
you can't otherwise delete a payment which has failed then succeeded.

But insist on neither or both being specified, at least for now.

Changelog-Added: JSON-RPC: `delpay` takes optional `groupid` and `partid` parameters to specify exactly what payment to delete.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell 05095313f5 lightningd: listsendpays always has groupid.
Schema was too loose since we did deprecations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell fa7d732ba6 lightningd: allow a connection to specify db batching.
Previous commit was a hack which *always* batched where possible, this
is a more sophisticated opt-in varaint, with a timeout sanity check.

Final performance for cleaning up 1M pays/forwards/invoices:

```
$ time l1-cli autoclean-once succeededpays 1
{
   "autoclean": {
      "succeededpays": {
         "cleaned": 1000000,
         "uncleaned": 26895
      }
   }
}

real	6m9.828s
user	0m0.003s
sys	0m0.001s
$ time l2-cli autoclean-once succeededforwards 1
{
   "autoclean": {
      "succeededforwards": {
         "cleaned": 1000000,
         "uncleaned": 40
      }
   }
}

real	3m20.789s
user	0m0.004s
sys	0m0.001s
$ time l3-cli autoclean-once paidinvoices 1
{
   "autoclean": {
      "paidinvoices": {
         "cleaned": 1000000,
         "uncleaned": 0
      }
   }
}

real	6m47.941s
user	0m0.001s
sys	0m0.000s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `batching` command to allow database transactions to cross multiple back-to-back JSON commands.
2022-09-22 15:19:46 +02:00
Rusty Russell 612f3de0d4 doc: manpages and schemas for autoclean-status.
And remove deprecated autocleaninvoice docs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `autoclean-status` command to see what autoclean is doing.
Changelog-Deprecated: JSON-RPC: `autocleaninvoice` (use option `autoclean-expiredinvoices-age`)
2022-09-22 15:19:46 +02:00
Rusty Russell 3079afb024 lightningd: add `delforward` command.
Changelog-Added: JSON-RPC: `delforward` command to delete listforwards entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell 7420a7021f lightningd: add `listhtlcs` to list all the HTLCs we know about.
Using `listfowards` for this wrong; expose this directly if people
care (and unlike listforwards, which could be deleted, we have to
remember these while the channel is still open!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listhtlcs` new command to list all known HTLCS.
2022-09-22 15:19:46 +02:00
Rusty Russell 311807ff1f lightningd: add in_htlc_id / out_htlc_id to listforwards.
And document that we never know payment_hash.

Changelog-Added: JSON-RPC: `listforwards` now shows `in_htlc_id` and `out_htlc_id`
Changelog-Changed: JSON-RPC: `listforwards` now never shows `payment_hash`; use `listhtlcs`.
2022-09-22 15:19:46 +02:00
Kristaps Kaupe 775d6ba193 doc: Fix wrong_funding description in manpage and type in schema 2022-09-22 10:41:05 +02:00
Rusty Russell 0868fa9f1e lightningd: allow extra tlv types in non-experimental mode.
The old `experimental-accept-extra-tlv-types` is now `accept-htlc-tlv-types`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `accept-htlc-tlv-types` lets us accept unknown even HTLC TLV fields we would normally reject on parsing (was EXPERIMENTAL-only `experimental-accept-extra-tlv-types`).
2022-09-22 17:19:11 +09:30
Christian Decker 54b4baabb0 opening: Add `dev-allowdustreserve` option to opt into dust reserves
Technically this is a non-conformance with the spec, hence the `dev`
flag to opt-in, however I'm being told that it is also implemented in
other implementations. I'll follow this up with a proposal to the spec
to remove the checks we now bypass.
2022-09-21 11:25:47 +02:00
Christian Decker 5c1de8029a openingd: Add `reserve` to `fundchannel` and `multifundchannel`
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel` and `fundchannel_start` now accept a `reserve` parameter to indicate the absolute reserve to impose on the peer.
2022-09-21 11:25:47 +02:00
Rusty Russell 4ca6b36439 lightningd: refuse to upgrade db on non-released versions by default.
This is a good sanity check that users understand that if they upgrade
to master mid-cycle they can't go back!

Suggested-by: @wtogami
Changelog-Added: Config: `--database-upgrade=true` required if a non-release version wants to (irrevocably!) upgrade the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-15 13:25:58 +09:30
Rusty Russell a6d4756d08 commando: make rune alternatives a JSON array.
This avoids having to escape | or &, though we still allow that for
the deprecation period.

To detect deprecated usage, we insist that alternatives are *always*
an array (which could be loosened later), but that also means that
restrictions must *always* be an array for now.

Before:

```
# invoice, description either A or B
lightning-cli commando-rune '["method=invoice","pnamedescription=A|pnamedescription=B"]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '["method=invoice","pnamedescription=A\\|B"]'
```

After:

```
# invoice, description either A or B
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A", "pnamedescription=B"]]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A|B"]]'
```

Changelog-Deprecated: JSON-RPC: `commando-rune` restrictions is always an array, each element an array of alternatives.  Replaces a string with `|`-separators, so no escaping necessary except for `\\`.
2022-09-14 17:46:43 +02:00
Christian Decker cb3ee0ac2e wallet: Load and value `completed_at` timestamp from DB 2022-09-14 13:14:10 +02:00
Rusty Russell 6438ee4126 doc: disallow additional properties in sendcustommsg.
We have to scatter this everywhere in our schemas, as there's no way
to make it the default :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
greggzigler ce9232af37 Changelog-None 2022-08-22 11:21:19 -04:00
Rusty Russell 25b4249f54 doc: fix decode schema for bolt11 routehints.
I decode a routehint in the next patch, and it barfed:

```
>       assert only_one(l1.rpc.decode(inv['bolt11'])['routes'])['short_channel_id'] == alias23

tests/test_opening.py:1515: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:321: in wrapper
    return self.call(name, payload=args)
contrib/pyln-testing/pyln/testing/utils.py:691: in call
    schemas[1].validate(res)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Validator(schema={'$schema': 'http://json-...ft-07/schema#', 'allOf': [{'if': {'properties': {'type': {'enum': [...], ...iption': 'if this is f... diagnostics!', 'type': 'boolean'}}, 'required': ['type', 'valid'], ...}, format_checker=None)
args = ({'amount_msat': 10msat, 'created_at': 1659923931, 'currency': 'bcrt', 'description': 'desc', ...},), kwargs = {}
error = <ValidationError: "1msat is not of type 'u32'">

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           jsonschema.exceptions.ValidationError: 1msat is not of type 'u32'
E           
E           Failed validating 'type' in schema['allOf'][6]['then']['properties']['routes']['items']['items']['properties']['fee_base_msat']:
E               {'description': 'the base fee for payments', 'type': 'u32'}
E           
E           On instance['routes'][0][0]['fee_base_msat']:
E               1msat

```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:31:36 -05:00
niftynei 3fcf60ab7c bkpr: track channel rebalances, display in listincome
Track rebalances, and report income events for them.

Previously `listincome` would report:
	- invoice event, debit, outgoing channel
	- invoice_fee event, debit, outgoing channel
	- invoice event, credit, inbound channel

Now reports:
	- rebalance_fee, debit, outgoing channel
	(same value as invoice_fee above)

Note: only applies on channel events; if a rebalance falls to chain
we'll use the older style of accounting.

Changelog-None
2022-08-09 11:57:18 +09:30
niftynei 4e503f7d0a bkpr/listpeeers: add lease_fees back to funds; separate out in listpeers
First, how we record "our_funds" and then apply pushes vs lease_fees
(for liquidity ad buys/sales) was exactly opposite.

For pushes we were reporting the total funded into the channel, with the
push representing how much we'd later moved to the peer.

For lease_fees we were rerporting the total in the channel, with the
push representing how much was already moved to the peer.

We fix this (from a view perspective) by re-adding lease fees to what's
reported in the channel funding totals. Since this is now new behavior
(for leased channel values), we added new fields so we can take the old
field names thru a deprecation cycle.

We also make it possible to differentiate btw a push and a lease_fee
(before they were all the same), by adding to new fields to `listpeers`:
`fee_paid_msat` and `fee_rcvd_msat`.

This allows us to avoid math in the bookkeeper, instead we just pick
the numbers out directly and record them.

Fixes #5472

Changelog-Added: JSON-RPC: `listpeers` now has a few new fields for `funding` (`remote_funds_msat`, `local_funds_msat`, `fee_paid_msat`, `fee_rcvd_msat`).
Changelog-Deprecated: JSON-RPC: `listpeers`.`funded` fields `local_msat` and `remote_msat` are now deprecated.
2022-07-31 21:53:05 +09:30
William Casarin 10d66c25c4 commando-rune: show warning when creating runes with no restrictions
Changelog-Added: Show warning when creating runes with no restrictions
Signed-off-by: William Casarin <jb55@jb55.com>
2022-07-29 08:46:14 -07:00
niftynei 71c03bc082 bkpr: Add an option to set the database to something else (postgres)
`lightningd` has an option --wallet that lets you supply a database dsn
string to connect to a sqlite3/postgres database that's hosted/stored
elsewhere.

This adds the `--bookkeeper-db` option which does the same, except for
the bookkeeping data for a node!

Note that the default is to go in the `lightning-dir` in a database
called `accounts.sqlite3`
2022-07-28 12:08:18 +09:30
niftynei 352b419755 bkpr: save invoice description data to the database and display it
It'll be really nice to be able to read description data about an
invoice, if we've got it!
2022-07-28 12:08:18 +09:30
niftynei 563910e667 bkpr: add docs, change names to 'bkpr-*'
Adds schema definitions and manpages for bkpr- commands; also renames
the commands to all start with 'bkpr-', so they're easier to identify/
make runes about.
2022-07-28 12:08:18 +09:30