Commit Graph

11684 Commits

Author SHA1 Message Date
Rusty Russell 56dde2cb77 lightningd: multiple log-file options allow more than one log output.
I've wanted this for a while: the ability to log to multiple places
at once.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: lightningd: `log-file` option specified multiple times opens multiple log files.
2022-06-27 17:21:35 +09:30
Christian Decker 92f10f2c34 pyln: Fix relative path dependencies when publishing to PyPI
So this was quite a journey:

 - We want relative depdendencies (using the `path` argument) whenever
   developing locally. Otherwise we would have to install each
   dependency every time we change a single character, which
   undoubtedly would cause us to waste time trying to debug an issue
   just because we forgot to install.
 - When publishing however we want to rely on the version number,
   since the repo context gets lost upon publishing, and path
   dependencies cause failures.

The solution then it seems is to use `dev-dependencies` (not that
surprising once you find it) with relative paths, so that `poetry
install` uses these over the normal dependencies (no idea how they
dedup them) and use `dependencies` when publishing. The paths are
still in there when publishing, but `pip install` ignores them.

I checked that `poetry install` from an unrelated project doesn't
accidentally use the path dependencies, even when adding them as
dev-dependencies. This should hopefully also allow installing them
as a repo link, though I can't test that right now.
2022-06-26 13:54:01 +09:30
Rusty Russell a196c77fe6 common: downgrade LND 'internal error' properly.
Thanks to @zerofeerouting for another report.

"desc" here is the sanitized message, eg:
"ERROR error channel 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef: internal error"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-26 13:52:14 +09:30
Rusty Russell e48c0dda85 common: downgrade "internal error" errors from lnd.
Prior to 0.11.0 we had cases where we would treat errors
as warnings: regretfully, this is still needed.  This message
in particular has been widely reported, and it now causes
channel force closes.

Downgrade and log.  I did insert some snarky log message earlier,
but hey, I'm sure CLN has done worse things to our peers!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: treat LND "internal error" as warnings, not force close events (as we did in v0.10).
2022-06-26 13:52:14 +09:30
Christian Decker 5444a843b6 git: Ignore some more generated files and the cln-grpc plugin 2022-06-24 16:27:13 +02:00
Vincenzo Palazzo d4bc4f6460 signmessage: improve the UX of the rpc command when zbase is not a valid one
Changelog-Fixed: signmessage: improve the UX of the rpc command when zbase is not a valid one

Stacktrace generated with a bad `zbase`

```
lightningd: lightningd/signmessage.c:59: from_zbase32: Assertion `len == tal_bytelen(u8arr)' failed
lightningd: FATAL SIGNAL 6 (version v0.11.1)
0x55b9b1b4e617 send_backtrace
[...]
```

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-06-24 14:02:04 +09:30
Dustin Dettmer 49c6459148 Update hsm error formatting.
Changelog-None
2022-06-24 13:57:17 +09:30
Rusty Russell 0b7897302e common/gossip_store: optimize case where entries are filtered out.
@whitslack complained of large CPU usage by connectd at startup;
I ran perf record on connectd on my machine (which sees a little spike, only)
and I see the cost of reading and discarding the entries:

```
-   95.52%     5.24%  lightning_conne  lightning_connectd  [.] gossip_store_next
   - 90.28% gossip_store_next
      + 40.27% tal_alloc_arr_
      + 22.78% tal_free
      + 11.74% crc32c
      + 9.32% fromwire_peektype
      + 4.10% __libc_pread64 (inlined)
        1.70% be32_to_cpu
```

Much of this is caused by the search for our own gossip: keeping this separately
would be even better, but this fix is minimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: connectd: reduce initial CPU load when connecting to peers.
2022-06-24 10:33:04 +09:30
Rusty Russell 37403e471c pytest: add (failing) test for db with old Tor v2 address.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-24 10:24:39 +09:30
Rusty Russell a0c6447611 lightningd: remove overzealous assertion.
This is being hit: it's possible if connectd and lightningd get desynchronized,
and we'll handle this later when peer is activated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-24 10:16:18 +09:30
Justin Moon acc78397eb Remove native-tls dependency from cln-rpc 2022-06-23 15:45:06 +02:00
Jan Sarenik bb53a17855 doc: Tor needs CookieAuthFile set
The message was

    [warn] CookieAuthFileGroupReadable is set, but will have no effect: you must specify an explicit CookieAuthFile to have it group-readable.

Changelog-None
2022-06-23 09:40:47 +02:00
Christian Decker e6442d798e cln-plugin: Make the proxy-related configuration Option<>
These are only populated if a proxy was specified, see
lightningd/plugin.c:1855, so we were getting upset when we expected
them and they weren't set.
2022-06-22 12:07:55 +02:00
Justin Moon 5ec424bc86 cln-plugin: Configuration struct
Represents the "configuration" part of the "init" message during
plugin initialization.

Changelog-Added: cln_plugin: persist cln configuration from init msg
2022-06-22 12:07:55 +02:00
Justin Moon 318b6e803e cln-plugin: Add unittest for parsing "init" message 2022-06-22 12:07:55 +02:00
Justin Moon 14483901cd cln-plugin: Save "configuration" from "init" method 2022-06-22 12:07:55 +02:00
Christian Decker 9a880a0932 clnrs: Implement backwards compatible mode for Amount as bare u64
Since we want to transition to raw `u64` values we need to accept both
for some time.
2022-06-21 06:52:35 +09:30
Rusty Russell 55f94322e5 doc/PLUGINS: update to remove deprecated fields, formats.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 60bd70be85 JSON: deprecate printing msat fields as strings.
This changes many fields: in non-deprecated mode, they're now raw integers.
This was always the intention, but the transition was never completed.

Suggested-By: @ShahanaFarooqui
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: "_msat" fields can be raw numbers, not "123msat" strings (please handle both!)
Changelog-Deprecated: JSON-RPC: "_msat" fields as "123msat" strings (will be only numbers)
2022-06-21 06:52:35 +09:30
Rusty Russell 5531de99de lease_rates: prepare for msats fields as raw numbers.
We would otherwise multiply them by 1000.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 6461815dd9 plugins/funder: fix parameter parsing.
This code was buggy: handing "1000" as a parameter to
min_their_funding_msat, don't turn that into "1000sat"!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 6139319b60 funder: prepare for msats fields as raw numbers.
We would otherwise multiply them by 1000.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 993f44f289 libplugin: don't be so strict on msat fields.
Let the parsing handle if they're the wrong type (soon they'll be u64).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 98c264de66 fundchannel, txprepare, multiwithdraw: don't assume "excess_msat" will have "msat" appended.
After next patch, it's a raw u64, and this code broke.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 08d5776ebc lightningd: deprecate `msatoshi` in `sendpay` `route`.
We should be using amount_msat always.  Many tests were not.  Plus,
deprecating it simplifies the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSONRPC: `sendpay` `route` elements `msatoshi` (use `amount_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell c3efba16ff JSON: don't print deprecated amount fields any more
A small change in one routine creates a lot of changes!  We actually
recommended moving away from these in v0.7.0 (2019-02-28), but never
deprecated them formally.

Changelog-Deprecated: JSON-RPC: `pay`, `decode`, `decodepay`, `getroute`, `listinvoices`, `listpays` and `listsendpays` `msatoshi` fields (use `amount_msat`).
Changelog-Deprecated: JSON-RPC: `getinfo` `msatoshi_fees_collected` field (use `fees_collected_msat`).
Changelog-Deprecated: JSON-RPC: `listpeers` `channels`: `msatoshi_to_us`, `msatoshi_to_us_min`, `msatoshi_to_us_max`, `msatoshi_total`, `dust_limit_satoshis`, `our_channel_reserve_satoshis`, `their_channel_reserve_satoshis`, `spendable_msatoshi`, `receivable_msatoshi`, `in_msatoshi_offered`, `in_msatoshi_fulfilled`, `out_msatoshi_offered`, `out_msatoshi_fulfilled`, `max_htlc_value_in_flight_msat` and `htlc_minimum_msat` (use `to_us_msat`, `min_to_us_msat`, `max_to_us_msat`, `total_msat`, `dust_limit_msat`, `our_reserve_msat`, `their_reserve_msat`, `spendable_msat`, `receivable_msat`, `in_offered_msat`, `in_fulfilled_msat`, `out_offered_msat`, `out_fulfilled_msat`, `max_total_htlc_in_msat` and `minimum_htlc_in_msat`).
Changelog-Deprecated: JSON-RPC: `listinvoices` and `pay` `msatoshi_received` and `msatoshi_sent` (use `amount_received_msat`, `amount_sent_msat`)
Changelog-Deprecated: JSON-RPC: `listpays` and `listsendpays` `msatoshi_sent` (use `amount_sent_msat`)
Changelog-Deprecated: JSON-RPC: `listforwards` `in_msatoshi`, `out_msatoshi` and `fee` (use `in_msat`, `out_msat` and `fee_msat`)
Changelog-Deprecated: JSON-RPC: `listfunds` `outputs` `value` (use `amount_msat`)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 6afc0affef pytest: don't use deprecated amount fields
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 5c208c1b06 pyln-client: convert every _msat field to Millisatoshi
This prepares for when they start being u64, not strings with msat appended.

This has a strange side effect on our schema: despite the name,
decodepay's `fee_base_msat` is actually a u64, which we now convert to
msat on decode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 01411d70be common: enforce that msat fields are called "xxx_msat".
Now we've fixed them all, make sure no new ones slip in!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell fccb11a641 fetchinvoice: amount_msat not msat.
The name in the spec is `msat`, but I don't want to make an API exception.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fetchinvoice` `changes` `amount_msat`
Changelog-Deprecated: JSON-RPC: `fetchinvoice` `changes` `msat` (use `amount_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell f117225436 pay: use amount_msat for amount report inside attempts.
Nobody really parses this though, fortunately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `pay` `attempts` `amount_msat` field.
Changelog-Deprecated: JSON-RPC: `pay` `attempts` `amount` field (use `amount_msat`).
2022-06-21 06:52:35 +09:30
Rusty Russell f6b4dbc65a lightningd: use amount_msat not amount in htlc_accepted_hook.
Changelog-Added: Plugins: `htlc_accepted_hook` `amount_msat` field.
Changelog-Deprecated: Plugins: `htlc_accepted_hook` `amount` field (use `amount_msat`)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell cd7e784d6f lightningd: change `msatoshi` args to `amount_msat`.
This is consistent with our output changes, and increases consistency.
It also keeps future sanity checks happy, that we only use JSON msat
helpers with '_msat' fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument is now called `amount_msat` to match other fields.
Changelog-Deprecated: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice` `msatoshi` (use `amount_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell ca69e293d1 coinmvt: don't use msats in fields not called "_msat".
The new msat fields are turned into Millisatoshi, so handle that correctly
too in tests too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `coin_movement` notification: `balance`, `credit`, `debit` and `fees` (use `balance_msat`, `credit_msat`, `debit_msat` and `fees_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell e2f0ca9cbe lightningd: don't add null for unset plugin options.
In general, we don't like to use `null` in JSON: simply omit the
field.  I found this one because it broke our 'msat' parsing (made
stricter in followup) which doesn't allow `null`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: `listconfigs` `plugins` `options` which are not set are omitted, not `null`.
2022-06-21 06:52:35 +09:30
Rusty Russell a52bdeee01 common: add msat to sat convert helper.
We don't always want to round down, sometimes we want to fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 36a2491a89 json: fix up msat amounts in non-_msat fields.
We had json_add_amount_msat_only(), which was designed to be used to
print out msat fields, if we had sats.

However, we misused it, so split it into the three different cases:
1. json_add_amount_sat_msat: We are using it correctly, with a field called
   xxx_msat.
2. json_add_amount_sats_deprecated: We were using it wrong, so deprecate
   the old field and create a new one which does end in _msat.
3. json_add_sats: we were using it to hand sats as a JSON parameter to an
   interface, where "XXXsat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `rbf_channel` and `openchannel2` hooks `their_funding` (use `their_funding_msat`)
Changelog-Deprecated: Plugins: `openchannel2` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `funding_satoshis` (use `funding_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `channel_reserve_satoshis` (use `channel_reserve_msat`)
Changelog-Deprecated: Plugins: `channel_opened` notification `amount` (use `funding_msat`)
Changelog-Deprecated: JSON-RPC: `listtransactions` `msat` (use `amount_msat`)
Changelog-Deprecated: Plugins: `htlc_accepted` `forward_amount` (use `forward_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell 6e2a775ef2 common/param: support renaming options using "|<deprecatedname>".
This is *much* easier to do inside parsing than in the caller!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell c5b032598e lightningd: fix outgoing IO logging for JSONRPC.
Changelog-Fixed: lightningd: `log-level` `io` shows JSONRPC output, as well as input.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell 4daa1b37ec contrib/pylightning: remove lightning-pay helper.
This example predates the pay plugin!  It's obsolete, unmaintained,
and probably doesn't work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
pn cff859331d tests: Update expected log prefix length to match truncated log 2022-06-19 16:42:11 +09:30
pn 83c31f548f log: Add termination to log level 2022-06-19 16:42:11 +09:30
pn 47a7b4a55b log: Add termination to prefix log 2022-06-19 16:42:11 +09:30
Justin Litchfield ed7624e4f9 Warning added to PLUGIN documentation re: `stdin`
Adds some warning/documentation about the `println!`/`dbg!` causing my bad times as a plugin developer.  =)
2022-06-18 18:32:20 +09:30
fiatjaf 1eaec223b7 expose short_channel_id and htlc id to htlc_accepted.
Changelog-Added: Plugins: `htlc_accepted` now exposes the `short_channel_id` for the channel from which that HTLC is coming from and the low-level per-channel HTLC `id`, which are necessary for bridging two different Lightning Networks when MPP is involved.
2022-06-18 14:53:32 +09:30
Christian Decker 928a81f24a docs: Fix manpage title
It was causing the first sublevel of headings to be shown on RTD.
Changelog-None
2022-06-18 13:47:04 +09:30
Igor Bubelov e7393121b1 Fix typo and convert paragraph to a list 2022-06-18 13:46:35 +09:30
Rusty Russell b17db120b1 bitcoin: add to check-source-bolt, and (minor) quotes fixup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-18 13:45:09 +09:30
Rusty Russell d946de6814 bitcoin: fix header order for make check-source.
It'll matter once we actually start including bitcoin/ in `make check-source`

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-18 13:45:09 +09:30
AutonomousOrganization 1e9ecc55e7 Mistaken default directory in plugin docs
directory .lightningd -> .lightning
2022-06-17 20:36:36 +09:30