Commit Graph

1267 Commits

Author SHA1 Message Date
Christian Decker 1621c6216f rs: Fix the manifests for publication 2023-10-26 15:51:07 +02:00
Christian Decker ee8fe11fbd cln-plugin: Make logging optional
Under some circumstances we may want to not log to `lightningd`
directly, but rather configure the logging ourselves. This is useful
for example if we want to use `tracing` and `tracing-subscriber` to
add custom handling, or add opentelemetry span tracing.

Changelog-Changed: cln-plugin: Suppress internal logging handler via `with_logging(false)`
2023-10-26 12:26:01 +02:00
niftynei 411bb2884a fallback: json add fallback cleanup
Expand to handle P2TR addresses, use the common script encoding utility
function instead of reimplementing it.
2023-10-26 19:11:17 +10:30
Christian Decker 535d3d953d rs: Bump crates cln-rpc and cln-grpc to 0.1.5 and cln-plugin to 0.1.6 2023-10-24 10:24:52 +10:30
Rusty Russell c0782ee841 clnrest: correctly omit `method` to checkrune call if none is supplied.
Specifying the empty string is very different from not specifying it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:58:40 +10:30
Tony Aldon 7bea339f2f cln-grpc: do not generate cerficates when no autostart.
Changelog-None
2023-10-13 10:25:36 +02:00
Rusty Russell 7d3f13ec0d renepay: work around change in fundchannel in tests.
The tests will wait until it's locally enabled, but it might not have
the update in the gossip store.  So have renepay enhance its local
view even if it already knows about the channel (this is correct
anyway, it just isn't very important usually).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell bb60c77859 plugins/topology: convert listpeerchannels into local overlay.
This prepares us for when the gossmap doesn't contain private channel info.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell 8c0932e46a plugins/topology: split getroute logic.
Mechanical change to bundle into struct getroute_info for next
patch which uses callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell ddb16991ee plugins/topology: remove unused fuzz factor.
This breaks Dijkstra, which is presumably why it was actually
disabled.  Remove the code altoghether, instead.

Changelog-Fixed: JSON-RPC: `getroute` now documents that it ignores `fuzzpercent`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell e11b35cb3a common/memleak: implement callback arg for dump_memleak.
This makes it easier to use outside simple subds, and now lightningd can
simply dump to log rather than returning JSON.

JSON formatting was a lot of work, and we only did it for lightningd, not for
subdaemons.  Easier to use the logs in all cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-03 10:05:55 +02:00
ShahanaFarooqui 77e3c3a38e plugin/clnrest: Adjusting flask-cors install after rebase 2023-09-29 16:00:05 +09:30
ShahanaFarooqui e010eece1b plugins/clnrest: Updated clnrest's requirements.txt
Even after migrating from pip install to poetry install for clnrest,
we still need to keep `plugins/clnrest/requirements.txt` for Fedora-28-amd64
build. `Dockerfile.builder.fedora` does not have poetry and will need
requirements.txt to install libraries with pip.

It is also required for non-developers who want to build cln from a tagged or master version.
2023-09-29 16:00:05 +09:30
ShahanaFarooqui 6a756256c0 plugins/clnrest: Adding pyproject.toml for Poetry install
Reference links:
https://github.com/ElementsProject/lightning/issues/6637
https://github.com/nix-community/poetry2nix

Changelog-Changed: Upgraded clnrest to poetry project.
2023-09-29 16:00:05 +09:30
Rusty Russell 577161a4fa renepay: remove custom debug, in favor of normal operations.
As a bonus, unit tests no longer leave files in /tmp.

Reported-by: https://github.com/whitslack
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-25 13:59:13 +09:30
Rusty Russell 2bcae6fc76 plugins/renepay: remove DEVELOPER build-time, in favor of runtime.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell f725edad62 plugins: remove #if DEVELOPER.
And rename dev-only-option `use_shadow` to `dev_use_shadow`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell ef87999f9a common: add option for dev-only parameters.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell 1ecf31bae7 db: remove #if DEVELOPER in favor of runtime flag inside db struct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell a9f26b7d07 common/daemon.c: remove #ifdef DEVELOPER in favor of runtime flag.
Also requires us to expose memleak when !DEVELOPER, however we only
ever used the memleak tracking when the LIGHTNINGD_DEV_MEMLEAK
environment variable was set, so keep that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell c8c6dd183d common/bolt12: roll all necessary checks into offer_decode.
Some of these were done by fetchinvoice (and it did some that
offer_decode already did).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 13:31:03 +09:30
ShahanaFarooqui 3108ff93b8 plugins/clnrest: Update content security policy for server 2023-09-19 11:40:15 +09:30
ShahanaFarooqui 51578e1f77 plugins/clnrest: Add `rest-csp` option for config 2023-09-19 11:40:15 +09:30
ShahanaFarooqui eca3a33e69 plugins/clnrest: Update Websocket server to dynamically add CORS settings
Changelog-Added: New configurable Cross-Origin-Resource-Sharing(CSP) header for clnrest
2023-09-19 11:40:15 +09:30
ShahanaFarooqui b0430cc897 plugins/clnrest: Update App server to dynamically add CORS settings 2023-09-19 11:40:15 +09:30
ShahanaFarooqui 632e7abd05 plugins/clnrest: Add `rest-cors-origins` option for config 2023-09-19 11:40:15 +09:30
ShahanaFarooqui 224a5dcbcf plugins/clnrest: Add CORS library 2023-09-19 11:40:15 +09:30
ShahanaFarooqui 6aa697ea3d plugins/clnrest: Rune authentication for websocket server 2023-09-19 11:40:15 +09:30
ShahanaFarooqui 9147996372 plugins/clnrest: Generate all three certs if missing
Generate client, server and CA certs for https and trusted CA update
2023-09-19 11:40:15 +09:30
ShahanaFarooqui 761172001b plugin/clnrest: Deleting redundant README.md 2023-09-12 16:10:15 +09:30
Rusty Russell a4644550a2 clnrest: don't require nodeid (FIXME: needs screenshot updates!)
This was a misunderstanding: nodeid is useful for commando, where it's the
peer's nodeid, and Noise-XK guarantees that we know who that is.  It's
not useful for clnrest, so don't require it (it was our node id, which
is redundant).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-12 15:19:02 +09:30
ShahanaFarooqui e788f76f99 plugin/clnrest: do not read json payload if data length is zero
OpenAPI readme always includes `content-type: application/json` header, even when body parameters are empty.
But the server expects data if the content-type has been sent.
This results in a "Server Error" response for non-param requests from readme doc.
This only affects readme requests as it is designed to send the header by default.

Changelog-None
2023-08-31 16:13:35 +09:30
Rusty Russell f4f4ab34f3 renepay: fix case where shadow increases amount past payment amount.
Without this, sendpay will refuse to pay.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 16:13:21 +09:30
Rusty Russell 3285aa0eea renepay: fix shadow route extension logic.
pseudorand(1) is always 0, and capacity test was the wrong way.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 16:13:21 +09:30
Rusty Russell 7a92eb2d89 renepay: calculate fee and delay accumulation correctly.
1. When we add a shadow amount, we were using the wrong channel for
   the fee calculation.
2. Similarly, when calculating the delay amount.

The result is that we can get WIRE_INCORRECT_CLTV_EXPIRY repeatedly
from nodes.

Reported-by: https://github.com/Sjors
Fixes: #6620
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changlog-Experimental: Fixed: `renepay` handles ctlv correctly when it varies along a path.
2023-08-31 16:13:21 +09:30
Rusty Russell 14e28a51ce renepay: don't crash if we get failure from sendpay.
This happened while I was hacking (it shouldn't normally!): cmd here is NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 16:13:21 +09:30
Christian Decker 8e2e7b1c51 rs: Bump crate versions before publishing
Changelog-None
2023-08-28 11:19:30 +02:00
Dusty Daemon 8d5be61fcb clnrest: Change process method to fork
On mac the clnrest plugin hangs on log messages, causing it to never run.

This guide suggests switching to fork’ing instead of spwan’ing
https://superfastpython.com/multiprocessing-common-errors/#Error_2_print_Does_Not_Work_In_Child_Processes

Switching to fork fixes the hang on Mac.

Changelog-None
2023-08-24 19:36:42 +09:30
Rusty Russell 7cd4359521 renepay: hack in workaround spendable reduction for release.
```
Flow 391: amount=23528000msat prob=0.000 fees=1023msat delay=140 path=-2471854x37x4/1(min=max=23528783msat)->-2414928x98x0/0->
Flow 391: Failure of 23529023msat for 2471854x37x4/1 capacity [23528783msat,23528783msat] -> [23528783msat,23528783msat]
```

We added fees and went over capacity!  This screams of a deeper logic
bug, but renepay is experimental and it's release day so hack around
it for now...

Reported-by: https://github.com/daywalker90
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 13:22:27 +09:30
Rusty Russell a984202576 plugins/renepay: don't drop min to 0 if it's shown wrong, but drop 50%.
Eduardo is on holiday right now, but he pinged me asking for this.  It
makes some sense, and using half the *failed value* covers the case where
it's less than half what we expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 32f5ae8d1f renepay: remove remaining calls to debug_paynote in favor of payflow_note().
Some are redundant, others simply convert.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 574716dfd5 plugins/renepay: convenience routing to log when we disable a chan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 00e9af57f5 plugins/renepay: neaten the command notifications.
It now looks like (for test_hardmpp):

```
# we have computed a set of 1 flows with probability 0.328, fees 0msat and delay 23
#   Flow 1: amount=1800000000msat prob=0.328 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0->-103x3x0/1->
#   Flow 1: Failed at node #1 (WIRE_TEMPORARY_CHANNEL_FAILURE): failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
#   Flow 1: Failure of 1800000000msat for 103x5x0/0 capacity [0msat,3000000000msat] -> [0msat,1799999999msat]
# we have computed a set of 2 flows with probability 0.115, fees 0msat and delay 23
#   Flow 2: amount=500000000msat prob=0.475 fees=0msat delay=12 path=-103x6x0/0(min=max=4294967295msat)->-103x1x0/1->-103x4x0/1->
#   Flow 3: amount=1300000000msat prob=0.242 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0(max=1799999999msat)->-103x3x0/1->
#   Flow 3: Failed at node #1 (WIRE_TEMPORARY_CHANNEL_FAILURE): failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
#   Flow 3: Failure of 1300000000msat for 103x5x0/0 capacity [0msat,1799999999msat] -> [0msat,1299999999msat]
# we have computed a set of 2 flows with probability 0.084, fees 0msat and delay 23
#   Flow 4: amount=260000000msat prob=0.467 fees=0msat delay=12 path=-103x6x0/0(500000000msat in 1 htlcs,min=max=4294967295msat)->-103x1x0/1(500000000msat in 1 htlcs)->-103x4x0/1(500000000msat in 1 htlcs)->
#   Flow 5: amount=1040000000msat prob=0.179 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0(max=1299999999msat)->-103x3x0/1->
#   Flow 5: Failed at node #1 (WIRE_TEMPORARY_CHANNEL_FAILURE): failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
#   Flow 5: Failure of 1040000000msat for 103x5x0/0 capacity [0msat,1299999999msat] -> [0msat,1039999999msat]
# we have computed a set of 2 flows with probability 0.052, fees 0msat and delay 23
#   Flow 6: amount=120000000msat prob=0.494 fees=0msat delay=12 path=-103x6x0/0(760000000msat in 2 htlcs,min=max=4294967295msat)->-103x1x0/1(760000000msat in 2 htlcs)->-103x4x0/1(760000000msat in 2 htlcs)->
#   Flow 7: amount=920000000msat prob=0.105 fees=0msat delay=12 path=-103x2x0/1(min=max=4294967295msat)->-103x5x0/0(max=1039999999msat)->-103x3x0/1->
#   Flow 7: Success
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 7db6aaa523 renepay: clarify chan_extra_can_send.
I am doing to add more more debugging, but sent here is 0.
Document that clearly, and put a real value in sent.

Also: since we already sub 1 msat from x, amount_msat_less_eq should
be amount_msat_less (it may be equal to our min, in theory).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 7287ade705 plugins/renepay: use struct short_channel_id_dir.
No changes, just use this convenience type for handing
around, and arrays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell a6c2cd77a1 plugins/renepay: clean up unused fmt_payflows.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell eef57488f9 pay_flow: downgrade messages about shadow route delays.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 4196f97571 plugins/renepay: Add infra for per-flow notes.
And set logging levels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-23 10:47:28 +09:30
Rusty Russell 763ec6ad43 plugins/renepay: don't free waiting-for-addgossip flow!
```
Aug 18 13:45:13  lightningd: 0x7fa921f8ffcf ???
Aug 18 13:45:13  lightningd:         ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
Aug 18 13:45:13  lightningd: 0x55b3bb54e6d3 pay_flow_finished_adding_gossip
Aug 18 13:45:13  lightningd:         plugins/renepay/pay_flow.c:675
Aug 18 13:45:13  lightningd: 0x55b3bb54af25 addgossip_done
Aug 18 13:45:13  lightningd:         plugins/renepay/pay.c:171
```

The assert we fail is almost certainly due to the flow being freed:

```
struct pf_result *pay_flow_finished_adding_gossip(struct pay_flow *pf)
{
	assert(pf->state == PAY_FLOW_FAILED_GOSSIP_PENDING);
```

Reported-by: https://github.com/daywalker90
Fixes: #6567
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:13:48 +09:30
Vincenzo Palazzo d2e39f7af8 meta: ignore renepay binary
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-08-21 10:35:41 +09:30