Commit Graph

10728 Commits

Author SHA1 Message Date
Rusty Russell 9d18180172 lightningd: really do allow two Torv3 addresses.
This surprised me, since the CHANGELOG for [0.8.2] said:

	We now announce multiple addresses of the same type, if given. ([3609](https://github.com/ElementsProject/lightning/pull/3609))

But it lied!

Changelog-Fixed: We really do allow providing multiple addresses of the same type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell 3dcab9793d doc/TOR.md: simplify, and don't cover autotor.
autotor is older, but statictor is better.  Your options are really
"use HiddenServiceDir in torrc" vs "use statictor", with the issue
that statictor requires you to configure Tor for control access by
c-lightning.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell 2a35f33cb0 doc/TOR.md: Make it clear that Tor == Torv3.
And switch ```` to ``` (emacs colorization was confused!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell 2f247c7bfb torv2: remove support for advertizing and connecting.
October was the date Torv2 is no longer supported by the Tor Project;
it will probably not work at all by next release, so we should remove
it now even though it's not quite the 6 months we prefer for
deprecation cycles.

I still see 110 nodes advertizing Torv2 (vs 10,292 Torv3); we still
parse and display it, we just don't advertize or connect to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell ecdc15591b custommsg hook: remove message field.
Changelog-Removed: Plugins:  The `message` field on the `custommsg` hook (deprecated in v0.10.0)
2021-11-14 18:49:46 +01:00
Rusty Russell 0c0a301062 fundchannel_complete: remove deprecated txid/txout params.
Changelog-Removed: JSON-RPC: `fundchannel_complete` `txid` and `txout` parameters (deprecated in v0.10.0)
2021-11-14 18:49:46 +01:00
Rusty Russell c00202a0da pytest: remove test_shutdown_alternate_txid.
We're about to require that fundchannel_complete() take a PSBT, where it
does sanity checks to avoid this error, making this a difficult mistake
to make.

Is it time to remove this functionality anyway?  @cdecker?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
ZmnSCPxj jxPCSnmZ c3847484d8 Makefile: Avoid overriding `PYTHONPATH`.
On some distributions (e.g. Gnu Guix) Python packages are not installed in
some standard directory, rather they are installed in different places and
the `PYTHONPATH` variable is modified to include the different places.
So, we must not use the name `PYTHONPATH` in our `Makefile` since `make`
will replace the `PYTHONPATH` environment variable, preventing e.g.
`tools/generate-wire.py` from finding `python-mako` installed on such
distributions.
2021-11-11 14:49:30 +10:30
Rusty Russell 78ebddeece subd: clean up our fd shuffling logic.
It's both complex and flawed, as ZmnSCPxj points out.  Make a generic
fd ordering routine, and use it.

Plus, test it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-10 10:27:15 +10:30
Christian Decker 12689674c7 doc: Update CHANGELOG.md for v0.10.2 release 2021-11-03 15:11:01 +01:00
Christian Decker 000508ca12 doc: Update CHANGELOG.md for v0.10.2rc2 2021-10-31 17:04:43 +01:00
Christian Decker 3d8293dbf4 gci: Add `-f` flag to git fetch
This avoids the warning of clobbering tags when fetching tags

Changelog-None
2021-10-31 16:57:56 +01:00
Christian Decker d680e3b158 doc: Remove non-ASCII character from docs
mrkd doesn't like non-ASCII characters in ubuntu:18.04, used for
repro-builds. This is the easiest way to sidestep this issue until we
deprecate the old ubuntu version.
2021-10-31 16:57:56 +01:00
Christian Decker 0351940593 peer: Wrap the HTLC out destructor in a DB transaction
Changelog-Fixed: peer: Fixed a crash when a connection is lost outside of a DB transaction
2021-10-31 13:11:34 +01:00
Christian Decker 344579b233 wallet: Re-sync the payments.id sequence with postgresql
Fixes #4883
Fixes #4879
Changelog-None: Unreleased bug being fixed.
2021-10-31 13:11:34 +01:00
Christian Decker 15b403531e db: Allow DB-specific queries in the migrations
Sometimes we need some DB-specific magic, and marking migrations with
`/*PSQL*/` or `/*SQLITE*/` will now give us that.
2021-10-31 13:11:34 +01:00
Vincenzo Palazzo e0745358d1 Fix typo in the error message.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-30 11:12:21 +02:00
Vincenzo Palazzo b031f07b8c Ignore all the deamon inside the git history
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-28 15:51:38 +02:00
ZmnSCPxj jxPCSnmZ d330e6e5b4 wallet/db_postgres.c: Fix obvious typo in error message.
ChangeLog-None
2021-10-28 15:51:25 +02:00
Rusty Russell ba77fce453 websocket: switch to address type 6, since DNS will take 5.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-28 15:51:11 +02:00
Christian Decker 091a6d9151 meta: Add version 0.10.2rc1 changelog 2021-10-23 13:22:20 +02:00
Christian Decker 012c2ecf17 pytest: Create less dust in `test_delpay_payment_split`
We were triggering the dust panic.
2021-10-23 12:59:13 +02:00
niftynei 303fbce479 tests: raise dust limit on mpp test
Fails liquid-regtest otherwise; liquid tends to hit the dust limit
earlier than non-liquid tx, and MPP exacerbates this by divvying up
payments into dusty bits then attempting to shove them through the same
channel, hitting the dust max. The MPP then fails as not all the parts
were able to arrive at their destination.
2021-10-23 12:59:13 +02:00
niftynei b57fed047a dusty htlcs: don't fail the channel, make it error a whole bunch
Let's make this a softer launch by just warning on the channel til the
feerates go back down.

You can also 'fix' this by upping your dust limit with
the `max-dust-htlc-exposure-msat` config.
2021-10-23 12:59:13 +02:00
niftynei cad082a763 channeld: add in RFC notes for max_htlc_dust_exposure_msat
And update some behavior to check both sides on receipt of a
update_fee, as per the proposed spec.

https://github.com/lightningnetwork/lightning-rfc/pull/919
2021-10-23 12:59:13 +02:00
niftynei 150d065088 tests: check for incoming + outgoing dust limits 2021-10-23 12:59:13 +02:00
niftynei b193eb06d3 dusty-htlcs: enforce limit on dusty htlcs
for every new added htlc, check that adding it won't go over our 'dust
budget' (which assumes a slightly higher than current feerate, as this
prevents sudden feerate changes from overshooting our dust budget)

note that if the feerate changes surpass the limits we've set, we
immediately fail the channel.
2021-10-23 12:59:13 +02:00
niftynei 42e40c1ced htlcs: add flag to 'fail immediately'
If we're over the dust limit, we fail it immediatey *after* commiting
it, but we need a way to signal this throughout the lifecycle, so we add
it to htlc_in struct and persist it through to the database.

If it's supposed to be failed, we fail after the commit cycle is
completed.
2021-10-23 12:59:13 +02:00
niftynei 208b161226 tests: bump the ceiling for dust, fixes failures for liquid
Liquid's threshold for dust is a bit higher, so we bump up the max limit
here so we can actually get the whole MPP'd payment sent over the wire
2021-10-23 12:59:13 +02:00
niftynei 1fe829c546 lightningd: new option for htlc dust limit
To reduce the surface area of amount of a channel balance that can be
eaten up as htlc dust, we introduce a new config
'--max-dust-htlc-exposure-msat', which sets the max amount that any
channel's balance can be added as dust

Changelog-Added: config: new option --max-dust-htlc-exposure-msat, which limits the total amount of sats to be allowed as dust on a channel
2021-10-23 12:59:13 +02:00
Christian Decker 0f0a77e322 doc: Add M1 mac + homebrew fix
The paths for this platform seem to be slightly different, so mention
how they must be set.
2021-10-22 16:05:23 +02:00
Rene Pickhardt 0ba1bc30fe pay: Use log probability based bias in channel selection
Changelog-Changed: pay: The route selection will now use the log-propability-based channel selection to increase success rate and reduce time to completion
2021-10-22 16:04:37 +02:00
Rusty Russell 233d339061 libplugin-pay: bias towards larger channels.
We bias by channel linearly by capacity, scaled by median fee.
This means that we effectively double the fee if we would use the
entire capacity, and only increase it by 50% if we would only use
1/2 the capacity.

This should drive us towards larger channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `pay` now biases towards larger channels, improving success probability.
2021-10-22 16:04:37 +02:00
Rusty Russell 0b3c79b3c2 common/dijkstra: hand channel direction to path_score callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 16:04:37 +02:00
ZmnSCPxj jxPCSnmZ e733fdf62e lightningd/lightningd.c: Only impose fd limit if absolutely needed.
Fixes: #4868

ChangeLog-Fixed: We now no longer self-limit the number of file descriptors (which limits the number of channels) in sufficiently modern systems, or where we can access `/proc` or `/dev/fd`.  We still self-limit on old systems where we cannot find the list of open files on `/proc` or `/dev/fd`, so if you need > ~4000 channels, upgrade or mount `/proc`.
2021-10-22 13:17:37 +02:00
ZmnSCPxj jxPCSnmZ 5356267f15 *: Use new closefrom module from ccan.
This also inadvertently fixes a latent bug: before this patch, in the
`subd` function in `lightningd/subd.c`, we would close `execfail[1]`
*before* doing an `exec`.
We use an EOF on `execfail[1]` as a signal that `exec` succeeded (the
fd is marked CLOEXEC), and otherwise use it to pump `errno` to the
parent.
The intent is that this fd should be kept open until `exec`, at which
point CLOEXEC triggers and close that fd and sends the EOF, *or* if
`exec` fails we can send the `errno` to the parent process vua that
pipe-end.

However, in the previous version, we end up closing that fd *before*
reaching `exec`, either in the loop which `dup2`s passed-in fds (by
overwriting `execfail[1]` with a `dup2`) or in the "close everything"
loop, which does not guard against `execfail[1]`, only
`dev_disconnect_fd`.
2021-10-22 13:17:37 +02:00
ZmnSCPxj jxPCSnmZ 5a84abb09e ccan: update to include closefrom
Signed-off-by: ZmnSCPxj jxPCSnmZ <ZmnSCPxj@protonmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 13:17:37 +02:00
Rusty Russell ed6eaf9171 experimental-websocket-port: option to create a WebSocket port.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell 80a47f1111 lightning_websocketd: simple proxy for websockets.
WebSocket is a bit weird:
1. It starts like an HTTP connection, but they send special headers.
2. We reply with special headers, one of which involves SHA1 of one of theirs.
3. We are then in WebSocket mode, where each frame starts with a 2-20 byte
   header.

We relay data in a simplistic way: if either side sends something, we
read it and relay it synchronously.  That avoids any gratuitous
buffering.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell f78184ce46 connectd: listen on ports for which we should spawn a proxy.
If the port is set, we spawn it (lightning_websocketd) on any
connection to that port.  That means websocketd is a per-peer daemon,
but it means every other daemon uses the connection normally (it's
just actually talking to websocketd instead of the client directly).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell b013b3ab0c patch websocket-address-support.patch 2021-10-22 11:56:30 +02:00
Rusty Russell 81b2092cfc CCAN: add base64 module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell 9c254c8387 gossipd: when we refresh channel, don't override pending updates.
We can have an update pending because it's too fast, but
refresh_local_channel is supposed to make sure we're up-to-date, so
force immediate application in that case.

Otherwise, we call update_local_channel at the bottom which frees the
pending update.  This can mean that we miss a change in fees, for example.

Changelog-Fixed: errors: Errors returning a `channel_update` no longer return an outdated one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-21 20:05:54 +02:00
Rusty Russell 3018113012 common: reltimer_arg to access a timer argument.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-21 20:05:54 +02:00
Vincenzo Palazzo 09459a97c7 Move log level from warning to info when creating a new ln directory
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-18 02:15:56 +00:00
ZmnSCPxj jxPCSnmZ 3e902e2816 doc/BACKUP.md: Warn about version if using Litestream, `sqlite3` `.dump` is safe(r) now. 2021-10-17 23:58:39 +00:00
ZmnSCPxj jxPCSnmZ ab026fb4de wallet/db_sqlite3.c: Increase busy timeout to 60 seconds.
Closes: #4860

ChangeLog-Added: With `sqlite3` db backend we now use a 60-second busy timer, to allow backup processes like `litestream` to operate safely.
2021-10-17 23:58:39 +00:00
Rusty Russell c012a71dec CI: suppress postgres vaccuuming.
```
[gw1] [ 98%] PASSED tests/test_wallet.py::test_hsmtool_dump_descriptors 
tests/test_wallet.py::test_fundchannel_listtransaction 
[gw0] [ 98%] PASSED tests/test_plugin.py::test_channel_opened_notification 
tests/test_wallet.py::test_hsmtool_generatehsm 
[gw0] [ 98%] PASSED tests/test_wallet.py::test_hsmtool_generatehsm 
tests/test_wallet.py::test_withdraw_nlocktime_fuzz 
[gw1] [ 98%] ERROR tests/test_wallet.py::test_fundchannel_listtransaction 
tests/test_wallet.py::test_fundchannel_listtransaction 
tests/test_wallet.py::test_withdraw_nlocktime_fuzz 
tests/test_wallet.py::test_fundchannel_listtransaction 
[gw0] [ 99%] ERROR tests/test_wallet.py::test_withdraw_nlocktime_fuzz 
tests/test_wallet.py::test_multiwithdraw_simple 
[gw1] [ 99%] ERROR tests/test_wallet.py::test_fundchannel_listtransaction 
tests/test_wallet.py::test_withdraw_nlocktime 
tests/test_wallet.py::test_multiwithdraw_simple 
tests/test_wallet.py::test_withdraw_nlocktime 
tests/test_wallet.py::test_multiwithdraw_simple 
tests/test_wallet.py::test_withdraw_nlocktime 
[gw0] [ 99%] ERROR tests/test_wallet.py::test_multiwithdraw_simple 
tests/test_wallet.py::test_repro_4258 
[gw1] [ 99%] ERROR tests/test_wallet.py::test_withdraw_nlocktime
...
2021-10-12 06:36:09.203 UTC [224552] STATEMENT:  SELECT version FROM version LIMIT 1
2021-10-12 06:36:09.566 UTC [224523] PANIC:  could not write to file "pg_wal/xlogtemp.224523": No space left on device
2021-10-12 06:36:09.566 UTC [224523] STATEMENT:  VACUUM FULL;
Error vacuuming db: BEGIN command failed: PANIC:  could not write to file "pg_wal/xlogtemp.224523": No space left on device
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
```
2021-10-15 12:09:36 +02:00
Rusty Russell da03985996 wallet: only hand onchaind the HTLCs it needs to know.
This will make closing long-lived channels more efficient, and it's
just nicer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell 3b5c24ada1 onchaind: queue any unexpected messages while waiting for htlcs.
In particular, we could get depth notifications.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00