Commit Graph

9678 Commits

Author SHA1 Message Date
Christian Decker 8c94d1a358 make: Remove hardcoded timeout to pytest
This should really be set by the environment by creating either a
pytest.ini or setting PYTEST_OPTS envvar.
2021-01-29 10:29:09 +10:30
Christian Decker 1ea0dd6af2 gci: Add pytest.ini in order to randomize the groups
We have a couple of very heavy tests bunched together, randomization
could potentially lessen the peak load.
2021-01-29 10:29:09 +10:30
Christian Decker 6119f1f5fb gci: Format the build script
We were using a lot of docker conventions, which are not necessary in
the script itself.
2021-01-29 10:29:09 +10:30
Christian Decker cd9aa267b4 pyln: Adjust maximum load allowed by the throttler 2021-01-29 10:29:09 +10:30
Christian Decker 542f3225e3 pytest: Parameterize process waits for hsmtool calls
We were sometimes waiting only 5 seconds, which is way too short on a
heavily loaded machine such as CI. Making it 30 seconds and collecting
it in a single place so we can adjust more easily.
2021-01-29 10:29:09 +10:30
Christian Decker 4c3ee04bb7 pyln: Use a fair FS lock to throttle node startups
We were getting a couple of starvations, so we need a fair filelock. I
also wasn't too happy with the lock as is, so I hand-coded it quickly.

Should be correct, but the overall timeout will tell us how well we
are doing on CI.
2021-01-29 10:29:09 +10:30
Christian Decker 8cc62d76e4 pytest: Stabilize test_channel_{spendable,receivable}
They were using TIMEOUT / 2 which may be way too long (hit against
test timeout), so we use a still ludicrous 30 seconds instead.
2021-01-29 10:29:09 +10:30
Christian Decker 6384cadd69 pytest: Stabilize the negotiation tests
We also make the logic a bit nicer to read. The failure was due to
more than one status message being present if we look at the wrong
time:

```
arr = ['CLOSINGD_SIGEXCHANGE:We agreed on a closing fee of 20334 satoshi for tx:17f1e9d377840edf79d8b6f1ed0faba59bb307463461...9b98', 'CLOSINGD_SIGEXCHANGE:Waiting for another closing fee offer: ours was 20334 satoshi, theirs was 20332 satoshi,']                                                                                                   │

     def only_one(arr):
         """Many JSON RPC calls return an array; often we only expect a single entry
         """
 >       assert len(arr) == 1
 E       AssertionError
```
2021-01-29 10:29:09 +10:30
Christian Decker 04ed93f5f8 pytest: Stabilize test_funding_external_wallet_corners 2021-01-29 10:29:09 +10:30
Christian Decker 18483ca582 pytest: Disable test_v2_open if not developer
It requires `--dev-force-features` which isn't available without
`DEVELOPER=1`
2021-01-29 10:29:09 +10:30
Christian Decker 7962db821c pytest: Stabilize test_channel_state_changed_bilateral 2021-01-29 10:29:09 +10:30
Christian Decker 07f5054700 pytest: Stabilize test_setchannelfee_state
Synching with the blockchain was slower than our timeout...
2021-01-29 10:29:09 +10:30
Christian Decker 03449e3cf0 pytest: Stabilize test_gossip_persistence
We weren't waiting for the `dev_fail` transaction to hit the mempool,
throwing the results off.
2021-01-29 10:29:09 +10:30
Christian Decker ae40c10bcb pytest: Stabilize test_onchain_timeout
The timeout on the pay future was too short under valgrind.
2021-01-29 10:29:09 +10:30
Christian Decker fc677e331a pyln: Update dependencies for all pyln packages
We were getting a number of incompatibility warning due to the
dependencies being expressed too rigidly. This losens the requirement
definitions to being compatible with a known good version, and while
we're at it we also bump all outdated requirements.
2021-01-29 10:29:09 +10:30
Christian Decker c564f165fa pytest: Stabilize test_penalty_htlc_tx_timeout
We weren't waiting for the transactions to enter the mempool which
could cause all of our fine-tuned block counts to be off. Now just
waiting for the expected number of txs.
2021-01-29 10:29:09 +10:30
Christian Decker c0f06f2779 pytest: Simplify and stabilize test_reconnect_no_update 2021-01-29 10:29:09 +10:30
Christian Decker 2b12cac31e pytest: Skip hsm encryption test if we don't have a TTY 2021-01-29 10:29:09 +10:30
Christian Decker be77cd7669 gci: Expand matrix to include all CI configurations 2021-01-29 10:29:09 +10:30
Christian Decker b447944285 gci: Add a tester Dockerfile 2021-01-29 10:29:09 +10:30
Christian Decker 6934f71bf8 gci: Add basic configuration for Github Actions as CI 2021-01-29 10:29:09 +10:30
Christian Decker 6bed85bac2 pyln: Bump pyln version to 0.9.3 before publishing on PyPI
Let's just keep pyln synched up with the c-lightning version
2021-01-27 11:29:56 +01:00
Christian Decker 483579f8b6 pyln: Pin the mypy dependency to 0.790 since 0.800 has a regression 2021-01-26 15:45:13 +01:00
Christian Decker 46af65d2e3 git: Ignore some more generated files and artifacts 2021-01-26 15:45:13 +01:00
Christian Decker 527cae0d4c doc: Make the release process a bit more readable 2021-01-26 15:45:13 +01:00
Christian Decker f76a6b99e0 make: Explicitly set PYTHON_VERSION=3 for libwally
Otherwise it can end up complaining that it can't find python2, which
we don't have listed as a dependency and is deprecated anyway.

Also reformatted a bit to make the options more readable and minimize
future conflicts.
2021-01-26 15:45:13 +01:00
Christian Decker f1962c1f4b tools: Pretty print the credits 2021-01-26 15:45:13 +01:00
Christian Decker 46cd5d8923 tools: Allow versions to either have a v prefix or not
Sometimes it's v0.9.3 and sometimes it's 0.9.3, and I always get it
wrong. This is painful when you have to create a signed tag after
committing to one of the two only to find out you chose wrong and have
to redo the tagging.
2021-01-26 15:45:13 +01:00
Christian Decker 25a7bf6aa3 doc: Update release process document 2021-01-26 15:45:13 +01:00
Christian Decker 25aa05d26e devtools: Make the raw changelog more readable
The links were rather annoying in the flowing text, so extract them
and add them to the footer of the release instead. This uses the
shorthand and footnote style of links.
2021-01-26 15:45:13 +01:00
nicolas.dorier 3105a7a204 Fix dockerfiles 2021-01-26 15:03:35 +01:00
niftynei d1a59b6f0d nit df: correct spelling mistake 2021-01-25 10:05:17 +10:30
niftynei 8591f11188 wallet nit: consolidate stmt free into one line 2021-01-25 10:05:17 +10:30
niftynei 6778ea5d05 wallet: cleanup fee_states alloc on error 2021-01-25 10:05:17 +10:30
Christian Decker 015ac37d92 release: Update the changelog for v0.9.3 2021-01-20 12:00:52 +01:00
Antoine Poinsot 48595674fa hsmtool: don't streq() on NULL
This would cause a segfault on the default network parameter for
`dumponchaindescriptors`.

Introduced in 1513a2d07e

Changelog-Fixed: hsmtool: fix a segfault on `dumponchaindescriptors` without network parameter
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-20 10:22:31 +01:00
ZmnSCPxj jxPCSnmZ 4dfbee47f7 wallet/db.c: Speed up deletion of single peers.
ChangeLog-Fixed: Database: Speed up deletion of peer especially when there is a long history with that peer.
2021-01-19 09:56:12 +01:00
Dave Scotese 6f924e63c2 Added link to https://lightning.readthedocs.io/
Fixes #4335
2021-01-17 13:30:39 +01:00
Christian Decker 0ff1493cf2 changelog: Add v0.9.3rc1..v0.9.3rc2 changes 2021-01-15 19:26:38 +01:00
Rusty Russell 130280d457 doc/PLUGINS.md: Document the onion_message and onion_message_blinded hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-14 13:56:13 +01:00
Rusty Russell 79f32751d2 lightningd: don't expose (undocumented) experimental-dual-fund.
It doesn't make sense currently without EXPERIMENTAL_FEATURES, and
check-manpages complains about it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-14 13:56:13 +01:00
Rusty Russell 424800fba6 doc: note where experimental-offers is required or affects commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-14 13:56:13 +01:00
Rusty Russell c4fe2e7f5a doc: Manual page for sendonionmessage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-14 13:56:13 +01:00
Rusty Russell 4d1214b432 lightningd: fix double-free when forking subdaemon fails.
payload is owned by the peer, which is freed in this case, then we
free payload (again).

==1404== Invalid read of size 8
==1404==    at 0x1F39E8: to_tal_hdr (tal.c:174)
==1404==    by 0x1F43A4: tal_free (tal.c:479)
==1404==    by 0x14B3D1: peer_connected_hook_cb (peer_control.c:1087)
==1404==    by 0x15D6E9: plugin_hook_call_ (plugin_hook.c:288)
==1404==    by 0x14B40E: plugin_hook_call_peer_connected (peer_control.c:1090)
==1404==    by 0x14B5B8: peer_connected (peer_control.c:1135)
==1404==    by 0x122FCF: connectd_msg (connect_control.c:310)
==1404==    by 0x160291: sd_msg_read (subd.c:480)
==1404==    by 0x15FBE7: read_fds (subd.c:308)
==1404==    by 0x1E37D1: next_plan (io.c:59)
==1404==    by 0x1E434E: do_plan (io.c:407)
==1404==    by 0x1E438C: io_ready (io.c:417)
==1404==  Address 0x2fcd2268 is 24 bytes inside a block of size 336 free'd
==1404==    at 0x4C32D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404==    by 0x1F416E: del_tree (tal.c:421)
==1404==    by 0x1F40F2: del_tree (tal.c:412)
==1404==    by 0x1F442C: tal_free (tal.c:486)
==1404==    by 0x148816: delete_peer (peer_control.c:120)
==1404==    by 0x148899: maybe_delete_peer (peer_control.c:136)
==1404==    by 0x13A970: destroy_uncommitted_channel (opening_common.c:29)
==1404==    by 0x1F3BB1: notify (tal.c:240)
==1404==    by 0x1F40A0: del_tree (tal.c:402)
==1404==    by 0x1F442C: tal_free (tal.c:486)
==1404==    by 0x13D3E9: peer_start_openingd (opening_control.c:911)
==1404==    by 0x14B3C2: peer_connected_hook_cb (peer_control.c:1086)
==1404==  Block was alloc'd at
==1404==    at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404==    by 0x1F3C1B: allocate (tal.c:250)
==1404==    by 0x1F41B4: tal_alloc_ (tal.c:428)
==1404==    by 0x14B454: peer_connected (peer_control.c:1105)
==1404==    by 0x122FCF: connectd_msg (connect_control.c:310)
==1404==    by 0x160291: sd_msg_read (subd.c:480)
==1404==    by 0x15FBE7: read_fds (subd.c:308)
==1404==    by 0x1E37D1: next_plan (io.c:59)
==1404==    by 0x1E434E: do_plan (io.c:407)
==1404==    by 0x1E438C: io_ready (io.c:417)
==1404==    by 0x1E6552: io_loop (poll.c:445)
==1404==    by 0x12E2AD: io_loop_with_timers (io_loop_with_timers.c:24)

Fixes: #4329
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-14 11:13:19 +01:00
Rusty Russell 1be4d42ca3 fetchinvoice: don't abort() if we try to fetch an invoice from ourselves.
blindedpath.c does:

```
	size_t num = tal_count(route);

	if (!num)
		abort();
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-14 11:12:11 +01:00
Christian Decker d953c68d58 travis: Bump number of valgrind configs to 12 2021-01-13 14:45:36 +01:00
Rusty Russell 001b5d6416 offers: make it a runtime option.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
2021-01-13 14:45:36 +01:00
Rusty Russell a4bc3e5673 common/bolt12: include correct bolt12 wiregen header.
Don't include exp directly, use an ifdef in common/bolt12
(like we do for peer and onion wiregen files).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell 6dab04ebd5 tools/generate-wire.py: include digits in #ifndef idempotent header.
Otherwise check complains that it's '#ifndef LIGHTNING_WIRE_BOLT_WIREGEN_H':

wire/bolt12_wiregen.h seems to be missing the expected include guard:
  #ifndef LIGHTNING_WIRE_BOLT12_WIREGEN_H
  #define LIGHTNING_WIRE_BOLT12_WIREGEN_H
  ...
  #endif /* LIGHTNING_WIRE_BOLT12_WIREGEN_H */

make: *** [Makefile:458: check-includes] Error 1

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell 27c006f7aa libplugin: make init return a string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: init can return a non-NULL string to disable the plugin.
2021-01-13 14:45:36 +01:00