Commit Graph

204 Commits

Author SHA1 Message Date
Rusty Russell e5f7e5b577 pytest: allow variable-order coin_moves, and give more information on failure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell cbd0be5c0e EXPERIMENTAL_FEATURES: enable option_anchor_outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell 8cf2fff91b pytest: generalize tests.
Anchor outputs break many assumptions in our tests:

1. Remove some hardcoded numbers in favor of a fee calc, so we only have to
   change in one place.
   FIXME: This should also be done for elements!
2. Do binary search to get feerate for a given closing fee.
3. Don't assume output #0: anchor outputs perturb them.
4. Don't assume we can make 1ksat channels (anchors cost 660 sats!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell 01a82d38f7 pytest: add slow_test marker.
And when it's set, and we're SLOW_MACHINE, simply disable valgrind.

Since Travis (SLOW_MACHINE=1) only does VALGRIND=1 DEVELOPER=1 tests,
and VALGRIND=0 DEVELOPER=0 tests, it was missing tests which needed
DEVELOPER and !VALGRIND.

Instead, this demotes them to non-valgrind tests for SLOW_MACHINEs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 7aa8ffa2a0 bitcoin: add weight calculation helpers.
These are pulled from wallet/wallet.c, with the fix now that we grind sigs.

This reduces the fees we pay slightly, as you can see in the coinmoves changes.

I now print out all the coin moves in suitable format before we match:
you only see this if the test fails, but it's really helpful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
niftynei 8185474bfb liquid-compat: update coinmove check to be liquid compatible
🎉 :waterfall:
2020-07-01 19:50:02 -05:00
Rusty Russell ce9e559aed features: do not set option_support_large_channel in channel_announcement.
Spec is wrong (it says it should be compulsory), and Eclair doesn't set it
at all, leading to an error when they send their announcement_signatures.

Fixes: #3703
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: large-channels: negotiate successfully with Eclair nodes.
2020-05-26 19:57:29 -05:00
Rusty Russell f4f8a363dd pytest: fix feature mask for EXPERIMENTAL_FEATURES, add wumbo support.
There are various places where our tests failed with
--enable-expimental-features.  And our plugin test overlapped an
existing feature.

We make our expected_feature functions more generic, and use them
everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
lisa neigut 8acbbca05d coins: use the chain's BIP173 name instead of a 'unit of account'
Updates the unit of account to be the chain_id, which is the BIP173 name
of the chain that the coins moved on.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut ffd9467f14 coin moves: persist the coin movement index counter to disk
Should make it easier to track when coin moves in the plugin are
disjoint from what c-lightning says it's broadcast already.
2020-05-12 15:46:58 +09:30
lisa neigut 9c4c0f10fb coin_mvt: add integration tests for in-channel htlc movements 2020-05-12 15:46:58 +09:30
Rusty Russell 046b402c18 gossipd: return channel_announcement features for listchannels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON API: `listchannels` now shows channel `features`.
2020-05-07 08:44:58 +09:30
lisa neigut d6d26dd056 features: split expected feature bits into node/peer sets
The new `keysend` plugin modifies the node features that we send to
peers. This commit breaks out the 'expected_features' we use for tests
to encompass this differentiation.
2020-04-28 10:39:07 -05:00
darosior 5aafef1484 pytest: skip some test when DEPRECATED_APIS is enabled. 2020-04-05 11:06:27 +09:30
Rusty Russell 839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Rusty Russell 5d413e0f79 common: offer option_basic_mpp for EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 8393d21503 common/features: add `payment_secret` feature if EXPERIMENTAL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Christian Decker d58339b872 pytest: Migrate to in-tree pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker b7e7e535c8 pytest: Migrate `env` to pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker f6a016c17d pytest: Consolidate access to environment variables
We use `env()` to look up configuration variables and allow them to be
overridden by the environment.
2019-11-12 21:23:55 +01:00
Rusty Russell a3f6ce1f8f pytest: add filters arg to query_gossip()
Code changes mean we're going to get gossip_timestamp_filter messages from
peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
darosior c9982f9f49 pytest: allow to set stdin, stdout and stderr at lightningd startup
And also allow to not wait for it to be started.
Passing stderr=subprocess.STDOUT can be useful to wait_for_log() also on
stderr messages.
2019-10-09 22:00:38 -05:00
Christian Decker 37fb912524 pytest: Exclude elements-cli from valgrind when following children
Without this we'd be attaching valgrind to all elements-cli call, resulting in
a huge overhead and timeouts.
2019-10-03 04:32:57 +00:00
Christian Decker 67fc8ffbcf elements: Use chainparams when the test relies on addresses
We were checking against hardcoded hrp and prefixes. Now we parametrize via
the chainparams.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker f078941a3d pytest: Do not edit the global copy of the bitcoind config
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 9025a3cd39 pytest: Allow switching chaind depending on a config option
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 13ba5b3d20 pytest: Add an ElementsD subclass of BitcoinD to control Elements
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 3abd4801f0 pytest: Make the config section configurable
Since we will soon be writing the `liquid-regtest` section instead of the
`regtest` section we should make that configurable.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 25c830a533 pytest: Have bitcoind remember its config file
There were a few places we were rebuilding the config path by appending
`bitcoin.conf` to the bitcoin directory. So now we just remember it and
reference it instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell 025b0e6a09 pytest: add helper to send node a gossip query and get response(s).
Note the use of sqrt, which makes a 13 second timeout under Travis
(180 second), or 7 seconds normally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Christian Decker 96a22b4003 pytest: Add db_provider and db instances for configurable backends
We will soon have a postgres backend as well, so we need a way to control the
postgres process and to provision DBs to the nodes. The two interfaces are the
dsn that we pass to the node, and the python query interface needed to query
from tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell 147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 9aca38b279 pytest: increase non-developer timeout.
We overrode this with travis, but it's not a travis issue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell 714e7fb670 pytest: test db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
lisa neigut df1d92a7a2 test-fixtures: return error set, don't throw exception
Throwing an exception while killing all nodes meant that
we aren't cleaning up all the nodes properly. Instead,
collect the errors, and return them back to the upper level,
where we report them and terminate as expected.
2019-09-05 01:32:48 +00:00
Christian Decker 245dff26e0 pytest: Updated requirements.txt to latest versions
This is an issue that was raised in #2665: some of the dependencies where
causing warnings to be added to the logs about deprecated dependencies. Since
I did not get these warnings I just blanket updated all the dependencies in
the hopes of getting the warnings to resolve.

Signed-off-by: Christian Decker <@cdecker>
2019-09-02 16:31:36 +02:00
Rusty Russell 6310a183af pytest: ignore occasional bad gossip in reorg tests.
It can happen, as expected when we reorg and update for a now-invalid
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 854e3c5303 pytest: wait for sync by default when starting lightningd.
Otherwise we get some spurious test failures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell 54ce4ed1cf pytest: fail tests if we get any LOG_BROKEN level messages, unless flagged.
And clean up some dev ones which actually happen (mainly by calling
channel_fail_permanent which logs UNUSUAL, rather than
channel_internal_error which logs BROKEN).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
lisa neigut 1b2a593b05 tests: default all addresses to bech32
Needed for composing a transaction externally to c-lightning, using
bitcoind util.
2019-06-12 02:22:43 +00:00
Rusty Russell 65505adbab pytest: test that we unreserve txprepare inputs across shutdown/crash.
We fail this at the moment, since we rely on shutdown to do the cleanups
for us.

(Also had to fix the unclean shutdown path: the caller checks the rc unless
 mayfail is set, and of course it's not zero since we just SIGTERM'd it).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Michael Schmoock dd87024060 test: adds wait mempool param to generate_block 2019-06-06 00:04:59 +00:00
Simon Vrouwe 7726681aa6 pytest: added BitcoinD.simple_reorg() method, which can change tx height and/or txindex
Now without bitcoind restart.
bitcoin-cli `prioritisetransaction` came to the rescue!
Its argument `fee_delta` (apparently) lowers the txs _effective_ feerate
soo low that bitcoind wont mine it ... untill we raise it when we want
it to be mined.
2019-05-27 13:29:32 +02:00
Simon Vrouwe 24f43e5910 pytest: fundwallet tighten up, make sure we see `CONFIRMED` in log line 2019-05-27 13:29:32 +02:00
Christian Decker a0fc915674 pytest: Start the Bitcoin RPC proxy as soon as it is created
Avoids attempting to rebind on restart.
2019-05-26 16:19:10 +02:00
Rusty Russell d054fef0f0 pytest: fix test_bech32_funding
We try to look up the funding tx, but it's already spent that to fund
the channel, so we need txindex if this test is to work reliably.

It's not clear to me why this *ever* worked, but if fails on my new
ThreadRipper build machine with valgrind:

  >	wallettx = l1.bitcoin.rpc.getrawtransaction(wallettxid, True)
...
  E           bitcoin.rpc.InvalidAddressOrKeyError: {'code': -5, 'message': 'No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions.'}

  /usr/lib/python3/dist-packages/bitcoin/rpc.py:231: InvalidAddressOrKeyError
 
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-14 04:58:52 +02:00
Christian Decker bc5dbb6a80 pytest: Ensure lightningd instances have the correct numeric ids
For performance reasons we start the lightningd instances in
parallel. However, if we only assign the numeric ids (used for log-prefixes
and home directories) when we are already running in parallel, we are not
guaranteed to get the numeric ids matching the return value of `get_nodes` or
`line_graph`. With this patch we now select numeric ids before parallelizing
the start.

Signed-off-by: Christian Decker <@cdecker>
2019-05-04 04:47:06 +00:00
Rusty Russell 049f2351d4 newaddr: deprecate 'address' output.
It doesn't make sense with 'all', and it's ok to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell 7887e5c00a pytest: add support for starting nodes with a pre-canned db.
With xz, the db is only 9120 bytes, vs 163840.  And lzma is a builtin
in Python 3.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00