Commit Graph

190 Commits

Author SHA1 Message Date
Rusty Russell 839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Rusty Russell 5d413e0f79 common: offer option_basic_mpp for EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 8393d21503 common/features: add `payment_secret` feature if EXPERIMENTAL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Christian Decker d58339b872 pytest: Migrate to in-tree pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker b7e7e535c8 pytest: Migrate `env` to pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker f6a016c17d pytest: Consolidate access to environment variables
We use `env()` to look up configuration variables and allow them to be
overridden by the environment.
2019-11-12 21:23:55 +01:00
Rusty Russell a3f6ce1f8f pytest: add filters arg to query_gossip()
Code changes mean we're going to get gossip_timestamp_filter messages from
peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
darosior c9982f9f49 pytest: allow to set stdin, stdout and stderr at lightningd startup
And also allow to not wait for it to be started.
Passing stderr=subprocess.STDOUT can be useful to wait_for_log() also on
stderr messages.
2019-10-09 22:00:38 -05:00
Christian Decker 37fb912524 pytest: Exclude elements-cli from valgrind when following children
Without this we'd be attaching valgrind to all elements-cli call, resulting in
a huge overhead and timeouts.
2019-10-03 04:32:57 +00:00
Christian Decker 67fc8ffbcf elements: Use chainparams when the test relies on addresses
We were checking against hardcoded hrp and prefixes. Now we parametrize via
the chainparams.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker f078941a3d pytest: Do not edit the global copy of the bitcoind config
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 9025a3cd39 pytest: Allow switching chaind depending on a config option
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 13ba5b3d20 pytest: Add an ElementsD subclass of BitcoinD to control Elements
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 3abd4801f0 pytest: Make the config section configurable
Since we will soon be writing the `liquid-regtest` section instead of the
`regtest` section we should make that configurable.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 25c830a533 pytest: Have bitcoind remember its config file
There were a few places we were rebuilding the config path by appending
`bitcoin.conf` to the bitcoin directory. So now we just remember it and
reference it instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell 025b0e6a09 pytest: add helper to send node a gossip query and get response(s).
Note the use of sqrt, which makes a 13 second timeout under Travis
(180 second), or 7 seconds normally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Christian Decker 96a22b4003 pytest: Add db_provider and db instances for configurable backends
We will soon have a postgres backend as well, so we need a way to control the
postgres process and to provision DBs to the nodes. The two interfaces are the
dsn that we pass to the node, and the python query interface needed to query
from tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell 147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 9aca38b279 pytest: increase non-developer timeout.
We overrode this with travis, but it's not a travis issue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell 714e7fb670 pytest: test db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
lisa neigut df1d92a7a2 test-fixtures: return error set, don't throw exception
Throwing an exception while killing all nodes meant that
we aren't cleaning up all the nodes properly. Instead,
collect the errors, and return them back to the upper level,
where we report them and terminate as expected.
2019-09-05 01:32:48 +00:00
Christian Decker 245dff26e0 pytest: Updated requirements.txt to latest versions
This is an issue that was raised in #2665: some of the dependencies where
causing warnings to be added to the logs about deprecated dependencies. Since
I did not get these warnings I just blanket updated all the dependencies in
the hopes of getting the warnings to resolve.

Signed-off-by: Christian Decker <@cdecker>
2019-09-02 16:31:36 +02:00
Rusty Russell 6310a183af pytest: ignore occasional bad gossip in reorg tests.
It can happen, as expected when we reorg and update for a now-invalid
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 854e3c5303 pytest: wait for sync by default when starting lightningd.
Otherwise we get some spurious test failures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell 54ce4ed1cf pytest: fail tests if we get any LOG_BROKEN level messages, unless flagged.
And clean up some dev ones which actually happen (mainly by calling
channel_fail_permanent which logs UNUSUAL, rather than
channel_internal_error which logs BROKEN).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
lisa neigut 1b2a593b05 tests: default all addresses to bech32
Needed for composing a transaction externally to c-lightning, using
bitcoind util.
2019-06-12 02:22:43 +00:00
Rusty Russell 65505adbab pytest: test that we unreserve txprepare inputs across shutdown/crash.
We fail this at the moment, since we rely on shutdown to do the cleanups
for us.

(Also had to fix the unclean shutdown path: the caller checks the rc unless
 mayfail is set, and of course it's not zero since we just SIGTERM'd it).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Michael Schmoock dd87024060 test: adds wait mempool param to generate_block 2019-06-06 00:04:59 +00:00
Simon Vrouwe 7726681aa6 pytest: added BitcoinD.simple_reorg() method, which can change tx height and/or txindex
Now without bitcoind restart.
bitcoin-cli `prioritisetransaction` came to the rescue!
Its argument `fee_delta` (apparently) lowers the txs _effective_ feerate
soo low that bitcoind wont mine it ... untill we raise it when we want
it to be mined.
2019-05-27 13:29:32 +02:00
Simon Vrouwe 24f43e5910 pytest: fundwallet tighten up, make sure we see `CONFIRMED` in log line 2019-05-27 13:29:32 +02:00
Christian Decker a0fc915674 pytest: Start the Bitcoin RPC proxy as soon as it is created
Avoids attempting to rebind on restart.
2019-05-26 16:19:10 +02:00
Rusty Russell d054fef0f0 pytest: fix test_bech32_funding
We try to look up the funding tx, but it's already spent that to fund
the channel, so we need txindex if this test is to work reliably.

It's not clear to me why this *ever* worked, but if fails on my new
ThreadRipper build machine with valgrind:

  >	wallettx = l1.bitcoin.rpc.getrawtransaction(wallettxid, True)
...
  E           bitcoin.rpc.InvalidAddressOrKeyError: {'code': -5, 'message': 'No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions.'}

  /usr/lib/python3/dist-packages/bitcoin/rpc.py:231: InvalidAddressOrKeyError
 
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-14 04:58:52 +02:00
Christian Decker bc5dbb6a80 pytest: Ensure lightningd instances have the correct numeric ids
For performance reasons we start the lightningd instances in
parallel. However, if we only assign the numeric ids (used for log-prefixes
and home directories) when we are already running in parallel, we are not
guaranteed to get the numeric ids matching the return value of `get_nodes` or
`line_graph`. With this patch we now select numeric ids before parallelizing
the start.

Signed-off-by: Christian Decker <@cdecker>
2019-05-04 04:47:06 +00:00
Rusty Russell 049f2351d4 newaddr: deprecate 'address' output.
It doesn't make sense with 'all', and it's ok to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell 7887e5c00a pytest: add support for starting nodes with a pre-canned db.
With xz, the db is only 9120 bytes, vs 163840.  And lzma is a builtin
in Python 3.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Christian Decker b7222531fe pytest: Stabilize the test_pay_direct test
It was waiting for a remote channel, but not for all the interesting
channels we want to check. It can sometimes happen that further away
channels are added before closer ones are added, depending on
propagation path, flush timers and bitcoind poll timers. This now just
checks for all channels, which also reduces the ambiguity of whether
we selected a path solely because we were lacking alternatives.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-08 01:08:44 +00:00
Christian Decker 5a55972f1a pytest: Have bitcoind own its proxies
We were restarting the with the nodes before, which was causing some
port contention. This is more natural since `bitcoind` will take care
of terminating all proxies it returned.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Rusty Russell 139604a618 pytest: test that pay command retries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell 4c57bf002e pytest: neaten fund_channel scid extraction.
We used to have a bug where decoderawtransaction would fail, fixed in
fedcfd661 (pytest: hand 'True' to decoderawtransaction so it doesn't
get confused.).

So we can remove the fallback decode, and might as well extract the
ugliness into a helper function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell 9f1f79587e short_channel_id_dir: new primitive for one direction of short_channel_id
Currently only used by gossipd for channel elimination.

Also print them in canonical form (/[01]), so tests need to be
changed.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell 5b3abd80b1 pytest: don't skip valgrinding plugins.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Christian Decker 659a26ea5a misc: Update short_channel_id representation to use 'x' separators
Reported-by: Alex Bosworth <@alexbosworth>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-15 03:50:27 +00:00
Rusty Russell 61420bf79b configure: (mostly) revert b7a56f0531
Plugins are a first-class citizen again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:52:13 +00:00
Christian Decker b7a56f0531 plugin: Gate the plugin subsystem with the --enable-plugins configure flag
Since we are planning to release a bug fix release, and the plugin
subsystem is not yet complete, it is better to make plugin support
opt-in while we continue testing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-10 17:15:10 -08:00
Christian Decker 25ca49c444 pytest: Add a test for the event subscription and notification
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-30 14:36:02 +01:00
lisa neigut a39c97c960 channeld: support private channel creation, fixes #2125
Adds a new 'announce' field for `fundchannel`, which if false
won't broadcast a `channel_announcement`.
2018-12-08 15:15:55 -08:00
lisa neigut eab992cecd py-tests: rename 'announce' to 'wait_for_announce'
Better description of what the option actually does -- if true
waits for the announcement messages to be generated and exchanged.
2018-12-08 15:15:55 -08:00
Rusty Russell fab5027d64 configure: add --experimental-features flag (default: == DEVELOPER).
This will be used for option_simplified_commitment and other
still-being-specified options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Rusty Russell ac5002a79e pytest: add hack to force options into a given order.
Needed for testing plugin options which are order-senditive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-05 01:22:55 +01:00