Commit Graph

123 Commits

Author SHA1 Message Date
Rusty Russell ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
darosior 7dac7a8c52 pytest: deactivate shadow routing for some tests that use 'pay'
So that we can assert payment values
2019-11-08 03:27:58 +00:00
lisa neigut 181e0d4982 test: amend 'behind sync' tests
Check explicitly for both 'fundchannel' case and funding a transaction;
remove unnecessary 'second funding'.
2019-11-04 17:52:48 +01:00
Rusty Russell ab48ecf07c pylightning: wrap signmessage/checkmessage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 17:15:23 +02:00
Rusty Russell e5d9c7effc lightningd: checkmessage can intuit pubkey in some cases.
*If* we know the key has signed something else (as is the case for
channel_announcement) then we can effectively trust the key derivation.

This matches how LND's VerifyMessage works, though in the next patch
we will document it exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell 150598e061 tests: add test for signmessage and lightning-checkmessage.
Thanks Twitter helpers @duck1123 and @jochemin for tests!
And @bitconner for the initial test vector.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell ca53c1b699 gossipd: push our own gossip messages harder.
I had a report of a 0.7.2 user whose node hadn't appeared on 1ml.  Their
node_announcement wasn't visible to my node, either.

I suspect this is a consequence of recent version reducing the amount of
gossip they send, as well as large nodes increasingly turning off gossip
altogether from some peers (as we do).  We should ignore timestamp filters
for our own channels: the easiest way to do this is to push them out
directly from gossipd (other messages are sent via the store).

We change channeld to wrap the local channel_announcements: previously
we just handed it to gossipd as for any other gossip message we received
from our peer.  Now gossipd knows to push it out, as it's local.

This interferes with the logic in tests/test_misc.py::test_htlc_send_timeout
which expects the node_announcement message last, so we generalize
that too.

[ Thanks to @trueptolmy for bugfix! ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 15:00:37 -05:00
darosior 43e55fc8e4 pytest: test lightningd startup with a relative config dir 2019-10-08 22:43:30 +00:00
Christian Decker e23d8a4aff pytest: Mark some tests as skippable that are too bitcoin-specific 2019-10-03 04:32:57 +00:00
Christian Decker 858b3f2b93 pytest: Mark tests using network specific addresses under elements
We may be able to re-enable them later, but this is just painful right now.
2019-10-03 04:32:57 +00:00
Christian Decker 8a69ea7bf5 elements: Parametrize the sample addresses we test against
Since elements addresses look quite different from the bitcoin mainnet
addresses I just added a sample to the chainparams fixture. In addition I
extracted some of the fixed strings to reference chainparams instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker e4c343a386 elements: Search for the correct address and invoice prefixes
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 67fc8ffbcf elements: Use chainparams when the test relies on addresses
We were checking against hardcoded hrp and prefixes. Now we parametrize via
the chainparams.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker cd93a855ce elements: Give pytest some information about the test chain
We are checking against chain-dependent constants, so let's make sure we are
using the ones for the correct chain.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell 15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
Rusty Russell 4e8141aa36 features: remove INITIAL_ROUTING_SYNC.
It only had an effect if the peer didn't support option_gossip_queries, but
still, we don't want a gossip blast any more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Christian Decker efc4aa94a3 pytest: Skip some tests that assume we have a sqlite3 db on postgres
These will not work since they touch the DB file itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 96a22b4003 pytest: Add db_provider and db instances for configurable backends
We will soon have a postgres backend as well, so we need a way to control the
postgres process and to provision DBs to the nodes. The two interfaces are the
dsn that we pass to the node, and the python query interface needed to query
from tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell 27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
lisa neigut 904a1384ef tests: de-flake lightningd_still_loading 2019-09-11 23:56:27 +00:00
Rusty Russell 226e2aee48 option_static_remotekey: update to latest draft.
531c8d7d9b

In this one, we always send my_current_per_commitment_point, though it's
ignored.  And we have our official feature numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Christian Decker 245dff26e0 pytest: Updated requirements.txt to latest versions
This is an issue that was raised in #2665: some of the dependencies where
causing warnings to be added to the logs about deprecated dependencies. Since
I did not get these warnings I just blanket updated all the dependencies in
the hopes of getting the warnings to resolve.

Signed-off-by: Christian Decker <@cdecker>
2019-09-02 16:31:36 +02:00
Rusty Russell 6901732ee0 lightningd: create --list-features-only which lists what features we support.
This allows the lightning-rfc protocol tests to automatically query what
features we support.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 6310a183af pytest: ignore occasional bad gossip in reorg tests.
It can happen, as expected when we reorg and update for a now-invalid
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 854e3c5303 pytest: wait for sync by default when starting lightningd.
Otherwise we get some spurious test failures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell ca28c30eff funding: don't allow funding new channels until we're synced.
This is probably worth preventing.

1. Our depth estimate would be inaccurate possibly leading to us
   timing out too early.
2. If we're not up-to-date our onchain funds are unknown.
3. We wouldn't be able to send or receive HTLCs until we're synced anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell c3a35416da lightningd: don't allow channeld to accept HTLCs if we're not synced.
We want to still allow incoming connections, and reestablishment of
channels, but if one tries to give us an HTLC, stall until we're
synced.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell 6195a878f7 lightningd: don't allow sending of HTLCs while still syncing.
If we don't know block height, we shouldn't be sending HTLCs.  This
stops us forwarding HTLCs as well as new payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell 3eebd0cc20 lightningd: add flag for whether we're synced, and callback infrastructure.
We consider ourselves synced when bitcoind is synced and we're synced
with that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell faded9a9cf bitcoind: detect when it's still syncing, add field to getinfo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell 118150227e cli: restore 0.7.0-style whitespace printing.
@renepickhardt has a shell script we broke.  While we still produce
perfectly valid JSON, we should not gratuitously change tool output.

Plus, I prefer the missing space before the ':'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:16:48 +08:00
Rusty Russell b73a85a75e lightningd: don't say 'killing channel' when HTLC times out.
We're actually only killing the connection.  I saw this in my logs,
but it was all OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-07 21:12:52 +08:00
Rusty Russell 979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell fc024f81d6 pytest: check that --daemon still means we recognize our own children.
We create our children then fork, so we're not a parent.  I noticed this
because 'lightning-cli stop' takes a long time: this is because it tries to
wait for them and they don't respond.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell d59e2b1b4b developer: add --dev-force-bip32-seed to force a specific BIP32 seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell fb6870c139 param: implement helpers for multiplex commands.
Our previous param support was a bit limited in this case.

We create a dev- command multiplexer, so we can exercise it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-16 21:39:14 +00:00
Rusty Russell 54ce4ed1cf pytest: fail tests if we get any LOG_BROKEN level messages, unless flagged.
And clean up some dev ones which actually happen (mainly by calling
channel_fail_permanent which logs UNUSUAL, rather than
channel_internal_error which logs BROKEN).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
Rusty Russell d817735dc2 lightningd: use ccan/json_out.
This is now a fairly simple transition, which only effects the internals
of json_stream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 26cdf9d3dc plugins/pay: don't retry routehint if it contains already-eliminated channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 260febd88b plugins/pay: fix attempt counter on failure message.
An "attempt" is when we actually try to send, not every route lookup
we do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Simon Vrouwe ea8a508a48 pytest: cleanup test_funding_reorg_remote_lags
- mock_rpc function now returns full JSON-RPC response, is much cleaner
- Since reached_announce_depth counting is fixed when starting
  channeld, we don't need the 7th block to tell depth anymore.
2019-06-07 10:34:34 +02:00
Rusty Russell 0543149b89 pytest: move wallet tests into new file tests/test_wallet.py
We're going to add some more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Michael Schmoock c40250ad0b test: adds wait mempool to flaky tests 2019-06-06 00:04:59 +00:00
Rusty Russell ba036b767f newaddr: fix return value for deprecated "address" field.
It would always return bech32; fix that, and don't bother printing
it if they use the (new) 'all' parameter.

This API was introduced in 3e67c09d5e,
which means it wasn't in a release so no CHANGELOG entry necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-05 15:38:42 -07:00
Michael Schmoock 107381aca2 test: fix flakyness of funding_reorg_private openchannel_hook 2019-06-03 19:30:19 +02:00
ZmnSCPxj 48df6c8566 lightningd/io_loop_with_timers.c: Move mainloop to its own source file, have chaintopology use it.
Fixes: #2687
2019-05-31 17:57:10 +02:00
Christian Decker 883d4b4e6e pytest: Add a test that reproduces #2687
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-05-31 17:57:10 +02:00