Commit Graph

6665 Commits

Author SHA1 Message Date
trueptolemy 74055eada5 pytest: Test `txprepare` with multiple outputs 2019-09-05 16:05:36 -05:00
trueptolemy 42d5d753db cleanup: Remove unused structure `struct withdrawal` 2019-09-05 16:05:36 -05:00
trueptolemy cdcafdaf74 API: `txprepare` now support mutiple outputs 2019-09-05 16:05:36 -05:00
trueptolemy 0ab3817715 common: Fix a memleak in `withdraw_tx` 2019-09-05 16:05:36 -05:00
trueptolemy b660531216 common: `withdraw_tx()` now use the array of `struct bitcoin_tx_output` as parameter 2019-09-05 16:05:36 -05:00
trueptolemy 6570c743c4 bitcoin: Add the interface to add multiple outputs to bitcoin transaction 2019-09-05 16:05:36 -05:00
darosior 4672795e1a lightningd: don't join the config_dir to the pid_file path 2019-09-05 19:10:33 +02:00
lisa neigut 10ed2ebdb4 wallet: track our change outputs
Add change scripts to our txfilter so that we mark them as confirmed
(and can subsequently spend them)
2019-09-05 15:48:49 +02:00
lisa neigut 01e0ba8232 test: add check that withdraw's change is confirmed
We're not tracking change outputs for withdraws, so they're not
confirming. This test exposes the problem.
2019-09-05 15:48:49 +02:00
lisa neigut 4030ccf94c fix memleak: remove duplicated new_channel creation
We call this same exact code inside `funder_finalize_channel_setup`;
which results in a memleak since we don't free the previous before
clobbering it.
2019-09-05 01:32:48 +00:00
lisa neigut 4d2f91f7ae test-fixtures: use helper for checking errors
log files were being deleted on memleak errors, since
we weren't marking the node has having an error.

this helper function is designed to exactly handle this, so
we use the helper function and modify it to print any additional
error messages that are handed back from killall.
2019-09-05 01:32:48 +00:00
lisa neigut df1d92a7a2 test-fixtures: return error set, don't throw exception
Throwing an exception while killing all nodes meant that
we aren't cleaning up all the nodes properly. Instead,
collect the errors, and return them back to the upper level,
where we report them and terminate as expected.
2019-09-05 01:32:48 +00:00
lisa neigut 63c80e7aa9 test-fixtures: move 'broken' check to after memleak
Memleaks appear in the logs as 'broken', so the broken log
check captures them as well. This moves broken to after memleak
so we get more informative error messages.
2019-09-05 01:32:48 +00:00
lisa neigut 5663ecc599 listfunds: add 'blockheight' for confirmed transactions
Needed to calculate the value of all inputs for 'all' in
externalized fundchannel
2019-09-04 14:08:08 +02:00
Michael Schmoock 76f27f47dc doc: HACKING - point out how to run the pylightning tests 2019-09-03 19:02:04 +02:00
Michael Schmoock bc5add399b pylightning: adds a millisatoshi test class 2019-09-03 19:02:04 +02:00
Michael Schmoock c0069d8944 pylightning: adds Millisatoshi __radd__ method for sum() 2019-09-03 19:02:04 +02:00
darosior 0e2ec27576 doc/requirements.txt: update 'mrkd' dependency 2019-09-02 16:31:36 +02:00
Christian Decker 7fafd934cf pytest: Remove --duration argument
We're not going to optimize travis any longer so let's no fill logs with
useless measurements.
2019-09-02 16:31:36 +02:00
Christian Decker fd1e8a5999 pytest: Fix directory deletion on passed tests
We were checking the test request against the searched for string. This fixes
it by actually looking at the outcome instead and should clean up correctly
if tests do not fail.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-02 16:31:36 +02:00
Christian Decker 245dff26e0 pytest: Updated requirements.txt to latest versions
This is an issue that was raised in #2665: some of the dependencies where
causing warnings to be added to the logs about deprecated dependencies. Since
I did not get these warnings I just blanket updated all the dependencies in
the hopes of getting the warnings to resolve.

Signed-off-by: Christian Decker <@cdecker>
2019-09-02 16:31:36 +02:00
Christian Decker c3254e6639 docs: Update manpages to be recognized correctly by mrkd
mrkd started enforcing the `name -- short description` style of top-level
headings somewhere, and was thus failing to build the man-pages. I swapped
the title and with the existing short description to make it work
again. `mrkd` will automatically infer the section from the filename so no
need to put it in the title as well.

In addition I removed the "last updated" lines at the bottom since they are
out of date at best, and misleading at the worst. If we want to keep them, I'd
suggest generating them from the commit that last touched them.
2019-09-02 16:31:36 +02:00
Rusty Russell 049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell f0ab4d72da pytest: add test that htlc_accepted hook only gets called after init.
We fail this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
nicolas.dorier 49496ab3ba [Docker] Use stretch-slim instead of Alpine 2019-08-30 16:17:51 +02:00
darosior 8f8e955421 Update installation instructions and Dockerfile
Get rid of the 'asciidoc' dependency, make developers also install 'doc/requirements.txt'
2019-08-30 00:34:11 +00:00
darosior 092f20aeeb README: Replace broken links to manpages 2019-08-30 00:34:11 +00:00
Rusty Russell 3ef5160fa7 travis: add --enable-experimental-features (build-only) test.
Also, add a 32-bit build-only test, and move all the build-only tests
to the top.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 15:51:36 +02:00
Rusty Russell 02b9b7f6e6 tests: update mocks for --enable-experimental-features builds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 15:51:36 +02:00
Christian Decker b77270fa46 docs: Add the title of the manpage to TOC and sort alphabetically
Fixes #2987
2019-08-29 14:56:24 +02:00
Rusty Russell 5e3e2f4e17 run-secret_eq_consttime: try harder.
This will never be reliable under high load, without making it unable
to detect real errors.  But the test is useful because if we don't
have this test we'll never notice if we break the const-timedness of
our implementation.

So, move the calloc out of the test loop (which seems to make it more
reliable), and then after we've run it, check the 1-minute load
average.  Too high, we don't complain about results.  It's not
perfect, but it's better.

Running 100 times (-O3) serially gave 100 successes with the following results:
	Constant: Within 5% 562-926(832.89+/-73)/1000 times
        Non-constant: More than 5% slower 860-990(956.35+/-26)/1000 times

More importantly, if we swap the const and non-const tests, we get
the expected 100 failures:
	Non-constant: Within 5% 14-79(41.17+/-14)/1000 times
	Constant: More than 5% slower 44-231(111.89+/-33)/1000 times

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 6901732ee0 lightningd: create --list-features-only which lists what features we support.
This allows the lightning-rfc protocol tests to automatically query what
features we support.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 17edcfa24d features: don't assume we'll always only advertize optional features.
Generalize things a bit so OPTIONAL_FEATURE() and COMPULSORY_FEATURE()
work with either odd or even features, then explicitly use OPTIONAL_FEATURE
in our internal feature array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell a38131f349 EXPERIMENTAL: advertize the extended_queries feature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell d943d8abbc lightningd: expose full onion error when we have it.
Mainly useful for testing.  In particular, we don't save it to the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 189b2f1313 BOLT: update CSV to latest bolt version.
This removes the WIRE_FINAL_EXPIRY_TOO_SOON which leaked too much info,
and adds the blockheight to WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 2600a6ed2e channeld: get current block height when an HTLC fails.
We need it to put in the error code for
WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 6349222ea2 Spec: Update to latest BOLT, include our first global feature definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 6310a183af pytest: ignore occasional bad gossip in reorg tests.
It can happen, as expected when we reorg and update for a now-invalid
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 20a2bf9547 pytest: make test_forward_event_notification more explicit.
We seem to be getting intermittant failures, but it's hard
to disgnose.  Simplify it by moving all the test logic into
the test itself, and making the plugin dumber.  This means we'll
see exactly what the differences are if it fails again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell dbc1ebe06a Makefile: fix race when checking manpages.
By not depending on plugins, we can call lightningd before plugins are
built, and not get all options:

    diff of command names vs manpage names:
    --- /dev/fd/63  2019-08-22 05:06:55.265659216 +0000
    +++ /dev/fd/62  2019-08-22 05:06:55.265659216 +0000
    @@ -3,6 +3,8 @@
     allow-deprecated-apis=
     always-use-proxy=
     announce-addr=
    +autocleaninvoice-cycle=
    +autocleaninvoice-expired-by=
     autolisten=
     bind-addr=
     bitcoin-cli=

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 394aef2391 Makefile: fix race with 'make check'
installcheck calls $(MAKE) under the covers, which can race with the
current builds, and we can try installing something which is still
being built:

    collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
    compilation terminated.
    /usr/bin/ld: can not read symbols: File truncated
    /usr/bin/ld: .eh_frame/.stab edit: File truncated
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.interp' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.ABI-tag' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.gnu.build-id' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.hash' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynsym' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynstr' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version_r' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.dyn' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.text' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rodata' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.eh_frame' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data.rel.ro' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynamic' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_json_command' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_hooks' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_type_to_string' not in segment
    Makefile:390: recipe for target 'lightningd/lightningd' failed
    make[1]: *** [lightningd/lightningd] Error 1
    make[1]: Leaving directory '/home/rusty/lightning'
    Makefile:553: recipe for target 'installcheck' failed
    make: *** [installcheck] Error 2

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 3f1ca18f2f pytest: optimize test_closing_torture
VALGRIND=1, SLOW_MACHINE=0:
Before: 197.74 seconds
After: 135.43 seconds

Note that we now spend about 13 seconds in teardown, could probably
be optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 686df235a2 pytest: fix race in test_block_backfill.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 11833192a1 tools: fix Makefile warning
Makefile:424: warning: overriding recipe for target 'clean'
    tools/Makefile:12: warning: ignoring old recipe for target 'clean'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 28185c397c gossipd: fix gossip send in case query_flags cause no output.
Fortunately, again, only happens with EXPERIMENTAL_FEATURES.

If the query causes us not to actually send anything, we won't
get called again.  This can validly happen if they only asked for
the node_announcements, for example.

(Found by protocol tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-27 12:35:25 +02:00
Rusty Russell 855dff704c gossipd: test crc32 routines using test vectors from PR.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-27 12:35:25 +02:00
Rusty Russell d1a1592cc8 gossipd: fix calculation of crc32 of update.
Currently EXPERIMENTAL_FEATURES only, fortunately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-27 12:35:25 +02:00
Simon Vrouwe 5b5d70d640 lightningd/plugin: extend plugins->startup window to include plugins_config
Fixes incorrect configuration[`startup`] in plugin `init`, modified tests
to test this.
2019-08-27 00:02:20 +00:00
Rusty Russell c2f58aec9d contrib/plugins: cowsay
___________________________________________________________________________
/ The most important plugin. Ever. Much thanks to @practicalswift and @jb55 \
\ for their improvements!                                                   /
 ---------------------------------------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-26 23:10:36 +00:00