Commit Graph

8431 Commits

Author SHA1 Message Date
Rusty Russell 4a9c162211 pytest: disable test_mpp_interference_2.
We're failing this too often: we'd fail it more but it's disabled
with VALGRIND (it shouldn't be: @slow_test removes VALGRIND if SLOW_MACHINE
is set).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 13:51:22 +02:00
niftynei c8579b99d0 pyln-proto: use vals for subtype parsing 2020-08-27 10:20:16 +09:30
Christian Decker 21d87f7075 pay: Implement simple presplit fix for ludicrous amounts
This is the simplest possible fix: increase the target amount until we get
the desired number of parts, while still bucketizing payments together that
are in approximately the same size.

The current logic puts all payments that are in the range x < amount <= 16*x
in the same bucket, making them harder to distinguish.

Changelog-Fixed: pay: The `presplit` modifier now supports large payments without exhausting the available HTLCs.
2020-08-27 10:19:21 +09:30
Christian Decker 3f6809c934 pytest: Reproduce ludicrous presplit issue
Huge payment + presplit modifier = too many HTLCs...
2020-08-27 10:19:21 +09:30
Christian Decker f5985171bd gossipd: Make gossipd much quieter
We are logging way too much from gossipd, causing noisy logs. This PR reduces
logs for incoming messages to those that actually caused a change in our
internal state (duplicate and old messages are just dropped silently now).

Changelog-Changed: gossipd: The `gossipd` is now a lot quieter, and will log only when a message changed our network topology.
2020-08-27 09:45:35 +09:30
Christian Decker bdad5a6f94 pytest: Stabilize test_onchain_middleman
It wasn't waiting for the tx to be in the mempool
2020-08-27 09:45:35 +09:30
Christian Decker 8f028272f4 pytest: Stabilize test_pay_get_error_with_update 2020-08-27 09:45:35 +09:30
Christian Decker 437c411769 pyln-testing: Have LightningNode.fund_channel poll 2020-08-27 09:45:35 +09:30
Christian Decker 0431a29a26 pyln-testing: Poll for channel active instead of watching logs 2020-08-27 09:45:35 +09:30
Christian Decker 8f05d28049 pytest: Stabilize test_onchaind_replay 2020-08-27 09:45:35 +09:30
ZmnSCPxj jxPCSnmZ 128adf0938 plugins/libplugin-pay.c: Round-robin routehints when splitting.
This improves the success rate of `test_mpp_interference_2`, though
still not quite up to the level that we can remove `@flaky` from it.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ ba3f38077e lightningd/invoice.c: Use round-robin channel selection.
Changelog-Changed: We now make MPP-aware routehints in invoices.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ a9a11265dd lightningd/lightningd.h: Maintain a round-robin list of channels. 2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ f50951a0d6 tests/test_pay.py: Motivating scenario for round-robin routehinting. 2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ 8c763bb7be contrib/pyln-testing/pyln/testing/utils.py: Add a fundbalancedchannel method to LightningNode, for when you want to Thanos your channels. 2020-08-26 09:29:46 +09:30
Sergi Delgado Segura 3edbacbb3c pyln: hexlify -> bytes.hex() in pyln.proto.wire 2020-08-26 06:09:53 +09:30
Sergi Delgado Segura 27e495efa5 pyln: Updates proto to use coincurve for Public and Private keys 2020-08-26 06:09:53 +09:30
ZmnSCPxj jxPCSnmZ c27d7a3110 plugins/libplugin-pay.c: Store the route description, and re-report on failure. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 6468616c02 plugins/libplugin-pay.c: Propagate local_id from parent to child payment object. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ d8678467fa plugins/libplugin-pay.c: Show routes being tried, also print updates to channel hints not just initial creations. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ d89c77c0ce plugins/libplugin-pay.c: Describe the bits of unrecognized failure codes. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 0d2d85ab5f plugins/libplugin-pay.c: Also print events that create new sub-payments. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ 98583e84b5 plugins/libplugin-pay.c: Give cmd id and partid for each log message.
Changelog-None: internal debugging

Makes it easier to debug payments with tons of splits.
2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ e80d733399 .gitignore: Add missing gitignores. 2020-08-25 12:10:05 +02:00
Rusty Russell dc8458d1e2 tools: fix comment in template.
It's (usually) a .csv not an _csv file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 398b4806b9 connectd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell fe8564555f closingd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 6d09c0eba9 openingd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 8ae6740bb1 Makefile: create generic wiregen rules, use for channeld.
This means some files get renamed, and I took the opportunity to clarify
our naming (the *d* is important!)

1. channeld/channel_wire.csv -> channeld/channeld_wire.csv
2. channeld/gen_channel_wire.h -> channeld/channeld_wiregen.h
3. enum channel_wire_type -> enum channeld_wire
4. WIRE_CHANNEL_FUNDING_DEPTH -> WIRE_CHANNELD_FUNDING_DEPTH.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 7804d89ef1 Makefile: don't output on stderr if file not yet checking into git.
And if both aren't checked in, consider them changed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell b38b913694 Makefile: reduce clutter if they haven't run ./configure
It's hard to see the error:

Before:

    $ make
    File config.vars not found: you must run ./configure before running make.
    CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning"    -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include    -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS   -DBUILD_ELEMENTS=1 -c -o
    LD: cc  -fsanitize=address     -Lexternal/x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/lib -lm -lgmp -lsqlite3 -lz  -o

After:

    $ make
    File config.vars not found: you must run ./configure before running make.
    make: *** No rule to make target 'config.vars', needed by 'show-flags'.  Stop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell b9ac032329 doc: force refresh of all manpages.
The idea is that you regenerate the man pages in the same commit you
alter them: that's how we know whether to try regenerating them or not
(git doesn't store timestamps, so it can't really tell).

Travis will now check this, so force them all to sync to this commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 67cde96397 Makefile: respect --quiet and V=1 in doc/Makefile.
And don't regen doc/index.rst unless there's good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
niftynei 6f114e06a6 bugfix: save max_witness_len as big-endian 2020-08-25 12:34:56 +09:30
niftynei 0305c0a57e psbt: if psbt is null, return null for get_bytes 2020-08-25 12:34:56 +09:30
niftynei df7c122cb8 json: add "json-to-psbt" helper
Pull a PSBT out of a json buffer
2020-08-25 12:34:56 +09:30
niftynei 10237ccb8e psbt: handle 'NULL' or empty byte PSBTs
Don't crash if the PSBT is null / zero-byte len.
2020-08-25 12:34:56 +09:30
Rusty Russell ef3fbab551 invoice: don't allow zero-value invoices.
You can't pay them anyway, and at least one person used 0 instead of "any".

Closes: #3808
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice` no longer accepts zero amounts (did you mean "any"?)
2020-08-25 12:22:45 +09:30
nicolas.dorier 2e51f23a95 Add python to runtime docker image (arm32/64) 2020-08-24 20:23:25 +02:00
Rusty Russell 57dd5be2fd gossipd: prune channels unless *both* peers have refreshed.
See https://github.com/lightningnetwork/lightning-rfc/pull/767

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: channels now pruned after two weeks unless both peers refresh it (see lightning-rfc#767)
2020-08-24 20:55:47 +09:30
Rusty Russell 7ad8fde060 bolt11: update ctlv expiry, always write it.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: protocol: bolt11 invoices always include CLTV fields (see lightning-rfc#785)
2020-08-24 20:55:47 +09:30
Rusty Russell b09e519669 options: be more conservative with timeouts on mainnet.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: config: the default CLTV expiry is now 34 blocks, and final expiry 18 blocks as per new BOLT recommendations.
2020-08-24 20:55:47 +09:30
Rusty Russell 6e4e267f2c doc: update BOLT now option_anchor_outputs is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell 007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell 95294e25ea wallet: do valgrind checks when binding db statements.
Otherwise valgrind gets upset when we *run* the statements: better
to get a backtrace when we bind, so we can tell which field it is!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 11:25:51 +09:30
Rusty Russell f762f7e247 db: remove assertion on validity of node_ids read from db.
We've never hit this, we do check them on insert, and it's slowing
down some operations unnecessarily.

$ time lightning-cli -R --network=regtest --lightning-dir /tmp/ltests-k8jhvtty/test_pay_stress_1/lightning-1/ listpays > /dev/null

Before:
	real	0m1.781s
	user	0m0.127s
	sys	0m0.013s

After:
	real	0m1.545s
	user	0m0.124s
	sys	0m0.024s

Also, the raw listsendpays drops from 0.983s to 0.676s.

(With -O3 -flto, listsendpays is 0.416s, listpays 0.971s).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell 28884864d0 plugins/pay: don't bother parsing nodeid for summary.
We have sanity checks in there that it's a valid point.  Simply store
the JSON token like we do with others.

time lightning-cli -R --network=regtest --lightning-dir /tmp/ltests-k8jhvtty/test_pay_stress_1/lightning-1/ listpays > /dev/null

Before:
	real	0m2.054s
	user	0m0.114s
	sys	0m0.024s

After:
	real	0m1.781s
	user	0m0.127s
	sys	0m0.013s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30