Commit Graph

8298 Commits

Author SHA1 Message Date
Vincenzo Palazzo 1521c29fcf listpays mod 1: add destination inside the response when bolt11 is null
Changelog-Added: JSON-RPC: `listpays` now lists the `destination` if it was provided (e.g., via the `pay` plugin or `keysend` plugin)
2020-08-09 16:03:03 +02:00
Vincent 700897f06e listpays: fixed bolt11 null with keysend and update doc command
listpays: make doc-all missed
Changelog-Added: JSON-RPC: `listpays` can be used to query payments using the `payment_hash`
Changelog-Added: JSON-RPC: `listpays` now includes the `payment_hash`
2020-08-09 16:03:03 +02:00
Christian Decker 723b7223b7 pay: Add timestamp of first part to `listpays`
Changelog-Added: JSON-RPC: The result returned by `listpays` now includes the timestamp of the first part of the payment
2020-08-08 12:53:12 +02:00
Rusty Russell 01a82d38f7 pytest: add slow_test marker.
And when it's set, and we're SLOW_MACHINE, simply disable valgrind.

Since Travis (SLOW_MACHINE=1) only does VALGRIND=1 DEVELOPER=1 tests,
and VALGRIND=0 DEVELOPER=0 tests, it was missing tests which needed
DEVELOPER and !VALGRIND.

Instead, this demotes them to non-valgrind tests for SLOW_MACHINEs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 51aae9cce7 pytest: make valgrind a per-node option.
Next patch will turn it off for slow-marked tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 79278b880f pytest: optimize join_nodes a little.
We can query all the txids at once, rather than one at a time.

Doesn't make any measurable difference to full runtime testing here
though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell fde353ab00 pytest: use get_nodes more widely.
I started replacing all get_node() calls, but got bored, so then just did the
tests which call get_node() 3 times or more.

Ends up not making a measurable speed difference, but it does make some
things neater and more standard.

Times with SLOW_MACHINE=1 (given that's how Travis tests):

Time before (non-valgrind):
	393 sec (had 3 failures?)
Time after (non-valgrind):
	410 sec

Time before (valgrind):
	890 seconds (had 2 failures)
Time after (valgrind):
	892 sec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 929fd3e2f6 pytest: make sure all nodes see funds using sync_blockheight
We might have funds prior to calling join_nodes(), so testing that
we've all seen the block is better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 7bb461acd8 pytest: fix flake in test_channel_opened_notification
We can have the message before the node ready message which line_graph
waits for.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 7aa40b6bae pytest: fix assumption in test_onchain_their_unilateral_out
In fact, the 5 blocks generate above were not seen by nodes until
later, making me very confused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 02b413a4dc pytest: make join_nodes / line_graph wait for updates in both dirs.
This is what fund_channel() does, which is more thorough than what
we were doing.  But since the order of the logs is undefined, we need
to be a little careful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 8f455c8b98 pytest: new join_nodes to allow you to get all the nodes then join some of them.
This lets you get_nodes() and join some later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell 64d40414b2 pytest: speed up test_restart_many_payments.
I thought this was timing out because I made it slow with the
change to txprepare as a plugin.  In fact, it was timing out
because sometimes gossip comes so fast it gets suppressed
and we never get the log messags.

Still, before this it took 98 seconds under valgrind and
24 under non-valgrind, so it's an improvement to time as
well as robustness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
ZmnSCPxj jxPCSnmZ 6fadd5aea2 Makefile: Remove gen_version.h from ALL_GEN_HEADERS.
This prevents recompiling everything when you are changing just a doc, or
touching only one file among hundreds of sources, just because the
`gen_version.h` is changed, especially since only one source actually
depends on that header.
2020-08-07 10:41:37 +09:30
ZmnSCPxj jxPCSnmZ a5fc66c382 plugins/Makefile: Only link in libplugin-pay for plugins that need it.
Changelog-None: cleanup only.

Before this change:

```
$ ls -l plugins/bcli
-rwxrwxr-x 1 zmnscpxj zmnscpxj 1914976 Aug  5 21:54 plugins/bcli
```

After this change:

```
$ ls -l plugins/bcli
-rwxrwxr-x 1 zmnscpxj zmnscpxj 1830552 Aug  5 22:00 plugins/bcli
```

We already duplicate a lot of code between `lightningd` and eeach
builtin plugin because we do not use .so for `common/`, but including
an object file with code that is not referenced in the executable is
fairly low-hanging size optimization fruit.
2020-08-07 10:40:10 +09:30
Jon Griffiths 908a8399e2 psbt: Allocate correctly sized buffer for psbt_to_bytes
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths fef155a9e2 psbt: Use wallys function to check PST finalization status
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths 1fe53880a7 psbt: use new wally functions to add PSBT inputs/outputs
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths 68ffecba63 psbt: Use the newly exposed wally function to clone PSBTs
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths a51c6550ec psbt: avoid allocations when adding psbt outputs
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths 499395636c psbt: remove now-unneeded const casts
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths 21ee2c3a9c psbt: Remove workaround for now-fixed wally tx flag behaviour
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths 95d3d65c62 wally: update to the latest wally version
Includes:
psbt: Use renamed functions for new wally version
psbt: Set the transaction directly to avoid script workarounds
psbt: Use low-S grinding when computing signatures
tx: Use wally_tx_clone from libwally now that its exported

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Rusty Russell 06372e13d8 pyln.proto.message: don't let Message() init set implicit lengths.
We'll override them from field length anyway!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 17:53:25 +02:00
ZmnSCPxj jxPCSnmZ 4fde45669c plugins/pay.c: Fix the routehints/presplit conflict.
Changelog-Fixed: pay: Fixed a bug where routehints would be ignored if the payment exceeded 10,000 satoshi. This is particularly bad if the payee is only reachable via routehints in an invoice.
2020-08-06 17:36:58 +02:00
ZmnSCPxj jxPCSnmZ a9b992a94a tests/test_pay.py: Add test for conflict between presplit and routehints paymods. 2020-08-06 17:36:58 +02:00
Sjors Provoost 2e10729690 Add missing test dependencies to requirements.txt
Supplements the consolidation in b68066e
2020-08-06 09:46:04 +09:30
Rusty Russell fa829f23db amount: add amount_msat_scale, amount_msat_ratio, amount_{msat,sat}_div
It's not all that rare to do these operations, and requiring annotations
for it is a little painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell ffbb409b47 amount: use initializers everywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell 36d43b871f amount: add amount_msat and amount_sat initializers.
Generally, importing amounts needn't be checked, and it cuts down on
the warnings we get.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
niftynei 28b839f306 amount: add helper to convert u64 sats to amount_sat type 2020-08-06 09:36:47 +09:30
ZmnSCPxj jxPCSnmZ 1ded3fc52f lightningd/plugin.c: Add a `--dev-builtin-plugins-unimportant` for developers who want to mess around with the builtin plugins. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 1aa076845a tests/test_plugin.py: Test builtin plugins are important. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 4ca2e49812 lightningd/plugin.c: Make builtin plugins important.
Changelog-Changed: plugin: Builtin plugins are now marked as important, and if they crash, will cause C-lightning to stop as well.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 48f36904c8 tests/test_plugin.py: Add test for --important-plugin. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ a847487bbe lightningd/plugin.c: Add important plugins, which if they terminate, lightningd also terminates.
Changelog-Added: New option `--important-plugin` loads a plugin is so important that if it dies, `lightningd` will exit rather than continue.  You can still `--disable-plugin` it, however, which trumps `--important-plugin` and it will not be started at all.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 50600dce95 lightningd/lightningd.c: Create API to exit lightningd with an exit code. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 3df2333d5d lightningd/plugin.c: Add specific function to give the directory for built-in plugins. 2020-08-04 13:27:51 -05:00
Christian Decker 3dafddd717 channel: Base the channel forget timeout on the headercount
We were basing the 2016 block timeout on the blockchain height that we had
processed at the time completed the funding, which could lag considerably
behind the network-wide blockheight. For example this could happen if we
started with `--rescan` from a low height, taking some time to catch up.

This means that we could end up forgetting channels even before reaching the
network blockheight. This patch instead uses the headercount reported by the
backend plugin if we aren't caught up yet. While the chances of this happening
are still there, the window it might happen are much reduced, since headers
can be synced in a couple of minutes.

Reported-by: Riccardo Masutti
Changelog-Changed: Funding timeout is now based on the header count reported by the bitcoin backend instead of our current blockheight which might be lower.
2020-08-04 09:14:17 +09:30
ZmnSCPxj jxPCSnmZ 094eac4e95 plugins/libplugin-pay.c: Properly handle exclusions for routehints with two hops or more.
Arguably a low-priority bug since no current node ever generates routehints longer
than one hop.

However, it is possible as an edge case, if the destination is directly accessible
*and* supports multiple channels, that we route through the destination, one of the
*other* channels it has not in the routehint, to the entry point, and then through
the routehint.

This change removes the risk of the above edge case.

Changelog-None: arguably a low-priority bug.
2020-08-03 15:20:29 +09:30
Antoine Poinsot 3aad86ff9d pytest: test the blocksonly sanity checkin bcli
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Antoine Poinsot 5b45334d93 pyln: allow to signal may_fail in get_node()
This or `expect_fail`, as it implies that, well, the node may fail.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Antoine Poinsot 905730341a bcli: explicitly check at startup that bitcoind does relay transactions
We've had problems with blocksonly in the past and bitcoind still allows
to use outdated (thus potentially dangerous estimates) while running
bitcoin with -blocksonly.

ZmnSCPxj mentionned that we still don't document this, but i figured
that in this specific case an explicit check and error seems preferable.

Changelog-Added: We now explicitly check at startup that our default Bitcoin backend (bitcoind) does relay transactions.

Proposed-by: ZmnSCPxj <zmnscpxj@protonmail.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Antoine Poinsot 5be07c5fe3 bcli: poll `getnetworkinfo` at startup
This allows us to kill two birds with one stone: once connected, we use
the output of the successful call to do some sanity checks (only
checking the version for now, but more are yet to come!).

Changelog-Added: We now explicitly check at startup the version of our default Bitcoin backend (bitcoind).
Co-Authored-by: ZmnSCPxj <zmnscpxj@protonmail.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Christian Decker a456d08ad0 pay: Be less aggressive when estimating channel capacity
We'd previously take the failed attempt and estimate the failing channel's
capacity at 3/4 of the attempted amount, which is rather aggressive. This
reduces this aggressiveness to use the exact amount tried, but excluding on
equality. This still skips attempting the same route with the same amount, but
also permits attempts that are in the range [3/4, 1] of the failed attempt
amount to still be attempted.
2020-08-03 12:15:08 +09:30
ZmnSCPxj jxPCSnmZ e7d89cd7de lightningd/invoice.c: Improve programmatic error reporting for `delinvoice`.
Changelog-Changed: JSON-RPC: `delinvoice` will now report specific error codes: 905 for failing to find the invoice, 906 for the invoice status not matching the parameter.
2020-07-31 23:57:32 +00:00
ZmnSCPxj jxPCSnmZ b646b96756 lightningd/chaintopology.h: Remove unused `txnums` field from `struct block`.
Changelog-None internal cleanup
2020-07-31 23:55:23 +00:00
Christian Decker af4eec7b66 release: Update changelog for release v0.9.0 2020-07-31 18:41:04 +02:00
Christian Decker d9680414cb release: Add changes since v0.9.0rc3 to changelog 2020-07-29 14:27:36 +02:00
niftynei d87f31f9a8 utxo: clean up NULL handling of scriptpubkey
Now that we're *guaranteed* to have a scriptpubkey entry in the
database, we remove the NULL handling for it.
2020-07-29 13:13:46 +02:00