Commit Graph

1315 Commits

Author SHA1 Message Date
Christian Decker cbc959ef30 pytest: Mark test_funding_push as network-specific
It checks for both exact values, that may differ due to fees, as well as the
unit which varies from network to network.
2020-06-15 16:14:53 +02:00
niftynei 971f615695 psbt: affirm database upgrade works for last_tx -> psbt
We use a database snapshot with 3 channels -- two of which have HTLCs
dangling and one is an initial open channel tx in the 'old' tx hex
format in last_tx and confirm that they are successfully updated to PSBT
format on start.
2020-06-11 13:13:13 +02:00
Rusty Russell ce9e559aed features: do not set option_support_large_channel in channel_announcement.
Spec is wrong (it says it should be compulsory), and Eclair doesn't set it
at all, leading to an error when they send their announcement_signatures.

Fixes: #3703
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: large-channels: negotiate successfully with Eclair nodes.
2020-05-26 19:57:29 -05:00
Rusty Russell b3eef81f19 pytest: add test for channel_announcement feature bits.
And neaten current feature mangling code now we have a wumbo flag for
expected_peer_features().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
Rusty Russell f4f8a363dd pytest: fix feature mask for EXPERIMENTAL_FEATURES, add wumbo support.
There are various places where our tests failed with
--enable-expimental-features.  And our plugin test overlapped an
existing feature.

We make our expected_feature functions more generic, and use them
everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
Antoine Poinsot 5dfd2436db wallet: show input txid in little endian for json_listtransactions
Changelog-Fixed: jsonrpc: `listtransactions` now displays all txids as little endian

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-22 15:11:51 +02:00
Antoine Poinsot 1061cd95c9 pytest: add a sanity check for the listtransactions command
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-22 15:11:51 +02:00
lisa neigut 1fb9a078b6 txprepare: return psbt serialized version of tx as well
Changelog-Added: JSON-API: `txprepare` returns a psbt version of the created transaction
2020-05-21 18:45:07 +09:30
Antoine Poinsot 0088147d9e pytest: add sanity check for --commit-fee
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-20 06:09:24 +09:30
Rusty Russell cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell abb16b4226 cli: implement new 'flattened JSON' mode.
Much nicer for grepping, since `{ "foo": { "bar": [7] } }` is turned into
`foo.bar[0]=7`.

Changelog-Added: cli: New `--flat` mode for easy grepping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:25:44 +02:00
Rusty Russell 679d3494b4 cli: tweak -H output to remove format-hint fields.
-H removes the top-level if there's only one, and 'format-hint'
breaks this heuristic, so we end up with:

```
help=command=autocleaninvoice [cycle_seconds] [expired_by]
category=plugin
description=Set up autoclean of expired invoices.
verbose=Perform cleanup every {cycle_seconds} (default 3600), or disable autoclean if 0. Clean up expired invoices that have expired for {expired_by} seconds (default 86400).
command=check command_to_check
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:25:44 +02:00
Rusty Russell 1fb505d118 pytest: extend timeout for test_payment_duplicate_uncommitted
We've been seeing some Travis timeouts under VALGRIND, with the
10 second timeout here: use TIMEOUT as per standard.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-12 21:57:43 +02:00
lisa neigut d35149e5ca try to fix travis test flakes 2020-05-12 15:46:58 +09:30
lisa neigut 487771aee5 tests: skip longer htlc tests on valgring+slow machines
These guys take a while to run, so let's just skip them on the
valgrind/slow-machine combos :/
2020-05-12 15:46:58 +09:30
lisa neigut 6e4b4c50f0 coin tests: try to make a bit more travis robust 2020-05-12 15:46:58 +09:30
lisa neigut 8acbbca05d coins: use the chain's BIP173 name instead of a 'unit of account'
Updates the unit of account to be the chain_id, which is the BIP173 name
of the chain that the coins moved on.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut 4bfbb58c56 coins: fix feerate for withdraw so test work? 2020-05-12 15:46:58 +09:30
lisa neigut 087ab166b3 coins, fix: don't crash if the to_us amount is greater than our_msat
It's possible for our peer to publish a commitment tx that has already
updated our balance for an htlc before we've completed removing it from
our commitment tx (aka before we've updated our balance). This used to
crash, now we just update our balance (and the channel balance logs!)
and keep going.

If they've removed anything from our balance, we'll end up counting it
as chain_fees below. Not ideal but fine... probably.
2020-05-12 15:46:58 +09:30
lisa neigut ffd9467f14 coin moves: persist the coin movement index counter to disk
Should make it easier to track when coin moves in the plugin are
disjoint from what c-lightning says it's broadcast already.
2020-05-12 15:46:58 +09:30
lisa neigut de86e29e16 coin moves: log all withdrawals when confirmed in a block
This moves the notification for our coin spends from when it's
successfully submited to the mempool to when they're confirmed in a
block.

We also add an 'informational' notice tagged as `spend_track` which
can be used to track which transaction a wallet output was spent in.
2020-05-12 15:46:58 +09:30
lisa neigut e9d26a46e0 coin moves: actually record the blockheight for all chain moves
Previously we were annotating every movement with the blockheight of
lightningd at notification time. Which is lossy in terms of info, and
won't be helpful for reorg reconciliation. Here we switch over to
logging chain moves iff they've been confirmed.

Next PR will fix this up for withdrawals, which are currently tagged
with a blockheight of zero, since we log on successful send.
2020-05-12 15:46:58 +09:30
lisa neigut 9caf20f636 coin moves: don't log coin moves in onchaind if we're replaying
On node start we replay onchaind's transactions from the database/from
our loaded htlc table.  To keep things tidy, we shouldn't notify the
ledger about these, so we wrap pretty much everything in a flag that
tells us whether or not this is a replay.

There's a very small corner case where dust transactions will get missed
if the node crashes after the htlc has been added to the database but
before we've successfully notified onchaind about it.

Notably, most of the obtrusive updates to onchaind wrappings are due to
the fact that we record dust (ignored outputs) before we receive
confirmation of its confirmation.
2020-05-12 15:46:58 +09:30
lisa neigut 1b5221cbf5 coin moves tests: push_msat and the wallet withdrawal
Check that we account for push_msat and wallet withdrawal/deposits
correctly
2020-05-12 15:46:58 +09:30
lisa neigut 8c986d67db coin moves: test all of the onchaind variants
Mostly we update existing tests to account for channel balances. In a
few places, new tests were needed as there wasn't an existing pathway
that tested the chain-fees for a few penalty cases
2020-05-12 15:46:58 +09:30
lisa neigut 41d3471c7f coin moves tests: save updates to disk
If we don't save to disk, if the node restarts we'll lose them all
and the resulting balance check at the end will be incorrect.
2020-05-12 15:46:58 +09:30
lisa neigut 6ee6cdc280 coin moves: record their attempted cheat (and our handling thereof)
For cheats, we do a little bit of weird accounting. First we 'update'
our on-ledger balance to be the entirety of the channel's balance. Then,
as outputs get resolved, we record the fees and outputs as withdrawals
from this amount.

It's possible that they might successfully 'cheat', in which case we
record those as 'penalty' but debits (not credits).
2020-05-12 15:46:58 +09:30
lisa neigut 9c4c0f10fb coin_mvt: add integration tests for in-channel htlc movements 2020-05-12 15:46:58 +09:30
Christian Decker 8f2ce1e638 pytest: Add a test for the commitment_revocation hook 2020-05-07 15:05:39 +09:30
Rusty Russell 046b402c18 gossipd: return channel_announcement features for listchannels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON API: `listchannels` now shows channel `features`.
2020-05-07 08:44:58 +09:30
Rusty Russell 855debcfe1 gossipd: upgrade v7 gossip_store to v8.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Rusty Russell 46793bdaaf pytest: test gossip_store upgrade from version 7 to version 8.
The previous patch changed the gossip_store, but in a trivial way.
The next patch will implement upgrading, so this is the test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Rusty Russell 78ffea61e1 channeld: tell gossipd what the features are for our local channels.
This msg is stored in the gossip_store, so it means a version bump.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Antoine Poinsot 4bb7b4621f pay: respect maxfeepercent when choosing a shadow route
And the percentage of the initial amount, not the constently increasing
one !

Changelog-Fixed: pay: we now respect maxfeepercent, even for tiny amounts.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-05 14:10:42 +09:30
Antoine Poinsot f83a1e0f2d Reproduce issue #3684
Fixed-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-05 14:10:42 +09:30
Rusty Russell fe365f930f lightningd: list disabled plugins in listconfig.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell 24063ca972 lightningd: have plugin-disable be more persistent.
The previous implementation was a bit lazy: in particular, since we didn't
remember the disabled plugins, we would load them on rescan.

Changelog-Changed: config: the `plugin-disable` option works even if specified before the plugin is found.
2020-05-05 13:45:17 +09:30
Rusty Russell 8c59de5ee5 lightningd: make plugin_kill take a simple string.
That's more convenient for most callers, which don't need a fmt.

Fixed-by: Darosior <darosior@protonmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell 051cbf7cc4 lightningd: make plugin_kill() free the plugin.
This is what I expected from plugin_kill, and now all the callers do the
equivalent anywat, it's easy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell 4de11950ec lightningd: unify dynamic and static plugin initialization.
This means we now clean up options in startup plugins (that was only
done by dynamic code!), and now they both share the 60 second timeout
instead of 20 seconds for dynamic.

For the dynamic case though, it's 60 seconds to both complete
getmanifest and init, which seems fair.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell 006ab1e367 pytest: test loading all plugins at once, including failing ones.
We modify the slow_init() so it doesn't go too slowly for this test.

This demonstrates a crash, where we currently try to fail a command
multiple times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell 6441233d2b pytest: add test for a plugin which falls over outside a command.
This actually passes fine, but it's an interesting case to test.

Fixed-by: Darosior <darosior@protonmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Christian Decker 7b752e00d5 pyln: Set the bitcoin datadir when running tests
Telling `lightningd` to pass a `-datadir` to `bitcoin-cli` so it doesn't go
snooping where it doesn't belong (i.e., the user's home directory and config).

Changelog-None
Suggested-by: Simon Vrouwe <@SimonVrouwe>
Signed-off-by: Christian Decker <@cdecker>
2020-05-05 13:28:54 +09:30
rbndg 241fa00e97 plugin:added invoice creation event
New invoice_creation event triggered when an new invoice is created

Changelog-Added: plugin: New invoice_creation plugin event
2020-05-04 19:18:05 +02:00
Rusty Russell 7abff309e5 pytest: Fix flakiness in test_feerate_stress.
We occasionally get a temporary_channel_failure too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
Rusty Russell 4eb1233ccb lightningd: don't report spurious temporary_node_failure on local failures.
I noticed the following in logs for tests/test_connection.py::test_feerate_stress:

```
DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Failing HTLC 18446744073709551615 due to peer death
DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: local_routing_failure: 8194 (WIRE_TEMPORARY_NODE_FAILURE)
```

This is because it reports the (transient) node_failure error, because
our channel_failure message is incomplete.  Fix this wart up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
Rusty Russell 75496ad168 pytest: fix flaky test_peerinfo
If we don't wait for close tx to reach mempool, it might not get to
depth 100, and we don't get 'onchaind complete, forgetting peer'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
Rusty Russell 4df9b2c5f2 pytest: fix flake in test_onchain_different_fees()
We didn't wait until l2 processed the final state of HTLC #2, so
it might not include it when it drops onchain, leading to us only
getting 3 (not 4) sendrawtx calls.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
lisa neigut 0e20e3c5e7 df: rename 'funder' to 'opener'
Previously we've used the term 'funder' to refer to the peer
paying the fees for a transaction; v2 of openchannel will make
this no longer true. Instead we rename this to 'opener', or the
peer sending the 'open_channel' message, since this will be universally
true in a dual-funding world.
2020-05-04 10:22:26 +09:30
Christian Decker e7b555dd90 pytest: Stabilize test_penalty_{in,out}htlc tests
They were looking for specific amounts which proved to be rather flaky. Now
they look for specific outputs being available in the `listfunds` result after
everything was settled.
2020-05-01 18:20:33 -05:00