Commit Graph

1742 Commits

Author SHA1 Message Date
Rusty Russell 001b5d6416 offers: make it a runtime option.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
2021-01-13 14:45:36 +01:00
Rusty Russell 27c006f7aa libplugin: make init return a string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: init can return a non-NULL string to disable the plugin.
2021-01-13 14:45:36 +01:00
Rusty Russell 529ae0d766 plugins: allow plugins to disable themselves at startup.
By returning 'disable: <reason>' inside getmanifest or init result.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: plugins can now disable themselves by returning `disable`, even if marked important.
2021-01-13 14:45:36 +01:00
Rusty Russell fc3e679c97 lightningd: control onion messages by `experimental-onion-messages` option.
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
2021-01-13 14:45:36 +01:00
niftynei 7e324177de df-tests: simultaneous openchannel_init (while in progress)
Reject a peer's request to open a channel while we're already in progress
2021-01-10 13:44:04 +01:00
niftynei 2c9ce2566c df-tests: test_lightningd_still_loading 2021-01-10 13:44:04 +01:00
niftynei a357e75c5d df-tests: test_disconnect_half_signed + test_reconnect_signed 2021-01-10 13:44:04 +01:00
niftynei fec1d9b707 df-tests: test_reconnect_openingd 2021-01-10 13:44:04 +01:00
niftynei 7eb6213fca df-tests: test_reconnect_sender_add1, others
We're still ignoring multifundchannel failure tests, but now we use the
'dual-fund' flag instead of a blanket true.
2021-01-10 13:44:04 +01:00
niftynei d9c68a708a df-tests: test_reconnect_receiver_add 2021-01-10 13:44:04 +01:00
niftynei 96c42262ab df-tests: test_peerinfo 2021-01-10 13:44:04 +01:00
niftynei 135cd178f7 df-tests: test_channel_persistence 2021-01-10 13:44:04 +01:00
niftynei a82bfa83ff df-tests: have the df_accepter plugin keep track of attempts
Test connection/reconnection handling for v2 opens. We needed to fixup
the accepter plugin so that we were freeing up inputs on
disconnect/failure.
2021-01-10 13:44:04 +01:00
niftynei 88b4b5b2bb df-tests: test_disconnect_opener 2021-01-10 13:44:04 +01:00
niftynei 3e330d7186 df-tests: test_forward_local_failed_stats 2021-01-10 13:44:04 +01:00
niftynei 973224dd14 df-tests: if we're using fundbalancechannel, set up plugin
nodes need to be using the accepter plugin, so they'll match funding.
we used to use push_msat for this, but v2 gets rid of it.

*sad trombone*
2021-01-10 13:44:04 +01:00
niftynei f822a93cf4 df-tests: add `setacceptmaxfunding` to df_accepter plugin
Method for setting a ceiling on the fund matching capabilities of
df_accepter. Setting it to zero means we don't fund the channel anymore.
2021-01-10 13:44:04 +01:00
niftynei e964c31369 df-tests: test_payment_success|failed_persistence 2021-01-10 13:44:04 +01:00
niftynei 09da171381 df-tests: test_io_logging
Move pid collection down to when dualopend is definitely dead
2021-01-10 13:44:04 +01:00
niftynei 609d509ca5 df-tests: test_channel_state_changed_unilateral 2021-01-10 13:44:04 +01:00
niftynei ac88c72f1e df-tests: test_channel_state_changed_bilateral 2021-01-10 13:44:04 +01:00
niftynei 450a094d26 df-tests: test_coin_movement_notices 2021-01-10 13:44:04 +01:00
niftynei 923e1f5c80 df-tests: test_feature_set 2021-01-10 13:44:04 +01:00
niftynei db65771f38 df-tests: test_plugin_feature_announce 2021-01-10 13:44:04 +01:00
niftynei 21b9bec93b df-tests: test_channel_state_change_history 2021-01-10 13:44:04 +01:00
niftynei 959ca793b0 df-tests: test_openchannel_hook_error_handling 2021-01-10 13:44:04 +01:00
niftynei 803d9701d3 df-tests: test_openchannel_hook_chaining 2021-01-10 13:44:04 +01:00
niftynei b0a8b10b41 df-tests: test_openchannel_hook
Have it check both openchannel + openchannel2
2021-01-10 13:44:04 +01:00
niftynei 0d45823b82 df-tests: test_funding_reorg_private 2021-01-10 13:44:04 +01:00
niftynei 09834bbc8d df-tests: test_node_reannounce correct features 2021-01-10 13:44:04 +01:00
niftynei c055198869 df-tests: test_fail_unconfirmed, duplicate with correct wires 2021-01-10 13:44:04 +01:00
niftynei dfa0eb0f2f df-tests: test_shutdown_awaiting_lockin check correct state 2021-01-10 13:44:04 +01:00
niftynei 5c7cc15a75 df-tests: adjust logsearch/features for test_dataloss_protection 2021-01-10 13:44:04 +01:00
niftynei e661d87440 df-tests: fixup expected features for dual-funding channels 2021-01-10 13:44:04 +01:00
niftynei 08200f03a7 df-tests: test_multifunding_v2_v1_mixed update to use flag
Now that we've got a fancy flag to enable dual funding, let's use it
2021-01-10 13:44:04 +01:00
niftynei 8fb97fd7e1 df-tests: updates for plugin hook_accepter helper
Fixes so that test_funding_close_upfront works with v2 opens
2021-01-10 13:44:04 +01:00
niftynei f4b7904ce5 tests: don't enforce daemon name on opening logline 2021-01-10 13:44:04 +01:00
niftynei 9951638d5f df: add flag to signal to run v2 channel open tests 2021-01-10 13:44:04 +01:00
niftynei 99a621dd99 df-reconnects: allow tx-sigs in channeld iff we're reconnecting
There's a case where a dropped funding_locked will result in the peer
moving onto channeld, while we stay in dualopend. As we haven't
received their funding_locked, we retransmit tx_sigs, which channeld
will need to handle.

With the patch the peer drops it on the floor; the peer will resend
funding_locked on reconnect, which will correctly advance us to
channeld and CHANNELD_NORMAL
2021-01-10 13:44:04 +01:00
Rusty Russell af46a4f57d fetchinvoice: allow amounts to be specified.
As per lastest revision of the spec, we can specify amounts in invoice
requests even if the offer already specifies it, as long as we exceed
the amount given.  This allows for tipping, and amount obfuscation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell 4bb05e46e9 offers: convert currency when they request an invoice.
Means a reshuffle of our logic: we want to multiply by quantity before
conversion for maximum accuracy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell 9681d491df offer: allow offers in other currencies if we can convert.
This avoids a footgun where they create an offer then we can't create
the invoice because they don't have a converter plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell 23af9d4972 onion_message: support variable-length onion messages.
Updated to the BOLT, and a few tweaks, and we can send giant onion_messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell d867fab3a7 sendinvoice: call waitinvoice directly to on invoice payment.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell 1aa7e8e011 offers: split offer send_invoice generation into new JSON command
We split `send_invoice` offers inoo offerout (for want of a better name).
This simplifies the API.

Also took the opportunity to move the `vendor` tag to immediately
follow `description` (our tests use arguments by keywords, so no
change there).

Suggested-by: shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell 55983827b6 fetchinvoice: try direct peers if we can't route.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell bdfcec3eb6 fetchinvoice: check we're in the period before attempting to fetch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell dae477175c fetchinvoice: implement timeout.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell 075c25fc08 plugins/fetchinvoice: handle sendinvoice timeout, error or payment.
If they pay the invoice, they don't bother replying; that's just for
errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell 43f5304539 plugins/offer: allow `offer` to create `send_invoice`, `refund` and `single_use` offers.
refund offers are implied send_invoice offers.  And send_invoice offers
are implied single-use offers, though it can also make sense to have
a non-send_invoice offer be single-use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Christian Decker 6b6e0fe34c pytest: Mark `test_lockup_drain` as disabled
It's been causing me headaches for some time, and it's just getting
annoying. Will file an issue to track progress.
2021-01-08 19:28:30 +01:00
niftynei d0dcb7cb42 tests: wait for drop to chain test_onchain_middleman_their_unilateral_in
this was catching the wrong 'sendrawtx exit 0' and consequently
not mining the close tx into a block
2021-01-08 19:28:30 +01:00
Rusty Russell 1a8978100e invoice: add ctlv option.
This is required if we want to create a "bouncer" plugin (in my copious free time!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
2021-01-08 10:45:56 +01:00
Christian Decker 4b2efd69d7 pay: Fix a crash when waitblockheight times out
Fixes #4309
Changelog-Fixed: pay: Fixed an issue where waiting for the blockchain height to sync could time out.
2021-01-08 09:19:24 +01:00
Christian Decker ca204eacfa pytest: Reproduce waitblockheight timeout issue #4309 2021-01-08 09:19:24 +01:00
Christian Decker 70410b8ee8 pytest: Test LightningRpc and plugin command notification support 2021-01-07 08:32:40 +01:00
Christian Decker 84b3653606 pyln: Add command notification support for plugins
Changelog-Added: pyln-client: Plugin methods can now report progress or status via the `Request.notify` function
2021-01-07 08:32:40 +01:00
Christian Decker a1d33c3781 pytest: Add a test for `listinvoices`
We now have some more logic in the query, so let's test it
exhaustively.
2021-01-06 18:48:06 +01:00
Antoine Poinsot a4f07a3e78 tests/fuzz: add a fuzz target for hsm_encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 506d524508 libfuzz: add a NULL-termination in to_string
It's more useful if we actually want to use the output as, well, a
string..

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
niftynei 0c9072272f df-plugin: add `signonly` to signing in df_accepter
We're the accepter, so we look up which inputs we need to sign and only
sign those.
2021-01-05 19:12:00 +01:00
niftynei e3a1d1a7f3 pyln-client: to_whole_satoshi returns the rounded up satoshi value
A fractional satoshi value isn't really useful; rounding up loses
precision but that's why you called "whole satoshi", wasn't it?

Changelog-Changed: pyln-client: Millisatoshi has new method, `to_whole_satoshi`; *rounds value up* to the nearest whole satoshi
2021-01-05 19:12:00 +01:00
niftynei cce0020568 pyln-client: add missing `min_witness_weight` to fundpsbt/utxopsbt
We need to use it for the 'df_accepter' plugin, so we get the feerate
correct.

Changelog-Added: pyln-client: `fundpsbt`/`utxopsbt` now support `min_witness_weight` param
2021-01-05 19:12:00 +01:00
Antoine Poinsot 19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot 1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker 5b3af00e28 pytest: Really only check DB statements if the env-var is "1" 2021-01-04 11:25:50 +01:00
Christian Decker 94820ad559 pytest: Disable test_funding_cancel_race on slow machines
The CI regularly trips up on this very taxing test (100 nodes) so only
run it if we have the required horsepower.
2021-01-04 11:25:50 +01:00
Christian Decker b3fd47e5f6 pytest: Stabilize test_routing_gossip
We were getting bad gossip because some nodes discarded the channel
announcement for being in the future. This is because the node was, at
that time, below the confirmation height. It'd then discard the
followup messages because not preceded by an announcement, and getting
upset about that.
2021-01-04 11:25:50 +01:00
Christian Decker 7e867e5ee6 pytest: Add throttler to limit load on the test system
Both my machine and apparently the CI tester machines regularly run
into issues with load on the system, causing timeouts (and
unresponsiveness). The throttler throttles the speed with which new
instances of c-lightning get started to avoid overloading. Since the
plugin used for parallelism when testing spawns multiple processes we
need to lock on the fs. Since we have that file open already, we'll
also write a couple of performance metics to it.
2021-01-04 11:25:50 +01:00
Rusty Russell 52af729641 plugins/offer and plugins/fetchinvoice: send and recv errors.
This also lets us extend our testing to cover error cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 96caf9f4ab fetchinvoice: return the next period for recurring offers.
This is useful for the caller to know when to call again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 19af1d10e6 invoice: hack in merkle of invoice as "payment_secret" (EXPERIMENTAL_FEATURES)
This lets actually pay the invoice that fetchinvoice returns.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 2baa410e2b pytest: finally test fetchinvoice send, and offer plugin receive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 08e110b568 JSON: offers plugin `offer` command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Karol Hosiawa e582c0ac0a Added listfunds test 2020-12-22 13:00:02 +01:00
Rusty Russell 723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell a33e39b7e8 pay, wallet: rename internal `bolt11` vars to `invstring`.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell d971e3de98 Plugin: support extra args to "start".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `start` command can now take plugin-specific parameters.
2020-12-15 09:28:56 +10:30
Rusty Russell 8a9976c4c1 plugins: support concatenation of multiple args.
"multi" means that specifying a parameter twice will append, not override.
Multi args are always given as a JSON array, even if only one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: new "multi" field allows an option to be specified multiple times.
2020-12-15 09:28:56 +10:30
Rusty Russell 646c564ec5 plugins: remove deprecated string plugin options.
This was fixed in 0.8.2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: plugins: options to init are no longer given as strings if they are bool or int types (deprecated in 0.8.2).
2020-12-15 09:28:56 +10:30
Christian Decker f7cdf1dd98 pytest: Debug test_lockup_drain 2020-12-11 16:55:55 -06:00
Christian Decker 5a69b94f9a pytest: Add test for keysend feature support 2020-12-11 15:50:09 +01:00
Christian Decker 3b599b846e keysend: Check that the destination supports keysend upon init
We were blindly initiating the keysend payment, which could lead to
confusing outcomes. This adds a very specific error message to the
error returned.

Changelog-Fixed: keysend: Keysend now checks whether the destination supports keysend before attempting a payment. If not a more informative error is returned.
2020-12-11 15:50:09 +01:00
Christian Decker 71fafd23fd pytest: Add a test to reproduce #4258 2020-12-09 06:56:21 +10:30
Christian Decker eae49727b1 pytest: Disable flaky test_fundchannel_start_alternate
This test has been causing most CI runs to fail. See tracking issue #4265
for discussion on fixing and re-enabling it.

Changelog-None
2020-12-08 09:14:25 +10:30
Michael Schmoock ce1a2f134a pytest: tests fee_base and fee_ppm on listpeers 2020-12-07 14:26:36 +01:00
Christian Decker bb0910999e pytest: Use datetime.strptime instead of datetime.fromisoformat
The latter is available only starting with Python 3.7, and it's the
only test that fails with <3.7. This allows us to keep on testing with
older versions.
2020-12-04 18:34:12 -06:00
ZmnSCPxj jxPCSnmZ a437936c78 lightningd/log.c: Fix up handling of SIGHUP.
Fixes: #4240

ChangeLog-Fixed: log: Do not terminate on the second received SIGHUP.
2020-12-02 16:48:48 +10:30
ZmnSCPxj jxPCSnmZ 67d8fdcc75 tests/test_misc.py: Add a test for issue #4240. 2020-12-02 16:48:48 +10:30
niftynei b23c6c50a0 df-tests: use multifundchannel pathways instead of custom plugin
Since fundchannel/multifundchannel now handles v2 opens, we use that
instead of this hacked together plugin.
2020-12-02 14:19:08 +10:30
Michael Schmoock 899022e452 pytest: use default fundamount on two tests 2020-12-01 11:15:30 +10:30
ZmnSCPxj jxPCSnmZ 32de621886 lightningd/plugin_hook.c: Make `db_write` a chained hook.
Fixes: #4219

Changelog-Changed: Plugins: Multiple plugins can now register `db_write` hooks.
2020-11-30 10:40:11 +10:30
niftynei 405453859c mfc-df: add 'happy path' tests for the v1+v2 things
We can't test disconnects et.al. quite yet because the 'cancel' flow for
openchannelv2 still needs to be resolved
2020-11-23 12:41:05 -06:00
niftynei da98a9d0af df-accepter plugin: temporarily dont pass in signpsbt 2020-11-23 12:41:05 -06:00
niftynei 78d32b12d0 nit,df test plugin: change up how feerate is formatted
Suggested-by: @cdecker
2020-11-23 12:41:05 -06:00
niftynei 7e60ef6974 tests: make connection details correct 2020-11-16 20:00:51 -06:00
niftynei 4a3ee19a22 connectd: Update connection list with new address
If we're already attempting to connect to a peer, we would ignore
new connection requests. This is problematic if your node has bad
connection details for the node -- you can't update it while inflight.

This patch appends new connection suggestions to the list of connections
to try.

Fixes #4154
2020-11-16 20:00:51 -06:00
niftynei cd7d5cdff9 pytest: use the 'contrib' plugin set, not the 'test' set
They're clobbering each other, and the test hangs
2020-11-13 14:28:29 -06:00
Michael Schmoock 183f7698c3 chore: reduce CI load merging a testcase 2020-11-13 14:25:02 -06:00
Michael Schmoock 4134c24cc9 pytest: fix a probable flake on test_channel_opener 2020-11-13 14:25:02 -06:00
Michael Schmoock 90a4476f16 pytest: tests closer on unilateral close 2020-11-13 14:25:02 -06:00
fiatjaf c5aa2ea911 failure_onion: a test that only ensures lightningd doesn't crash. 2020-11-10 19:03:23 -06:00
niftynei 72dc97c0c1 pytest: silence pytest warnings about unregistered 'slow_test' marker
We've been getting a bunch of 'warnings' about an unknown mark. This
silences the warning, by registering the mark as expected.

=========================================================== warnings summary ===========================================================
tests/test_closing.py:152
  /home/niftynei/dev/lightning/tests/test_closing.py:152: PytestUnknownMarkWarning: Unknown pytest.mark.slow_test - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    @pytest.mark.slow_test

tests/test_closing.py:214
  /home/niftynei/dev/lightning/tests/test_closing.py:214: PytestUnknownMarkWarning: Unknown pytest.mark.slow_test - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    @pytest.mark.slow_test

tests/test_closing.py:704
2020-11-10 16:17:04 -06:00
Antoine Poinsot fc964edc06 pytest: test onchain wallet descriptor import on bitcoind
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot 4f2ae48c54 fuzz: add a fuzz target for common/descriptor_checksum
A small one just to check that we don't crash nor go out of bounds!

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Antoine Poinsot bf0a2f2b85 fuzz: add a 'to_string' function to libfuzz
We want to use it outside of fuzz-amount

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
Rusty Russell 9a575a98a0 gossipd: simplify large reply code.
We used to create the entire reply, the if it was too big, split in
half and retry.

Now that the main network is larger, this always happens with a full
request, which is inefficient.

Instead, produce a reply assuming no compression, then compress as a
bonus.  This is simpler and more efficient, at cost of sending more
packets.

I also renamed an internal dev var to make it clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 17:37:10 -06:00
Rusty Russell 4745e7e3d0 pytest: make test_hook_dep tests more stable.
I think this is what Travis is having an issue with, but it work
fine locally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Michael Schmoock f3804daabb pytest: simplify test_openchannel_hook_chaining now order is deterministic. 2020-11-09 15:22:33 -06:00
Rusty Russell fb295ffb51 plugin: sort topological candidates by specified order.
We previously registered hooks up in who-replies-to-getmanifest-first
order, but then if any had dependencies it would scatter that order.

This allows users to manually set dependencies developers have
forgotten by specifying the plugins manually in their configuration or
cmdline.  This was an excellent consideration by @mschmook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell d429e21db3 pytest: test that we maintain load order unless hook deps require a change.
Suggested-by: @mschmook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell 852e14c947 plugins: check order once all plugins have returned from getmanifest.
This means we need to stop at this stage even in the runtime-loaded
case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell a4f79e8e7e pytest: test that we don't complain about missing dependencies at startup.
We fail this, because we check dependencies as they come in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell 62c52fe868 libplugin: add support for before and after deps on hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell e16ed0e207 pyln: add support for dependent hooks.
And use that to add simple tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
ZmnSCPxj jxPCSnmZ 0fd87b85da openingd/: Fail `fundchannel_start` if we already are, or will become, the fundee.
Fixes: #4108

Changelog-Fixed: Network: Fixed a race condition when us and a peer attempt to make channels to each other at nearly the same time.
2020-11-07 17:18:49 +00:00
ZmnSCPxj jxPCSnmZ 712e0796fc tests/test_connection.py: Test for #4108. 2020-11-07 17:18:49 +00:00
Michael Schmoock 4a8722bf1d pytest: state_change history 2020-11-06 14:47:04 -06:00
Michael Schmoock 68ce25c92d feat: adds timestamp to state_change notification 2020-11-06 14:47:04 -06:00
Michael Schmoock 67f4970eb1 pytest: test opener and closer of listpeers 2020-11-06 14:47:04 -06:00
Michael Schmoock c52efe0f37 pytest: channel state change cause and message 2020-11-06 14:47:04 -06:00
Rusty Russell ae4139ba10 lightningd: note whether payment secret is used or not.
This will allow nodes (with log-level=debug) to gather how many payments
are made without payment_secrets.  We need to know this so we know when
we can make them compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 12:03:14 +01:00
niftynei 4ea9d9e928 df: Pass new feerate options through to plugin, set reasonable bounds
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.

We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.

In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
2020-10-27 19:52:05 -05:00
Rusty Russell 1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 44f6b8a746 pytest: add notifications to tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Antoine Poinsot 85aa53c778 fuzz: add a runner Python script for fuzz targets
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot c149a88bbe fuzz: add a fuzz target for initial_channel
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 1aa94ccd9e fuzz: add a fuzz target for the close tx
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 33c7eccb3b fuzz: add a fuzz target for channel_id derivation
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot f0061d0606 fuzz: add a fuzz target for common/bip32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot a89a018475 fuzz: add a fuzz target for common/bigsize
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 5d987f2dec fuzz: add a target for base32/64 encoding
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 63b1ae54a1 fuzz: add a target for common/bech32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 2f90c45454 fuzz: add a fuzz target for amounts parsing
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot 62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
niftynei ede5f5be3c mfc: blackbox test for commitment vs funding tx feerates
Liquid is excluded because the mempool entry doesn't contain a 'weight'
field.
2020-10-21 11:24:16 +10:30
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
Rusty Russell 639eddf840 common/gossmap: digest private channel information too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell 83aea6b2bb gossip_store: make private channels more similar to channel_announcement
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys).  This makes it far
easier for gossmap to handle local channels.

Backwards compatible update, since we update old stores.

We also fix devtools/dump-gossipstore to know about the tombstone markers.

Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
Rusty Russell c6625943b5 pytest: test that route can see private channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-21 08:58:34 +10:30
niftynei a7d4b8780e elementsd: disable dual-funding test; don't add non-witness utxo
Elementsd needs to be updated to v0.20.0 in order for this to work as
expected
2020-10-20 12:50:31 +10:30
niftynei b696ec89a5 df-open: use channel_id for openchannel_update and openchannel_signed
Be as specific as possible is a good rule for things, I think
2020-10-20 12:50:31 +10:30
niftynei 5fbd25f938 dual-fund: test (turned off) and plugin for executing an openchannel
Proof that v2 openchannel works, plus a plugin for sewing all the
requisite bits together
2020-10-20 12:50:31 +10:30
niftynei e2a6fd7112 common: pull up `param_psbt`
Usable other places, not just in wallet
2020-10-20 12:50:31 +10:30
niftynei 8f253b2adc test: simplify test for close_to
Since `fundchannel` now supports the 'close_to' argument, we can remove
all the logic needed to call fundchannel_start here.

Underneath, we're still calling `fundchannel_start`, we're just one (or
two, if you count multifundchannel) call levels away from it now.
2020-10-16 13:51:57 +10:30
niftynei de34f08b82 tests,fundchannel: return the result from the fundchannel rpc call
We need this so we can verify the 'close_to' result
2020-10-16 13:51:57 +10:30
Rusty Russell a3c30441d3 channeld: order htlcs by id before retransmission.
We had one report of this, and then Eugene and Roasbeef of Lightning
Labs confirmed it; they saw misordered HTLCs on reconnection too.

Since we didn't enforce this when we receive HTLCs, we never noticed :(

Fixes: #3920
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)
2020-10-14 19:58:57 +10:30
Rusty Russell d04597cbb6 pytest: add test for HTLC ordering.
[ Includes tidyups by Christian Decker ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-14 19:58:57 +10:30