Commit Graph

545 Commits

Author SHA1 Message Date
Christian Decker 6c67afeb03 make: Add CPPCHECK_OPTS to allow excluding files from the check
With swig we now have C files that are generated with tools that are
not under our control, so provide an escape hatch for them.
2021-05-15 09:31:12 +09:30
Christian Decker a4c19ba6ba libhsmd: Clean up the build descriptors for libhsmd_python 2021-05-15 09:31:12 +09:30
Christian Decker 7f15f86718 libhsmd: First working build script for libhsmd-python
Changelog-Added: libhsmd: Added python bindings for `libhsmd`
2021-05-15 09:31:12 +09:30
niftynei 3a2d602922 tests: add marker for v1/v2 channel opens
Tests that will only run when !EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v1')
	def test_...()

Tests that will only run when EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v2')
	def test_...()
2021-05-12 11:25:41 +09:30
niftynei 484d6bde83 tests: move EXP_DF into the testing utils 2021-05-12 11:25:41 +09:30
niftynei 09b5c906fd funder, startupregtest: fixup default config to be 100
Fails on start otherwise
2021-05-12 11:25:41 +09:30
Rusty Russell d8e68893f5 bcli: become less aggressive with onchain fee levels.
Users are more upset recently with the cost of unilateral closes
than they are the risk of being cheated.  While we complete our
anchor implementation so we can use low fees there, let's
get less aggressive (we already have 34 or 18 blocks to close
in the worst case).

The changes are:

- Commit transactions were "2 CONSERVATIVE" now "6 ECONOMICAL".
- HTLC resolution txs were "3 CONSERVATIVE" now "6 ECONOMICAL".
- Penalty txs were "3 CONSERVATIVE" now "12 ECONOMICAL".
- Normal txs were "4 ECONOMICAL" now "12 ECONOMICAL".

There can be no perfect levels, but we have had understandable
complaints recently about how high our default fee levels are.

Changelog-Changed: Protocol: channel feerates reduced to bitcoind's "6 block ECONOMICAL" rate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Christian Decker 083b41f090 plugin: Add a list of notification topics registered by plugin
We will eventually start emitting and dispatching custom notifications
from plugins just like we dispatch internal notifications. In order to
get reasonable error messages we need to make sure that the topics
plugins are asking for were correctly registered. When doing this we
don't really care about whether the plugin that registered the
notification is still alive or not (it might have died, but
subscribers should stay up and running), so we keep a list of all
topics attached to the `struct plugins` which gathers global plugin
information.
2021-05-03 11:20:15 +09:30
niftynei 473067859f pyln-testing: use provided outnum instead of trying to find it
Dual-funded channels won't match the old amount check. Good news is
we're already returning the outnum so we just use that.
2021-05-03 11:06:10 +09:30
niftynei bc7875864b contrib: if you're in dev mode, use dual-funding (with matching)
If you're using the regtest node, turn on dual funding and
automatically attempt to dual fund at a 100% match for every channel
open that you do.
2021-05-03 11:06:10 +09:30
Rusty Russell 32d650f9df lightningd: don't abort on incorrect versions, but try to re-exec.
You still shouldn't do this (you could get some transient failures),
but at least you have a decent chance if you reinstall over a running
daemon, instead of getting confusing internal errors if message
formats have changed.

Changelog-Added: lightningd: we now try to restart if subdaemons are upgraded underneath us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #4346
2021-04-24 13:56:58 +09:30
Rusty Russell 402f7f90c0 pytest: handle case where funding tx is not tx #1.
e.g. in test_closing_id we can get a spend from the first (closed) channel
in the same block as the open of the second.  Half the time, we'll choose
the wrong one as scid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-12 23:03:47 +02:00
Christian Decker 8e0d0c3e7b pyln: Pin mypy to version 0.790 since later has import path issues 2021-04-08 10:34:14 +09:30
Christian Decker 0dd57cbf3c pyln-spec: Clean up the setup.py files
This centralizes the setup.py file, and parametrizes it so it can
auto-detect which bolt we are building. It also uses trick 3 from [1]
to avoid importing the package itself during the manifest creation,
which'd cause an import error due to missing dependencies.

[1] https://packaging.python.org/guides/single-sourcing-package-version/
2021-04-08 10:34:14 +09:30
Christian Decker 6b0a7b173c pyln: Change the setup.py file not to import the package
This would lead to errors about missing dependencies when attempting
to install using `pyhon setup.py install`. This is because the
`setup.py` file effectively is the manifest file used to discover
which dependencies are needed, so when using it to detect dependencies
we obviously don't have them yet.

See https://packaging.python.org/guides/single-sourcing-package-version/
2021-04-08 10:34:14 +09:30
Rusty Russell fad4aee5da pyln: remove deprecated fundchannel/fundchannel_start variants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: pyln: removed deprecated fundchannel/fundchannel_start `satoshi` arg.
2021-04-07 14:34:39 +09:30
Rusty Russell 0ed614d5c4 pyln: remove deprecated pay/sendpay variants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: pyln: removed deprecated pay/sendpay `description` arg.
2021-04-07 14:34:39 +09:30
Rusty Russell 10dee030f1 pyln: remove deprecated close variant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: pyln: removed deprecated close `force` variant.
2021-04-07 14:34:39 +09:30
niftynei cbde1f8158 pyln-proto: make an error message more detailed
Easier to figure out what's going wrong
2021-03-30 13:44:34 +10:30
niftynei 5142dc81f6 pyln-proto: write out length of arrays of subtypes to wire
We weren't writing out the length of a nested subtype's
dynamicarraylenght, now we do. The trick is to iterate through the
fields on a subtype (since the length field is added separately)
and to also iterate down through the otherfield values as we 'descend'
2021-03-30 13:44:34 +10:30
Rusty Russell dc44737fb8 python: handle psbt-version of fundchannel_complete, deprecate old.
I didn't change the callers yet, so I can test the backwards compat code
works.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:10:07 +10:30
Rene Pickhardt da2ee188f3 added the default bitcoin path for macos to startup_regtest.sh
[ Neatened shell script a little, removed unwanted submodule update --RR ]
2021-03-16 10:40:34 +10:30
niftynei 9776e23b83 contrib: add the dev- options to the config iff DEVELOPER=1 configured
We scan config.vars to figure out if you configured developer on or off.
If it's on, we add the dev-only options to the config.

Fixes: #4400
Reported-By: Jonathan Harvey-Buschel @jharveyb
2021-03-16 10:26:51 +10:30
niftynei bec96a6c5b df: add openchannel_abort command
Allows us to clean up an in-progress open that we won't be completing

Changelog-Added: EXPERIMENTAL JSON-RPC: Permit user-initiated aborting of in-progress opens. Only valid for not-yet-committed opens and RBF-attempts
2021-03-15 14:08:44 +10:30
Alexey Zagarin 43c46ceb2d Fix wallet creation in contrib/startup_regtest.sh 2021-03-15 12:16:38 +10:30
Christian Decker 1e6626ff43 pyln: Fix listfunds call defaulting to spent=False
The semantics don't change, since `lightningd` will use false as
default as well, however setting it to something other than `None`
causes the RPC library to include the parameter in the query, and
since the parameter was introduced only in 0.9.3 and pyln may be used
with older versions this then results in an error about an unknown
parameter.
Setting this to `None` makes sure pyln filters out the argument before
calling.

Changelog-Fixed: pyln: Fixed an error when calling `listfunds` with an older c-lightning version causing an error about an unknown `spent` parameter
2021-03-11 14:07:47 +10:30
Christian Decker ba083c8869 pyln: Plugins can be nice if invoked from CLI too
I had way too much fun with this and got a bit carried away with the
letter writing. The idea is to be helpful when users start the plugin
from the command line, rather than run it under the control of
lightningd. We also print detailed information about the user-visible
things such as the methods and options exposed by the plugin.

Changelog-Added: pyln: Plugins that are run from the command line print helpful information on how to configure c-lightning to include them and print metadata about what RPC methods and options are exposed.
Suggested-by: Rusty Russell <@rustyrussell>
2021-03-10 21:02:55 +10:30
niftynei 1e9a900918 df: start of RPC for bumping the fee!
Doesn't call into dualopend yet
2021-03-03 16:19:04 -06:00
Karol Hosiawa 7b72ea7061 Added listforwards test 2021-03-03 09:19:20 +10:30
Rusty Russell 8ad6972b25 contrib: make startup_regtest.sh work for modern bitcoind.
They need us to create a wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 14:44:03 +10:30
niftynei ea95ad9c12 fund/utxopsbt: flag 'excess_as_change' to add a change output for excess
In the case where you want a PSBT and also want the output to be added
as a change address, use `excess_as_change` = true.

Generates a change address to use. If you want to pay the excess
elsewhere, you will have to add separately.

Changelog-Added: JSON-RPC: Add new parameter `excess_as_change` to fundpsbt+utxopsbt
2021-02-04 13:25:34 -06:00
Rusty Russell f3159ec4ac pytest: detect warnings, too.
Since we turned many errors into warnings, we want our tests to fail
when they happen unexpectedly.  We make WARNING clear in the strings
we print, too, to help out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:52 +10:30
Michael Schmoock c8198d9820 pyln-testing: add `scids` param to `wait_for_htlcs` 2021-02-04 12:02:15 +10:30
Michael Schmoock d76ca6ed35 pyln-testing: improve `wait_for` a bit
This 'fixes' the `wait_for` helper by removing a pointless final
`time.sleep()`, thus potentially making the method return quicker.

The old code could have had three final states:
 - success() := True
 - Timeout and success() := True
 - Timeout and success() := False

The new code has just two final state:
 - success() := True
 - Timeout and success() := False

It ensures the final `time.sleep()` is just the right amount before timeout.
And more importantly making it more readable :-)

Changelog-None
2021-02-04 12:02:15 +10:30
Rusty Russell 3c5502426b lightningd: addgossip API to inject gossip messages.
Importantly, this is synchronous, so pay will be able to use it
reliably.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-02 13:44:01 +01:00
Christian Decker a5f16ab5b1 pyln: Catch OSError when cleaning up test directories 2021-01-29 10:29:09 +10:30
Christian Decker ea67710e01 pyln: Pretty print RPC calls in the testing framework
We are printing `repr(obj)` which is not pretty-printed, hard to read,
and can't even be copied and inspected to JSON tools. We now print the
JSONified and indented calls and responses for easier debugging based
on solely the logs (useful for CI!).

Changelog-Added: pyln-testing: The RPC client will now pretty-print requests and responses to facilitate log-based debugging.
2021-01-29 10:29:09 +10:30
Christian Decker cd9aa267b4 pyln: Adjust maximum load allowed by the throttler 2021-01-29 10:29:09 +10:30
Christian Decker 4c3ee04bb7 pyln: Use a fair FS lock to throttle node startups
We were getting a couple of starvations, so we need a fair filelock. I
also wasn't too happy with the lock as is, so I hand-coded it quickly.

Should be correct, but the overall timeout will tell us how well we
are doing on CI.
2021-01-29 10:29:09 +10:30
Christian Decker fc677e331a pyln: Update dependencies for all pyln packages
We were getting a number of incompatibility warning due to the
dependencies being expressed too rigidly. This losens the requirement
definitions to being compatible with a known good version, and while
we're at it we also bump all outdated requirements.
2021-01-29 10:29:09 +10:30
Christian Decker b447944285 gci: Add a tester Dockerfile 2021-01-29 10:29:09 +10:30
Christian Decker 6bed85bac2 pyln: Bump pyln version to 0.9.3 before publishing on PyPI
Let's just keep pyln synched up with the c-lightning version
2021-01-27 11:29:56 +01:00
Christian Decker 483579f8b6 pyln: Pin the mypy dependency to 0.790 since 0.800 has a regression 2021-01-26 15:45:13 +01:00
nicolas.dorier 3105a7a204 Fix dockerfiles 2021-01-26 15:03:35 +01:00
niftynei ea1895fc1e pyln-tests: tweak `fundbalancedchannel` to assume peer will match
we got rid of push_msats for dual funded channels. this assumes that hte
peer will match an equal amount of sats as ours (the df_accepter.py
plugin will do this)
2021-01-10 13:44:04 +01:00
niftynei f4b7904ce5 tests: don't enforce daemon name on opening logline 2021-01-10 13:44:04 +01:00
niftynei 3c1d90086f pyln-testing: add 'config' method to a node
Query for a config's value. If not set, returns 'none'
2021-01-10 13:44:04 +01:00
Christian Decker a91254de11 pyln: Add ammag key to onion keyset
This was missing, and is required to wrap error responses.
2021-01-08 19:28:30 +01:00
Rusty Russell 1a8978100e invoice: add ctlv option.
This is required if we want to create a "bouncer" plugin (in my copious free time!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
2021-01-08 10:45:56 +01:00
Christian Decker b6650425b9 pyln: Add notifications support to LightningRpc
Changelog-Added: pyln-client: Added support for command notifications to LightningRpc via the `notify` context-manager.
2021-01-07 08:32:40 +01:00