Commit Graph

351 Commits

Author SHA1 Message Date
Rusty Russell b33dc9847c pyln.proto.message: python-fluency feedback from @darosior
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell 8f38bc7230 pyln.proto.message: support adding two namespaces.
They must not have duplicate names!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell 85bb93618b new modules: pyln.proto.message.{bolt1,bolt2,bolt4,bolt7}
These are autogenerated, but now they export their own
MessageNamespace, as well as the raw csv.

They also expose their SubtypeTypes, MessageTypes and TlvStreamTypes,
though in theory these could clash (they don't for now, and it'd be
kinda awkward if they did).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell 59bb6b90ac pyln: new module pyln.proto.message.bolts
This contains the CSVs for the current bolts (autogenerated).  It's a
separate module because I expect it to be updated alongside the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: new module pyln.proto.message.bolts
2020-06-12 15:35:35 +02:00
Rusty Russell daa707d213 pyln.proto.message: separate fundamental types from other subtypes.
This will be useful for the next patch, which introduces per-bolt
modules.  This makes it easier for them generate variables for each
field type they parse (they don't want to export u16, for example)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell e4b5679f5d message: support option fields.
These are (probably) going away soon, but just tag them for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell 9992a577b4 pyln: add (undocumented) u8 fundamental type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell 42aab89b9f pyln.proto.message: expose fundamental MessageTypes as variables.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell 47631cc23c pyln.proto.message: use BufferedIOBase instead of bytes for binary ops.
Instead of val_to_bin/val_from_bin which deal with bytes, we implement
read and write which use streams.  This simplifies the API. 

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell ed4eadc8f3 patch message-export-types.patch 2020-06-12 15:35:35 +02:00
Rusty Russell eb73a0dd8f pyln: add pyln.proto.message.
This supports infrasructure for creating messages.  In particular, it
can be fed CSV from the spec's `tools/extract-formats.py` and then convert
them all to and from strings and binary formats.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: new module pyln.proto.message
2020-06-12 15:35:35 +02:00
Rusty Russell 5b4be02ed3 pyln: add Makefile
This runs flake8 and the python tests.  Helps me, at least!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
niftynei 9845eb41a9 contrib: allow stderr printing for startup_regtest
Moves the 'daemon'ization from c-lightning to the process level, so that
stderr print messages appear in the terminal. Easier debugging!

Changelog-None
2020-05-19 11:44:54 +02:00
mb300sd b6285ffa5d Fix bash completion.
At some point lightning-cli help defaulted to human readable format, the additional -H broke the bash completion.

Changelog-Fixed: bash completion on lightning-cli now works again
2020-05-15 15:25:33 +02:00
Rusty Russell 97e3d61748 pyln: fix incorrect python syntax.
contrib/pyln-proto/pyln/proto/bech32.py:120
  /home/rusty/devel/cvs/lightning/contrib/pyln-proto/pyln/proto/bech32.py:120: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    assert decode(hrp, ret) is not (None, None)

I think this warning is correct (though I don't see the warning once I installed coincurve:
are we suppressing warnings?)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:21:50 +02:00
Candle fbfe0e6c0b Comment optimisations
I quickly ran through the comments in lightning.py and saw a few small inconsistencies:

- upper/lower case for the "B" in "Bitcoin" unified (see https://github.com/lnbook/lnbook/pull/98)
- added missing "." after a complete sentence
- removed unnecessary double spaces
2020-05-12 22:19:34 +02:00
Michal Rostecki db0a2c082a pyln-proto: Use only coincurve for libsecp256k1 bindings
secp256k1 Python library is not maintained anymore and coincurve was
already used in the `wire` module.

Changelog-None
Signed-off-by: Michal Rostecki <mrostecki@mailfence.com>
2020-05-07 11:37:34 +02:00
Christian Decker 7b752e00d5 pyln: Set the bitcoin datadir when running tests
Telling `lightningd` to pass a `-datadir` to `bitcoin-cli` so it doesn't go
snooping where it doesn't belong (i.e., the user's home directory and config).

Changelog-None
Suggested-by: Simon Vrouwe <@SimonVrouwe>
Signed-off-by: Christian Decker <@cdecker>
2020-05-05 13:28:54 +09:30
Christian Decker 9bfdf234f3 pyln-proto: Avoid circular dependency in setup.py
It seems that loading the version from the source files triggers imports that
may not yet have been installed.
2020-05-05 13:28:54 +09:30
Sebastian Falbesoner 94c15f5cc0 contrib: add executable flag for bootstrap-node.sh
Changelog-None
2020-05-05 13:27:55 +09:30
rbndg 83b78fd327 Unpacking variable for cleaner code for testing 2020-05-04 19:18:05 +02:00
rbndg 241fa00e97 plugin:added invoice creation event
New invoice_creation event triggered when an new invoice is created

Changelog-Added: plugin: New invoice_creation plugin event
2020-05-04 19:18:05 +02:00
Christian Decker 64b0487228 pyln-testing: Just warn if we get an unexpected fee request
This was breaking a couple of tests if the pyln version was not synced up with
`lightningd`, so now we just warn (these are then collected when running in
pytest and highlighted).
2020-05-01 18:20:33 -05:00
Christian Decker 264b2d1975 pyln-testing: Add more meaningful error for mock_estimatefees
We only support a very limited number of argument combinations, and apparently
sometimes we trigger a case we aren't handling. This adds a more useful error
message, including the params we didn't match.
2020-04-20 07:48:32 +09:30
lisa neigut 9b42d9d8ba build: add mkrd to docker install? 2020-04-19 08:07:47 +09:30
Christian Decker f3315ca110 pyln-proto: Add Tu32Field and Tu64Field helpers
These are necessary for the interim keysend plugin
2020-04-16 18:03:35 +09:30
darosior 11acd902ff pyln-testing: specify fallbackfee for newer versions of bitcoind
Changelog-None
2020-04-13 19:50:05 +09:30
darosior af76bce0fa pyln-client: send proper JSONRPC2 errors on failed call
Changelog-Added: pyln now sends proper error on bad calls to plugin methods

Co-Authored-By: Sergi Delgado Segura <sergi.delgado.s@gmail.com>
2020-04-11 12:17:51 +09:30
Christian Decker 3b09662768 pyln: Allow non-empty directory when creating node in node_factory
So far we've always cleared the node directory when provisioning the node, but
while testing some plugins we noticed that pre-generating some files in that
directory is useful. This just adds yet another flag to `get_node` that
disables deleting any existing node directory.
2020-04-11 08:43:06 +09:30
Vasil Dimov 158d2212c2 closingd: configurable closing fee negotiation step
When negotiating the transaction fee for closing a channel [1], we used
to always pick the middle of the range between our proposal and the
peer's proposal.

Introduce a new option `fee_negotiation_step` to the close command, so
the peer who initiates the close can choose his back off step.

Partially resolves https://github.com/ElementsProject/lightning/issues/3270

[1] https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#closing-negotiation-closing_signed

Changelog-Added: New optional parameter to the `close` command to control the closing transaction fee negotiation back off step
2020-04-07 13:52:48 +09:30
darosior 242ddb5660 pyln-testing: allow to customize --allow-deprecated-apis 2020-04-05 11:06:27 +09:30
darosior 7c0af81c21 bcli: use a more urgent feerate for HTLCs and penalty transactions
A CONSERVATIVE/3 target for them.

Some noisy changes to the tests as we had to update the estimatesmartfee
mock.

Changelog-Changed: We now use a higher feerate for resolving onchain HTLCs and for penalty transactions
2020-04-01 23:02:47 -05:00
darosior dce2e87928 chaintopology: better feerate targets differentiation
We kept track of an URGENT, a NORMAL, and a SLOW feerate. They were used
for opening (NORMAL), mutual (NORMAL), UNILATERAL (URGENT) transactions
as well as minimum and maximum estimations, and onchain resolution.

We now keep track of more fine-grained feerates:
- `opening` used for funding and also misc transactions
- `mutual_close` used for the mutual close transaction
- `unilateral_close` used for unilateral close (commitment transactions)
- `delayed_to_us` used for resolving our output from our unilateral close
- `htlc_resolution` used for resolving onchain HTLCs
- `penalty` used for resolving revoked transactions

We don't modify our requests to our Bitcoin backend, as the next commit
will batch them !

Changelog-deprecated: The "urgent", "slow", and "normal" field of the `feerates` command are now deprecated.
Changelog-added: The fields "opening", "mutual_close", "unilateral_close", "delayed_to_us", "htlc_resolution" and "penalty" have been added to the `feerates` command.
2020-03-30 20:17:18 +10:30
Michael Schmoock e71e44ac8b pyln-testing: cleanup of pay() helper function 2020-03-27 12:33:23 +01:00
Michael Schmoock 80ff9c5b63 test: refactor lockup_drain add helpers drain and force_feerates 2020-03-27 12:33:23 +01:00
Christian Decker 7201cb7315 pytest: Configure logging in a fixture to match stdout capturing
pytest captures the output by monkey patching out `sys.stdout`. This may
conflict with our use of `sys.stdout` when configuring logging, resulting in
the "Write to closed file" issue that is spamming the logs. By making the
logging configuration a fixture hopefully we always use the correct
stdout (after pytest has monkey-patched it).
2020-03-24 09:52:33 +10:30
Christian Decker cf8c972883 pyln-testing: Print a list of files if we can't remove the test dir
For some reason we fail to remove the test directory in some cases. My
hypothesis is that it is a daemon that is not completely shut down yet, and
still writes to the directory. This commit intercepts the error, prints any
files in the directory and re-raises the error. This should allow us to debug
the reappears.
2020-03-24 09:52:33 +10:30
Christian Decker 3e3b05e1b2 pyln: Migrate remaining uses of the deprecated pylightning module
`pylightning` is not much more than an alias for `pyln-client`, so this
removes the need to install that as well just to run the tests.
2020-03-24 09:52:33 +10:30
Christian Decker 24aaf73982 pytest: Actually make sure that the direcory exists
Some tests may not spawn a node at all, so make sure that our assumption that
the directory exists in the fixture cleanup is correct by creating the
directory.
2020-03-24 09:52:33 +10:30
lisa neigut 42cce55b45 plugins: add 'flag' type for plugin options
Updates the plugin docs to include more detailed info about how options
work.

Changelog-Added: Plugins: 'flag'-type option now available.
2020-03-21 16:29:52 +10:30
lisa neigut d19cddf00f pyln: enforce types of options
we loosely enforce that the specified type must be one of the listed
options. you can still cause an error because we're not checking the
default value you're passing in ...

not sure if this is totally necessary, should we jsut let clightning
enforce the input?
2020-03-10 13:25:36 +10:30
lisa neigut 34cef2cac3 pyln-testing: add flag 'expect_fail' to node factory get_node
if the node fails to start (and we're expecting it to) return to us the
node object anyway

we also signal to collect all of its stderr logs by setting stderr
on the tailableproc that backs the node
2020-03-10 13:25:36 +10:30
lisa neigut 0cf3e19e0b pyln-testing: save stderr logs for checking
just for convenience's sake
2020-03-10 13:25:36 +10:30
ZmnSCPxj jxPCSnmZ d9b2482415 lightningd/hsm_control.c: Implement `getsharedsecret`.
ChangeLog-Added: New `getsharedsecret` command, which lets you compute a shared secret with this node knowing only a public point. This implements the BOLT standard of hashing the ECDH point, and is incompatible with ECIES.
2020-02-28 14:45:50 +10:30
Christian Decker 7f6f324590 python2: Remove python2 dependency from docs and dockerfiles 2020-02-27 09:18:24 +10:30
Christian Decker 87e924e270 docker: Remove Dockerfile for i386 builder
We aren't using it and it's broken with the specified upstream image, so
remove it outright.
2020-02-27 09:18:24 +10:30
Michael Schmoock dd334df80d fix: makes testutils ln.pay assert status complete
`waitsendpay` in `lightningd/pay.c` can also return 'pending' and 'failed',
we should check that.
2020-02-18 10:10:17 +10:30
Michael Schmoock 3e2607bf2d chore: updates coincurve requirement to v13
Update to v13 as v12 started to raise undefined symbol exceptions agains latest
libsecp256k1.

Note: any version of `pip install coincurve` fails if no libsecp256k1 headers
are installed, should we point this out somewhere/somehow?

Changelog-None
2020-02-18 10:10:17 +10:30
Christian Decker 966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
Christian Decker a13591400a pyln-client: Add facility to register featurebits from plugins
we have 4 venues in which we can add features, 3 of which are unilaterally
controlled (`init`, `node_announcement`, and `invoices`) the
`channel_announcement` is co-signed by both parties, so we can't add
featurebits without additional coordination overhead.

Each location is encoded as a key-value pair in a dict called `featurebits` in
the manifest (omitted if no custom featurebits are set).
2020-02-11 13:53:31 +10:30