Commit Graph

9272 Commits

Author SHA1 Message Date
Antoine Poinsot a441485a35 lightningd: regroup hsm_secret password input logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 917f78a4f8 lightningd: group hsm_secret encryption key derivation
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot 38204f9b6d hsmtool: use errx() instead of err() everywhere
errx() was printing the confusing errno as well ("Error could not [...] :Success")

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Rusty Russell d41b383cf7 pyln-testing: suppress mypy on FileLock
3.0.12 here:
```
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-06 12:15:39 +01:00
niftynei 0c9072272f df-plugin: add `signonly` to signing in df_accepter
We're the accepter, so we look up which inputs we need to sign and only
sign those.
2021-01-05 19:12:00 +01:00
niftynei d98bd39d60 dual_open: neaten spacing for macro 2021-01-05 19:12:00 +01:00
niftynei 55bbdf1866 df: update an error message to give more info 2021-01-05 19:12:00 +01:00
niftynei 713bb7491b json: allow helper to take/free psbt
Makes things a bit cleaner elsewhere
2021-01-05 19:12:00 +01:00
niftynei e3a1d1a7f3 pyln-client: to_whole_satoshi returns the rounded up satoshi value
A fractional satoshi value isn't really useful; rounding up loses
precision but that's why you called "whole satoshi", wasn't it?

Changelog-Changed: pyln-client: Millisatoshi has new method, `to_whole_satoshi`; *rounds value up* to the nearest whole satoshi
2021-01-05 19:12:00 +01:00
niftynei cce0020568 pyln-client: add missing `min_witness_weight` to fundpsbt/utxopsbt
We need to use it for the 'df_accepter' plugin, so we get the feerate
correct.

Changelog-Added: pyln-client: `fundpsbt`/`utxopsbt` now support `min_witness_weight` param
2021-01-05 19:12:00 +01:00
Antoine Poinsot 19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot dd1def335c hsmtool: reuse read_stdin_pass for `generatehsm` passphrase
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Antoine Poinsot 1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker 5b3af00e28 pytest: Really only check DB statements if the env-var is "1" 2021-01-04 11:25:50 +01:00
Christian Decker 94820ad559 pytest: Disable test_funding_cancel_race on slow machines
The CI regularly trips up on this very taxing test (100 nodes) so only
run it if we have the required horsepower.
2021-01-04 11:25:50 +01:00
Christian Decker dbb83a1bc3 pyln: Do not rstrip() the return value of .append()
This was causing the following error

```
Exception in thread Thread-553:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/tmp/cirrus-ci-build/contrib/pyln-testing/pyln/testing/utils.py", line 232, in tail
    self.err_logs.append(line.rstrip().decode('UTF-8', 'replace')).rstrip()
AttributeError: 'NoneType' object has no attribute 'rstrip'

[gw5] [ 33%] FAILED tests/test_misc.py::test_bitcoin_failure
```

Notice the second call to `.rstrip()` on the return value of `.append()`
2021-01-04 11:25:50 +01:00
Christian Decker b3fd47e5f6 pytest: Stabilize test_routing_gossip
We were getting bad gossip because some nodes discarded the channel
announcement for being in the future. This is because the node was, at
that time, below the confirmation height. It'd then discard the
followup messages because not preceded by an announcement, and getting
upset about that.
2021-01-04 11:25:50 +01:00
Christian Decker 1ff57f07f8 pyln: Check that the stderr line is not none in pyln-testing tail
I stumbled over this in a test run and it seems benign.
2021-01-04 11:25:50 +01:00
Christian Decker 7e867e5ee6 pytest: Add throttler to limit load on the test system
Both my machine and apparently the CI tester machines regularly run
into issues with load on the system, causing timeouts (and
unresponsiveness). The throttler throttles the speed with which new
instances of c-lightning get started to avoid overloading. Since the
plugin used for parallelism when testing spawns multiple processes we
need to lock on the fs. Since we have that file open already, we'll
also write a couple of performance metics to it.
2021-01-04 11:25:50 +01:00
Christian Decker f2a0a4abfc pyln: Add logging to bitcoind RPC calls
Useful if we want to debug a bit better
2021-01-04 11:25:50 +01:00
Christian Decker 8b09ac38aa pyln: Log mempool when we generate blocks in BitcoinD fixture
We sometimes have very specific sequences of tx broadcasts and blocks
being generated to confirm them. If the confirmation is missed the
test can completely get out of sync. Make debugging this easier by
logging what we confirmed.
2021-01-04 11:25:50 +01:00
fiatjaf 6af8adde26 fix memset for route_hop json parsing when no channel_id exists.
Changelog-None
2021-01-04 11:24:55 +01:00
niftynei 6030d0542a bcli: print error message in debug log for broadcast failures 2020-12-24 11:38:49 +01:00
niftynei 47d9ff88df bcli: only print error if exitcode is non-zero 2020-12-24 11:38:49 +01:00
niftynei 08c8581461 bcli: allocate temp string off of short-lived cmd
`cmd` gets cleaned up almost immediately, and is the right scope for a
temporary string allocation
2020-12-24 11:38:49 +01:00
niftynei f9a2489424 bcli: print sendrawtx every time (even on failures) 2020-12-24 11:38:49 +01:00
Rusty Russell 52af729641 plugins/offer and plugins/fetchinvoice: send and recv errors.
This also lets us extend our testing to cover error cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 96caf9f4ab fetchinvoice: return the next period for recurring offers.
This is useful for the caller to know when to call again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 19af1d10e6 invoice: hack in merkle of invoice as "payment_secret" (EXPERIMENTAL_FEATURES)
This lets actually pay the invoice that fetchinvoice returns.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 2baa410e2b pytest: finally test fetchinvoice send, and offer plugin receive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 69ddf818c7 plugins/fetchinvoice: catch invoice_request reply.
Using onionmessage hook, we get the response and either present it
to the user (invoice) or return the error to the user.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 3eada358e9 plugins/fetchinvoice: plugin to send an invoice_request for a given offer
Doesn't catch the reply yet, but prepares the invoice request based on
the offer and sends it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 43b71de897 lightningd: low-level createinvoicerequest API (EXPERIMENTAL_FEATURES)
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database.  We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 59efd160c1 hsmd: code to sign bolt12 messages with a tweaked key.
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 9d75fbe237 plugins/offers: add code to catch invoice_request onion messages.
We prepare an invoice, but there's no code to send an error reply yet
(future patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 08e110b568 JSON: offers plugin `offer` command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 26e0cbeb7c plugin/offers: skeleton for offers plugin (EXPERIMENTAL_FEATURES)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 1293852542 common/bolt12: add calculations for offer periods.
It's not trivial, but fortunately gmtime and mktime exist already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 353e4cf3c2 wire/Makefile: make generated wire files depend on Makefile.
That way they rebuild when we change flags in Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Karol Hosiawa e582c0ac0a Added listfunds test 2020-12-22 13:00:02 +01:00
Karol Hosiawa 4a3dec660f Added spent option to listfunds in pyln-client 2020-12-22 13:00:02 +01:00
Karol Hosiawa a2c208e121 Added spent option to listfunds
Changelog-Added: JSON-RPC: The `listfunds` method now includes spent outputs if the `spent` parameter is set to true.
2020-12-22 13:00:02 +01:00
Rusty Russell eb4062bb14 setchannelfee: fix crash when channel is not in valid state.
You can't fail a cmd when you've already started streaming
a successful response:

lightningd: ccan/ccan/json_out/json_out.c:343: json_out_finished: Assertion `tal_count(jout->wrapping) == 0' failed.
lightningd: FATAL SIGNAL 6 (version v0.9.2-119-gf7cdf1d)
0x1847d1 send_backtrace
	common/daemon.c:38
0x184877 crashdump
	common/daemon.c:51
0x5bda03f ???
	???:0
0x5bd9fb7 ???
	???:0
0x5bdb920 ???
	???:0
0x5bcb489 ???
	???:0
0x5bcb501 ???
	???:0
0x1e07a8 json_out_finished
	ccan/ccan/json_out/json_out.c:343
0x18db0a json_stream_double_cr
	common/json_stream.c:95
0x18dbf3 json_stream_close
	common/json_stream.c:117
0x12fd98 command_raw_complete
	lightningd/jsonrpc.c:459
0x12fec9 command_failed
	lightningd/jsonrpc.c:488
0x12ffb9 command_fail
	lightningd/jsonrpc.c:503
0x14dc20 json_setchannelfee
	lightningd/peer_control.c:2052

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSONRPC: `setchannelfee` would fail an assertion if channel wasn't in normal state.
2020-12-21 17:10:28 -06:00
fiatjaf 0fb4c1112d add a note to "unknown parameter" error about -k autodetect failure. 2020-12-21 12:17:43 +00:00
Rusty Russell 77478408f9 pyln: add RpcException for finer method failure control.
Allows caller to set code and exact message to be returned.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln-client: plugins can now raise RpcException for finer control over error returns.
2020-12-16 12:37:14 +01:00
ZmnSCPxj jxPCSnmZ 423f3944e3 doc/BACKUP.md: Minor typo due to rearranged text.
Changelog-None
2020-12-16 12:36:52 +01:00
Jan Sarenik 0401c60d43 common: change default network to mainnet for all
Changelog-changed: Default network on both new and old
installs is now bitcoin. The warning "default network
changing in 2020" was removed.

Reverts 36c517b
Fixes #4159
2020-12-16 11:48:25 +01:00
ZmnSCPxj jxPCSnmZ 0e326dd3a7 doc/BACKUP.md: Document backup strategies for `lightningd`.
ChangeLog-Added: Document: `doc/BACKUP.md` describes how to back up your C-lightning node.
2020-12-15 12:16:50 +01:00