Commit Graph

9358 Commits

Author SHA1 Message Date
Antoine Poinsot 1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker 5b3af00e28 pytest: Really only check DB statements if the env-var is "1" 2021-01-04 11:25:50 +01:00
Christian Decker 94820ad559 pytest: Disable test_funding_cancel_race on slow machines
The CI regularly trips up on this very taxing test (100 nodes) so only
run it if we have the required horsepower.
2021-01-04 11:25:50 +01:00
Christian Decker dbb83a1bc3 pyln: Do not rstrip() the return value of .append()
This was causing the following error

```
Exception in thread Thread-553:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/tmp/cirrus-ci-build/contrib/pyln-testing/pyln/testing/utils.py", line 232, in tail
    self.err_logs.append(line.rstrip().decode('UTF-8', 'replace')).rstrip()
AttributeError: 'NoneType' object has no attribute 'rstrip'

[gw5] [ 33%] FAILED tests/test_misc.py::test_bitcoin_failure
```

Notice the second call to `.rstrip()` on the return value of `.append()`
2021-01-04 11:25:50 +01:00
Christian Decker b3fd47e5f6 pytest: Stabilize test_routing_gossip
We were getting bad gossip because some nodes discarded the channel
announcement for being in the future. This is because the node was, at
that time, below the confirmation height. It'd then discard the
followup messages because not preceded by an announcement, and getting
upset about that.
2021-01-04 11:25:50 +01:00
Christian Decker 1ff57f07f8 pyln: Check that the stderr line is not none in pyln-testing tail
I stumbled over this in a test run and it seems benign.
2021-01-04 11:25:50 +01:00
Christian Decker 7e867e5ee6 pytest: Add throttler to limit load on the test system
Both my machine and apparently the CI tester machines regularly run
into issues with load on the system, causing timeouts (and
unresponsiveness). The throttler throttles the speed with which new
instances of c-lightning get started to avoid overloading. Since the
plugin used for parallelism when testing spawns multiple processes we
need to lock on the fs. Since we have that file open already, we'll
also write a couple of performance metics to it.
2021-01-04 11:25:50 +01:00
Christian Decker f2a0a4abfc pyln: Add logging to bitcoind RPC calls
Useful if we want to debug a bit better
2021-01-04 11:25:50 +01:00
Christian Decker 8b09ac38aa pyln: Log mempool when we generate blocks in BitcoinD fixture
We sometimes have very specific sequences of tx broadcasts and blocks
being generated to confirm them. If the confirmation is missed the
test can completely get out of sync. Make debugging this easier by
logging what we confirmed.
2021-01-04 11:25:50 +01:00
fiatjaf 6af8adde26 fix memset for route_hop json parsing when no channel_id exists.
Changelog-None
2021-01-04 11:24:55 +01:00
niftynei 6030d0542a bcli: print error message in debug log for broadcast failures 2020-12-24 11:38:49 +01:00
niftynei 47d9ff88df bcli: only print error if exitcode is non-zero 2020-12-24 11:38:49 +01:00
niftynei 08c8581461 bcli: allocate temp string off of short-lived cmd
`cmd` gets cleaned up almost immediately, and is the right scope for a
temporary string allocation
2020-12-24 11:38:49 +01:00
niftynei f9a2489424 bcli: print sendrawtx every time (even on failures) 2020-12-24 11:38:49 +01:00
Rusty Russell 52af729641 plugins/offer and plugins/fetchinvoice: send and recv errors.
This also lets us extend our testing to cover error cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 96caf9f4ab fetchinvoice: return the next period for recurring offers.
This is useful for the caller to know when to call again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 19af1d10e6 invoice: hack in merkle of invoice as "payment_secret" (EXPERIMENTAL_FEATURES)
This lets actually pay the invoice that fetchinvoice returns.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 2baa410e2b pytest: finally test fetchinvoice send, and offer plugin receive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 69ddf818c7 plugins/fetchinvoice: catch invoice_request reply.
Using onionmessage hook, we get the response and either present it
to the user (invoice) or return the error to the user.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 3eada358e9 plugins/fetchinvoice: plugin to send an invoice_request for a given offer
Doesn't catch the reply yet, but prepares the invoice request based on
the offer and sends it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 43b71de897 lightningd: low-level createinvoicerequest API (EXPERIMENTAL_FEATURES)
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database.  We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 59efd160c1 hsmd: code to sign bolt12 messages with a tweaked key.
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 9d75fbe237 plugins/offers: add code to catch invoice_request onion messages.
We prepare an invoice, but there's no code to send an error reply yet
(future patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 08e110b568 JSON: offers plugin `offer` command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 26e0cbeb7c plugin/offers: skeleton for offers plugin (EXPERIMENTAL_FEATURES)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 1293852542 common/bolt12: add calculations for offer periods.
It's not trivial, but fortunately gmtime and mktime exist already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell 353e4cf3c2 wire/Makefile: make generated wire files depend on Makefile.
That way they rebuild when we change flags in Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Karol Hosiawa e582c0ac0a Added listfunds test 2020-12-22 13:00:02 +01:00
Karol Hosiawa 4a3dec660f Added spent option to listfunds in pyln-client 2020-12-22 13:00:02 +01:00
Karol Hosiawa a2c208e121 Added spent option to listfunds
Changelog-Added: JSON-RPC: The `listfunds` method now includes spent outputs if the `spent` parameter is set to true.
2020-12-22 13:00:02 +01:00
Rusty Russell eb4062bb14 setchannelfee: fix crash when channel is not in valid state.
You can't fail a cmd when you've already started streaming
a successful response:

lightningd: ccan/ccan/json_out/json_out.c:343: json_out_finished: Assertion `tal_count(jout->wrapping) == 0' failed.
lightningd: FATAL SIGNAL 6 (version v0.9.2-119-gf7cdf1d)
0x1847d1 send_backtrace
	common/daemon.c:38
0x184877 crashdump
	common/daemon.c:51
0x5bda03f ???
	???:0
0x5bd9fb7 ???
	???:0
0x5bdb920 ???
	???:0
0x5bcb489 ???
	???:0
0x5bcb501 ???
	???:0
0x1e07a8 json_out_finished
	ccan/ccan/json_out/json_out.c:343
0x18db0a json_stream_double_cr
	common/json_stream.c:95
0x18dbf3 json_stream_close
	common/json_stream.c:117
0x12fd98 command_raw_complete
	lightningd/jsonrpc.c:459
0x12fec9 command_failed
	lightningd/jsonrpc.c:488
0x12ffb9 command_fail
	lightningd/jsonrpc.c:503
0x14dc20 json_setchannelfee
	lightningd/peer_control.c:2052

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSONRPC: `setchannelfee` would fail an assertion if channel wasn't in normal state.
2020-12-21 17:10:28 -06:00
fiatjaf 0fb4c1112d add a note to "unknown parameter" error about -k autodetect failure. 2020-12-21 12:17:43 +00:00
Rusty Russell 77478408f9 pyln: add RpcException for finer method failure control.
Allows caller to set code and exact message to be returned.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln-client: plugins can now raise RpcException for finer control over error returns.
2020-12-16 12:37:14 +01:00
ZmnSCPxj jxPCSnmZ 423f3944e3 doc/BACKUP.md: Minor typo due to rearranged text.
Changelog-None
2020-12-16 12:36:52 +01:00
Jan Sarenik 0401c60d43 common: change default network to mainnet for all
Changelog-changed: Default network on both new and old
installs is now bitcoin. The warning "default network
changing in 2020" was removed.

Reverts 36c517b
Fixes #4159
2020-12-16 11:48:25 +01:00
ZmnSCPxj jxPCSnmZ 0e326dd3a7 doc/BACKUP.md: Document backup strategies for `lightningd`.
ChangeLog-Added: Document: `doc/BACKUP.md` describes how to back up your C-lightning node.
2020-12-15 12:16:50 +01:00
Rusty Russell 723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 0fb0b94e8d common/blindedpath: fix type of enctlv.
This didn't compile, but since nothing depends yet on common/blindedpath.o
we don't build it yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 9fe612db20 plugins/pay: allow bolt12 invoices (EXPERIMENTAL_FEATURES).
Note that we remove the redundant "is this the correct chain?"
check, since bolt11_decode and bolt12_decode do that internally
anyway (this was changed in 924cc04bd2).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell a33e39b7e8 pay, wallet: rename internal `bolt11` vars to `invstring`.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 4c4288e3e5 invoice: handle bolt12 strings if EXPERIMENTAL_FEATURES.
The database still calls them `bolt11`, but we treat them depending on
prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell fab0842d31 lightningd: createinvoice low-level invoice creation command.
This takes an unsigned bolt11 (or bolt12 if EXPERIMENTAL_FEATURES) string
and signs it and puts it in the database.

The invoice command could now be moved out to a plugin, in fact.

Changelog-Added: JSON-RPC: `createinvoice` new low-level invoice creation API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 32d66b2740 bolt11: expose routine to parse bolt11 string except signature.
This means we can hand around unsigned bolt11s.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell d66dbd473a lightningd: add low-level offer interfaces.
The real work is done in a plugin, but provide enough API that we can
manipulate the db.
2020-12-15 12:05:02 +01:00
Rusty Russell 3f4683e3f8 sendpay: optional argument to link local offer.
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer.  We mark it used when it's paid, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 963f6b5d67 invoice: add an optional local_offer_id.
This allows us to mark an offer used when an invoice derived from it
is paid, and importantly, avoid any other invoices for the offer being
paid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 381b0f456c db: offer table.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell 0f2c9cf5d5 common/iso4217: make find_iso4217 a little more usable.
We often have the currency as a tal string.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell d971e3de98 Plugin: support extra args to "start".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `start` command can now take plugin-specific parameters.
2020-12-15 09:28:56 +10:30
Rusty Russell 8a9976c4c1 plugins: support concatenation of multiple args.
"multi" means that specifying a parameter twice will append, not override.
Multi args are always given as a JSON array, even if only one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: new "multi" field allows an option to be specified multiple times.
2020-12-15 09:28:56 +10:30