Commit Graph

7508 Commits

Author SHA1 Message Date
Christian Decker e1b1f47c53 pytest: Avoid starting the btcproxy twice 2019-12-11 16:18:34 +01:00
Christian Decker 15f04d5fff pytest: Give details about which node exited with a return code != 0 2019-12-11 16:18:34 +01:00
Christian Decker 518c43a9c2 pytest: Remember a node's return code so we can check it later 2019-12-11 16:18:34 +01:00
Vasil Dimov cd30f76539 build: fix ./configure options' descriptions
The description of

--enable/disable-experimental-features was a bogus "Developer mode, good
for testing" and

--enable/disable-valgrind was "Valgrind binary to use for tests" which
gave the false impression that it should be set to something like
/usr/local/bin/valgrind whereas it is a boolean option.
2019-12-11 00:50:20 +01:00
darosior fcbd11f0c5 plugins/libplugin: hook support
Changelog-Added: plugins: libplugin now supports writing plugins which register to hooks
2019-12-09 16:18:28 +01:00
darosior 3371f0cf78 plugins/libplugin: notifications support
Changelog-Added: plugins: libplugin now supports writing plugin which registers to notifications
2019-12-09 16:18:28 +01:00
Rusty Russell f7ebbb2ec5 common: make sphinx code ignorant of payload format.
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).

This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
   anything about "style".

The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Rusty Russell bb538a1862 common: don't crash on bad sphinx payload.
It's cleanest to eliminate the SPHINX_INVALID_PAYLOAD altogether.

lightning_channeld: FATAL SIGNAL (version v0.7.3-242-gb1583bb-modded)
0x55a8169eed08 send_backtrace
	common/daemon.c:41
0x55a8169fc3eb status_failed
	common/status.c:206
0x55a8169fc657 status_backtrace_exit
	common/subdaemon.c:25
0x55a8169eedbb crashdump
	common/daemon.c:57
0x7f0eaff8446f ???
	???:0
0x7f0eaff843eb ???
	???:0
0x7f0eaff63898 ???
	???:0
0x55a8169fb29f route_step_decode
	common/sphinx.c:759
0x55a8169fb60a process_onionpacket
	common/sphinx.c:834
0x55a8169d9b34 get_shared_secret
	channeld/channeld.c:605
0x55a8169d9d35 handle_peer_add_htlc
	channeld/channeld.c:649
0x55a8169dd88d peer_in
	channeld/channeld.c:1838
0x55a8169e11a8 main
	channeld/channeld.c:3233
0x7f0eaff651e2 ???

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Rusty Russell f8d8348c9f doc: add 'id' to rpc_command
It's required, so at least hint about its presence!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-08 15:26:28 +01:00
Michael Schmoock bd369edd10 chore: reduce rpc load in is_channel_active helper 2019-12-07 21:27:28 +01:00
Michael Schmoock 44c23ede04 chore: adds announce_channel parameter to pyln fund_channel 2019-12-07 21:27:28 +01:00
Rusty Russell 30c8db148d lightningd: make sure rpc_command replacement is well-formed.
In particular:
1. It must redirect to an existing command.
2. It must contain method, params and id.

And update the docs to show the id, which is vital.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell e572b487d3 pytest: demonstrate failing rpc_command hook rewrite to plugin.
lightningd: lightningd/plugin.c:648: plugin_rpcmethod_dispatch: Assertion `idtok != NULL' failed.
lightningd: FATAL SIGNAL 6 (version v0.7.3-234-gf3aeb51-modded)
0x557652363eb1 send_backtrace
	common/daemon.c:41
0x557652363f5b crashdump
	common/daemon.c:54
0x7f5b065a346f ???
	???:0
0x7f5b065a33eb ???
	???:0
0x7f5b06582898 ???
	???:0
0x7f5b06582768 ???
	???:0
0x7f5b06594005 ???
	???:0
0x557652355901 plugin_rpcmethod_dispatch
	lightningd/plugin.c:648
0x557652331836 command_exec
	lightningd/jsonrpc.c:588
0x557652331bad rpc_command_hook_callback
	lightningd/jsonrpc.c:655
0x557652358745 plugin_hook_callback
	lightningd/plugin_hook.c:90
0x55765235486a plugin_response_handle
	lightningd/plugin.c:258
0x557652354a16 plugin_read_json_one
	lightningd/plugin.c:356
0x557652354b6d plugin_read_json
	lightningd/plugin.c:388
0x5576523b5e6c next_plan
	ccan/ccan/io/io.c:59
0x5576523b6a51 do_plan
	ccan/ccan/io/io.c:407
0x5576523b6a93 io_ready
	ccan/ccan/io/io.c:417
0x5576523b8d2b io_loop
	ccan/ccan/io/poll.c:445
0x55765232dc6f io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x557652334a6e main
	lightningd/lightningd.c:848
0x7f5b065841e2 ???
	???:0
0x5576523193ed ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20191206093336

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell f6ed7f2e89 plugin: handle corner case where rpc_command is to stop the plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
Rusty Russell d8fc5332c3 pytest: test rpc_command hook when it handles command to stop itself.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
lisa neigut a30ee2b7cd fundchannel: attempt to connect before funding channel
Attempts to connect to peer before funding the channel
2019-12-04 15:32:31 -06:00
lisa neigut f6ff5e5b19 connectd: make failure message more descriptive 2019-12-04 15:32:31 -06:00
darosior ff412a6c9a plugins/fundchannel: try to connect to peer before funding
Changelog-Added: RPC: The 'fundchannel' command now tries to connect to the peer before funding the channel, no need to 'connect' before 'fundchannel' anymore !
2019-12-04 15:32:31 -06:00
Rusty Russell 4683d096dd doc: feerate is not obvious, refer to fundchannel documentation:
It has a whole *two paragraphs* on it:

*feerate* is an optional feerate used for the opening transaction and as
initial feerate for commitment and HTLC transactions. It can be one of
the strings *urgent* (aim for next block), *normal* (next 4 blocks or
so) or *slow* (next 100 blocks or so) to use lightningd’s internal
estimates: *normal* is the default.

Otherwise, *feerate* is a number, with an optional suffix: *perkw* means
the number is interpreted as satoshi-per-kilosipa (weight), and *perkb*
means it is interpreted bitcoind-style as satoshi-per-kilobyte. Omitting
the suffix is equivalent to *perkb*.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-04 08:35:50 -06:00
Saibato 2c16b410ab Add basic tests
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
Saibato b18510a7e3 Reflect the changes in documentation
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
Saibato f6006f43a9 Init commit to be able to create a tor static service on the fly.
We  want to have a static Tor service created from a blob bound to
our node on cmdline

Changelog-added: persistent Tor address support
Changelog-added: allow the Tor inbound service port differ from 9735

Signed-off-by: Saibato <saibato.naga@pm.me>

Add base64 encode/decode to common

We need this to encode the blob for the tor service

Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
fiatjaf 99ff86f6fe update docs with `payment_preimage`.
Changelog-Added: JSON-RPC: `listinvoices` now displays the payment preimage if the invoice was paid.
2019-12-03 01:16:07 +00:00
fiatjaf c0368a717a also return preimage on listinvoices. 2019-12-03 01:16:07 +00:00
fiatjaf 770789684c jsonrpc: Add json_add_preimage helper 2019-12-03 01:16:07 +00:00
Christian Decker db92c2ac5e tlv: Remove unused TLV deserialization function 2019-12-03 00:37:15 +00:00
Christian Decker 626675c83c tlv: Migrate tlv serialization to typesafe function 2019-12-03 00:37:15 +00:00
Christian Decker e12b5c3824 tlv: Add a typesafe serialization function for tlv namespaces
This is the counterpart to the typesafe deserialization function implemented
in an earlier commit.
2019-12-03 00:37:15 +00:00
Christian Decker 5a78671d9f wire: Remove unused fromwire_tlvs
We are now using the typesafe variant everywhere.
2019-12-03 00:37:15 +00:00
Christian Decker 838001af4c tlv: Migrate run-tlvstream to typesafe helpers 2019-12-03 00:37:15 +00:00
Christian Decker 69c17d2d31 wire: Let the TLV _is_valid function actually return validity
I got this one wrong myself, since the function name implied a boolean
result. So I changed it to take the optional err_index as argument.
2019-12-03 00:37:15 +00:00
darosior d9aeeb4d14 pytest: skip test_sendonion if not DEVELOPER 2019-12-02 14:41:13 +01:00
darosior 4661dfa590 tests: update requirements
cheroot release(/"changes"?) notes:
    #218 via PR #219: Fix HTTP parser to return 400 on invalid major-only HTTP version in Request-Line.
    #198 via 9f7affe: Fix race condition when toggling stats counting in the middle of request processing.
    Improve post Python 3.9 compatibility checks.
    Fix support of abstract namespace sockets.
    #222 via 621f4ee: Fix socket.SO_PEERCRED constant fallback value under PowerPC
    Revisit PR #85 under PR #221. Now backports.functools_lru_cache is only required on Python 3.2 and earlier.
    CherryPy #1206 via PR #204: Fix race condition in threadpool shrink code.
    PR #224: Refactored “open URL” behavior in webtest to rely on retry_call. Callers can no longer pass raise_subcls or ssl_context positionally, but must pass them as keyword arguments.
    #231 via PR #232: Remove custom setup.cfg parser handling, allowing the project (including sdist) to build/run on setuptools 41.4. Now building cheroot requires setuptools 30.3 or later (for declarative config support) and preferably 34.4 or later (as indicated in pyproject.toml).
    Workers are now request-based, addressing the long-standing issue with keep-alive connections (#91 via PR #199).
    Deprecated use of negative timeouts as alias for infinite timeouts in ThreadPool.stop.
    CherryPy #1662 via PR #74: For OPTION requests, bypass URI as path if it does not appear absolute.
    CherryPy #1818: Restore support for None default argument to WebCase.getPage().

https://github.com/cherrypy/cheroot/blob/master/CHANGES.rst

flaky changes:
    Bugfixes - Reraise KeyboardInterrupt when running tests under pytest.

https://github.com/box/flaky/blob/v3.6.1/HISTORY.rst#361-2019-08-06

python-bitcoinlib:
    New RPC `generatetoaddress(self,numblocks,addr)`.
    Fixed Python 2.7 incompatibility.
    Various OpenSSL fixes, including a memory leak.

https://github.com/petertodd/python-bitcoinlib/blob/python-bitcoinlib-v0.10.2/release-notes.md#v0102

pytest release notes:
    A lot of misc fixes, see https://docs.pytest.org/en/latest/changelog.html.
2019-12-02 14:41:13 +01:00
Rusty Russell 7f36a4e3dd lightningd: fix reference to out-of-scope var.
cppcheck found this:

	[lightningd/options.c:1137] -> [lightningd/options.c:1120] -> [lightningd/options.c:1193]: (error) Using pointer to local variable 'buf' that is out of scope.

Indeed, answer can point into buf, which is no longer in scope at the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-02 09:44:39 +01:00
Christian Decker 0b61781746 json-rpc: Only show the amount_msat field if we know it in payments
If we initiated the payment using an externally generated onion we don't know
what the final hop gets, or even who it is, so we don't display the amount in
these cases. I chose to show `null` instead in order not to break dependees
that rely on the value being there.
2019-12-01 15:40:47 +01:00
Christian Decker 82255e2401 json-rpc: Add helper for an array of secrets
Suggested-by: Rusty Russell <@rustyrussell>
2019-12-01 15:40:47 +01:00
Christian Decker 21b5b59c8c json-rpc: Rename onion payload type to style
Suggested-by: Rusty Russell <@rustyrussell>
2019-12-01 15:40:47 +01:00
Christian Decker 55d8dcc907 param: Encapsulate hops parsing in a param_hops_array helper
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2019-12-01 15:40:47 +01:00
Christian Decker f569b52681 sphinx: Make the sphinx_hop struct public
We're about to create a param helper for sphinx hops and this struct seems
like the correct place to store the result.
2019-12-01 15:40:47 +01:00
Christian Decker 2080416a89 doc: Add manpages for `createonion` and `sendonion`
Changelog-Added: Added `createonion` and `sendonion` JSON-RPC methods allowing the implementation of custom protocol extensions that are not directly implemented in c-lightning itself.
2019-12-01 15:40:47 +01:00
Christian Decker 705e189f0a pay: Allow payments initiated with `sendonion` to be retried 2019-12-01 15:40:47 +01:00
Christian Decker 16b6c31010 json-rpc: Add the error onion if we stored it in the DB
If we can't decode the onion, because the onion got corrupted or we used
`sendonion` without specifying the `shared_secrets` used, the best we can do
is tell the caller instead.
2019-12-01 15:40:47 +01:00
Christian Decker 1cf3c12a81 cleanup: The failchannel is not tal-allocated when first assigned
This makes the copy on write redundant.
2019-12-01 15:40:47 +01:00
Christian Decker 27547ce8d4 pay: Allow `sendonion` callers to provide `shared_secrets`
This means that c-lightning can now internally decrypt an eventual error
message, and not force the caller to implement the decryption. The main
difficulty was that we now have a new state (channels and nodes not specified,
while shared_secrets are specified) which needed to be handled.
2019-12-01 15:40:47 +01:00
Christian Decker 7e41d6c9c4 pay: Make `erring_channel` optional in `struct routing_failure`
Same rationale as the previous commit: we may not have the channels in the
path so we don't try to infer the failing channel from the error.
2019-12-01 15:40:47 +01:00
Christian Decker 88961aa4b0 pay: Make `erring_node` optional in `struct routing_failure`
When using `sendonion` with `shared_secrets` we may be able to decode the
onioned error message but we cannot infer which node reported the failure
since we don't know which nodes where involved.
2019-12-01 15:40:47 +01:00
Christian Decker 43fb2a6ed0 json: Add helper to extract a secret from JSON 2019-12-01 15:40:47 +01:00
Christian Decker b8ce175fd2 pay: Handle payment failures resulting from sendonion correctly
We are breaking with a couple of assumptions, namely that we have the
`path_secrets` to decode the error onion. If this happens we just want it to
error out.
2019-12-01 15:40:47 +01:00
Christian Decker cb912c9ae8 pytest: Add a test for the `sendonion` method 2019-12-01 15:40:47 +01:00
Christian Decker 42277a7ad4 json-rpc: Add the `sendonion` RPC command 2019-12-01 15:40:47 +01:00