Commit Graph

6859 Commits

Author SHA1 Message Date
trueptolemy f715e3ad84 pay: A cleanup for the comment about wait_payment()
json_waitsendpay_on_resolve() has been replaced by wait_payment(), so correct it here.
2019-09-11 00:57:39 +00:00
trueptolemy 619362ae19 pytest: Add a test for 'sendpay_success' and 'sendpay_failure' 2019-09-11 00:57:39 +00:00
trueptolemy 5fa3a50896 pytest: Add a simple plugin to test 'sendpay_success' and 'sendpay_failure' 2019-09-11 00:57:39 +00:00
trueptolemy 086f096b86 Pay: Notify 'sendpay_success' and 'sendpay_failure' when sendpay succeeds and fails 2019-09-11 00:57:39 +00:00
trueptolemy 82e8db4ba4 plugin: Another new notification type, 'sendpay_failure'
(The json when sendpay successes is too different when sendpay fails, so
divide the sendpay result into two notifications: `sendpay_success` and
`sendpay_failure`)

`sendpay_failure`

A notification for topic `sendpay_failure` is sent every time a sendpay
success(with `failed` status). The json is same as the return value of
command `sendpay`/`waitsendpay` when this cammand fails.

```json
{
  "sendpay_failure": {
  "code": 204,
  "message": "failed: WIRE_UNKNOWN_NEXT_PEER (reply from remote)",
  "data": {
    "id": 2,
    "payment_hash": "9036e3bdbd2515f1e653cb9f22f8e4c49b73aa2c36e937c926f43e33b8db8851",
    "destination": "035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d",
    "msatoshi": 100000000,
    "amount_msat": "100000000msat",
    "msatoshi_sent": 100001001,
    "amount_sent_msat": "100001001msat",
    "created_at": 1561395134,
    "status": "failed",
    "erring_index": 1,
    "failcode": 16394,
    "failcodename": "WIRE_UNKNOWN_NEXT_PEER",
    "erring_node": "022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59",
    "erring_channel": "103x2x1",
    "erring_direction": 0
    }
  }
}
```
`sendpay` doesn't wait for the result of sendpay and `waitsendpay`
returns the result of sendpay in specified time or timeout, but
`sendpay_failure` will always return the result anytime when sendpay
fails if is was subscribed.
2019-09-11 00:57:39 +00:00
trueptolemy 9d8f46149a API: Add payment fields(if not NULL) into return value when sendpay fails
pPayment field includes the basic information of the payment, so the return valves of 'sendpay_success()' and 'sendpay_fail()' should include this field.
Note "immediate_routing_failure" is before payment creation, and for this case, return won't include payment fields.
2019-09-11 00:57:39 +00:00
trueptolemy 507f8d46df pay: Warp the json process of payment fail field
We will also call this warped function in the json process of the 'sendpay_failure' notification.
2019-09-11 00:57:39 +00:00
trueptolemy 07f85cbf72 plugin: A new notification type, 'sendpay_success'
`sendpay_success`

A notification for topic `sendpay_success` is sent every time a sendpay
success(with `complete` status). The json is same as the return value of
command `sendpay`/`waitsendpay` when these cammand succeeds.

```json
{
	"sendpay_success": {
  "id": 1,
  "payment_hash": "5c85bf402b87d4860f4a728e2e58a2418bda92cd7aea0ce494f11670cfbfb206",
  "destination": "035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d",
  "msatoshi": 100000000,
  "amount_msat": "100000000msat",
  "msatoshi_sent": 100001001,
  "amount_sent_msat": "100001001msat",
  "created_at": 1561390572,
  "status": "complete",
  "payment_preimage": "9540d98095fd7f37687ebb7759e733934234d4f934e34433d4998a37de3733ee"
  }
}
```
`sendpay` doesn't wait for the result of sendpay and `waitsendpay`
returns the result of sendpay in specified time or timeout, but
`sendpay_success` will always return the result anytime when sendpay
successes if is was subscribed.
2019-09-11 00:57:39 +00:00
Rusty Russell 226e2aee48 option_static_remotekey: update to latest draft.
531c8d7d9b

In this one, we always send my_current_per_commitment_point, though it's
ignored.  And we have our official feature numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell e467e6a5c6 Makefile: fix race if we run `make update-mocks` in a clean tree.
/bin/sh: 1: ccan/ccan/cdump/tools/cdump-enumstr: Text file busy
    make[1]: *** [common/Makefile:81: common/gen_htlc_state_names.h] Error 2
    make[1]: *** Waiting for unfinished jobs....

The fix is to make sure all generated headers are made first, and
thus cdump-enumstr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 0861279b65 channeld: don't exchange my_current_per_commitment_point if option_static_remotekey
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell ee3480e56b derive_keyset: don't rotate key for remote iff option_static_remotekey.
The largest change is inside hsmd: it hands a null per-commitment key
to the wallet to tell it to spend the to_remote output.

It can also now resolve unknown commitments, even if it doesn't have a
possible_remote_per_commitment_point from the peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 160f27061e common/utxo: make commitment_point optional in close_info.
We don't rotate key for option_static_remotekey, so we don't need
this point for such channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 87f0ee6351 channeld: set option_static_remotekey when negotiated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 11ee089d4b db: store option_static_remotekey for each channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 5203847025 common/features: if EXPERIMENTAL_FEATURES, advertise `option_static_remotekey`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell ded8eb7b31 wire/extracted_peer_experimental_csv: option_static_remotekey
Aka. BOLTVERSION=930a9b44076a8f25a8626b31b3d5a55c0888308c from
https://github.com/lightningnetwork/lightning-rfc/pull/642

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 6a293fd135 tools/generate-wire.py: accept multiple comma-separated options.
Somehow this change got lost, but it's needed for option_static_remotekey,
to quote gen_peer_wire_csv:

    msgtype,channel_reestablish,136
    msgdata,channel_reestablish,channel_id,channel_id,
    msgdata,channel_reestablish,next_commitment_number,u64,
    msgdata,channel_reestablish,next_revocation_number,u64,
    msgdata,channel_reestablish,your_last_per_commitment_secret,byte,32,option_data_loss_protect,option_static_remotekey
    msgdata,channel_reestablish,my_current_per_commitment_point,point,,option_data_loss_protect

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 51d7a1404f tools/check-bolt: don't get confused by 'BOLT #1' in middle of a comment.
Insist it be prefixed with '* '.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
darosior e52af0d1ea doc: Document the new dependency 2019-09-10 02:17:27 +00:00
darosior 0b0ad4c22d transition from status_trace() to status_debug 2019-09-10 02:02:51 +00:00
darosior 0504a51211 channeld/full_channel: use louder warning when peer is about to be failed 2019-09-10 02:02:51 +00:00
darosior 6193d22594 onchaind: Use louder warning where needed 2019-09-10 02:02:51 +00:00
darosior a7cbe93fb8 closingd: retransmit 'funding_locked' if we reconnect without any update
As per BOLT02 #message-retransmission :
if `next_commitment_number` is 1 in both the `channel_reestablish` it sent and received:
    - MUST retransmit `funding_locked`
2019-09-10 02:02:51 +00:00
darosior 57a8951b7a channeld: check sha256_of_onion in update_fail_malformed_htlc
This just logs for the moment
2019-09-10 02:02:51 +00:00
darosior ea6c95b2b3 gossipd: don't ignore wrong chain in 'query_channel_range'
Give a NULL reply with the 'complete' flag to 0 instead
2019-09-10 02:02:51 +00:00
darosior 870d75613f connectd: expand 'seedname()' to allow multiple DNS seeds 2019-09-10 02:02:51 +00:00
darosior b2bb97adde connectd: get multiple addresses from hostname 2019-09-10 02:02:51 +00:00
darosior 9be28fe40f daemons tour: minor typos correction 2019-09-10 02:02:51 +00:00
trueptolemy ffb2ee9b21 bitcoind: Remove unused type `bitcoind_mode` 2019-09-09 04:26:03 +00:00
trueptolemy 7ffa4ad0bc plugin: Register `forward_event` notification 2019-09-08 16:02:00 +02:00
trueptolemy a3eb2942de plugin: Register `channel_opened` notification 2019-09-08 16:02:00 +02:00
trueptolemy 2907e430d5 plugin: Register `invoice_payment` notification 2019-09-08 16:02:00 +02:00
trueptolemy 55fe2b03a2 plugin: Register `warning` notification 2019-09-08 16:02:00 +02:00
trueptolemy 74d8969967 plugin: Register `disconnect` notification 2019-09-08 16:02:00 +02:00
trueptolemy c289fa8f19 plugin: Register `connect` notification 2019-09-08 16:02:00 +02:00
trueptolemy 25d79c5c3f notification: Add registeration interface 2019-09-08 16:02:00 +02:00
darosior d0fe0d658f doc: remove duplicate lines from index.rst 2019-09-08 15:59:04 +02:00
Rusty Russell b65cbc7dc3 plugins: fix false-positive memleak.
This moves field initialization into plugins_new(), and
adds a memleak helper to search the request map:

=================================== ERRORS ====================================
___________________ ERROR at teardown of test_plugin_command ___________________
[gw0] linux -- Python 3.7.1 /opt/python/3.7.1/bin/python3.7
>       lambda: ihook(item=item, **kwds),
        when=when,
    )
../../../.local/lib/python3.7/site-packages/flaky/flaky_pytest_plugin.py:306:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/fixtures.py:112: in node_factory
    ok = nf.killall([not n.may_fail for n in nf.nodes])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <utils.NodeFactory object at 0x7f873b245278>, expected_successes = [True]
    def killall(self, expected_successes):
        """Returns true if every node we expected to succeed actually succeeded""
        unexpected_fail = False
        for i in range(len(self.nodes)):
            leaks = None
            # leak detection upsets VALGRIND by reading uninitialized mem.
            # If it's dead, we'll catch it below.
            if not VALGRIND:
                try:
                    # This also puts leaks in log.
                    leaks = self.nodes[i].rpc.dev_memleak()['leaks']
                except Exception:
                    pass

            try:
                self.nodes[i].stop()
            except Exception:
                if expected_successes[i]:
                    unexpected_fail = True

            if leaks is not None and len(leaks) != 0:
                raise Exception("Node {} has memory leaks: {}".format(
                    self.nodes[i].daemon.lightning_dir,
>                   json.dumps(leaks, sort_keys=True, indent=4)
                ))
E               Exception: Node /tmp/ltests-qm87my20/test_plugin_command_1/lightnng-1/ has memory leaks: [
E                   {
E                       "backtrace": [
E                           "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E                           "lightningd/jsonrpc.c:1112 (jsonrpc_request_start_)",
E                           "lightningd/plugin.c:1041 (plugin_config)",
E                           "lightningd/plugin.c:1072 (plugins_config)",
E                           "lightningd/plugin.c:846 (plugin_manifest_cb)",
E                           "lightningd/plugin.c:252 (plugin_response_handle)",
E                           "lightningd/plugin.c:342 (plugin_read_json_one)",
E                           "lightningd/plugin.c:367 (plugin_read_json)",
E                           "ccan/ccan/io/io.c:59 (next_plan)",
E                           "ccan/ccan/io/io.c:407 (do_plan)",
E                           "ccan/ccan/io/io.c:417 (io_ready)",
E                           "ccan/ccan/io/poll.c:445 (io_loop)",
E                           "lightningd/io_loop_with_timers.c:24 (io_loop_with_tiers)",
E                           "lightningd/lightningd.c:840 (main)"
E                       ],
E                       "label": "lightningd/jsonrpc.c:1112:struct jsonrpc_reques",
E                       "parents": [
E                           "lightningd/plugin.c:66:struct plugin",
E                           "lightningd/lightningd.c:103:struct lightningd"
E                       ],
E                       "value": "0x55d6385e4088"
E                   },
E                   {
E                       "backtrace": [
E                           "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E                           "lightningd/jsonrpc.c:1112 (jsonrpc_request_start_)",
E                           "lightningd/plugin.c:1041 (plugin_config)",
E                           "lightningd/plugin.c:1072 (plugins_config)",
E                           "lightningd/plugin.c:846 (plugin_manifest_cb)",
E                           "lightningd/plugin.c:252 (plugin_response_handle)",
E                           "lightningd/plugin.c:342 (plugin_read_json_one)",
E                           "lightningd/plugin.c:367 (plugin_read_json)",
E                           "ccan/ccan/io/io.c:59 (next_plan)",
E                           "ccan/ccan/io/io.c:407 (do_plan)",
E                           "ccan/ccan/io/io.c:417 (io_ready)",
E                           "ccan/ccan/io/poll.c:445 (io_loop)",
E                           "lightningd/io_loop_with_timers.c:24 (io_loop_with_tiers)",
E                           "lightningd/lightningd.c:840 (main)"
E                       ],
E                       "label": "lightningd/jsonrpc.c:1112:struct jsonrpc_reques",
E                       "parents": [
E                           "lightningd/plugin.c:66:struct plugin",
E                           "lightningd/lightningd.c:103:struct lightningd"
E                       ],
E                       "value": "0x55d6386529d8"
E                   }
E               ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-07 16:54:03 +02:00
lisa neigut 6e991eaebb docs: add install line 2019-09-07 12:21:22 +02:00
Rusty Russell ddab2cb287 pytest: remove flake8 warnings for tests/fixtures.py
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-07 09:51:03 +00:00
Christian Decker cb96be9a6e pytest: Allow running tests in a shared memory directory
It seems we spend a lot of time waiting for `bitcoind` and `lightningd` to
talk to disks. This adds the `TEST_DIR` environment variable, allowing for
example to use `/dev/shm`, or a faster disk than the disk `/tmp` is on, as the
root directory for all test-related files.

Testing this on one of our builder machines cut the time to run the entire
suite under valgrind roughly in half (180-200 seconds vs 440-490 seconds).
2019-09-07 09:51:03 +00:00
Christian Decker b90b4b4bb1 pytest: Consolidate node teardown checks a bit
These are following the same pattern over and over again, so I just added a
tiny wrapper so we reduce the amount of clutter.
2019-09-07 09:51:03 +00:00
Christian Decker ce2bdeec70 pytest: Add a class that collects teardown checks
My machine would accumulate a number of zombie lightningd and bitcoind
processes over time while testing. Investigating this showed that if a fixture
raised an exception during fixture teardown then other fixtures that have not
been torn down would linger around. The issue is that pytest treats exceptions
in fixtures as non-recoverable and therefore will not catch them and call the
remaining ones.

This commit adds a new fixture, that is there just to collect eventual errors
from other fixtures and ensure that anything that needs to clean up something,
e.g., processes started by the fixture, are cleaned up before we raise an
eventual exception. This is achieved by making any fixture that needs cleaning
up dependent on the teardown_checks fixture, which also serves as central
point to collect errors and printer of eventual errors.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-07 09:51:03 +00:00
Rusty Russell 1c0d435f5e pytest: remove flaky part of test_gossip's test_gossip_no_empty_announcements
This "wait_for" failed on Travis, but it's unnecessary anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell 7514a87b5e common/gossip_rcvd_filter: avoid DoS.
If they don't send us a gossip timestamp filter, we won't be sending
them any gossip, thus won't be aging the gossip_rcvd_filter.  So
restrict it to 10,000 elements just to be sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell 5292f11818 pytest: test (fail) that we don't repeat gossip back to the node we got it from
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell 1e7b1276a9 common/gossip_rcvd_filter: cache for received gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell 2652cc9704 common/memleak: simplify notleak() handling.
Use the same "child of tal object" trick to mark things "notleak".

That simplifies things and means we don't have to track them being
reallocated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00