Commit Graph

3771 Commits

Author SHA1 Message Date
Rusty Russell 9d8e3cf3da gossip: handle Tor proxy better.
1. Only force proxy use if we don't announce any non-TOR address.
   There's no option to turn it off, so this makes more sense.
2. Don't assume we want an IPv4 socket to reach proxy, use the family
   from the struct addrinfo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell c3ccc14f19 Tor: remove --tor prefix from SOCKS5 options.
It's usually for Tor, but we can use a socks5 proxy without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 003cd29733 tor: clean up io_tor_connect.
Instead of storing a wireaddr and converting to an addrinfo every
time, just convert once (which also avoids the memory leak in the
current code).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d87a6c3a48 wireaddr: more helpers, to convert to addrinfo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 2d840706df tor: don't immediately fail if we can't open cookie file.
We might still be offered password authentication, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d5e4d52563 common/base32: make this a simple tal-wrapper around ccan/base32.
And use it in wireaddr.

We fix up the double '.onion' in the test case, which seems like an error?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 53c6ceeab9 ccan: import base32 module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e91975c22d wireaddr: use fmt_wireaddr_without_port in fmt_wireaddr.
Avoids duplicate code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 7900318216 wireaddr: fix tor address lengths.
This variable does NOT include the 2 bytes for the port.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 7d95dbbd0a wireaddr: remove FQDN_ADDRLEN in favor of LARGEST_ADDRLEN.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell c1e0a4d572 gossip/tor: rearrange functions to avoid predeclarations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 8b0215549c options: check that combinations of TOR options are valid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e229f113b9 gossipd: don't try to reach tor if we don't have a proxy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 11db7ca9e6 options: use NULL for unset Tor settings.
Rename tor_proxyaddrs and tor_serviceaddrs to tor_proxyaddr and tor_serviceaddr:
the 's' at the end suggests that there can be more than one.

Make them NULL or non-NULL, rather than using all-zero if unset.

Hand them the same way to gossipd; it's a bit of a hack since we don't
have optional fields, so we use a counter which is always 0 or 1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell ef09961713 lightningd: rewrite tor service interaction to be synchronous, robust.
There's no reason to do this async, and far easier to follow using normal
read/write.

The previous parsing was deeply questionable, using substring searches
only, and relying on the fact that a single non-blocking read would get
the entire response.  This is changed to do (somewhat) proper parsing
using ccan/rbuf.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 57115f4914 ccan: update and import rbuf module.
This is a simple helper for dealing with buffered I/O.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 85eff42d9a common/tor: move into lightningd.
This is simply the code to set up the automatic hidden service, so move
it into lightningd.

I removed the undefined parse_tor_wireaddr, and added a parameter name
to the create_tor_hidden_service_conn() declaration for update-mocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d9f13230cf gossip/tor.c: new file for socks proxy code.
All gossipd needs from common/tor is do_we_use_tor_addr(), so move
that and the rest of the tor-specific handshake code into gossip/tor.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 6d69e7b066 netaddress: fix up IsTor()
We don't actually use it, but let's fix it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato 877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Isidoro Ghezzi 855d0b9401 "sizeof(sun->sun_path)" is not always the same as "sizeof(addr->u.sockname)"
on all platforms;

because of that BUILD_ASSERT was failing on macOS.

(on macOS "sizeof(sun->sun_path) == 104" and
"sizeof(addr->u.sockname) == 108")

[ Linux man page says it can be as small as 92, so let's use the real value.
  I also cleaned up the incorrect comment order on that struct! --RR ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 09:10:45 +00:00
Rusty Russell c33bbb2639 test_lightningd.py: wait longer test_permfail for l2 to notice new blocks.
Reproduced this failure locally: l2 hadn't seen the block yet.  Timeout
was too aggressive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell 2caeab506b tests/test_lightningd.py: make test_blockchaintrack safe to re-run.
It gets confused if re-run due to flaky, since:
1. Using the same HSM, it generates the same spend and sees a previous one.
2. The block height numbers are off.

Fixes: #1479
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell ecebbe700b tests/test_lightningd.py: remove unneeded btc variable.
We have a 'bitcoind' global: getting it from inside one of the daemons
was a mistake I've copied widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
ZmnSCPxj 757c059cd4 test_lightningd: Add closingd torture test. 2018-05-09 04:43:21 +00:00
ZmnSCPxj 926b41b7da closingd: Ensure proper closing of TCP socket.
Fixes: #1457
2018-05-09 04:43:21 +00:00
Felix 80b298a15f help - add preimage to param list 2018-05-09 03:47:53 +00:00
Christian Decker 783327622f pytest: Mark test_blockchaintrack as flaky
The flakyness is probably due to timing issues when reorging.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker fac1d24ce3 pytest: Use the port from the lightning node
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker 81dc82de14 gossip: Clean up stale `store` argument to `handle_gossip_msg`
This is a leftover from before splitting the `gossip_store` injection path from
the handling of gossip messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker 5f7b0fdf92 pytest: Marking `test_permfail` as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Mark Beckwith 9153688b77 Added fundchannel man page.
This describes 'fundchannel' as it works today.

It definitely could use better error codes instead of -1 for everything.
2018-05-08 07:11:35 +00:00
ZmnSCPxj c79b3de4d6 payalgo: Report reason to delay before clearing try memory.
We allocate the reason to delay, if any, from the `pay->try_parent`.
So we should not clear the `pay->try_parent` until after we print
the reason.
2018-05-08 07:08:37 +00:00
Christian Decker 104645ba3a pytest: Fix flaky `test_closing_id` test
It simply wasn't waiting for us to register the peer before attempting to open a
connection. Moved into a separate test to be able rerun multiple times.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 05:43:35 +00:00
Rusty Russell 79902b48c7 test_lightningd.py: remove gratuitous 'addr' option from test_connect_by_gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 73cd009a4c gossipd/lightningd: use wireaddr_internal.
This replacement is a little menial, but it explicitly catches all
the places where we allow a local socket.  The actual implementation of
opening a AF_UNIX socket is almost hidden in the patch.

The detection of "valid address" is now more complex:

	p->addr.itype != ADDR_INTERNAL_WIREADDR || p->addr.u.wireaddr.type != ADDR_TYPE_PADDING

But most places we do this, we should audit: I'm pretty sure we can't
get an invalid address any more from gossipd (they may be in db, but
we should fix that too).

Closes: #1323
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 6b2282fc1d wireaddr_internal: new type for where we can also use a local socket.
This was something @icota implemented, but it fits logically into this
cleanup series.  We create a new type which is the internal generalization
of a wireaddr (which is defined by the spec), and add a case here for
a socket name.

Based-on-the-true-story-by: @icota
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell e6c678e5df gossipd: take over address determination, from master.
It does all the other address handling, do this too.  It also proves useful
as we clean up wildcard address handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 356e5dcea8 wireaddr: helpers to convert to/from IPv4/v6 addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 9c0de76019 lightningd: still bind to local ports even if address not public.
Now we only bind to addresses in our wireaddrs array, we would not
autobind to local sockets if they couldn't reach google's nameserver.

That's clearly wrong: we should only not bind if there's a protocol
issue (eg. no IPv6 support).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 00537fde43 lightningd: deprecate --ipaddr in favor of --addr.
We're going to add sockets, and later onion addresses, so the current name
is bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell ed466a8523 lightningd: make explicit listen and reconnect flags.
We set no_reconnect with --offline, but that doesn't work if !DEVELOPER.
Make the flag positive, and non-DEVELOPER mode for gossipd.

We also don't override portnum with --offline, but have an explicit
'listen' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 6ba9a7cb57 tests/test_lightningd.py: move port into node.
We're about to change the JSONRPC, so let's put an explicit 'port' into
our node class.

We initialize it at startup time: in future I hope to use ephemeral ports
to make our tests more easily parallelizable.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 7d27501c94 tests/utils.py: remove unused connect routine.
It's broken anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 99bba2a23c channeld: don't send ANNOUNCEMENT_SIGNATURES if we've send shutdown.
Our closingd doesn't handle it:

lightningd(2968): 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1:
 Peer permanent failure in CLOSINGD_SIGEXCHANGE: lightning_closingd: sent ERROR Expected closing_signed:
 0103ff54517293892ec3f214f2343c54cbfbf24aa6ffb8d5585d3bc1b543eae0a272000067000001000146390e0c043c777226927eacd2186a03f064e4bdc30f891cb6e4990af49967d34b338755e99d728987e3d49227815e17f3ab40092434a59e33548e870071176d26d19a4e4d8f7715c13ac2d6bf3238608a1ccf9afd91f774d84d170d9edddebf7460c54d49bd6cd81410bc3eeeba2b7278b1b5f7e748d77d793f31086847d582

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 12:48:19 +02:00
Christian Decker 6231e9969f pytest: Give lightningd nodes a numeric ID to prefix logs
This used to be the port, but since we no longer have fixed ports, and we start
them in random order we can't easily distinguish them by the port anymore. Just
use a numeric ID that matches their lightning-dirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00