Commit Graph

4002 Commits

Author SHA1 Message Date
ZmnSCPxj b79bb1dbe6 gitignore: Ignore `run-features` test. 2018-05-16 04:59:29 +00:00
Rusty Russell 7bfb9978de wire: remove --check-alignment arg from spec parsing tool.
It's no longer functional, and is being removed upstream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 23:39:09 +00:00
practicalswift ba20ace064 Do not use potentially uninitialized value needed_dns 2018-05-15 15:05:16 +02:00
Rusty Russell 4359f46ffd features: fix features_supported() function.
1. We need to test all bits, not all bytes.
2. Both local and global features need to be supported.
3. Untested code is broken code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Rusty Russell 149e967ba3 features: fix endian.
Closes: https://github.com/lightningnetwork/lightning-rfc/issues/424

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Saibato 8659745ca6 update TOR.md to latest changes
Signed-off-by: Saibato <saibato.naga@protonmail.com>
2018-05-15 07:16:43 +00:00
nicolas.dorier 59154ebd34 Fix build on linux-musl
[ Done by updating CCAN with the ccan/endian fix --RR ]
2018-05-15 05:54:15 +00:00
practicalswift 3abd9ad4f6 wireaddr: Avoid memory corruption in fmt_wireaddr_without_port() 2018-05-15 05:30:43 +00:00
practicalswift fab3b214b4 Avoid static analyzer warning about integer wraparound 2018-05-15 05:26:29 +00:00
Rusty Russell 1125682ceb wireaddr: new type, ADDR_INTERNAL_FORPROXY, use it if we can't/wont resolve.
Tor wasn't actually working for me to connect to anything, but it worked
for 'ssh -D' testing.

Note that the resulting 'netaddr' is a bit weird, but I guess it's honest.

    $ ./cli/lightning-cli connect 021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b
    {
      "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b"
    }
    $ ./cli/lightning-cli listpeers
    {
      "peers": [
        {
          "state": "GOSSIPING", 
          "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b", 
          "netaddr": [
            "ln1qg0je0lugpzu5ttsv78vlrkhteyg9yy8fjw68qr57mfhsfyrxurzkq522ah.lseed.bitcoinstats.com:9735"
          ], 
          "connected": true, 
          "owner": "lightning_gossipd"
        }
      ]
    }

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 2a0acd3492 tor: log proxy communications using status_io.
Good for debugging (you have to send SIGUSR1 to lightning_gossipd to turn
it on though, and --log-level=io on the lightningd cmdline to have it
output IO messages by default).

I also noticed that io_tor_connect_after_req_host() does a useless
test on reach->buffer[0] after it's *written*: remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 570283bc76 gossipd: don't use fake addrhint for non-addrhint resolutions.
Use a wireaddr_internal directly (which is what we want).

Also, don't hardcode 9735, use DEFAULT_PORT internally in
seed_resolve_addr().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell de063edb54 gossip: extract function to derive seedname.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 0d23f4fb4a gossipd: hand io_tor_connect the host as a string.
Previously it converted the wireaddr to a string internally: to support
unresolved names we need that done externally.

We actually tell the SOCKS5 proxy to do a domain lookup already, even
though we give use IP/IPv6 address, so this change is sufficient to
support connect-by-name.

Note replacement of assert() with an explicit case statement, which
has the benefit that the compiler complains when we add new
ADDR_INTERNAL types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell a1dc4eef56 wireaddr: tell caller that we failed due to wanting DNS lookup, don't try.
This is useful for the next patch, where we want to hand the unresolved
name through to the proxy.

This also addresses @Saibato's worry that we still called getaddrinfo()
(with the AI_NUMERICHOST option) even if we didn't want a lookup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 5345e43354 gossipd: rename use_tor to use_proxy,
Not all of them, but it's really about using the SOCKS proxy rather than
really using Tor at this level.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell bcb047a729 gossipd: fix uninitialized var.
We assert() that it's set by one of the branches (it should be!) but
if we don't hit one it's uninitialized, not NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 011d75c618 lightningd: don't allow --announce-addr with a wildcard address.
It doesn't make sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 1ab9a5def6 wireaddr: fix parsing of empty address with no port.
Port should become the default in not specified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell cca791d1cb routing: clean up channel public/active states.
1. If we have a channel_announcement, the channel is public, otherwise
   it's not.  Not all channels are public, as they can be local: those
   have a NULL channel_announcement.

2. If we don't have a channel_update, we know nothing about that half
   of the channel, and no other fields are valid.

3. We can tell if a half channel is disabled by the flags field directly.

Note that we never send halfchannels without an update over
gossip_getchannels_reply so that marshalling/unmarshalling can be
vastly simplified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell 9d1e496b11 gossipd: use a real update in local_add_channel.
We generate one now, so let's use it.  That lets us simplify the
code, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell c71e16f784 broadcast: invert ownership of messages.
Make the update/announce messages own the element in the broadcast map
not the other way around.

Then we keep a pointer to the message, and when we free it
(eg. channel closed, update replaces it), it gets freed from the
broadcast map automatically.

The result is much nicer!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell 8940528bdb gossipd: don't include private announcements into broadcast map.
Basically, if we don't have an announcement for the channel, stash it,
and once we get an announcement, replay if necessary.

Fixes: #1485
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell cdded4f53d tests/utils.py: make default timeout configurable via $TIMEOUT
This is useful when you're expecting a test to fail, you've just added
diagnostics.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell d1b28f832d gossipd: when reconnecting, make sure we free old connection.
Looks like old connection got a callback, and we blew up since
the old peer was freed:

2018-05-06T10:57:11.865Z lightning_gossipd(14387): ...will try again in 300 seconds
2018-05-06T10:57:16.397Z lightning_gossipd(14387): peer_out WIRE_INIT
2018-05-06T10:57:16.405Z lightning_gossipd(14387): peer_in WIRE_INIT
2018-05-06T10:57:16.406Z lightning_gossipd(14387): peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4: reconnect for local peer
2018-05-06T10:57:16.406Z lightning_gossipd(14387): peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4 now remote
2018-05-06T10:57:16.406Z lightning_gossipd(14387): UPDATE WIRE_GOSSIP_PEER_CONNECTED
2018-05-06T10:57:16.406Z lightning_gossipd(14387): UPDATE WIRE_GOSSIP_PEER_CONNECTED
2018-05-06T10:57:16.406Z lightning_gossipd(14387): Handing back peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4 to master
2018-05-06T10:57:16.420Z lightning_gossipd(14387): hand_back_peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4: now local again
2018-05-06T10:57:16.420Z lightning_gossipd(14387): FATAL SIGNAL 11
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: common/daemon.c:42 (crashdump) 0x416991
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x7f70cf57a4af
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: common/msg_queue.c:38 (msg_dequeue) 0x418232
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: gossipd/gossip.c:816 (peer_pkt_out) 0x404ac4
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x4316db
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/io.c:427 (io_do_always) 0x4322ce
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/poll.c:228 (handle_always) 0x433abd
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/poll.c:249 (io_loop) 0x433b48
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: gossipd/gossip.c:2407 (main) 0x4093aa
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x7f70cf56582f
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x402ad8
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0xffffffffffffffff
2018-05-06T10:57:16.421Z lightning_gossipd(14387): STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL

Fixes: #1469
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:11:00 +02:00
Rusty Russell 2bc9d7594c doc/TOR.md: update for new options.
Currently the Tor port for autotor is always 9735; the --port option
is deprecated, though we could append an optional '[@portnum]' to
'autotor' if people want it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 89c76a5a78 Move always-use-proxy auto-override to master daemon.
This means it will effect connect commands too (though it's too
late to stop DNS lookups caused by commandline options).

We also warn that this is one case where we allow forcing through Tor
without a proxy set: it just means all connections will fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 1106c40217 tor: add new 'autotor:' address option.
This takes the Tor service address in the same option, rather than using
a separate one.  Gossipd now digests this like any other type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell a8c0bca6a8 gossipd: take over negotiation of autogenerated Tor addresses.
For the moment, this is a straight handing of current parameters through
from master to the gossip daemon.  Next we'll change that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e93682e3bf status: make status_io a more generic mechanism.
Currently it's always for messages to peer: make that status_peer_io and
add a new status_io for other IO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 5a0bc83b20 Tor: don't do seed queries if we're supposed to always use proxy.
Risks leakage.  We could do lookup via the proxy, but that's a TODO.

There's only one occurance of getaddrinfo (and no gethostbyname), so
we add a flag to the callers.

Note: the use of --always-use-proxy suppresses *all* DNS lookups, even
those from connect commands and the command line.

FIXME: An implicit setting of use_proxy_always is done in gossipd if it
determines that we are announcing nothing but Tor addresses, but that
does *not* suppress 'connect'.

This is fixed in a later patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 9d8e3cf3da gossip: handle Tor proxy better.
1. Only force proxy use if we don't announce any non-TOR address.
   There's no option to turn it off, so this makes more sense.
2. Don't assume we want an IPv4 socket to reach proxy, use the family
   from the struct addrinfo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell c3ccc14f19 Tor: remove --tor prefix from SOCKS5 options.
It's usually for Tor, but we can use a socks5 proxy without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 003cd29733 tor: clean up io_tor_connect.
Instead of storing a wireaddr and converting to an addrinfo every
time, just convert once (which also avoids the memory leak in the
current code).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d87a6c3a48 wireaddr: more helpers, to convert to addrinfo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 2d840706df tor: don't immediately fail if we can't open cookie file.
We might still be offered password authentication, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d5e4d52563 common/base32: make this a simple tal-wrapper around ccan/base32.
And use it in wireaddr.

We fix up the double '.onion' in the test case, which seems like an error?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 53c6ceeab9 ccan: import base32 module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e91975c22d wireaddr: use fmt_wireaddr_without_port in fmt_wireaddr.
Avoids duplicate code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 7900318216 wireaddr: fix tor address lengths.
This variable does NOT include the 2 bytes for the port.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 7d95dbbd0a wireaddr: remove FQDN_ADDRLEN in favor of LARGEST_ADDRLEN.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell c1e0a4d572 gossip/tor: rearrange functions to avoid predeclarations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 8b0215549c options: check that combinations of TOR options are valid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e229f113b9 gossipd: don't try to reach tor if we don't have a proxy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 11db7ca9e6 options: use NULL for unset Tor settings.
Rename tor_proxyaddrs and tor_serviceaddrs to tor_proxyaddr and tor_serviceaddr:
the 's' at the end suggests that there can be more than one.

Make them NULL or non-NULL, rather than using all-zero if unset.

Hand them the same way to gossipd; it's a bit of a hack since we don't
have optional fields, so we use a counter which is always 0 or 1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell ef09961713 lightningd: rewrite tor service interaction to be synchronous, robust.
There's no reason to do this async, and far easier to follow using normal
read/write.

The previous parsing was deeply questionable, using substring searches
only, and relying on the fact that a single non-blocking read would get
the entire response.  This is changed to do (somewhat) proper parsing
using ccan/rbuf.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 57115f4914 ccan: update and import rbuf module.
This is a simple helper for dealing with buffered I/O.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 85eff42d9a common/tor: move into lightningd.
This is simply the code to set up the automatic hidden service, so move
it into lightningd.

I removed the undefined parse_tor_wireaddr, and added a parameter name
to the create_tor_hidden_service_conn() declaration for update-mocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d9f13230cf gossip/tor.c: new file for socks proxy code.
All gossipd needs from common/tor is do_we_use_tor_addr(), so move
that and the rest of the tor-specific handshake code into gossip/tor.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 6d69e7b066 netaddress: fix up IsTor()
We don't actually use it, but let's fix it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00