Commit Graph

4319 Commits

Author SHA1 Message Date
Rusty Russell 85a062164a test_lightningd: test_closing_torture timeout increase.
I hit this timeout on pytest and valgrind with -n5; use the timeout variable
from utils.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell f92712f22c channeld: always send either a temporary or final update if we can.
If we hit depth 6, we would start exchanging announcement signatures.
However, we should still send a temporary update while waiting for the
reply; make the logic clear in this case that we should always send
one or the other.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 00d4c04208 channeld: fix conditions under which we can send a channel update.
The condition in send_channel_update is wrong: it needs to match the
conditions under which we send announcements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell c546b1bbb6 gossipd: specify origin of updates in errors.
@cdecker points out that in test_forward, where we manually create a route,
we get an error back which contains an update for an unknown channel.

We should still note this, but it's not an error for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 8ee60e2d8e testing: make sure we don't see gossip in bad order.
This is something which generally shouldn't happen, but we didn't
notice it previously.

We ignore this warning in the case where a channel was deleted: this
happens because one side can send an update while the other notices
that the channel is closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 177a1fc88e gossipd: handle local channel creation separately from update.
Note: this will break the gossip_store if they have current channels,
but it will fail to parse and be discarded.

Have local_add_channel do just that: the update is logically separate
and can be sent separately.

This removes the ugly 'bool add_to_store' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell f0231b0e6f channeld: don't send temporary announcements more than once.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell f014cbb78c channeld: clean up announcement state tests.
We always call:

	send_temporary_announcement(peer);
	send_announcement_signatures(peer);

We should handle these in one place, since the conditional at the top
of them actually makes sure only one is effective.  We also make the
caller set the peer->have_sigs[LOCAL] flag, instead of doing it
inside send_announcement_signatures().

We were sending announcements at the wrong time (on restart) somtimes.

We also move announce_channel() into the same logic, so it's always
together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 323472225c channeld: simplify announce/locked-in callback,
Just have a "new depth" callback, and let channeld do the right thing.

This makes the channeld paths a bit more straightforward.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 981ffb83f7 channeld: don't send updates for 0:0:0.
Some paths (eg reconnect) were unconditionally sending a channel_update.
valgrind wasn't catching it because we unmarshal short_channel_ids[LOCAL]
as all-zeroes, so it's technically "initialized".

Create a wrapper to do this, and change the 'bool disabled' flag to be
the explicit disable flag value for clarity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 540c68d7ca gossipd/gossip_constants.h: Single place for BOLT constants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell b965ef7d1d routing: make sure we fail if we can't unmarshal announcements.
This is how we notice if the gossip store is corrupt!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell d52564dc05 check-source: add check for tal_free(tmpctx).
My older rebased patches use this, and I want to catch it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell cae25ca5db devtools/print_wire: add return for numerical fields.
Before:
    $ ./devtools/decodemsg 0102c2bd3f4a94ff390ce764caf51925d0ed38fa95b6539945b42124f5c4e625da63351380c79230a05550d0e5def9c2412f4f164478f9f9491140e505f79c0d716506226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00006700000100015afd0da1000000060000000000000000000000010000000a
    WIRE_CHANNEL_UPDATE:
    signature=3045022100c2bd3f4a94ff390ce764caf51925d0ed38fa95b6539945b42124f5c4e625da630220351380c79230a05550d0e5def9c2412f4f164478f9f9491140e505f79c0d7165
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    short_channel_id=103:1:1
    timestamp=1526533537flags=0cltv_expiry_delta=6htlc_minimum_msat=0fee_base_msat=1fee_proportional_millionths=10

After:

    WIRE_CHANNEL_UPDATE:
    signature=3045022100c2bd3f4a94ff390ce764caf51925d0ed38fa95b6539945b42124f5c4e625da630220351380c79230a05550d0e5def9c2412f4f164478f9f9491140e505f79c0d7165
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    short_channel_id=103:1:1
    timestamp=1526533537
    flags=0
    cltv_expiry_delta=6
    htlc_minimum_msat=0
    fee_base_msat=1
    fee_proportional_millionths=10

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell bd1c16d284 update-ccan: clean up the two unwanted imports.
We don't import these, though they're technically dependencies.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell 6fe2c037dc Add .rej files to gitignore.
They turn up for me when I force-push a guilt patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Christian Decker e604891910 pytest: Do not set test-groups unless explicitly requested
Setting these options could fail the `pytest` target if `pytest-test-groups` is
not installed.

Fixes #1508

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-16 19:42:44 -04:00
ZmnSCPxj b79bb1dbe6 gitignore: Ignore `run-features` test. 2018-05-16 04:59:29 +00:00
Rusty Russell 7bfb9978de wire: remove --check-alignment arg from spec parsing tool.
It's no longer functional, and is being removed upstream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 23:39:09 +00:00
practicalswift ba20ace064 Do not use potentially uninitialized value needed_dns 2018-05-15 15:05:16 +02:00
Rusty Russell 4359f46ffd features: fix features_supported() function.
1. We need to test all bits, not all bytes.
2. Both local and global features need to be supported.
3. Untested code is broken code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Rusty Russell 149e967ba3 features: fix endian.
Closes: https://github.com/lightningnetwork/lightning-rfc/issues/424

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Saibato 8659745ca6 update TOR.md to latest changes
Signed-off-by: Saibato <saibato.naga@protonmail.com>
2018-05-15 07:16:43 +00:00
nicolas.dorier 59154ebd34 Fix build on linux-musl
[ Done by updating CCAN with the ccan/endian fix --RR ]
2018-05-15 05:54:15 +00:00
practicalswift 3abd9ad4f6 wireaddr: Avoid memory corruption in fmt_wireaddr_without_port() 2018-05-15 05:30:43 +00:00
practicalswift fab3b214b4 Avoid static analyzer warning about integer wraparound 2018-05-15 05:26:29 +00:00
Rusty Russell 1125682ceb wireaddr: new type, ADDR_INTERNAL_FORPROXY, use it if we can't/wont resolve.
Tor wasn't actually working for me to connect to anything, but it worked
for 'ssh -D' testing.

Note that the resulting 'netaddr' is a bit weird, but I guess it's honest.

    $ ./cli/lightning-cli connect 021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b
    {
      "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b"
    }
    $ ./cli/lightning-cli listpeers
    {
      "peers": [
        {
          "state": "GOSSIPING", 
          "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b", 
          "netaddr": [
            "ln1qg0je0lugpzu5ttsv78vlrkhteyg9yy8fjw68qr57mfhsfyrxurzkq522ah.lseed.bitcoinstats.com:9735"
          ], 
          "connected": true, 
          "owner": "lightning_gossipd"
        }
      ]
    }

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 2a0acd3492 tor: log proxy communications using status_io.
Good for debugging (you have to send SIGUSR1 to lightning_gossipd to turn
it on though, and --log-level=io on the lightningd cmdline to have it
output IO messages by default).

I also noticed that io_tor_connect_after_req_host() does a useless
test on reach->buffer[0] after it's *written*: remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 570283bc76 gossipd: don't use fake addrhint for non-addrhint resolutions.
Use a wireaddr_internal directly (which is what we want).

Also, don't hardcode 9735, use DEFAULT_PORT internally in
seed_resolve_addr().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell de063edb54 gossip: extract function to derive seedname.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 0d23f4fb4a gossipd: hand io_tor_connect the host as a string.
Previously it converted the wireaddr to a string internally: to support
unresolved names we need that done externally.

We actually tell the SOCKS5 proxy to do a domain lookup already, even
though we give use IP/IPv6 address, so this change is sufficient to
support connect-by-name.

Note replacement of assert() with an explicit case statement, which
has the benefit that the compiler complains when we add new
ADDR_INTERNAL types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell a1dc4eef56 wireaddr: tell caller that we failed due to wanting DNS lookup, don't try.
This is useful for the next patch, where we want to hand the unresolved
name through to the proxy.

This also addresses @Saibato's worry that we still called getaddrinfo()
(with the AI_NUMERICHOST option) even if we didn't want a lookup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 5345e43354 gossipd: rename use_tor to use_proxy,
Not all of them, but it's really about using the SOCKS proxy rather than
really using Tor at this level.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell bcb047a729 gossipd: fix uninitialized var.
We assert() that it's set by one of the branches (it should be!) but
if we don't hit one it's uninitialized, not NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 011d75c618 lightningd: don't allow --announce-addr with a wildcard address.
It doesn't make sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 1ab9a5def6 wireaddr: fix parsing of empty address with no port.
Port should become the default in not specified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell cca791d1cb routing: clean up channel public/active states.
1. If we have a channel_announcement, the channel is public, otherwise
   it's not.  Not all channels are public, as they can be local: those
   have a NULL channel_announcement.

2. If we don't have a channel_update, we know nothing about that half
   of the channel, and no other fields are valid.

3. We can tell if a half channel is disabled by the flags field directly.

Note that we never send halfchannels without an update over
gossip_getchannels_reply so that marshalling/unmarshalling can be
vastly simplified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell 9d1e496b11 gossipd: use a real update in local_add_channel.
We generate one now, so let's use it.  That lets us simplify the
code, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell c71e16f784 broadcast: invert ownership of messages.
Make the update/announce messages own the element in the broadcast map
not the other way around.

Then we keep a pointer to the message, and when we free it
(eg. channel closed, update replaces it), it gets freed from the
broadcast map automatically.

The result is much nicer!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell 8940528bdb gossipd: don't include private announcements into broadcast map.
Basically, if we don't have an announcement for the channel, stash it,
and once we get an announcement, replay if necessary.

Fixes: #1485
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell cdded4f53d tests/utils.py: make default timeout configurable via $TIMEOUT
This is useful when you're expecting a test to fail, you've just added
diagnostics.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell d1b28f832d gossipd: when reconnecting, make sure we free old connection.
Looks like old connection got a callback, and we blew up since
the old peer was freed:

2018-05-06T10:57:11.865Z lightning_gossipd(14387): ...will try again in 300 seconds
2018-05-06T10:57:16.397Z lightning_gossipd(14387): peer_out WIRE_INIT
2018-05-06T10:57:16.405Z lightning_gossipd(14387): peer_in WIRE_INIT
2018-05-06T10:57:16.406Z lightning_gossipd(14387): peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4: reconnect for local peer
2018-05-06T10:57:16.406Z lightning_gossipd(14387): peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4 now remote
2018-05-06T10:57:16.406Z lightning_gossipd(14387): UPDATE WIRE_GOSSIP_PEER_CONNECTED
2018-05-06T10:57:16.406Z lightning_gossipd(14387): UPDATE WIRE_GOSSIP_PEER_CONNECTED
2018-05-06T10:57:16.406Z lightning_gossipd(14387): Handing back peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4 to master
2018-05-06T10:57:16.420Z lightning_gossipd(14387): hand_back_peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4: now local again
2018-05-06T10:57:16.420Z lightning_gossipd(14387): FATAL SIGNAL 11
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: common/daemon.c:42 (crashdump) 0x416991
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x7f70cf57a4af
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: common/msg_queue.c:38 (msg_dequeue) 0x418232
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: gossipd/gossip.c:816 (peer_pkt_out) 0x404ac4
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x4316db
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/io.c:427 (io_do_always) 0x4322ce
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/poll.c:228 (handle_always) 0x433abd
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/poll.c:249 (io_loop) 0x433b48
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: gossipd/gossip.c:2407 (main) 0x4093aa
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x7f70cf56582f
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x402ad8
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0xffffffffffffffff
2018-05-06T10:57:16.421Z lightning_gossipd(14387): STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL

Fixes: #1469
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:11:00 +02:00
Rusty Russell 2bc9d7594c doc/TOR.md: update for new options.
Currently the Tor port for autotor is always 9735; the --port option
is deprecated, though we could append an optional '[@portnum]' to
'autotor' if people want it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 89c76a5a78 Move always-use-proxy auto-override to master daemon.
This means it will effect connect commands too (though it's too
late to stop DNS lookups caused by commandline options).

We also warn that this is one case where we allow forcing through Tor
without a proxy set: it just means all connections will fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 1106c40217 tor: add new 'autotor:' address option.
This takes the Tor service address in the same option, rather than using
a separate one.  Gossipd now digests this like any other type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell a8c0bca6a8 gossipd: take over negotiation of autogenerated Tor addresses.
For the moment, this is a straight handing of current parameters through
from master to the gossip daemon.  Next we'll change that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e93682e3bf status: make status_io a more generic mechanism.
Currently it's always for messages to peer: make that status_peer_io and
add a new status_io for other IO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 5a0bc83b20 Tor: don't do seed queries if we're supposed to always use proxy.
Risks leakage.  We could do lookup via the proxy, but that's a TODO.

There's only one occurance of getaddrinfo (and no gethostbyname), so
we add a flag to the callers.

Note: the use of --always-use-proxy suppresses *all* DNS lookups, even
those from connect commands and the command line.

FIXME: An implicit setting of use_proxy_always is done in gossipd if it
determines that we are announcing nothing but Tor addresses, but that
does *not* suppress 'connect'.

This is fixed in a later patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 9d8e3cf3da gossip: handle Tor proxy better.
1. Only force proxy use if we don't announce any non-TOR address.
   There's no option to turn it off, so this makes more sense.
2. Don't assume we want an IPv4 socket to reach proxy, use the family
   from the struct addrinfo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell c3ccc14f19 Tor: remove --tor prefix from SOCKS5 options.
It's usually for Tor, but we can use a socks5 proxy without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00