Commit Graph

26 Commits

Author SHA1 Message Date
Rusty Russell fd9c0c8543 routing: move struct node_connection into struct routing_channel.
No need to have pointers since they're always there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 33726b0a08 gossip: instead of refresh interval, have routing know prune_timeout.
This is twice the 'update_channel_interval' we get handed.

We delete the non-existent channel_add_connection and delete_connection
declarations from the header too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 942d04ba87 gossipd: simplify channel_announce handling.
We make new_routing_channel() populate both connections
(active=false), so local_add_channel becomes simpler.  We also
suppress listchannels output of active=false unannounced channels, to
avoid breaking tests (also, these are unusable, so it makes sense to
omit them)

It also seems the logic in add_channel_direction is legacy: a
channel_announce cannot replace the scid (that would be a different
channel), we don't allow duplicate announcements, and the announcement
is never NULL.

And since we disallow repeated channel_announce already, I believe
'forward' is always true, greatly simplifying the logic in
handle_pending_cannouncement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell 9b900138d0 gossip: put 'routing_channel' in charge of 'node_connection'.
This makes 'routing_channel' the primary object in the system; it can have
one or two 'node_connection's attached, and points to two nodes.

The nodes are freed when no more routing_channel refer to them.  The
routing_channel are freed when they contain no more 'node_connection'.
This fixes #1072 which I surmise was caused by a dangling
routing_channel after pruning.

Each node contains a single array of 'routing_channel's, not one for
each direction.  The 'routing_channel' itself orders nodes in key
order (conveniently the index is equal to the direction flag we use),
and 'node_connection' with source in the same order.

There are helpers to assist with common questions like "which
'node_connection' leads out of this node?".

There are now two ways to find a channel:
1. Direct scid lookup via rstate->channels map.
2. Node key lookup, followed by channel traversal.

Several FIXMEs are inserted for where we can now do things more optimally.

Fixes: #1072
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
ZmnSCPxj 6767434ea9 routing: Use siphash24 for route randomization
Primary idea by @rustyrussell
2018-02-26 02:36:27 +00:00
ZmnSCPxj aff52ce5a8 gossipd: Implement improved randomization of routes.
Fixes: #928
2018-02-26 02:36:27 +00:00
practicalswift 91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
Rusty Russell e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell 26617d1d64 gossipd/test: update mocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell fd498be7ca status: generate messages rather than marshal/unmarshal manually.
Now we have wirestring, this is much more natural.  And with the
24M length limit, we needn't be so concerned about dumping 64k peer
messages in hex.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell 84bf60f934 status: add multiple levels of logging.
status_trace maps to status_debug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Christian Decker 26b5588656 gossip: Use replace_broadcast when enqueuing a channel_announcement
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
ZmnSCPxj 67e2275383 gossipd: Add handling of channel_update to routing failures. 2018-02-01 00:46:06 +00:00
ZmnSCPxj b255d82ae2 gossipd: Implement `gossip_routing_failure` for master to report routing failures. 2018-02-01 00:46:06 +00:00
Rusty Russell 047a2ea043 gossip: don't use assert around code with side effects.
The use of status_failed() requires a stubs update, which fails
with unnamed parameters, so tweak the status.h header as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-22 15:47:17 +01:00
Christian Decker 83caf1fdab routing: Add tracking of eviction for broadcasts
We should never be evicting channel_announcements because a) they were
deeply buried and should not change the short_channel_id/tag, b) they
are static.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker 1b28220b64 Revert "broadcast: don't reorder channel_announce when we get the real one."
This reverts commit f293ff0a6a.
2017-12-22 15:47:17 +01:00
Rusty Russell 887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell f293ff0a6a broadcast: don't reorder channel_announce when we get the real one.
If channel_announce is rebroadcast, it should replace the existing one
in-place.  We currently only do this if we start from the unsigned one
and replace it with the signed one when we hit 6 confirms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell bb601a1eeb gossipd/test/run-bench-find_route: don't abort if we try to route to ourselves
This would fail, and we'd free an uninitialized pointer.

Also, add us to .gitignore and clear up a comment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 13:57:28 +01:00
Rusty Russell bd27eba6f8 bench: reduce defaults for travis run.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell a7eee0b669 routing: precalc per-block risk factor.
Saves a little by doing up-front calculation.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 42863 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell 589cb673ce routing: remove negative fee support.
We can't get them; channel_update doesn't support it.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 47677 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell 21cc904b03 gossipd/test/run-bench-find_route.c: add perfme support.
Compile this, and link from perfme-start and perfme-stop in your path:

/* Simple wrapper to allow a program to perf itself. 
 * Copyright Rusty Russell, Blockstream 2015.
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * See <http://www.gnu.org/licenses/>.
 */
#include <ccan/err/err.h>
#include <ccan/str/str.h>
#include <sys/types.h>
#include <unistd.h>
#include <signal.h>
#include <errno.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>

#define PERFME_PREFIX "/tmp/perfme."
#define MAX_ENV_ARGS 20

static void write_noerr(int fd)
{
	int e = errno;
	if (write(fd, "", 1) != 1)
		/* Complain about warn_unused_result fascist bullshit */ ;
	errno = e;
}

/* Child.  Setup pid, run perf. */
static void exec_perf(int pfd[2], const char *perfpid, const char *perfout,
		      pid_t parent)
{
	char pid[STR_MAX_CHARS(pid_t)];
	int i, fd;
	char *cmd, *args[MAX_ENV_ARGS + 5];
	
	fd = open(perfpid, O_CREAT|O_EXCL|O_WRONLY, 0400);
	if (fd < 0) {
		write_noerr(pfd[1]);
		err(1, "opening %s", perfpid);
	}

	sprintf(pid, "%u", getpid());
	if (write(fd, pid, strlen(pid)) != strlen(pid)) {
		write_noerr(pfd[1]);
		err(1, "writing to %s", perfpid);
	}
	close(fd);

	sprintf(pid, "%u", parent);
	cmd = getenv("PERFME");
	if (!cmd)
		cmd = "perf record --call-graph dwarf -q";
	cmd = strdup(cmd);
	for (i = 0; i < MAX_ENV_ARGS; i++) {
		args[i] = strtok(i == 0 ? cmd : NULL, " ");
		if (!args[i])
			break;
	}
	if (i == 0 || i == MAX_ENV_ARGS)
		errx(1, "Too %s args in $PERFME: '%s'",
		     i ? "many" : "few", getenv("PERFME"));

	args[i++] = "-p";
	args[i++] = pid;
	args[i++] = "-o";
	args[i++] = (char *)perfout;
	args[i++] = NULL;

	execvp(args[0], args);
	write_noerr(pfd[1]);
	err(1, "Execing %s", args[0]);
}

int main(int argc, char *argv[])
{
	pid_t parent = argv[1] ? atoi(argv[1]) : getppid();
	char perfout[sizeof(PERFME_PREFIX) + STR_MAX_CHARS(parent)];	
	char perfpid[sizeof(perfout) + sizeof(".pid")];

	err_set_progname(argv[0]);

	sprintf(perfpid, PERFME_PREFIX "%u.pid", parent);
	if (strends(argv[0], "perfme-stop")) {
		char pid[STR_MAX_CHARS(pid_t)];
		int r, fd = open(perfpid, O_RDONLY);
		if (fd < 0)
			err(1, "Opening %s", perfpid);
		r = read(fd, pid, sizeof(pid) - 1);
		if (r < 0)
			err(1, "Reading %s", perfpid);
		pid[r] = 0;
		if (unlink(perfpid) != 0)
			warn("Unlinking %s", perfpid);
		if (atoi(pid) <= 0)
			errx(1, "Invalid pid '%s' from %s", pid, perfpid);
		if (kill(atoi(pid), SIGTERM) != 0)
			err(1, "Stopping %s", pid);
		exit(0);
	} else if (strends(argv[0], "perfme-start")) {
		int pfd[2];

		sprintf(perfout, PERFME_PREFIX "%u", parent);

		/* Use pipe to detect successful exec. */
		if (pipe(pfd) != 0)
			err(1, "Creating pipe");
		
		switch (fork()) {
		case 0:
			close(pfd[0]);
			fcntl(pfd[1], F_SETFD,
			      fcntl(pfd[1], F_GETFD)|FD_CLOEXEC);

			exec_perf(pfd, perfpid, perfout, parent);
		case -1:
			err(1, "Forking");
		default:
			/* Parent.  Wait for child. */
			close(pfd[1]);
			if (read(pfd[0], perfpid, 1) == 1)
				exit(1);
			fprintf(stderr, "Perf recording into %s\n", perfout);
			sleep(1);
			exit(0);
		}
	}
	errx(1, "Unknown name: am I perfme-start or perfme-stop?");
}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell 4a54884d4b gossipd: routing benchmark.
Initial run (100,000 nodes):

$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 15646 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00