Commit Graph

437 Commits

Author SHA1 Message Date
Rusty Russell dba756ef38 common: fix up BOLT 3 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell 0dc406c07a common/bolt11.c: fix up BOLT 11 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell f5d637059d common/cryptomsg.c: fix up bolt 8 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell 20e5bc938e wireaddr: zero entire sockaddr structure.
2018-06-14T01:09:03.495Z lightningd(23766): HSM: created new hsm_secret file
==23785== Syscall param socketcall.bind(my_addr.sin6_flowinfo) points to uninitialised byte(s)
==23785==    at 0x5731877: bind (syscall-template.S:78)
==23785==    by 0x11767C: make_listen_fd (gossip.c:2405)
==23785==    by 0x117DA2: handle_wireaddr_listen (gossip.c:2558)
==23785==    by 0x1183B7: setup_listeners (gossip.c:2653)
==23785==    by 0x118E86: gossip_activate (gossip.c:2871)
==23785==    by 0x11AC42: recv_req (gossip.c:3543)
==23785==    by 0x143FF1: next_plan (io.c:59)
==23785==    by 0x144AEE: do_plan (io.c:387)
==23785==    by 0x144B2C: io_ready (io.c:397)
==23785==    by 0x146719: io_loop (poll.c:310)
==23785==    by 0x11B0B0: main (gossip.c:3687)
==23785==  Address 0x1ffeffffa4 is on thread 1's stack
==23785==  in frame #2, created by handle_wireaddr_listen (gossip.c:2539)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 13:08:10 +02:00
Rusty Russell 02b55e35b8 daemon: print version in crash backtrace.
Sometimes people only send us the crash, not the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-12 23:18:53 +02:00
Rusty Russell 3e07971582 features: define LOCAL_GOSSIP_QUERIES feature.
From BOLT #9 proposed update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Mark Beckwith 7f437715d5 Added error code parameter to command_fail
Until now, `command_fail()` reported an error code of -1 for all uses.
This PR adds an `int code` parameter to `command_fail()`, requiring the
caller to explicitly include the error code.

This is part of #1464.

The majority of the calls are used during parameter validation and
their error code is now JSONRPC2_INVALID_PARAMS.

The rest of the calls report an error code of LIGHTNINGD, which I defined to
-1 in `jsonrpc_errors.h`.  The intention here is that as we improve our error
reporting, all occurenaces of LIGHTNINGD will go away and we can eventually
remove it.

I also converted calls to `command_fail_detailed()` that took a `NULL` `data`
parameter to use the new `command_fail()`.

The only difference from an end user perspecive is that bad input errors that
used to be -1 will now be -32602 (JSONRPC2_INVALID_PARAMS).
2018-05-26 12:17:36 +02:00
Rusty Russell 4359f46ffd features: fix features_supported() function.
1. We need to test all bits, not all bytes.
2. Both local and global features need to be supported.
3. Untested code is broken code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Rusty Russell 149e967ba3 features: fix endian.
Closes: https://github.com/lightningnetwork/lightning-rfc/issues/424

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
practicalswift 3abd9ad4f6 wireaddr: Avoid memory corruption in fmt_wireaddr_without_port() 2018-05-15 05:30:43 +00:00
Rusty Russell 1125682ceb wireaddr: new type, ADDR_INTERNAL_FORPROXY, use it if we can't/wont resolve.
Tor wasn't actually working for me to connect to anything, but it worked
for 'ssh -D' testing.

Note that the resulting 'netaddr' is a bit weird, but I guess it's honest.

    $ ./cli/lightning-cli connect 021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b
    {
      "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b"
    }
    $ ./cli/lightning-cli listpeers
    {
      "peers": [
        {
          "state": "GOSSIPING", 
          "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b", 
          "netaddr": [
            "ln1qg0je0lugpzu5ttsv78vlrkhteyg9yy8fjw68qr57mfhsfyrxurzkq522ah.lseed.bitcoinstats.com:9735"
          ], 
          "connected": true, 
          "owner": "lightning_gossipd"
        }
      ]
    }

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell a1dc4eef56 wireaddr: tell caller that we failed due to wanting DNS lookup, don't try.
This is useful for the next patch, where we want to hand the unresolved
name through to the proxy.

This also addresses @Saibato's worry that we still called getaddrinfo()
(with the AI_NUMERICHOST option) even if we didn't want a lookup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 1ab9a5def6 wireaddr: fix parsing of empty address with no port.
Port should become the default in not specified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell 89c76a5a78 Move always-use-proxy auto-override to master daemon.
This means it will effect connect commands too (though it's too
late to stop DNS lookups caused by commandline options).

We also warn that this is one case where we allow forcing through Tor
without a proxy set: it just means all connections will fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 1106c40217 tor: add new 'autotor:' address option.
This takes the Tor service address in the same option, rather than using
a separate one.  Gossipd now digests this like any other type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell a8c0bca6a8 gossipd: take over negotiation of autogenerated Tor addresses.
For the moment, this is a straight handing of current parameters through
from master to the gossip daemon.  Next we'll change that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e93682e3bf status: make status_io a more generic mechanism.
Currently it's always for messages to peer: make that status_peer_io and
add a new status_io for other IO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 5a0bc83b20 Tor: don't do seed queries if we're supposed to always use proxy.
Risks leakage.  We could do lookup via the proxy, but that's a TODO.

There's only one occurance of getaddrinfo (and no gethostbyname), so
we add a flag to the callers.

Note: the use of --always-use-proxy suppresses *all* DNS lookups, even
those from connect commands and the command line.

FIXME: An implicit setting of use_proxy_always is done in gossipd if it
determines that we are announcing nothing but Tor addresses, but that
does *not* suppress 'connect'.

This is fixed in a later patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d87a6c3a48 wireaddr: more helpers, to convert to addrinfo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d5e4d52563 common/base32: make this a simple tal-wrapper around ccan/base32.
And use it in wireaddr.

We fix up the double '.onion' in the test case, which seems like an error?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e91975c22d wireaddr: use fmt_wireaddr_without_port in fmt_wireaddr.
Avoids duplicate code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 7900318216 wireaddr: fix tor address lengths.
This variable does NOT include the 2 bytes for the port.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 7d95dbbd0a wireaddr: remove FQDN_ADDRLEN in favor of LARGEST_ADDRLEN.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 85eff42d9a common/tor: move into lightningd.
This is simply the code to set up the automatic hidden service, so move
it into lightningd.

I removed the undefined parse_tor_wireaddr, and added a parameter name
to the create_tor_hidden_service_conn() declaration for update-mocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell d9f13230cf gossip/tor.c: new file for socks proxy code.
All gossipd needs from common/tor is do_we_use_tor_addr(), so move
that and the rest of the tor-specific handshake code into gossip/tor.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato 877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Isidoro Ghezzi 855d0b9401 "sizeof(sun->sun_path)" is not always the same as "sizeof(addr->u.sockname)"
on all platforms;

because of that BUILD_ASSERT was failing on macOS.

(on macOS "sizeof(sun->sun_path) == 104" and
"sizeof(addr->u.sockname) == 108")

[ Linux man page says it can be as small as 92, so let's use the real value.
  I also cleaned up the incorrect comment order on that struct! --RR ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 09:10:45 +00:00
ZmnSCPxj 926b41b7da closingd: Ensure proper closing of TCP socket.
Fixes: #1457
2018-05-09 04:43:21 +00:00
Rusty Russell d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 6b2282fc1d wireaddr_internal: new type for where we can also use a local socket.
This was something @icota implemented, but it fits logically into this
cleanup series.  We create a new type which is the internal generalization
of a wireaddr (which is defined by the spec), and add a case here for
a socket name.

Based-on-the-true-story-by: @icota
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 356e5dcea8 wireaddr: helpers to convert to/from IPv4/v6 addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 666e1b320f lightningd: fix double-specified args.
We need to make sure the arg is a tal object, as we'll free it next time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Mark Beckwith 7767b68ee9 Removed redundancies in withdraw and fundchannel.
No new functionality, just a continuation of my work toward completing #665.

I removed the common members of `struct withdrawal` and `struct fund_channel`
and placed them in a new `struct wallet_tx`.  Then it was fairly straightforward
to reimplement the existing code in terms of `wallet_tx`.

Since I made some structural changes I wanted to get this approved before I
go any farther.

Added 'all' to fundchannel help message.
2018-05-03 18:20:20 +02:00
practicalswift abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
Rusty Russell bf1076080b common: typo fix.
Old gossip is rarely interesting.

Reported-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell b68fb24758 read_peer_msg: handle incoming gossip from gossipd.
This means that openingd and closingd now forward our gossip.  But the real
reason we want to do this is that it gives an easy way for gossipd to kill
any active daemon, by closing its fd: previously closingd and openingd didn't
read the fd, so tended not to notice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell ab9d9ef3b8 gossipd: drain fd instead of passing around gossip index.
(This was sitting in my gossip-enchancement patch queue, but it simplifies
this set too, so I moved it here).

In 94711969f we added an explicit gossip_index so when gossipd gets
peers back from other daemons, it knows what gossip it has sent (since
gossipd can send gossip after the other daemon is already complete).

This solution is insufficient for the more general case where gossipd
wants to send other messages reliably, so replace it with the other
solution: have gossipd drain the "gossip fd" which the daemon returns.

This turns out to be quite simple, and is probably how I should have
done it originally :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Christian Decker a0c1b7af1f moveonly: Move DNS resolution to wireaddr conversion
This is the simple getaddrinfo lookup and conversion into wireaddr.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 12:34:55 +02:00
Christian Decker c635396766 common: Moving some bech32 related utilities to bech32_util
These were so far only used for bolt11 construction, but we'll need them for the
DNS seed as well, so here we just pull them out into their own unit and prefix
them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 12:34:55 +02:00
practicalswift 0eff28c80f Assert our parse_wireaddr assumptions in run-ip_port_parsing.c
We assume that the parse_wireaddr(...) calls return true.
Otherwise addr.port will be uninitialized.
2018-04-15 17:45:47 +02:00
Christian Decker 723b64036f jsonrpc: Pretty-print the json results
Just a small cleanup of the indentation code, so we don't have to reformat all
the issue reports to become readable. This is much closer to what `jq` or
`json_pp` spit out and doesn't have those infinitely long lines.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-08 08:26:00 +00:00
Rusty Russell 09c4203767 bolt11: allow multiple fallback addresses.
We can have more than one; eg we might offer both bech32 and a p2sh
address, and in future we might offer v1 segwit, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-06 14:26:53 +02:00
Rusty Russell 21fbae6df8 openingd: ensure that initial channel can cover fees and reserve.
This is probably covered by our "channel capacity" heuristic which
requires the channel be significant, but best to be explicit and sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Justin Litchfield d3d195dd85 bt_print and bt_exit is enclosed in the conditional 2018-04-05 18:17:55 +02:00
practicalswift 66fc0a047b Make version output simpler/more user-friendly 2018-04-04 02:32:44 +00:00
Rusty Russell feb6b52f0f lightningd: don't try to perform backtrace if LIGHTNINGD_DEV_NO_BACKTRACE
Valgrind gets upset.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Rusty Russell 1a4a59d221 common/daemon: common routines for all daemons.
In particular, the main daemon and subdaemons share the backtrace code,
with hooks for logging.

The daemon hook inserts the io_poll override, which means we no longer
need io_debug.[ch].  Though most daemons don't need it, they still link
against ccan/io, so it's harmess (suggested by @ZmnSCPxj).

This was tested manually to make sure we get backtraces still.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Rusty Russell 8975fc2ceb libwally: use their secp context for all our daemons.
I didn't convert all tests: they can still use a standalone context.
It's just marginally more efficient to share the libwally one for all
our daemons which link against it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Rusty Russell 20bbd92564 utils: add subdaemon_shutdown() to consolidate subdaemon cleanup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
practicalswift e25297dd0a Remove unused functions not covered by unit tests 2018-03-28 11:22:05 +02:00
practicalswift 8df29d169c The overflow check mul_overflows_s64(int64_t, int64_t) overflows and triggers UB :-) Remove it
The overflow check `mul_overflows_s64(int64_t, int64_t)` overflows.
Since this is an signed integer overflow this triggers UB, which
in turn means that we cannot trust the check.

Luckily mul_overflows_s64(int64_t, int64_t) is unused. Removing it.
2018-03-27 23:18:48 +00:00
Christian Decker ba7341ec87 cleanup: Make blockheights unsigned
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
practicalswift 6269a4c55d Remove unused functions not covered by unit tests 2018-03-26 23:35:56 +00:00
practicalswift 7e9750ffee Reduce variable scopes 2018-03-26 01:31:21 +00:00
practicalswift 94ca824cf5 Use correct format strings for signed integers 2018-03-26 01:20:22 +00:00
conanoc e7c8b95be3 Fix include order 2018-03-26 00:30:39 +00:00
conanoc c4700a13a7 Add guard for BACKTRACE_SUPPORTED 2018-03-26 00:30:39 +00:00
Rusty Russell dfce8b5efd common/json: move declaration into correct header.
Reported-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell 7ae013202f json: make json_add_string do partial escapes.
This is useful when we log a JSON-escaped string, so we don't double-escape.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell ed9093fcbd json: allow strange characters through our JSON parser.
Fixes: #387
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell 9f7d4312ff bolt11: undo json encoding for description bytes.
We don't handle \u, since we assume everyone sane is using UTF-8.  We'd
still have to reject '\u0000' and maybe other weird cases if we did.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell d92579f627 common/json_escaped: new type which explicitly notes a string is already JSON.
Trivial to use as a string, but it still means you should be careful
around it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift 98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift 8f76581005 Use pointer to const where possible 2018-03-16 01:17:40 +00:00
practicalswift f0667d239a Remove duplicate va_end(...) call. va_end(...) is the responsibility of the caller. 2018-03-16 01:17:40 +00:00
Rusty Russell e63b7bb539 take: allocate temporary variables off NULL.
If we're going to simply take() a pointer, don't allocate it off a random
object.  Using NULL makes our intent clear, particularly with allocating
packets we're going to take() onto a queue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell 0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell ccc9414356 status: remove trc context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell 4845445079 memleak: ignore tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell ef2a063169 utils: add a global tmpctx.
I did a brief audit of tmpctx uses, and we do leak them in various
corner cases.  Fortunely, all our daemons are based on some kind of
I/O loop, so it's fairly easy to clean a global tmpctx at that point.

This makes things a bit neater, and slightly more efficient, but also
clearer: I avoided creating a tmpctx in a few places because I didn't
want to add another allocation.  With that penalty removed, I can use
it more freely and hopefully write clearer code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell 5e333b75b9 daemon_conn: simplify msg_queue_cleared_cb.
Now it just returns true if it queued something.  This allows it
to queue multiple packets, and lets it share code paths with other code
in future patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell 46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell 5eea772c53 type_to_string: support sha256_double.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell 61f048bbf1 gossip: rename is_gossip_msg to is_msg_for_gossipd.
We're going to expand the range of messages which go through gossipd
when we support queries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Christian Decker 08bfb740f5 onion: Move cli onion tool to devtools/onion
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-09 17:57:55 +01:00
Corné Plooy c3a46de129 Fix order of includes 2018-03-06 19:26:21 +01:00
Corné Plooy dbbc61bc4e Coding style fix 2018-03-06 19:26:21 +01:00
Corné Plooy b857b2e843 Add assertions in various places to ensure tal_fmt doesn't receive NULL as argument for strings. 2018-03-06 19:26:21 +01:00
Christian Decker 227dc36146 utxo: Add blockheight and spendheight to outputs to track state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Rusty Russell 30faa6485a subdaemons: copy backtraces to stderr.
We didn't get the entire thing in the parent when gossipd crashed: stderr
is suitable for these I think.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
ZmnSCPxj f7f55edcdb wireaddr: Do hostname resolution in parse_wireaddr.
Fixes: #834
2018-02-26 03:13:55 +00:00
Rusty Russell 9cffa03647 peer_failed: set permanent slot when we fail the peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell d4a172e221 common: peer_billboard() helper for updating the billboard.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell 26b004e5af subd: handle status_peer_billboard messages from subdaemons.
We use a callback which updates the appropriate slot.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell b8c636514b pong: embed version string into ping replies if DEVELOPER=1.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-22 12:24:26 +01:00
Rusty Russell 3e7d98a52d status: don't log gossip messages in channeld.
Looking at an example log from #968, 288612 of 289244 lines were simply
channeld logging incoming and outgoing gossip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-22 08:22:44 +00:00
practicalswift 91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
Rusty Russell e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell cfa50d393a openingd: use peer_failed like normal instead of boutique negotiation_failed.
Because peer_failed would previously drop the connection, we had a
special 'negotiation_failed' message which made the master hand it
back to gossipd.  We don't need that any more.

This also meant we no longer need a special hook in read_peer_msg
for openingd to send this message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell dba08f9d1b peer_failed: don't send error ourselves.
gossipd actually does that now, so we don't need this synchronous send
hack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 02d469b3d4 peer_failed: hand fds back to master when we fail.
master now hands it back to gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell f76ff90485 status: split off error messages into a new 'peer_status' type.
Several daemons (onchaind, hsm) want to use the status messages, but
don't communicate with peers.  The coming changes made them drag in
more code they didn't need, so instead we have a different
non-overlapping type.

We combine the status_received_errmsg and status_sent_errmsg
into a single status_peer_error, with the presence or not of the
'error_for_them' field indicating direction. 

We also rename status_fatal_connection_lost() to
peer_failed_connection_lost() to fit in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 65ff5f8bb1 read_peer_msg: ignore errors not destined for this channel.
We quoted the spec, but somehow the implementation disappeared.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 201d498e39 peer_failed: automatically hand PEER_FD, GOSSIP_FD; add gossip_index
We make it a macro, since everyone uses PEER_FD and GOSSIP_FD constants
(they're actually always the same, but this is slightly safer), and
add a gossip_index arg: this is groundwork for when we want to hand
the peer back to master for gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 611ecc60ae lightningd: rename peer_state -> channel_state, remove OPENINGD.
And now we can finally do the db upgrade to remove any OPENINGD
channels once, since we never put them back.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 7c512f91ce memleak: ignore tal_link artifacts.
We use strends, because that works with or without CCAN_TAL_DEBUG (which
prepends file and line).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
John Barboza 7a77271922 lightningd: encode fallback address in 5 bits
The bolt11 specification requires that the version number of the
Fallback on-chain address should be 5 bits wide instead of 8
bits.
2018-02-19 02:01:37 +00:00
Rusty Russell 55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell e76a0b4ddc gossipd: fix race where we can handoff peer with bad cryptostate.
DEBUG:root:lightningd(16333): 2018-02-08T02:12:21.158Z lightningd(8262): lightning_openingd(0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199): Failed hdr decrypt with rn=2

We only hand off the peer if we've not started writing, but that was
insufficient: we increment the sn twice on encrypting packet, so there's
a window before we've actually started writing where this is now
wrong.

The simplest fix is only to hand off from master when we've just written,
and have the read-packet path simply wake the write-packet path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-13 12:10:35 +01:00
ZmnSCPxj e1284b1df1 common/json: Add json_add_double. 2018-02-09 12:44:33 +01:00
practicalswift 4f4756bd20 Fix a-vs-an typos 2018-02-08 22:49:34 +01:00
Rusty Russell eb0603bd13 wireaddr: rework port parsing for weird addresses.
We save wireaddr to databases as a string (which is pretty dumb) but
it turned out that my local node saved '[::ffff:127.0.0.1]:49150'
which our parser can't parse.

Thus I've reworked the parser to make fewer assumptions:
parse_ip_port() is renamed to separate_address_and_port() and is now
far more accepting of different forms, and returns failure only on
grossly malformed strings.  Otherwise it overwrites its *port arg only
if there's a port specified.  I also made it static.

Then fromwire_wireaddr() hands the resulting address to inet_pton to
figure out if it's actually valid.

Cc: William Casarin <jb55@jb55.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:14:21 +01:00
Rusty Russell cc9ca82821 status: separate types for peer failure vs "impossible" failures.
Ideally we'd rename status_failed() to status_fatal(), but that's
too much churn for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell fd498be7ca status: generate messages rather than marshal/unmarshal manually.
Now we have wirestring, this is much more natural.  And with the
24M length limit, we needn't be so concerned about dumping 64k peer
messages in hex.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell c7b693d7ce status: remove unused status_send_sync.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell 526d3a232e tools/generate_wire.py: generate varlen arrays properly.
These are now logically arrays of pointers.  This is much more natural,
and gets rid of the horrible utxo array converters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell ad8dfaca1c tools/generate_wire.py: make varlen structs self-allocate.
If we tell it a struct is variable length, make fromwire() allocate
and return it off ctx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell 99e246becd channeld: rely on io_logging, not our own boutique logging.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Rusty Russell c01f3267d5 common: only log io if they set --debug-subdaemon-io=<daemon> or with SIGUSR1.
Otherwise we just log the type of msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Rusty Russell de56dc0ffc common: add logging for peer packets, with status_io.
We log the plaintext, not the encrypted ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Rusty Russell b7acd93578 log: separate levels for IO directions, allow msg + io data.
We currently don't handle LOG_IO properly, and we turn it into a string
before handing it to the ->print function, which makes it ugly for
the case where we're using copy_to_parent_log, and also means in
that case we lose *what peer* the IO is coming from.

Now, we handle the io as a separate arg, which is much neater.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Rusty Russell 84bf60f934 status: add multiple levels of logging.
status_trace maps to status_debug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Rusty Russell 8be1c1df32 Updates for changed external/jsmn API change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-05 04:08:16 +00:00
Saibato 7dcaf27bf5 allow --rpc-file option to change default value
short onliner as rustyrussel suggested
2018-02-02 20:35:44 +01:00
Rusty Russell 91a22dc496 jsonprc: make json_get_params() fail the command, for better error reporting.
We move it into jsonrpc where it belongs, and make it fail the command.
This means it can tell us exactly what was wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 00:05:00 +01:00
Rusty Russell 47577e5c4e jsonrpc: check that arguments to calls are valid.
This change could break users who accidentally have typos in scripts,
so we need to check sooner rather than later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 00:05:00 +01:00
Rusty Russell 7f03e15e03 json_add_string_escape: for escaping internally-generated strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 00:05:00 +01:00
Rusty Russell ef9b16399c common: read_peer_msg helpers for per-peer subds.
It's a bit ugly because each caller has slightly different needs, but
we have three hooks and standard helpers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-01 05:57:56 +00:00
rvandermeer 4c0f7dbd04 Spelling corrections (#824)
* Small spelling fixes, and clarity for 'iff'

[ Squashed commit --RR ]
2018-01-29 04:46:54 +00:00
practicalswift 689db5b7c1 Onboarding: Make log output texts friendlier to new users 2018-01-29 03:22:27 +00:00
ZmnSCPxj 8c527f3931 common/json: Implement json_add_snum for signed numbers. 2018-01-27 14:32:06 +01:00
Ronald van der Meer fa992ecaab clarified --lightning-dir syntax on help 2018-01-26 03:00:29 +00:00
practicalswift 2c17546cbd Onboarding: Make "lightning-cli" (without arguments) output the equivalent of "lightning-cli --help; lightning-cli help"
New users invoking lightning-cli are likely interested in what RPC
commands they can invoke via the command.
2018-01-26 00:35:13 +00:00
Carl Dong 8da65854f0 build: Add needed UNIX standard includes. 2018-01-23 16:10:19 +01:00
Björge Dijkstra 2f4ba73c77 Allocate hex buffer on heap since it can be very large. 2018-01-23 13:33:29 +01:00
Rusty Russell 6b740e78bd json: more sanity checks on JSON output.
We should never have an unnamed element, nor an named array field.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
practicalswift a87c8a74b5 Avoid segfault on CLI command "decodepay 1111111" (invalid short bech32 string)
Before this patch:

```
$ cli/lightning-cli decodepay 1111111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 111111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 11111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 1111111
lightning-cli: Non-object response ''
$ cli/lightning-cli decodepay 1111111
lightning-cli: Connecting to 'lightning-rpc': Connection refused
```

After this patch:

```
$ cli/lightning-cli decodepay 1111111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 111111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 11111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 1111111
"Invalid bolt11: Bad bech32 string"
$ cli/lightning-cli decodepay 1111111
"Invalid bolt11: Bad bech32 string"
```
2018-01-15 19:32:00 +00:00
Rusty Russell 1950583612 subdaemon: make debugging a bit easier.
Use a volatile global, so debugger can flip it easily.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 19:26:08 +00:00
William Casarin 4a12cafe7f debugging: add -ex return and cont to gdb command
Have gdb execute return and continue when attaching to a subdaemon

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-15 19:26:08 +00:00
William Casarin 44f9863192 permute_tx: bail on empty permute_{inputs/outputs} arguments
permute_outputs is sometimes called with empty arguments from initial_commit_tx.
Make sure we guard against that case. We also do the same in permute_inputs for
good measure.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-15 06:35:02 +00:00
ZmnSCPxj 83e76e3ac3 features: Move feature-handling code to a common/features.c source. 2018-01-13 11:29:42 +01:00
Björge Dijkstra 648e4feee2 Extend json unit test with tests for json_tok_bitcoin_amount() 2018-01-13 00:15:03 +01:00
Björge Dijkstra 82a2d2f0a6 Fix parsing of txout value. Force decimal base. 2018-01-13 00:15:03 +01:00
Rusty Russell e34ec8da2d peer_failed: use towire_errorfmtv() which doesn't add nul terminator.
This code was actually wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-12 09:43:01 +01:00
Rusty Russell 86e1a61165 json: fix json_tok_bitcoin_amount()
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
practicalswift dcb4039a96 Check lseek(...) return value 2018-01-09 13:52:12 +01:00
Rusty Russell 0f97b8cf36 subdaemon.c: subdaemon_setup() routine for all daemons.
Our handling of SIGPIPE was incoherent and inconsistent, and we had much
cut & paste between the daemons.  They should *ALL* ignore SIGPIPE, and
much of the rest of the boilerplate can be shared, so should be.

Reported-by: @ZmnSCPxj
Fixes: #528
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:16:39 +01:00
Christian Decker d9df831a2f Don't attempt to permute a single input
This was annoying valgrind since it would attempt to swap an input
with itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 248879e203 wire: Move bitcoin_tx serialization from htlc_wire to wire
Removes the need to keep a second transaction around and marking it as
`noleak`, just to make sure that dependencies are not free'd along
with the original tx.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 59128a86ee hsm: Make sure to pass close_info along with the UTXO
The close_info is needed to re-derive the secret key that is supposed
to be used to sign the input spending the output.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 0bb264e1a2 wallet: Added unilateral close info to utxo
This is necessary to grad the their_unilateral/to-us outputs since
they aren't being harvested by `onchaind`

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
practicalswift aae991f28d Add missing call to va_end()
Each invocation of va_start() must be matched by a corresponding
invocation of va_end() in the same function.
2018-01-02 01:19:03 +00:00
practicalswift db362e2e23 Remove redundant assignments 2018-01-02 00:38:20 +00:00
practicalswift f84828be36 Avoid writing uninitialized value data->realm in serialize_hop_data(...) (via create_onionpacket(...)) 2017-12-28 16:09:46 +01:00
practicalswift bfce5b41f9 Fix typo (a vs. an) 2017-12-28 16:04:38 +01:00
Rusty Russell 047a2ea043 gossip: don't use assert around code with side effects.
The use of status_failed() requires a stubs update, which fails
with unnamed parameters, so tweak the status.h header as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-22 15:47:17 +01:00
Rusty Russell 887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell 0237e0b28c bitcoin: create new wrapper type bitcoin_txid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_txid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell ed2158c334 Revert "Functions for encoding reversed hex"
This reverts commit ef5678956d2eedf9904ce32f12f6367bc26c59e7.
2017-12-21 11:05:38 +00:00
Andrew Chow 19116b6561 Functions for encoding reversed hex 2017-12-21 11:05:38 +00:00
William Casarin ce1d709d44 wireaddr: fix ipv6 formatting with ports in fmt_wireaddr
Correctly format ipv6 address with ports. This will also make it more compatible
with the new parse_wireaddr, which has been updated to parse ports. They are
inverses now.

Also add some tests that check this.

Signed-off-by: William Casarin <jb55@jb55.com>
2017-12-21 09:56:20 +00:00
William Casarin 3e7aabe256 test: add tests for parse_ip_port
This tests the basic functionality of parse_ip_port, which is used in
parse_wireaddr.

Signed-off-by: William Casarin <jb55@jb55.com>
2017-12-21 09:56:20 +00:00
William Casarin 7ecccd50b9 wireaddr: add ip[:port] parsing
* Add port parsing support to parse_wireaddr. This is in preparation for storing
addresses in the peers table. This also makes parse_wireaddr a proper inverse of
fmt_wireaddr.

* Move parse_wireaddr to common/wireaddr.c this seems like a better place for
it. I bring along parse_ip_port with it for convenience. This also fixes some
issues with the upcoming ip/port parsing tests.

Signed-off-by: William Casarin <jb55@jb55.com>
2017-12-21 09:56:20 +00:00
Rusty Russell bbac67f108 withdraw_tx: don't create empty output if no change.
We were using changekey as the flag to produce change, not changesat,
but the caller was using changesat as the flag.

Also, don't allocate changekey at all if we don't need it; this means
valgrind will complain if we use it at all, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:56:15 +01:00
Christian Decker ee25547576 Re-enable builds with DEVELOPER=0
Two changes:
 - Fixed the function signature of noleak_ to match in both
   configurations
 - Added memleak.o to linker for tests

Generating the stubs for the unit tests doesn't really work since the
stubs are checked in an differ between the two configurations, so
adding memleak to the linker fixes that, by not requiring stubs to be
generated in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 12:43:10 +01:00
Rusty Russell 8c4eecc089 sphinx: fix leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 795a03e8c4 fromwire_bitcoin_tx: quieten leak reporting.
We create a temporary tx which is a child of the real tx, for simplicity of
marshalling.  That's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 20fcb5d6ef memleak: allow marking of entire tal trees to be noleak.
In particular, the pay command attaches all kinds of stuff to the
struct command; they're not really a leak, since commands expire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 7a8532830f memleak: track reallocs and frees in notleak()
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 70d01b22e1 memleak: add second exclude arg to exclude current commands' jcon.
This is not a child of cmd, since they have independent lifetimes, but
we don't want to noleak them all, since it's only the one currently in
progress (and its children) that we want to exclude.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 19b1b35d31 memleak: explicitly exclude ccan/io allocation.
It's a single, full-lifetime allocation; make sure we only exclude that
one, though this is fragile: tests will break if it moves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell dfc132b2fe memleak: add backtrace to allocations.
We use the tal notifiers to attach a `backtrace` object on every
allocation.

This also means moving backtrace_state from log.c into lightningd.c, so
we can hand it to memleak_init().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell c956d9f5eb lightningd: tal memleak detection, dev-memleak command.
This is a primitive mark-and-sweep-style garbage detector.  The core is
in common/ for later use by subdaemons, but for now it's just lightningd.
We initialize it before most other allocations.

We walk the tal tree to get all the pointers, then search the `ld`
object for those pointers, recursing down.  Some specific helpers are
required for hashtables (which stash bits in the unused pointer bits,
so won't be found).

There's `notleak()` for annotating things that aren't leaks: things
like globals and timers, and other semi-transients.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 0cab9d1dd1 daemon_conn: helper to release daemon_conn.
We'll want this for the next change, where gossipd migrates remote peers
back to local ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell afea2520ba wireaddr: marshal empty address properly.
On unmarshal, we stop unmarshaling on a 0 (ADDR_TYPE_PADDING) type.  So
we should also stop marshaling in that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell b00525b8c8 options: don't leak memory when people override options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-13 22:46:10 +01:00
ZmnSCPxj 5fd74f9933 Change withdraw_tx to accept scriptpubkey rather than plain address. 2017-12-13 03:10:04 +00:00
Rusty Russell 0610f66c34 bolt11: handle r value fee spec change.
We don't use it yet, but now we'll decode correctly.

See: https://github.com/lightningnetwork/lightning-rfc/pull/317
lightning-rfc commit: ef053c09431442697ab46e83f9d3f86e3510a18e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-12 11:45:44 +01:00
practicalswift 61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Christian Decker 9ba99d2b2d hsm: Cleanup after merging control and client libraries
Change all calls to use the correct serialization and deserialization
functions, include the correct headers and remove the control
messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Rusty Russell 02411b04e4 channel: remove awaiting_revoke_and_ack flag.
We can tell this more generically because the count of revocations
received != count of commitments sent.  This is the correct condition
which allows us to restore the test we had to eliminate in
c3cb7f1c85.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:52:18 +01:00
Rusty Russell ed60e9ee57 close_tx: use version 2.
As per the pending spec change: the other implementations use version 2,
and the current best practice advice (from maaku) is to use the latest
defined tx version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-01 18:30:57 +01:00
Rusty Russell af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Christian Decker 9bcb01e710 fixup! common/test: add spec tests for bolt11. 2017-11-24 13:22:18 +01:00
Rusty Russell ae6faf7f9c common/test: add spec tests for bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell 60c30214c1 bolt11: fix encoding of x and c fields.
Fixes: #374
Reported-by: Nadav Ivgi
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell 45f5bb7fac bolt11: move to common/ and sign via callback.
JSON stuff is moved to lightningd/invoice.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell 78d0a5e840 bech32: move to common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
ueno 4312deddfa typo 2017-11-23 23:10:08 +01:00
Rusty Russell b836b452dc feerate: keep feerates separately for each side.
When we support changing them, they can be different during the transition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell f1e4cad9d4 feerate: use u32 everywhere.
The wire protocol uses this, in the assumption that we'll never see feerates
in excess of 4294967 satoshi per kiloweight.

So let's use that consistently internally as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 0a131c6389 channel: use flag to indicate we're awaiting revoke_and_ack.
We currently scan through HTLCs: this isn't enough if we've only got a
feechange in the commitment, so use a flag (but keep both for now for
debugging).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell c328a76438 channeld: use flags to track whether changes are pending.
This is required when we have non-HTLC changes (ie. fees).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 7ece90038b derive_basepoints: actually derive a separate HTLC key.
This means we'll find anywhere still using the payment key,
even though we still expose the private payment key to channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell 4db460903a htlc_tx: wire up the htlc points.
All the callers need to pass it in: currently channeld and openingd just
fake it by copying the payment point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell fe5614a489 basepoints/secrets: add htlc entry
Currently derive_basepoints just sets it to match the payment point/secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell 5502a19d1e json: reject incoming JSON which has any unusual characters in tokens.
ie. non-printable, quotes or escapes.  We allow these outside tokens
(we expect tabs and \n for example).

This is a big hammer, but if someone really wants we can add support
later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell 112ae0d0f5 common/test/run-json: test JSON escaping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell ab634dfcdd common/test_sphinx: change to standard run- format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell 8f17effeeb json: just blatt weird characters in string.
Don't try to escape them.  It's whack-a-mole and they shouldn't do it anyway.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 02c1d10c9f json: escape strings we output in JSON.
We're going to output description strings, which are untrusted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 753f15f503 common/hash_u5: routines to hash 5 bit values.
Needed for creating/verifying signatures on bolt11 invoices.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell a7d6326bef type_to_string: format wireaddr.
Good for printing, and removes some code from peer_control.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell 78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell bd1cac34ce netaddr: remove.
We use ipaddr everywhere now, so we can remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell ac92138603 common: remove unused assert() headers.
Auditing for assert/abort in common/ code used by lightningd, this is all
that showed up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 81db5896e1 common/json: remove asserts() which may trigger from user input.
They don't currently, since callers check, but be safe.  In addition,
handle NULL returns from these in the bitcoind code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 4e14185961 cryptomsg: add helpers to determine if we're partway through msg read/write.
For message read, we do it as header then body, so we can have
io_plan_in_started(conn) false, but we're between header and body.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell 74e684cc0d is_all_channels: rename to channel_id_is_all
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-22 16:24:10 +02:00
Rusty Russell 2273ce783e dev_disconnect: support multiple disconnects in the same daemon.
We currently assume the daemon gives up; gossipd won't, and we want to
test it there too.

This reveals a bug (returning io_close() is bad if the call is to
duplex()), and breaks a test which now continues after dropping a
packet..

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 6ceec17943 dev_disconnect: make commit suppression a "-nocommit" modifier.
Useful if we want to drop & suppress, for example.  We change '=' to mean
do nothing to the packet.

We use this to clean up the test_reconnect_sender_add test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 8430e33f3b common/status: add status_tracev() for making status wrappers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 9b589fb5ba common/wire_error: helpers to create/parse WIRE_ERROR messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 2394c9a2e7 crypto_state: move to its own file.
In particular, the main daemon needs to pass it about (marshal/unmarshal)
but it won't need to actually use it after the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 0969626918 close_tx: make version 1, not version 2.
Fixes: #311
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-18 13:00:29 +02:00
Rusty Russell f32e0b35ad Revert "Send/receive cltv_expiry_delta in open/accept channel."
This reverts commit 18e3f9820f.
2017-10-11 11:54:50 +02:00
Rusty Russell 18e3f9820f Send/receive cltv_expiry_delta in open/accept channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell e137e2527f Update BOLT references with typo fixes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Christian Decker 3dda72c885 db: Create table for HTLCs
Also added a small warning to one of the used enums not to reorder or
insert values. They'd break the update path.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Rusty Russell 3404c393b7 common/io_debug: replacement for ccan/io's poll which does sanity checks.
For now we just check for outstanding take() or tal_tmpctx().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell e587ec3bd3 tal_tmpctx: keep information around so we can find leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell ce160d9b17 lightnind: _ dev-disconnect argument to suppress commit timer.
Required for catching daemon in exact state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 6ef64cd52b script: make "sig_and_empty" more generic, make htlc tx witness fns clearer.
For non-delayed HTLC success spends, we have a similar pattern ("<sig>
<preimage> <wscript>") so a we want to use the same function.

The other routines don't say "witness" in them, and should.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 9123d6d45e type_to_string: support preimage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell 86313383a7 type_to_string: support ripemd160.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell a21e31b2b2 subdaemons: catch backtrace on fatal signals.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell ef28b6112c status: use common status codes for all the failures.
This change is really to allow us to have a --dev-fail-on-subdaemon-fail option
so we can handle failures from subdaemons generically.

It also neatens handling so we can have an explicit callback for "peer
did something wrong" (which matters if we want to close the channel in
that case).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell 4f6ae3209b common/test/test_sphinx: build fixes.
It isn't currently included in make check, so fix it to build and do that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-11 14:29:20 +02:00
Christian Decker a9f02a8985 sphinx: Remove annoying warning when MACs do not match
This was printing to stderr during the tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Christian Decker a44f893838 sphinx: Switch to version 0x00 like the spec says
We were using the wrong version, oops.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Rusty Russell 5889ad5fc4 dev-disconnect: add blackhole option.
To reproduce the next bug, I had to ensure that one node keeps thinking it's
disconnected, then the other node reconnects, then the first node realizes
it's disconnected.

This code does that, adding a '0' dev-disconnect modifier.  That means
we fork off a process which (due to pipebuf) will accept a little
data, but when the dev_disconnect file is truncated (a hacky, but
effective, signalling mechanism) will exit, as if the socket finally
realized it's not connected any more.

The python tests hang waiting for the daemon to terminate if you leave
the blackhole around; to give a clue as to what's happening in this
case I moved the log dump to before killing the daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 19:11:21 +02:00
Rusty Russell bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 739b163f8b Makefiles: simplify dependencies.
Gather all binaries and objects and make the depend on external
requirements and common headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell f42f34b82d external: new subdirectory for all external libraries and submodules.
You will want to 'make distclean' after this.

I also removed libsecp; we use the one in in libwally anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8c22bd9ee1 headers: fix up header idempotent lines.
For future reference, done via:
	for f in `find wire/ bitcoin/ common/ lightningd -name '*.h' ! -name 'gen*'`; do ID=`echo -n LIGHTNING/$f | tr 'a-z' 'A-Z' | tr -cs 'A-Z0-9' _`; sed 's/^#\(ifndef\|define\) .*_H$/#\1 '$ID/ < $f | sed 's,#endif /..*_H ./$,#endif /* '$ID' */,' | bagto $f; done

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell a37c165cb9 common: move some files out of lightningd/
Basically all files shared by different daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8375857116 common: absorb remaining files from daemon/
Also, we split the more sophisticated json_add helpers to avoid pulling in
everything into lightning-cli, and unify the routines to print struct
short_channel_id (it's ':',  not '/' too).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell a3c51168db routing: use status() instead of log-to-nowhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 85ff95e829 common: new directory for any shared objects.
To avoid everything pulling in HTLCs stuff to the opening daemon, we
split the channel and commit_tx routines into initial_channel and
initial_commit_tx (no HTLC support) and move full HTLC supporting versions
into channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00