Commit Graph

4261 Commits

Author SHA1 Message Date
Rusty Russell 7b735fbeee gossipd: fix json_listpeers printing node information.
json_listpeers returns an array of peers, and an array of nodes: the latter
is a subset of the former, and is used for printing alias/color information.

This changes it so there is a 1:1 correspondance between the peer information
and nodes, meaning no more O(n^2) search.

If there is no node_announce for a peer, we use a negative timestamp
(already used to indicate that the rest of the gossip_getnodes_entry
is not valid).

Other fixes:
1. Use get_node instead of iterating through the node map.
2. A node without addresses is perfectly valid: we have to use the timestamp
   to see if the alias/color are set.  Previously we wouldn't print that
   if it didn't also advertize an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
William Casarin a9cf73380a cli: launch a manpage when help is passed an argument
example

  lightning-cli help pay
  lightning-cli help cli

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-07 05:01:31 +00:00
Christian Decker 13796f0d0f travis: Add xmllint to the builder image
Seems that it is required for the manpage conversion.
2018-07-05 00:24:33 +00:00
Mark Beckwith fc2d955b01 Fixed spacing in run-params
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith 7d9ad89010 params: removed the param_opt_tok macro
There doesn't seeem to be a need for this anymore (unless I'm missing something).
I added the sendpay_nulltok() unit test to confirm.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith 1b50ea2abd params: removed tal context.
@rustyrussell showed we don't need temporary objects for params.

This means params no longer need a tal context.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Rusty Russell 32ccfa5b29 test/run-params: suppress stderr for expected failures.
And use err() instead of perror/exit(0) (we should have exit(1) there anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 6ff901d7b0 params: simplify lifetimes of params.
@wythe points out we don't need to keep the around now param_is_set()
is removed.  We can in fact go further and avoid marshalling them into
temporary objects at the caller altogether.

This means internally we have an array of struct param, rather than an
array of 'struct param *', which causes most of the noise in this
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 3f6f9e6fe0 param: make sure the name is a string literal.
We're using a macro anyway, so appending "" make it a compile-time check.

Complicates testing a bit, since we actually use generated names there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 899ff02e36 params: use asort.
It's a little neater than qsort here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 9f83a9ae4d params: make optional args do allocation for you.
This is a bit more natural, IMHO.  The only issue is that json_tok_tok is
special, so we end up with param_opt_tok() if you really want an optional
generic token.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Mark Beckwith 4d1d0438e1 Typesafe callback system for parsing json
This is part of #1464 and incorporates Rusty's suggested updates from #1569.

See comment in param.h for description, here's the basics:

	unsigned cltv;
	const jsmntok_t *note;
	u64 msatoshi;
	struct param * mp;

	if (!param_parse(cmd, buffer, tokens,
			 param_req("cltv", json_tok_number, &cltv),
			 param_opt("note", json_tok_tok, &note),
			 mp = param_opt("msatoshi", json_tok_u64, &msatoshi),
			 NULL))
		return;

	if (param_is_set(mp))
		do_something()

There is a lot of developer mode code to make sure we don't make mistakes,
like trying to unmarshal into the same variable twice or adding a required param
after optional.

During testing, I found a bug (of sorts) in the current system.  It allows you
to provide two named parameters with the same name without error; e.g.:

	# cli/lightning-cli -k newaddr addresstype=p2sh-segwit addresstype=bech32
	{
		  "address": "2N3r6fT65PhfhE1mcMS6TtcdaEurud6M7pA"
	}

It just takes the first and ignores the second.  The new system reports this as an
error for now.  We can always change this later.
2018-07-05 00:19:16 +00:00
Rusty Russell fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell 4a1ca0fb99 gossipd: don't use raw secp256k1_pubkey in routing.
We wrap it in 'struct pubkey' for typesafety and consistency, and the
next patch takes advantage of that when we move to pubkey_eq.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Jan Sarenik 9f519afc5d tools/check-includes.sh: shellcheck recommended fixes
$ shellcheck --version
    ShellCheck - shell script analysis tool
    version: 0.5.0
    license: GNU General Public License, version 3
    website: https://www.shellcheck.net
    $ make check-source
    ...

    In tools/check-includes.sh line 14:
        if [[ $(grep -cE "^#((ifndef|define) ${HEADER_ID}|endif /\* ${HEADER_ID} \*/)$" "${HEADER_FILE}") != 3 ]]; then
                                                                 ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".

    In tools/check-includes.sh line 28:
        git ls-files | grep -v 'ccan/' | grep -E "\.${1}"'$'
                                                  ^-- SC1117: Backslash is literal in "\.". Prefer explicit escaping: "\\.".
2018-07-04 01:48:21 +00:00
Christian Decker 582ea1a33b jsonrpc: Remove `dev-blockheight` in favor of `getinfo`
`getinfo` has been providing the blockheight for a good while and doesn't
require the `DEVELOPER=1` flag during compilation, so it should be the preferred
method to retrieve the blockchain height.
2018-07-04 00:08:14 +00:00
William Casarin 4a1b68502e wallet: fix up semi-cryptic wallet error message
There is evidence that this message can be confusing[1].
Suggest a way to resolve the error in the message.

[1] https://www.reddit.com/r/Bitcoin/comments/7ybcq9/programming_how_can_i_resolve_the_below_error/

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-03 23:58:55 +00:00
Sjors Provoost f7dabd4fd8 Armbian build instructions 2018-07-03 11:49:55 +02:00
Jan Sarenik cb53ad0706 wallet/wallet.c: bind_null if (payment_key) 2018-07-02 05:00:08 +00:00
Jan Sarenik da80447ba3 wallet/wallet.com: bind_null if (out->preimage) 2018-07-02 05:00:08 +00:00
Jan Sarenik 5205de7b8e wallet/wallet.c: bind_null if (in->preimage) 2018-07-02 05:00:08 +00:00
Jan Sarenik 580b8ba8a9 wallet/wallet.c: bind_null if (chan->remote_shutdown_scriptpubkey) 2018-07-02 05:00:08 +00:00
Jan Sarenik 2090ddd1fa wallet/wallet.c: bind_null if (chan->scid) 2018-07-02 05:00:08 +00:00
Jan Sarenik 0640ac23cb wallet/wallet.c: Add missing bind_null
In order to address #338
2018-07-02 05:00:08 +00:00
Christian Decker fe405f49be bitcoind: Smooth fee changes over a number of estimates
Implements an EWMA for the fee estimation. Achieves 90% influence of the newer
fee after 5 minutes, and adjusts to the polling rate that is configured.
2018-07-02 01:41:42 +00:00
William Casarin a88ab1634f cli: add manpage hint at the end of human help output
It may not immediately obvious that this is available

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-02 00:09:59 +02:00
Rusty Russell 32af9d1e19 Travis: re-enable BOLT text check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 17:37:03 +02:00
Rusty Russell 82ff891202 Update to latest BOLT version.
And remove the FIXMEs now that the gossip_query extension is merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 17:37:03 +02:00
Rusty Russell c46f373205 options: refuse two --announce-addr of the same type.
Gossipd will ignore the second one, but doing it in the front end
gives an explicit error message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell f67182ff20 gossipd: order node_announcement addresses correctly, remove duplicate types.
Fixes: #1596
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell e6f31b3c66 pytest: allow duplicated options.
Python dict can't have duplicate entries, but some options can be specified
multiple times.  The easiest way is to put a list in the dict.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell 284f0a04c9 gossipd: don't announce bound address if given with --bind-addr, even if public.
Only --addr implies announce-if-public: --bind-addr does not.

It's also possible to have --bind-addr to an automatic Tor address:
you'd have to dig the onion address out of the logs or getinfo to use
it, but it's possible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell 01c02fd617 devtools/decodemsg: decode encoded_short_ids.
$ devtools/decodemsg 010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f000000000000ffff0100160178da6360486760606400824c285d00a60111710144
$ devtools/decodemsg 010506226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00110000006700000100000000690000010000

Before:
    WIRE_REPLY_CHANNEL_RANGE:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    first_blocknum=0
    number_of_blocks=65535
    complete=1
    encoded_short_ids=[0178da6360486760606400824c285d00a60111710144]
    WIRE_QUERY_SHORT_CHANNEL_IDS:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    encoded_short_ids=[0000006700000100000000690000010000]

After:

    WIRE_REPLY_CHANNEL_RANGE:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    first_blocknum=0
    number_of_blocks=65535
    complete=1
    encoded_short_ids=[ (ZLIB) 103:1:0 105:1:0 112:1:0 ]
    WIRE_QUERY_SHORT_CHANNEL_IDS:
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    encoded_short_ids=[ (UNCOMPRESSED) 103:1:0 105:1:0 ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell 9d3ce87700 decode_short_ids: move to common.
We want to use it in devtools/decodemsg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell 0bd82a8138 devtools/decodemsg: decode node_announcement.addresses
$ ./devtools/decodemsg 01014bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e97fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab1300005b3315de0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c035180266e4e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000004d010102030404d202000000000000000000000000000000002607039216a8b803f3acd758aa260704e00533f3e8f2aedaa8969b3d0fa03a96e857bbb28064dca5e147e934244b9ba50230032607

Before:
    WIRE_NODE_ANNOUNCEMENT:
    signature=304402204bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e902207fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab13
    features=[]
    timestamp=1530074590
    node_id=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
    rgb_color=[0266e4]
    alias=[ナンセンス 1杯 e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000 ]
    addresses=[010102030404d202000000000000000000000000000000002607039216a8b803f3acd758aa260704e00533f3e8f2aedaa8969b3d0fa03a96e857bbb28064dca5e147e934244b9ba50230032607]

After:
    WIRE_NODE_ANNOUNCEMENT:
    signature=304402204bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e902207fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab13
    features=[]
    timestamp=1530074590
    node_id=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
    rgb_color=[0266e4]
    alias=[ナンセンス 1杯 e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000 ]
    addresses=[ 1.2.3.4:1234 [::]:9735 silkroad6ownowfk.onion:9735 4acth47i6kxnvkewtm6q7ib2s3ufpo5sqbsnzjpbi7utijcltosqemad.onion:9735 ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell bf4dc09910 devtools/decodemsg: decode node_announcement.alias
$ ./devtools/decodemsg 01014bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e97fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab1300005b3315de0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c035180266e4e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000004d010102030404d202000000000000000000000000000000002607039216a8b803f3acd758aa260704e00533f3e8f2aedaa8969b3d0fa03a96e857bbb28064dca5e147e934244b9ba50230032607

Before:
    WIRE_NODE_ANNOUNCEMENT:
    signature=304402204bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e902207fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab13
    features=[]
    timestamp=1530074590
    node_id=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
    rgb_color=[\x02f\xe4]
    alias=[\xe3\x83\x8a\xe3\x83\xb3\xe3\x82\xbb\xe3\x83\xb3\xe3\x82\xb9 1\xe6\x9d\xaf\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00]
    addresses=[\x01\x01\x02\x03\x04\x04\xd2\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00&\x07\x03\x92\x16\xa8\xb8\x03\xf3\xac\xd7X\xaa&\x07\x04\xe0\x053\xf3\xe8\xf2\xae\xda\xa8\x96\x9b=\x0f\xa0:\x96\xe8W\xbb\xb2\x80d\xdc\xa5\xe1G\xe94$K\x9b\xa5\x020\x03&\x07]

After:
    WIRE_NODE_ANNOUNCEMENT:
    signature=304402204bfa4585cb36194ce7c308e8d3d9032ff4e42ed4764a4c6d0a9a58da0a4e57e902207fbd463577a5fd14347c60cc7bef2b40bd644337153564320b310b4d155cab13
    features=[]
    timestamp=1530074590
    node_id=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
    rgb_color=[0266e4]
    alias=[ナンセンス 1杯 e3838ae383b3e382bbe383b3e382b92031e69daf000000000000000000000000 ]
    addresses=[010102030404d202000000000000000000000000000000002607039216a8b803f3acd758aa260704e00533f3e8f2aedaa8969b3d0fa03a96e857bbb28064dca5e147e934244b9ba50230032607]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell c02ff11506 print_wire: hand field names to print routines.
This lets us override how we print them.

Also, add dependency on header for devtools/Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Rusty Russell 2639b1e9a9 ccan: add UTF-8 module for checking alias fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
arowser fecef618bc remove duplicate line 2018-07-01 04:14:10 +00:00
Christian Decker 6c82d1b8b0 pylightning: Subclass ValueError to access to the returned error 2018-07-01 04:13:04 +00:00
Christian Decker e35da01d11 travis: Add asciidoc, excluding latex, to the builder images
Fixes #1641
2018-07-01 04:12:45 +00:00
arowser 2eab1b66ff add alias and color to getinfo 2018-06-30 08:24:50 +00:00
arowser 25f60f9456 remove unused return value 2018-06-30 04:27:34 +00:00
nicolas.dorier aeafe4dbe7 [Docker] Fix: Bind socat to the right port 2018-06-27 13:40:02 +02:00
Corné Plooy 50efe34e3b ncc: we need to set variables like CC in a different way, to avoid them being overwritten by configure-specified values. 2018-06-26 10:34:17 +00:00
nicolas.dorier e59489f49b Add configure and .gitmodules to .gitattributes 2018-06-24 18:50:02 +02:00
Christian Decker 0d3c3a225e docker: Add a usable dockerfile
This is based on @NicolasDorier's excellent proposal for a Dockerfile, sans the
writing of a config file.

Co-authored-by: Nicolas Dorier <nicolas.dorier@gmail.com>
Co-authored-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 23:51:57 +00:00
Christian Decker 1899e000d6 docker: Add two more dependencies to the fedora builder
Strictly speaking not needed for the build and test, but if we want to use the
image also for packaging this is required.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 23:04:10 +00:00
Christian Decker 2c56019b46 make: Specify the commit hash length for the bin-tarball name
Different versions of git have different default lengths. This uses a fixed
length hash prefix.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 23:04:10 +00:00
Rusty Russell 9071b853b6 Makefile: add bin-tarball rule to produce clightning-<version>-<distro>.tar.xz
Places everything under opt/clightning, as per LSB.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 14:02:31 +02:00