Commit Graph

5862 Commits

Author SHA1 Message Date
Christian Decker 9a4cf7dda7 pytest: Mark test_option_upfront_shutdown_script as flaky
Signed-off-by: Christian Decker <@cdecker>
2019-06-04 00:27:15 +00:00
Christian Decker 80b3d8aef3 json: Add some more methods to add integers of various size to json
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Rusty Russell 763697eb4c gossipd: fix gossip_store calling delete.
Now we handle node_announcements properly, we have a failure case where we
try to move them when a channel is deleted while loading the store.

We're going to remove this soon, in favor of in-place delete, so
workaround this for now to avoid an assert() when we try to write to
the store while loading.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell 21fe518513 gossip_store: fix 'bad node_announcement' by allowing node_announcement on un-updated channel.
When we first receive a channel_update, we write both the
channel_announcement and that channel_update to the store: we need
that first update so we can set the channel_announcement timestamp.

However, the channel_update can be replaced later.  This means we can
have a channel_announcement, a node_update which relies on it, then
the channel_update later.

So move the "this applies to a pending announcement" check lower, where
gossip_store can use it too.  Has a nice side-effect of avoiding
one lookup of the node id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell 048a650a6b pytest: more comprehensive tests for test_gossip_store_compact.
First, we should have a channel_update so we actually do some compaction!
(Reported-by @SimonVrouwe).  But we should also handle the cases where:

1. A channel_announcement is *not* directly followed by a
   channel_update (happens when the channel_update is replaced).
2. A node_announcement predates a channel_update for the peer
   (again, can happen once a channel_update is replaced).
3. A local/private channel_creation is not directly followed by an
   update.

In addition, we might as well check that we can *load* such a store,
before compaction.

This checks the corner cases which occur in real gossip stores.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell 1147e65602 pytest: make test_gossip_notices_close more reliable.
It's possible that it hasn't got the node_announcement messages;
it will still list the nodes, however (the channel_announcement tells
it the nodes exist).  Check for the alias field instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell a40fe40ad2 pytest: catch more gossip-related errors.
Basically, any "Bad" message from gossipd is something we should look
at.  This covers failures loading the gossip_store, too!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Michael Schmoock 45645e3a85 test: fix flakyness test_no_fee_estimate 2019-06-03 19:30:19 +02:00
Michael Schmoock 107381aca2 test: fix flakyness of funding_reorg_private openchannel_hook 2019-06-03 19:30:19 +02:00
Rusty Russell 7961a815aa travis: add compilation to SOURCE_CHECK_ONLY.
Our "-O3" CI check wasn't doing what I thought.  But building with -flto
is hard with older gccs, so remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell 7092d215fa configure: detect modern gcc, and only suppress necessary errors.
Drive-by fix of CONFIGURATOR_CC setting when CC is overridden on cmdline
(not via env var).

Note that clang defines __GNUC__ (to 4) :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell ed08ca6df3 configure: handle --reconfigure better.
`./configure CC=newcc --reconfigure` didn't set CC, because reconfigure
simply replaced all values: only make it replace undefined values.

Also, it didn't change COPTFLAGS or CWARNFLAGS, even if they were previously
the defaults (eg. --reconfigure --enable-developer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell 5b7776cf13 wire/test: remove unused padding code.
Turns out the peer part of the spec no longer uses padding (it's used only
in the onion), and GCC-9 with -O3 warns we're padding NULL to memcpy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell c233fc5063 gossipd: fix spurious unused error with gcc-9 -O3.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell 9820d0664b connectd: handle IPv6 correctly.
Untested, but the current code is clearly wrong (caught by gcc-4.8 -O3).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell c091a4ee40 gossipd: fix spurious gcc warning.
It turns out that we don't look at type when we return 0, but gcc isn't
quite smart enough for that.  Initializing to -1 is good practice anyway
for the failure path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell fae593c448 lightningd: fix uninitialized variable in DEVELOPER path
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
William Casarin 6f635b46fd gossipstore: fix uninitialized input fd
Initialize infd to STDIN_FILENO if the input file argument is missing.

Caught with gcc version: 7.4.0

devtools/create-gossipstore.c: In function ‘main’:
devtools/create-gossipstore.c:130:9: error: ‘infd’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
  while (read_all(infd, &be_inlen, sizeof(be_inlen))) {

Suggested-by: @ZmnSCPxj <https://github.com/ElementsProject/lightning/pull/2674#issuecomment-495617253>
Signed-off-by: William Casarin <jb55@jb55.com>
2019-06-03 00:07:11 +00:00
William Casarin 3f035cb3cc gossipd: fix uninitialized free on short_route in goto path
Fix a path where tal_free is called on an uninitialized variable

If the first `goto bad_total` executes, then that path has
uninitialized `short_route` but bad_total passes through to `out`
whose first call is tal_free(short_route).

This was noticed by a maybe-uninitialized heuristic on gcc 7.4.0:

gossipd/routing.c: In function ‘find_shorter_route’:
gossipd/routing.c:1096:2: error: ‘short_route’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
  tal_free(short_route);

Reported-by: @ZmnSCPxj <https://github.com/ElementsProject/lightning/pull/2674#issuecomment-495617253>
Signed-off-by: William Casarin <jb55@jb55.com>
2019-06-03 00:07:11 +00:00
darosior 4640d18ef0 Plugins: allow to specify plugin option type in libplugin.c, then specify it to autoclean options 2019-06-03 00:06:12 +00:00
darosior 3775a32659 Pylightning: allow to specify the option type at its creation 2019-06-03 00:06:12 +00:00
darosior a6753fd812 Plugins: accept options of different types
A new struct containing the plugin options value as different types is created and appended to the plugin_option structure
2019-06-03 00:06:12 +00:00
darosior fba050e891 CHANGELOG: Add some informations about the new category field in commands 2019-06-03 00:02:25 +00:00
darosior 78326a8d90 Pylightning: Allow a plugin to set the category of the RPC commands it creates 2019-06-03 00:02:25 +00:00
darosior b1bbafb19c cli: Sort human_readable output by categories
A new field is available in the command json object : 'category' (corresponding to whether "bitcoin", "channel", "network", "payment", "plugin", "utility", "developer"). We use it to printf commands ordered by categories.

credits @rustyrussel
2019-06-03 00:02:25 +00:00
darosior 9d34121ec5 libplugin.c: Handle command categories then set them in pay and autoclean 2019-06-03 00:02:25 +00:00
darosior 5b0bf0ba1f jsonrpc: allow to set the command category plugin side 2019-06-03 00:02:25 +00:00
darosior 323adb467a jsonrpc: Add a category field to commands.
A new string field is added to the command structure and is specified at the creation of each native command, and in the JSON created by 'json_add_help_command()'.
2019-06-03 00:02:25 +00:00
Rusty Russell 0c189fe3a7 devtools/decode-iolog: tool to decode hexstrings from io logging.
Slow, but useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-31 18:36:38 +02:00
Rusty Russell 695bec531c plugin: fix printing of bad plugin responses.
Before:
	Plugin for invoice_payment returned non-result response 

	"subscriptions": [], "hooks": ["invoice_payment"]}}

	�V

After:
	Plugin for invoice_payment returned non-result response {"jsonrpc": "2.0", "id": 6, "error": "Error while processing invoice_payment: ValueError(\"invalid literal for int() with base 10: '5.0'\")"}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-31 18:36:38 +02:00
Rusty Russell 8f9c48254b plugins: do I/O logging.
I was trying to trace a problem with a plugin, and needed this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-31 18:36:38 +02:00
ZmnSCPxj 48df6c8566 lightningd/io_loop_with_timers.c: Move mainloop to its own source file, have chaintopology use it.
Fixes: #2687
2019-05-31 17:57:10 +02:00
Christian Decker 883d4b4e6e pytest: Add a test that reproduces #2687
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-05-31 17:57:10 +02:00
ZmnSCPxj 3466261b20 lightningd/channel.c: Transfer peer to connectd only if connectd alive.
Fixes: #2677
2019-05-31 15:01:58 +02:00
ZmnSCPxj 37440e9447 lightningd/subd.c: Return NULL from subd_shutdown.
And set pointers to shut down daemons as NULL in lightningd.
2019-05-31 15:01:58 +02:00
trueptolemy 11412435ce Channeld: init channel with remote announcement info when restart
1. Add remote_ann_node_sigs and remote_bitcoin_sigs fields in channel_init message;
2. Master add announcement signatures into channel_init message, and send this message to Channeld.
Channeld will initial the channel with this signatures when it reenables the channel.
2019-05-29 11:46:33 +02:00
trueptolemy d93f61407a Channeld: Add new wire type:channel_got_announcement
Channeld sends announcement signatures to Master by this message.
When Channeld receive a new channel announcement msg, (After channel locking)it will sends announcement signatures to Master by this message.
2019-05-29 11:46:33 +02:00
trueptolemy a645fbdecd run-wallet: Add the check of case that channel received ann_sigs 2019-05-29 11:46:33 +02:00
trueptolemy b41d529b28 DB: Store the remote channel announcement signatures into DB
1. Add the fields of remote channel announcement signatures into TABLE channels
2. Add the related function
2019-05-29 11:46:33 +02:00
Rene Pickhardt 22600faed7 manually added a list-forward manpage because I have no idea how a2x works 2019-05-27 17:35:06 +02:00
Simon Vrouwe 52c84b2387 pytest: add test_funding_reorg_remote_lags
Nodes may disagree about short_channel_id before channel
announcement.
2019-05-27 13:29:32 +02:00
Simon Vrouwe 5b0c174251 pytest: add test_funding_reorg_private, reorg changes funding height
Reorg changes short_channel_id after lockin of private channel, while
one node restarts.

test that:
- peer->depth_togo in billboard decrements
- reorg and scid change is detected by running node and restarting node
- both `old` and `new` scids are in rtable

Also added a comment to test_blockchaintrack to clarify.
2019-05-27 13:29:32 +02:00
Simon Vrouwe 54a67c2444 channeld: rephrase depth_togo line in billboard_update 2019-05-27 13:29:32 +02:00
Simon Vrouwe 8c1bbf33e5 lightningd: funding_lockin_cb, handle reorgs that change short_channel_id
Keep watching and updating scid until ANNOUNCE_MIN_DEPTH, even when channel is private.
When scid changes, we fail channeld so it will restart and initialize with updated
scid and add it to rtable. Reorgs can change funding tx's height/index after lockin,
which could happen with small minimum_depth=1.
2019-05-27 13:29:32 +02:00
Simon Vrouwe 7726681aa6 pytest: added BitcoinD.simple_reorg() method, which can change tx height and/or txindex
Now without bitcoind restart.
bitcoin-cli `prioritisetransaction` came to the rescue!
Its argument `fee_delta` (apparently) lowers the txs _effective_ feerate
soo low that bitcoind wont mine it ... untill we raise it when we want
it to be mined.
2019-05-27 13:29:32 +02:00
Simon Vrouwe 24f43e5910 pytest: fundwallet tighten up, make sure we see `CONFIRMED` in log line 2019-05-27 13:29:32 +02:00
Simon Vrouwe 245807a085 pytest: tighten a log test `Owning output...` with address type and confirmation
Because the call (wallet_extract_owned outputs) that prints that line can happen
_before_ or _after_ confirmation in block, adding `CONFIRMED` in the later.
2019-05-27 13:29:32 +02:00
Simon Vrouwe eb3495c23d chaintopology: add log line when we remove stale block from topo->tip
Added comments
2019-05-27 13:29:32 +02:00
Simon Vrouwe b261c5ba0b chaintopology: add REORG to logline when tx watch is fired after reorg
This makes clear that txwatch_fire was called with depth=0 when
chain tip got removed after a reorg. And now we can test for that.
2019-05-27 13:29:32 +02:00
Simon Vrouwe db57d9c5d2 lightningd: suppress IO_OUT logging for getlog command
Before this, the response of `getlog io` blew up quickly
when called multiple times.
2019-05-26 23:53:20 +00:00