Commit Graph

6350 Commits

Author SHA1 Message Date
Rusty Russell af0200f9d0 fixup! doc: fix up documentation about when we move into lightning-dir. 2019-08-08 18:17:12 +08:00
Rusty Russell bcff30bb3f fixup! doc: fix up documentation about when we move into lightning-dir. 2019-08-08 18:17:12 +08:00
Rusty Russell 202ab91234 doc: fix up documentation about when we move into lightning-dir.
And make sure that plugins know that they should not touch things
until their init call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:17:12 +08:00
Rusty Russell 118150227e cli: restore 0.7.0-style whitespace printing.
@renepickhardt has a shell script we broke.  While we still produce
perfectly valid JSON, we should not gratuitously change tool output.

Plus, I prefer the missing space before the ':'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:16:48 +08:00
Rusty Russell 2f30cdc731 contrib: simple script to bootstrap node.
The helpme plugin is more comprensive, but this at least connects to a
few random nodes, and doesn't require python libraries in path or anything.

I selected the nodes from helpme.py, eliminating ones I couldn't reach.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-07 21:13:17 +08:00
Rusty Russell b73a85a75e lightningd: don't say 'killing channel' when HTLC times out.
We're actually only killing the connection.  I saw this in my logs,
but it was all OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-07 21:12:52 +08:00
darosior cd91c06ce9 lightningd/notification: Add missing includes for 'forward_event'
And update test mocks
2019-08-07 01:55:38 +00:00
Christian Decker a85d3dc220 pytest: Make test_funding_cancel_race SLOW_MACHINE compatible
This test is spawning 100 nodes concurrently, which is a lot even when not
running with `valgrind`, especially when executing tests in parallel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-07 01:54:43 +00:00
darosior 5fbb15bd59 Document the 'dev' command 2019-08-07 01:50:42 +00:00
darosior ed63496152 Pylightning: update to use the 'dev' multiplex command
Update 'dev_crash()' and 'dev_rhash()' methods which used outdated RPC commands, add the missing 'dev_slowcmd()' method.
2019-08-07 01:50:42 +00:00
darosior f3f33dceb1 lightningd/jsonrpc: Remove unused dev-rhash command code
'dev-rhash' is now part of the 'dev' multiplex command
2019-08-07 01:50:42 +00:00
Christian Decker 820b52207e lightningd: Defer creating the PID until we actually want to start
This was causing `--help` to fail if we already had a `lightningd` running
with the same `--lightning-dir`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
Christian Decker e2f9a410f4 pytest: Run plugin --help tests in the test directory
This is a followup to #2892. Since we now attempt to lock the PID file before
starting plugins we need to make sure that we actually use a unique lightning
directory for anything that attempts to call `--help`. If not we may be
conflicting with a `lightningd` that is running against that directory.

Notice that this still means that we will be unable to call `--help` on
`lightningd` if we have a running instance, but isolation in this case is
good, otherwise we'd be reading the default config anyway.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
darosior a2c00d42f2 Remove the 'signal_startup' member of the plugin struct
It's set but not used
2019-08-05 23:06:55 -05:00
Rusty Russell 596972366d wire: always ignore unknown odd messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-05 09:54:30 +00:00
Rusty Russell e78a80495b log: make --log-file an early arg (since we move to dir early now).
Otherwise we bleed plugin log messages to stderr, as they're initted before
log-file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-05 17:09:57 +08:00
Christian Decker 9d4148ce68 pylightning: Warn users of plugins that may break due to extra args
We recently noticed that the way we unpack the call arguments for hooks and
notifications in pylightning breaks pretty quickly once you start changing the
hook and notification params. If you add params they will not get mapped
correctly causing the plugin to error out.

This can be fixed by adding a `VAR_KEYWORD` argument to the calbacks, i.e., by
adding a single `**kwargs` argument at the end of the signature. This commit
adds a check that such a catch-all argument exists, and emits a warning if it
doesn't.

It also fixes up the plugins that we ship ourselves.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-05 00:19:48 +00:00
Rusty Russell e808aaa1bb lightningd: clean up pidfile test, crashlog.
1. Now checking the pid file really does precede touching the db and
   starting plugins, which is far safer.
2. Crashlog is now activated just after daemon parent release, and just
   before the main loop, which means no "crash" on startup if we call fatal().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell 979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell fc024f81d6 pytest: check that --daemon still means we recognize our own children.
We create our children then fork, so we're not a parent.  I noticed this
because 'lightning-cli stop' takes a long time: this is because it tries to
wait for them and they don't respond.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Christian Decker 8ed77753ef json-rpc: Add size and cumulative size to dev-memdump
Since we are walking the entire allocation tree anyway, and access the tal
metadata anyway, we can just as well also track the size of the memory
allocations to simplify debugging of memory use.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-04 20:54:53 +02:00
darosior 0cd3823c98 libplugin: pass configuration to plugin's 'init' callback
So that a plugin can, for example, know if it has been loaded on startup
2019-08-03 13:15:40 +02:00
darosior 017ee7c916 libplugin: Add 'getmanifest' 'dynamic' field 2019-08-03 13:15:40 +02:00
Rusty Russell f57f068592 gossipd: don't use O_APPEND on the gossip_store.
We always know the length, so we don't need it.  It causes much extra work
when we want to delete a record, which I suspect may cause issues amongst
some users who've been seeing gossip_store corruption.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 12:50:51 +02:00
Rusty Russell 4c9bfa351a lightningd: handle --version before trying to move to lightning-dir.
Otherwise it creates the lightning-dir.  This can't be helped for --help
(at least, if plugins are present), but --version simply prints and exits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell fd63b8bf53 lightningd: chdir as soon as we know lightning dir.
This is easy since we did the option parsing cleanup, but it has the
effect that plugins are launched from the lightning-dir.  Now
we have dynamic plugins, this means startup and post-startup plugins
experience the same environment.

This is absolutely a desirable thing: they can just drop files in
their cwd rather than having to move (including, I might note, core
files!).

We also highlight the change in various places (and a drive-up update
of PLUGINS.md which says you have to use --plugin).

The next patch adds a backwards compatibility wedge for old users of
relative plugin paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell 4fcd94a091 pytest: use absolute paths for plugin arguments.
We're going to change this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell c24573b0d2 Makefile: set PYTHONPATH to be absolute when running tests.
My test machine started failing on dynamic plugin tests, unable to
find the lightning module:

  ModuleNotFoundError: No module named 'lightning'

This is because plugins at startup are run from whatever directory
you're in, but on refresh are run from the lightning-dir.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
lisa neigut c7f3fa34b2 funding tx: include segwit marker + flag in fee calculation
Noticed an off by one error when running tests for dual-funding;
we're not including the two 'header' segwit bytes in our weight
calculations.
2019-08-03 05:19:24 +00:00
lisa neigut b0b6ddb66f devtools: rm repetitive field (it's included in remotesecrets) 2019-08-03 05:19:24 +00:00
lisa neigut abcde461df devtools: print witnesses along with signatures for mkfunding
we need the witnesses for dual-funding transactions
2019-08-03 05:19:24 +00:00
Rusty Russell 913a1a9b59 bolt: update to 8b2cf0054660bece9e1004f42a500c6a1a77efd3
This contains only typo fixes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 8774070a31 bolt: Update to latest bolt, including TLV onion format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell fafd5ed4b0 wire/test/run-tlvstream.c: use generated defs for test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 4b5cd5ff84 tools/generate-wire.py: only use 'struct node_id' in announcements.
Naturally, it's a struct pubkey.  However, those are large, and take
time to marshal, so gossipd treats them as node_id which is a simple
array.  It adds explicit checks at the right points to make sure
they're valid pubkeys.

However, the next commit adds TLV test vectors, which assumes we treat
node_id as a point (thus catch invalid values when parsing).  The best
solution is to restrain our types here to exactly those we've
optimized for.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 00ca52ea07 bolt: update to aa33af0c4d7ae0180c04ef98e61af49c1f876a36
This introduces the TLV test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell b17b4c3eb6 wire: include definition of struct node_id.
Next BOLT update puts it in a TLV, so we need the definition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell b10e0e08bb tools/generate-wire.py: add option to expose tlv_record_type
Next update adds TLV test vectors: without this, we get a warning
about them being unused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 9b88fd4c60 bolt: update to 950b2f5481c2a4b57ef1102e2374543e81c4aa88
Just a simple field renaming which only alters comments,
though I updated variable names too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 2b3003f25b channeld: delay sending channel_announcement by 60 seconds.
We currently send channel_announcement as soon as we and our
peer agree it's 6 blocks deep.  In theory, our other peers might
not have seen that block yet though, so delay a little.

This is mitigated by two factors:
1. lnd will stash any "not ready yet" channel_announcements anyway.
2. c-lightning doesn't enforce the 6 depth minimum at all.

We should not rely on other nodes' generosity or laxity, however!

Next release, we can start enforcing the depth limit, and maybe stashing
ones which don't quite make it (or simply enforce depth 5, not 6).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 16:50:45 +02:00
ZmnSCPxj 3e74ca4b86 gossipd/routing.c: Correctly handle a duplicated entry in `exclude` of `getroute`. 2019-08-02 16:06:15 +02:00
ZmnSCPxj a5fb37298c tests/test_gossip.py: Add test to check that duplicated exclusions in `getroute` have no lasting effect. 2019-08-02 16:06:15 +02:00
Rusty Russell 359433f374 lightningd: convert the compiler-wanted-init FIXME.
I'm sure there are others, but this was the only one which showed up
in grep.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 79345cc0b9 CCAN: update to suppress path_readlink warning.
Reported-by: Chirimen-Jako
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 4f89d9be86 cppcheck: catch json_list_for_each
cppcheck on bionic doesn't like this it seems.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 1510ea5717 wallet/walletrpc.c: fix uninitialized warning.
The withdraw_tx function shouldn't use it, but GCC is right it's uninitialized:

wallet/walletrpc.c: In function ‘json_prepare_tx’:
wallet/walletrpc.c:202:15: error: ‘changekey’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 296cfe8d1b wire/tlvstream: suppress gcc -O3 warning about prev_type.
Use a pointer, so it's explicit and gcc is happy.  We avoid the
allocation by pointing it to another stack var.

./wire/tlvstream.c:81:22: error: ‘prev_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 02609773c0 lightningd: suppress gcc-7.4.0 error
In file included from wallet/test/run-wallet.c:15:0:
./lightningd/peer_htlcs.c: In function ‘htlcs_reconnect’:
./lightningd/peer_htlcs.c:2060:15: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   }  else if (failcode) {
               ^~~~~~~~
./lightningd/peer_htlcs.c:2056:19: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
          failcode != 0
          ~~~~~~~~~^~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 7835a25121 common: helper to suppress 'may be used uninitialized' warnings.
When gcc still warns at -O3 and you are sure it's not necessary,
this marks it appropriately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00