Commit Graph

6290 Commits

Author SHA1 Message Date
darosior f3f33dceb1 lightningd/jsonrpc: Remove unused dev-rhash command code
'dev-rhash' is now part of the 'dev' multiplex command
2019-08-07 01:50:42 +00:00
Christian Decker 820b52207e lightningd: Defer creating the PID until we actually want to start
This was causing `--help` to fail if we already had a `lightningd` running
with the same `--lightning-dir`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
Christian Decker e2f9a410f4 pytest: Run plugin --help tests in the test directory
This is a followup to #2892. Since we now attempt to lock the PID file before
starting plugins we need to make sure that we actually use a unique lightning
directory for anything that attempts to call `--help`. If not we may be
conflicting with a `lightningd` that is running against that directory.

Notice that this still means that we will be unable to call `--help` on
`lightningd` if we have a running instance, but isolation in this case is
good, otherwise we'd be reading the default config anyway.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
darosior a2c00d42f2 Remove the 'signal_startup' member of the plugin struct
It's set but not used
2019-08-05 23:06:55 -05:00
Rusty Russell 596972366d wire: always ignore unknown odd messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-05 09:54:30 +00:00
Rusty Russell e78a80495b log: make --log-file an early arg (since we move to dir early now).
Otherwise we bleed plugin log messages to stderr, as they're initted before
log-file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-05 17:09:57 +08:00
Christian Decker 9d4148ce68 pylightning: Warn users of plugins that may break due to extra args
We recently noticed that the way we unpack the call arguments for hooks and
notifications in pylightning breaks pretty quickly once you start changing the
hook and notification params. If you add params they will not get mapped
correctly causing the plugin to error out.

This can be fixed by adding a `VAR_KEYWORD` argument to the calbacks, i.e., by
adding a single `**kwargs` argument at the end of the signature. This commit
adds a check that such a catch-all argument exists, and emits a warning if it
doesn't.

It also fixes up the plugins that we ship ourselves.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-05 00:19:48 +00:00
Rusty Russell e808aaa1bb lightningd: clean up pidfile test, crashlog.
1. Now checking the pid file really does precede touching the db and
   starting plugins, which is far safer.
2. Crashlog is now activated just after daemon parent release, and just
   before the main loop, which means no "crash" on startup if we call fatal().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell 979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell fc024f81d6 pytest: check that --daemon still means we recognize our own children.
We create our children then fork, so we're not a parent.  I noticed this
because 'lightning-cli stop' takes a long time: this is because it tries to
wait for them and they don't respond.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Christian Decker 8ed77753ef json-rpc: Add size and cumulative size to dev-memdump
Since we are walking the entire allocation tree anyway, and access the tal
metadata anyway, we can just as well also track the size of the memory
allocations to simplify debugging of memory use.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-04 20:54:53 +02:00
darosior 0cd3823c98 libplugin: pass configuration to plugin's 'init' callback
So that a plugin can, for example, know if it has been loaded on startup
2019-08-03 13:15:40 +02:00
darosior 017ee7c916 libplugin: Add 'getmanifest' 'dynamic' field 2019-08-03 13:15:40 +02:00
Rusty Russell f57f068592 gossipd: don't use O_APPEND on the gossip_store.
We always know the length, so we don't need it.  It causes much extra work
when we want to delete a record, which I suspect may cause issues amongst
some users who've been seeing gossip_store corruption.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 12:50:51 +02:00
Rusty Russell 4c9bfa351a lightningd: handle --version before trying to move to lightning-dir.
Otherwise it creates the lightning-dir.  This can't be helped for --help
(at least, if plugins are present), but --version simply prints and exits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell fd63b8bf53 lightningd: chdir as soon as we know lightning dir.
This is easy since we did the option parsing cleanup, but it has the
effect that plugins are launched from the lightning-dir.  Now
we have dynamic plugins, this means startup and post-startup plugins
experience the same environment.

This is absolutely a desirable thing: they can just drop files in
their cwd rather than having to move (including, I might note, core
files!).

We also highlight the change in various places (and a drive-up update
of PLUGINS.md which says you have to use --plugin).

The next patch adds a backwards compatibility wedge for old users of
relative plugin paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell 4fcd94a091 pytest: use absolute paths for plugin arguments.
We're going to change this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell c24573b0d2 Makefile: set PYTHONPATH to be absolute when running tests.
My test machine started failing on dynamic plugin tests, unable to
find the lightning module:

  ModuleNotFoundError: No module named 'lightning'

This is because plugins at startup are run from whatever directory
you're in, but on refresh are run from the lightning-dir.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
lisa neigut c7f3fa34b2 funding tx: include segwit marker + flag in fee calculation
Noticed an off by one error when running tests for dual-funding;
we're not including the two 'header' segwit bytes in our weight
calculations.
2019-08-03 05:19:24 +00:00
lisa neigut b0b6ddb66f devtools: rm repetitive field (it's included in remotesecrets) 2019-08-03 05:19:24 +00:00
lisa neigut abcde461df devtools: print witnesses along with signatures for mkfunding
we need the witnesses for dual-funding transactions
2019-08-03 05:19:24 +00:00
Rusty Russell 913a1a9b59 bolt: update to 8b2cf0054660bece9e1004f42a500c6a1a77efd3
This contains only typo fixes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 8774070a31 bolt: Update to latest bolt, including TLV onion format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell fafd5ed4b0 wire/test/run-tlvstream.c: use generated defs for test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 4b5cd5ff84 tools/generate-wire.py: only use 'struct node_id' in announcements.
Naturally, it's a struct pubkey.  However, those are large, and take
time to marshal, so gossipd treats them as node_id which is a simple
array.  It adds explicit checks at the right points to make sure
they're valid pubkeys.

However, the next commit adds TLV test vectors, which assumes we treat
node_id as a point (thus catch invalid values when parsing).  The best
solution is to restrain our types here to exactly those we've
optimized for.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 00ca52ea07 bolt: update to aa33af0c4d7ae0180c04ef98e61af49c1f876a36
This introduces the TLV test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell b17b4c3eb6 wire: include definition of struct node_id.
Next BOLT update puts it in a TLV, so we need the definition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell b10e0e08bb tools/generate-wire.py: add option to expose tlv_record_type
Next update adds TLV test vectors: without this, we get a warning
about them being unused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 9b88fd4c60 bolt: update to 950b2f5481c2a4b57ef1102e2374543e81c4aa88
Just a simple field renaming which only alters comments,
though I updated variable names too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 2b3003f25b channeld: delay sending channel_announcement by 60 seconds.
We currently send channel_announcement as soon as we and our
peer agree it's 6 blocks deep.  In theory, our other peers might
not have seen that block yet though, so delay a little.

This is mitigated by two factors:
1. lnd will stash any "not ready yet" channel_announcements anyway.
2. c-lightning doesn't enforce the 6 depth minimum at all.

We should not rely on other nodes' generosity or laxity, however!

Next release, we can start enforcing the depth limit, and maybe stashing
ones which don't quite make it (or simply enforce depth 5, not 6).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 16:50:45 +02:00
ZmnSCPxj 3e74ca4b86 gossipd/routing.c: Correctly handle a duplicated entry in `exclude` of `getroute`. 2019-08-02 16:06:15 +02:00
ZmnSCPxj a5fb37298c tests/test_gossip.py: Add test to check that duplicated exclusions in `getroute` have no lasting effect. 2019-08-02 16:06:15 +02:00
Rusty Russell 359433f374 lightningd: convert the compiler-wanted-init FIXME.
I'm sure there are others, but this was the only one which showed up
in grep.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 79345cc0b9 CCAN: update to suppress path_readlink warning.
Reported-by: Chirimen-Jako
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 4f89d9be86 cppcheck: catch json_list_for_each
cppcheck on bionic doesn't like this it seems.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 1510ea5717 wallet/walletrpc.c: fix uninitialized warning.
The withdraw_tx function shouldn't use it, but GCC is right it's uninitialized:

wallet/walletrpc.c: In function ‘json_prepare_tx’:
wallet/walletrpc.c:202:15: error: ‘changekey’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 296cfe8d1b wire/tlvstream: suppress gcc -O3 warning about prev_type.
Use a pointer, so it's explicit and gcc is happy.  We avoid the
allocation by pointing it to another stack var.

./wire/tlvstream.c:81:22: error: ‘prev_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 02609773c0 lightningd: suppress gcc-7.4.0 error
In file included from wallet/test/run-wallet.c:15:0:
./lightningd/peer_htlcs.c: In function ‘htlcs_reconnect’:
./lightningd/peer_htlcs.c:2060:15: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   }  else if (failcode) {
               ^~~~~~~~
./lightningd/peer_htlcs.c:2056:19: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
          failcode != 0
          ~~~~~~~~~^~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 7835a25121 common: helper to suppress 'may be used uninitialized' warnings.
When gcc still warns at -O3 and you are sure it's not necessary,
this marks it appropriately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell df298ba7e3 configure: suppress "maybe uninitialized" warnings altogether unless -O3.
We were just telling GCC not to treat them as errors: this suppresses them
entirely unless at -O3.  People keep trying to "fix" them, when in fact
they're false positives, as revealed with "./configure COPTFLAGS=-O3".

Fixes: #2856
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 5a3ba1ce99 travis: don't run full testsuite with gcc-4.8
Build test (non-developer) is probably enough.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 6a2a118467 travis: upgrade to bionic
This means we'll start enforcing no "maybe uninitialized" warnings at
-O3, since xenial was using gcc 5.4 or gcc 4.8 which are too primitive.

Seems like `sudo: false` is deprecated (those deps weren't being
installed); you simply install in the `before_install` hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 641876a20a ccan: update to latest.
configurator failed under clang:

checking for #pragma omp and -fopenmp support... ccan/tools/configurator/configurator: Test for HAVE_OPENMP failed with 32512:
./configurator.out: error while loading shared libraries: libomp.so: cannot open shared object file: No such file or directory

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
trueptolemy 5c153de81e CHANGELOG: Add the record for `forward_event` 2019-08-01 18:49:25 +08:00
trueptolemy 7b59431a6c doc: Add the description for 'forward_event' notification 2019-08-01 18:49:25 +08:00
trueptolemy 31e2e70f17 invoice: a cleanup for the json of struct sha256
Here should't be accessed directly to the underlying of struct sha256.
2019-08-01 18:49:25 +08:00
trueptolemy 5ac02907bb pytest: Add the test for 'forward_event' in test_plugin 2019-08-01 18:49:25 +08:00
trueptolemy d2423c6f9a pytest: Add a simple plugin to test 'forward_event' 2019-08-01 18:49:25 +08:00
trueptolemy 294f05dc98 Call notification 'forward_event' when forward payment status changes 2019-08-01 18:49:25 +08:00