As per https://github.com/lightningnetwork/lightning-rfc/pull/785
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: config: the default CLTV expiry is now 34 blocks, and final expiry 18 blocks as per new BOLT recommendations.
Changelog-Added: New option `--important-plugin` loads a plugin is so important that if it dies, `lightningd` will exit rather than continue. You can still `--disable-plugin` it, however, which trumps `--important-plugin` and it will not be started at all.
Reduces VALGRIND=1 node_factory.line_graph(5) time on my laptop from 42s to 36s.
This is simply because forking all the subdaemons just to check the
version is very expensive under valgrind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We did not take the value of --commit-fee into account : this removes
the unused option from lightningd and instead registers it in bcli,
where we set the actual feerate of commitment transactions. This also
corrects the documentation.
Changelog-Fixed: config: we now take the --commit-fee parameter into account.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
The previous implementation was a bit lazy: in particular, since we didn't
remember the disabled plugins, we would load them on rescan.
Changelog-Changed: config: the `plugin-disable` option works even if specified before the plugin is found.
This will let us unify the startup and runtime-started infrastructure.
Note that there are two kinds of notifications:
1. Starting a single plugin (i.e. `plugin start`)
2. Starting multiple plugins (i.e. `plugin rescan` or `plugin startdir`).
In the latter case, we want the command to complete only once *all*
the plugins are dead/finished.
We also call plugin_kill() in all cases, and correctly return afterwards
(it matters once we use the same paths for dynamic plugins, which don't
cause a fatal error if they don't startup).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's almost always "their_features" and "our_features" respectively, so
make those names clear.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is to prepare for dynamic features, including making plugins first
class citizens at setting them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: lightningd: Added --subdaemon command to allow alternate subdaemons.
[ Wow, that was mammoth; 44 comments over 12 commits. Feels almost unfair to squash it into one commit, so I wanted to note @ksedgwic's perseverence here! --RR ]
We could use sendonion to do this, but it actually takes a different path through
pay, and I wanted to test all of it, so I made a new dev flag.
We currently get upset with the response:
lightningd/pay.c:556: payment_failed: Assertion `!hout->failcode' failed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
json_listconfigs() returns in the middle; the name0 is not always freed.
It will be freed later with the response, but our memleak detection doesn't
know that, and Travis caught it:
Global errors:
E - Node /tmp/ltests-5mfrzh5v/test_hsmtool_secret_decryption_1/lightning-1/ has memory leaks: [
E {
E "backtrace": [
E "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E "ccan/ccan/tal/tal.c:466 (tal_alloc_arr_)",
E "ccan/ccan/tal/tal.c:794 (tal_dup_)",
E "ccan/ccan/tal/str/str.c:32 (tal_strndup_)",
E "lightningd/options.c:1122 (add_config)",
E "lightningd/options.c:1282 (json_listconfigs)",
E "lightningd/jsonrpc.c:588 (command_exec)",
E "lightningd/jsonrpc.c:679 (rpc_command_hook_callback)",
E "lightningd/plugin_hook.c:123 (plugin_hook_call_)",
E "lightningd/jsonrpc.c:729 (plugin_hook_call_rpc_command)",
E "lightningd/jsonrpc.c:736 (call_rpc_command_hook)",
E "common/timeout.c:39 (timer_expired)",
E "lightningd/io_loop_with_timers.c:32 (io_loop_with_timers)",
E "lightningd/lightningd.c:871 (main)"
E ],
E "label": "lightningd/options.c:1122:char[]",
E "parents": [
E "lightningd/json_stream.c:49:struct json_stream",
E "ccan/ccan/io/io.c:91:struct io_conn",
E "lightningd/lightningd.c:104:struct lightningd"
E ],
E "value": "0x5569ada057a8"
E }
E ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We want to have a static Tor service created from a blob bound to
our node on cmdline
Changelog-added: persistent Tor address support
Changelog-added: allow the Tor inbound service port differ from 9735
Signed-off-by: Saibato <saibato.naga@pm.me>
Add base64 encode/decode to common
We need this to encode the blob for the tor service
Signed-off-by: Saibato <saibato.naga@pm.me>
cppcheck found this:
[lightningd/options.c:1137] -> [lightningd/options.c:1120] -> [lightningd/options.c:1193]: (error) Using pointer to local variable 'buf' that is out of scope.
Indeed, answer can point into buf, which is no longer in scope at the end.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were using sleeps to hope we catch the password prompt. This makes the test
flaky. So I added a help text followed by a `fflush` to make sure we catcht he
right moment, instead of guessing. The `fflush` is also useful for debugging
if a user ever pipes the output to a file it'd get buffered and the user would
wait forever. The same applies for automated systems such as `expect` or
`pexpect` based scripts that enter the password on prompt.
Do the same thing '--help' does with them; append `...`.
Valgrind noticed that we weren't NUL-terminarting if answer was over
78 characters.
Changelog-Fixed: JSONRPC: listconfigs appends '...' to truncated config options.
They're already qualified with network name, and there's little point
moving them; it might even be dangerous if multiple are running.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. "conf" can't be specified in a configuration file.
2. "lightning-dir" can't be specified in a configuration file unless the file
was explicitly set with --conf=.
3. "network" options can't be set in a per-network configuration file.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This lets you have a default, but also a network-specific config.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: `config` and <network>/`config` read by default.
lightning-cli is going to need to know what network we're on, so
it will need to parse the config files. Move the code which does
the initial bootstrap parsing into common, as well as the config
file parsing core.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows finegrained logging control of particular subdaemons or
subsystems.
To do this, we defer setting the logging levels for each log object
until after early argument parsing (since e.g. "bitcoind" log object
is created early).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: log-level can now specify different levels for different subsystems.
--dev-force-tmp-channel-id flag takes a 64-character hex string
to use as the temporary channel id. Useful for spec tests
[ Fixed crash in non-DEVELOPER mode --RR ]
Changelog-None
Add a new startup option which will, if set, prompt the user for a
password to derive a key from. This key will later be used to encrypt
and/or decrypt `hsm_secret`.
This was made a noarg option even if it would have been preferable to
let the user the choice of how to specify the password. Since we have
to chose, better to not let the password in the commands history.
Make --announce-addr with autotor: also
a meaningful use case.
The option --announce-addr=autotor: is more
intuitive than to use the --addr=autotor: option
Signed-off-by: Saibato <saibato.naga@pm.me>
Declare opt_add_addr at top of option.c
We we use opt_add_addr and opt_announce_addr vice versa.
To make compiler happy, we declare it at top.
Signed-off-by: Saibato <saibato.naga@pm.me>
We now have a pointer to chainparams, that fails valgrind if we do anything
chain-specific before setting it.
Suggested-by: Rusty Russell <@rustyrussell>
Using a global variable is a bit lazy, but weaving the network type through
the entire stack is a daunting task. Maybe we can make that happen at a later
stage.
Most of the changes in `chainparams.c` are just formatting the
`genesis_blockhash` a bit nicer (`clang-format` to the rescue).
Signed-off-by: Christian Decker <decker.christian@gmail.com>
The Fairy Tail version of .onion option calls on cmdline
reroute the call to --announce-addr or just drop it in case of bind.
Signed-off-by: Saibato <saibato.naga@pm.me>
With enable-autotor-v2 defined in cmdline the default behavior to create
v3 onions with the tor service call, is set to v2 onions.
Signed-off-by: Saibato <saibato.naga@pm.me>
I was seeing some accidental pruning under load / Travis, and in
particular we stopped accepting channel_updates because they were 103
seconds old. But making it too long makes the prune test untenable,
so restore a separate flag that this test can use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Will be demuxed into starting the selected DB backend in one of the next
commits. Defaults to the old database location.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.
Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.
Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The simplest case is to explicitly load it when we see it's been
set.
This involves neatening the default config setup, to remove it from
opt_parse_from_config() and into the caller. It also seems we don't
need to call it anymore before parsing early options: none of them
need ld->config set.
Closes: #3030
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Dumb programs which have a --daemon option call fork() early. This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.
However, forking late introduced a heap of problems with changing
pids. Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us. We release our parent
with an explicit action just before the main loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Otherwise it creates the lightning-dir. This can't be helped for --help
(at least, if plugins are present), but --version simply prints and exits.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is easy since we did the option parsing cleanup, but it has the
effect that plugins are launched from the lightning-dir. Now
we have dynamic plugins, this means startup and post-startup plugins
experience the same environment.
This is absolutely a desirable thing: they can just drop files in
their cwd rather than having to move (including, I might note, core
files!).
We also highlight the change in various places (and a drive-up update
of PLUGINS.md which says you have to use --plugin).
The next patch adds a backwards compatibility wedge for old users of
relative plugin paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No code changes, just move.
Put all the dev options into the one function, and register (and
comment on) the early args first.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I noticed that --network=regtest didn't override 'network=bitcoin' in
the config file.
Normally we parse the config file first, then the commandline (so the cmdline
wins). But for early options, we do cmdline first so we can find the config
file. That was fine when the only early option was the location of the
config file, but now it includes plugins and the network setting.
So do a boutique cmdline parse *just* to find the config file, then parse
the config file early options, then the cmdline early options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.
this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
"result" should always be an object (so that we can add new fields),
so make that implicit in json_stream_success.
This makes our primitives well-formed: we previously used NULL as our
fieldname when calling the first json_object_start, which is a hack
since we're actually in an object and the fieldname is 'result' (which
was already written by json_object_start).
There were only two cases which didn't do this:
1. dev-memdump returned an array. No API guarantees on this.
2. shutdown returned a string.
I temporarily made shutdown return an empty object, which shouldn't
break anything, but I want to fix that later anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These are generalized from our internal implementations.
The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.
The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A new string field is added to the command structure and is specified at the creation of each native command, and in the JSON created by 'json_add_help_command()'.
We now have a test blockchain for MCP which has the correct channels,
so this is not needed.
Also fix a benchmark script bug where 'mv "$DIR"/log
"$DIR"/log.old.$$' would fail if you log didn't exist from a previous run.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The old value of 1000 sat was too small to cover the dust reserves.
This lead to the situation when trying to open a channel with minimal
amount, the channels got refused because they were not able cover the
commitment fees.
For this reason the minimal capacity should be increased to i.e. 10k
satoshi, as the technical minimum that also accounts for fees and
reserves is somewhere around 6k sat.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.
We didn't get much space saving in gossipd, even though we should save
24 bytes per node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Pubkeys are not not actually DER encoding, but Pieter Wuille corrected
me: it's SEC 1 documented encoding.
Results from 5 runs, min-max(mean +/- stddev):
store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
38922-39297(39180.6+/-1.3e+02),2880728,41.040000-41.160000(41.106+/-0.05),2.270000-2.530000(2.338+/-0.097),44.570000-53.980000(49.696+/-3),32.840000-33.080000(32.95+/-0.095),43.060000-44.950000(43.696+/-0.72)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- add config value min_capacity_sat that will replaces the magic value
min_effective_htlc_capacity = AMOUNT_MSAT(1000000)
- add config switch min_capacity_sat
This lets us benchmark without a valid blockchain.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup!_gossipd__dev_option_to_allow_unknown_channels.patch':
fixup! gossipd: dev option to allow unknown channels.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This patch will properly set fee_per_satoshi to _unsigned_ integer,
as support for negative fees was removed from overall design.
This change does not break any tests, so I assume its
better this way.
Christian and I both unwittingly used it in form:
*tal_arr_expand(&x) = tal(x, ...)
Since '=' isn't a sequence point, the compiler can (and does!) cache
the value of x, handing it to tal *after* tal_arr_expand() moves it
due to tal_resize().
The new version is somewhat less convenient to use, but doesn't have
this problem, since the assignment is always evaluated after the
resize.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we are planning to release a bug fix release, and the plugin
subsystem is not yet complete, it is better to make plugin support
opt-in while we continue testing.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Usually, this means they return 'command_param_failed()' if param()
fails, and changing 'command_success(); return;' to 'return
command_success()'.
Occasionally, it's more complex: there's a command_its_complicated()
for the case where we can't exactly determine what the status is,
but it should be considered a last resort.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Handers of a specific form are both designed to be used as callbacks
for param(), and also dispose of the command if something goes wrong.
Make them return the 'struct command_result *' from command_failed(),
or NULL.
Renaming them just makes sense: json_tok_XXX is used for non-command-freeing
parsers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These are only supposed to be used when you want the token contents including
surrounding "". We should use this when reporting errors, but usually
we just want to access the tok members directly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
json_escaped.[ch], param.[ch] and jsonrpc_errors.h move from lightningd/
to common/. Tests moved too.
We add a new 'common/json_tok.[ch]' for the common parameter parsing
routines which a plugin might want, taking them out of
lightningd/json.c (which now only contains the lightningd-specific
ones).
The rest is mainly fixing up includes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This (will) avoid the plugin having to walk back from the params object
as it currently does.
No code changes; I removed UNUSED and UNNEEDED labels from the other
parameters though (as *every* json_rpc callback needs to call param()
these days, they're *always* used).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This currently just invokes GDB, but we could generalize it (though
pdb doesn't allow attaching to a running process, other python
debuggers seem to).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We also make `--help` a non-early arg so it allows for the plugins to
register their options before printing the help message. The options
themselves are stored in a separate struct inbetween them being
registered and them being forwarded to the plugin. Currently only
supports string options.
Signed-off-by: Christian Decker <@cdecker>
Rusty Russell
ZmnSCPxj jxPCSnmZ
Christian Decker
Antoine Poinsot
lisa neigut
Ken Sedgwick
nicolas.dorier
Vasil Dimov
Saibato
gorazdko
Rene Pickhardt
Simon Vrouwe
Karl-Johan Alm
Michael Schmoock
Richard Bondi
arowser