Commit Graph

6787 Commits

Author SHA1 Message Date
Christian Decker 3418e59d76 pyln: Split pylightning into multiple pyln modules
This is the first step to transition to a better organized python module
structure. Sadly we can't reuse the `pylightning` module as a namespace module
since having importable things in the top level of the namespace is not
allowed in any of the namespace variants [1], hence we just switch over to the
`pyln` namespace. The code the was under `lightning` will now be reachable
under `pyln.client` and we add the `pyln.proto` module for all the things that
are independent of talking to lightningd and can be used for protocol testing.

[1] https://packaging.python.org/guides/packaging-namespace-packages/

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-30 13:27:37 +02:00
Christian Decker 4e0c3f098c pylightning: Implement the lightning handshake and wire protocol
Simple transcript of the specification in python :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-30 13:27:37 +02:00
Rusty Russell aab9e9f010 gossipd: remove internal dev helpers for queries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 8b3a298ce6 gossipd: generalize query_short_channel_ids to use a callback.
We currently use a flag, but that's inflexible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell c07dff21dc gossipd: generalize query_channel_range to use a callback.
This means we'll be able to call it for internal reasons, not just dev testing
as now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 18aabc3596 pytest: use query_gossip in test_gossip_query_channel_range.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 1386fedfb6 pytest: use query_gossip in test_query_short_channel_id.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell d534a146d2 pytest: clean up test_gossip_timestamp_filter, use query_gossip.
It relies on the fact that nodes don't do their own gossip queries.
Use devtools instead.

This revealed that the entire logic was broken!  It just happened to work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 025b0e6a09 pytest: add helper to send node a gossip query and get response(s).
Note the use of sqrt, which makes a 13 second timeout under Travis
(180 second), or 7 seconds normally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 38524d907b devtools/gossipwith: Add timeout for better use in tests.
Useful for testing you *didn't* send anything else.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 72aa83026a devtools/mkquery: tool to generate gossip query messages.
These helpers would be better autogenerated in python, perhaps as part
of cdecker's pyln package?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell 15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
darosior 274e32925f connectd: add a BOLT10 reference when doing DNS lookup 2019-09-30 01:18:22 +00:00
Saibato 1b9cec8e91 Don't exit w/ INTERNAL_ERROR, if --addr or --bind-addr is used with an .onion address
The Fairy Tail version of .onion option calls on cmdline
reroute the call to --announce-addr or just drop it in case of bind.

Signed-off-by: Saibato <saibato.naga@pm.me>
2019-09-30 00:35:22 +00:00
darosior d0ccd15ac0 plugins: Add a destructor to the plugin request, in case it dies
Authored-by: @rustyrussell
2019-09-30 00:20:16 +00:00
darosior 61c07bca45 doc: Document the changes to the 'plugin' command 2019-09-30 00:20:16 +00:00
darosior 5bdf349771 plugins: make the default plugins directory a member of 'plugins' 2019-09-30 00:20:16 +00:00
darosior 73bbf4f6a3 plugins: cleanup shared headers between dynamic and static plugins 2019-09-30 00:20:16 +00:00
darosior 32d98f0a87 plugins: refactor the tests for dynamic plugins
This adapts the test to the new 'plugin' command: no more sleeping,
since we are synchronous !

This tests the timeout by increasing the 'slowinit' plugin sleep
duration at init reception.

This adds a broken plugin to make sure we won't crash because of a
misbehaving plugin (unmet dependency is the most common case).
2019-09-30 00:20:16 +00:00
darosior 05a747c5ba contrib: Make cowsay answer 'init'
__________________________________
< I was not plugin specs compliant >
 ----------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
2019-09-30 00:20:16 +00:00
darosior 3491a98cca plugins: remove dynamic plugins configuration code from lightningd/plugin
This merges back plugins_init and plugins_start, removes conditions on
startup, and removes the CONFIGURING state.
2019-09-30 00:20:16 +00:00
darosior 7f181edebb plugins: make use of dynamic_plugin_list() in 'plugin list'
And make a function for the 'rescan' subcommand so that we are
consistent and have only subcommand logic in the main function,
which return command_still_pending() in any case (but 'stop').

patched-by: @rustyrussell
2019-09-30 00:20:16 +00:00
darosior b9e40d4cc7 plugins: make 'plugin stop' a function
To keep only subcommands logic into the main
function.
2019-09-30 00:20:16 +00:00
darosior 3a5211048d plugins: make 'plugin startdir' 's runtime path independant
This does the same as for the 'start' subcommand for the 'startdir'
one.

Note that we could fail to start the last plugin of a directory, but
have succesfully started the precedent plugins. This will make us return
an error to the user while some of the plugins have been started, but we
still don't end up in a transient state with
half-configured-half-errored plugins.
2019-09-30 00:20:16 +00:00
darosior edd3ffc7c4 plugins: make 'plugin start' 's runtime path independant
This remove the reliance on startup plugins' function "plugin_start" in
order to use a distinct runtime path for a dynamically started plugin,
which will allow startup plugins' code to be (almost) agnostic of
dynamic plugins.

This also makes the 'start' subcommand return only if the plugin is
either started, or killed : no weird middle state where the plugin
mishbehaving could crash lightningd.
2019-09-30 00:20:16 +00:00
darosior 603e2f0ca9 plugins: split manifest_cb and plugin_config
This reduces error details for getmanifest response,
but avoids too much code duplication in the next commit.
2019-09-30 00:20:16 +00:00
darosior 6694dda8a3 plugins: return also on register error in add_plugin_dir() 2019-09-30 00:20:16 +00:00
darosior fcc358c9bc plugins: return plugin in plugin_register, make conn initializers publics 2019-09-30 00:20:16 +00:00
Rusty Russell 6dca80a375 checkchannels tool.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-29 22:01:50 +02:00
darosior 2c43606895 doc: update mrkd version
The new tag includes @cdecker's modifications for better list handling
2019-09-29 14:06:40 +02:00
darosior a4204226b4 Tidy up parse_wireaddr_from_hostname 2019-09-28 02:49:24 +00:00
Saibato 32a1b782f3 Add lines in regard to enable-autotor-v2-mode flag to doc files
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-09-28 00:31:02 +02:00
Saibato 0f9ba53581 Add enable-autotor-v2 config variable
With enable-autotor-v2 defined in cmdline the default behavior to create
v3 onions with the tor service call, is set to v2 onions.

Signed-off-by: Saibato <saibato.naga@pm.me>
2019-09-28 00:31:02 +02:00
Saibato e638224092 Change default behavior of autotor onions to Version 3 onions. 2019-09-28 00:31:02 +02:00
Rusty Russell 4bf0bc1f28 gossipd: age txout_failures map.
We do this by keeping a current and an old map, and moving the current to old
every hour or 10,000 entries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 02:32:53 +00:00
Rusty Russell 1450a13c1f gossipd: don't expose scids of unannounced channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 02:32:53 +00:00
Rusty Russell 722b4942ed common: rename decode_short_channel_ids.{c,h} to decode_array.{c.h}
This encoding scheme is no longer just used for short_channel_ids, so make
the names more generic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 02:32:53 +00:00
Rusty Russell aa9024db51 gossipd: fix memleak false-positive
2019-09-26T02:00:47.832Z DEBUG lightning_hsmd(89822): Client: Received message 33 from client'
2019-09-26T02:00:47.837Z **BROKEN** lightning_gossipd(89828): MEMLEAK: 0x55eddc5d1fd8
2019-09-26T02:00:47.838Z **BROKEN** lightning_gossipd(89828):   label=gossipd/routing.c:1579:struct unupdated_channel
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):   backtrace:
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     ccan/ccan/tal/tal.c:437 (tal_alloc_)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     gossipd/routing.c:1579 (routing_add_channel_announcement)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     gossipd/routing.c:1867 (handle_pending_cannouncement)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     gossipd/gossipd.c:1543 (handle_txout_reply)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     gossipd/gossipd.c:1726 (recv_req)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     common/daemon_conn.c:31 (handle_read)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     ccan/ccan/io/io.c:59 (next_plan)
2019-09-26T02:00:47.838Z DEBUG lightning_gossipd(89828):     ccan/ccan/io/io.c:407 (do_plan)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 00:01:34 +00:00
Rusty Russell d24c850899 gossipd: restore a flag for fast pruning
I was seeing some accidental pruning under load / Travis, and in
particular we stopped accepting channel_updates because they were 103
seconds old.  But making it too long makes the prune test untenable,
so restore a separate flag that this test can use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 00:01:34 +00:00
Jorge Timón 152f9b66b9 pylightning: Responses may not be iterable 2019-09-26 11:37:53 +02:00
Christian Decker 48728d3828 docker: Use system libraries, don't bring your own
We were compiling them from source, but then not adding them in the final
docker image. This just drops the source based ones in favor of system
provided ones.

Fixes #3074
2019-09-25 22:49:41 +00:00
Christian Decker 07e6f43e1d master: Set our own rlimit for number of fds
We should never open more than 1024 file descriptors anyway, and under some
situations, namely running as root or in docker, would give us huge
allowances. This then results in a huge, unneeded, cleanup for subprocesses,
which we use a lot.

Fixes #2977
2019-09-25 22:49:41 +00:00
lambrosini 6999a4ce2d Remove the python-dev dependency which is useless in python3 2019-09-25 20:37:50 +02:00
lambrosini eb28268bd7 Install in the OS dependencies required to successfully install the python requirements 2019-09-25 20:37:50 +02:00
lambrosini 08573fbc4b Clone the repository before than installing requirements, which are specified in the repository 2019-09-25 20:37:50 +02:00
trueptolemy 12da10cd15 sendpay: Compatible with old parameters `description` 2019-09-25 12:29:53 +02:00
trueptolemy 2d184d464f pylightning: Fix the parameter `label` of `sendpay`
`description` is renamed as `label` in 0.7.0
2019-09-25 12:29:53 +02:00
Rusty Russell 4e8141aa36 features: remove INITIAL_ROUTING_SYNC.
It only had an effect if the peer didn't support option_gossip_queries, but
still, we don't want a gossip blast any more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell 2b47922ea5 gossipd: move query functions into their own file.
The only real change is dump_gossip() used to call
maybe_create_next_scid_reply(), but now I've simply renamed
that to maybe_send_query_responses() and we call it directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell 38124ec287 gossipd: don't ask peers for gossip until we're synced with bitcoind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00