Commit Graph

3732 Commits

Author SHA1 Message Date
Rusty Russell 6b2282fc1d wireaddr_internal: new type for where we can also use a local socket.
This was something @icota implemented, but it fits logically into this
cleanup series.  We create a new type which is the internal generalization
of a wireaddr (which is defined by the spec), and add a case here for
a socket name.

Based-on-the-true-story-by: @icota
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell e6c678e5df gossipd: take over address determination, from master.
It does all the other address handling, do this too.  It also proves useful
as we clean up wildcard address handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 356e5dcea8 wireaddr: helpers to convert to/from IPv4/v6 addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 9c0de76019 lightningd: still bind to local ports even if address not public.
Now we only bind to addresses in our wireaddrs array, we would not
autobind to local sockets if they couldn't reach google's nameserver.

That's clearly wrong: we should only not bind if there's a protocol
issue (eg. no IPv6 support).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 00537fde43 lightningd: deprecate --ipaddr in favor of --addr.
We're going to add sockets, and later onion addresses, so the current name
is bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell ed466a8523 lightningd: make explicit listen and reconnect flags.
We set no_reconnect with --offline, but that doesn't work if !DEVELOPER.
Make the flag positive, and non-DEVELOPER mode for gossipd.

We also don't override portnum with --offline, but have an explicit
'listen' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 6ba9a7cb57 tests/test_lightningd.py: move port into node.
We're about to change the JSONRPC, so let's put an explicit 'port' into
our node class.

We initialize it at startup time: in future I hope to use ephemeral ports
to make our tests more easily parallelizable.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 7d27501c94 tests/utils.py: remove unused connect routine.
It's broken anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell 99bba2a23c channeld: don't send ANNOUNCEMENT_SIGNATURES if we've send shutdown.
Our closingd doesn't handle it:

lightningd(2968): 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1:
 Peer permanent failure in CLOSINGD_SIGEXCHANGE: lightning_closingd: sent ERROR Expected closing_signed:
 0103ff54517293892ec3f214f2343c54cbfbf24aa6ffb8d5585d3bc1b543eae0a272000067000001000146390e0c043c777226927eacd2186a03f064e4bdc30f891cb6e4990af49967d34b338755e99d728987e3d49227815e17f3ab40092434a59e33548e870071176d26d19a4e4d8f7715c13ac2d6bf3238608a1ccf9afd91f774d84d170d9edddebf7460c54d49bd6cd81410bc3eeeba2b7278b1b5f7e748d77d793f31086847d582

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 12:48:19 +02:00
Christian Decker 6231e9969f pytest: Give lightningd nodes a numeric ID to prefix logs
This used to be the port, but since we no longer have fixed ports, and we start
them in random order we can't easily distinguish them by the port anymore. Just
use a numeric ID that matches their lightning-dirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 65cac1ba85 pytest: Add flaky dependency and mark as test_closing_different_fees
This is mainly just a stopgap solution until we get to stabilize the test.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 957b32affe contrib: Add the flaky dependency to the builder image
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker e00c7012cc pytest: Ensure unique test directories per test even if rerun
We add an attempt number to the test directory to improve the test-isolation and
allow for multiple reruns of the same test, without re-using any of the
lightning-dirs or bitcoin-datadirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 71857bad8e builder: Add the ephemeral-port-reserve dependency to the builder
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 071ef628db pytest: Use random ports for bitcoind and lightningd to allow parallel testing
Adds a new dependency, but totally worth it :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker bbeb44faac pytest: Make the directory of the NodeFactory configurable
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 727d115296 pytest: Add py.test fixtures and migrate first example test
This is the first example of the py.test style fixtures which should allow us to
write much cleaner and nicer tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker b307df0002 pytest: Let `connect` start nodes in parallel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker e3bac12683 pytest: Allow stdout to be dropped for bitcoind
We never really look at the output, and it's rather noisy, so we just stop
writing to the log.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 4f63acc77c pytest: Add helper to start a number of nodes in parallel
Especially with valgrind this allows us to safe quite some time on multi-core
machines since startup is about the most expensive operation in the tests. In a
simple test spinning up 3 nodes this gave me about 25% - 30% test time
reduction. The effects will be smaller for single core machines, hoping Travis
handles these gracefully.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 81b715aae9 pytest: Move wait_for to utils.py and make fund_channel a member
Slowly moving towards a more pythonic approach to the testing code.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker 7aa13cc949 channel: Queue a channel_update to the peer upon funding_locked
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 01:10:48 +00:00
Christian Decker 9cfd09dc4a gossip: HalfChans are public if we have an update and the Chan is
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 01:10:48 +00:00
Christian Decker b028a363d8 gossip: Make sure we never add a channel twice
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 01:10:48 +00:00
practicalswift 8cc02f63bc gossipd: Handle failed lseek(...) 2018-05-06 20:45:10 +02:00
Christian Decker e308423954 travis: Fix the locale in the builder images
These were spamming the logs and making the log view almost unusable.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-06 15:45:52 +02:00
Christian Decker 700a505727 doc: Remove libsodium-dev dependency, it's in-tree
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Perlover <@Perlover>
2018-05-06 14:07:13 +02:00
Rusty Russell 78f3e8d852 lightningd: Stop the BigTCoin scam!
And it will surprise nobody that bigtcoin.{com,org} are already taken.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Rusty Russell e44f951b6c Makefile: use -modded instead of -with-local-modifications
Since we include the tail of the version in the default aliasname with
DEVELOPER, we want to see more of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Rusty Russell 3bbc708ccc lightningd: --mainnet and --testnet convenience options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Rusty Russell ac51231166 lightningd: remove --dev-hsm-seed option.
We can create the hsm file from python directly; that works even if we
don't have DEVELOPER set, and is simpler.

We add a test that the aliases are correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Rusty Russell 2ecfbf46e3 hsmd: drop newdir logic.
Originally we were supposed to tell the HSM we had just created the directory,
otherwise it wouldn't create a new seed.  But we modified it to check if
there was a seed file anyway: just move that logic into a branch of hsmd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Rusty Russell 666e1b320f lightningd: fix double-specified args.
We need to make sure the arg is a tal object, as we'll free it next time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Christian Decker 6139fee31d fixup! make: Disable DEVELOPER by default 2018-05-04 01:01:41 +02:00
Christian Decker 00e75fee0c make: Disable DEVELOPER by default
We had quite a few users running into issues because the `--dev-xyz` options and
`dev-xyz` RPC calls were available. Before a release we should make sure that
the default compilation flags are safe.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-04 01:01:41 +02:00
ZmnSCPxj b1ffcc3b52 test_lightningd: Update close to use scid, add test for peer id and full cid. 2018-05-03 22:47:07 +02:00
ZmnSCPxj e588737511 peer_control: Have `close` accept channel IDs also. 2018-05-03 22:47:07 +02:00
Mark Beckwith 7767b68ee9 Removed redundancies in withdraw and fundchannel.
No new functionality, just a continuation of my work toward completing #665.

I removed the common members of `struct withdrawal` and `struct fund_channel`
and placed them in a new `struct wallet_tx`.  Then it was fairly straightforward
to reimplement the existing code in terms of `wallet_tx`.

Since I made some structural changes I wanted to get this approved before I
go any farther.

Added 'all' to fundchannel help message.
2018-05-03 18:20:20 +02:00
Saibato 09407b9fd8 fix Litecoin testnet bip173 name
Signed-off-by: Saibato <saibato.naga@protonmail.com>
2018-05-03 14:09:50 +02:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ 333dcbf373 lightningd: Move onchaind replay and gossipd activation after daemonization
Fixes: #1445

Hacky fix, possibly.  First cut at avoiding starting up onchaind and gossipd (which might make queries of chaintopology, which might start up a bitcoin-cli) before we can daemonize.
2018-05-03 12:31:43 +02:00
Rusty Russell c6af2a8cb2 lightningd: loosen feerate minimum.
We're getting spurious closures, even on mainnet.  Using --ignore-fee-limits
is dangerous; it's slightly less so to lower the minimum (which is the
usual cause of problems).

So let's halve it, but beware the floor.

This is a workaround, until we get independent feerates in the spec.

Fixes: #613
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-01 18:45:04 +02:00
practicalswift 5db73c6e27 Avoid static analyzer warnings about potentially uninitialized values 2018-05-01 17:14:33 +02:00
Rusty Russell f083a699e2 gossipd: separate init and activate.
This means gossipd is live and we can tell it things, but it won't
receive incoming connections.  The split also means that the main daemon
continues (eg. loading peers from db) while gossipd is loading from the store,
potentially speeding startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-30 12:01:36 +02:00
Christian Decker 61317859f8 master: Move the gossipd initialization after the other inits
If we start accepting peer connections before we initialized some of the other
parts (mainly the chaintopology) we could end up asking for stuff that isn't
ready yet (blockchain head for example).

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-30 12:01:36 +02:00
Rusty Russell 91d149b990 lightningd: insert db statement checking in io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Rusty Russell 0f0c045aca wallet: fix statement leaks.
We do this before enabling leak checking, but of course I wrote the
commits in the other order!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Rusty Russell ae17c64c4a db: full location tags for callers, make it implicit.
For better leak tracking.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Rusty Russell b036948219 db: track open sqlite3_stmt in DEVELOPER mode.
I would have liked to make it a tal object, then we'd catch most
things with our memleak detection.  However, sqlite3 doesn't seem to
allow allocator overrides.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
practicalswift abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00