Commit Graph

708 Commits

Author SHA1 Message Date
Christian Decker 91b17d45d8 sphinx: Removing last vestiges of the end-to-end payload
So far this was simply set to a zero-length end-to-end payload. We
don't have any plans of re-adding it for the moment, so let's get rid
of the unused code.
2017-01-16 11:14:15 +10:30
Christian Decker 285b8b4698 sphinx: Use libsecp256k1 to generate shared secrets
So far we did it on our own, but since the spec specifies that we use
the libsecp256k1 version anyway, we can remove our own implementation.
2017-01-16 11:08:36 +10:30
Christian Decker 679dec3e6a sphinx: Actually use the full pubkey in ECDH key generation
The spec says that we use the libsecp256k1 style ECDH, which uses the
full compressed pubkey from the scalar multiplication which is then
hashed. This is in contrast to the btcsuite implementation which was
only using the hashed X-coordinate.
2017-01-16 11:08:36 +10:30
Rusty Russell 6bf3c30a10 lightningd/Makefile: fix check-source
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-13 10:51:57 +10:30
Rusty Russell 73d07ce441 libsodium: use our local submodule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-11 10:04:26 +10:30
Rusty Russell e39d09d5a5 daemon/test/scripts: maek slightly more general for lightningd/lightningd
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell 015eb072b8 Makefile: split CORE_SRC into CORE_SRC, CORE_TX_SRC and CORE_PROTOBUF_SRC
With the lightningd daemon split, we don't need them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell a0ac5c276e status: API for status reporting.
The API formalizes how daemons should report their statuses back to
the main lightningd.  It's a simple write API, which includes tracing
support (currently it always sends traces, later it could send iff
there's a failure, for example).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:24:20 +10:30
Rusty Russell 42f474af18 wire: make lengths of variable fields implied by tal_count()
This is a much nicer interface, and works better in practice too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell 7b1a4fc765 utils: add tal_hex() helper.
This is a shortcut when the data being dumped is a tal array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell 4a233090ae log: rename struct log_record to struct log_book.
I think "log entry" when I see "log record", so this name is better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:18:26 +10:30
Rusty Russell 350d8edb7e log: don't include dstate any more.
Before we had a global secp256k1_ctx we needed to hold this to print
out pubkeys, now it's completely orthogonal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:17:51 +10:30
Christian Decker af43cc5e2c makefile: Be more verbose in `check-daemon-headers`
So far it was failing silently, now it diffs the Makefile state
against the directory listing. This also fixes a bug when the locale
was not set the sort order would not match.
2017-01-10 09:26:06 +10:30
Rusty Russell b59fe5a2b6 daemon/Makefile: objects depend on wire headers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-06 13:23:50 +10:30
Christian Decker 96af89139e base58: Unittests need in-tree libbase58 2017-01-05 12:12:30 +10:30
Lucas Betschart 3e6bb958eb Add libbase58 as git submodule
Pointing to the latest release (0.1.4)
2017-01-05 12:11:18 +10:30
Rusty Russell bdc4972df6 wire/gen_peer_wire_csv: update to latest spec #2
828eda61df5a7be27051c605f7808e4f690739e4, in particular, it has the
new address format for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell d3bdb073b5 wire/gen_peer_wire_csv: update to latest spec #1
In particular, add features bitfields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell bdaa22e247 generate-wire.py: allow NULL len parameter from fromwire_*
It implies tal_count() gives the length. Great for almost all callers which
don't care if there are extra bytes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell e076d56709 generate-wire.py: include type bytes in towire/fromwire routines.
This removes some redundancy in creating messages, but also allows
a lazy form or parsing without explicitly checking the type.

A helper fromwire_peektype() is added to look up the type and handle
the too-short-for-type problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell c864b28068 generate-wire.py: don't generate structures, hand in all values.
This is a bit more awkward for large structures, but avoids
indirection for the simpler ones (I copied the structures for the test
code, however).  We also remove explicit padding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell a08a2105ea generate-wire.py: generalize, move to tools.
We're going to want to use this for inter-daemon comms, so generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell b7789bf065 Makefile: generalize whitespace check.
Spread to individual Makefiles, and include headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 0861ec33a5 check-source-bolt: generalize.
This way sub-Makefiles can add their own files to check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell c2cc164d6d daemon: disable old BOLT checks.
This is useful for the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 05feefbb8a test: move mockup script outside daemon/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell d30b67a3fe test: generalize update-mocks
So we can use it in other directories.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell dec3b9d030 peer: don't ever set up listener on dynamic port.
Simplifies the logic somewhat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:15 +10:30
Rusty Russell 12b30ab4fc jsonrpc: register base on linkage.
Other than being neater (no more global list to edit!), this lets the
new daemon and old daemon have their own separate routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:08:15 +10:30
Rusty Russell c7b69abdaa type_to_string: move formatting to appropriate files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:07:15 +10:30
Rusty Russell f8eb454139 type_to_string: move pretty printing of types from log
It's not just useful for logging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:05:15 +10:30
Rusty Russell 756b2a0530 options: --help and --version are early args.
If they use these, we definitely don't want to create the config dir...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:04:15 +10:30
Rusty Russell 93471e06c1 options: return true if we created new directory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:03:15 +10:30
Rusty Russell 038ef0250a options: move option and config code into its own file.
For the moment, the new lightningd will share all this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 13:22:29 +10:30
Rusty Russell ca4d03aa2a lightningd_state: move invoices into separate structure.
It's all maintained by invoice.c, so make it private to that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 13:18:47 +10:30
Christian Decker 570003235e gossip: Fixing minor issues
Thanks @rustyrussell for the review ^^
2017-01-03 15:08:05 +10:30
Christian Decker 95bc8f6987 gossip: Removing origin from staggered broadcast
Keeping a pointer to the peer that initially sent us a message
could (actually will!) result in dangling pointers. Removing this
results in some additional messages, which will be discarded by the
recipient, so that should not be a problem.
2017-01-03 15:08:05 +10:30
Christian Decker 0ed23c46a9 gossip: Implemented routing table sync with newly connected peers 2017-01-03 15:08:05 +10:30
Christian Decker 5ce1f7c2f3 gossip: Set announce timeout to 5 hours 2017-01-03 15:08:05 +10:30
Christian Decker 71a18fdd2e gossip: Trigger announce on channel establishment
Since we ultimately want to have a very long re-announcement timeout
we better trigger an announcement upon channel state change to normal.
2017-01-03 15:08:05 +10:30
Christian Decker e54c0adced gossip: Do not consider half-open connections for routes
Connections are in a half-open state after receiving the
`channel_announcement` and before the `channel_update` makes them
usable, so we need to ignore channels that are not yet fully open.
2017-01-03 15:08:05 +10:30
Christian Decker 611f4833c5 gossip: Implemented the staggered broadcast 2017-01-03 15:08:05 +10:30
Christian Decker fca9bf1a16 gossip: Partially sign channel_announcements
We can already add our own signatures, will have to defer remote
signatures until we implement the channel setup messages from the spec.
2017-01-03 15:08:05 +10:30
Christian Decker d93ce12afa gossip: Broadcasting and handling gossip messages 2017-01-03 15:08:05 +10:30
Christian Decker e8d68757f6 gossip: Tracking tx index in topology
The txlocator was reporting the index in the purged array of
transactions instead of the index in the original block.
2017-01-03 15:08:05 +10:30
Christian Decker 57d5ae96a4 routing: Added channel_id and directions to connections
The gossip protocol spec refers to channels by their `channel_id` and
a direction. Furthermore, inbetween the `channel_announcement` and the
`channel_update` for either direction, the channel direction is in an
undefined state and cannot be used, so added the `half_add_connection`
function and an `active` flag to differentiate usable connections from
unusable ones.
2017-01-03 15:08:05 +10:30
Christian Decker db481d881a proto: Added handling for nested packets 2017-01-03 15:08:05 +10:30
Rusty Russell a44e4fb631 daemon/json, test/test_protocol: avoid gcc -O warnings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:49:51 +10:30
Rusty Russell e109977df7 peer: don't print uninitialized commit_num.
Reported-by: Lucas Betschart <lucasbetschart@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:44:48 +10:30
Rusty Russell 2a68a984c3 peer: fix unassigned error packet on receiving non-ASCII error.
Reported-by: Lucas Betschart <lucasbetschart@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:42:24 +10:30
Rusty Russell 1b170c85bb Revert "Fix -Wsometimes-uninitialized and -Wuninitialized"
This reverts commit ff9cdda4fa.

These are real bugs, so I prefer to fix them explicitly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:41:31 +10:30
Lucas Betschart 900ab78d46 Fix -Wtautological-constant-out-of-range-compare 2016-12-14 08:47:21 +01:00
Lucas Betschart ff9cdda4fa Fix -Wsometimes-uninitialized and -Wuninitialized 2016-12-14 08:45:40 +01:00
Lucas Betschart c163fe04ac Fix -Wenum-conversion 2016-12-14 08:43:48 +01:00
Lucas Betschart 33d6cbe6bc Fix -Wgnu-variable-sized-type-not-at-end 2016-12-14 08:41:30 +01:00
Rusty Russell 45ec071e58 lightningd: remove whitespace failing make check-source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-05 14:03:06 +10:30
Rusty Russell 36c8fc7ef8 lightningd: remove secpctx
Use the global in the few remaining places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:13:27 +10:30
Rusty Russell a4fdaab5b3 Use global secp256k1_ctx instead of passing it around.
If I'd known how large this patch would be (though trivial), I'd
have done it in parts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:12:58 +10:30
Rusty Russell c938ebb5c0 utils: add a global secp, fix wire to use it.
This repairs make check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:11:06 +10:30
Rusty Russell 1f447688bc test: make generic driver for unit tests.
Not just in daemon.  This fixes 'make check' when it tries to
run 'make wire-tests'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:10:13 +10:30
Christian Decker e40509c6c3 wire: Linking against wire objs 2016-12-01 13:50:02 +10:30
Rusty Russell 0ba93cb948 lightningd: don't listen at all if no port number set.
When we support the Milan protocol, we'll use a default port.  But
for now, don't listen at all unless a port is specified.

Fixes: #54
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 17:13:29 +10:30
Rusty Russell fe14b3f4d9 Merge remote-tracking branch 'origin/pr/92'
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 17:05:42 +10:30
Rusty Russell 97a155bad5 Merge remote-tracking branch 'origin/pr/97'
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 14:28:22 +10:30
Christian Decker 42a33df42e irc: Checking for param count before accessing 2016-11-16 22:55:29 +01:00
Christian Decker 188b3c3e19
sphinx: Fixed a buffer overflow in hmac generation
Our HMACs are truncated to 20 byte, but sodium still generates 32 byte
HMACs and we were handing in a buffer that was too small, so we
overflowing the buffer by 12 bytes. This manifested itself only in the
32 bit variant because of different alignment in the 64bit version.

Fixes #94.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2016-11-16 14:41:19 +01:00
Christian Decker b2a3e8010f Merge pull request #96 from ElementsProject/minor-sphinx-cleanups
Minor sphinx cleanups
2016-11-16 14:35:19 +01:00
Christian Decker bf81f7af15 Merge pull request #93 from ElementsProject/fix-87-open-too-small-for-fee
json_connect: don't try to double-fail if amount insufficient.
2016-11-16 13:04:26 +01:00
Rusty Russell 41299b679c sphinx: process_onionpacket doesn't modify the packet.
Make it const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:22:14 +10:30
Rusty Russell 423572190b sphinx: use assignment instead of memcpy where possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:22:08 +10:30
Rusty Russell 6f9dedbe7f sphinx: remove redundant initialization
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:04:50 +10:30
Rusty Russell 8599d63256 sphinx: add brackets around constant definition
Always do this, otherwise "MACRO * foo" can have unexpected results.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:02:56 +10:30
Rusty Russell 5465b61784 json: fix 32-bit build.
Assume we have strtoull everywhere (it's C99 and was common before).

Fixes: #91
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 13:34:56 +10:30
Christian Decker 686ff6edef irc: Switched to LFNet.org IRC server
This is a temporary fix until we finish #88.
2016-11-14 23:19:32 +01:00
Rusty Russell 94adbd241d json_connect: don't try to double-fail if amount insufficient.
Closes: #87
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-14 11:46:09 +10:30
Christian Decker 34b4134cb0 bugfix: Removed spurious printf in peer.c
I must've left that on in. Fixes #89
2016-11-14 00:29:01 +01:00
Christian Decker e5b44ff232 Merge pull request #85 from ElementsProject/whitespace
Whitespace
2016-11-11 16:40:23 +01:00
Christian Decker 9848b4ac3e travis-ci: Added travis-ci config
Added .travis.yml to get travis-ci to build and run tests for us.

In addition this fixes a flaky test due to the fact that when lightning2
connects to lightning3 and we tell lightning3 to restart, then
lightning2 will back-off its reconnection attempts, potentially causing
a timeout to trigger during tests. This was triggered by travis-ci
relatively consistently since the restart would take quite some
time. Now simply restarting them in reverse order and a small timeout
seems to fix this consistently.
2016-11-11 14:25:41 +01:00
Rusty Russell f5c00deec7 Remove trailing whitespace from source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-11 09:32:04 +10:30
Rusty Russell 716da21f85 chaintopology: reduce how far back we start on testnet.
When initially reading the blockchain, we start 100 back from the
current block, or at the first block with a funding transaction,
whichever is earlier.

This slows testing slightly, so use whatever the "forever" value is
(10 on testnet, still 100 on mainnet).

make check -j12 times:
	Before:	7m52.005s
	After:  6m31.896s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:53:48 +10:30
Rusty Russell ce96812ef0 test.sh: split into multiple scripts.
This significantly reduces re-testing of the same paths, and simplifies
the addition of new tests.

make check -j12 times:
	Before: 9m24.973s
	After:	7m52.005s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:44:49 +10:30
Rusty Russell fae7f68731 helpers.sh: make check_tx_spend able to check for a specific transaction.
It currently takes an unused "who to blame" argument, which doesn't
always make sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:34:35 +10:30
Rusty Russell c6a187a5d7 helpers.sh: set up funding in start_lightningd()
Every test wants this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:33:35 +10:30
Rusty Russell 9e777ae922 test.sh: move core routines to scripts/helpers.sh
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:32:35 +10:30
Rusty Russell 973ef3c932 test.sh: move valgrind PREFIX setting to vars.sh
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:31:35 +10:30
Rusty Russell 700c536a27 Makefile: remove all the old Elements Alpha support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:30:35 +10:30
Rusty Russell bff81c0ecf lightningd: ignore SIGPIPE.
It can happen when a peer hangs up, but also a JSON connection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:29:35 +10:30
Rusty Russell e7e7345596 peer: fix steal (penalty tx) generation code.
In particular, we got a segv because we were measuring the wrong
wscript, then we miswired the inputs.  It only worked because our
current steal tests don't have a to_us_idx output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:28:35 +10:30
Rusty Russell 3372645d8e peer_disconnect: simply free if in STATE_INIT.
db_forget_peer() was harmless, but we haven't been entered into the
database yet anyway, and it asserted that we should have been STATE_CLOSED.

Closes: #67
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 11:21:16 +10:30
Rusty Russell c2d71497ff opt, utils: new arg for wrappers due to low-level tal change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:56:38 +10:30
Rusty Russell 9708c7a019 timers: use time_mono, as that's what ccan/timers now wants.
Fixes: #58
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:56:15 +10:30
Rusty Russell 4bed6c8c67 controlled_time: remove
We don't need it for testing at the moment, and if we do it'll have
to change to relative anyway now we're going to use time_mono().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:54:15 +10:30
Rusty Russell c5de5d4c39 timeout: remove absolute timers.
We don't actually use them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:53:15 +10:30
Rusty Russell 3aca5c87e3 init: rebroadcast anchors on restart if we haven't seen them.
It's possible that we won't have sent the anchor, but state is
committed in db.  And our current philosophy is that we retransmit all
the txs dumbly, all the time.

Our --restart --timeout-anchor test trigger this case, too, so
re-enable that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:52:15 +10:30
Rusty Russell 0f09605e17 test: restore anchor timeout tests.
Importantly, they're now entirely block driven.  We don't use
dev-setmocktime at all any more.

This also fixes a bug if we run the test twice against the same
bitcoind; we need to extract the time from the block header rather
than assuming bitcoind is on the current time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:50:54 +10:30
Rusty Russell 6f360422d4 chaintopology: restore anchor timeout.
Instead of using wall-clock time, we use blocks.  This is simpler and
better for database restores.  And both sides will time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:14:22 +10:30
Rusty Russell 1fe83f47a9 dev-broadcast: control whether we send out transactions.
Good for testing; we also flush broadcast when it's re-enabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:14:21 +10:30
Rusty Russell f715752e71 chaintopology: keep txs to send in a struct.
This allows us to add a new field for a callback at the end, but
more subtle, ensures broadcast in order (which simplifies testing).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:14:10 +10:30
Rusty Russell 56b0f03c5a peer: fix retransmission before open packet.
Re-enabling the next test revealed bugs: if we need to retransmit the
initial open_commit_sig packet, we currently tried to send it as an
UPDATE_COMMIT, which isn't allowed.  Fixing that revealed that if
we have to retransmit the initial open, we didn't do that either.

Thus the initial open should count towards the ack count, and we should
special case transmissions of 0 (pkt_open) and 1
(pkt_open_commit_sig).

We also save those early state changes to the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 08:04:28 +10:30
Rusty Russell 41f3f8e067 db: store anchor input.
If we die before we've established connection, we should remember how
we were going to fund it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 08:04:28 +10:30