Commit Graph

724 Commits

Author SHA1 Message Date
Christian Decker 2c06524165 refactor: Moving legacy sync method out of routing
This was the only time we actually reference non-routing structs in
routing, so moving this out should allow us to get it working in the
new subdaemons.
2017-02-03 05:52:11 +10:30
Christian Decker 5d10093da3 refactor: Moving the node_map definition into routing.h
This allows us to move some legacy functions closer to where they are
actually used, and not worry about them when including routing.h into
the new subdaemons. `struct peer` is the main culprit here.
2017-02-03 05:52:11 +10:30
Christian Decker 3cb576d69d refactor: Moving gossip/routing specific state into its own struct
This used to be part of `lightningd_state` which is being split up for
the various subdaemons. The main change is the addition of the `struct
routing_state` in `routing.h` and the addition of `rstate` in `struct
lightningd_state` for backwards compatibility.
2017-02-03 05:52:11 +10:30
Rusty Russell 940053d000 htlc: move enum side crom daemon/channel.h to daemon/htlc.h
lightningd wants htlcs, but not the old struct channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-02 14:48:00 +10:30
Rusty Russell d97e3489a0 daemon/htlc_state: split off from daemon/htlc.
We want to use struct htlc inside lightningd, and we need the state
bits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-02 14:48:00 +10:30
Rusty Russell c6997f15c7 bitcoin/preimage: struct preimage.
We had a hack for 'struct rval' in protobuf_convert.h; make an
explicit header and put it in bitcoin/preimage.h.  It's not really
bitcoin-specific, but it's better than having bitcoin/script depend on
an external header.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-02 14:48:00 +10:30
Rusty Russell 279f216208 gen_peer_wire_csv: update to latest spec.
In particular, 860990fa0afb55f839e882a5e9abe8abe6ccb981 reordered
channel_announcement and c93bf5cf8c48eab1b028e85214cb35feeeffcbb3
reordered the update_fail_malformed_htlc message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-02 14:48:00 +10:30
Rusty Russell 8522a5ea64 struct bitcoin_tx: remove explicit lengths, use tal_len()/tal_count()
They're always tal objects, so we can simply ask tal for the length,
simplifying the API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-25 11:03:55 +10:30
Rusty Russell 8159c4458a bitcoin/script.h: remove struct bitcoin_signature
Technically this incudes the sighash flags, but we only handle SIGHASH_ALL
anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-25 11:03:55 +10:30
Rusty Russell 1edce4878d bitcoin/signature.h: remove struct signature.
It's a wrapper around secp256k1_ecdsa_signature, so use that directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-25 11:03:55 +10:30
Christian Decker 091c2fc8f5 log: Flushing logs on each new entry
This is to speed up the python testing framework and should not have a
big impact on performance.
2017-01-23 10:37:34 +01:00
Christian Decker 3f79a0e117 log: Flushing logs on every line
This is needed for the new testing framework since we wait for
messages to be printed on stdout. Buffering delays this
arbitrarily. Flushing so often should not have much of a performance
impact.
2017-01-23 10:45:36 +10:30
Christian Decker d6ccf90063 jsonrpc: Renamed `awaitpayment` -> `waitinvoice` -> `waitanyinvoice`
As suggested by Rusty
2017-01-23 10:22:13 +10:30
Christian Decker 1a9bfe7be4 jsonrpc: Added `awaitpayment` method
`awaitinvoice` can be used to wait on a specific invoice to be
completed. If the invoice was previously paid, then the command
returns immediately, otherwise it'll block until the invoice is
paid. This complements `waitinvoice` which uses a highwatermark and
waits for the next invoice. I found waitinvoice a bit hard to use
since it doesn't allow waiting for a specific invoice to be completed,
just the next in the insertion order.
2017-01-23 10:22:13 +10:30
Christian Decker ba83430b5f trivial: Removing unused constant 2017-01-22 16:19:11 +01:00
Christian Decker cae283087d sphinx: Committing the onion packet to the payment-hash
The sphinx onion packet now commits to the HTLC payment-hash it is
associated with. This prevents replay attacks with the same onion.
2017-01-16 11:14:30 +10:30
Christian Decker 91b17d45d8 sphinx: Removing last vestiges of the end-to-end payload
So far this was simply set to a zero-length end-to-end payload. We
don't have any plans of re-adding it for the moment, so let's get rid
of the unused code.
2017-01-16 11:14:15 +10:30
Christian Decker 285b8b4698 sphinx: Use libsecp256k1 to generate shared secrets
So far we did it on our own, but since the spec specifies that we use
the libsecp256k1 version anyway, we can remove our own implementation.
2017-01-16 11:08:36 +10:30
Christian Decker 679dec3e6a sphinx: Actually use the full pubkey in ECDH key generation
The spec says that we use the libsecp256k1 style ECDH, which uses the
full compressed pubkey from the scalar multiplication which is then
hashed. This is in contrast to the btcsuite implementation which was
only using the hashed X-coordinate.
2017-01-16 11:08:36 +10:30
Rusty Russell 6bf3c30a10 lightningd/Makefile: fix check-source
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-13 10:51:57 +10:30
Rusty Russell 73d07ce441 libsodium: use our local submodule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-11 10:04:26 +10:30
Rusty Russell e39d09d5a5 daemon/test/scripts: maek slightly more general for lightningd/lightningd
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell 015eb072b8 Makefile: split CORE_SRC into CORE_SRC, CORE_TX_SRC and CORE_PROTOBUF_SRC
With the lightningd daemon split, we don't need them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell a0ac5c276e status: API for status reporting.
The API formalizes how daemons should report their statuses back to
the main lightningd.  It's a simple write API, which includes tracing
support (currently it always sends traces, later it could send iff
there's a failure, for example).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:24:20 +10:30
Rusty Russell 42f474af18 wire: make lengths of variable fields implied by tal_count()
This is a much nicer interface, and works better in practice too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell 7b1a4fc765 utils: add tal_hex() helper.
This is a shortcut when the data being dumped is a tal array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell 4a233090ae log: rename struct log_record to struct log_book.
I think "log entry" when I see "log record", so this name is better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:18:26 +10:30
Rusty Russell 350d8edb7e log: don't include dstate any more.
Before we had a global secp256k1_ctx we needed to hold this to print
out pubkeys, now it's completely orthogonal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:17:51 +10:30
Christian Decker af43cc5e2c makefile: Be more verbose in `check-daemon-headers`
So far it was failing silently, now it diffs the Makefile state
against the directory listing. This also fixes a bug when the locale
was not set the sort order would not match.
2017-01-10 09:26:06 +10:30
Rusty Russell b59fe5a2b6 daemon/Makefile: objects depend on wire headers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-06 13:23:50 +10:30
Christian Decker 96af89139e base58: Unittests need in-tree libbase58 2017-01-05 12:12:30 +10:30
Lucas Betschart 3e6bb958eb Add libbase58 as git submodule
Pointing to the latest release (0.1.4)
2017-01-05 12:11:18 +10:30
Rusty Russell bdc4972df6 wire/gen_peer_wire_csv: update to latest spec #2
828eda61df5a7be27051c605f7808e4f690739e4, in particular, it has the
new address format for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell d3bdb073b5 wire/gen_peer_wire_csv: update to latest spec #1
In particular, add features bitfields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell bdaa22e247 generate-wire.py: allow NULL len parameter from fromwire_*
It implies tal_count() gives the length. Great for almost all callers which
don't care if there are extra bytes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell e076d56709 generate-wire.py: include type bytes in towire/fromwire routines.
This removes some redundancy in creating messages, but also allows
a lazy form or parsing without explicitly checking the type.

A helper fromwire_peektype() is added to look up the type and handle
the too-short-for-type problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell c864b28068 generate-wire.py: don't generate structures, hand in all values.
This is a bit more awkward for large structures, but avoids
indirection for the simpler ones (I copied the structures for the test
code, however).  We also remove explicit padding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell a08a2105ea generate-wire.py: generalize, move to tools.
We're going to want to use this for inter-daemon comms, so generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell b7789bf065 Makefile: generalize whitespace check.
Spread to individual Makefiles, and include headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 0861ec33a5 check-source-bolt: generalize.
This way sub-Makefiles can add their own files to check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell c2cc164d6d daemon: disable old BOLT checks.
This is useful for the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 05feefbb8a test: move mockup script outside daemon/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell d30b67a3fe test: generalize update-mocks
So we can use it in other directories.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell dec3b9d030 peer: don't ever set up listener on dynamic port.
Simplifies the logic somewhat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:15 +10:30
Rusty Russell 12b30ab4fc jsonrpc: register base on linkage.
Other than being neater (no more global list to edit!), this lets the
new daemon and old daemon have their own separate routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:08:15 +10:30
Rusty Russell c7b69abdaa type_to_string: move formatting to appropriate files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:07:15 +10:30
Rusty Russell f8eb454139 type_to_string: move pretty printing of types from log
It's not just useful for logging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:05:15 +10:30
Rusty Russell 756b2a0530 options: --help and --version are early args.
If they use these, we definitely don't want to create the config dir...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:04:15 +10:30
Rusty Russell 93471e06c1 options: return true if we created new directory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:03:15 +10:30
Rusty Russell 038ef0250a options: move option and config code into its own file.
For the moment, the new lightningd will share all this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 13:22:29 +10:30
Rusty Russell ca4d03aa2a lightningd_state: move invoices into separate structure.
It's all maintained by invoice.c, so make it private to that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 13:18:47 +10:30
Christian Decker 570003235e gossip: Fixing minor issues
Thanks @rustyrussell for the review ^^
2017-01-03 15:08:05 +10:30
Christian Decker 95bc8f6987 gossip: Removing origin from staggered broadcast
Keeping a pointer to the peer that initially sent us a message
could (actually will!) result in dangling pointers. Removing this
results in some additional messages, which will be discarded by the
recipient, so that should not be a problem.
2017-01-03 15:08:05 +10:30
Christian Decker 0ed23c46a9 gossip: Implemented routing table sync with newly connected peers 2017-01-03 15:08:05 +10:30
Christian Decker 5ce1f7c2f3 gossip: Set announce timeout to 5 hours 2017-01-03 15:08:05 +10:30
Christian Decker 71a18fdd2e gossip: Trigger announce on channel establishment
Since we ultimately want to have a very long re-announcement timeout
we better trigger an announcement upon channel state change to normal.
2017-01-03 15:08:05 +10:30
Christian Decker e54c0adced gossip: Do not consider half-open connections for routes
Connections are in a half-open state after receiving the
`channel_announcement` and before the `channel_update` makes them
usable, so we need to ignore channels that are not yet fully open.
2017-01-03 15:08:05 +10:30
Christian Decker 611f4833c5 gossip: Implemented the staggered broadcast 2017-01-03 15:08:05 +10:30
Christian Decker fca9bf1a16 gossip: Partially sign channel_announcements
We can already add our own signatures, will have to defer remote
signatures until we implement the channel setup messages from the spec.
2017-01-03 15:08:05 +10:30
Christian Decker d93ce12afa gossip: Broadcasting and handling gossip messages 2017-01-03 15:08:05 +10:30
Christian Decker e8d68757f6 gossip: Tracking tx index in topology
The txlocator was reporting the index in the purged array of
transactions instead of the index in the original block.
2017-01-03 15:08:05 +10:30
Christian Decker 57d5ae96a4 routing: Added channel_id and directions to connections
The gossip protocol spec refers to channels by their `channel_id` and
a direction. Furthermore, inbetween the `channel_announcement` and the
`channel_update` for either direction, the channel direction is in an
undefined state and cannot be used, so added the `half_add_connection`
function and an `active` flag to differentiate usable connections from
unusable ones.
2017-01-03 15:08:05 +10:30
Christian Decker db481d881a proto: Added handling for nested packets 2017-01-03 15:08:05 +10:30
Rusty Russell a44e4fb631 daemon/json, test/test_protocol: avoid gcc -O warnings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:49:51 +10:30
Rusty Russell e109977df7 peer: don't print uninitialized commit_num.
Reported-by: Lucas Betschart <lucasbetschart@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:44:48 +10:30
Rusty Russell 2a68a984c3 peer: fix unassigned error packet on receiving non-ASCII error.
Reported-by: Lucas Betschart <lucasbetschart@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:42:24 +10:30
Rusty Russell 1b170c85bb Revert "Fix -Wsometimes-uninitialized and -Wuninitialized"
This reverts commit ff9cdda4fa.

These are real bugs, so I prefer to fix them explicitly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:41:31 +10:30
Lucas Betschart 900ab78d46 Fix -Wtautological-constant-out-of-range-compare 2016-12-14 08:47:21 +01:00
Lucas Betschart ff9cdda4fa Fix -Wsometimes-uninitialized and -Wuninitialized 2016-12-14 08:45:40 +01:00
Lucas Betschart c163fe04ac Fix -Wenum-conversion 2016-12-14 08:43:48 +01:00
Lucas Betschart 33d6cbe6bc Fix -Wgnu-variable-sized-type-not-at-end 2016-12-14 08:41:30 +01:00
Rusty Russell 45ec071e58 lightningd: remove whitespace failing make check-source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-05 14:03:06 +10:30
Rusty Russell 36c8fc7ef8 lightningd: remove secpctx
Use the global in the few remaining places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:13:27 +10:30
Rusty Russell a4fdaab5b3 Use global secp256k1_ctx instead of passing it around.
If I'd known how large this patch would be (though trivial), I'd
have done it in parts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:12:58 +10:30
Rusty Russell c938ebb5c0 utils: add a global secp, fix wire to use it.
This repairs make check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:11:06 +10:30
Rusty Russell 1f447688bc test: make generic driver for unit tests.
Not just in daemon.  This fixes 'make check' when it tries to
run 'make wire-tests'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:10:13 +10:30
Christian Decker e40509c6c3 wire: Linking against wire objs 2016-12-01 13:50:02 +10:30
Rusty Russell 0ba93cb948 lightningd: don't listen at all if no port number set.
When we support the Milan protocol, we'll use a default port.  But
for now, don't listen at all unless a port is specified.

Fixes: #54
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 17:13:29 +10:30
Rusty Russell fe14b3f4d9 Merge remote-tracking branch 'origin/pr/92'
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 17:05:42 +10:30
Rusty Russell 97a155bad5 Merge remote-tracking branch 'origin/pr/97'
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 14:28:22 +10:30
Christian Decker 42a33df42e irc: Checking for param count before accessing 2016-11-16 22:55:29 +01:00
Christian Decker 188b3c3e19
sphinx: Fixed a buffer overflow in hmac generation
Our HMACs are truncated to 20 byte, but sodium still generates 32 byte
HMACs and we were handing in a buffer that was too small, so we
overflowing the buffer by 12 bytes. This manifested itself only in the
32 bit variant because of different alignment in the 64bit version.

Fixes #94.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2016-11-16 14:41:19 +01:00
Christian Decker b2a3e8010f Merge pull request #96 from ElementsProject/minor-sphinx-cleanups
Minor sphinx cleanups
2016-11-16 14:35:19 +01:00
Christian Decker bf81f7af15 Merge pull request #93 from ElementsProject/fix-87-open-too-small-for-fee
json_connect: don't try to double-fail if amount insufficient.
2016-11-16 13:04:26 +01:00
Rusty Russell 41299b679c sphinx: process_onionpacket doesn't modify the packet.
Make it const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:22:14 +10:30
Rusty Russell 423572190b sphinx: use assignment instead of memcpy where possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:22:08 +10:30
Rusty Russell 6f9dedbe7f sphinx: remove redundant initialization
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:04:50 +10:30
Rusty Russell 8599d63256 sphinx: add brackets around constant definition
Always do this, otherwise "MACRO * foo" can have unexpected results.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:02:56 +10:30
Rusty Russell 5465b61784 json: fix 32-bit build.
Assume we have strtoull everywhere (it's C99 and was common before).

Fixes: #91
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 13:34:56 +10:30
Christian Decker 686ff6edef irc: Switched to LFNet.org IRC server
This is a temporary fix until we finish #88.
2016-11-14 23:19:32 +01:00
Rusty Russell 94adbd241d json_connect: don't try to double-fail if amount insufficient.
Closes: #87
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-14 11:46:09 +10:30
Christian Decker 34b4134cb0 bugfix: Removed spurious printf in peer.c
I must've left that on in. Fixes #89
2016-11-14 00:29:01 +01:00
Christian Decker e5b44ff232 Merge pull request #85 from ElementsProject/whitespace
Whitespace
2016-11-11 16:40:23 +01:00
Christian Decker 9848b4ac3e travis-ci: Added travis-ci config
Added .travis.yml to get travis-ci to build and run tests for us.

In addition this fixes a flaky test due to the fact that when lightning2
connects to lightning3 and we tell lightning3 to restart, then
lightning2 will back-off its reconnection attempts, potentially causing
a timeout to trigger during tests. This was triggered by travis-ci
relatively consistently since the restart would take quite some
time. Now simply restarting them in reverse order and a small timeout
seems to fix this consistently.
2016-11-11 14:25:41 +01:00
Rusty Russell f5c00deec7 Remove trailing whitespace from source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-11 09:32:04 +10:30
Rusty Russell 716da21f85 chaintopology: reduce how far back we start on testnet.
When initially reading the blockchain, we start 100 back from the
current block, or at the first block with a funding transaction,
whichever is earlier.

This slows testing slightly, so use whatever the "forever" value is
(10 on testnet, still 100 on mainnet).

make check -j12 times:
	Before:	7m52.005s
	After:  6m31.896s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:53:48 +10:30
Rusty Russell ce96812ef0 test.sh: split into multiple scripts.
This significantly reduces re-testing of the same paths, and simplifies
the addition of new tests.

make check -j12 times:
	Before: 9m24.973s
	After:	7m52.005s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:44:49 +10:30
Rusty Russell fae7f68731 helpers.sh: make check_tx_spend able to check for a specific transaction.
It currently takes an unused "who to blame" argument, which doesn't
always make sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:34:35 +10:30
Rusty Russell c6a187a5d7 helpers.sh: set up funding in start_lightningd()
Every test wants this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:33:35 +10:30
Rusty Russell 9e777ae922 test.sh: move core routines to scripts/helpers.sh
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:32:35 +10:30