Commit Graph

324 Commits

Author SHA1 Message Date
Rusty Russell 240abf6c46 common/io_lock: add helper to query if lock is taken.
Not just for debugging; we actually need to know if a write is active for
the coming hack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell 305795b01e common/json: move JSON creation routines into lightningd/
It's the only user of them, and it's going to get optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossip.pydiff --git a/common/test/run-json.c b/common/test/run-json.c
index 956fdda35..db52d6b01 100644
2018-10-19 22:02:11 +00:00
Rusty Russell 71a40faae7 withdraw: fix incorrect error when we have an empty wallet.
This also highlights the danger of searching the logs: that error
appeared previously in the logs, so we didn't notice that the actual
withdraw call gave a different error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell 6af8f29392 bolt11: better message when you try to 'lightning-cli pay' an onchain addr.
Give a clear error at the beginning if it's not bolt11 payment,
rather than falling foul of other checks.

This will work at least until some altcoin adapts the 'ln' prefix :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:59:20 +00:00
lisa neigut 66ca2a333f channeld: use u64 fo `htlc_minimum_msat`
As per spec.
2018-10-16 03:32:27 +00:00
Christian Decker 0ee0da0eb8 common: Add a test for io_lock
Totally forgot to add this test. It just shows how a writer can take
exclusive access of a socket over multiple `io_write` calls, and
queuing all others behind it.
2018-10-11 01:43:55 +00:00
Christian Decker bb38541d9e common: Added a small locking mechanism to simplify how we lock IO
We've done this a number of times already where we're getting
exclusive access to either the out direction of a connection, or we
try to lock out the read side while we are responding to a previous
request. They usually are really cumbersome because we reach around to
the other direction to stop it from proceeding, or we flag our
exclusive access somewhere, and we always need to know whom to notify.

PR ElementsProject/lightning#1970 adds two new instances of this:

 - Streaming a JSON response requires that nothing else should write
   while the stream is active.
 - We also want to stop reading new requests while we are responding
   to one.

To remove the complexity of having to know whom to stop and notify
when we're done, this adds a simple `io_lock` primitive that can be
used to get exclusive access to a connection. This inverts the
requirement for notifications, since everybody registers interest in
the lock and they get notified if the lock holder releases it.
2018-10-11 01:43:55 +00:00
Rusty Russell df27fc55af More renaming of gfeatures to globalfeatures.
Use the BOLT #1 naming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell 76e7f8aa5c wireaddr: add wireaddr_eq.
We can't STRUCTEQ_DEF() it since addrlen is variable, so open-code it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell 41b0872f58 Use localfeatures and globalfeatures consistently.
That's what BOLT #1 calls them; make it easier for people to grep.

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell a24ed20dee common/cryptomsg: remove unused async routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell a1bdaa8f99 connectd/peer_exchange_initmsg: handle peer comms ourselves.
connectd is the only user of the cryptomsg async APIs; better to
open-code it here.  We need to expose a little from cryptomsg(),
but we remove the 'struct peer' entirely from connectd.

One trick is that we still need to defer telling lightningd when a
peer reconnects (until it tells us the old one is disconnected).  So
now we generate the message for lightningd and send it once we're woken.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell d1552bd1eb test/run-cryptomsg.c: move out of lightningd/ into common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell dfaaa09bc6 common/cryptomsg: remove unused 'reading_body' flag and 'peer_in_started'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell 96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Christian Decker c3f433ec66 json: Support streaming JSON messages
It turns out we were heavily relying on the fact that after each message from
the client there'd be a flush, and that there would not be anything after the
JSON object we read. This will no longer be the case once we start streaming
things or we are very quick in issuing the JSON-RPC requests.

This just takes one of the error paths (incomplete read) and makes it into a
successful path if we have indeed read a full root element.
2018-09-25 05:47:23 +00:00
lisa neigut b1f15c2605 BOLT updates: broken link fixes
See a9195a84d0
2018-09-21 00:24:12 +00:00
lisa neigut b287f2f007 BOLT 11 human-readable formatting changes 2018-09-21 00:24:12 +00:00
Rusty Russell e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Saibato d5c3626fa7 parse autotor: address before separate_address_and_port
this enables addr like --addr=autotor:127.0.0.1 or
--addr=autotor:localhost to just use the default tor service port

Signed-off-by: Saibato <Saibato.naga@pm.me>
2018-09-20 09:09:11 +02:00
Christian Decker dc88c35d7f channeld: Do not fail if we get a chain_hash we don't know 2018-09-14 21:18:11 +02:00
Christian Decker 2402c524cc channeld: Keep track of the chainparams for the chain we are using 2018-09-14 21:18:11 +02:00
Rusty Russell 7b9341e762 subdaemon: better GDB support.
It was annoying me, so I made it much nicer to use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:36:05 +02:00
Rusty Russell 76f116daf1 lightningd: minor cleanups
Code changes:
1. Expose daemon_poll() so lightningd can call it directly, which avoids us
   having store a global and document it.
2. Remove the (undocumented, unused, forgotten) --rpc-file="" option to disable
   JSON RPC.
3. Move the ickiness of finding the executable path into subd.c, so it doesn't
   distract from lightningd.c overview.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Mark Beckwith aa60057134 param: upgraded json_tok_escaped_string
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Mark Beckwith c32f7910cc param: upgraded json_tok_label
Added utility function json_tok_is_num so I would avoid using goto.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Rusty Russell 90d1062a55 daemon_conn: fix memory leak when passing an fd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell a2828ed40b memleak: allow for scanning non-talloc regions.
For some daemons we'll be handing it non-talloc memory to scan for pointers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 8733015836 memleak: don't require a root pointer.
We can just track everything from NULL (the ultimate parent) down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 37ea0d3c7f memleak: fix exclude check.
We want to exclude the child from being entered into the htable:
if we wanted the parent we could do this outside the loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 112b7336a3 memleak: create and use a generic htable helper and generic intmap helper.
memleak can't see into htables, as it overloads unused pointer bits.
And it can't see into intmap, since they use malloc (it only looks for tal
pointers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 12a39b8a79 lightningd: fix backtraces in memleak detection.
We were using a *different* backtrace_state var, which was always NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 7f37fa3263 derive_basepoints: harden checking.
I managed to crash the HSM by asking for point -1 (shachain_index has an
assert).  Fail in this case, instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 8340d8c070 secret_eq: remove in favor of constant time variant.
To be safe, we should never memcmp secrets.  We don't do this
currently outside tests, but we're about to.

The tests to prove this as constant time are the tricky bit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 4c891f4661 common: log when we toggle IO logging, don't edit env in tests!
Tests were failing when in the same thread after a test which set
log_all_io=True, because SIGUSR1 seemed to be turning logging *off*.

This is due to Python using references not copies for assignment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell 9f044305db pytest: dev env var LIGHTNINGD_DEV_LOG_IO turns io logging on immediately.
This is required for the next test, which has to log messages from channeld
as soon as it starts (so might be too late if it sends SIGUSR1).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell ebaf5eaf2e channeld: send option_data_loss_protect fields.
We ignore incoming for now, but this means we advertize the option and
we send the required fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell b123b1867d shachain: shachain_get_secret helper.
This is a wrapper around shachain_get_hash, which converts the
commit_num to an index and returns a 'struct secret' rather than a
'struct sha256' (which is really an internal detail).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Mark Beckwith 6254d15efd param: upgraded json_tok_sha256
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 947752b9cc param: upgraded json_tok_percent
Made it a local static since its only used once.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith e5918f4e5a param: upgraded json_tok_double
Also renamed old version to json_to_double for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8f17191099 param: upgraded json_tok_tok to advanced callback
This was a very simple change and allowed us to remove the special
`json_opt_tok` macro.

Moved the callback out of `common/json.c` to `lightningd/json.c` because the new
callbacks are dependent on `struct command` etc.
(I already started on `json_tok_number`)

My plan is to:
	1. upgrade json_tok_X one a time, maybe a PR for each one.
	2. When done, rename macros (i.e, remove "_tal").
	3. Remove all vestiges of the old callbacks
	4. Add new callbacks so that we no longer need json_tok_tok!
	   (e.g., json_tok_label, json_tok_str, json_tok_msat)

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-13 23:46:35 +00:00
Rusty Russell db22d2366e crypto_sync: sync_crypto_write_no_delay to flush TCP after critical packets.
Avoid that 200ms loss.  We don't want to disable nagle generally,
since it's great for gossip and other traffic; we just want to push at
critical times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell b4e6a0fcad peer_failed: write error message to peer directly.
We currently hand the error back to the master, who then stores it for
future connections and hands it back to another openingd to send and exit.

Just send directly; it's more reliable and simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 17f7f50814 wireaddr: correctly parse ':portnum' (meaning IPv4 and IPv6)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-08 15:04:06 +02:00
Rusty Russell 136f10e4a3 common/read_peer_msg: remove.
Also means we simplify the handle_gossip_msg() since everyone wants it to
use sync_crypto_write().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell 0b08601951 sync_crypto_write/sync_crypto_read: just fail, don't return NULL.
There's only one thing the caller ever does, just do that internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00