Commit Graph

616 Commits

Author SHA1 Message Date
Rusty Russell 31b2865791 common/json_tok: expose routines to parse addresses.
These are currently inside lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell a9427f1a8d bitcoin/feerate: new exposure for feerate parsing outside lightningd.
This exposes the numeric part of param_feerate() as param_feerate_val().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
niftynei 57488cde13 hsm: decouple hsm from wallet; init before wallet
We're going to use the hsm for a migration, so we need to set up the HSM
before we get to the wallet migration code.

All that this requires is removing the places in HSM init that we touch
the database struct -- easy enough to accomplish by passing the required
field back out from init, and then associating it onto the wallet after
it's been initialized.
2020-06-11 13:13:13 +02:00
niftynei a04f0fe250 psbt: remove input_amounts from bitcoin tx
Instead we will stash them into the PSBT as a utxo/witness record (which
includes the amount)
2020-06-11 13:13:13 +02:00
Rusty Russell cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell f77d70d546 common/json: move rest of bitcoin/lightning-specific json functions to json_helpers.
This dramatically reduces the linking requirements of lightning-cli.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
lisa neigut ffd9467f14 coin moves: persist the coin movement index counter to disk
Should make it easier to track when coin moves in the plugin are
disjoint from what c-lightning says it's broadcast already.
2020-05-12 15:46:58 +09:30
lisa neigut 9caf20f636 coin moves: don't log coin moves in onchaind if we're replaying
On node start we replay onchaind's transactions from the database/from
our loaded htlc table.  To keep things tidy, we shouldn't notify the
ledger about these, so we wrap pretty much everything in a flag that
tells us whether or not this is a replay.

There's a very small corner case where dust transactions will get missed
if the node crashes after the htlc has been added to the database but
before we've successfully notified onchaind about it.

Notably, most of the obtrusive updates to onchaind wrappings are due to
the fact that we record dust (ignored outputs) before we receive
confirmation of its confirmation.
2020-05-12 15:46:58 +09:30
Rusty Russell 1e4f85a539 lightningd: refactor to extract getmanifest paths.
This will allow the dynamic starting code to use them too.

Also lets us move dev_debug_subprocess under #if DEVELOPER.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell 77094b7df8 lightningd: avoid plugin timer indirection.
Now we know whether the command completed or not, we can correctly
call command_still_pending() if it didn't complete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell f8cdb523dd plugin_hook_call: return indication whether we called the callback or not.
This will allow us to simplify the caller's command handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
rbndg 241fa00e97 plugin:added invoice creation event
New invoice_creation event triggered when an new invoice is created

Changelog-Added: plugin: New invoice_creation plugin event
2020-05-04 19:18:05 +02:00
Rusty Russell 4eb1233ccb lightningd: don't report spurious temporary_node_failure on local failures.
I noticed the following in logs for tests/test_connection.py::test_feerate_stress:

```
DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Failing HTLC 18446744073709551615 due to peer death
DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: local_routing_failure: 8194 (WIRE_TEMPORARY_NODE_FAILURE)
```

This is because it reports the (transient) node_failure error, because
our channel_failure message is incomplete.  Fix this wart up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
lisa neigut 0e20e3c5e7 df: rename 'funder' to 'opener'
Previously we've used the term 'funder' to refer to the peer
paying the fees for a transaction; v2 of openchannel will make
this no longer true. Instead we rename this to 'opener', or the
peer sending the 'open_channel' message, since this will be universally
true in a dual-funding world.
2020-05-04 10:22:26 +09:30
Christian Decker 27ea47ae37 plugins: Fix undefined deallocation order in `struct plugins`
We use the new function `plugins_free` to define the correct deallocation
order on shutdown, since under normal operation the allocation tree is
organized to allow plugins to terminate and automatically free all dependent
resources. During shutdown the deallocation order is under-defined since
siblings may get freed in any order, but we implicitly rely on them staying
around.
2020-04-16 18:03:35 +09:30
Rusty Russell deac09950a plugins: make chained hooks have two different callbacks.
One is called on every plugin return, and tells us whether to continue;
the other is only called if every plugin says ok.

This works for things like payload replacement, where we need to process
the results from each plugin, not just the final one!

We should probably turn everything into a chained callback next
release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 9aedb0c61f plugin: simplify hooks calling methods, and make lifetime requirements explicit.
They callback must take ownership of the payload (almost all do, but
now it's explicit).

And since the payload and cb_arg arguments to plugin_hook_call_() are
always identical, make them a single parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 2c3543e42d lightningd: fix crash when failing htlc once channeld dies.
We were reaching through it to get `ld`, but channeld is already dead.
Caught this on test_onchaind_replay (without valgrind) on my test machine:

    INFO 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
    **BROKEN** lightningd: FATAL SIGNAL 11 (version f6e1735)
    **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x5634dc83dc55
    **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x5634dc83dca9
    **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fd4b7c0b46f
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:285 (failmsg_incorrect_or_unknown) 0x5634dc82625a
    **BROKEN** lightningd: backtrace: lightningd/htlc_set.c:109 (htlc_set_add) 0x5634dc801e5c
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:496 (handle_localpay) 0x5634dc826997
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1008 (htlc_accepted_hook_callback) 0x5634dc827e60
    **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:197 (plugin_hook_callback) 0x5634dc831ea1
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:261 (plugin_response_handle) 0x5634dc82d2c3
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:359 (plugin_read_json_one) 0x5634dc82d46f
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:391 (plugin_read_json) 0x5634dc82d5c6
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x5634dc896319
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x5634dc896efe
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x5634dc896f40
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:39:49 +09:30
Rusty Russell b29d1ed3ff channeld: support HTLCs with blinding (EXPERIMENTAL_FEATURES)
Note that it's channeld which calculates the shared secret, too.  This
minimizes the work that lightningd has to do, at cost of passing this
through.

We also don't yet save the blinding field(s) to the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Vasil Dimov 3ce0552dd4 build: use locale-independent sort for mocks
Use `LC_ALL=C sort` instead of `sort` so that mocks get sorted in
the same way on all developers' environments.

Re-record the result of `make update-mocks`.

Changelog-None
2020-04-07 13:52:48 +09:30
Rusty Russell 3b4a06f52b common: generalize ecdh function.
common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV.  But how it does ecdh is daemon-specific.

We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh().  This just makes it more general.

So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-04 16:08:49 +10:30
Rusty Russell 41ebaffba3 lightningd: return `features` in connect response.
This is useful in general, but in particular it allows fundchannel to avoid YA
query to figure out if it can wumbo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON: `connect` returns `features` of the connected peer on success.
2020-04-03 13:13:21 +10:30
Rusty Russell cf43e44378 common/features: don't use internal global.
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell b7db588a8a plugin: remove boutique features.
We now manipulate the global features directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell 590b2db88e lightningd: make local htlc failures pass a wiremsg for errors, not a failcode.
Unfortunately the invoice_payment_hook can give us a failcode, so I simply
restrict it to the two sensible ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-deprecated: plugins: invoice_payment_hook "failure_code" only handles simple cases now, use "failure_message".
2020-02-25 11:12:12 +10:30
darosior c79ab0f1b0 lightningd/bitcoind: use the Bitcoin plugin for getutxout 2020-02-12 11:45:07 +10:30
darosior 947f5ddde1 lightningd/bitcoind: use the Bitcoin plugin to send transactions
This restrains the informations we get about how the sending went to
an errmsg as we cant rely on bitcoin-cli specific output nor its exit code.
2020-02-12 11:45:07 +10:30
Christian Decker 58c9a6a004 plugin: Include featurebits registered by plugins in invoices 2020-02-11 13:53:31 +10:30
darosior 3510c29e5d common: move json_stream helpers to common/json
Now that we have json_stream in common/, we can move all the related
helpers from lightningd/json to common/json. This way everyone can
benefit of them (including libplugin, the plugins themselves,
potentially lightning-cli), not lightningd alone!

Note that the Makefile of the common/test/ had to be modified, because
the new helpers make use of common/wireaddr... Which turns out to
\#include <lightingd/lightningd.h> ! So we couldnt just include the .c
and add mocks if we redefined some structs (hello run-param).
2020-02-04 13:24:32 +10:30
darosior d35387f489 common: move json_stream from lightningd/ to common/
It's not lightningd-specific and we are going to need it for libplugin. The only
drawback is the log_io removal in json_stream_output_write()..
2020-02-04 13:24:32 +10:30
Ken Sedgwick 5fd0ed79f4
lightningd: Added --subdaemon command to allow alternate subdaemons.
Changelog-Added: lightningd: Added --subdaemon command to allow alternate subdaemons.

[ Wow, that was mammoth; 44 comments over 12 commits. Feels almost unfair to squash it into one commit, so I wanted to note @ksedgwic's perseverence here! --RR ]
2020-02-04 10:44:13 +10:30
Vasil Dimov 18a40c0c5d build: re-record the result of `make update-mocks`
Changelog-None
2020-02-03 15:38:11 +00:00
Vasil Dimov 55173a56b7 Use dedicated type for error codes
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.

To resolve this:

* Introduce an error code type with a known size:
  `typedef s32 errcode_t`.

* Change all error code macros to constants of type `errcode_t`.
  Constants also play better with gdb - it would visualize the name of
  the constant instead of the numeric value.

* Change all functions that take error codes to take the new type
  `errcode_t` instead of `int`.

* Introduce towire / fromwire functions to send / receive the newly added
  type `errcode_t` and use it instead of `towire_int()`.

In addition:

* Remove the now unneeded `towire_int()`.

* Replace a hardcoded error code `-2` with a new constant
  `INVOICE_EXPIRED_DURING_WAIT` (903).

Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
2020-01-31 06:02:47 +00:00
Christian Decker 3ad8438d91 json-rpc: Add sendcustommsg command
This command injects a custom message into the encrypted transport stream to
the peer, allowing users to build custom protocols on top of c-lightning
without requiring any changes to c-lightning itself.
2020-01-28 23:50:52 +01:00
ZmnSCPxj 54cc735201 lightningd/peer_control.c: Implement waitblockheight.
This is needed to fully implement handling of blockheight disagreements
between us and payee.
If payee believes the blockheight is higher than ours, then `pay`
should wait for our node to achieve that blockheight.

Changelog-Add: Implement `waitblockheight` to wait for a specific blockheight.
2020-01-21 22:23:21 +01:00
Rusty Russell 11dc1b341c gossipd: hand all candidates up to lightningd to select routeboost.
This lets us do more flexible filtering in the next patch.  But it also
keeps some weird logic out of gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-04 08:07:22 +08:00
Rusty Russell 72aa315b5e lightningd: save the fee_states into the database.
This is the final step: we pass the complete fee_states to and from
channeld.

Changelog-Fixed: "Bad commitment signature" closing channels when we sent back-to-back update_fee messages across multiple reconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 8e3234e67a lightningd: sew in htlc set.
The invoice_try_pay code now takes a set, rather than a single htlc, but
it's basically the same thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 12985331f7 htlcs: remove origin_htlc_id from htlc_out.
This is a transient field, so rework things so we don't leave it in
struct htlc_out.  Instead, load htlc_in first and connect htlc_out to
them as we go.

This also changes one place where we use it instead of the am_origin
flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
fiatjaf 770789684c jsonrpc: Add json_add_preimage helper 2019-12-03 01:16:07 +00:00
Rusty Russell 8393d21503 common/features: add `payment_secret` feature if EXPERIMENTAL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell e5247a68b6 lightningd: check payment secret on htlc receipt.
We don't set the secret to compulsory (yet!) but put code in for the
future.  Meanwhile, if there is a secret, check it is correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 854c64ffee common/bolt11: add secret support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 81c89aaef8 wallet: add invoice features into db.
In a future version, we will use features to insist that payers
provide the secret.  In transition, we may have old invoices which
didn't insist on that, so we need to know this on a per-invoice basis.

Not sure if I got the right syntax for adding an empty blob though!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 8b1aa3ef8b lightningd: move basic parameter parsing into common/configdir
lightning-cli is going to need to know what network we're on, so
it will need to parse the config files.  Move the code which does
the initial bootstrap parsing into common, as well as the config
file parsing core.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell edbcb6fa15 lightningd: remove chainparams local parameter from wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-20 20:41:53 +01:00
Rusty Russell b210c9d91f lightningd: use a simple array for less memory usage.
The tal overhead of 5 pointers, the linked list node is 2; and we also
tal'd the string.  That's 96 bytes per entry.

Use a simple array instead, though it means more work on deletion
since each log_entry is no longer a tal object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell 4fc498f901 lightningd: enable io logging on subdaemons iff we're going to print it.
This simplifies our tests, too, since we don't need a magic option to
enable io logging in subdaemons.

Note that test_bad_onion still takes too long, due to a separate minor
bug, so that's marked and left dev-only for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell 0607f998d1 options: allow --log-level <level>:<prefix> for finegrained log control.
This allows finegrained logging control of particular subdaemons or
subsystems.

To do this, we defer setting the logging levels for each log object
until after early argument parsing (since e.g. "bitcoind" log object
is created early).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: log-level can now specify different levels for different subsystems.
2019-11-18 04:50:22 +00:00
Rusty Russell ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell 86fb54a33b lightningd: remove per-peer log book.
We had a separate logbook for each peer, and copy log entries above
the printable log level into the master logbook.  This didn't always
work well, since we didn't dump it on crash for example.

Keep a single global logbook instead, and remove this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
darosior f075b87137 bitcoind: remove the chainparams member
We now have a global constant, prefer to use it instead of having
two variables with the same utility.
2019-11-15 13:14:08 +01:00
lisa neigut 963a1da958 addr: handle P2SH/P2PKH in scriptpubkey encoding
Previously, returned null if a scriptpubkey was not Segwit; now
handles encoding to Base58 for other types.
2019-11-13 03:31:20 +00:00
darosior a0df49718a lightningd/jsonrpc: Add a 'rpc_command' hook
The 'rpc_command' hook allows a plugin to take over any RPC command.
It sends the complete JSONRPC request to the plugin, which can then respond
with :
- {'continue'}: executes the command normally
- {'replace': {a_jsonrpc_request}}: replaces the request made
- {'return': {'result': {}}}: send a custom response
- {'return': {'error': {}}}: send a custom error

This way, a plugin can modify (/reimplement) or restrict the usage of
any of `lightningd`'s commands.

Changelog-Added: Plugin: A new plugin hook, `rpc_command` allows a plugin to take over `lightningd` for any RPC command.
2019-11-11 22:30:01 +01:00
Rusty Russell ca53c1b699 gossipd: push our own gossip messages harder.
I had a report of a 0.7.2 user whose node hadn't appeared on 1ml.  Their
node_announcement wasn't visible to my node, either.

I suspect this is a consequence of recent version reducing the amount of
gossip they send, as well as large nodes increasingly turning off gossip
altogether from some peers (as we do).  We should ignore timestamp filters
for our own channels: the easiest way to do this is to push them out
directly from gossipd (other messages are sent via the store).

We change channeld to wrap the local channel_announcements: previously
we just handed it to gossipd as for any other gossip message we received
from our peer.  Now gossipd knows to push it out, as it's local.

This interferes with the logic in tests/test_misc.py::test_htlc_send_timeout
which expects the node_announcement message last, so we generalize
that too.

[ Thanks to @trueptolmy for bugfix! ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 15:00:37 -05:00
Rusty Russell bd55f6d940
common/features: only support a single feature bitset.
This is mainly an internal-only change, especially since we don't
offer any globalfeatures.

However, LND (as of next release) will offer global features, and also
expect option_static_remotekey to be a *global* feature.  So we send
our (merged) feature bitset as both global and local in init, and fold
those bitsets together when we get an init msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-11 02:52:04 +00:00
Rusty Russell 9485919a81 queries: make sure scids are in order.
I thought LND had a bug, but turns out it doesn't like out-of-order
short_channel_ids: in fact, the spec says they have to be in order!

This means we use uintmap instead of a htable for unknown_scids and
stale_scids so they're nicely ordered.

But our nodes-missing-announcements probe is harder since they can
also contain duplicates: we switch that to iterate through channels
rather than nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
lisa neigut af4ffe5fcd dev-forget-channel: accept passing in channel_id
This patch adds a channel_id parameter to allow for specifying
channels that are lacking a short_channel_id.

Useful in the case where a peer has 1) multiple channels (ONCHAIN etc)
and 2) a channel where the funding transaction hasn't been
broadcast/mined.
2019-10-10 05:57:45 +00:00
trueptolemy 019c052123 JSON-API: Allow `close` channel to specified address
Command format: close id [unilateraltimeout] [destination]

Close the channel with peer {id}, forcing a unilateral
close after {unilateraltimeout} seconds if non-zero, and
the to-local output will be sent to {destination}. If
{destination} isn't specified, the default is the address
of lightningd.

Also change the pylightning:
update the `close` API to support `destination` parameter
2019-10-09 21:04:16 -05:00
trueptolemy 75e946d256 json: Move `param_bitcoin_address` from wallet/walletrpc.c to lightningd/json.c
It's a useful helper, and it will be used to prase address in `close` command.
2019-10-09 21:04:16 -05:00
trueptolemy 7db3f1b8a6 json: Rename `json_tok_address_scriptpubkey` to `json_to_address_scriptpubkey`
Our json naming style is like `json_to_***`.
2019-10-09 16:51:42 -05:00
Christian Decker af53e3494b wallet: Annotate only the funding output instead of the whole tx 2019-10-07 20:03:24 -05:00
Rusty Russell 6f9c5f2936 gossipd: get fed the blockheight from lightningd when we know it.
This will let gossipd be more intelligent about gossiping before we're
synced, and also it might know how far behind we are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell 27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
lisa neigut 19e65068a9 json: move param node_id parser to common
Going to need this for a plugin that's external
2019-09-11 23:56:27 +00:00
Rusty Russell acf3952acc JSON: remove handling of pre-Adelaide (B:T:N) short_channel_ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Christian Decker b6d583c26a db: Move tracking of pending statements into the `struct db`
We now have a much stronger consistency check from the combination of
transaction wrapping, tal memory leak detection. Tramsaction wrapping ensures
that each statement is executed before the transaction is committed. The
commit is also driven by the `io_loop`, which means that it is no longer
possible for us to have statements outside of transactions and transactions
are guaranteed to commit at the round's end.

By adding the tal-awareness we can also get a much better indication as to
whether we have un-freed statements flying around, which we can test at the
end of the round as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 803007ecdf db: Make the `db` struct private and provide accessors instead
We will soon generalize the DB, so directly reaching into the `struct db`
instance to talk to the sqlite3 connection is bad anyway. This increases
flexibility and allows us to tailor the actual implementation to the
underlying DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Rusty Russell 049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell 855dff704c gossipd: test crc32 routines using test vectors from PR.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-27 12:35:25 +02:00
trueptolemy 4929034a40 json: Make payment_hash use `json_add_sha256` 2019-08-21 09:32:21 +08:00
lisa neigut 0c96c89d67 db-fix: resolve crash on fundchannel
Fixes error introduced by 1dbdc74bc where a new fundchannel
can cause a crash after start if the max dbid is for a closed
channel.
2019-08-10 02:52:13 +00:00
Rusty Russell 979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell cc70b9c4ec wire: use common/bigsize routines
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell 39e8e98b49 options: make option registration a single function.
No code changes, just move.

Put all the dev options into the one function, and register (and
comment on) the early args first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 05:16:22 +00:00
Rusty Russell f68c9fa9c9 opt: make sure early cmdline options override config file.
I noticed that --network=regtest didn't override 'network=bitcoin' in
the config file.

Normally we parse the config file first, then the commandline (so the cmdline
wins).  But for early options, we do cmdline first so we can find the config
file.  That was fine when the only early option was the location of the
config file, but now it includes plugins and the network setting.

So do a boutique cmdline parse *just* to find the config file, then parse
the config file early options, then the cmdline early options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 05:16:22 +00:00
Rusty Russell dd79813a75 common: add peer_error flag to treat this error as "soft".
The spec says to close the channel if they send us an error, but we
need to be more lenient to preserve channels with other
implementations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-26 03:53:03 +00:00
darosior b5bb7f191f Plugins: Add a notification for invoice payment
Similarly to the 'invoice_payment' hook
2019-07-25 11:19:47 +08:00
lisa neigut 32eaae0cb9 wire-gen: move in-house wire delcarations to new format
tidying things up!
2019-07-24 06:31:46 +00:00
Rusty Russell fb6870c139 param: implement helpers for multiplex commands.
Our previous param support was a bit limited in this case.

We create a dev- command multiplexer, so we can exercise it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-16 21:39:14 +00:00
Christian Decker 86d4362b65 wallet: Don't delete channels from DB, mark them closed.
Since we now have a couple of long-lived dependents it is time we stop
removing channels from the table once they are fully closed, and instead just
mark them as closed. This allows us to keep forwards and transactions foreign
keys intact, and it may help us debug things after the fact.

Fixes #2028

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-30 16:41:30 +09:30
lisa neigut 7046d0220c makefiles: move all unit tests under `make check-units`
Isolate unit tests under their own make directive.
2019-06-30 16:41:30 +09:30
Rusty Russell f1bea55395 lightningd: fix occasional missing txid detection.
I was working on rewriting our (somewhat chaotic) tx watching code
for 0.7.2, when I found this bug: we don't always notice the funding
tx in corner cases where more than one block is detected at
once.

This is just the one commit needed to fix the problem: it has some
unnecessary changes, but I'd prefer not to diverge too far from my
cleanup-txwatch branch.

Fixes: #2352
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-28 03:31:43 +00:00
Rusty Russell f1b57063f7 bitcoin/tx: use fromwire_fail in pull_bitcoin_tx.
This is the correct way to mark failure: it also sets *max to 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 03:56:59 +00:00
Rusty Russell d817735dc2 lightningd: use ccan/json_out.
This is now a fairly simple transition, which only effects the internals
of json_stream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 7f75043ab2 lightningd: tighten interal json_stream API.
Move it closer to ccan/json_out, in preparation for using that as a
replacement.

In particular:

1. Add a 'quote' field in json_add_member.
2. json_add_member now always escapes if 'quote' is true.
3. json_member_direct is exposed to allow avoiding of escaping.
4. json_add_hex can use this, so no longer needs to be in json_stream.c.
5. We don't make JSON manually, but always use helpers.
6. We now flush the stream (wake reader) only when we close it, or mark
   command as pending.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell bb7bbd03c5 lightningd: have json_stream_success start the "result" object.
"result" should always be an object (so that we can add new fields),
so make that implicit in json_stream_success.

This makes our primitives well-formed: we previously used NULL as our
fieldname when calling the first json_object_start, which is a hack
since we're actually in an object and the fieldname is 'result' (which
was already written by json_object_start).

There were only two cases which didn't do this:
1. dev-memdump returned an array.  No API guarantees on this.
2. shutdown returned a string.

I temporarily made shutdown return an empty object, which shouldn't
break anything, but I want to fix that later anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 220449e1cd ccan: import ccan/json_out and ccan/json_escape.
These are generalized from our internal implementations.

The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.

The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell ad24d94c7b lightningd: make 'spendable_msatoshi' more realistic.
Take into account the fee we'd have to pay if we're the funder, and
also drop to 0 if the amount is less than the smallest HTLC the peer
will accept.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
Christian Decker b6b548a983 wallet: Rip out the txtypes type in favor of enum wallet_tx_type
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Christian Decker 05dbf1a2a9 onchaind: Store and annotate close transaction when we drop to chain
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Christian Decker 7f898aa2a4 wallet: Annotate funding transaction in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
trueptolemy 96135dab5e log: add 'warning' notification when log
- Related Changes for `warning` notification

Add a `bool` type parameter in `log_()` and `lov()`, this `bool` flag
 indicates if we should call `warning` notifier.

1) The process of copying `log_book` of every peer to the `log_book` of
`ld` is usually included in `log_()` and `lov()`, and it may lead to
repeated `warning` notification. So a `bool`, which explicitly indicates
if the `warning` notification is disabled during this call, is necessary
.
2) The `LOG_INFO` and `LOG_DEBUG` level don't need to call
warning, so set that `bool` paramater as `FALSE` for these log level and
only set it as `TRUE` for `LOG_UNUAUSL`/`LOG_BROKEN`. As for `LOG_IO`,
it use `log_io()` to log, so we needn't think about notifier for it.
2019-06-07 01:23:51 +00:00
Rusty Russell e1dbc0b12b wallet: clean reserved inputs on startup.
We reserve inputs when we're going to send a transaction, but we don't
unreserve them if we crash.  This is most graphically demonstrated by
the txprepare case, which makes it easier to trigger.

Instead, we should query bitcoind to see whether the tx made it out or
not, as we would do manually with dev-rescan-outputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Rusty Russell 1a3886c116 wallet: keep a list of unreleased transactions.
We're going to use this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Rusty Russell 7e02fbe6ec lightningd: add json_add_tx helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
lisa neigut a4a193840f json: add a param parser for a txid 2019-06-06 04:47:44 +00:00
Rusty Russell 5591c0b5d8 gossipd: don't send gossip stream, let per-peer daemons read it themselves.
Keeping the uintmap ordering all the broadcastable messages is expensive:
130MB for the million-channels project.  But now we delete obsolete entries
from the store, we can have the per-peer daemons simply read that sequentially
and stream the gossip itself.

This is the most primitive version, where all gossip is streamed;
successive patches will bring back proper handling of timestamp filtering
and initial_routing_sync.

We add a gossip_state field to track what's happening with our gossip
streaming: it's initialized in gossipd, and currently always set, but
once we handle timestamps the per-peer daemon may do it when the first
filter is sent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell 38d2899fbb common/per_per_state: generalize lightningd/peer_comm Part 1
Encapsulating the peer state was a win for lightningd; not surprisingly,
it's even more of a win for the other daemons, especially as we want
to add a little gossip information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Christian Decker 1561ffaea0 hooks: Add cltv_expiry_delta to the htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Corné Plooy <@bitonic-cjp>
2019-06-04 00:27:15 +00:00
ZmnSCPxj 48df6c8566 lightningd/io_loop_with_timers.c: Move mainloop to its own source file, have chaintopology use it.
Fixes: #2687
2019-05-31 17:57:10 +02:00
Rusty Russell 0b79538b18 lightningd: hang up on clients if they make us run out of memory.
This happened with the 800M JSON for the MCP listchannels on the raspberry
pi, and tal calls abort() by default.

We switch to raw malloc here; we could override the error hook for
tal, but this is neater since we're doing low-level things anyway,

I tested it manually with this patch:

   diff --git a/lightningd/json_stream.c b/lightningd/json_stream.c
   index cec9f5771..206ba37c0 100644
   --- a/lightningd/json_stream.c
   +++ b/lightningd/json_stream.c
   @@ -43,6 +43,14 @@ static void free_json_stream_membuf(struct json_stream *js)
    	free(membuf_cleanup(&js->outbuf));
    }
    
   +static void *membuf_realloc_hack(struct membuf *mb, void *rawelems,
   +				 size_t newsize)
   +{
   +	if (newsize > 1000000000)
   +		return NULL;
   +	return realloc(rawelems, newsize);
   +}
   +
    struct json_stream *new_json_stream(const tal_t *ctx,
    				    struct command *writer,
    				    struct log *log)
   @@ -53,7 +61,7 @@ struct json_stream *new_json_stream(const tal_t *ctx,
    	js->reader = NULL;
    	/* We don't use tal here, because we handle failure externally (tal
    	 * helpfully aborts with a msg, which is usually right) */
   -	membuf_init(&js->outbuf, malloc(64), 64, membuf_realloc);
   +	membuf_init(&js->outbuf, malloc(64), 64, membuf_realloc_hack);
    	tal_add_destructor(js, free_json_stream_membuf);
    #if DEVELOPER
    	js->wrapping = tal_arr(js, jsmntype_t, 0);

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-22 11:28:44 +00:00
Rusty Russell 401bd9f8ef json: rename json_add_amount_sat to json_add_amount_sat_compat.
New fields don't have to be spelled out twice.

The raw version are called _only, so we don't miss a call
accidentally.  We can rename them when we finally deprecated old
fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-20 20:31:07 -04:00
Rusty Russell f3d30f1267 openingd: check with lightningd when we receive an offer.
Instead of lightningd telling us when it's ready, we ask it.
This also provides an opportunity to have a plugin hook at this point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-20 20:31:07 -04:00
trueptolemy 0f429853fc wallet: cleanup the tal context in wallet_channels_load_active() and wallet_stmt2channel()
The original idea is to "tal" channel on the "ctx"(In fact, we'd like to set ctx as "ld").
But we already tal channel on "ld" in new_channel(), so "ctx" is unused.
2019-05-18 02:35:33 +00:00
Rusty Russell eaac0d7105 lightningd: group crypto_state and fds into a convenient structure.
These are always handed to subdaemons as a set, so group them.  This makes
it easier to add an fd (in the next patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-13 05:16:18 +00:00
GreenAddress fb07265663 remove libbase58, use base58 from libwally (#2594)
* remove libbase58, use base58 from libwally

This removes libbase58 and uses libwally instead.

It allocates and then frees some memory, we may want to
add a function in wally that doesn't or override
wally_operations to use tal.

Signed-off-by: Lawrence Nahum lawrence@greenaddress.it
2019-04-30 23:07:31 +02:00
Simon Vrouwe c053dc9a6a lightningd: fix/refactor select_inchan for invoice route-hint, use fractional excess as weight
Refactored the weighted-reservoir-sampling algo to make it more straightforward.
It now uses the excess as fraction of capacity as weight. This favors channels that
are more _relatively_ unbalanced to be used for incoming payment.

Now passes test_invoice_routeboost_private() when using max fundamount=16777215.
2019-04-16 21:22:13 +00:00
Rusty Russell 6630b99cf7 lightningd: move local invoice resolution into invoice.c function.
We're going to make it async, so start by moving the core code into
invoice.c and having that directly call fail/success functions for the
htlc.

We add an extra check in fulfill_htlc() that the HTLC state is correct:
that can't happen now, but may once we're async.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-12 03:32:48 +00:00
Rusty Russell dc6d53e787 lightningd: don't bother pretty-printing JSON.
This doesn't result in a speedup for our benchmark, since we use the
cli tool which does the formatting.

MCP results from 5 runs, min-max(mean +/- stddev):
	store_load_msec:33422-36830(35196.2+/-1.2e+03)
	vsz_kb:2637488
	store_rewrite_sec:36.030000-37.630000(36.794+/-0.52)
	listnodes_sec:0.720000-0.950000(0.86+/-0.077)
	listchannels_sec:40.300000-41.080000(40.668+/-0.29)
	routing_sec:30.440000-31.030000(30.69+/-0.2)
	peer_write_all_sec:50.060000-52.800000(51.416+/-0.91)

MCP notable changes from 2 patches ago (>1 stddev):
	-listchannels_sec:48.560000-55.680000(52.642+/-2.7)
	+listchannels_sec:40.300000-41.080000(40.668+/-0.29)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell a2fa699e0e Use node_id everywhere for nodes.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.

We didn't get much space saving in gossipd, even though we should save
24 bytes per node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell b4455d517c common/node_id: new type.
Node ids are pubkeys, but we only use them as pubkeys for routing and checking
gossip messages.  So we're packing and unpacking them constantly, and wasting
some space and time.

This introduces a new type, explicitly the SEC1 compressed encoding
(33 bytes).  We ensure its validity when we load from the db, or get it
from JSON.  We still use 'struct pubkey' for peer messages, which checks
validity.

Results from 5 runs, min-max(mean +/- stddev):
	store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
	39475-39572(39518+/-36),2880732,41.150000-41.390000(41.298+/-0.085),2.260000-2.550000(2.336+/-0.11),44.390000-65.150000(58.648+/-7.5),32.740000-33.020000(32.89+/-0.093),44.130000-45.090000(44.566+/-0.32)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell 837a095d68 pubkey: rename PUBKEY_DER_LEN to PUBKEY_CMPR_LEN.
Pubkeys are not not actually DER encoding, but Pieter Wuille corrected
me: it's SEC 1 documented encoding.

Results from 5 runs, min-max(mean +/- stddev):
	store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
	38922-39297(39180.6+/-1.3e+02),2880728,41.040000-41.160000(41.106+/-0.05),2.270000-2.530000(2.338+/-0.097),44.570000-53.980000(49.696+/-3),32.840000-33.080000(32.95+/-0.095),43.060000-44.950000(43.696+/-0.72)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell 87cd628f52 log: truncate giant IO logging.
Adding a giant IO message simply causes it to be pruned immediately,
so truncate it if it's more than 1/64 the max size.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 04:41:43 +00:00
trueptolemy 92b40cb68a fix:Add infor about how many blocks needed until funding is confirmed
1. Rename channel_funding_locked to channel_funding_depth in
channeld/channel_wire.csv.
2. Add minimum_depth in struct channel in common/initial_channel.h and
change corresponding init function: new_initial_channel().
3. Add confirmation_needed in struct peer in channeld/channeld.c.
4. Rename channel_tell_funding_locked to channel_tell_depth.
5. Call channel_tell_depth even if depth < minimum, and still call
lockin_complete in channel_tell_depth, iff depth > minimum_depth.
6. channeld ignore the channel_funding_depth unless its >
minimum_depth(except to update billboard, and set
peer->confirmation_needed = minimum_depth - depth).
2019-04-07 23:45:35 +00:00
Michael Schmoock 5d0390f637 json: add cmd setchannelfee and wire to channeld
* adds the channeld_specific_feerates wire message 1029
* adds a json command handler
* adds u32 access methods for amount_msat
2019-03-15 02:48:18 +00:00
Rusty Russell bac9a594b8 wallet: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 3ac0e814d0 daemons: use amount_msat/amount_sat in all internal wire transfers.
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell b8e484b508 struct channel_config: use amount_sat / amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 85b8b25749 bitcoin/chainparams: use amount_sat / amount_msat
Simple changes, but ripples through the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 7e3928359a listpeers: add all the alternate "msat" and "sat" fields for channels.
These are undocumented, unfortunately, but at least that means I don't
have to update the docs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 3ba544bfde common/bolt11: use struct amount_msat
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 28ec65fd79 lightningd: add json_add_amount_msat and json_add_amount_sat helpers.
These create two fields, one old one which is purely numeric,
and a modern on with a suffix, eg "msatoshi" and "amount_msat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell a25e22737c common/json_tok: add param_msat / param_sat.
The current param_sat accepts "any": rename and move that to invoice.c
where it's called.  We rename it to param_msat_or_any and invoice.c
is our first (trivial) amount_msat user.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 7fad7bccba common/amount: new types struct amount_msat and struct amount_sat.
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.

The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 53423e8a55 lightningd: re-enable IO logging for JSON output.
Hex format is terrible, but sometimes it's the only way to tell WTF is
going on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Christian Decker ff0ddee56e json: Add utility to add a null-member
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-20 15:37:59 +01:00
Christian Decker ca3db290ba plugin: Add `connected` hook
This hook is used to let a plugin decide whether we should continue
with the connection normally, or whether we should be dropping the
connection. Possible use-cases include policy enforcement (dropping
connections based on previous interactions), draining a node by
allowing only peers with an active channel to reconnect, or
temporarily preventing a channel from making progress.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-20 15:37:59 +01:00
Rusty Russell b99293fbb6 short_channel_id: don't accept :-separated in JSON if --allow-deprecated-apis=false
We need to still accept it when parsing the database, but this flag
should allow upgrade testing for devs building on top

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-08 16:52:30 -08:00
Rusty Russell 5770e0c700 jsonrpc: probe sites for usage information once, at start.
We store it in a strmap.  This means we call the jsonrpc handler earlier,
so all callers need to call param() before they do anything else; only
json_listaddrs and json_help needed fixing.

Plugins still use '[usage]' for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell c506d42679 plugins: don't keep redundant jsonrpc pointer.
We have ld already, just use that in the one place we need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell dabdefefae jsonrpc: populate ld->jsonrpc ourselves, so we can use it.
Next patch will call commands to get usage inside jsonrpc_new(): to do
this it will need access to ld->jsonrpc, so we can't use the current
pattern.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell 1567238dd9 invoice: option to expose/not-expose private channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell e65b680807 json: move bitcoin/lightning specific helpers into common/json_helpers.
We don't need them in common/json, since lightning-cli doesn't need these,
but plugins want them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell 26dda57cc0 utils: make tal_arr_expand safer.
Christian and I both unwittingly used it in form:

	*tal_arr_expand(&x) = tal(x, ...)

Since '=' isn't a sequence point, the compiler can (and does!) cache
the value of x, handing it to tal *after* tal_arr_expand() moves it
due to tal_resize().

The new version is somewhat less convenient to use, but doesn't have
this problem, since the assignment is always evaluated after the
resize.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Christian Decker 26f17e87a3 plugin: Add connect and disconnect notifications
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-30 14:36:02 +01:00
Rusty Russell add822a072 jsonrpc: don't be coy with details for command_its_complicated().
Obviously the Facebook relationship status joke was a bit subtle, but I've
continued it anyway because I'm especially susceptible to Dad jokes.

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell 68bb36b210 json-rpc: make commands return 'struct command_result *'.
Usually, this means they return 'command_param_failed()' if param()
fails, and changing 'command_success(); return;' to 'return
command_success()'.

Occasionally, it's more complex: there's a command_its_complicated()
for the case where we can't exactly determine what the status is,
but it should be considered a last resort.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell bc41ab2cb9 param: make json_tok_ handlers all return command_result, rename to param_
Handers of a specific form are both designed to be used as callbacks
for param(), and also dispose of the command if something goes wrong.

Make them return the 'struct command_result *' from command_failed(),
or NULL.  

Renaming them just makes sense: json_tok_XXX is used for non-command-freeing
parsers too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell 93bf7c4839 param: make command sinks (fail/success) return a special type.
These routines free the 'struct command': a common coding error is not
to return immediately.

To catch this, we make them return a non-NULL 'struct command_result
*', and we're going to make the command handlers return the same (to
encourage 'return command_fail(...)'-style usage).

We also provide two sources for external use:
1. command_param_failed() when param() fails.
2. command_its_complicated() for some complex cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell d7e233e47d Move json and param core functionality into common, for plugins.
json_escaped.[ch], param.[ch] and jsonrpc_errors.h move from lightningd/
to common/.  Tests moved too.

We add a new 'common/json_tok.[ch]' for the common parameter parsing
routines which a plugin might want, taking them out of
lightningd/json.c (which now only contains the lightningd-specific
ones).

The rest is mainly fixing up includes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 3f16c9a665 param: abstract 'struct command' so param doesn't need to access it.
I want to use param functions in plugins, and they don't have struct
command.

I had to use a special arg to param() for check to flag it as allowing
extra parameters, rather than adding a one-use accessor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 86c517ac9b common/json: add context arg to json_parse_input.
All callers currently just hand the same arg twice, but plugins might
want this different.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 8015e7dcfb jsonrpc: add the obj token to the callback.
This (will) avoid the plugin having to walk back from the params object
as it currently does.

No code changes; I removed UNUSED and UNNEEDED labels from the other
parameters though (as *every* json_rpc callback needs to call param()
these days, they're *always* used).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell 6323cc1898 plugins: allow --dev-debugger=<pluginname>.
This currently just invokes GDB, but we could generalize it (though
pdb doesn't allow attaching to a running process, other python
debuggers seem to).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Christian Decker be7674ed6c plugin: Added .params.configuration to init call
This tells the plugin both the `lightning-dir` as well as the
`rpc-filename` to use to talk to `lightningd`. Prior to this they'd
had to guess.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-07 17:17:53 +01:00
Rusty Russell dffe2f516a signature: wrap almost all signatures in struct bitcoin_signature.
This is prep work for when we sign htlc txs with
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY.

We still deal with raw signatures for the htlc txs at the moment, since
we send them like that across the wire, and changing that was simply too
painful (for the moment?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Rusty Russell 23540fe956 common: make funding_tx and withdraw_tx share UTXO code.
They both do the same thing: convert utxos into tx inputs.  Share code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Rusty Russell a046af4416 lightningd/test: move some tests to common/ and channeld/
These unit tests stayed under lightningd/ even though the units they test
are elsewhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Mark Beckwith 164c76454c test: add json_tok_remove unit tests
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-12-06 02:06:03 +00:00
Rusty Russell eb03b33655 plugins: add and install built-in plugin dir, add clear and disable options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-05 01:22:55 +01:00
Christian Decker 3e1138951b plugin: Add pointer to jsonrpc so we can add new methods
Signed-off-by: Christian Decker <@cdecker>
2018-12-02 22:55:47 +00:00
Christian Decker 83775e7cea jsonrpc: Split the jsonrpc object creation from starting to listen
This is needed in order to be able to add methods while initializing
the plugins, but before actually moving to the config dir and starting
to listen.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-12-02 22:55:47 +00:00
Christian Decker 01c7bc5884 jsonrpc: Make an explicit jsonrpc struct
This wraps the listener, a separate log and the registered
commands. This is mainly needed once we dynamically add
sjson_command`s to the JSON-RPC.
2018-12-02 22:55:47 +00:00
Rusty Russell 55306fc3eb onchaind: wire up dev_memleak.
For onchaind we need to remove globals from memleak consideration;
we also change the htlc pointer to an htlc copy, which simplifies
things as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell d3ea9bf8bf channeld: wire up dev_memleak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Conor Scott 795bc1b3a7 [test] update test mocks for getinfo rpc update 2018-11-21 00:37:04 +00:00
Rusty Russell 3f5487e247 jsonrpc: dev_slowcmd, a command which starts output then delays.
This lets us explicitly test that our JSON outputs don't intermingle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell e17f69ce2d json_stream: disentangle JSON handling from command.
We promote 'struct json_stream' to contain the membuf; we only attach
the json_stream to the command when we actually call
json_stream_success / json_stream_fail.

This means we are closer to 'struct json_stream' being an independent
layer; the tests are already modified to use it directly to create
JSON.

This is also the first step toward re-enabling non-serial command
execution.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Christian Decker b02861bfe1 plugin: Give each plugin their own log-prefix
There is no good naming just yet, so we just number them 1 to n.
2018-11-13 00:44:50 +01:00
Christian Decker e6ef675ea1 plugin: Send the configure request once we collected all options
This is the final step to get the plugins working. After parsing the
early options (including `--plugin`), then starting and asking the
plugins for options, and finally reading in the options we just
registered, we just need to assemble the options and send them over.

Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker 10338983a6 plugin: Add logs to plugin and add method to kill a plugin
Also includes some sanity checks for the results returned by the
plugin, such as ensuring that the ID is as expected and that we have
either an error or a real result.
2018-11-13 00:44:50 +01:00
Christian Decker 8e83d43c39 opts: Split early from non-early args so plugins can register theirs
The idea is that `plugin` is an early arg that is parsed (from command
line or the config file). We can then start the plugins and have them
tell us about the options they'd like to add to the mix, before we
actually parse them.

Signed-off-by: Christian Decker <@cdecker>
2018-11-13 00:44:50 +01:00
Christian Decker d0de6e59c6 plugin: Start each plugin and setup the connection to it
Mostly copied from bitcoind.c
2018-11-13 00:44:50 +01:00
Christian Decker 59e37c12cd plugin: Add plugins to lightningd and register arguments 2018-11-13 00:44:50 +01:00
Rusty Russell 7eec2253e9 lightningd: fix up compile errors from bad merge.
The new listforwards was old-style, and the new API was merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-20 03:34:58 +00:00
Rusty Russell 0dcd66880c Rename `struct json_result` to `struct json_stream` (RENAMEONLY)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell c403415caa lightningd: format JSON directly into json connection membuf.
My test case is a mainnet gossip store with 22107 channels, and
time to do `lightning-cli listchannels`:

Before: `lightning-cli listchannels` DEVELOPER=0
	real	0m1.303000-1.324000(1.3114+/-0.0091)s

After:
	real	0m0.629000-0.695000(0.64985+/-0.019)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell e46ce0fc84 jsonrpc: declare up front whether a response is success or fail.
Such an API is required for when we stream it directly.  Almost all our
handlers fit this pattern already, or nearly do.

We remove new_json_result() in favor of explicit json_stream_success()
and json_stream_fail(), but still allowing command_fail() if you just
want a simple all-in-one fail wrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell 12adcda370 test/run-param: don't reach into json_result to get the string.
It's about to change, so refactor this first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell 305795b01e common/json: move JSON creation routines into lightningd/
It's the only user of them, and it's going to get optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossip.pydiff --git a/common/test/run-json.c b/common/test/run-json.c
index 956fdda35..db52d6b01 100644
2018-10-19 22:02:11 +00:00
lisa neigut b9331e5ac8 gossipd: parse and respect optional `htlc_maximum_msat`
If another channel has set the optional `htlc_maximum_msat` field,
we should correctly parse that field and respect it when drawing up
routes for payments.
2018-10-09 23:22:52 +00:00
Rusty Russell c8c3294a23 json-invoice: add routeboost, warnings.
We split json_invoice(), as it now needs to round-trip to the gossipd,
and uniqueness checks need to happen *after* gossipd replies to avoid
a race.

For every candidate channel gossipd gives us, we check that it's in
state NORMAL (not shutting down, not still waiting for lockin), that
it's connected, and that it has capacity.  We then choose one with
probability weighted by excess capacity, so larger channels are more
likely.

As a side effect of this, we can tell if an invoice is unpayble (no
channels have sufficient incoming capacity) or difficuly (no *online*
channels have sufficient capacity), so we add those warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell d1552bd1eb test/run-cryptomsg.c: move out of lightningd/ into common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Mark Beckwith 30b67c0334 param: call param() all the time
Now call param() even for commands that don't accept any parameters.

This is a bugfix of sorts.  For example, before you could call:

	bitcoin-cli getinfo blah

and the blah parameter would be ignored.

Now you will get an error: "too many parameters: got 1, expected 0"

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Mark Beckwith d91b94a812 param: add command mode
Added the concept of a "command mode".  The
behavior of param() changes based on the mode.

Added and tested the command mode of CMD_USAGE for
setting the usage of a command without running it.

Only infrastructure and test.  No functional changes.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Christian Decker cf52b7161f json-rpc: Remove upper limit for percentage
The `json_tok_percentage` parser is used for the `fuzzpercent` in `getroute` and
`maxfeepercent` in `pay`. In both cases it seems reasonable to allow values
larger than 100%. This has bitten users in the past when they transferred single
satoshis to things like satoshis.place over a route longer than 2 hops.
2018-09-20 02:41:28 +00:00
Rusty Russell 76f116daf1 lightningd: minor cleanups
Code changes:
1. Expose daemon_poll() so lightningd can call it directly, which avoids us
   having store a global and document it.
2. Remove the (undocumented, unused, forgotten) --rpc-file="" option to disable
   JSON RPC.
3. Move the ickiness of finding the executable path into subd.c, so it doesn't
   distract from lightningd.c overview.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell 168bec0974 lightningd: move channel/peer/htlc load into own function.
Also, wallet has no business wiring up HTLCs; move that code to
peer_htlcs.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell 3e53a63cf2 wallet: do wallet_invoice init during preparation.
We have a transaction anyway, and it's simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Mark Beckwith ca40cfa0ce param: started adding callback unit tests
Well its about time.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Mark Beckwith 8590dbedfb param: make json_tok_label non-static
Needed to do this so I could remove the implementation in
the run-param test.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Mark Beckwith 4f81cd3852 param: added json_tok_msat
This could have been a local static but its used by the run-param test,
so putting it in json.c made things easier.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Rusty Russell db3c387264 feerate: allow names 'urgent' 'normal' and 'slow'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell e0952ceff2 feerate: use suffix, not separate argument.
And, reluctantly, default to bitcoind style.
"It's wrong to be right too soon."

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell 2db77f5d1d gossipd: minor modifications for memleak detection to work.
1. Move the list to the start of `struct peer`: memleak walks the
   list correctly this way.
2. Don't create tal parent loop daemon->conn->daemon.

The second one is silly anyway: we exit via master_gone when the master
conn is closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Mark Beckwith a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 2a0c2601c0 param: upgraded json_tok_newaddr
Made it a local static since its a one-off.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 47555efc0d param: rebased master
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith e5918f4e5a param: upgraded json_tok_double
Also renamed old version to json_to_double for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith 8f17191099 param: upgraded json_tok_tok to advanced callback
This was a very simple change and allowed us to remove the special
`json_opt_tok` macro.

Moved the callback out of `common/json.c` to `lightningd/json.c` because the new
callbacks are dependent on `struct command` etc.
(I already started on `json_tok_number`)

My plan is to:
	1. upgrade json_tok_X one a time, maybe a PR for each one.
	2. When done, rename macros (i.e, remove "_tal").
	3. Remove all vestiges of the old callbacks
	4. Add new callbacks so that we no longer need json_tok_tok!
	   (e.g., json_tok_label, json_tok_str, json_tok_msat)

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-13 23:46:35 +00:00
Mark Beckwith bd5bf1f168 Enhanced param parsing
[ Squashed into single commit --RR ]

This adds two new macros, `p_req_tal()` and `p_opt_tal()`. These support
callbacks that take a `struct command *` context.  Example:

	static bool json_tok_label_x(struct command *cmd,
                                      const char *name,
				      const char *buffer,
				      const jsmntok_t *tok,
				      struct json_escaped **label)

The above is taken from the run-param unit test (near the bottom of the diff).
The return value is true on success, or false (and it calls command_fail itself).

We can pretty much remove all remaining usage of `json_tok_tok` in the codebase
with this type of callback.
2018-08-10 02:15:30 +00:00
Rusty Russell 52303029aa fundchannel: cap 'all' at 2^24-1 satoshi.
The easiest way to do this is to play with the 'wallet_tx' semantics
and have 'amount' have meaning even when 'all_funds' is set.

Note that we change the string 'Cannot afford funding transaction' to
'Cannot afford transaction' as this code is also used for withdrawls.

Inspired-by: molz on #c-lightning
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:46:41 +02:00
Rusty Russell 5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Mark Beckwith 1851ebbc56 programer -> developer
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith 381c32af75 Remove sigaction from run-param
Developer errors result in command_fail being called
just like other errors.  The bad_programmer() Test is now updated
and passing.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith 2ca1786931 Added check_fail function to run-param
Just a utility function to check that command_fail was called.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith b7203b0c0c Remove Developer Asserts from param.c
They now just call command_fail() and cause param() to return false.
Temporarily disabled all the run-param.c tests that redirect
asserts so CI would still pass.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Rusty Russell 3d3d2ef9af gossipd: remove connectd functionality, enable connectd.
This patch guts gossipd of all peer-related functionality, and hands
all the peer-related requests to channeld instead.

gossipd now gets the final announcable addresses in its init msg, since
it doesn't handle socket binding any more.

lightningd now actually starts connectd, and activates it.  The init
messages for both gossipd and connectd still contain redundant fields
which need cleaning up.

There are shims to handle the fact that connectd's wire messages are
still (mostly) gossipd messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 92d66a5451 gossipd: take connectd fd on initialization.
connectd has a dedicated fd to gossipd, so it can ask for a new gossip_fd
for a peer.

gossipd has a standalone routine to create a remote peer (this will
eventually be the only way gossipd creates a new peer).

For now lightningd creates a socketpair but doesn't run connectd, so
gossipd never sees any requests on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 289e39a0a3 bitcoin/pubkey: add pubkey_from_secret.
Really, we should have a 'struct point' since we don't use all points
as pubkeys.  But this is the minimal fix to avoid type cast nastiness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Mark Beckwith f850849486 Modern param style for all remaining files
Removed `json_get_params`.

Also added json_tok_percent and json_tok_newaddr. Probably should
have been a separate PR but it was so easy.

[ Squashed comment update for gcc workaround --RR ]
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-20 01:14:02 +00:00
Mark Beckwith 19a0b78fec param: renamed files
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-16 03:51:21 +00:00
Mark Beckwith cf12130627 params: shortened names
This is a cosmetic change only. No functional changes.

I shortened the names of macros and changed param_parse() to param().

Also went through params.h with a fine-toothed comb and updated the comments
to reflect the current API.

I wanted to change the files:

	params.c -> param.c
	params.h -> param.h
	run-params.c -> run->param.c

but that confused `git diff` for params.h so its best left for another PR.

I'm keeping #1682 updated locally with all these changes.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 23:44:50 +00:00
Rusty Russell cf86c74870 params: add helper to provide default initialization.
@wythe points out that many cases want a default value, not NULL.
Nicer to do it in the param_parse() call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-13 11:21:39 +00:00
Rusty Russell b14cc0c9f7 lightningd/params: fix typesafe check.
typesafe_cb isn't suitable here, as it is simply a conditional cast,
and the result is passed through '...' and doesn't matter.

Reported-by: @wythe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 23:49:25 +00:00
Mark Beckwith fc2d955b01 Fixed spacing in run-params
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith 7d9ad89010 params: removed the param_opt_tok macro
There doesn't seeem to be a need for this anymore (unless I'm missing something).
I added the sendpay_nulltok() unit test to confirm.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith 1b50ea2abd params: removed tal context.
@rustyrussell showed we don't need temporary objects for params.

This means params no longer need a tal context.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Rusty Russell 32ccfa5b29 test/run-params: suppress stderr for expected failures.
And use err() instead of perror/exit(0) (we should have exit(1) there anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 6ff901d7b0 params: simplify lifetimes of params.
@wythe points out we don't need to keep the around now param_is_set()
is removed.  We can in fact go further and avoid marshalling them into
temporary objects at the caller altogether.

This means internally we have an array of struct param, rather than an
array of 'struct param *', which causes most of the noise in this
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 3f6f9e6fe0 param: make sure the name is a string literal.
We're using a macro anyway, so appending "" make it a compile-time check.

Complicates testing a bit, since we actually use generated names there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 899ff02e36 params: use asort.
It's a little neater than qsort here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell 9f83a9ae4d params: make optional args do allocation for you.
This is a bit more natural, IMHO.  The only issue is that json_tok_tok is
special, so we end up with param_opt_tok() if you really want an optional
generic token.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Mark Beckwith 4d1d0438e1 Typesafe callback system for parsing json
This is part of #1464 and incorporates Rusty's suggested updates from #1569.

See comment in param.h for description, here's the basics:

	unsigned cltv;
	const jsmntok_t *note;
	u64 msatoshi;
	struct param * mp;

	if (!param_parse(cmd, buffer, tokens,
			 param_req("cltv", json_tok_number, &cltv),
			 param_opt("note", json_tok_tok, &note),
			 mp = param_opt("msatoshi", json_tok_u64, &msatoshi),
			 NULL))
		return;

	if (param_is_set(mp))
		do_something()

There is a lot of developer mode code to make sure we don't make mistakes,
like trying to unmarshal into the same variable twice or adding a required param
after optional.

During testing, I found a bug (of sorts) in the current system.  It allows you
to provide two named parameters with the same name without error; e.g.:

	# cli/lightning-cli -k newaddr addresstype=p2sh-segwit addresstype=bech32
	{
		  "address": "2N3r6fT65PhfhE1mcMS6TtcdaEurud6M7pA"
	}

It just takes the first and ignores the second.  The new system reports this as an
error for now.  We can always change this later.
2018-07-05 00:19:16 +00:00
Rusty Russell e549bc6ecf lightningd: fix up BOLT references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Christian Decker 2415f48723 topo: Tell chain_topology about the min and max block height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Christian Decker 024dca0fff wallet: Return both min and max block heights
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
ZmnSCPxj 097a8e72d1 channel_control: Forget if unconfirmed for a long time and we are fundee.
We should forget this as it is a potential DoS if we remember every
funding txid that an attacker gave in a `funding_created` but never
broadcasted.
2018-05-23 14:37:32 -07:00
Rusty Russell c8cc8fd83f option cleanup: --dev-bitcoind-poll
Make --bitcoind-poll a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell a8c0bca6a8 gossipd: take over negotiation of autogenerated Tor addresses.
For the moment, this is a straight handing of current parameters through
from master to the gossip daemon.  Next we'll change that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell e93682e3bf status: make status_io a more generic mechanism.
Currently it's always for messages to peer: make that status_peer_io and
add a new status_io for other IO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell ef09961713 lightningd: rewrite tor service interaction to be synchronous, robust.
There's no reason to do this async, and far easier to follow using normal
read/write.

The previous parsing was deeply questionable, using substring searches
only, and relying on the fact that a single non-blocking read would get
the entire response.  This is changed to do (somewhat) proper parsing
using ccan/rbuf.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 85eff42d9a common/tor: move into lightningd.
This is simply the code to set up the automatic hidden service, so move
it into lightningd.

I removed the undefined parse_tor_wireaddr, and added a parameter name
to the create_tor_hidden_service_conn() declaration for update-mocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato 877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 2ecfbf46e3 hsmd: drop newdir logic.
Originally we were supposed to tell the HSM we had just created the directory,
otherwise it wouldn't create a new seed.  But we modified it to check if
there was a seed file anyway: just move that logic into a branch of hsmd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Rusty Russell f083a699e2 gossipd: separate init and activate.
This means gossipd is live and we can tell it things, but it won't
receive incoming connections.  The split also means that the main daemon
continues (eg. loading peers from db) while gossipd is loading from the store,
potentially speeding startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-30 12:01:36 +02:00
Rusty Russell 91d149b990 lightningd: insert db statement checking in io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
practicalswift abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
Christian Decker 96352858d6 chaintopology: Simplify rescan offset computation
Simplification of the offset calculation to use the rescan parameter, and rename
of `wallet_first_blocknum`. We now use either relative rescan from our last
known location, or absolute if a negative rescan was given. It's all handled in
a single location (except the case in which the blockcount is below our
precomputed offset), so this should reduce surprises.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker 4b22760cf9 onchaind: Replay stored channeltxs to restore onchaind state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Rusty Russell 21fbae6df8 openingd: ensure that initial channel can cover fees and reserve.
This is probably covered by our "channel capacity" heuristic which
requires the channel be significant, but best to be explicit and sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 1764d6c907 grind_htlc_tx_fee: benchmark.
Takes 15 seconds on my laptop to do the worst-case grind:

	$ onchaind/test/run-grind_feerate 250001
	250001 iterations in 15893 msec = 63574 nsec each

It's not worth optimizing as it's 75% in libsecp:

    29.65%  run-grind_feera  run-grind_feerate  [.] secp256k1_fe_mul_inner
    23.51%  run-grind_feera  run-grind_feerate  [.] secp256k1_fe_sqr_inner
    11.04%  run-grind_feera  run-grind_feerate  [.] secp256k1_gej_double_var.part.6.constprop.34
     9.56%  run-grind_feera  run-grind_feerate  [.] secp256k1_scalar_reduce_512
     5.70%  run-grind_feera  run-grind_feerate  [.] Round

Even forcing a compile with -O3 -flto, it's only 13883 msec = 55534 nsec each.

Fixes: #291
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 1a4a59d221 common/daemon: common routines for all daemons.
In particular, the main daemon and subdaemons share the backtrace code,
with hooks for logging.

The daemon hook inserts the io_poll override, which means we no longer
need io_debug.[ch].  Though most daemons don't need it, they still link
against ccan/io, so it's harmess (suggested by @ZmnSCPxj).

This was tested manually to make sure we get backtraces still.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Rusty Russell 54431d2b08 lightningd: escape our own alias when we print it in logs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
ZmnSCPxj 5737853123 options: Add --autocleaninvoice-* options. 2018-03-20 17:25:51 +01:00
Rusty Russell 0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell ef2a063169 utils: add a global tmpctx.
I did a brief audit of tmpctx uses, and we do leak them in various
corner cases.  Fortunely, all our daemons are based on some kind of
I/O loop, so it's fairly easy to clean a global tmpctx at that point.

This makes things a bit neater, and slightly more efficient, but also
clearer: I avoided creating a tmpctx in a few places because I didn't
want to add another allocation.  With that penalty removed, I can use
it more freely and hopefully write clearer code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell e76f863329 txfilter: clean up prototypes.
Use const, add TAKES to declaration.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell b7ed5670d5 lightningd: close and reopen db across fork for daemonize
Fixes: #1092
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-27 00:09:34 +01:00
practicalswift 91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
Rusty Russell a2c6ec6c9b lightningd: use tal_link for log_book.
BackgroundL Each log has a log_book: many logs can share the same one,
as each one can have a separate prefix.

Testing tickled a bug at the end of this series, where subd was
logging to the peer's log_book on shutdown, but the peer was already
freed.  We've already had issues with logging while lightningd is
shutting down.

There are times when reference counting really is the right answer,
this seems to be one of them: the 'struct log' share the 'struct
log_book' and the last 'struct log' cleans it up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Christian Decker b2819f9f97 wallet: Add check that wallet matches the network on startup
Adds a simple check that compares genesis-blockhashes from the
chainparams against the blockhash that the wallet was created
with. The wallet is network specific, so mixing is always a bad idea.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-17 11:36:17 +00:00
Rusty Russell 6620305606 wallet: use last_processed_block to determine scan start.
With fallback depending on chainparams: this means the first upgrade
will be slow, but after that it'll be fast.

Fixes: #990
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 0e93fb932a lightningd: bitcoind and topology routines take channel, not peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell 409fef582d subd: keep pointer to channel, not peer.
This rolls through many other functions, making them take channel not peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell 32411de90e lightningd: split struct peer into struct peer and struct channel.
Much like the database; peer contains id, address, channel contains
per-channel information.  Where we create a channel, we always create
the peer too.

For the moment, peer->log and channel->log coexist side-by-side, to
reduce some of the churn.

Note that this changes the API to dev-forget-channel: if we have more
than one channel, we insist they specify the short-channel-id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell 15eaf56d79 wallet: add ld pointer.
This will be required to give it direct access to the ld->peers list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell de56dc0ffc common: add logging for peer packets, with status_io.
We log the plaintext, not the encrypted ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Rusty Russell 84bf60f934 status: add multiple levels of logging.
status_trace maps to status_debug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
ZmnSCPxj 1e6747c28e wallet: Pass in timers object during construction.
In preparation for expiration.
2018-02-05 08:52:42 +00:00
ZmnSCPxj 93dc90990f invoices: Extensive reorganization of invoice system. 2018-01-16 13:03:54 +01:00
Christian Decker 0bb264e1a2 wallet: Added unilateral close info to utxo
This is necessary to grad the their_unilateral/to-us outputs since
they aren't being harvested by `onchaind`

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Rusty Russell 36316957e3 lightningd: set parent correctly for loaded peers.
The current code makes the channel the parent, which is a cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 1b41335121 chain_topology: two-stage startup.
Load the first block we're possibly interested in, then load the peers so
we can restore the tx watches, then finally replay to the current tip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 7b735e5de8 lightningd: scan blockchain from first possible block.
Eventually we want to save blockchain in db to avoid this scan, but
for the moment, we need to reload as far back as we may be interested in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
practicalswift 66e7c54810 Avoid out-of-bounds read in increase(u32 feerate_per_kw) 2018-01-02 18:37:22 +01:00
practicalswift 84dd65485c Remove duplicate includes 2017-12-28 16:09:24 +01:00
Rusty Russell 0237e0b28c bitcoin: create new wrapper type bitcoin_txid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_txid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Christian Decker ee25547576 Re-enable builds with DEVELOPER=0
Two changes:
 - Fixed the function signature of noleak_ to match in both
   configurations
 - Added memleak.o to linker for tests

Generating the stubs for the unit tests doesn't really work since the
stubs are checked in an differ between the two configurations, so
adding memleak to the linker fixes that, by not requiring stubs to be
generated in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 12:43:10 +01:00
Rusty Russell dfc132b2fe memleak: add backtrace to allocations.
We use the tal notifiers to attach a `backtrace` object on every
allocation.

This also means moving backtrace_state from log.c into lightningd.c, so
we can hand it to memleak_init().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell c956d9f5eb lightningd: tal memleak detection, dev-memleak command.
This is a primitive mark-and-sweep-style garbage detector.  The core is
in common/ for later use by subdaemons, but for now it's just lightningd.
We initialize it before most other allocations.

We walk the tal tree to get all the pointers, then search the `ld`
object for those pointers, recursing down.  Some specific helpers are
required for hashtables (which stash bits in the unused pointer bits,
so won't be found).

There's `notleak()` for annotating things that aren't leaks: things
like globals and timers, and other semi-transients.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 0610f66c34 bolt11: handle r value fee spec change.
We don't use it yet, but now we'll decode correctly.

See: https://github.com/lightningnetwork/lightning-rfc/pull/317
lightning-rfc commit: ef053c09431442697ab46e83f9d3f86e3510a18e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-12 11:45:44 +01:00
practicalswift 4452e3f3f1 Remove redundant code 2017-12-11 03:38:37 +00:00
Christian Decker c29923a623 topology: Add transaction filtering to connect_block
The filter is being populated while initializing the daemon and by
adding new keys as they are being generated. The filter is then used
in connect_block to identify transactions of interest.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Rusty Russell f1e4cad9d4 feerate: use u32 everywhere.
The wire protocol uses this, in the assumption that we'll never see feerates
in excess of 4294967 satoshi per kiloweight.

So let's use that consistently internally as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 40315bfb91 test: fix dependencies.
Test objects must be added to $(ALL_OBJS) so they correctly depend on
CCAN headers etc.

Also, each test in a subdir must depend on headers and src in the parent
directory, as it will often #include them directly.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 37b4ab306e run-channel.c: move under channeld/
This also fixes dependencies, since it actually depends on channeld objects.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 5fb4577890 channeld/full_channel: fix incorrect reutrn from channel_rcvd_revoke_and_ack.
It was always returning false; it was supposed to return true if
we had added pending changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 4db460903a htlc_tx: wire up the htlc points.
All the callers need to pass it in: currently channeld and openingd just
fake it by copying the payment point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell fe5614a489 basepoints/secrets: add htlc entry
Currently derive_basepoints just sets it to match the payment point/secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell 956350e62e lightningd: check peers don't leave dangling HTLCs when they die.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-11 01:29:51 +00:00
Rusty Russell 4fb472b7a7 db: we must always be in a transaction, remove nested, call fatal()
We save location where transaction was started, in case we try to nest.
There's now no error case; db_exec_mayfail() is the only one.

This means the tests need to override fatal() if they want to intercept
these errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell 7586f3ed54 timers: wrap all calls in transactions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell a55ce607a1 bitcoind: contain ld pointer.
This is a subset of a "bitcoind: wrap callbacks in transaction." from
the everything-in-transaction branch, but we need the ld pointer now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell 112ae0d0f5 common/test/run-json: test JSON escaping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell ac92138603 common: remove unused assert() headers.
Auditing for assert/abort in common/ code used by lightningd, this is all
that showed up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell b6a2b8c58b Add --rgb and --alias options.
And derive random ones from nodeid if they don't choose.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 09:16:14 +00:00
Rusty Russell 7e022b522c gossipd: don't try to handle padding inside fromwire_ipaddr.
It makes it impossible to embed an ipaddr in another structure, since we
always try to skip over any zeroes, which may swallow a following field.

Do the skip specially for the case where we're parsing routing messages:
we never use padding for our own internal messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell 474887512d gossipd: rewrite to do the handshake internally.
Now the flow is much simpler from a lightningd POV:

1. If we want to connect to a peer, just send gossipd `gossipctl_reach_peer`.
2. Every new peer, gossipd hands up to lightningd, with global/local features
   and the peer fd and a gossip fd using `gossip_peer_connected`
3. If lightningd doesn't want it, it just hands the peerfd and global/local
   features back to gossipd using `gossipctl_handle_peer`
4. If a peer sends a non-gossip msg (eg `open_channel`) the gossipd sends
   it up using `gossip_peer_nongossip`.
5. If lightningd wants to fund a channel, it simply calls `release_channel`.

Notes:
* There's no more "unique_id": we use the peer id.
* For the moment, we don't ask gossipd when we're told to list peers, so
  connected peers without a channel don't appear in the JSON getpeers API.
* We add a `gossipctl_peer_addrhint` for the moment, so you can connect to
  a specific ip/port, but using other sources is a TODO.
* We now (correctly) only give up on reaching a peer after we exchange init
  messages, which changes the test_disconnect case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell e137e2527f Update BOLT references with typo fixes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Christian Decker 11903aed6c wallet: Wiring in invoice persistence into JSON-RPC and master
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker c1493ae60c lightningd: Added loading of HTLCs upon startup
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Rusty Russell 66c1da795f lightningd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell 7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell ce160d9b17 lightnind: _ dev-disconnect argument to suppress commit timer.
Required for catching daemon in exact state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell d39c27a304 lightningd: search path for our own directory.
Needed slight reformatting of log.h for stubs autogeneration.

Fixes: #277
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-15 13:44:34 +02:00
Rusty Russell 4f5e7a4d92 lightningd/Makefile: clean up.
1. Remove reference to old $(LIGHTNINGD_OLD_LIB_OBJS) var (in handshaked too).
2. Make check depend directly on unit tests, insteadof weird lightningd/tests
   variable.
3. check-source-bolt and check-whitespace are automatic for $(ALL_TEST_PROGRAMS)
   so we don't need them here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-11 14:29:20 +02:00
Christian Decker 0137aea7fc test: Don't assume size_t == u64 in tests 2017-09-08 16:56:07 +09:30
Rusty Russell 5889ad5fc4 dev-disconnect: add blackhole option.
To reproduce the next bug, I had to ensure that one node keeps thinking it's
disconnected, then the other node reconnects, then the first node realizes
it's disconnected.

This code does that, adding a '0' dev-disconnect modifier.  That means
we fork off a process which (due to pipebuf) will accept a little
data, but when the dev_disconnect file is truncated (a hacky, but
effective, signalling mechanism) will exit, as if the socket finally
realized it's not connected any more.

The python tests hang waiting for the daemon to terminate if you leave
the blackhole around; to give a clue as to what's happening in this
case I moved the log dump to before killing the daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 19:11:21 +02:00
Rusty Russell bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 739b163f8b Makefiles: simplify dependencies.
Gather all binaries and objects and make the depend on external
requirements and common headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell c6976cd947 shachain: always build 48 bit version.
No more special Makefile hacks required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell a37c165cb9 common: move some files out of lightningd/
Basically all files shared by different daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 04db39558d htlc_tx: use keyset abstraction.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30