Commit Graph

1068 Commits

Author SHA1 Message Date
Rusty Russell 5698a133c2 JSONRPC: rename getpeers to listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 34a994c7db JSONRPC: listpayments not return a naked array.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 361fdccfca JSONRPC: help returns an object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 843ede78c6 JSONRPC: listinvoices: wrap array in 'invoices' field.
This allows us to add other fields, such as version information,
warnings or invoiceless payments, later.

(Note: the deprecated listinvoice is unchanged)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 1a641c823c JSONRPC: Make listinvoice an deprecated alias for listinvoices.
This matches the other names, and also the return value is about to change.

This will be removed before release!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 6e703ad977 JSONRPC: listpayments can list just a specific bolt11 or payment_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 06c4f6ddca JSONRPC: add optional short_channel_id argument to listchannels
In order to just list one (though it may return two entries, one for each
channel direction!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 7fdb30cbda JSONRPC: rename getchannels to listchannels
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 8cf1870465 jsonrpc: add json_tok_short_channel_id helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell e1e7f289fb JSONRPC listnodes: return timestamp, alias and color.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell ced572b973 JSONRPC: getnodes: rename to listnodes.
Like listinvoice, and add optional 'id' parameter to ask about a
specific node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell e745572dfb JSONPRC: decodepay: print 'c' value.
If they want to call getroute manually, they'll need this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell 1f6392fa83 lightningd: --deprecated-api option to turn off deprecated APIs.
This can be used for upgrades to make sure you're not using deprecated
options, JSON commands, JSON fields, etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Christian Decker ced486e727 jsonrpc: Add the network name to the getinfo output
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-18 23:34:51 +01:00
ZmnSCPxj a88c73a41b invoices: Add `paid_timestamp` field.
Fixes: #615
2018-01-18 17:37:10 +01:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ a06f0dfeb1 Make `fundchannel` return txid too
For voyeurism.

Fixes: #402
2018-01-18 02:03:28 +00:00
Rusty Russell 9ed7041c46 bitcoind: if callback says don't call on error, dont.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell 38e8601cf6 wallet: abstract away delayed entry of wallet_payment.
For performance, we delay entering the 'wallet_payment' into the db
until we actually commit to the HTLC (when we have to touch the DB
anyway).

This opens a race where we can try to pay twice, and since it's not in
the database yet, we don't notice the duplicate.

So remove the temporary payment field from htlc_out, which was always
an uncomfortable hack, and make the wallet code abstract over the
deferred entry a little by maintaining a 'unstored_payments' list
and incorporating that in results.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell 469faa476e test_lightning.py: test for persistence.
Also note that failing an in-progress payment (instead of waiting) is
pretty weird.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell 02e05ba6ff pay: remove struct pay_command.
It's all in wallet_payment, which is persistent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell 0f8f273410 lightningd: move cmd pointer from struct pay_command directly into htlc_out.
Set if a command is responsible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell 559010f525 wallet: add path_secrets to payment table.
We need these to decode any returned errors.

We remove it from struct pay_command too, and load directly from db
when we need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell 2cbe5b65c7 wallet: add preimage to db.
We should be saving this, as it's our proof of payment.  Also, we return
it if they try to pay again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell dea0aef52f wallet: use wallet_payment only for *outgoing* payments.
Incoming payment information is completely covered by invoices.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 22:43:14 +01:00
ZmnSCPxj 93dc90990f invoices: Extensive reorganization of invoice system. 2018-01-16 13:03:54 +01:00
Rusty Russell 79dc44713b channeld: --ignore-fee-limits as a hack for fee disparities.
This, of course, should never be used.  But it helps maintain connections
for the moment while we dig deeper into feerates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-16 12:22:06 +01:00
practicalswift e91a8dff12 Change log level for some common debug messages from "info" to "debug" 2018-01-16 03:20:27 +00:00
Christian Decker 5319ff1bc0 bitcoind: Do not crash when getblock fails
This is a common occurence on pruned nodes. By calling the callback
upon failures, we communicate that we couldn't verify the txoutput. We
fail safe rejecting any channel we can't verify.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-15 19:29:01 +00:00
Rusty Russell 4b663e596a lightningd: don't crash when txout lookup fails.
Gossipd already correctly handles the "empty output means lookup failed" case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 19:29:01 +00:00
Rusty Russell c66df31674 subd: pass absolute path as argv[0].
This means we print out the correct path with --debugger, which
can be vital if there are multiple binaries (eg. compiled vs installed).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 19:26:08 +00:00
Rusty Russell 904a3e4ae3 MAX_FUNDING_SATOSHI: clean up def to be max, fix name, share with openingd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 12:45:12 +01:00
Christian Decker 351718e90f jsonrpc: Return an error if funding amount is too large
Fixes #516

Reported-by: @instagibbs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-15 12:45:12 +01:00
Rusty Russell dbef4a1827 pay: remove gratuitous check for msatoshitok being 'null'.
json_get_params does this for us.

Fixes: 78adf0b (pay: allow 'null' msatoshi field.)
Reported-by: ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 12:44:02 +01:00
Christian Decker dd599706cd peer: Save whenever we change last_was_revoke as well
Pulling up the save call from `peer_save_commitsig_received` into its
caller `peer_got_commitsig` and adding a call to
`peer_sending_commitsig`

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-15 12:43:22 +01:00
practicalswift 4214c5ad57 Use consistent formatting for CLI command descriptions 2018-01-15 04:20:30 +00:00
practicalswift caf57b24ea Use correct description for the "help" CLI command 2018-01-15 04:20:30 +00:00
practicalswift 447afa923d Avoid segfault on CLI command "dev-memleak" when $LIGHTNINGD_DEV_MEMLEAK is not set 2018-01-15 02:42:59 +00:00
Björge Dijkstra 44836ef509 Fix double free of message buffer.
Message buffer `why` is allocated in the `peer` context and also freed when peer is freed.
Only explicitly free the buffer when peer itself is not freed yet.
2018-01-15 02:12:10 +00:00
William Casarin 3c0d2813a0 bitcoind: properly handle spent outputs in gettxout
exit status is not enough to detect spent outputs. gettxout will return a
success exit code and 0 bytes.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-14 23:49:59 +00:00
Christian Decker 7fc8e3a4e2 gossip: Pass --channel-update-interval to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker 0419688b0c gossip: Added --channel-update-interval argument
We'll pass this down to gossip and make sure to re-announce/update
channels every so often. This is also used as a pruning timer, i.e.,
channels that have not been updated in 2 x channel-update-interval
will be pruned from the local view.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Rusty Russell 6d7c8c8dd0 doc: manpages for pay, listpayments and decodepay.
Also contains some help message clarifications.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell a1a0b33577 pay: allow 'null' msatoshi field.
Since most callers use positional arguments, we should allow a 'null'
literal where we require no value at all.

Also adds some more value tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell 2f2fb0c2a1 invoice: add msatoshi_received field.
Paid invoices need to know how much was actually paid: both for the case
where no 'msatoshi' amount was specified, and for the normal case, where
clients are permitted to overpay in order to help them disguise their
payments.

While we migrate the db, we leave this field as 0 for old paid
invoices.  This is unhelpful for accounting, but at least clearly
indicates what happened if we find this in the wild.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell 66885163c9 JSON: Rename rhash to payment_hash in delinvoice, invoice, listinvoice, waitinvoice, waitanyinvoice.
'rhash' is the old terminology, but 'payment_preimage' and
'payment_hash' were decided on for the BOLTs, so we should fix that here.

We still use rhash internally, but that's much easier to fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell cc4caf9d69 invoice: common routine for invoice to json.
Different commands (listinvoice, delinvoice, waitinvoice,
waitanyinvoice) returned different fields, as not all were updated.
This makes them uniform.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell cce432b77f wallet_invoice_nextpaid: return a struct invoice.
This reuses the same code internally, and also now means that we deal
correctly with "any" msatoshi invoices: the old code would a return
'msatoshi' of 0 in that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
ZmnSCPxj 83e76e3ac3 features: Move feature-handling code to a common/features.c source. 2018-01-13 11:29:42 +01:00
ZmnSCPxj 2d8304f176 pay: Change online help of 'listinvoices' to use 'msatoshi' as per actual response returned. 2018-01-13 03:06:05 +00:00
ZmnSCPxj c402d834e7 invoice: Rename 'amount' argument of 'invoice' RPC to 'msatoshi'.
The manfile and the online help use 'msatoshi', the returned
response uses 'msatoshi', nearly every invoice-related
monetary amount is labelled 'msatoshi' and not 'amount'.
2018-01-13 03:06:05 +00:00
William Casarin 2e796ffa67 peer_control: send addrhints in activate_peer
Since we now have addresses in the database, we can resend them as hints to
gossipd on startup.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-12 19:47:36 +01:00
Rusty Russell 2bc2502b23 wire: move extract_channel_id to here.
It can be useful for other daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-12 09:43:01 +01:00
Igor Cota e97857847a peer_control: make hardcoded feerates consistent between json_fund_channel and peer_offer_channel
Stops some transactions from failing on the other side due to insufficient funds.
2018-01-12 03:40:31 +00:00
ZmnSCPxj 816298123f invoice: Allow "any" msatoshi, for amountless invoices.
Fixes: #534
2018-01-12 01:43:55 +00:00
ZmnSCPxj caab95b922 wallet_payment: Make msatoshi field nullable. 2018-01-12 01:43:55 +00:00
ZmnSCPxj 50471bf4fe invoice: Make msatoshi field nullable.
Note that the database itself has the field nullable (sqlite3
does not even have non-nullable columns) but our in-memory
structures did not.
2018-01-12 01:43:55 +00:00
ZmnSCPxj 7221476553 invoice: Clarify comments about the two loops in resolve_invoice. 2018-01-12 01:28:32 +00:00
practicalswift a900551815 Use tal_hex(...) instead of tal_hexstr(...) 2018-01-12 00:55:46 +00:00
Rusty Russell d4c8210a9e gossipd: don't hang if we try to connect to already-connected peer.
Closes: #287
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:47:47 +01:00
Rusty Russell e22dd5ad71 lightningd: implement gossip_get_txout/gossip_get_txout_reply.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 2b1eb6a677 bitcoind: getoutput so we can check short_channel_ids.
It would be nice if bitcoind had an RPC to do this in one, but that's
a bit much to ask for.  We could also hand around proofs, for lite nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell 9a8eca4fee Remove obsolete FIXME messages.
1. htlc->fail has been changed to a u8 *.
2. wallet_get_newindex saves to the db.
3. peer->next_htlc_id is saved to the db in peer_save_commitsig_sent() below.
4. We do store commit in peer_save_commitsig_received(peer, commitnum),
   and the fixme below talks about HTLC sigs.
5. We do commit shachain and next_per_commit_point in wallet_shachain_add_hash
   and update_per_commit_point respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 55909a7a39 lightningd: call peer_last_tx() to update peer->last_tx before saving to db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 8cf97e904d lightningd: remove unused "forever_confirms" parameter.
It's 100, but we never wired this up, so remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 7e0bc88e2a gossip: fix default broadcast interval, move option.
This now makes a few more gossip tests time out without --dev-broadcast-interval

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 171ce689dc lightningd/chaintopology: remove unneeded chain_topology pointer from block.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell 3332cd6395 chain_topology: remove unneeded struct outgoing_tx topo pointer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
practicalswift ab0c26be91 Check return value of json_get_params(...) call in json_getlog(...) and json_getpeers(...)
All other users of json_get_params(...) check the return value:

```
lightningd/chaintopology.c:     if (!json_get_params(buffer, params,
lightningd/chaintopology.c:     if (!json_get_params(buffer, params,
lightningd/dev_ping.c:  if (!json_get_params(buffer, params,
lightningd/gossip_control.c:    if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params, "label", &labeltok, NULL)) {
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/jsonrpc.c:   if (!json_get_params(buffer, params,
lightningd/pay.c:       if (!json_get_params(buffer, params,
lightningd/pay.c:       if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
wallet/walletrpc.c:     if (!json_get_params(buffer, params,
wallet/walletrpc.c:     if (!json_get_params(buffer, params, "tx", &txtok, NULL)) {
```
2018-01-10 23:43:50 +00:00
Rusty Russell 467ca0a996 lightningd: suppress feerate debug messages when nothing changes.
Every 30 seconds, these flood the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-10 16:55:36 +01:00
ZmnSCPxj e717963210 lightningd: Do not search for ourself in the daemon dir.
In preparation for putting our daemons into pkglibexecdir; lightningd
itself will be in bindir, separate from our subdaemons.
2018-01-09 14:52:48 +01:00
ZmnSCPxj 7f88585d84 lightningd: When searching subd fall back to pkglibexecdir. 2018-01-09 14:52:48 +01:00
ZmnSCPxj c572c3bbee lightningd: Add --test-daemons-only option. 2018-01-09 14:52:48 +01:00
practicalswift 4bdd2452f2 Make sure fsync, connect and close are never accidentally passed negative arguments 2018-01-09 14:50:50 +01:00
Rusty Russell 795e42d757 chaintopology: fix use-after-free which sometimes causes fatal() in clear_otx_peer
I've only seen this under travis, so I can't verify that this fixes it,
but it's certainly a bug which could cause that issue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-09 14:49:52 +01:00
practicalswift dcb4039a96 Check lseek(...) return value 2018-01-09 13:52:12 +01:00
Rusty Russell a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Rusty Russell 7975a1cbfc json_getchannels: short_id -> short_channel_id, use standard format.
We include flags separately, so name this field as per spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Christian Decker 3db4474465 hsm: Generate fully signed funding transaction in the HSM
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 37c2873c88 wallet: Store outputs from onchaind in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 0bb264e1a2 wallet: Added unilateral close info to utxo
This is necessary to grad the their_unilateral/to-us outputs since
they aren't being harvested by `onchaind`

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 8b6ab6a901 wallet: Remember scriptpubkey that we pass to onchaind on init
This is the scriptpubkey that onchaind spends all funds to, except for
the their_unilateral/to-us case, so we better recognize that address.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 5f29a218b7 onchain: Add message to notify about their_unilater/to-us outputs
This is the only case in which we don't respend to a simple keyindex'd
pubkey, so we need to handle this for future spends.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 85eb743e02 wallet: Detect close transactions and track their outputs in DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker 2a3c9e96f0 txfilter: Expose the txfilter_add_scriptpubkey function
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker e4e9e22fe8 wallet: Use newindex primitive to generate change key index
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker b13a972929 gossip: Do not return empty address-objects for getnodes
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 09:48:15 +00:00
practicalswift d02b890b55 assert(tal_count(log) > 0) in log_to_json(...) 2018-01-08 09:39:45 +00:00
Rusty Russell 05a3db886b peer_control: fix memleak in dev-disconnect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 16:08:50 +01:00
Rusty Russell ee939c8dcb lightingd: fix memleak when we reload from db.
We always arm the funding_lockin_cb, even if we don't need to.  If we
have an short_channel_id already from the db, this was replacing it
and leaking the old one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell fb81e6c14b chaintopology: always start 100 blocks back.
Since we panic when we see our root reorg out, even if we're not doing
anything yet, restoring the 100 block margin is the simplest fix.

Unfortunately this means adding a 100-block spacer in the tests, so things
don't get confused.

Fixes: #511
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 36316957e3 lightningd: set parent correctly for loaded peers.
The current code makes the channel the parent, which is a cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell a5c65e2c9b lightningd: reactivate peers.
This is surprisingly simple.  We set up the watches for funding tx
depth and the funding output, then if it's not onchain we ask gossipd
to reconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 1b41335121 chain_topology: two-stage startup.
Load the first block we're possibly interested in, then load the peers so
we can restore the tx watches, then finally replay to the current tip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 7b735e5de8 lightningd: scan blockchain from first possible block.
Eventually we want to save blockchain in db to avoid this scan, but
for the moment, we need to reload as far back as we may be interested in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 93c589efe8 wallet: save block height when we first create channel.
This gives us a lower bound on where funding tx could be.

In theory, it could be lower than this if we get a reorganization, but
in practice this is already a 1-block buffer (since we can't get into
current block, only the next one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell 4dea3c4747 lightningd: remove unused db stubs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell ba22484901 lightningd: simplify permanent failure.
Turns out everyone wanted a formatted string anyway.

Inspired-by: practicalswift
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-03 19:56:28 +00:00
ZmnSCPxj 5eceaa7be9 invoice: Modify waitanyinvoice interface to use pay_index. 2018-01-03 01:12:49 +00:00
ZmnSCPxj 3003b7346c invoice: Add pay_index member to struct invoice.
In preparation for change in interface of waitanyinvoice.
2018-01-03 01:12:49 +00:00
practicalswift 66e7c54810 Avoid out-of-bounds read in increase(u32 feerate_per_kw) 2018-01-02 18:37:22 +01:00
practicalswift 3d39312212 Fix typos 2018-01-02 15:09:36 +01:00
Rusty Russell 1e51275792 lightningd: don't save wallet_channel twice on creation.
peer_channel_new() does exactly that already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-02 13:21:25 +01:00
Rusty Russell 7e062fd637 onchaind: don't assert() when htlc fulfilled twice.
In the normal (peer-to-peer) path, the HTLC state prevents us fulfilling
twice, but this goes out the window with onchain HTLCs.

The actual assert which caught it was lightningd/pay.c:70 (payment_succeeded)
in the test_htlc_in_timeout test, after the next commit.

So add an assert earlier (in fulfill_our_htlc_out) and check in the
one caller where it can be true.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-02 13:21:25 +01:00
ZmnSCPxj f2f116d613 lightningd: Add missing final cleanups. 2018-01-02 01:31:04 +00:00
practicalswift c6b5e72e3b Pass bool true instead of "true" to json_add_bool(..., bool value) 2018-01-02 01:18:42 +00:00
ZmnSCPxj 8e3c97762e invoice: Properly implement waitinvoice.
Fixes: #444
2018-01-02 00:36:16 +00:00
ZmnSCPxj 773f7524dd invoice: Factor out invoice deletion code.
In preparation for adding a deletion trigger for
invoice waiters on individual invoices.
2018-01-02 00:36:16 +00:00
ZmnSCPxj 1de339eff6 invoice: Use wallet_invoice_nextpaid in waitanyinvoice implementation.
Fixes: #442
2018-01-01 23:13:31 +00:00
practicalswift 84dd65485c Remove duplicate includes 2017-12-28 16:09:24 +01:00
Rusty Russell 6debaccfd5 chaintopology: only do callbacks once chain has settled.
This is only important for testing, where we add 100 blocks at once
and time out under valgrind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 0650653658 bitcoind: delete chaintips code.
We don't need it any more, with the simpler topology approach.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 1d9a8e5484 chaintopology: load forwards, not backwards.
We used to load the new tip and work backwards until we joined up with
the previous tip.  That consumed quite a lot of memory if there were
many blocks.

Instead, just poll on blocknum+1, and grab it once that succeeds.  If
prev is different from what we expect (reorg), we free the current tip
and try again.

We could theoretically miss a reorg which is the same length (2 block
reorg with more work due to difficulty adjustment), but even if that
happened we'd catch up on the next block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 6f6d7a5e44 chaintopology: get fees using a timer, not on each block.
It definitely changes when we get a block, but it also changes between
blocks as mempool fills.  So put it on its own timer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 985a0b431f getblockhash: don't get upset if we fail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell 810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell 0237e0b28c bitcoin: create new wrapper type bitcoin_txid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_txid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell 553ebc936b json: add json_add_txid.
I prefer the typesafety of specific functions, rather than having the
caller know that txids are traditionally reversed in bitcoin.

And we already have a bitcoin_txid_to_hex() function for this.

Closes: #411
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
William Casarin 7ecccd50b9 wireaddr: add ip[:port] parsing
* Add port parsing support to parse_wireaddr. This is in preparation for storing
addresses in the peers table. This also makes parse_wireaddr a proper inverse of
fmt_wireaddr.

* Move parse_wireaddr to common/wireaddr.c this seems like a better place for
it. I bring along parse_ip_port with it for convenience. This also fixes some
issues with the upcoming ip/port parsing tests.

Signed-off-by: William Casarin <jb55@jb55.com>
2017-12-21 09:56:20 +00:00
Rusty Russell 90a5ba043c htlc_out: initialize hout->key.id, in case we destroy it before setting.
We set hout->key.id when channeld tells us what it is, but if channeld
dies before that we free the hout, and our destructor logs it:

Valgrind error file: valgrind-errors.20312
==20312== Use of uninitialised value of size 8
==20312==    at 0x53ABC9B: _itoa_word (_itoa.c:179)
==20312==    by 0x53B041F: vfprintf (vfprintf.c:1642)
==20312==    by 0x53B17D5: buffered_vfprintf (vfprintf.c:2330)
==20312==    by 0x53AEAA5: vfprintf (vfprintf.c:1301)
==20312==    by 0x53B7D63: fprintf (fprintf.c:32)
==20312==    by 0x128BAC: hout_subd_died (peer_htlcs.c:316)
==20312==    by 0x16D8E0: notify (tal.c:240)
==20312==    by 0x16DD95: del_tree (tal.c:400)
==20312==    by 0x16DDE7: del_tree (tal.c:410)
==20312==    by 0x16DDE7: del_tree (tal.c:410)
==20312==    by 0x16E1B4: tal_free (tal.c:509)
==20312==    by 0x162B5C: io_close (io.c:443)
==20312==    by 0x12D563: sd_msg_read (subd.c:508)
==20312==    by 0x161EA5: next_plan (io.c:59)
==20312==    by 0x1629A2: do_plan (io.c:387)
==20312==    by 0x1629E0: io_ready (io.c:397)
==20312==    by 0x164319: io_loop (poll.c:305)
==20312==    by 0x118E21: main (lightningd.c:334)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell 3543530172 build_utxos: fix weight calculation, and make more accurate.
Accuracy improvements:

1. We assumed the output was a p2wpkh, but it can be user-supplied now.
2. We assumed we always had change; remove this for wallet_select_all.

Calculation out-by-one fixes:

1. We need to add 1 byte (4 sipa) for the input count.
2. We need to add 1 byte (4 sipa) for the output count.
3. We need to add 1 byte (4 sipa) for the output script length for each output.
4. We need to add 1 byte (4 sipa) for the input script length for each input.
5. We need to add 1 byte (4 sipa) for the PUSH optcode for each P2SH input.

The results are now a slight overestimate (due to guessing 73 bytes
for signature, whereas they're 71 or 72 in practice).

Fixes: #458
Reported-by: Jonas Nick @jonasnick
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:56:15 +01:00
Rusty Russell abafcae5a6 lightningd/build_utxos: don't tell them they can pass NULL for change.
They can't: wallet_select_coins() doesn't allow it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:56:15 +01:00
Rusty Russell e695e5db09 lightningd: require explicit LIGHTNINGD_DEV_MEMLEAK env var to do memleak.
Otherwise every allocation and free is slowed down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 6b9c525f35 lightningd: use env var not cmdline to suppress backtrace.
We now set it up *before* parsing cmdline, so this is more convenient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker ee25547576 Re-enable builds with DEVELOPER=0
Two changes:
 - Fixed the function signature of noleak_ to match in both
   configurations
 - Added memleak.o to linker for tests

Generating the stubs for the unit tests doesn't really work since the
stubs are checked in an differ between the two configurations, so
adding memleak to the linker fixes that, by not requiring stubs to be
generated in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 12:43:10 +01:00
Rusty Russell 3ed496e45b lightningd: free failed output htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell ba6904f833 pay: close leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 85f065a6e7 peer_control: close leak in sign_last_tx.
We can call this multiple times.  The best solution is to add and remove
the signature so it's always unsigned as we expect it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell ed8baf5f88 funding_spent: fix leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 8c4eecc089 sphinx: fix leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell f19e02eeaa pay: close leak in payment_failed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 38a0ee8638 jsonrpc: current pending commands are not a leak.
The pay command in particular, attaches a reasonable number of
temporaries to cmd, knowing they'll be freed once cmd is done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 1356913143 invoice: remove strange code.
This is called when we load from database: clearly our tests aren't thorough
enough because we were allocating and initializing `r` in an unused structure.

invs is also the owner already; functions which steal are a bit surprising
to callers, so we either document them, or just don't do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell adf8740b96 peer_control: fix opening fundee leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 4957921e02 log: mark ltmp as notleak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell e64320c76c options: suppress "leak" from opt_table allocation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell ccb7047291 lightningd: add notleak annotations.
We have things which we don't keep a pointer to, but aren't leaks.
Some are simply eternal (eg. listening sockets), others cases are
io_conn tied to the lifetime of an fd, and timers which expire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 6c951bf7e0 invoice: move link to the top of the structure.
memleak doesn't detect pointers to within an object, only pointers to their
exact address (it's simpler this way).  Moving the linked list to the
top of the structure means it can follow the chain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell cec87ba08b peer_control: move link to the top of the structure.
memleak doesn't detect pointers to within an object, only pointers to their
exact address (it's simpler this way).  Moving the linked list to the
top of the structure means it can follow the chain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 70d01b22e1 memleak: add second exclude arg to exclude current commands' jcon.
This is not a child of cmd, since they have independent lifetimes, but
we don't want to noleak them all, since it's only the one currently in
progress (and its children) that we want to exclude.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell dfc132b2fe memleak: add backtrace to allocations.
We use the tal notifiers to attach a `backtrace` object on every
allocation.

This also means moving backtrace_state from log.c into lightningd.c, so
we can hand it to memleak_init().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell c956d9f5eb lightningd: tal memleak detection, dev-memleak command.
This is a primitive mark-and-sweep-style garbage detector.  The core is
in common/ for later use by subdaemons, but for now it's just lightningd.
We initialize it before most other allocations.

We walk the tal tree to get all the pointers, then search the `ld`
object for those pointers, recursing down.  Some specific helpers are
required for hashtables (which stash bits in the unused pointer bits,
so won't be found).

There's `notleak()` for annotating things that aren't leaks: things
like globals and timers, and other semi-transients.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 95df553813 lightningd: add dev-memdump command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell 8c665835fa jsonrpc: make explicit call to indicate cmd is still pending.
jsonrpc handlers usually directly call command_success or
command_fail; not doing that implies they're waiting for something
async.

Put an explicit call (currently a noop) there, and add debugging
checks to make sure it's used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker 2988e290cf gossip: Added internal gossip message to add a local channel
Couldn't find a good place to put these messages, we probably want to
do the same capability based request routing that we did for the HSM,
but for now this just defines the message in the master messages file.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Rusty Russell 477a529856 pay: make sure we don't think payment in progress if it immediately fails.
If send_htlc_out() fails, it doesn't initialize pc->out; that can
make us think it's still in progress.

Reported-by: Jonas Nick
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-19 14:40:26 +01:00
Christian Decker a8a6d1d669 channel: Directly send announcements and updates to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell 94711969f9 gossipd: hand out gossip_index to other daemons.
When gossipd sends a message, have a gossip_index.  When it gets back a
peer, the current gossip_index is included, so it can know exactly where
it's up to.

Most of this is mechanical plumbing through openingd, channeld and closingd,
even though openingd and closingd don't (currently) read gossip, so their
gossip_index will be unchanged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell 67aa95c194 gossipd: hand back peer, don't hand a new peer.
All peers come from gossipd, and maintain an fd to talk to it.  Sometimes
we hand the peer back, but to avoid a race, we always recreated it.

The race was that a daemon closed the gossip_fd, which made gossipd
forget the peer, then master handed the peer back to gossipd.  We stop
the race by never closing the gossipfd, but hand it back to gossipd
for closing.

Now gossipd has to accept two fds, but the handling of peers is far
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell b83ac58a98 subd: if a required daemon exits, wait instead of killing it.
Otherwise we always say it died because we killed it, so we don't get
the exit status.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell 6b232de7b1 openingd: return to master for more gossip when negotiation fails.
We can open other channels, if we want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell 899bf3fde9 subd: add transaction to subd exit corner case.
As demonstrated in the test at the end of this series, openingd dying
spontaneously causes the conn to be freed which causes the subd to be
destroyed, which fails the peer, which hits the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell 20d7ddcb6e json_fund_channel: give more details than "peer died".
Rather than using the destructor, hook up the cmd so we can close it.
peers are allocated off ld, so they are only destroyed explicitly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Christian Decker b5f737e563 gossip: Passing alias, color and wireaddrs through to gossipd
First step towards `gossipd` managing the `node_announcement`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker 67c6d4d1f1 opts: Change alias to be u8*, better matches the unicode nature
We are still generating only char* style aliases, but the field is
defined to be unicode, which doesn't mix too well with char.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell 65fd7ce132 bitcoind: don't leak memory on every call to bitcoin-cli.
Fixes: #412
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-13 22:46:10 +01:00
Rusty Russell b00525b8c8 options: don't leak memory when people override options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-13 22:46:10 +01:00
Rusty Russell 0610f66c34 bolt11: handle r value fee spec change.
We don't use it yet, but now we'll decode correctly.

See: https://github.com/lightningnetwork/lightning-rfc/pull/317
lightning-rfc commit: ef053c09431442697ab46e83f9d3f86e3510a18e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-12 11:45:44 +01:00
practicalswift 4452e3f3f1 Remove redundant code 2017-12-11 03:38:37 +00:00
ZmnSCPxj 7c09b6f478 Load defaults for purpose of help. 2017-12-11 03:21:07 +00:00
practicalswift 2ed53025f9 Remove redundant remote_shutdown_scriptpubkey assignment 2017-12-09 16:17:58 +01:00
practicalswift 61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Rusty Russell 5444084625 onchaind: fix reversed CSV delays.
Our config -> their CSV delay, their config -> our CSV delay.

Fixes: #395
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-04 13:00:57 +01:00
Rusty Russell af5dbbc9f8 json_connect: separate port arg so we can parse IPv6 addresses.
Fixes: #391
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-04 11:15:59 +01:00
Rusty Russell 0069787eb6 lightningd: fix crash on bad connect host arg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-04 11:15:59 +01:00
Christian Decker 9ba99d2b2d hsm: Cleanup after merging control and client libraries
Change all calls to use the correct serialization and deserialization
functions, include the correct headers and remove the control
messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Christian Decker 8ff69e0307 hsm: channeld now also just uses the handle_client entry point
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Christian Decker 6ce264d3a2 hsm: Remove special case of hsmfd_ecdh, replaced with client and cap
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Rusty Russell 933c1794a1 gossipd: gossip_get_update / gossip_get_update_reply from channeld.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell 2bed246e10 channeld: generate error messages instead of having master do it.
The master now hands channeld either an error code, and channeld
generates the error message, or an error message relayed from another
node to pass through.

This doesn't fill in the channel_update yet: we need to wire up gossipd
to give us that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell ee8274b7a1 channeld: handle wrapping error messages messages.
Currently lightningd does this, but channeld is perfectly capable of doing it.
channeld is also in a far better position to add channel_updates to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell be35895847 lightningd: instead of malformed code, keep generic error code.
This is in preparation for generating the actual error messages inside
channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell 06d7c40c63 lightningd: make minimum feerate significantly lower.
estimatesmartfee 4 ECONOMICAL was too high for lnd, so drop it, with some
increased security risk.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-01 18:30:57 +01:00
Christian Decker 00cad3a2ff fixup! txfilter: Add a simple transaction filter 2017-11-29 14:39:12 +01:00
Christian Decker c29923a623 topology: Add transaction filtering to connect_block
The filter is being populated while initializing the daemon and by
adding new keys as they are being generated. The filter is then used
in connect_block to identify transactions of interest.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Christian Decker 1eb3c9d2b3 txfilter: Add a simple transaction filter
This is mainly used to filter for transactions that may be of interest
to us, i.e., whether one of our keys is the recipient. It currently
does onyl simple scriptpubkey checks, but will eventually be extended
to use bloomfilters and add more sophisticated checks.

For now the goal is to speed up the processing of blocks during startup.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Christian Decker 26aa0e2aa4 topogoly: Extract owned funds from transactions in blocks
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Christian Decker 9c4f075c6e topogoly: Reduce log noise
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Rusty Russell af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Rusty Russell 18db1105c7 json_invoice: return absolute expiry_time
Requested-by: Nadav Ivgi <nadav@shesek.info>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell 45f5bb7fac bolt11: move to common/ and sign via callback.
JSON stuff is moved to lightningd/invoice.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell 78d0a5e840 bech32: move to common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Christian Decker 79443d6f94 htlc: Defer saving the outgoing payment until we store the HTLC
This addresses a performance regression introduced by
6ceb375650. We were storing it in an
otherwise empty DB transaction, which means that DB transaction was no
longer a no-op. Now we defer storing until we need to store the
corresponding HTLC anyway, so we can just piggyback on top of that
transaction.

This is also more consistent since we'd be forgetting the payment
anyway if we restart between adding the HTLC and committing to it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-23 23:45:28 +00:00
Rusty Russell 1a38c98a7e chaintopology: implement dev-setfees to force feerate changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 552e56d002 channeld: send update_fee messages.
We only send them when we're not awaiting revoke_and_ack: our
simplified handling can't deal with multiple in flights.

Closes: #244
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 062f18127d channeld: handle incoming feerate changes.
We allow from the 100-confirm economical rate to 5x the immediate rate.

Closes: #243
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell b836b452dc feerate: keep feerates separately for each side.
When we support changing them, they can be different during the transition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 24b4326b30 onchaind: use normal feerate, not last feerate from transaction.
This is used by onchaind when creating its own transactions.  Note we
don't send updates yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell ccf86a138a chaintopology: add notify_feerate_change() callback.
We'll use this to tell peers to change feerate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell d44088db83 fund-channel: use real feerate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell e2287ce545 openingd: use real feerates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell f1e4cad9d4 feerate: use u32 everywhere.
The wire protocol uses this, in the assumption that we'll never see feerates
in excess of 4294967 satoshi per kiloweight.

So let's use that consistently internally as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 88af0f5bf8 tests: change --override-fee-rate to --override-fee-rates and use in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 7151c65535 chaintopology: track three different feerates.
Depending on what we're doing, we can want different ones.  So use
IMMEDIATE (estimatesmartfee 2 CONSERVATIVE), NORMAL (estimatesmartfee
4 ECONOMICAL) and SLOW (estimatesmartfee 100 ECONOMICAL).

If one isn't available, we try making each one half the previous.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell ef4d54df94 chaintopology: use satoshi-per-kw everywhere.
This means we convert it when retrieving from bitcoind; internally it's
always satoshi-per-1000-weight aka millisatoshi-per-weight.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell b954662f06 bitcoind: use estimatesmartfee.
estimatefee is deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell 35a6ab8151 invoice: don't allow payments in expired invoices.
Fixes: #363
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell 1e6d5d82aa invoice: record expiry in data structure.
We don't enforce it yet, but you can query it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell 2d05644dfd json_invoice: add optional expiry argument.
This just includes it in the bolt11 return for the moment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell fbc5ba12e4 invoice: simplify find_invoice_by_label.
Passing a list_head is a remnant of when we had two lists.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell 70f15fb068 json_invoice: remove 'r' argument.
It was only for debugging early on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell 40315bfb91 test: fix dependencies.
Test objects must be added to $(ALL_OBJS) so they correctly depend on
CCAN headers etc.

Also, each test in a subdir must depend on headers and src in the parent
directory, as it will often #include them directly.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 37b4ab306e run-channel.c: move under channeld/
This also fixes dependencies, since it actually depends on channeld objects.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 593d0fa625 lightningd: initialize dev_subdaemon_fail to false.
Our testsuite uses --dev-fail-on-subdaemon-fail, so I didn't notice this
until I turned that off to chase a bug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell 5fb4577890 channeld/full_channel: fix incorrect reutrn from channel_rcvd_revoke_and_ack.
It was always returning false; it was supposed to return true if
we had added pending changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Christian Decker 77497b526f jsonrpc: Add listpayments method
Returns a list of transfers and their state.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-20 06:28:03 +00:00
Christian Decker e0d86376e2 wallet: Track outgoing payments in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-20 06:28:03 +00:00
Christian Decker 6ceb375650 wallet: Record issued invoices in the payments table
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-20 06:28:03 +00:00
Rusty Russell 5281541ec6 closingd: start with proper maximum fee, not our guesstimate.
Fixes: #348
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:41:45 +01:00
Rusty Russell 46f2e17905 openingd: update to BOLT with htlckey.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell fc05779f78 subdaemons: pass back and forth the htlc points.
Openingd sets it to the same as the payment point for the remote side.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell 654c2f4eb8 onchaind: use HTLC key for htlc signatures.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell 4db460903a htlc_tx: wire up the htlc points.
All the callers need to pass it in: currently channeld and openingd just
fake it by copying the payment point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell fe5614a489 basepoints/secrets: add htlc entry
Currently derive_basepoints just sets it to match the payment point/secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Christian Decker adfe494b08 invoice: Fix waitanyinvoice
There were two bugs: we weren't returning the next from the given
label but the one matching the label, and we were appending new
invoices to the head instead of the tail, which meant we'd be
traversing in the wrong order.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-15 00:07:44 +00:00
Rusty Russell f95afc55d8 rpc: report netaddr as array.
Thought we don't handle it at the moment, nodes can certainly have multiple
addresses, and we should display them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-12 12:16:22 +01:00
Christian Decker 89f016f524 jsonrpc: Only print netaddr in getpeers when we know it
Fixes #285
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-12 12:16:22 +01:00
Rusty Russell 956350e62e lightningd: check peers don't leave dangling HTLCs when they die.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-11 01:29:51 +00:00
Rusty Russell e9337820a0 onchaind: remove htlcs when peer is irrevocably committed.
We don't track them accurately when in onchaind, but we don't want to:
onchaind can be restarted at any time.

Once it's all settled, we're clear to clean them up.

Before this, valgrind could complain about deferncing hout->key.peer:

Valgrind error file: valgrind-errors.10876
==10876== Invalid read of size 4
==10876==    at 0x41F8AF: peer_on_chain (peer_control.h:127)
==10876==    by 0x42340D: notify_new_block (peer_htlcs.c:1461)
==10876==    by 0x40A08D: connect_block (chaintopology.c:96)
==10876==    by 0x40A96B: topology_changed (chaintopology.c:313)
==10876==    by 0x40AC85: add_block (chaintopology.c:384)
==10876==    by 0x40ABF0: gather_previous_blocks (chaintopology.c:363)
==10876==    by 0x4051B3: process_rawblock (bitcoind.c:410)
==10876==    by 0x4044DD: bcli_finished (bitcoind.c:155)
==10876==    by 0x454665: destroy_conn (poll.c:183)
==10876==    by 0x454685: destroy_conn_close_fd (poll.c:189)
==10876==    by 0x45DF89: notify (tal.c:240)
==10876==    by 0x45E43A: del_tree (tal.c:400)
==10876==  Address 0x6929208 is 2,120 bytes inside a block of size 2,416 free'd
==10876==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10876==    by 0x45E513: del_tree (tal.c:421)
==10876==    by 0x45E849: tal_free (tal.c:509)
==10876==    by 0x41A8E9: handle_irrevocably_resolved (peer_control.c:1172)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-11 01:29:51 +00:00
Christian Decker 07e5a9ef9f htlc: Allow for exactly min_final_cltv_expiry cltv delta
We are announcing that we are willing to accept incoming payments with
current_height + min_final_cltv_expiry + slack, assuming that the
sender adds some slack. In particular we'd reject the payment if
slack=0 which is allowed by the spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-11 01:28:03 +00:00
Rusty Russell 4fb472b7a7 db: we must always be in a transaction, remove nested, call fatal()
We save location where transaction was started, in case we try to nest.
There's now no error case; db_exec_mayfail() is the only one.

This means the tests need to override fatal() if they want to intercept
these errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell 7133a2f9b3 wallet: assume db errors will be fatal, don't check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell bccd2f8cf4 bitcoind: wrap callbacks in transaction.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell 8229769438 lightningd: do initial database load within a transaction.
Safest, and we can then assert that all db calls are in transactions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell 7586f3ed54 timers: wrap all calls in transactions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell 01361ab21b jsonrpc: wrap all calls in transaction. 2017-11-06 10:24:34 +01:00
Rusty Russell 3a596d6dda subd: wrap all message callbacks in a transaction.
Including destructors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell 8cef36cbd7 lightningd: fail htlcs we fulfill if peer unresponsive after deadline.
Closes: #241
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell 1142c44c29 lightningd: fail htlcs we offer if peer unresponsive after deadline.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell 9662589ed8 lightningd: move notify_new_block() callback to peer_htlcs.
And change prototype to take the lightningd structure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell bdabb24734 lightningd: note our complience on not forwarding an HTLC post deadline.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell c14b159166 lightningd: remove --deadline-blocks option.
We will derive it from other factors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell a55ce607a1 bitcoind: contain ld pointer.
This is a subset of a "bitcoind: wrap callbacks in transaction." from
the everything-in-transaction branch, but we need the ld pointer now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell 860a76b1c9 lightningd: update to add `wire_expiry_too_far`.
From recently-merged BOLT update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell 112ae0d0f5 common/test/run-json: test JSON escaping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell b7774fcb0a bolt11: comment on weird assignment-then-check test.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 6c21da69e6 bolt11: 'c' support for min_final_cltv_expiry.
Based on latest draft spec, using variable length encoding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell e1dc75e954 pay: new all-in-one RPC command.
Closes: #240
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell d577769350 invoice: provide bolt11 invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 8d09734536 bolt11: add decodepay RPC helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 15e1e4b099 bolt11: support for encoding/decoding and checking.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 9ec5cb7ba2 script: enhance is_p2sh/is_p2pkh/is_p2wsh/is_p2wpkh to extract addr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 7ed81cdf7b pay: factor out actual payment sending from json part of sendpay.
We're going to reuse this for the new 'pay' all-in-one command.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 79df10f782 bech32: pull in bech32 sample code, almost untouched.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell c2a0c51c30 lightningd: check for excessive HTLC locktimes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell 0000dc5908 lightningd: Allow in excess of ctlv_expiry.
We add a test which both overpays and over-sets the delta on the
intermediate and final nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell 71e794a046 lightningd: split ctlv_expiry and final_cltv.
These need to be different for testing the example in BOLT 11.

We also use the cltv_final instead of deadline_blocks in the final hop:
various tests assumed 5 was OK, so we tweak utils.py.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell 68dae5648d gossipd: route correctly using final CLTV value.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell bc9918ad46 dev: option not to do backtracing.
It crashes under valgrind, causing a valgrind error: valgrind gives us a
backtrace anyway, so we don't need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 04:14:33 +00:00