Commit Graph

1266 Commits

Author SHA1 Message Date
ZmnSCPxj f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
ZmnSCPxj 5a267eb831 pay, payalgo: Show erring_node as compressed DER pubkey.
For consistency with other node pubkeys.
2018-04-04 14:17:07 +02:00
Rusty Russell 1764d6c907 grind_htlc_tx_fee: benchmark.
Takes 15 seconds on my laptop to do the worst-case grind:

	$ onchaind/test/run-grind_feerate 250001
	250001 iterations in 15893 msec = 63574 nsec each

It's not worth optimizing as it's 75% in libsecp:

    29.65%  run-grind_feera  run-grind_feerate  [.] secp256k1_fe_mul_inner
    23.51%  run-grind_feera  run-grind_feerate  [.] secp256k1_fe_sqr_inner
    11.04%  run-grind_feera  run-grind_feerate  [.] secp256k1_gej_double_var.part.6.constprop.34
     9.56%  run-grind_feera  run-grind_feerate  [.] secp256k1_scalar_reduce_512
     5.70%  run-grind_feera  run-grind_feerate  [.] Round

Even forcing a compile with -O3 -flto, it's only 13883 msec = 55534 nsec each.

Fixes: #291
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 5f1c77d249 test_lightning.py: add test for onchain with different feerates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 6bb47276ce lightningd: put min/max feerates into db, struct channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 4234321f7e onchain: get feerate min/max from master.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell 1f9ad06056 lightningd: allow us to close channel while still awaiting lockin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 23:22:59 +02:00
Rusty Russell 1773b03380 channeld: accept SHUTDOWN before channel is locked.
Fixes: #1308
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 23:22:59 +02:00
Rusty Russell 1a4a59d221 common/daemon: common routines for all daemons.
In particular, the main daemon and subdaemons share the backtrace code,
with hooks for logging.

The daemon hook inserts the io_poll override, which means we no longer
need io_debug.[ch].  Though most daemons don't need it, they still link
against ccan/io, so it's harmess (suggested by @ZmnSCPxj).

This was tested manually to make sure we get backtraces still.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Rusty Russell 8975fc2ceb libwally: use their secp context for all our daemons.
I didn't convert all tests: they can still use a standalone context.
It's just marginally more efficient to share the libwally one for all
our daemons which link against it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
ZmnSCPxj 26f7014813 payalgo: Add maximum delay.
Fixes: #1086
2018-04-03 04:29:48 +00:00
Christian Decker 63f22d70b5 gossip: Store channel deletions so we don't re-add them on restart
If we only remember the actions that added channels then we'd restore them when
re-reading the gossip_store, so put a tombstone in there to remember to delete
it. These will be cleared upon re-writing the store since the announcements wont
be written anymore.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker 633f893ec4 gossip: Add function to notify gossipd about an outpoint spend
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker a8d587c418 wallet: Return any eventual outpoint scid when marking it spent
Just return the short_channel_id matching the outpoint that we just marked as
spent.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker 5571f2143e gossip: Added message to notify gossipd of outpoint spends
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
ZmnSCPxj c1a43a04af payalgo: Be willing to overpay up to maxfeepercent, for privacy.
This obscures how far an intermediate hop is from the ultimate
payee, and also obscures slightly the exact payment value.

Fixes: #1089
2018-03-30 15:40:32 +02:00
ZmnSCPxj bc5fc692d1 pay: Let `sendpay` modify the recorded `msatoshi` of payments. 2018-03-30 15:40:32 +02:00
Rusty Russell a85ead7058 invoice: allow numeric labels again.
Fixes: #1291
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-30 00:01:10 +02:00
practicalswift e25297dd0a Remove unused functions not covered by unit tests 2018-03-28 11:22:05 +02:00
Christian Decker 5519717144 onchaind: Pass the funding spend height through when adding a UTXO
This is necessary since we have onchaind tell us about the
their_unilateral/to_us output, after it is already in a block.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
Christian Decker ba7341ec87 cleanup: Make blockheights unsigned
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
Christian Decker aba0b0e01b opening: Add the change scriptpubkey to the txfilter
This is necessary to track confirmations of or change output.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
Christian Decker 27db2d4ce2 wallet: Also track confirmations of their_unilateral/to_us outpoints
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
practicalswift 6269a4c55d Remove unused functions not covered by unit tests 2018-03-26 23:35:56 +00:00
ZmnSCPxj b914062465 pay: Also record how much we actually ended up sending. 2018-03-26 05:01:49 +00:00
practicalswift 7e9750ffee Reduce variable scopes 2018-03-26 01:31:21 +00:00
practicalswift 94ca824cf5 Use correct format strings for signed integers 2018-03-26 01:20:22 +00:00
ZmnSCPxj 74f3662a3b lightningd/subd.h: Add missing wire/wire.h.
If not included, a source file containing only
`#include<lightningd/subd.h>` will file compilation.
2018-03-26 01:09:59 +00:00
ZmnSCPxj 0bb9bcc0f1 wallet: Track some channel usage statistics.
Fixes: #1049
2018-03-26 01:08:52 +00:00
conanoc e7c8b95be3 Fix include order 2018-03-26 00:30:39 +00:00
conanoc c4700a13a7 Add guard for BACKTRACE_SUPPORTED 2018-03-26 00:30:39 +00:00
Rusty Russell 9f7d4312ff bolt11: undo json encoding for description bytes.
We don't handle \u, since we assume everyone sane is using UTF-8.  We'd
still have to reject '\u0000' and maybe other weird cases if we did.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell 54431d2b08 lightningd: escape our own alias when we print it in logs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell 4139f06780 json_getnodes: escape the alias when we print it out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell 76e8a11380 wallet: use json_escaped for invoice label.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell d92579f627 common/json_escaped: new type which explicitly notes a string is already JSON.
Trivial to use as a string, but it still means you should be careful
around it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Christian Decker 0a5ea76d77 gossip: Add message types to store gossip msgs and associate data
Since we may want to extend the on-disk format by adding custom information we
may as well just go the extra mile and reuse the serialization primitives we
already have.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
practicalswift a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift 98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
Igor Cota 8c00e4f98d Add --bitcoin-rpcport option to pass to bitcoin-rpc 2018-03-25 23:17:36 +02:00
practicalswift 0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
Mark Beckwith 489d8d04a6 Fund all (#1249)
* Funding channel with "all"
* Added test for fund all
2018-03-21 00:06:04 +01:00
ZmnSCPxj 5737853123 options: Add --autocleaninvoice-* options. 2018-03-20 17:25:51 +01:00
ZmnSCPxj 159e736fc1 invoices: Implement autocleaninvoice. 2018-03-20 17:25:51 +01:00
ZmnSCPxj 3b0081aebb invoices: Add delexpiredinvoice command. 2018-03-20 17:25:51 +01:00
Rusty Russell 8966a175ee chaintopology: add debug message when we're adjusting due to feerate floor.
But only if we're actually going to change the feerate, otherwise we'd
log every time.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-20 16:08:45 +01:00
Rusty Russell 2e687b9b35 chaintopology: set a fee floor to avoid creating unrelayable txs.
Naively, this would be 250 satoshi per sipa, but it's not since bitcoind's
fee calculation was not rewritten to deal with weight, but instead bolted
on using vbytes.

The resulting calculations made me cry; I dried my tears on the thorns
of BUILD_ASSERT (I know that makes no sense, but bear with me here as I'm
trying not to swear at my bitcoind colleagues right now).

Fixes: #1194
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-20 16:08:45 +01:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ 044705a547 peer_control: Fix a use-after-free bug. (#1237)
This bug is a classic case of being lazy:
1. peer_accept_channel() allocated its return off the input message,
   rather than taking an explicit allocation context.  This concealed the
   lifetime nature of the return.
2. The context for sanitize_error was the error itself, rather than the
   more obvious tmpctx (connect_failed does not take).

The global tmpctx removes the "efficiency" excuse for grabbing a random
object to use as context, and is also nice and explicit.

All-the-hard-work-by: @ZmnSCPxj
2018-03-19 09:27:09 +00:00
practicalswift e56eee50c8 Make sure we never pass a negative value to dup2(...) 2018-03-19 09:25:39 +00:00
ZmnSCPxj 9efe123a0d lightningd/json: Move json helpers specific for lightningd to new module. 2018-03-19 00:27:55 +00:00
ZmnSCPxj 881eb80f6a payalgo: Make 'pay' return similar to 'listpayments' 2018-03-19 00:27:55 +00:00
ZmnSCPxj 3c39bcc0f7 pay: Make json_add_payment_fields accessible to other parts. 2018-03-19 00:27:55 +00:00
ZmnSCPxj d181ecbeea payalgo: Implement retry_for for pay command. 2018-03-19 00:27:55 +00:00
ZmnSCPxj 981af478f1 payalgo: Keep and report route and route failures. 2018-03-19 00:27:55 +00:00
ZmnSCPxj 234d67da34 channel_fail_permanent: Use a channel-level error, not an all-channels error.
Fixes: #1229
2018-03-17 13:26:58 +01:00
ZmnSCPxj d49915cdd2 peer_control: Have listpeers also return channel ID. 2018-03-17 13:26:58 +01:00
ZmnSCPxj 099acb2a66 opening_control: have fundchannel also return channel_id.
For voyeurism.
2018-03-17 13:26:58 +01:00
Rusty Russell 4ffd5e213b pay: work around bug #1212 for existing databases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 04:01:06 +00:00
Rusty Russell 1b9791f0f5 pay: delete HTLC when we delete payment.
This fixes the root cause of https://github.com/ElementsProject/lightning/issues/1212
where we deleted the payment because we wanted to retry, then retry failed
so we had an (old) HTLC without a matching payment.  We then fed that
HTLC to onchaind, which tells us it's missing, and we try to fail the
payment and deref a NULL pointer.

Fixes: #1212
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 04:01:06 +00:00
ZmnSCPxj 7bbc1948f7 chaintopology: Add missing `block_map_del`.
We would `block_map_add` inside `add_tip`, but we never
`block_map_del` inside `remove_tip`, which is dangerous as
we actually `tal_free` the block inside `remove_tip`.

Our CI did not reliably trap this problem since block
hashes are random and rerunning the `test_blockchaintrack`
often passed spuriously.
2018-03-16 01:45:08 +00:00
ZmnSCPxj 022cfccddd pay: sendpay and waitsendpay print similar to listpayments 2018-03-16 01:24:43 +00:00
ZmnSCPxj fa281d32ea pay: sendpay_result has payment on success or in-progress. 2018-03-16 01:24:43 +00:00
practicalswift f9bd43f266 Limit scope of variable failcode. Remove dead code. 2018-03-16 01:17:40 +00:00
practicalswift b557e7b0fc Fix misleading indentation (space before tab) 2018-03-16 01:17:40 +00:00
practicalswift 7bd1ae6b2e Fix misleading indentation (in the case of DEVELOPER) 2018-03-16 01:17:40 +00:00
practicalswift b206aed371 Remove redundant cast to same type 2018-03-16 01:17:40 +00:00
practicalswift 03e2e404f2 Mark unused parameters as such (using "UNUSED") 2018-03-16 01:17:40 +00:00
Rusty Russell e63b7bb539 take: allocate temporary variables off NULL.
If we're going to simply take() a pointer, don't allocate it off a random
object.  Using NULL makes our intent clear, particularly with allocating
packets we're going to take() onto a queue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell 0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell de5d84097e lightningd: remove ltmp context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell ef2a063169 utils: add a global tmpctx.
I did a brief audit of tmpctx uses, and we do leak them in various
corner cases.  Fortunely, all our daemons are based on some kind of
I/O loop, so it's fairly easy to clean a global tmpctx at that point.

This makes things a bit neater, and slightly more efficient, but also
clearer: I avoided creating a tmpctx in a few places because I didn't
want to add another allocation.  With that penalty removed, I can use
it more freely and hopefully write clearer code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell 41ef42ee94 memdump: fix overzealous assertions from b857b2e843
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
ZmnSCPxj a0c2686ebd pay: Have sendpay wait for payment to be saved.
The payment should be stored in a "timely" manner, i.e.
within 10ms.
2018-03-14 05:33:09 +00:00
ZmnSCPxj a7a18b96cf pay: Rename some sendpay functions and fields to waitsendpay
Nearer to their actual purposes at this point.  Also in preparation
for next change.
2018-03-14 05:33:09 +00:00
ZmnSCPxj 61882ed9df pay: Add hook for triggering storage of payments. 2018-03-14 05:33:09 +00:00
ZmnSCPxj 24ee5d0437 pay: Store payment on failure. 2018-03-14 05:33:09 +00:00
ZmnSCPxj 411c9583a7 pay: Save and recover failure information.
Needed for particular race condition: client calls `sendpay` with
intent to call `waitsendpay` later to get information, but the
payment fails after `sendpay` returns but before client can invoke
`waitsendpay`.
This lets client know of information even if it manages to invoke
`waitsendpay` "late".
2018-03-14 05:33:09 +00:00
ZmnSCPxj 1e4adb0359 pay: Make sendpay nonblocking. 2018-03-14 05:33:09 +00:00
ZmnSCPxj bb4661f008 pay: Factor out construction of sendpay_result 2018-03-14 05:33:09 +00:00
Rusty Russell 46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Christian Decker 74a444eb7a jsonrpc: Return the channel capacity for listchannels
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker 5690783148 gossip: Pass output value to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker a8e553098a gossip: Pass output value to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker cb4ba9bf4f gossip: Use the utxoset to shortcircuit the txout lookup
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
ZmnSCPxj 1de124277b peer_control: Print both `to_self_delay` in listpeers. 2018-03-12 14:54:29 +01:00
John Barboza b4f7ce61b4 Append to crash log instead of creating new one
Multiple crashes accumulate in case you have some kind
of auto-restart for daemons.
2018-03-11 14:30:34 +01:00
Isidoro Ghezzi f24a7b9356 help: listnodes and listchannels,have optional arg 2018-03-11 14:27:09 +01:00
practicalswift 161ed320f8 Improve onboarding experience by handling common failure scenarios for new users more gracefully
Improve usability in these scenarios:
* bitcoin-cli not available in PATH and/or bitcoind not running
* bitcoin-cli available in PATH but bitcoind is not running
2018-03-08 23:51:45 +00:00
Rusty Russell 6c1233de44 channel: reserve a bip32 index as soon as channel is opened.
This simplifies things, and means it's always in the database.  Our
previous approach to creating it on the fly had holes when it was
created for onchaind, causing us to use another every time we
restarted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell e76f863329 txfilter: clean up prototypes.
Use const, add TAKES to declaration.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell c9e271df90 pay: wrap missing route_channels handling in COMPAT_V052.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell 9d9c1c3db6 wallet: wrap missing last_processed_block handling in COMPAT_V052.
Our testing also reveals a bug: we start lightningd and shut it down
before fully processing the blockchain, so we don't set
last_processed_block.  Fix that by setting it immediately once we have
a block: worst case it goes backwards a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell 5d5c1a5da5 lightningd: don't discard const in get_chainparams().
In general, it is true that accessors should take const and discard it,
but chainparams is *always* const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
John Barboza 30b290cb8f Explicit error message when disconnect fails
Whether the peer is not connected or not gossiping.
2018-03-07 16:14:01 +01:00
John Barboza 0a1351fd51 gossipd: command to disconnect from a gossiping peer
lightning-cli disconnect <peer id>
2018-03-07 16:14:01 +01:00
Corné Plooy b2ce619bcd Fix order of includes 2018-03-06 19:26:21 +01:00
Corné Plooy 71c39e96f3 Instead of passing the line number behind the '\0' of an artificial command-line argument, store it in a global variable.
While it is still a bit of a hack, this makes the code much easier to read.
2018-03-06 19:26:21 +01:00
Corné Plooy b857b2e843 Add assertions in various places to ensure tal_fmt doesn't receive NULL as argument for strings. 2018-03-06 19:26:21 +01:00
Christian Decker e44ea0b363 topology: Add new P2WSH outpoints to the wallet utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker ae30942e3f chaintopo: Record outpoint spends for owned outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00