Commit Graph

567 Commits

Author SHA1 Message Date
Rusty Russell 7ead29b695 db: add support for remote end specify option_upfront_shutdown_script.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-02 01:18:33 +00:00
Lawrence Nahum a9e71d76b5 update libwally to latest version 0.6.9
Also removes a workaround caused by bug in libwally (!95) which has
been fixed.

Signed-off-by: Lawrence Nahum <lawrence@greenaddress.it>
2019-04-23 15:18:57 +02:00
Rusty Russell d0aefac5c3 wallet: fix crash when we have height requirement.
68fe5eacde introduced a skip in the iteration
of the available funds, which means utxos[i] may be off the end of utxos.

Reported-and-debugged-by: @nitramiz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-18 06:06:30 +00:00
Michael Schmoock 3fa539fc1b chore: increase min-capacity-sat to 10k
The old value of 1000 sat was too small to cover the dust reserves.
This lead to the situation when trying to open a channel with minimal
amount, the channels got refused because they were not able cover the
commitment fees.

For this reason the minimal capacity should be increased to i.e. 10k
satoshi, as the technical minimum that also accounts for fees and
reserves is somewhere around 6k sat.
2019-04-16 15:01:28 -07:00
Rusty Russell e0ec9ac521 libwally: update to 0.6.8.
This fixes block parsing on testnet; specifically, non-standard tx versions.

We hit a type bug in libwally (wallt_get_secp_context()) which I had to
work around for the moment, and the updated libsecp adds an optional hash
function arg to the ECDH function.

Fixes: #2563
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-13 18:55:15 +02:00
Rusty Russell 6630b99cf7 lightningd: move local invoice resolution into invoice.c function.
We're going to make it async, so start by moving the core code into
invoice.c and having that directly call fail/success functions for the
htlc.

We add an extra check in fulfill_htlc() that the HTLC state is correct:
that can't happen now, but may once we're async.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-12 03:32:48 +00:00
Christian Decker 27afc804d5 json-rpc: Include received and resolved time to listforward result
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Christian Decker 75de2e2f3c wallet: Record the received_time and resolved_time for HTLCs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Christian Decker 2ce9a1e10d wallet: Add `received_time` to `htlc_in` for forwarding times
We'd like to display the receive and resolution times in the forwardings
table. In order to remember the receive time we need to store it in the DB
along with the other information.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Christian Decker fcf133cd0a db: Add timestamp primitives so we can store them in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Rusty Russell a2fa699e0e Use node_id everywhere for nodes.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.

We didn't get much space saving in gossipd, even though we should save
24 bytes per node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell b4455d517c common/node_id: new type.
Node ids are pubkeys, but we only use them as pubkeys for routing and checking
gossip messages.  So we're packing and unpacking them constantly, and wasting
some space and time.

This introduces a new type, explicitly the SEC1 compressed encoding
(33 bytes).  We ensure its validity when we load from the db, or get it
from JSON.  We still use 'struct pubkey' for peer messages, which checks
validity.

Results from 5 runs, min-max(mean +/- stddev):
	store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
	39475-39572(39518+/-36),2880732,41.150000-41.390000(41.298+/-0.085),2.260000-2.550000(2.336+/-0.11),44.390000-65.150000(58.648+/-7.5),32.740000-33.020000(32.89+/-0.093),44.130000-45.090000(44.566+/-0.32)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell 837a095d68 pubkey: rename PUBKEY_DER_LEN to PUBKEY_CMPR_LEN.
Pubkeys are not not actually DER encoding, but Pieter Wuille corrected
me: it's SEC 1 documented encoding.

Results from 5 runs, min-max(mean +/- stddev):
	store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
	38922-39297(39180.6+/-1.3e+02),2880728,41.040000-41.160000(41.106+/-0.05),2.270000-2.530000(2.338+/-0.097),44.570000-53.980000(49.696+/-3),32.840000-33.080000(32.95+/-0.095),43.060000-44.950000(43.696+/-0.72)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Michael Schmoock c7af0c93c9 feat: add min_capacity_sat config value and switch
- add config value min_capacity_sat that will replaces the magic value
  min_effective_htlc_capacity = AMOUNT_MSAT(1000000)
- add config switch min_capacity_sat
2019-04-09 13:20:52 +02:00
lisa neigut ee1c82a7a7 db: fix memleak introduced with sqlite3_expanded_sql
wallet/test/run-wallet was failing the valgrind check; turns out
`sqlite3_expanded_sql` expects you to manage the memory of strings
it returns. from `sqlite3.h`:

** ^The string returned by sqlite3_expanded_sql(P), on the other hand,
** is obtained from [sqlite3_malloc()] and must be free by the application
** by passing it to [sqlite3_free()].
2019-04-09 07:19:15 +00:00
Rusty Russell 87cd628f52 log: truncate giant IO logging.
Adding a giant IO message simply causes it to be pruned immediately,
so truncate it if it's more than 1/64 the max size.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 04:41:43 +00:00
Rusty Russell f4d44d84bf db: typo fixed from review.
Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell 03108e6501 db: fall back to deprecated sqlite3_trace() for db hook on old sqlite3.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell 7ccf3af51d pytest: test the db hook plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell f0200027b8 db: keep track of changes.
For the moment we just dump them to stderr, but later this will go to a hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell e1e26ca69d db: create explicit separate API for select statements.
I was tempted to create a new db_select_stmt wrapper type, but that means
a lot of boilerplate around binding, which expects to work with db_prepare
*and* db_select_prepare.

This lets us clearly differentiate between db queries (which don't need to
go to a plugin) and db changes (which do).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell 8c4ef7f059 db: avoid the only failing db_exec.
Replace with an explicit test for existence.  This simplifies our db interface.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Christian Decker d651ce6f3b wally: Migrate main daemon to use wally transactions
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-08 00:00:00 +00:00
trueptolemy 92b40cb68a fix:Add infor about how many blocks needed until funding is confirmed
1. Rename channel_funding_locked to channel_funding_depth in
channeld/channel_wire.csv.
2. Add minimum_depth in struct channel in common/initial_channel.h and
change corresponding init function: new_initial_channel().
3. Add confirmation_needed in struct peer in channeld/channeld.c.
4. Rename channel_tell_funding_locked to channel_tell_depth.
5. Call channel_tell_depth even if depth < minimum, and still call
lockin_complete in channel_tell_depth, iff depth > minimum_depth.
6. channeld ignore the channel_funding_depth unless its >
minimum_depth(except to update billboard, and set
peer->confirmation_needed = minimum_depth - depth).
2019-04-07 23:45:35 +00:00
Michael Schmoock 5d0390f637 json: add cmd setchannelfee and wire to channeld
* adds the channeld_specific_feerates wire message 1029
* adds a json command handler
* adds u32 access methods for amount_msat
2019-03-15 02:48:18 +00:00
Michael Schmoock 1043df28be adds: new db fields and struct variables
- Intrduce DB update `channel` values: `feerate_base` and `feerate_ppm`
- Make fist use of now context realted DB migration
- Add `struct channel` members of the same name
- Use struct values instead of config when commiting new channels
2019-03-15 02:48:18 +00:00
Michael Schmoock 1853b399b0 fix: a pr2444 introduced valgrind complaint
This will add the testnet config copyNpaste to test_utils.c,
so that the test stups can set these.

Alternatively, to reduce code duplication, we can move the
testnet_config and mainnet_config from options.c to options.h.
2019-03-14 16:51:18 +01:00
Michael Schmoock 624ec6da27 tests: always call db_migrate() with non-NULL ld.
This will be useful when we add hooks.  For the moment, ld's contents is
undefined, but this can easily be changed.
2019-03-05 10:14:17 +00:00
Rusty Russell 049f2351d4 newaddr: deprecate 'address' output.
It doesn't make sense with 'all', and it's ok to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell 3e67c09d5e newaddr: support getting both bech32 and p2sh addresses.
Higher layers consume less addresses this way.

Fixes: #2390
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
neil saitug 7f89ce43cc Update wallet/db.h
Co-Authored-By: rustyrussell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell 60285187fd db: support more powerful migrations.
Allow a function as well as (or instead of!) an sql statement.  That
will let us do things like set per-channel values to the global
defaults, for example.

Since we remove the NULL termination, the final entry is ARRAY_SIZE()-1
not ARRAY_SIZE()-2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell bef7d45f1c db: remove manual db_migration_count function.
More efficient to measure the ARRAY_SIZE(), which is a runtime
constant.  We move it into the unit test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell b65c279557 db: don't enter into db_upgrades table on every startup.
Below this code appears:

	if (current != orig)
		db_exec(__func__, db,
			"INSERT INTO db_upgrades VALUES (%i, '%s');",
			orig, version());

But since the loop pre-increments current, this is always true.  I wondered
why there were so many duplicates in my db_upgrades table!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell 919f390a93 sendpay: rename 'description' to 'label'.
This field was used by `pay` to hold the bolt11 description if the bolt11
string used `h` to hash the description (which nobody ever did).  If the
`h` field wasn't present, it could contain anything, as it wasn't checked.

It's really useful to have a label for payments (eg. '1 Cuban'), but adding
yet-another option would be painful, so we simply rename 'description'
to 'label' except inside the db.

This means we need to do some tricky parameter parsing to handle array
and keyword JSON arguments, but only until we remove the old name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Rusty Russell 5e14274f41 sendpay: provide 'bolt11' parameter.
Without this, there's no proof of payment, since it is the signed invoice
that make the receipt valid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Christian Decker ed6a455a3c wallet: Display addresses derived from scriptPubKey where available
In particular this matches the case of `their_unilateral/to_us` outputs, which
were missing their addresses so far.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 11:15:24 -08:00
Christian Decker df4e0a0022 wallet: Add scriptPubkey to struct utxo
In order to avoid having to ask the HSM for public keys to
their_unilateral/to_us outputs we just store the `scriptPubkey` with the UTXO,
which can then be converted to the P2WPKH address.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 11:15:24 -08:00
Christian Decker 72f1c78a1e jsonrpc: Arm the minconf=1 parameter and deal with the fallout
We want to disallow using unconfirmed outputs by default, so making the
default 1 confirmation seems a good idea. This also matches `bitcoind`s
minimum confirmation requirement.

Arming however breaks some of our tests, so I used `minconf=0` for the
breaking tests and added a new test specifically for the `minconf` parameter
for `fundchannel`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Christian Decker c0d38aa99c jsonrpc: Expose the minconf parameter for fundchannel and withdraw
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Christian Decker 68fe5eacde wallet: Allow limiting the selection by confirmation height
This allows us to specify that an output must have been confirmed before the
given maximum height. This allows us to specify a minimum number of
confirmations for an output to be selected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Rusty Russell 6ed54dd34e db: fix dangling peers on db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell 38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell bac9a594b8 wallet: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 3ac0e814d0 daemons: use amount_msat/amount_sat in all internal wire transfers.
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell b8e484b508 struct channel_config: use amount_sat / amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 7e3928359a listpeers: add all the alternate "msat" and "sat" fields for channels.
These are undocumented, unfortunately, but at least that means I don't
have to update the docs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell b5dcb93e5f wallet: use amount_msat / amount_sat.
We change struct utxo to use amount_sat, and paper over the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 83adb94583 lightningd and routing: use struct amount_msat.
We use it in route_hop, and paper over it in the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00