Commit Graph

311 Commits

Author SHA1 Message Date
Christian Decker ec8efe02e1 db: Switch to indirect `db_last_insert_id` version
We are about to delete all the `sqlite3`-specific code from `db.c` and this is
one of the last uses of the old interface.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b06cb68330 db: Migrate wallet.c to the new abstraction layer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker f19dc2ecc1 db: Migrate invoices.c to new abstraction layer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker e4ab98459c wallet: Add tooling to extract SQL queries and generate driver info
This is the counterpart of the annotations we did in the last few commits. It
extracts queries, passes them through a driver-specific query rewriter and
dumps them into a driver-specific query-list, along with some metadata to
facilitate processing later on. The generated query list is then registered as
a `db_config` and will be loaded by the driver upon instantiation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 803007ecdf db: Make the `db` struct private and provide accessors instead
We will soon generalize the DB, so directly reaching into the `struct db`
instance to talk to the sqlite3 connection is bad anyway. This increases
flexibility and allows us to tailor the actual implementation to the
underlying DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 57f91246cf wallet: Change db_select_prepare to check for select instead
`db_select_prepare` was prepending the "SELECT" part in an attempt to limit
its use to read-only statements. This is leads to the queries in the code not
actually being well-formed, which we'll need in a later commit, and was also
resulting in extra allocations. This switches the behavior to just enforce a
"SELECT" prefix being present which allows us to have well-formed queries in
the code again and avoids the extra allocation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker f08fd9eaab wallet: Remove printf-like db_select variant
We need to have full DB queries that can be extracted at compile time later in
order to be able to rewrite them in other SQL dialects. In addition we had a
bit of unnecessary code-duplication in db_select and db_select_prepare. Now
the former uses the latter internally.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b856bb2f54 wallet: Expand select query field lists from macros
These will interfere with our query extraction process later on, and they were
really separating definition from use anyway, so let's expand these field lists.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
lisa neigut 10ed2ebdb4 wallet: track our change outputs
Add change scripts to our txfilter so that we mark them as confirmed
(and can subsequently spend them)
2019-09-05 15:48:49 +02:00
lisa neigut 5663ecc599 listfunds: add 'blockheight' for confirmed transactions
Needed to calculate the value of all inputs for 'all' in
externalized fundchannel
2019-09-04 14:08:08 +02:00
trueptolemy 23bfdc307f wallet: Use `struct sha256` for payment_hash in `struct forwarding` 2019-08-21 09:32:21 +08:00
lisa neigut 0c96c89d67 db-fix: resolve crash on fundchannel
Fixes error introduced by 1dbdc74bc where a new fundchannel
can cause a crash after start if the max dbid is for a closed
channel.
2019-08-10 02:52:13 +00:00
Rusty Russell 69b7ef1508 lightningd: fix up typesafe-cb bitcoind_getfilteredblock
`const struct filteredblock *` everywhere, as the typesafe_cb_preargs
macro required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-09 02:31:51 +00:00
Rusty Russell 99236f86f9 lightningd: rename 'satoshis' to 'amount' to avoid confusing check-source.
The type is enough (it's a struct amount_sat) to avoid confusion with
btc or msats.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-09 02:31:51 +00:00
Christian Decker 3dbaae38e3 wallet: Add function to add filteredblocks from backfilling
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-09 02:31:51 +00:00
lisa neigut c7f3fa34b2 funding tx: include segwit marker + flag in fee calculation
Noticed an off by one error when running tests for dual-funding;
we're not including the two 'header' segwit bytes in our weight
calculations.
2019-08-03 05:19:24 +00:00
trueptolemy 294f05dc98 Call notification 'forward_event' when forward payment status changes 2019-08-01 18:49:25 +08:00
trueptolemy ab60bcab7c wallet: Set the fee to 0 when forward doesn't have out channel with LOCAL_FAILED status 2019-08-01 18:49:25 +08:00
Karl-Johan Alm ff2bfe3bdb dynamically generate string of network names 2019-07-22 16:38:32 -05:00
Christian Decker 86d4362b65 wallet: Don't delete channels from DB, mark them closed.
Since we now have a couple of long-lived dependents it is time we stop
removing channels from the table once they are fully closed, and instead just
mark them as closed. This allows us to keep forwards and transactions foreign
keys intact, and it may help us debug things after the fact.

Fixes #2028

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-30 16:41:30 +09:30
Christian Decker 1dbdc74bc3 wallet: Add a final `CLOSED` state to channels
Instead of deleting the channels we will simple mark them as `CLOSED` from now
on. This is needed for some of the other tables not to end up with dangling
references that would otherwise survive the channel lifetime, e.g., forwards
and transactions.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-30 16:41:30 +09:30
Rusty Russell 220449e1cd ccan: import ccan/json_out and ccan/json_escape.
These are generalized from our internal implementations.

The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.

The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
darosior ac038b340b fundchannel: Add a new 'utxo' parameter
This new parameter takes a list of outpoints (as txid:vout) and fund a channel from the corresponding utxos.
Example : fundchannel <id> 10000 normal 1 [10767f0db0e568127fffd7f70a154d4599f42d62babf63230a7c3378bfce3cb0:0, c9e040e0b5fc8c59d5e7834108fbc5583001f414dd83faf0a05cff9d1a92d32c:0]
2019-06-11 23:24:07 +00:00
Christian Decker b6b548a983 wallet: Rip out the txtypes type in favor of enum wallet_tx_type
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Christian Decker 6efb1c00a6 wallet: Add function to retrieve transactions from the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Christian Decker ad4b9204ab wallet: Add function to annotate transactions aposteriori
Since we add the transactions while processing the blockchain, and before we
have enough context to annotate them correctly, i.e., in the txwatches, we add
them first and then annotate them aposteriori.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Christian Decker 1e2291c40c db: Add channel_id and type to the transactions table
Mainly used to differentiate channel-related transactions from on-chain wallet
transactions. Will be used to filter `listtransaction` results and bundle
transactions that belong to the same channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Rusty Russell e1dbc0b12b wallet: clean reserved inputs on startup.
We reserve inputs when we're going to send a transaction, but we don't
unreserve them if we crash.  This is most graphically demonstrated by
the txprepare case, which makes it easier to trigger.

Instead, we should query bitcoind to see whether the tx made it out or
not, as we would do manually with dev-rescan-outputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Rusty Russell 1a3886c116 wallet: keep a list of unreleased transactions.
We're going to use this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
trueptolemy b41d529b28 DB: Store the remote channel announcement signatures into DB
1. Add the fields of remote channel announcement signatures into TABLE channels
2. Add the related function
2019-05-29 11:46:33 +02:00
Rusty Russell 4370ffa978 autoclean: make this a plugin.
No change in behavior.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-22 00:18:29 +02:00
trueptolemy 0f429853fc wallet: cleanup the tal context in wallet_channels_load_active() and wallet_stmt2channel()
The original idea is to "tal" channel on the "ctx"(In fact, we'd like to set ctx as "ld").
But we already tal channel on "ld" in new_channel(), so "ctx" is unused.
2019-05-18 02:35:33 +00:00
trueptolemy c34548c847 wallet: add failcode and case for local fail in wallet_forwarded_payments_get() 2019-05-03 11:45:57 +02:00
trueptolemy 3a8fe5bf31 wallet: store failcode by adding failcode field in wallet_forwarded_payment_add() 2019-05-03 11:45:57 +02:00
Rusty Russell 7ead29b695 db: add support for remote end specify option_upfront_shutdown_script.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-02 01:18:33 +00:00
Rusty Russell d0aefac5c3 wallet: fix crash when we have height requirement.
68fe5eacde introduced a skip in the iteration
of the available funds, which means utxos[i] may be off the end of utxos.

Reported-and-debugged-by: @nitramiz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-18 06:06:30 +00:00
Christian Decker 27afc804d5 json-rpc: Include received and resolved time to listforward result
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Christian Decker 75de2e2f3c wallet: Record the received_time and resolved_time for HTLCs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Christian Decker 2ce9a1e10d wallet: Add `received_time` to `htlc_in` for forwarding times
We'd like to display the receive and resolution times in the forwardings
table. In order to remember the receive time we need to store it in the DB
along with the other information.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Rusty Russell a2fa699e0e Use node_id everywhere for nodes.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.

We didn't get much space saving in gossipd, even though we should save
24 bytes per node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell e1e26ca69d db: create explicit separate API for select statements.
I was tempted to create a new db_select_stmt wrapper type, but that means
a lot of boilerplate around binding, which expects to work with db_prepare
*and* db_select_prepare.

This lets us clearly differentiate between db queries (which don't need to
go to a plugin) and db changes (which do).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell 8c4ef7f059 db: avoid the only failing db_exec.
Replace with an explicit test for existence.  This simplifies our db interface.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Christian Decker d651ce6f3b wally: Migrate main daemon to use wally transactions
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-08 00:00:00 +00:00
Michael Schmoock 1043df28be adds: new db fields and struct variables
- Intrduce DB update `channel` values: `feerate_base` and `feerate_ppm`
- Make fist use of now context realted DB migration
- Add `struct channel` members of the same name
- Use struct values instead of config when commiting new channels
2019-03-15 02:48:18 +00:00
Rusty Russell 60285187fd db: support more powerful migrations.
Allow a function as well as (or instead of!) an sql statement.  That
will let us do things like set per-channel values to the global
defaults, for example.

Since we remove the NULL termination, the final entry is ARRAY_SIZE()-1
not ARRAY_SIZE()-2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell 919f390a93 sendpay: rename 'description' to 'label'.
This field was used by `pay` to hold the bolt11 description if the bolt11
string used `h` to hash the description (which nobody ever did).  If the
`h` field wasn't present, it could contain anything, as it wasn't checked.

It's really useful to have a label for payments (eg. '1 Cuban'), but adding
yet-another option would be painful, so we simply rename 'description'
to 'label' except inside the db.

This means we need to do some tricky parameter parsing to handle array
and keyword JSON arguments, but only until we remove the old name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Rusty Russell 5e14274f41 sendpay: provide 'bolt11' parameter.
Without this, there's no proof of payment, since it is the signed invoice
that make the receipt valid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Christian Decker df4e0a0022 wallet: Add scriptPubkey to struct utxo
In order to avoid having to ask the HSM for public keys to
their_unilateral/to_us outputs we just store the `scriptPubkey` with the UTXO,
which can then be converted to the P2WPKH address.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 11:15:24 -08:00
Christian Decker 68fe5eacde wallet: Allow limiting the selection by confirmation height
This allows us to specify that an output must have been confirmed before the
given maximum height. This allows us to specify a minimum number of
confirmations for an output to be selected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Rusty Russell 38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00