Commit Graph

691 Commits

Author SHA1 Message Date
Rusty Russell fe66b53fb9 jsonrpc: add more fields to listfunds.
This avoids having to correlate with listpeers for the most pertinent
information.

This API predates plugins, otherwise we'd have listutxos and listpeers
and this would simply combine them appropriately.  Still, it exists so
there's little reason not to make it more friendly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 01:03:09 +00:00
Christian Decker ac1c894255 db: Get old sqlite3_trace based db_changes working again
Seems I accidentally broke the fallback mechanism while working on the DB
abstraction.
2019-09-18 18:35:11 +02:00
lisa neigut 38e404af51 wallet: add flag to specify whether or not to include change output
Will allow for more intelligent change policy
2019-09-11 23:56:27 +00:00
lisa neigut 5c70890efa rpc: add 'utxos' parameter to txprepare + withdraw
Allow a user to select the utxo set that will be added to a
transaction, via the `utxos` parameter. Optional.

Format for utxos should be of the form ["txid:vout","..."]
2019-09-11 23:56:27 +00:00
trueptolemy 6c708b5854 API: `fundchannel_cancel` can cancel fundchannel process before funding broadcast 2019-09-11 17:04:14 -05:00
trueptolemy a3b43fa196 Cleanup: Remove the duplicated code of getting txid in `json_withdraw` 2019-09-11 17:04:14 -05:00
trueptolemy b7ae2f3ac7 Fix: Store the transaction(broadcast by `txsend`) into DB 2019-09-11 17:04:14 -05:00
Rusty Russell 160f27061e common/utxo: make commitment_point optional in close_info.
We don't rotate key for option_static_remotekey, so we don't need
this point for such channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 11ee089d4b db: store option_static_remotekey for each channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell aca2e4f722 common/memleak: add dynamic hooks for assisting memleak.
Rather than reaching into data structures, let them register their own
callbacks.  This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.

Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell acf3952acc JSON: remove handling of pre-Adelaide (B:T:N) short_channel_ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Rusty Russell 91dcd1d55a db: convert pre-Adelaide short_channel_ids.
We're about to remove them.

Includes fix to sqlite3_bind_short_channel_id to not assume `id` is a
tal object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Christian Decker 58f448904c db: Extract db config lookup into its own function
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Rusty Russell <@rustyrussell>
2019-09-05 23:41:05 +00:00
Christian Decker c6b6958ae6 db: Remove sqlite3 from db.c and db.h
Now that all the users are migrated to the abstraction layer we can remove the
legacy implementation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker a29bedb5b8 db: Switch to indirect db close
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker ec8efe02e1 db: Switch to indirect `db_last_insert_id` version
We are about to delete all the `sqlite3`-specific code from `db.c` and this is
one of the last uses of the old interface.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 742bcdb2bc db: Move statement expansion into the driver
It's better to let the driver decide when and how to expand. It can then
report the expanded statement back to the dispatch through the
`db_changes_add` function.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b6d583c26a db: Move tracking of pending statements into the `struct db`
We now have a much stronger consistency check from the combination of
transaction wrapping, tal memory leak detection. Tramsaction wrapping ensures
that each statement is executed before the transaction is committed. The
commit is also driven by the `io_loop`, which means that it is no longer
possible for us to have statements outside of transactions and transactions
are guaranteed to commit at the round's end.

By adding the tal-awareness we can also get a much better indication as to
whether we have un-freed statements flying around, which we can test at the
end of the round as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b06cb68330 db: Migrate wallet.c to the new abstraction layer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker f19dc2ecc1 db: Migrate invoices.c to new abstraction layer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 460299850f db: Add DB-specific db_last_insert_id
This is likely the last part we need to completely encapsulate the part of the
sqlite3 API that we were using. Like the `db_count_changes` call I decided to
pass in the `struct db_stmt` since really they refer to the statement that was
executed and not the db.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker dffd3d79bc db: Add type-safe column access functions
These are based on top of the basic column access functions, and act as a
small type-safe wrapper, that also does a bit of validation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker a4e5371732 db: Add more type-safe bindings to the interface
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker c68efdfcf4 db: Migrate to DB abstraction layer in db.c
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 716a3b11a5 sqlite3: Move begin transaction and commit into the driver
This has a slight side-effect of removing the actual begin and commit
statements from the `db_write` hooks, but they are mostly redundant anyway (no
harm in grouping pre-init statements into one transaction, and we know that
each post-init call is supposed to be wrapped anyway).

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 492d77f213 db: Add setup and teardown function to DB
These are used to do one-time initializations and wait for pending statements
before closing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker a2b5b1561e db: Add method to count changed rows of a db_stmt
I was hoping to get rid of these by using "ON CONFLICT" upserts, however
sqlite3 only started supporting them in version 3.24.0 which is newer than
some of our deployment targets.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker d0027b1036 db: Implement basic query capabilities
This is the first step towards being able to extract information from query
rows. Only the most basic types are exposed, the others will be built on top
of these primitives.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 98963c58e4 db: Track whether a db_stmt has been executed
For some of the query methods in the next step we need to have an idea of
whether the stmt was executed (db_step function) so let's track that
explicitly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker e65e4fcf4b wallet: Call db_stmt_free from the db_stmt destructor automatically
This is much more in line with the rest of our memory management.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 093f5bfbec db: Switch to new DB asbtraction for DB migrations
These do not require the ability to iterate over the result, hence they can be
migrated already.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 50a1af8908 db: Implement the sqlite3 driver
This is the DB-specific counterpart to the previous commit.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 38601afd17 db: Implement skaffolding for the dispatch of DB-specific functions
These functions implement the lookup of the query, and the dispatch to the
DB-specific functions that do the actual heavy lifting.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 70e8da4fbd wallet: Add read-only flag to extracted queries
This gets rid of the two parallel execution paths of read-only and write
queries, by explicitly stating with each query whether it is a read-only
query, we only need to remember the ones marked as write queries.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 455aa48da0 wallet: Move the struct db definition to db_common.h
All drivers will have to reach into it, so put it in a place that is reachable
from the drivers, along with all other definitions.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker e04772ec9c wallet: Move the db_fatal definition so we can use it in drivers
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker acedcc593f wallet: Look up the db_config for the given driver in db_open
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker e4ab98459c wallet: Add tooling to extract SQL queries and generate driver info
This is the counterpart of the annotations we did in the last few commits. It
extracts queries, passes them through a driver-specific query rewriter and
dumps them into a driver-specific query-list, along with some metadata to
facilitate processing later on. The generated query list is then registered as
a `db_config` and will be loaded by the driver upon instantiation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 803007ecdf db: Make the `db` struct private and provide accessors instead
We will soon generalize the DB, so directly reaching into the `struct db`
instance to talk to the sqlite3 connection is bad anyway. This increases
flexibility and allows us to tailor the actual implementation to the
underlying DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 66a47d2761 wallet: Annotate migrations using the SQL macro
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 5270c149e3 wallet: Add macro to annotate SQL statements
These two simple macros have a twofold use:

 1) They serve as annotations for the query extraction tool to find them when
 extracting queries from the C source code.
 2) They replace the actual queries with names that can be used to lookup the
 queries in a table again, once they have been rewritten into the target SQL dialect.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 57f91246cf wallet: Change db_select_prepare to check for select instead
`db_select_prepare` was prepending the "SELECT" part in an attempt to limit
its use to read-only statements. This is leads to the queries in the code not
actually being well-formed, which we'll need in a later commit, and was also
resulting in extra allocations. This switches the behavior to just enforce a
"SELECT" prefix being present which allows us to have well-formed queries in
the code again and avoids the extra allocation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker f08fd9eaab wallet: Remove printf-like db_select variant
We need to have full DB queries that can be extracted at compile time later in
order to be able to rewrite them in other SQL dialects. In addition we had a
bit of unnecessary code-duplication in db_select and db_select_prepare. Now
the former uses the latter internally.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b856bb2f54 wallet: Expand select query field lists from macros
These will interfere with our query extraction process later on, and they were
really separating definition from use anyway, so let's expand these field lists.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
trueptolemy 42d5d753db cleanup: Remove unused structure `struct withdrawal` 2019-09-05 16:05:36 -05:00
trueptolemy cdcafdaf74 API: `txprepare` now support mutiple outputs 2019-09-05 16:05:36 -05:00
trueptolemy b660531216 common: `withdraw_tx()` now use the array of `struct bitcoin_tx_output` as parameter 2019-09-05 16:05:36 -05:00
lisa neigut 10ed2ebdb4 wallet: track our change outputs
Add change scripts to our txfilter so that we mark them as confirmed
(and can subsequently spend them)
2019-09-05 15:48:49 +02:00
lisa neigut 5663ecc599 listfunds: add 'blockheight' for confirmed transactions
Needed to calculate the value of all inputs for 'all' in
externalized fundchannel
2019-09-04 14:08:08 +02:00
Rusty Russell 049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00