Commit Graph

345 Commits

Author SHA1 Message Date
Rusty Russell 72aa315b5e lightningd: save the fee_states into the database.
This is the final step: we pass the complete fee_states to and from
channeld.

Changelog-Fixed: "Bad commitment signature" closing channels when we sent back-to-back update_fee messages across multiple reconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 12985331f7 htlcs: remove origin_htlc_id from htlc_out.
This is a transient field, so rework things so we don't leave it in
struct htlc_out.  Instead, load htlc_in first and connect htlc_out to
them as we go.

This also changes one place where we use it instead of the am_origin
flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 345ca9b122 db: add partid field to htlc_out.
This is in preparation for partial payments.  For existing payments,
partid is 0 (to match the corresponding payment).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 2d18c3a209 db: add partid, total_msat fields to payment entries.
This is in preparation for partial payments.  For existing payments,
partid is 0 (arbitrarity) and total_msat is msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Christian Decker 16b6c31010 json-rpc: Add the error onion if we stored it in the DB
If we can't decode the onion, because the onion got corrupted or we used
`sendonion` without specifying the `shared_secrets` used, the best we can do
is tell the caller instead.
2019-12-01 15:40:47 +01:00
Christian Decker 1cf3c12a81 cleanup: The failchannel is not tal-allocated when first assigned
This makes the copy on write redundant.
2019-12-01 15:40:47 +01:00
Christian Decker 27547ce8d4 pay: Allow `sendonion` callers to provide `shared_secrets`
This means that c-lightning can now internally decrypt an eventual error
message, and not force the caller to implement the decryption. The main
difficulty was that we now have a new state (channels and nodes not specified,
while shared_secrets are specified) which needed to be handled.
2019-12-01 15:40:47 +01:00
Christian Decker b8ce175fd2 pay: Handle payment failures resulting from sendonion correctly
We are breaking with a couple of assumptions, namely that we have the
`path_secrets` to decode the error onion. If this happens we just want it to
error out.
2019-12-01 15:40:47 +01:00
Christian Decker 41221b6ecb pay: Make wallet_payment->destination optional
If we use `sendonion` we don't actually know the destination, so we
make the destination a pointer which is NULL if we don't know.
2019-12-01 15:40:47 +01:00
Rusty Russell 81c89aaef8 wallet: add invoice features into db.
In a future version, we will use features to insist that payers
provide the secret.  In transition, we may have old invoices which
didn't insist on that, so we need to know this on a per-invoice basis.

Not sure if I got the right syntax for adding an empty blob though!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell edbcb6fa15 lightningd: remove chainparams local parameter from wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-20 20:41:53 +01:00
Michael Schmoock 68b9ded24a fix: listtx set txindex only if blockheight is known 2019-11-18 11:01:40 +01:00
Rusty Russell ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Michael Schmoock fe4a25a780 wallet: fix skipping tx dups memory corruption
Changelog-Fixed: #3231 listtransactions crash
2019-11-15 03:53:56 +00:00
Michael Schmoock ed238758a2 wallet: fix accessing blockheight of unconfirmed transaction 2019-11-15 03:53:56 +00:00
Christian Decker ec8d774b29 db: Make column access way more pedantic
We should never access a nulled field, so add an assert to that effect.
2019-10-18 08:29:46 +02:00
Christian Decker 12f40f2227 db: Add _or_default variants for column accesses
We were implicitly relying on sqlite3 behavior that returns the zero-value for
nulled fields when accessing them. This adds the same behavior explicitly to
the DB abstraction in order to reduce `db_column_is_null` checks in the logic,
but still make it evident what is happening here.

Fixes https://github.com/fiatjaf/mcldsp/issues/1

Signed-off-by: Christian Decker <@cdecker>
2019-10-18 08:29:46 +02:00
trueptolemy 71b606e050 lightningd: Add a new field `shutdown_scriptpubkey[NUM_SIDES]`
`shutdown_scriptpubkey[REMOTE]` is original remote_shutdown_scriptpubkey;
`shutdown_scriptpubkey[LOCAL]` is the script used for "to-local" output when `close`. Add the default is generated form `final_key_idx`;

Store `shutdown_scriptpubkey[LOCAL]` into wallet;
2019-10-09 21:04:16 -05:00
Christian Decker 55af02b163 wallet: Fix a column access size mismatch when reading tx types 2019-10-07 20:03:24 -05:00
Christian Decker d981b58234 wallet: Retrieve transaction annotations when listing transactions
This triple join should be efficient to read, and to process. We have a
one-to-many (tx-to-annotations), followed by a
one-to-one (annotation-to-channel) join, so we are limited to annotations x
transactions results.
2019-10-07 20:03:24 -05:00
Christian Decker ddae604f3d wallet: Move tx annotation for deposits into the wallet
We have split the iteration over the txs and the output in different
functions, so pushing the annotation down, while keeping the transaction
addition atop. This showcases the need to not have the txid reference the
transactions.id in the DB: we annotate in a function that doesn't have the tx
index context, but only add the TX after we have finished extracting.
2019-10-07 20:03:24 -05:00
Christian Decker 3d14c18074 wallet: Add primitives to annotate a transaction input and output
We'll slowly migrate from the tx annotations to the input and output we are
interested in.
2019-10-07 20:03:24 -05:00
Christian Decker 7283efa5b5 elements: Add amount_asset to support more than just plain satoshis
Currently the only source for amount_asset is the value getter on a tx output,
and we don't hand it too far around (mainly ignoring it if it isn't the
chain's main currency). Eventually we could bubble them up to the wallet, use
them to select outputs or actually support assets in the channels.

Since we don't hand them around too widely I thought it was ok for them to be
pass-by-value rather than having to allocate them and pass them around by
reference. They're just 41 bytes currently so the overhead should be ok.

Signed-off-by: Christian Decker <@cdecker>
2019-10-03 04:32:57 +00:00
Christian Decker d5f0c08a88 elements: Remove global is_elements variable in favor of chainparams
No need to keep duplicate globals.
2019-10-03 04:32:57 +00:00
Christian Decker f197e3da83 elements: Fix transaction size estimate when selecting coins
In elements we add an explicit fee output, if we don't consider it when
selecting coins, we end up underpaying the fees.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 17af57e504 elements: Compute correct weight in wallet_select for elements TXs
They are a bit larger, so account for that.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker 639713b547 elements: Fix transaction handling for elements transactions
Skipping coinbase transactions and ensuring that the transaction is serialized
correctly when sending it onwards.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker b89222f2d9 db: Adjust queries to work with postgres
The DB field type has to match the size of the accessor-type, and we had to
split the `REPLACE INTO` and `INSERT INTO OR IGNORE` queries into two
queries (update and insert if not updated) since there is no portable UPSERT
operation, but impact should be minimal.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 1f935cbd85 db: Strengthen some null-checks on queries
sqlite3 will just report 0 for anything that it thinks should be numeric, or
is accessed using a numeric accessor. Postgres does not, so we need to check
for is_null before trying to read it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 6b26ce5764 db: Split the vars table to have type-specific columns
This was weird right from the start, so we just split the table into integers
and blobs, so each column has a well-defined format. It is also required for
postgres not to cry about explicit casts in the `paramTypes` array.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
lisa neigut 38e404af51 wallet: add flag to specify whether or not to include change output
Will allow for more intelligent change policy
2019-09-11 23:56:27 +00:00
trueptolemy 6c708b5854 API: `fundchannel_cancel` can cancel fundchannel process before funding broadcast 2019-09-11 17:04:14 -05:00
Rusty Russell 160f27061e common/utxo: make commitment_point optional in close_info.
We don't rotate key for option_static_remotekey, so we don't need
this point for such channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell 11ee089d4b db: store option_static_remotekey for each channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Christian Decker ec8efe02e1 db: Switch to indirect `db_last_insert_id` version
We are about to delete all the `sqlite3`-specific code from `db.c` and this is
one of the last uses of the old interface.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b06cb68330 db: Migrate wallet.c to the new abstraction layer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker f19dc2ecc1 db: Migrate invoices.c to new abstraction layer
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker e4ab98459c wallet: Add tooling to extract SQL queries and generate driver info
This is the counterpart of the annotations we did in the last few commits. It
extracts queries, passes them through a driver-specific query rewriter and
dumps them into a driver-specific query-list, along with some metadata to
facilitate processing later on. The generated query list is then registered as
a `db_config` and will be loaded by the driver upon instantiation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 803007ecdf db: Make the `db` struct private and provide accessors instead
We will soon generalize the DB, so directly reaching into the `struct db`
instance to talk to the sqlite3 connection is bad anyway. This increases
flexibility and allows us to tailor the actual implementation to the
underlying DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker 57f91246cf wallet: Change db_select_prepare to check for select instead
`db_select_prepare` was prepending the "SELECT" part in an attempt to limit
its use to read-only statements. This is leads to the queries in the code not
actually being well-formed, which we'll need in a later commit, and was also
resulting in extra allocations. This switches the behavior to just enforce a
"SELECT" prefix being present which allows us to have well-formed queries in
the code again and avoids the extra allocation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker f08fd9eaab wallet: Remove printf-like db_select variant
We need to have full DB queries that can be extracted at compile time later in
order to be able to rewrite them in other SQL dialects. In addition we had a
bit of unnecessary code-duplication in db_select and db_select_prepare. Now
the former uses the latter internally.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker b856bb2f54 wallet: Expand select query field lists from macros
These will interfere with our query extraction process later on, and they were
really separating definition from use anyway, so let's expand these field lists.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
lisa neigut 10ed2ebdb4 wallet: track our change outputs
Add change scripts to our txfilter so that we mark them as confirmed
(and can subsequently spend them)
2019-09-05 15:48:49 +02:00
lisa neigut 5663ecc599 listfunds: add 'blockheight' for confirmed transactions
Needed to calculate the value of all inputs for 'all' in
externalized fundchannel
2019-09-04 14:08:08 +02:00
trueptolemy 23bfdc307f wallet: Use `struct sha256` for payment_hash in `struct forwarding` 2019-08-21 09:32:21 +08:00
lisa neigut 0c96c89d67 db-fix: resolve crash on fundchannel
Fixes error introduced by 1dbdc74bc where a new fundchannel
can cause a crash after start if the max dbid is for a closed
channel.
2019-08-10 02:52:13 +00:00
Rusty Russell 69b7ef1508 lightningd: fix up typesafe-cb bitcoind_getfilteredblock
`const struct filteredblock *` everywhere, as the typesafe_cb_preargs
macro required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-09 02:31:51 +00:00
Rusty Russell 99236f86f9 lightningd: rename 'satoshis' to 'amount' to avoid confusing check-source.
The type is enough (it's a struct amount_sat) to avoid confusion with
btc or msats.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-09 02:31:51 +00:00
Christian Decker 3dbaae38e3 wallet: Add function to add filteredblocks from backfilling
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-09 02:31:51 +00:00
lisa neigut c7f3fa34b2 funding tx: include segwit marker + flag in fee calculation
Noticed an off by one error when running tests for dual-funding;
we're not including the two 'header' segwit bytes in our weight
calculations.
2019-08-03 05:19:24 +00:00