Commit Graph

768 Commits

Author SHA1 Message Date
Rusty Russell 72aa315b5e lightningd: save the fee_states into the database.
This is the final step: we pass the complete fee_states to and from
channeld.

Changelog-Fixed: "Bad commitment signature" closing channels when we sent back-to-back update_fee messages across multiple reconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 4270031d75 wallet: add per-channel feerate_state to the database.
The upgrade here is a bit tricky: we map the two values into the
feerate_state.  This is trivial if they're both the same, but if
they're different we don't know exactly what state they're in (this
being the source of the bug!).

So, we assume that the have received the update and not acked it,
as that would be the normal case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 8e3234e67a lightningd: sew in htlc set.
The invoice_try_pay code now takes a set, rather than a single htlc, but
it's basically the same thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 12985331f7 htlcs: remove origin_htlc_id from htlc_out.
This is a transient field, so rework things so we don't leave it in
struct htlc_out.  Instead, load htlc_in first and connect htlc_out to
them as we go.

This also changes one place where we use it instead of the am_origin
flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 345ca9b122 db: add partid field to htlc_out.
This is in preparation for partial payments.  For existing payments,
partid is 0 (to match the corresponding payment).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell 2d18c3a209 db: add partid, total_msat fields to payment entries.
This is in preparation for partial payments.  For existing payments,
partid is 0 (arbitrarity) and total_msat is msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Christian Decker 9660549b0c changelog: Update changelog for the 0.7.4 release 2019-12-12 00:15:23 +01:00
Christian Decker ff5f7b194f sphinx: Return the error in parse_onionpacket
As suggested by @niftynei here: https://github.com/ElementsProject/lightning/pull/3260#discussion_r347543999

Suggested-by: Lisa Neigut <@niftynei>
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2019-12-11 16:18:34 +01:00
Rusty Russell f7ebbb2ec5 common: make sphinx code ignorant of payload format.
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).

This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
   anything about "style".

The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Christian Decker 69c17d2d31 wire: Let the TLV _is_valid function actually return validity
I got this one wrong myself, since the function name implied a boolean
result. So I changed it to take the optional err_index as argument.
2019-12-03 00:37:15 +00:00
Christian Decker 16b6c31010 json-rpc: Add the error onion if we stored it in the DB
If we can't decode the onion, because the onion got corrupted or we used
`sendonion` without specifying the `shared_secrets` used, the best we can do
is tell the caller instead.
2019-12-01 15:40:47 +01:00
Christian Decker 1cf3c12a81 cleanup: The failchannel is not tal-allocated when first assigned
This makes the copy on write redundant.
2019-12-01 15:40:47 +01:00
Christian Decker 27547ce8d4 pay: Allow `sendonion` callers to provide `shared_secrets`
This means that c-lightning can now internally decrypt an eventual error
message, and not force the caller to implement the decryption. The main
difficulty was that we now have a new state (channels and nodes not specified,
while shared_secrets are specified) which needed to be handled.
2019-12-01 15:40:47 +01:00
Christian Decker b8ce175fd2 pay: Handle payment failures resulting from sendonion correctly
We are breaking with a couple of assumptions, namely that we have the
`path_secrets` to decode the error onion. If this happens we just want it to
error out.
2019-12-01 15:40:47 +01:00
Christian Decker 41221b6ecb pay: Make wallet_payment->destination optional
If we use `sendonion` we don't actually know the destination, so we
make the destination a pointer which is NULL if we don't know.
2019-12-01 15:40:47 +01:00
Rusty Russell b9a03a08ef wallet/Makefile: fix dependency line.
The .o files need regen, not the .c files!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-28 11:50:13 +01:00
Rusty Russell e5247a68b6 lightningd: check payment secret on htlc receipt.
We don't set the secret to compulsory (yet!) but put code in for the
future.  Meanwhile, if there is a secret, check it is correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 81c89aaef8 wallet: add invoice features into db.
In a future version, we will use features to insist that payers
provide the secret.  In transition, we may have old invoices which
didn't insist on that, so we need to know this on a per-invoice basis.

Not sure if I got the right syntax for adding an empty blob though!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Christian Decker e46f423ae9 sphinx: Cleanup route_step_decode_* functions
We have consolidated the two functions into a single `route_step_decode`
function, and made it static since we call it in the `process_onionpacket`
function. We remove the two exposed functions since they're no longer useful.
2019-11-22 04:40:25 +00:00
Christian Decker 2519f934aa tlv: Add validity check codegen for the tlv namespaces
Since the parser itself just parses and doesn't include validation anymore we
need to put that functionality somewhere. The validation consists of enforcing
that the types are in monotonically increasing order without duplicates and
that for the even types we know how to handle it.
2019-11-22 04:40:25 +00:00
Rusty Russell edbcb6fa15 lightningd: remove chainparams local parameter from wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-20 20:41:53 +01:00
Michael Schmoock 68b9ded24a fix: listtx set txindex only if blockheight is known 2019-11-18 11:01:40 +01:00
Rusty Russell 0607f998d1 options: allow --log-level <level>:<prefix> for finegrained log control.
This allows finegrained logging control of particular subdaemons or
subsystems.

To do this, we defer setting the logging levels for each log object
until after early argument parsing (since e.g. "bitcoind" log object
is created early).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: log-level can now specify different levels for different subsystems.
2019-11-18 04:50:22 +00:00
Rusty Russell ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell 86fb54a33b lightningd: remove per-peer log book.
We had a separate logbook for each peer, and copy log entries above
the printable log level into the master logbook.  This didn't always
work well, since we didn't dump it on crash for example.

Keep a single global logbook instead, and remove this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
darosior f075b87137 bitcoind: remove the chainparams member
We now have a global constant, prefer to use it instead of having
two variables with the same utility.
2019-11-15 13:14:08 +01:00
Michael Schmoock fe4a25a780 wallet: fix skipping tx dups memory corruption
Changelog-Fixed: #3231 listtransactions crash
2019-11-15 03:53:56 +00:00
Michael Schmoock ed238758a2 wallet: fix accessing blockheight of unconfirmed transaction 2019-11-15 03:53:56 +00:00
Rusty Russell b7bbccd6fa common/sphinx: handle decoding of TLV payload.
We add routines to decode the expected fields from both legacy and tlv
hop formats.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
lisa neigut 963a1da958 addr: handle P2SH/P2PKH in scriptpubkey encoding
Previously, returned null if a scriptpubkey was not Segwit; now
handles encoding to Base58 for other types.
2019-11-13 03:31:20 +00:00
Christian Decker 396e8224fa db: Add a migration to set received_time on forwards in rare cases
The case where this is needed is when the wallet had a forwarded payment
somewhere between commits 66a47d2 (which started tracking forwardings) and
d901304 (which added the `received_time` column). This just emulates the
behavior of sqlite3 for postgres as well.

Signed-off-by: Christian Decker <@cdecker>
2019-10-21 13:56:10 +02:00
Christian Decker 1ecad0cc53 db: Maybe a bit too pedantic?
Checking on whether we access a null field is ok, but should we crash right
away? Probably not. This reduces the access to a warning on sqlite3 and let's
it continue. We can look for occurences and fix them as they come up and then
re-arm the asserts once we addressed all cases.
2019-10-21 13:56:10 +02:00
Christian Decker 712595f0d2 db: Wire in the logs into the database so we can give feedback 2019-10-21 13:56:10 +02:00
Christian Decker ec8d774b29 db: Make column access way more pedantic
We should never access a nulled field, so add an assert to that effect.
2019-10-18 08:29:46 +02:00
Christian Decker 12f40f2227 db: Add _or_default variants for column accesses
We were implicitly relying on sqlite3 behavior that returns the zero-value for
nulled fields when accessing them. This adds the same behavior explicitly to
the DB abstraction in order to reduce `db_column_is_null` checks in the logic,
but still make it evident what is happening here.

Fixes https://github.com/fiatjaf/mcldsp/issues/1

Signed-off-by: Christian Decker <@cdecker>
2019-10-18 08:29:46 +02:00
Christian Decker 894627a287 db: Fix report and commit order for DB transactions
Reported-by: Simon Vrouwe <@SimonVrouwe>
Signed-off-by: Christian Decker <@cdecker>
2019-10-18 08:29:46 +02:00
Rusty Russell ca53c1b699 gossipd: push our own gossip messages harder.
I had a report of a 0.7.2 user whose node hadn't appeared on 1ml.  Their
node_announcement wasn't visible to my node, either.

I suspect this is a consequence of recent version reducing the amount of
gossip they send, as well as large nodes increasingly turning off gossip
altogether from some peers (as we do).  We should ignore timestamp filters
for our own channels: the easiest way to do this is to push them out
directly from gossipd (other messages are sent via the store).

We change channeld to wrap the local channel_announcements: previously
we just handed it to gossipd as for any other gossip message we received
from our peer.  Now gossipd knows to push it out, as it's local.

This interferes with the logic in tests/test_misc.py::test_htlc_send_timeout
which expects the node_announcement message last, so we generalize
that too.

[ Thanks to @trueptolmy for bugfix! ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 15:00:37 -05:00
Rusty Russell bd55f6d940
common/features: only support a single feature bitset.
This is mainly an internal-only change, especially since we don't
offer any globalfeatures.

However, LND (as of next release) will offer global features, and also
expect option_static_remotekey to be a *global* feature.  So we send
our (merged) feature bitset as both global and local in init, and fold
those bitsets together when we get an init msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-11 02:52:04 +00:00
lisa neigut 6e7cfcc76e walletrpc: explicitly note we discard utx on unsuccessful broadcast
If we can't broadcast the transaction, you're going to have to start
over regardless.  It's a tal child of cmd, which we free here.
2019-10-10 05:57:45 +00:00
lisa neigut af4ffe5fcd dev-forget-channel: accept passing in channel_id
This patch adds a channel_id parameter to allow for specifying
channels that are lacking a short_channel_id.

Useful in the case where a peer has 1) multiple channels (ONCHAIN etc)
and 2) a channel where the funding transaction hasn't been
broadcast/mined.
2019-10-10 05:57:45 +00:00
trueptolemy 019c052123 JSON-API: Allow `close` channel to specified address
Command format: close id [unilateraltimeout] [destination]

Close the channel with peer {id}, forcing a unilateral
close after {unilateraltimeout} seconds if non-zero, and
the to-local output will be sent to {destination}. If
{destination} isn't specified, the default is the address
of lightningd.

Also change the pylightning:
update the `close` API to support `destination` parameter
2019-10-09 21:04:16 -05:00
trueptolemy 71b606e050 lightningd: Add a new field `shutdown_scriptpubkey[NUM_SIDES]`
`shutdown_scriptpubkey[REMOTE]` is original remote_shutdown_scriptpubkey;
`shutdown_scriptpubkey[LOCAL]` is the script used for "to-local" output when `close`. Add the default is generated form `final_key_idx`;

Store `shutdown_scriptpubkey[LOCAL]` into wallet;
2019-10-09 21:04:16 -05:00
trueptolemy 75e946d256 json: Move `param_bitcoin_address` from wallet/walletrpc.c to lightningd/json.c
It's a useful helper, and it will be used to prase address in `close` command.
2019-10-09 21:04:16 -05:00
trueptolemy 7db3f1b8a6 json: Rename `json_tok_address_scriptpubkey` to `json_to_address_scriptpubkey`
Our json naming style is like `json_to_***`.
2019-10-09 16:51:42 -05:00
trueptolemy 36bc624928 Fix: fix the parameter prase of `txprepare`
The old prase process doesn't work!!
2019-10-09 16:51:42 -05:00
Rusty Russell 93865bb0f3 wallet: minor style fixes, and remove null JSON fields.
Our policy is generally to omit fields which aren't sensible.
Also, @niftynei points out the spacing in for loops.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-07 20:03:24 -05:00
Christian Decker 55af02b163 wallet: Fix a column access size mismatch when reading tx types 2019-10-07 20:03:24 -05:00
Christian Decker d840496e53 json-rpc: Make listtransaction non-experimental but hide annotations
We haven't tested the annotation for completeness yet, so let's not have
people start relying on them just yet.
2019-10-07 20:03:24 -05:00
Christian Decker 33bb4f7f58 json-rpc: Print details and annotations for listtransactions 2019-10-07 20:03:24 -05:00