Commit Graph

10702 Commits

Author SHA1 Message Date
Rusty Russell d287d7d229 gossipd: make request handlers return void.
They all returned the next io_plan, but it was always the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 0aad222c2d gossipd: rewrite update generation, remove local_chan.
local_chan was mainly around so we could "soft" disable channels (and
really disable them once we used the channel_update in an error
message).

Instead we introduce the idea of a "deferred_update": it's either
deferred indefinitely (a peer goes offline, if we need to send it in
an error we'll apply it immediatly), or simply delayed to avoid
spamming everyone.

The resulting rewrite is much clearer, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 8e37eb0028 gossipd: local_direction helper to generalize is_local_channel.
Increasingly we want to know is it local, and get the direction: it's
more efficient to do both at once.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 70ed47d77a channeld: add dev-disable-commit-after instead of dev-disconnect -nocommit
It was always a hack, but an impossible one once connectd will be
interpreting dev-disconnect!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 888745be16 dev_disconnect: remove @ marker.
Once connectd is doing this, we can't close as soon as we send,
and in fact we can't do 'fail write' either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 560fa06f42 pytest: disable dev-disconnect blackhole tests.
These would have to be done by connectd, not the local daemon, once it's
intermediating.  Otherwise the remote peer won't see any change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 2fb58772a4 common/socket_close: don't wait forever.
The "read until closed" trick doesn't work if the other end doesn't
close (as found in the next patch, where we use DEV_DISCONNECT_DISABLE_AFTER).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 35e3c1866e common: generalize extract_channel_id().
connectd is going to end up using this do demux; make it fast and complete.

Fixing this reveals a problem in openingd: it now extracts the channel_id
from funding_signed (which is where we transition off the temporary), and
gets upset.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 5111f39d2a connectd: clean up lightningd connection handling.
They all returned the same thing anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 6ce7e7ea60 common: is_msg_gossip_broadcast() for putting things in filter.
We don't need the other "gossip" messages in our echo-suppression filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 90b669857e lightningd: handle channel cleanups more explicitly.
1. Freeing an unconfirmed channel already releases the subd, so don't
   do that explicitly.
2. Use channel->owner to transfer ownership where possible, using
   channel_set_owner() which handles all the cases.

This simplifies the code and makes it more readable, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell 18526a3a5b lightningd: close one more fd for subdaemons.
Noticed by stracing for an unrelated problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Vincenzo Palazzo 9f02b6eb59 external: update lnprototest dependeces
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-12-28 09:49:09 +10:30
niftynei 70a73928cb balance-snaps: add a `balance_snapshot` event; fires after first catchup
Fire off a snapshot of current account balances (node wallet + every
'active' channel) after we've caught up to the chain tip for the *first*
time (in other words, on start).
2021-12-28 04:42:42 +10:30
niftynei f169597a02 test plugins/coin_moves: clean up/tidy coin moves plugin logic
removes unused in-memory stash of coin-moves; don't write the entire set
of coinmoves out on every update.
2021-12-28 04:42:42 +10:30
niftynei ec991e3af7 balance snap: first pass 2021-12-28 04:42:42 +10:30
niftynei 5570c6da08 coin_moves: cleanup tags list, update docs, remove unused method
We don't use journal_entry any more; remove it and update all of the
remaining tags (plus docs)
2021-12-28 04:42:42 +10:30
niftynei 8225a9decf coin_mvt: log events for pushes/lease_fees for leased channels
We need to stash/save the amount of the lease fees on a leased channel,
we do this by re-using the 'push' amount field on channel (which is
technically correct, since we're essentially pushing the fee amount to
the peer).

Also updates a bit of how the pushes are accounted for (pushed to now
has an event; their channel will open at zero but then they'll
immediately register a push event).

Leases fees are treated exactly the same as pushes, except labeled
differently.

Required adding a 'lease_fee' field to the inflights so we keep track of
the fee for the lease until the open happens.
2021-12-28 04:42:42 +10:30
niftynei bddd3694fa coin_mvt: record fees for an outbound htlc
If we initialized the payment, the fees are the entire fee-chain
(final hop amount - starting hop amount)

If it's a payment we routed, the fees are the diff between the
inbound htlc and the outbound (net gain by this routing)

Added to database so data persists nicely.
2021-12-28 04:42:42 +10:30
niftynei 29c6718297 coin_mvt: record new 'fees' field on htlc channel moves
We record the amount of fees collected for a routed payment. For
simplicity's sake on the data agg side, we record the fee payment on
*BOTH* the incoming htlc and the outgoing htlc. Note that this results
in double counting if you add up the fees from both an in-routed and
out-routed payment.
2021-12-28 04:42:42 +10:30
niftynei b6463174d6 coin moves: turn 'tag' into 'tags' array, add OPENER tag
Channels that the node has hopened will now be tagged with 'opener' in a
list of tags.
2021-12-28 04:42:42 +10:30
niftynei d3d6c09758 coin_moves: update some info about 'blockheight' 2021-12-28 04:42:42 +10:30
niftynei 4506f639fa coin_moves: remove 'index' for moves; bump version
Get rid of the 'movement_idx', since we replay events now.

Since we're removing a field from the 'coin_movement' event emission, we
bump the version type.

Changelog-Updated: `coin_movements` events have been revamped and are now on version 2.
2021-12-28 04:42:42 +10:30
niftynei 8098a4cd21 onchaind: remove 'is_replay' logic
we used this originally to suppress duplicate issuance of coin-move
events; we're assuming that any plugin expects duplicate events though
(and knows how to de-dupe them), so we no longer need this logic.
2021-12-28 04:42:42 +10:30
niftynei 27ff87c045 coin_moves: de-dupe moves as they come in
test_onchain_dust_out restarts a node, which produces duplicate events.
this is expected, but we need to de-duplicate the events stream to get
accurate results
2021-12-28 04:42:42 +10:30
niftynei d2c4d4aec2 coin_mvts: rewrite how onchain events are recorded, update tests
The old model of coin movements attempted to compute fees etc and log
amounts, not utxos. This is not as robust, as multi-party opens and dual
funded channels make it hard to account for fees etc correctly.

Instead, we move towards a 'utxo' view of the onchain events. Every
event is either the creation or 'destruction' of a utxo. For cases where
the value of the utxo is not (fully) debited/credited to our account, we
also record the output_value. E.g. channel closings spend a utxo who's
entire value we may not own.

Since we're now tracking UTXOs onchain, we can now do more complex
assertions about the onchain footprint of them. The integration tests
have been updated to now use more 'chain aware' assertions about the
ending state.
2021-12-28 04:42:42 +10:30
niftynei 07039fc2b4 onchaind:move some stuff around so we can call it 2021-12-28 04:42:42 +10:30
niftynei ade2242d14 coin_mvts: don't overwrite account names we add in onchaind
we assume that every event coming from onchaind is for that channel's
account, but now that we sometimes track external + wallet events from
onchaind, we should only add the channel account name if there's nothing
set there already
2021-12-28 04:42:42 +10:30
niftynei 3b3b8fb3ec coin_mvt: add "output value" to coin movement
When we log "external" wallet events, it's helpful to know what the
value of that output is, so let's log it
2021-12-28 04:42:42 +10:30
niftynei b7ca45514c coin_moves: fix account name serialization
Switch over to a more robust method of serializing strings
2021-12-28 04:42:42 +10:30
niftynei 5e1d8d6ad6 amount: helper for msat == 0
Shorthand for checking if value is zero
2021-12-28 04:42:42 +10:30
niftynei 43ade318d9 coin-move: the txid is now optional but outpoints are not
we're pivoting from a txid based world to a outpoint based world. every
coin movement (onchain) will correspond with a outpoint; only the spend
of an outpoint will have a tx_txid
2021-12-28 04:42:42 +10:30
niftynei 511c0df63a coin-mvts: on close, record an 'end' of our channel balance
For a mutual close, this is the only record for this that we need.
We remove all of the other tracking around mutual closes.
2021-12-28 04:42:42 +10:30
niftynei 737772f1ca coin-mvt: remove all the chain fees tracking
this will be impllicit going forward.
2021-12-28 04:42:42 +10:30
niftynei e791efe23b coin-mvt: remove all the spend-tracking
We're not going to do 'spend tracks' any more; instead we'll emit an
event whenever an output is included in a broadcast tx

(even if the broadcast fails!!)
2021-12-28 04:42:42 +10:30
niftynei 9b1a0f9b7b coin-moves: remove penalty + chain fee types
FIXME: still has coin tracking stuff!
2021-12-28 04:42:42 +10:30
Dustin Dettmer 6cee603f71 Add pip requirements install to Mac instructions 2021-12-24 15:49:01 +01:00
Anand Suresh 0c487a61ba Fix comment to reflect the code
The comment mentions a 100MB log book, but the code was modified 15 months ago to allocate a 10MB log book to preserve memory.
2021-12-21 10:59:25 +10:30
Vincenzo Palazzo 5bbe3feee7 doc: document the errors code
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-12-15 12:24:54 +10:30
Vincenzo Palazzo 43ff949ea7 lightningd: support hsm error code
Suggested-by: Rusty Russell
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Changed: Support hsm specific error error code in lightning-cli
2021-12-15 12:24:54 +10:30
Rusty Russell 9a85b02c6f wire: add missing spec patch.
You have to manually run "make extract-bolt-csv" to recreate the
spec files, so we didn't notice that I forgot to check this in :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-15 06:53:39 +10:30
Rusty Russell 064e239ae1 ccan: update, add graphql module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-15 06:53:23 +10:30
Christian Decker a98ccac777 pay: Disable channels that are not in normal state
Since we have the exact state of the channels from the `listpeers`
response we can filter out the ones that are not yet normal or not
anymore. This is more of a safety net, given that the `gossip_store`
should contain local disables, but better not be racy :-)

Changelog-None
2021-12-14 15:07:45 +10:30
Rusty Russell 1ac5a1b7e2 pytest: restore gossip advertizement tests for default port.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-14 13:41:31 +10:30
Vincenzo Palazzo 82124b7733 doc: force mistune==0.8.4 during pip package resolution.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-None: doc: force mistune==0.8.4 during pip package resolution.
2021-12-14 11:27:46 +10:30
lightning-developer bf71d250a7 Removed FIXME to match bolts PR 918
If PR 918 is merged to the bolts this requirement will be dropped from the spec and we do not need to have a FIXME in the code base.
2021-12-14 11:26:47 +10:30
Ken Sedgwick 335ef3fb69 hsmd: Add hsmd_ready_channel 2021-12-14 11:24:52 +10:30
Ken Sedgwick bb574be839 hsmd: Add hsmd_new_channel 2021-12-14 11:24:52 +10:30
Ken Sedgwick e8f43ef6ca wallet: make wallet_can_spend non-static because needed 2021-12-14 11:24:52 +10:30
ZmnSCPxj jxPCSnmZ d088288daa bitcoin/chainparams.h: Split BIP173 name into onchain and Lightning HRPs.
Fixes: #4937
2021-12-14 11:21:35 +10:30