Commit Graph

494 Commits

Author SHA1 Message Date
niftynei a0cb7bd9b4 df: update reserve calculation
We can use the funding amount to derive the reserve requirement.
2021-03-06 15:03:56 +10:30
niftynei d47358848f df-rbf: init an rbf for real, using openchannel_bumpfee
Reorg a bit of the RBF code so we use the same codepaths for we-init vs
they-init starts.
2021-03-06 15:03:56 +10:30
niftynei 063948d51a df-feerates: send back original + last known feerates
For calculating the next RBF attempt's feerate, we need some historical
information about the feerate used for this channel
2021-03-03 16:19:04 -06:00
niftynei fe688ab718 df: revert channel state in dualopend
Now that RBF's are possible, we revert the channel's state
after a failure to the previous channel info.
2021-03-03 16:19:04 -06:00
niftynei 628f9df320 df: consolidate callbacks for opener/accepter, move to open_attempt 2021-03-03 16:19:04 -06:00
niftynei 6474779e38 df-rbf: hook for rbf_init attempt, called "rbf_channel"
When we get an RBF request, we ask the/a plugin what they'd like to do
about it. This pipes the request through to the plugin
2021-03-03 16:19:04 -06:00
lisa neigut 0d8351155e df-rbf: validate that the candidate funding transaction is valid
We need to make sure that there's at least one input that's represented
in every single RBF-attempt for this channel, to prevent "parallel"
subsequent RBFs from succeeding/opening (the multi-channel backdoor?!)
2021-03-03 12:28:22 +10:30
niftynei b30489310b df-rbf: break out opener commitment codepath, use for rbf 2021-03-03 12:28:22 +10:30
niftynei 63b060066b df-rbf: opener setup for "interactive" protocol portion
If we're the opener, there's a bit of extra setup we need to do before
calling "run_tx_interactive", namely adding the funding_output to the
PSBT
2021-03-03 12:28:22 +10:30
niftynei e67d5d7559 df: move out logic for adding funding output to PSBT 2021-03-03 12:28:22 +10:30
niftynei 804b93c81f df-rbf: split out "commitment txs" portion of accepter
use it for received RBFs requests, when we're the "accepter"
2021-03-03 12:28:22 +10:30
niftynei 22078262f0 df-reinit: pass back channel_flags
We should also restore the channel_flags field on reinit; we use them if
we do an RBF after a reconnection
2021-03-03 12:28:22 +10:30
niftynei 940347b04e df: put channel_flags onto state
They need to exist through multiple open channel attempst (via RBF)
2021-03-03 12:28:22 +10:30
niftynei a489f92ee8 df-rbf: start of handling an incoming RBF attempt from peer
A peer init's an RBF, we start handling it
2021-03-03 12:28:22 +10:30
niftynei 067f1f2eb7 df-rbf: add method to fail an RBF
this is a bit different than straight up failing a channel. we want to
signal that the RBF attempt failed, only
2021-03-03 12:28:22 +10:30
niftynei 54d5cdb938 df-rbf: update wires with ack and fail rbf 2021-03-03 12:28:22 +10:30
niftynei cf2f0c32d1 df-rbf: refactor state for opening attempts
Since a channel can now have multiple successful funding transactions
constructed for it, we need space to express this in the states.
2021-03-03 12:28:22 +10:30
Rusty Russell d0946b75bc common: support opt_shutdown_anysegwit checks (EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-02 14:42:34 +10:30
Rusty Russell 26627bdf7d openingd: check upfront shutdown script.
The spec doesn't say to do this, but it makes sense, otherwise
they'll never be able to mutually close the channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-02 14:42:34 +10:30
Rusty Russell 93d3b0d126 doc: update BOLT quotes (to include LOW-S-standard rule).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 13:23:16 +10:30
niftynei fbaf8aa6d9 df: more descriptive error message 2021-02-24 10:06:42 -06:00
Rusty Russell 6b11cc8b8c common: disallow NULL channel_id to peer_failed_err.
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
2021-02-04 12:02:52 +10:30
Rusty Russell f4ee41a989 common: remove peer_failed in favor of peer_failed_warn/peer_failed_err
And make all the callers choose which one.  In general, I prefer warn,
which lets them reconnect and try again, however some places are either
stated that they must be errors in the spec itself, or in openingd
where we abandon the channel when we close the connection anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we now send warning messages and close the connection, except on unrecoverable errors.
2021-02-04 12:02:52 +10:30
Rusty Russell d14e273b04 common: treat all "all-channels" errors as if they were warnings.
This is in line with the warnings draft, where all-zeroes in a
channel_id is no longer special (i.e. it will be ignored).

But gossipd would send these if it got upset with us, so it's best
practice to ignore them for now anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we treat error messages from peer which refer to "all channels" as warnings, not errors.
2021-02-04 12:02:52 +10:30
Rusty Russell a7c5a1f1d2 lightningd: implement receiving warnings.
This takes from the draft spec at https://github.com/lightningnetwork/lightning-rfc/pull/834

Note that if this draft does not get included, the peer will simply
ignore the warning message (we always close the connection afterwards
anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now report the new (draft) warning message.
2021-02-04 12:02:52 +10:30
Rusty Russell 6dab04ebd5 tools/generate-wire.py: include digits in #ifndef idempotent header.
Otherwise check complains that it's '#ifndef LIGHTNING_WIRE_BOLT_WIREGEN_H':

wire/bolt12_wiregen.h seems to be missing the expected include guard:
  #ifndef LIGHTNING_WIRE_BOLT12_WIREGEN_H
  #define LIGHTNING_WIRE_BOLT12_WIREGEN_H
  ...
  #endif /* LIGHTNING_WIRE_BOLT12_WIREGEN_H */

make: *** [Makefile:458: check-includes] Error 1

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
niftynei 7e324177de df-tests: simultaneous openchannel_init (while in progress)
Reject a peer's request to open a channel while we're already in progress
2021-01-10 13:44:04 +01:00
niftynei 205a7057c9 df: use dev-env flagged upfront shutdown script
This lets the test_option_upfront_shutdown_script test pass
2021-01-10 13:44:04 +01:00
niftynei 5387f6736c df: free psbt on negotiation failed 2021-01-10 13:44:04 +01:00
niftynei 2cd95aa806 df: add a new 'channel_open_failed' notification
Let plugins know when a channel open has failed.

We need to notify accepters now too, so we remove the check on who's
funding the channel before sending the 'failed' message from
dualopend->master.
2021-01-10 13:44:04 +01:00
niftynei d4ec052668 df: include channel_id in openchannel2 hook
We already know what the channel id is, we should go ahead and pass it
on to any listening plugins -- this makes clean up easier/possible
if a open fails early on and we've got reserved utxos.
2021-01-10 13:44:04 +01:00
niftynei 99a621dd99 df-reconnects: allow tx-sigs in channeld iff we're reconnecting
There's a case where a dropped funding_locked will result in the peer
moving onto channeld, while we stay in dualopend. As we haven't
received their funding_locked, we retransmit tx_sigs, which channeld
will need to handle.

With the patch the peer drops it on the floor; the peer will resend
funding_locked on reconnect, which will correctly advance us to
channeld and CHANNELD_NORMAL
2021-01-10 13:44:04 +01:00
niftynei 0a9c562664 df: state, clean more things up 2021-01-10 13:44:04 +01:00
niftynei 2f3acb91fc df: actually send a temp-id to the peer in open_channel2 WIP
We actually need a channel_id in openchannel so we can properly
attribute any errors that we get back
2021-01-10 13:44:04 +01:00
niftynei f62f8b9c6b df: assume this is a 'negotiation failure'
The most likely reason this failed is that we didn't like parameters the
peer sent us.
2021-01-10 13:44:04 +01:00
niftynei 9cffa769b8 df: actually run 'reconnect' code 2021-01-10 13:44:04 +01:00
niftynei 9984c34f14 df: handle locked in etc in dualopend 2021-01-10 13:44:04 +01:00
niftynei 9fdf1ea32a df: reinitialize dual-funding
Prior to this, all reconnect logic lived in channeld. If you
disconnected before we finished building a funding transaction, that was
no big deal. Now, however, we're waiting for the funding to lock in in
dualopend, instead of handing straight to channeld to wait.

So we need a way to restart dualopend.
2021-01-10 13:44:04 +01:00
niftynei 1b6deaffc8 df: bunch of driveby formatting fixes 2021-01-10 13:44:04 +01:00
niftynei 924aaf180e temp_channel_id: move to common
We actually do need this for dualopend's, specifically for returning
errors to open_channel2
2021-01-10 13:44:04 +01:00
niftynei c3a387a6d8 dual-fund: use correct 'last_tx'
We were incorrectly saving the *remote's* commitment transaction as the
'last_tx' for a peer, not our own local one.

When we applied the 'remote_commit_sig' to it, it would fail since the
remote's signature doesn't validate for their commitment transaction.
2021-01-10 13:44:04 +01:00
niftynei 90d936bb60 df: handle shutdown (before lockedin) in dualopend
A channel can be closed before it locks in. This lets dualopend handle
shutdown messages from the peer before the channel is locked in.
2021-01-10 13:44:04 +01:00
niftynei ae532614a3 df: handle fails
We weren't handling failure messages, now we handle them.

Also, fail/kill dualopend if there's a failure.
2021-01-10 13:44:04 +01:00
niftynei d47992a46d billboard: add in billboard updates from dualopend
Uses shared messaging now!
2021-01-10 13:44:04 +01:00
niftynei 1ea4e63331 dual-open: handle sigs and channel lockin in dualopend, not channeld
This will make it possible to do RBF, since we can re-start the opening
process in dualopend while waiting for lock-in.

Note the new channel states are being used, DUALOPEND_INIT and
DUALOPEND_AWAITING_LOCKIN, to differentiate from openingd/channeld opens
2021-01-10 13:44:04 +01:00
niftynei e0a7907d79 dual-fund: don't kill dualopend after commitments exchanged
We're going to handle the funding tx sigs here, so we keep dualopend
open still, so we can get them in/out before moving on
2021-01-10 13:44:04 +01:00
niftynei 0bb483f9b1 whitespace changes, 80 chars etc 2021-01-10 13:44:04 +01:00
niftynei 21a1f71732 df nit: capitialize acronym
Suggested-By: Christian Decker @cdecker
2021-01-10 13:44:04 +01:00
niftynei 260bd04adb df: move tx_sigs from channeld to dualopend
Non-functional yet, but this gets all the pieces in the right places,
rips the signature signing functionality out of channeld.
2021-01-10 13:44:04 +01:00
niftynei 55bbdf1866 df: update an error message to give more info 2021-01-05 19:12:00 +01:00
Rusty Russell a11edebb7c utf8: handle UTF-8 arrays.
BOLT 12 introduces this as a new fundamental type, which lets us easily
validate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell ff8830876d wire/tlvstream: add tlv_make_fields helper to populate ->fields array.
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!

Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell 5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire.
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
niftynei 9f74dd9fb3 df: rename the dual_open wires to match other daemon wires 2020-12-02 14:19:08 +10:30
niftynei 5e71de7819 df-spec: fail the open if they send us too many tx-collab msgs 2020-11-23 12:41:05 -06:00
niftynei 1e173463c9 df: rename 'feerate_per_kw' to 'feerate_per_kw_commitment'
Differentiate btw feerate for commitment txs and feerate for funding
transaction.
2020-10-27 19:52:05 -05:00
niftynei 4ea9d9e928 df: Pass new feerate options through to plugin, set reasonable bounds
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.

We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.

In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
2020-10-27 19:52:05 -05:00
niftynei 97fd18f0b5 df: incorporate a few spec changes -- serial_id is now 64-bits
And we pass 3-params for feerate so it's a 'pick a range' conversation.
2020-10-27 19:52:05 -05:00
niftynei 7a2e72f56c dualopend: handle dev-memleaks; fix placement of return on DEV options 2020-10-26 13:04:42 +10:30
niftynei 396c406983 dualopend: re-order 'handle_dev_memleak' occurrence 2020-10-26 13:04:42 +10:30
Rusty Russell a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
niftynei ae825fff26 opening: use correct dust_limit for reserve floor
Fixes #4140

Reported-By: @PsySc0rpi0n
Changelog-Fixed: openingd now uses the correct dust limit for determining the allowable floor for a channel open (affects fundee only)
2020-10-21 11:18:31 +10:30
niftynei 4a1843a151 df, bugfix: use tal_wally around an allocation 2020-10-21 06:22:18 +10:30
niftynei 5c04ff1ad7 df: Pass the serial_id of the funding output to openchannel_init caller
This is handy/necessary for getting multifundchannel to work, as we need
to know what output to tell all the other peers about.

Changelog-Added: Experimental!! JSON-RPC: openchannel_init returns a field `funding_serial` that indicates the serial_id of the funding output in the provided PSBT
2020-10-21 06:22:18 +10:30
niftynei 8bf9b4132b df: simplify `check_balances`, add spec quotes
`check_balances` had a weird interface because it was meant to be able
to be used at any 'intermediate' point to verify that a single side had
a valid inputs/output balance.

This was worse than useless. Now it just straight checks for both sides'
balances are correct and that both sides pay their fees. Called after
transaction is constructed.
2020-10-21 06:22:18 +10:30
niftynei 6926711f45 dualopend, nit: move some lines to be within 80chars
Random tidy of a few lines to be <=80 characters
2020-10-21 06:22:18 +10:30
niftynei 4508584b21 dualfund: rearrange things so that the wire-dependent calls are separate
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.

This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
2020-10-20 14:27:19 +10:30
niftynei 8317957db2 feerate: remove duplicate method
is dupe of `bitcoin_tx_core_weight`
2020-10-20 14:27:19 +10:30
niftynei 4034d0c306 psbt: have the unknown map 'add' be a 'set' instead 2020-10-20 14:27:19 +10:30
niftynei 085c590a51 dualopen: use separate wire for passing updated PSBTs back to dualopend
Rusty pointed out that having an empty channel_id is suboptimal; adding
another call is probably the right idea rather than re-using an existing
one.

Suggested-By: @rustyrussell
2020-10-20 12:50:31 +10:30
niftynei 33769cca4f dualopend: reverse polarity of send_next
Suggested-By: @rustyrussell
2020-10-20 12:50:31 +10:30
niftynei 41ebf71e26 psbt: new methods for generating serial_ids for an input/output
we need to do this elsewhere later, pull it out so we can use it
2020-10-20 12:50:31 +10:30
niftynei a7f29f30db df-open: pathway for getting a commit back from peer
Goes all the way back to where we save it to the database and return
whatever command kicked this off
2020-10-20 12:50:31 +10:30
niftynei 3a405c33e6 dualfund: Pass in expected remote's serial parity
Now that we've got the opener in progress, we need to be able to toggle
which parity to check a remote's serial_ids for
2020-10-20 12:50:31 +10:30
niftynei 06c41a0547 dualfund: opener, openchannel_init command (1/3)
There are 3 commands for opening a channel with dualfunding.
`openchannel_init` is the first of these.

It initializes the open-channel dialog, and stops once we've run out of
updates (input/outputs) to send to the peer.
2020-10-20 12:50:31 +10:30
niftynei 2618ef10c3 tx_roles: pull up roles, rename
We're going to use these elsewhere/more widely so having them in common
and using a more generic name is a obvious place to start.
2020-10-20 12:50:31 +10:30
niftynei 99293844a1 dualopend: use status_failed, not peer_failed 2020-10-20 12:50:31 +10:30
niftynei 9c1675fcb9 dual-fund: remove flag for option_anchor_outputs
It's assumed true
2020-10-20 12:50:31 +10:30
niftynei 9af6e83393 dual-fund: check the max feerate also, not just min feerate
We cap beneath our max feerate
2020-10-20 12:50:31 +10:30
niftynei 82c0b48215 wires: towire/fromwire for wally_tx
We're eventually moving away from 'bitcoin_tx
2020-10-20 12:50:31 +10:30
niftynei 60be62a3bb openingd: patch test_opening_tiny_channel under EXP_FEAT
`test_opening_tiny_channel` fails if EXPERIMENTAL_FEATURES is on because
we don't include the anchor in our reserve if we're the channel opener.

Seems fine to include in all cases?
2020-10-16 10:27:07 +10:30
niftynei db2b013990 dual-fund: call daemon_shutdown only
daemon_shutdown calls common_shutdown internally; calling both frees the
same context twice
2020-09-23 15:51:13 +09:30
niftynei c5f748e716 dualopend: tiny billboard capitalization change 2020-09-23 15:51:13 +09:30
niftynei 5f6a2c05bd dualopend: check that we're not exceeding max allowable capacity 2020-09-23 15:51:13 +09:30
niftynei 8761468f42 dualopend: use status failed instead of peer failed 2020-09-23 15:51:13 +09:30
Rusty Russell f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 3c8049f32c bitcoin/psbt: psbt_input_add_unknown/psbt_output_add_unknown needs a tal ctx.
Since it allocates something, it needs a context (used in the next patch!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 480f671e91 bitcoin/psbt: psbt_txid needs a tal ctx.
It returns a wally_tx; it's an anti-pattern not to hand in a tal context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Antoine Poinsot 4b4fe2f4fb dual-openingd: msg_type should be dualopend_wire not u8
clang did the hard work here:

openingd/dualopend.c:958:42: error: result of comparison of constant 'WIRE_DUAL_OPEN_FAIL' (7003) with expression of type 'u8' (aka 'unsigned char') is always false
      [-Werror,-Wtautological-constant-out-of-range-compare]
        if ((msg_type = fromwire_peektype(msg)) == WIRE_DUAL_OPEN_FAIL) {
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~

Co-Authored-by: the clang compiler
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-23 13:46:02 +09:30
Matt Whitlock eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
niftynei b44e36b99e spec-update: get rid of max-witness-len
We can use a fixed value and close the channel if they don't cover their
amount; this wasn't really helping with anything other than setting a
floor for an expected feerate
2020-09-09 19:54:20 +09:30
niftynei aef5a5a0e5 psbt: psbt_has_serial_id -> psbt_find_serial_id
Cleans up some awkward spots in the code, makes the footprint a bit
neater

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei c50f377a85 psbt: pull out changeset logic into common, update API
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.

Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei 5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei 303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
niftynei cafdbbcd2d dualopend: new `dualopend` daemon, containing most of accepter side
oof. needs validation.
2020-09-09 19:54:20 +09:30
niftynei 72a098a4ad openingd: pull out check_configs into a common place
We'll re-use it for dualopend!
2020-09-09 19:54:20 +09:30
niftynei 864f2f3e21 channel_id: save to database, dont derive from funding_txid
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.

includes a migration for existing channels
2020-09-09 19:54:20 +09:30
Rusty Russell 0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes.
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
Rusty Russell ea819107eb common: remove funding_tx.
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell d8e8426b52 Makefile: remove EXPERIMENTAL_FEATURES marker from generated files
We're going to make experimental versions of these completely separate files.

Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn.  We can always force-rebuild when we change a rule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell 82f2f43425 Generated files: more merge fallout.
We changed the generation templates!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 11:10:21 +09:30
Rusty Russell 9b8f08a8a0 lightningd: return min-capacity-sat to its intended purpose.
And document exactly what it does: insist that an HTLC can pass of
this value (module assumptions of feerate).

Note that we remove the "is_opener" test from the capacity calculation
for anchor fees: it doesn't matter which side it is, someone has to pay
for anchor fees to it deducts from capacity.

This change breaks the test, which we rewrite.

Changelog-Changed: config: `min-capacity-sat` is now stricter about checking usable capacity of channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-01 12:27:39 -05:00
Rusty Russell 3c6af3efb4 Makefile: commit and preserve all the wiregenerated files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: we no longer require extra Python modules to build.
2020-08-31 21:33:26 -05:00
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 6d09c0eba9 openingd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell f495363bc4 openingd: add bolt quote about reserve calculation.
Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell a5d0c14d4d option_anchor_outputs: wire into all the subdaemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Jon Griffiths 95d3d65c62 wally: update to the latest wally version
Includes:
psbt: Use renamed functions for new wally version
psbt: Set the transaction directly to avoid script workarounds
psbt: Use low-S grinding when computing signatures
tx: Use wally_tx_clone from libwally now that its exported

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Rusty Russell fa829f23db amount: add amount_msat_scale, amount_msat_ratio, amount_{msat,sat}_div
It's not all that rare to do these operations, and requiring annotations
for it is a little painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
ZmnSCPxj jxPCSnmZ a6374ad431 openingd/openingd.c: Fix a mild deviation from BOLT#2.
Fixes: #3815

Changelog-Fixed: Fixed a deviation from BOLT#2: if both nodes advertised `option_upfront_shutdown_script` feature: MUST include ... a zero-length `shutdown_scriptpubkey`.
2020-07-06 14:53:49 +00:00
Rusty Russell 93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei c3ae44e296 psbt: don't crash if we can't add a partial sig
instead return a boolean indicating the success/failure of a sig set
2020-06-23 14:49:32 +02:00
niftynei 891f61ad48 channel_tx: add the commitment sig and pubkey data to the commit tx
needs to be update elsewhere too!
2020-06-11 13:13:13 +02:00
niftynei a04f0fe250 psbt: remove input_amounts from bitcoin tx
Instead we will stash them into the PSBT as a utxo/witness record (which
includes the amount)
2020-06-11 13:13:13 +02:00
niftynei bf4cac7fb8 tx: strip out witscript
now that witness script data is saved into the tx/psbt which is
serialized across the wire, there's no reason to use witscript to do
this. good bye witscript!
2020-05-21 18:45:07 +09:30
niftynei b076f40cf3 psbt: move witness script storage into the psbt
we can now keep associated witness data with the output in the psbt
struct, so we do that.
2020-05-21 18:45:07 +09:30
niftynei fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
Rusty Russell cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Christian Decker 30e4443eae openingd: Return the penalty base after funding completes 2020-05-07 15:05:39 +09:30
Christian Decker eb8eabcc3c txs: Move commit tx generation out of the signature computation
We need the txs around, so don't throw them away after generating them.
2020-05-07 15:05:39 +09:30
lisa neigut 0e20e3c5e7 df: rename 'funder' to 'opener'
Previously we've used the term 'funder' to refer to the peer
paying the fees for a transaction; v2 of openchannel will make
this no longer true. Instead we rename this to 'opener', or the
peer sending the 'open_channel' message, since this will be universally
true in a dual-funding world.
2020-05-04 10:22:26 +09:30
Rusty Russell 2f1502abf4 cleanup: make 'u8 *features' and 'struct feature_set *fset' more explicit.
It's almost always "their_features" and "our_features" respectively, so
make those names clear.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell 1d90f21833 lightningd / openingd: remove limits if we negotiate option_support_large_channel.
Note that now we check capacity once we've figured out which peer, which
broke a test (we returned "unknown peer" instead of "capacity exceeded"),
so we rework that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell cf43e44378 common/features: don't use internal global.
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell c95e58ad4b subdaemons: initialize feature routines with explicit feature_set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell d881a4bd66 BOLT: update to latest version.
This is all typo/clarity fixes, no substantive changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell 11149ef5a1 lightningd, openingd: remove active code inside assert().
We don't compile with NDEBUG defined, but if we did, this code would
vanish.  I did a quick audit, inspired by @ZmnSCPxj.

I actually hacked up something to compile with NDEBUG (many unused vars
resulted, and of course unit tests are allowed to rely on assert()), and
after this the testsuite still passes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-18 10:21:29 +10:30
Ken Sedgwick 5c8f881a75
hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation.
Changelog-Added: hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation by signing daemon.
2020-02-04 10:40:43 +10:30
Christian Decker 3af010f200 openingd: Forward custommsgs to lightningd 2020-01-28 23:50:52 +01:00
Christian Decker 3c88d5c8c4 openingd: Implement sendcustommsg handling in openingd
Most of the work is done in `lightningd`, here we just need to queue the
message itself.
2020-01-28 23:50:52 +01:00
Christian Decker a8fa55d275 wire: Add common messages that are independent of daemons
These messages may be exchanged between the master and any daemon. For now
these are just the daemons that a peer may be attached to at any time since
the first example of this is the custommsg infrastructure.
2020-01-28 23:50:52 +01:00
Rusty Russell 1099f6a5e1 common: use struct onionreply.
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Rusty Russell 1d0c433dc4 channeld: treat all incoming errors as "soft", so we retry.
We still close the channel if we *send* an error, but we seem to have hit
another case where LND sends an error which seems transient, so this will
make a best-effort attempt to preserve our channel in that case.

Some test have to be modified, since they don't terminate as they did
previously :(

Changelog-Changed: quirks: We'll now reconnect and retry if we get an error on an established channel. This works around lnd sending error messages that may be non-fatal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-13 16:36:18 +01:00
Rusty Russell 24d54f98ad channeld: use fee_states internally.
This is an intermediary step: we still don't save it to the database,
but we do use the fee_states struct to track it internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell b9d6e8bbaa common: add fee_states state machine for update_fee.
This uses the same state machine as HTLCs, but they're only
ever added, not removed.  Since we can only have one in each
state, we use a simple array; mostly NULL.

We could make this more space-efficient by folding everything into the
first 5 states, but that would be more complex than just using the
identical state machine.

One subtlety: we don't send uncommitted fee_states over the wire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell 493c2ab1d7 openingd: clean up and fix minor leak.
test_openchannel_hook_1:
MEMLEAK: 0x557593c164e8'
  label=wire/fromwire.c:320:char[]'
   backtrace:'
     ccan/ccan/tal/tal.c:437 (tal_alloc_)'
     ccan/ccan/tal/tal.c:466 (tal_alloc_arr_)'
     wire/fromwire.c:320 (fromwire_wirestring)'
     openingd/gen_opening_wire.c:205 (fromwire_opening_got_offer_reply)'
     openingd/openingd.c:1067 (fundee_channel)'
     openingd/openingd.c:1279 (handle_peer_in)'
     openingd/openingd.c:1535 (main)'
   parents:

fromwire_opening_got_offer_reply() allocates two fields off NULL:
err_reason and our_upfront_shutdown_script.  err_reason is used
immediately afterwards (and was the leak detected here), so fixing
that is easy.

To fix the leak of our_upfront_shutdown_script, it makes sense to simply
make it a member of 'state'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-28 11:50:13 +01:00
lisa neigut de16d0f0b4 openchannel hook: add new `close_to` field
Rounds out the application of `upfront_shutdown_script`, allowing
an accepting node to specify a close_to address.

Prior to this, only the opening node could specify one.

Changelog-Added: Plugins: Allow the 'accepter' to specify an upfront_shutdown_script for a channel via a `close_to` field in the openchannel hook result
2019-11-22 00:41:15 +00:00
Rusty Russell ce1049115a channeld: remove chainparams local parameter.
Use global everywhere.  This leaks into openingd a little, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-20 20:41:53 +01:00
lisa neigut 28cdccfb11 dev: add option flag for specifying temporary channel id
--dev-force-tmp-channel-id flag takes a 64-character hex string
to use as the temporary channel id. Useful for spec tests

[ Fixed crash in non-DEVELOPER mode --RR ]
Changelog-None
2019-11-13 05:51:02 +00:00
lisa neigut 422b4502d3 funding: add RPC arg to specify a 'close_to' address
Takes advantage of upfront-shutdown-script to permit users to
specify the close-to address for a channel at open, by adding
a `close_to` field to `fundchannel_start`.

Note that this only is in effect if `fundchannel_start` returns
with `close_to` set -- otherwise, peer doesn't
support `option_upfront_shutdown_script`.
2019-10-15 19:10:05 +02:00
Rusty Russell bd55f6d940
common/features: only support a single feature bitset.
This is mainly an internal-only change, especially since we don't
offer any globalfeatures.

However, LND (as of next release) will offer global features, and also
expect option_static_remotekey to be a *global* feature.  So we send
our (merged) feature bitset as both global and local in init, and fold
those bitsets together when we get an init msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-11 02:52:04 +00:00
lisa neigut 3458ee2a6e openingd: small formatting cleanups 2019-10-10 05:57:45 +00:00
Christian Decker d5f0c08a88 elements: Remove global is_elements variable in favor of chainparams
No need to keep duplicate globals.
2019-10-03 04:32:57 +00:00
Christian Decker ef7a63d8f8 elements: Move from a global is_elements to a global chainparams
We now have a pointer to chainparams, that fails valgrind if we do anything
chain-specific before setting it.

Suggested-by: Rusty Russell <@rustyrussell>
2019-10-03 04:32:57 +00:00
Christian Decker 3963b0040e elements: openingd needs to know it's using an elements chain
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell 3c9ca5d3e1 openingd: ensure that revents fields are initialized.
Valgrind error file: valgrind-errors.112365
==112365== Conditional jump or move depends on uninitialised value(s)
==112365==    at 0x1105E0: main (openingd.c:1504)
==112365== 
==112365== Conditional jump or move depends on uninitialised value(s)
==112365==    at 0x110604: main (openingd.c:1507)
==112365== 
==112365== Conditional jump or move depends on uninitialised value(s)
==112365==    at 0x110628: main (openingd.c:1510)
==112365==

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell 147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
lisa neigut a35677fa0f fundchannel: use plugin; delete interior impl
Switch over to using the fundchannel plugin.
2019-09-11 23:56:27 +00:00
lisa neigut 8854bc8ae6 fundchannel_start: don't return null message
When the peer sends back an error, we return null, but sending
a NULL message back to lightningd causes a parsing error on their
side. negotiation_failed already calls back the peer, all we need
to do here is exit.
2019-09-11 23:56:27 +00:00
Rusty Russell 87f0ee6351 channeld: set option_static_remotekey when negotiated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
darosior 0b0ad4c22d transition from status_trace() to status_debug 2019-09-10 02:02:51 +00:00
Rusty Russell c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
lisa neigut 4030ccf94c fix memleak: remove duplicated new_channel creation
We call this same exact code inside `funder_finalize_channel_setup`;
which results in a memleak since we don't free the previous before
clobbering it.
2019-09-05 01:32:48 +00:00
Rusty Russell 596972366d wire: always ignore unknown odd messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-05 09:54:30 +00:00
Rusty Russell cc70b9c4ec wire: use common/bigsize routines
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Christian Decker 9288a7906b tx: Add chainparams to struct bitcoin_tx as context
The way we build transactions, serialize them, and compute fees depends on the
chain we are working on, so let's add some context to the transactions.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
lisa neigut 32eaae0cb9 wire-gen: move in-house wire delcarations to new format
tidying things up!
2019-07-24 06:31:46 +00:00
Rusty Russell cd6d0ccde7 openingd: round *down* for reserve amount.
That's what other implementations do, so it simplifies the test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell 9fdcb1a122 openingd: remove unnecessary is_err flag.
It probably doesn't matter to "fundchannel_cancel" exactly why the
fundchannel didn't work (though it can read the error msg), and we
should always fail any pending fundchannel_complete command.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-08 15:14:31 +00:00
lisa neigut c00e0d2936 funding: rename fundchannel_continue -> _complete
Renaming. "complete" more accurately describes what we're doing here.
2019-06-12 02:22:43 +00:00
lisa neigut 5aad642c59 opening: add fundchannel_cancel command
Provide the option to cancel a funding-opening with a peer.
Must either call `fundchannel_cancel` or `fundchannel_continue`
2019-06-12 02:22:43 +00:00
lisa neigut 478762dbd2 funding: wire up funding_continue
Big wiring re-org for funding-continue

In openingd, we move the 'persistent' state (their basepoints,
pubkey, and the minimum_depth requirement for the opening tx) into
the state object. We also look to keep code-reuse between
'continue' and normal 'fundchannel' as high as possible. Both
of these call the same 'fundchannel_reply' at the end.

In opening_control.c, we remap fundchannel_reply such that it is
now aware of the difference between an external/internally funded
channel open. It's the same return path, with the difference that
one finishes making and broadcasting the funding transaction; the
other is skips this.
2019-06-12 02:22:43 +00:00
lisa neigut 7ea21c36b1 fundchannel: add txout field to RPC/API
We'll need the outpoint for the funding output.
2019-06-12 02:22:43 +00:00
lisa neigut dd11d3bd81 lightningd: add start for fundchannel_continue
Add an RPC method (not working at the moment) called
`fundchannel_continue` that takes as its parameters a
node_id and a txid for a transaction (that ostensibly has an output
for a channel)
2019-06-12 02:22:43 +00:00
lisa neigut 6e5928aa1e openingd: update billboard with funding info
Make it easier for a user to know what's going on with
a channel in `listpeers`.
2019-06-12 02:22:43 +00:00
lisa neigut 5920e656cf opening: wire up walking through open channel up thru accept
Fill in details to make fundchannel_start work.
2019-06-12 02:22:43 +00:00
lisa neigut 685aa47b65 openingd: pull out setup funder checks into separate method
Useful for adding the funder_start stuff
2019-06-12 02:22:43 +00:00
lisa neigut 8103acd9b3 opening: add entry point for `funding_start rpc command
Beginnings of wiring up the funding_start rpc command. missing
the part that actually starts the funding channel dance.
2019-06-12 02:22:43 +00:00
lisa neigut 846bc9cbc4 openingd: allow funding_failed to complete successfully
For the `fundchannel_cancel` we're going to want
to 'successfully' fail a funding channel operation. This allows
us to report it a failure back as an RPC success, instead of
automatically failing the RPC request.
2019-06-12 02:22:43 +00:00
Rusty Russell 728bb4e662 common/gossip_store: handle timestamp filtering.
This means we intercept the peer's gossip_timestamp_filter request
in the per-peer subdaemon itself.  The rest of the semantics are fairly
simple however.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell 5591c0b5d8 gossipd: don't send gossip stream, let per-peer daemons read it themselves.
Keeping the uintmap ordering all the broadcastable messages is expensive:
130MB for the million-channels project.  But now we delete obsolete entries
from the store, we can have the per-peer daemons simply read that sequentially
and stream the gossip itself.

This is the most primitive version, where all gossip is streamed;
successive patches will bring back proper handling of timestamp filtering
and initial_routing_sync.

We add a gossip_state field to track what's happening with our gossip
streaming: it's initialized in gossipd, and currently always set, but
once we handle timestamps the per-peer daemon may do it when the first
filter is sent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell 38d2899fbb common/per_per_state: generalize lightningd/peer_comm Part 1
Encapsulating the peer state was a win for lightningd; not surprisingly,
it's even more of a win for the other daemons, especially as we want
to add a little gossip information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell f1b4b14be5 channeld: don't queue gossip msgs while waiting for foreign_channel_update.
We ask gossipd for the channel_update for the outgoing channel; any other
messages it sends us get queued for later processing.

But this is overzealous: we can shunt those msgs to the peer while
we're waiting.  This fixes a nasty case where we have to handle
WIRE_GOSSIPD_NEW_STORE_FD messages by queuing the fd for later.

This then means that WIRE_GOSSIPD_NEW_STORE_FD can be handled
internally inside handle_gossip_msg(), since it's always dealt with
the same, simplifying all callers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell f3d30f1267 openingd: check with lightningd when we receive an offer.
Instead of lightningd telling us when it's ready, we ask it.
This also provides an opportunity to have a plugin hook at this point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-20 20:31:07 -04:00
Rusty Russell 6ee2cd8ce3 openingd: fix hangup when gossipd compacts.
My raspberry pi node hung up on my other node:
   lightning_openingd-... chan #1: Got bad message from gossipd: 0db1

This is because we didn't handle that message in one path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-16 15:54:17 -04:00
Rusty Russell f5a218f9d1 gossipd: send per-peer daemons offsets into gossip store.
Instead of reading the store ourselves, we can just send them an
offset.  This saves gossipd a lot of work, putting it where it belongs
(in the daemon responsible for the specific peer).

MCP bench results:
   store_load_msec:28509-31001(29206.6+/-9.4e+02)
   vsz_kb:580004-580016(580006+/-4.8)
   store_rewrite_sec:11.640000-12.730000(11.908+/-0.41)
   listnodes_sec:1.790000-1.880000(1.83+/-0.032)
   listchannels_sec:21.180000-21.950000(21.476+/-0.27)
   routing_sec:2.210000-11.160000(7.126+/-3.1)
   peer_write_all_sec:36.270000-41.200000(38.168+/-1.9)

Signficant savings in streaming gossip:
   -peer_write_all_sec:48.160000-51.480000(49.608+/-1.1)
   +peer_write_all_sec:35.780000-37.980000(36.43+/-0.81)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-13 05:16:18 +00:00
Rusty Russell d8db4e871f gossipd: provide new fd to per-peer daemons when we compact it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-13 05:16:18 +00:00
Rusty Russell 13717c6ebb gossipd: hand a gossip_store_fd to all subdaemons.
This will let them read from the gossip store directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-13 05:16:18 +00:00
Rusty Russell dddfdc7f10 pytest: add dev hack for testing option_upfront_shutdown_script.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-02 01:18:33 +00:00
Rusty Russell cfebe66762 openingd: support receipt of upfront_shutdown_script.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-02 01:18:33 +00:00
Michael Schmoock 41dd975aac chore: err_reason for initial_channel_tx initial_commit_tx 2019-04-09 13:20:52 +02:00
Michael Schmoock 898df57fa0 fix: openingd memory leaks when failing
- when opening was faling for reasons (i.e. channel too small) the fail
handling routing did not clean up *funding and *wscript.
2019-04-09 13:20:52 +02:00
trueptolemy 92b40cb68a fix:Add infor about how many blocks needed until funding is confirmed
1. Rename channel_funding_locked to channel_funding_depth in
channeld/channel_wire.csv.
2. Add minimum_depth in struct channel in common/initial_channel.h and
change corresponding init function: new_initial_channel().
3. Add confirmation_needed in struct peer in channeld/channeld.c.
4. Rename channel_tell_funding_locked to channel_tell_depth.
5. Call channel_tell_depth even if depth < minimum, and still call
lockin_complete in channel_tell_depth, iff depth > minimum_depth.
6. channeld ignore the channel_funding_depth unless its >
minimum_depth(except to update billboard, and set
peer->confirmation_needed = minimum_depth - depth).
2019-04-07 23:45:35 +00:00
arowser 8e8f5bbaba correct messge format 2019-03-19 00:07:39 +00:00
Rusty Russell 38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 28f5da7b2f tools/generate-wire: use amount_msat / amount_sat for peer protocol.
Basically we tell it that every field ending in '_msat' is a struct
amount_msat, and 'satoshis' is an amount_sat.  The exceptions are
channel_update's fee_base_msat which is a u32, and
final_incorrect_htlc_amount's incoming_htlc_amt which is also a
'struct amount_msat'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 3ac0e814d0 daemons: use amount_msat/amount_sat in all internal wire transfers.
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell bb00deeea4 channeld: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell b8e484b508 struct channel_config: use amount_sat / amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 85b8b25749 bitcoin/chainparams: use amount_sat / amount_msat
Simple changes, but ripples through the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 7fad7bccba common/amount: new types struct amount_msat and struct amount_sat.
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.

The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell b155dbb316 openingd: make sure we take utxos on success.
Otherwise recent additional checks in tal() complain that we're freeing a
take() pointer.  In this case, we're exiting so it's harmless, but it's
still a latent bug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-29 13:24:12 +01:00
Rusty Russell 9b9f9e4837 openingd: I can't believe we have even more typo fixes.
Reported-by: @wythe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell 2c3a53b88e openingd: even more typo fixes.
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell 6a04d2ced3 openingd: comment typo fixes
Reported-by: Conor Scott @connscott
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell 9d95eba6f2 openingd: document this in semi-literate style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell 338e4bab45 openingd: don't create trivial single-use is_all_channel_error function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell 1259e10761 openingd: plug UTXO leak on failed opening.
This existed previously, but code perturbations seem to have revealed it
now: test_bad_opening reports a leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell b50c670f08 openingd: plug a funding msg leak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell fc92ae515b openingd: don't send opening_funder_failed twice funder gets a general error.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell bc4a62d349 openingd: subtract *both* reserves for our "effective capacity" calculation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell 6d717d73a4 openingd: call it first_per_commitment_point not next_per_commit, as per BOLT 2.
And group struct state fields together into some kind of logical order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-12 11:42:24 -08:00
Rusty Russell dffe2f516a signature: wrap almost all signatures in struct bitcoin_signature.
This is prep work for when we sign htlc txs with
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY.

We still deal with raw signatures for the htlc txs at the moment, since
we send them like that across the wire, and changing that was simply too
painful (for the moment?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Rusty Russell a42c8bfb38 openingd: wire up dev_memleak.
This is a bit different from the other cases: we need to iterate through
the peers and ask all the ones in openingd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-22 05:15:42 +00:00
Rusty Russell 5c60d7ffb2 gossipd: split wire types into msgs from lightningd and msgs from per-peer daemons
This avoids some very ugly switch() statements which mixed the two,
but we also take the chance to rename 'towire_gossip_' to
'towire_gossipd_' for those inter-daemon messages; they're messages to
gossipd, not gossip messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
lisa neigut 73ea6d0038 BOLT 2 updates for fix placment of chain_hash req
See 4b62d26af9
2018-09-21 00:24:12 +00:00
Rusty Russell e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell 8f1f1784b3 hsmd: remove hsmd/client.c
It was only used by handshake.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Simon Vrouwe 94e42f2384 openingd: prioritize incoming peer traffic over handling (and sending out) gossip
- reduces probability for a deadlock where we block on sending data because
  the other peer cannot receive because it blocks on sending data etc.
- when either side sends so much data that it fills up the kernel/network buffer
- however sending out gossip can still block when (malicious) peer never receives
2018-09-20 03:28:42 +00:00
Christian Decker f417dfa0e1 chainparams: Always retrieve chainparams by the chain_hash 2018-09-14 21:18:11 +02:00
Christian Decker 0128bc7362 channeld: Use the chainparams to check msatoshi and funding_satoshi 2018-09-14 21:18:11 +02:00
Christian Decker 2402c524cc channeld: Keep track of the chainparams for the chain we are using 2018-09-14 21:18:11 +02:00
Rusty Russell 0d46a3d6b0 Put the 'd' back in the daemons.
@renepickhardt: why is it actually lightningd.c with a d but hsm.c without d ?

And delete unused gossipd/gossip.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell 11a1ec6e94 openingd: don't leak peer messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 36e31cacf9 openingd: don't leak messages from master.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell 035362e151 openingd: don't exit when we receive an error.
In particular, all opening_read_peer_msg() callers need to know there
was an error (presumably, negotiating) so they can stop, but we should
not exit.

This lets us reenable the final disabled test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 174c79acad openingd: tell master if funding failed, but don't exit.
We don't want to exit just because channel parameter negotiation
failed, but we do want to tell the master if it was a channel we were
trying to fund.

Note that lightningd still needs to fail the funding cmd if it gets a
fromwire_opening_fundee (they raced us and won), or an outright
failure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell b39ee8bef5 openingd: make remoteconf a non-pointer member.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 909cd4136b openingd: get told if we can't let them open a new channel.
Previously master would fail once the channel has been negotiated,
which is terrible, since the funder will have already broadcast tx.

Now we tell them if we have an active channel, and update if it goes away.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 50f5eb34b4 openingd: take peer before we're opening, wait for explicit funding msg.
Prior to this, lightningd would hand uninteresting peers back to connectd,
which would then return it to lightningd if it sent a non-gossip msg,
or if lightningd asked it to release the peer.

Now connectd hands the peer to lightningd once we've done the init
handshake, which hands it off to openingd.

This is a deep structural change, so we do the minimum here and cleanup
in the following patches.

Lightningd:
1. Remove peer_nongossip handling from connect_control and peer_control.
2. Remove list of outstanding fundchannel command; it was only needed to
   find the race between us asking connectd to release the peer and it
   reconnecting.
3. We can no longer tell if the remote end has started trying to fund a
   channel (until it has succeeded): it's very transitory anyway so not
   worth fixing.
4. We now always have a struct peer, and allocate an uncommitted_channel
   for it, though it may never be used if neither end funds a channel.
5. We start funding on messages for openingd: we can get a funder_reply
   or a fundee, or an error in response to our request to fund a channel.
   so we handle all of them.
6. A new peer_start_openingd() is called after connectd hands us a peer.
7. json_fund_channel just looks through local peers; there are none
   hidden in connectd any more.
8. We sometimes start a new openingd just to send an error message.

Openingd:
1. We always have information we need to accept them funding a channel (in
   the init message).
2. We have to listen for three fds: peer, gossip and master, so we opencode
   the poll.
3. We have an explicit message to start trying to fund a channel.
4. We can be told to send a message in our init message.

Testing:
1. We don't handle some things gracefully yet, so two tests are disabled.
2. 'hand_back_peer .*: now local again' from connectd is no longer a message,
   openingd says 'Handed peer, entering loop' once its managing it.
3. peer['state'] used to be set to 'GOSSIPING' (otherwise this field doesn't
   exist; 'state' is now per-channel.  It doesn't exist at all now.
4. Some tests now need to turn on IO logging in openingd, not connectd.
5. There's a gap between connecting on one node and having connectd on
   the peer hand over the connection to openingd.  Our tests sometimes
   checked getpeers() on the peer, and didn't see anything, so line_graph
   needed updating.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell 8f38a46584 lightningd: correctly store our own channel_reserve_satoshis
openingd calculates our reserve based on the channel amount (even if
we're funding, to keep the calculation in one place), but it wasn't
reporting it back to the master daemon.  We initialized it to 0 so that
valgrind wouldn't get upset, as it's part of a structure we send over
the wire.

Have openingd report back, and also initialize it to an impossible value
as extra assurance.  And remove a stray (harmless but weird) semicolon.

Reported-by: Gálli Zoltán
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-06 19:34:43 +02:00
Rusty Russell 136f10e4a3 common/read_peer_msg: remove.
Also means we simplify the handle_gossip_msg() since everyone wants it to
use sync_crypto_write().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell 90b43a6f47 openingd: use deconstructed read_peer_msg helpers.
This is more verbose but I still think it's clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell 0b08601951 sync_crypto_write/sync_crypto_read: just fail, don't return NULL.
There's only one thing the caller ever does, just do that internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell a52d522525 gossipd: handle ping messages for remote peers too.
This simplifies our ping handling: make gossipd always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 89b1cc69ef openingd: take hsmfd, use it to sign commitment transaction and get commitment_point
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-24 00:40:01 +02:00
Rusty Russell ad9dbaaa3f openingd: rename confusing 'our_commit' and 'their_commit'.
We use 'our_commit' for the commit we sign (ie. the remote commitment tx),
and vice versa.  Use local/remote nomenclature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell 6c98457ef2 per-peer seed is a 'struct secret' not a 'struct privkey'.
They're both 32 bytes, but it's not a privkey at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell 2d533dc82e channeld: don't manually disable channel.
gossipd will do it when peer dies anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell 82ff891202 Update to latest BOLT version.
And remove the FIXMEs now that the gossip_query extension is merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 17:37:03 +02:00
Rusty Russell 80182ce275 openingd: fix up BOLT references.
Some of these are from the master branch, and were not when the query-gossip
extensions were made, so I've had to mark some with FIXME.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell edf1b3cec9 More option cleanups.
Because we have too many which are never used and I don't want to document
them.

1. Remove unused anchor_onchain_wait.  When implemented, it should be
   hardcoded to 100 or more.
2. Remove anchor_confirms_max.  10 always reasonable, and we can readd
   an override option should someone need it.
3. max_htlc_expiry should be the same as locktime_max (which increases
   from 3 to 5 days by default): they're both a limit on how long
   funds can be locked up.
4. channel_update_interval should always be a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Saibato 877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
practicalswift abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
Rusty Russell b68fb24758 read_peer_msg: handle incoming gossip from gossipd.
This means that openingd and closingd now forward our gossip.  But the real
reason we want to do this is that it gives an easy way for gossipd to kill
any active daemon, by closing its fd: previously closingd and openingd didn't
read the fd, so tended not to notice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell ab9d9ef3b8 gossipd: drain fd instead of passing around gossip index.
(This was sitting in my gossip-enchancement patch queue, but it simplifies
this set too, so I moved it here).

In 94711969f we added an explicit gossip_index so when gossipd gets
peers back from other daemons, it knows what gossip it has sent (since
gossipd can send gossip after the other daemon is already complete).

This solution is insufficient for the more general case where gossipd
wants to send other messages reliably, so replace it with the other
solution: have gossipd drain the "gossip fd" which the daemon returns.

This turns out to be quite simple, and is probably how I should have
done it originally :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell 429c853fac opening: clearer messages for negotiation failures.
'negotiation_failed' is currently just a useless wrapper around
peer_failed (a vestige from when peer_failed would close the
connection).  Change it to send different local and remote messages,
and use it wherever we dislike their parameters: stick with
peer_failed if we dislike our own parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 21fbae6df8 openingd: ensure that initial channel can cover fees and reserve.
This is probably covered by our "channel capacity" heuristic which
requires the channel be significant, but best to be explicit and sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 441a5b8835 openingd: update check to latest bolt revision.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00
Rusty Russell 83f83f7685 openingd: make sure our their reserve isn't our dust, and vice versa.
This quotes from the BOLT proposal at https://github.com/lightningnetwork/lightning-rfc/pull/389

Don't try to fund channels which would do this, and don't allow others
to fund channels which would do this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-05 19:07:23 +02:00