Commit Graph

7046 Commits

Author SHA1 Message Date
darosior cd11c2050c devtools/gossipwith: add a "network" option
If specified, this will add the corresponding chain_hash to the init message.
2019-11-29 21:17:08 +01:00
darosior 3322048774 connectd: add network to init message
Changelog-Added: protocol: We now signal the network we are running on at init.
2019-11-29 21:17:08 +01:00
darosior 4b13b88f6c bitcoin/chainparams: add an utility to retrieve chainparams for all networks 2019-11-29 21:17:08 +01:00
darosior e6b8a02446 connectd: setup chainparams
We are going to signal the genesis block hash in the init message.
2019-11-29 21:17:08 +01:00
Christian Decker c84473f82c hsm: Stabilize the hsm encryption and decryption tests
We were using sleeps to hope we catch the password prompt. This makes the test
flaky. So I added a help text followed by a `fflush` to make sure we catcht he
right moment, instead of guessing. The `fflush` is also useful for debugging
if a user ever pipes the output to a file it'd get buffered and the user would
wait forever. The same applies for automated systems such as `expect` or
`pexpect` based scripts that enter the password on prompt.
2019-11-29 15:06:39 +01:00
Christian Decker 9e59740268 pyln-testing: Do not require a conftest.py to annotate tests
In the c-lightning tests we have `tests/conftest.py` which annotates test
function with the outcome. If we use pyln-testing outside of the c-lightning
tree we cannot rely on that annotation being there, so we assume it passed.
2019-11-29 15:06:39 +01:00
Christian Decker 3c5ed15b2c pyln: Add psycopg2-binary as a dependency
Using the psycopg2-binary package means that the apropriate compiled binary
for the user platform will be shipped alongside the python binaries. Otherwise
the python bindings and the C shims would be shipped which would then require
the postgres development packages as well.

This just makes things easier, since we don't require the build dependencies.
2019-11-29 15:06:39 +01:00
Rusty Russell b9a03a08ef wallet/Makefile: fix dependency line.
The .o files need regen, not the .c files!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-28 11:50:13 +01:00
Rusty Russell 493c2ab1d7 openingd: clean up and fix minor leak.
test_openchannel_hook_1:
MEMLEAK: 0x557593c164e8'
  label=wire/fromwire.c:320:char[]'
   backtrace:'
     ccan/ccan/tal/tal.c:437 (tal_alloc_)'
     ccan/ccan/tal/tal.c:466 (tal_alloc_arr_)'
     wire/fromwire.c:320 (fromwire_wirestring)'
     openingd/gen_opening_wire.c:205 (fromwire_opening_got_offer_reply)'
     openingd/openingd.c:1067 (fundee_channel)'
     openingd/openingd.c:1279 (handle_peer_in)'
     openingd/openingd.c:1535 (main)'
   parents:

fromwire_opening_got_offer_reply() allocates two fields off NULL:
err_reason and our_upfront_shutdown_script.  err_reason is used
immediately afterwards (and was the leak detected here), so fixing
that is easy.

To fix the leak of our_upfront_shutdown_script, it makes sense to simply
make it a member of 'state'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-28 11:50:13 +01:00
darosior 670f92002d tools/gen/impl_template: correct one-line for-loops indentation 2019-11-26 21:30:25 +01:00
darosior 5fb8e0aade tools/gen/impl_template: correct tlvs fromwire's for-loop 2019-11-26 21:30:25 +01:00
gorazdko a3961c4e06 unit test: run sphinx unit test by default 2019-11-26 21:25:25 +01:00
gorazdko d5a6072642 unit test: fix test vectors in run-sphinx 2019-11-26 21:25:25 +01:00
gorazdko e8ce9d0327 unit test: fix run-sphinx dependencies and update mocks 2019-11-26 21:25:25 +01:00
fiatjaf 17bb862cfe don't enforce description_hash checking 2019-11-25 18:15:35 +01:00
Michael Schmoock 6ed320189e feat: restructure plugin and options in listconfigs
This will change the command `listconfigs` output in several ways:

 - Deprecated the duplicated "plugin" JSON output by replacing it with
 - a "plugins" array with substructures for each plugin with:
 - path, name and their options

Changelog-Changed: JSON-RPC: `listconfigs` now structures plugins and include their options
Changelog-Deprecated: JSON-RPC: `listconfigs` duplicated "plugin" paths
2019-11-25 18:15:02 +01:00
Rusty Russell 8393d21503 common/features: add `payment_secret` feature if EXPERIMENTAL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 3b37c9d426 common/features: clean up feature handling for different cases.
The spec is (RSN!) going to explicitly denote where each feature should
be presented, so create that infrastructure.

Incorporate the new proposed bolt11 features, which need this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 9765642bf8 lightningd: use final_tlv if the payment_secret is supplied.
This implies the final node understands TLV onion.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell e5247a68b6 lightningd: check payment secret on htlc receipt.
We don't set the secret to compulsory (yet!) but put code in for the
future.  Meanwhile, if there is a secret, check it is correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell c2e8531e8e lightningd: add secrets (and associated feature) to invoices.
We don't need a new db column, since we can just derive the secret from
the preimage as required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 18e9144675 plugins/pay: hand payment_secret from bolt11 through to sendpay.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell dfb1f6a096 JSON: add `payment_secret` parameter to sendpay.
This is not documented yet, since it's ignored unless EXPERIMENTAL_FEATURES
is set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 854c64ffee common/bolt11: add secret support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 81c89aaef8 wallet: add invoice features into db.
In a future version, we will use features to insist that payers
provide the secret.  In transition, we may have old invoices which
didn't insist on that, so we need to know this on a per-invoice basis.

Not sure if I got the right syntax for adding an empty blob though!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 09cdbb70eb common: make BOLT11 use the normal feature array.
This was decided at a recent spec meeting: in particular, mpp and
var_onion_optin options will be used here.

We enhanced "features_supported" into "features_unsupported" so it
can return the first un-handlable bit number.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell ebac3d2a85 spec: update to experimental BOLTs with secret/total_amount.
Also pulls in a new onion error (mpp_timeout).  We change our
route_step_decode_end() to always return the total_msat and optional
secret.

We check total_amount (to prohibit mpp), but we do nothing with
secret for now other than hand it to the htlc_accepted hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 50d6941e89 lightningd: remove redundant htlc_accepted_hook_payload fields
Now we cache them in the route_step, don't need to copy them here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell fb040bc619 Spec: trivial update to latest.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell 14997f6237 lightningd: fix handling of truncated config options.
Do the same thing '--help' does with them; append `...`.

Valgrind noticed that we weren't NUL-terminarting if answer was over
78 characters.

Changelog-Fixed: JSONRPC: listconfigs appends '...' to truncated config options.
2019-11-23 22:42:34 +00:00
Rusty Russell f5831e5d10 config: limit depth on includes.
Don't spend too much effort on it, but this is better than running out
of memory and crashing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell badeaf1a67 pytest: upgrade tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell a06fa7f267 log: print UNUSUAL messages before log initialization too.
Otherwise we don't print out the upgrading messages when we move things!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell 57a32ee044 pytest: test for config parsing errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell 747bb99575 lightningd: keep pid files in top-level config dir.
They're already qualified with network name, and there's little point
moving them; it might even be dangerous if multiple are running.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell e3dbd78536 config: explicitly disallow nonsensical options.
1. "conf" can't be specified in a configuration file.
2. "lightning-dir" can't be specified in a configuration file unless the file
   was explicitly set with --conf=.
3. "network" options can't be set in a per-network configuration file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell 36c517bac5 common: change default network from testnet to mainnet for new installs.
Changelog-changed: Default network (new installs) is now bitcoin, not testnet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell a56f2b25b0 common: parse --allow-deprecated-apis extremely early.
We're going to want this for changing the default network.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell d512bcb85f lightningd: automatically move files for existing deployments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell dc23c308e4 config: Read both top-level and network-subdir config files.
This lets you have a default, but also a network-specific config.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: `config` and <network>/`config` read by default.
2019-11-23 22:42:34 +00:00
Rusty Russell 8b1aa3ef8b lightningd: move basic parameter parsing into common/configdir
lightning-cli is going to need to know what network we're on, so
it will need to parse the config files.  Move the code which does
the initial bootstrap parsing into common, as well as the config
file parsing core.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell 5673607ebc lightningd: don't use chainparams before param() call in json_fund_channel_start
With coming changes, this will segfault if we access it when param
code is trying to get usage from functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell 34c89cb226 config: Add include directive support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: configuration files now support `include`.
2019-11-23 22:42:34 +00:00
Rusty Russell 6defc69482 channeld: allow transient negative balance.
Travis randomly picked up an error in test_feerate_stress:
**BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: Cannot add htlc #0 10000msat to LOCAL (version a2541b9-modded)

This is because it hit an unlikely corner case involving applying multiple HTLCs
(similar to the previous c96cee9b8d).

In this case, the test sends a 500,000,000 "balancing" setup payment L1->L2.
It waits for L2 to get the preimage (which is the when pay() helper returns),
but crucially, it starts spamming with HTLCs before that HTLC is completely
removed.

From L2's point of view, the setup HTLC is in state RCVD_REMOVE_REVOCATION;
gone from L1's commitment tx, but still waiting for the commitment_signed
from L1 to remove it from L2's.

Note that each side keeps a local and remove view of both sides' current
balances: at this point, L2's view is REMOTE: "500,000,000 to L1, 499,900,000
to L2", LOCAL: "500,000,000 to L1, 0 to L2".

L2 sends a 10,000 msat HTLC to L1: legal, since L1 will allow it,
then the commitment_signed.  L1 sends the revoke-and-ack for this,
*then* belatedly follows with the commitment_signed which both completes the
removal of the setup HTLC and adds the new one.

But L2 processes the HTLCs in hashtable (i.e. random) order: so if it
tries to apply its own HTLC first, it freaks out because it doesn't have
funds in its local view.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Unlikely corner case is simultanous HTLCs near balance limits fixed.
2019-11-22 23:31:54 +00:00
Christian Decker d1df4d6959 htlc: Add a checker function tellung us whether we can continue
This function ensures we have all the infos we need to continue if the
htlc_accepted hook tells us to. It also enforces well-formedness of the TLV
payload if we have a TLV payload.

Suggested-by: List Neigut <@niftynei>
Signed-off-by: Christian Decker <@cdecker>
2019-11-22 04:40:25 +00:00
Christian Decker e46f423ae9 sphinx: Cleanup route_step_decode_* functions
We have consolidated the two functions into a single `route_step_decode`
function, and made it static since we call it in the `process_onionpacket`
function. We remove the two exposed functions since they're no longer useful.
2019-11-22 04:40:25 +00:00
Christian Decker d7b28ac480 htlc: Consolidate validation after the htlc_accepted hook returns
This now enforces all rules for validity, both for the TLV format and checking
that the required fields have been provided.
2019-11-22 04:40:25 +00:00
Christian Decker fc14e5eab0 htlcs: Make necessary payload fields optional and derfer validation
We make the fields in `htlc_accepted_payload` optional (NULL if not present in
the payload) and defer validation till after the hook call.
2019-11-22 04:40:25 +00:00
Christian Decker baffa84291 sphinx: Decode payload and place shortcuts in the route-step
We'll need to pass them around anyway, so just make them easier to access by
doing a bit more to `process_onionpacket`.
2019-11-22 04:40:25 +00:00