Commit Graph

2727 Commits

Author SHA1 Message Date
Rusty Russell c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Michael Schmoock a3e18a6aba openchannel: make openchannel hook chainable
This will make the `openchannel_hook` chainable. Logic is as follows:
 - The first plugin that rejects terminates the chain
 - If more than one plugin uses the `close_to` parameter, take the first
   value and log_unusual for the others.

Changelog-Added: openchannel_hook is now chainable
2020-09-08 06:14:29 +09:30
Christian Decker d7ff74e8f6 json: Only start parsing once we're sure we have a message
PR #3957 improved performance considerably, however we still look over the
entire message for the message separator. If instead we just look in the
incrementally read data, we remove the quadratic behavior for large messages.

This is safe since we then loop over the messages which would drain any
message separator from the buffer before we attempt the next read.

Changelog-Fixed: bcli: Significant speedups for block synchronization
2020-09-02 13:21:32 +02:00
Christian Decker f2797f303a block: Compute the txids only once
We're not going to mutate transactions in a block, so computing the txids
every time we need them is a waste, let's compute them upfront and use them
afterwards.
2020-09-02 13:21:32 +02:00
Rusty Russell e4b5a1e0e8 dev: option to set what features we offer.
And use it to smoke-test !option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 10:52:49 +09:30
Rusty Russell 39e4796ae3 json_command: command_fail_badparam helper.
It's common to want to complain a token is not what we expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 09:46:37 +09:30
Rusty Russell 9b8f08a8a0 lightningd: return min-capacity-sat to its intended purpose.
And document exactly what it does: insist that an HTLC can pass of
this value (module assumptions of feerate).

Note that we remove the "is_opener" test from the capacity calculation
for anchor fees: it doesn't matter which side it is, someone has to pay
for anchor fees to it deducts from capacity.

This change breaks the test, which we rewrite.

Changelog-Changed: config: `min-capacity-sat` is now stricter about checking usable capacity of channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-01 12:27:39 -05:00
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell dd8cd81e91 listpeers: add `features` array using BOLT9 names.
It's actually not possible to currently tell if you're using anchor_outputs
with a peer (since it depends on whether you both supported it at *channel open*).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-added: JSON-RPC: `listpeers` shows `features` list for each channel.
2020-08-28 14:46:09 -05:00
Rusty Russell 496c0dd1e6 common/random_select: central place for reservoir sampling.
Turns out we can make quite a simple API out of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
ZmnSCPxj jxPCSnmZ ba3f38077e lightningd/invoice.c: Use round-robin channel selection.
Changelog-Changed: We now make MPP-aware routehints in invoices.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ a9a11265dd lightningd/lightningd.h: Maintain a round-robin list of channels. 2020-08-26 09:29:46 +09:30
Rusty Russell 3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 398b4806b9 connectd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell fe8564555f closingd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 6d09c0eba9 openingd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 8ae6740bb1 Makefile: create generic wiregen rules, use for channeld.
This means some files get renamed, and I took the opportunity to clarify
our naming (the *d* is important!)

1. channeld/channel_wire.csv -> channeld/channeld_wire.csv
2. channeld/gen_channel_wire.h -> channeld/channeld_wiregen.h
3. enum channel_wire_type -> enum channeld_wire
4. WIRE_CHANNEL_FUNDING_DEPTH -> WIRE_CHANNELD_FUNDING_DEPTH.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell ef3fbab551 invoice: don't allow zero-value invoices.
You can't pay them anyway, and at least one person used 0 instead of "any".

Closes: #3808
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice` no longer accepts zero amounts (did you mean "any"?)
2020-08-25 12:22:45 +09:30
Rusty Russell b09e519669 options: be more conservative with timeouts on mainnet.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: config: the default CLTV expiry is now 34 blocks, and final expiry 18 blocks as per new BOLT recommendations.
2020-08-24 20:55:47 +09:30
Rusty Russell 007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell c284b5bbf4 lightningd: always do incremental parsing of JSON inputs.
time lightning-cli -R --network=regtest --lightning-dir /tmp/ltests-k8jhvtty/test_pay_stress_1/lightning-1/ listpays > /dev/null

Before:
	real	0m42.741s
	user	0m0.149s
	sys	0m0.016s

After:
	real	0m13.674s
	user	0m0.131s
	sys	0m0.024s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: significant speedups for plugins which create large JSON replies (e.g. listpays on large nodes).
2020-08-21 09:52:33 +09:30
Rusty Russell 3ed03f9c8f common: make json_parse_input API retry friendly.
The jsmn parser is a beautiful piece of code.  In particular, you can parse
part of a string, then continue where you left off.

We don't take advantage of this, however, meaning for large JSON objects
we parse them multiple times before finally having enough to complete.

Expose the parser state and tokens through the API, so the caller can pass
them in repeatedly.  For the moment, every caller is allocates each time
(except the unit tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell cbd0be5c0e EXPERIMENTAL_FEATURES: enable option_anchor_outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell a730cbdc96 onchaind: recognize (and ignore) anchor outputs.
Without this, onchaind fails to identify them.  So onchaind now needs
to know the funding pubkeys.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell a5d0c14d4d option_anchor_outputs: wire into all the subdaemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell e7423888ba initial_commit_tx, commit_tx: add anchor outputs if needed.
This also means we subtract 660 satoshis more everywhere we subtract
the base fee (except for mutual close, where the base fee is still
used).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell d8d1014ca4 channeld: implement htlc sig checking changes for option_anchor_outputs.
This is best done by passing `struct bitcoin_signature` around instead
of raw signatures.  We still save raw sigs to the db, and of course the
wire protocol uses them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell 85e3b43176 channeld, openingd: take into account option_anchor_outputs for fees.
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell edf60b4f9e openingd: remember if we negotiated option_anchor_outputs, to put in the db.
And hand it through to channeld just like option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell cf25ca11b8 sendpsbt: annotate transactions in wallet.
This is what txsend does, only we have a psbt so we have
to change the db interface to take a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-13 12:37:02 -05:00
Christian Decker 639eaaf2b4 json-rpc: Do not return a null amount_msat to listpays
Suggested-by: Rusty Russell <@rustyrussell>
Changelog-Deprecated: JSON-RPC: `listsendpays` will no longer add `null` if we don't know the `amount_msat` for a payment.
2020-08-11 08:43:18 +09:30
Rusty Russell 7a7a849fc2 lightningd: allow plugin commands and options to mark themselves deprecated.
This lets us handle it the same way we handle builtin commands, and also
lets us warn if they use deprecated apis and allow-deprecated-apis=false.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: can now mark their options and commands deprecated.
2020-08-11 08:43:18 +09:30
Rusty Russell 151bc47583 JSON-RPC: getmanifest passes allow-deprecated-apis flag.
This allows plugins to choose how to present things in getmanifest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: `getmanifest` may now include "allow-deprecated-apis" boolean flag.
Changelog-Deprecated: plugins: `getmanifest` without any parameters; plugins should accept any parameters for future use.
2020-08-11 08:43:18 +09:30
Rusty Russell 930e294801 help: better handing of deprecated commands
If allow-deprecated-apis=false, don't mention them at all (we already
disallow calling them in that case).  Otherwise, note that they're
deprecated in the help msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:43:18 +09:30
Rusty Russell 5e0b03fba9 common: hoist param_bitcoin_address where plugins can use it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:39:24 +09:30
Vincenzo Palazzo 1521c29fcf listpays mod 1: add destination inside the response when bolt11 is null
Changelog-Added: JSON-RPC: `listpays` now lists the `destination` if it was provided (e.g., via the `pay` plugin or `keysend` plugin)
2020-08-09 16:03:03 +02:00
Rusty Russell fa829f23db amount: add amount_msat_scale, amount_msat_ratio, amount_{msat,sat}_div
It's not all that rare to do these operations, and requiring annotations
for it is a little painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell 36d43b871f amount: add amount_msat and amount_sat initializers.
Generally, importing amounts needn't be checked, and it cuts down on
the warnings we get.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
ZmnSCPxj jxPCSnmZ 1ded3fc52f lightningd/plugin.c: Add a `--dev-builtin-plugins-unimportant` for developers who want to mess around with the builtin plugins. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 4ca2e49812 lightningd/plugin.c: Make builtin plugins important.
Changelog-Changed: plugin: Builtin plugins are now marked as important, and if they crash, will cause C-lightning to stop as well.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ a847487bbe lightningd/plugin.c: Add important plugins, which if they terminate, lightningd also terminates.
Changelog-Added: New option `--important-plugin` loads a plugin is so important that if it dies, `lightningd` will exit rather than continue.  You can still `--disable-plugin` it, however, which trumps `--important-plugin` and it will not be started at all.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 50600dce95 lightningd/lightningd.c: Create API to exit lightningd with an exit code. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 3df2333d5d lightningd/plugin.c: Add specific function to give the directory for built-in plugins. 2020-08-04 13:27:51 -05:00
Christian Decker 3dafddd717 channel: Base the channel forget timeout on the headercount
We were basing the 2016 block timeout on the blockchain height that we had
processed at the time completed the funding, which could lag considerably
behind the network-wide blockheight. For example this could happen if we
started with `--rescan` from a low height, taking some time to catch up.

This means that we could end up forgetting channels even before reaching the
network blockheight. This patch instead uses the headercount reported by the
backend plugin if we aren't caught up yet. While the chances of this happening
are still there, the window it might happen are much reduced, since headers
can be synced in a couple of minutes.

Reported-by: Riccardo Masutti
Changelog-Changed: Funding timeout is now based on the header count reported by the bitcoin backend instead of our current blockheight which might be lower.
2020-08-04 09:14:17 +09:30
ZmnSCPxj jxPCSnmZ e7d89cd7de lightningd/invoice.c: Improve programmatic error reporting for `delinvoice`.
Changelog-Changed: JSON-RPC: `delinvoice` will now report specific error codes: 905 for failing to find the invoice, 906 for the invoice status not matching the parameter.
2020-07-31 23:57:32 +00:00
ZmnSCPxj jxPCSnmZ b646b96756 lightningd/chaintopology.h: Remove unused `txnums` field from `struct block`.
Changelog-None internal cleanup
2020-07-31 23:55:23 +00:00
niftynei 65c2bac2f3 hsmd/wallet: pass the bip32_key down into migrations
we're about to add a migration that requires access to the bip32_key
in order to calculate missing scriptpubkeys.

prior to this patch, we don't have access to the bip32 key in the db
migration, as it's set on the wallet but after the db migrations are
run.

here we patch it through so that every migration can access it
2020-07-29 13:13:46 +02:00