Commit Graph

247 Commits

Author SHA1 Message Date
Rusty Russell 37ddf2e829 lightningd: rename sendonionmessage to sendobsonionmessage.
sendonionmessage is going to be the new one, and do much *less*.

As this is an internal experimental-only API, no deprecation cycle
required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell 83e581b12a lightningd: print out support for channel_types in --list-features.
It's not a separate option, but lnprototest needs it to know to expect
the tlvs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 2063049559 libplugin: plugin_exit helper which flushes stdout.
We weren't actually getting the last log out; this does that.

We have to fix test_bitcoin_failure which now notices the BROKEN
log message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: libplugin: Fatal error messages from plugin_exit() now logged in lightningd.
2021-09-05 15:16:56 +02:00
Rusty Russell fe86c117d9 datastore: turn keys into arrays
After some discussion with @shesek, and my own usage, we agreed that
a more comprehensive interface, which explicitly supports grouping,
is desirable.

Thus keys are now arrays, with the semantic that a key is either a
parent or has a value, never both.

For convenience in the JSON schema, we always return them as arrays,
though we accept simple strings as arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 533571a655 datastore: add generation, simple atomicity.
We add a generation counter, and allow update or del conditional
on a given generation.

Formalizes error codes, too, since we have more now.

Suggested-by: @shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 432508e65e datastore: allow replace/append.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell e711f6c589 datastore: allow strings.
It's common, and the simplest case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell dfe2693bbd datastore: docs, schemas, and a test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell 66af5f8a28 chaintopology: tell gossipd that channels no longer exist on reorg.
This actually caused the flake in test_funding_reorg_private, where
l1 and l2 might not mark the original channel disabled.  In fact, they
should *remove* it as it gets reorged out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:38 +09:30
niftynei 35bec51a97 printlogs for failing tests 2021-07-20 13:28:38 -04:00
niftynei 43ae30df21 chaintopology: new command 'parsefeerate'
Useful for parsing a passed in feerate before calling lightningd with
it, e.g. when you need to know what the feerate is for a fundpsbt before
calling fundpsbt

Changelog-Added: JSON-RPC: new command `parsefeerate` which takes a feerate string and returns the calculated perkw/perkb
2021-07-20 13:28:38 -04:00
niftynei 8654c817da sendcustommsg: promote to non-dev
Enable non-dev builds to send custom messages.

Preserves 'dev-' for compat-enabled builds.

Changelog-Changed: JSON-RPC: moved dev-sendcustommsg to sendcustommsg
2021-07-14 14:39:44 -05:00
Rusty Russell 9eb531868f lightningd: make invoices insist on payment_secret.
In fact, we make it compulsory, which means if you don't understand it
you'll hang up on us!

Add some logging for that in future.

Changelog-Changed: Protocol: All new invoices require a payment_secret (i.e. modern TLV format onion)
Changelog-Changed: Protocol: We can no longer connect to peers which don't support `payment_secret`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-14 14:38:00 -05:00
Rusty Russell 4e881e56ce pytest: always provide payment_secret when making payments.
They're about to become compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-14 14:38:00 -05:00
Rusty Russell adab9eb301 lightningd: add force-feerates option.
Useful for regtest and testnet.  Sure, you shouldn't use this on mainnet,
but I haven't restricted it because our users are usually pretty clever.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #1806
Changelog-Added: config: `force_feerates` option to allow overriding feerate estimates (mainly for regtest).
2021-07-09 07:26:09 +09:30
Rusty Russell 2910bb0235 pytest: fix flake in test_misc.py::test_funding_reorg_* xxremote_lags
If l2 doesn't have the funding locked in, the rest of the test fails
(we got a timeout on `wait_for(lambda: [c['active'] for c in l2.rpc.listchannels('103x1x0')['channels']] == [False, False])`)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-15 06:36:36 +09:30
Rusty Russell ec83d7a8a5 doc/schemas: disableoffer, disconnect, feerates, fetchinvoice, fundchannel, fundchannel_cancel, fundchannel_complete, fundchannel_start, fundpsbt, getinfo, getlog, getroute.
We also add a test for getlog, since it was never called by the
testsuite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-27 20:28:49 +09:30
Rusty Russell b6223eb117 lightningd: option_shutdown_anysegwit is no longer experimental.
https://github.com/lightningnetwork/lightning-rfc/pull/672 was merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: `option_shutdown_anysegwit` allows future segwit versions on shutdown transactions.
2021-05-26 20:01:03 +09:30
Rusty Russell f24dc9173d wallet: add "reserved_to_block" field to listfunds.
We already have this field in reserveinputs and unreserveinputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listfunds` has a new `reserved_to_block` field.
2021-05-26 15:08:01 +09:30
Rusty Russell 33736b860a lightningd: attach HTLC timeout to htlc itself, fix gratuitous disconnect bug.
We set the timeout on first HTLC, but didn't clear it if that HTLC failed.

It's saner to have a per-HTLC timeout (since that's what it is!) and
also our timer infra is specially coded to scale approximately infinitely so
trying to optimize this is vastly premature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We would sometimes gratuitously disconnect 30 seconds after an HTLC failed.
2021-05-21 14:45:05 +09:30
niftynei 558abe288a tests: explicitly mark tests to run both as v1+v2
By default, tests only run as v1 unless marked as v2.

These tests we want to run as both v1+v2

Includes fixes to have tests pass
2021-05-12 11:25:41 +09:30
niftynei d0bbf07655 tests: not DEVELOPER -> mark.developer
Nicer syntaxtic sugar for marking pytests as 'developer required'
2021-05-12 11:25:41 +09:30
Rusty Russell d8e68893f5 bcli: become less aggressive with onchain fee levels.
Users are more upset recently with the cost of unilateral closes
than they are the risk of being cheated.  While we complete our
anchor implementation so we can use low fees there, let's
get less aggressive (we already have 34 or 18 blocks to close
in the worst case).

The changes are:

- Commit transactions were "2 CONSERVATIVE" now "6 ECONOMICAL".
- HTLC resolution txs were "3 CONSERVATIVE" now "6 ECONOMICAL".
- Penalty txs were "3 CONSERVATIVE" now "12 ECONOMICAL".
- Normal txs were "4 ECONOMICAL" now "12 ECONOMICAL".

There can be no perfect levels, but we have had understandable
complaints recently about how high our default fee levels are.

Changelog-Changed: Protocol: channel feerates reduced to bitcoind's "6 block ECONOMICAL" rate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell 9825f32874 lightningd: implement --log-timestamps=false.
Fixes: #4494
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: config: New option `log-timestamps` allow disabling of timestamp prefix in logs.
2021-05-05 17:19:19 -05:00
Rusty Russell b352df4179 pytest: make it work with latest bitcoind master branch.
They seem to have changed the JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-24 13:57:16 +09:30
Rusty Russell 32d650f9df lightningd: don't abort on incorrect versions, but try to re-exec.
You still shouldn't do this (you could get some transient failures),
but at least you have a decent chance if you reinstall over a running
daemon, instead of getting confusing internal errors if message
formats have changed.

Changelog-Added: lightningd: we now try to restart if subdaemons are upgraded underneath us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #4346
2021-04-24 13:56:58 +09:30
Rusty Russell 9c3cf5aff9 newaddr: don't include "address" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `newaddr` no longer includes `address` field (deprecated in 0.7.1)
2021-04-07 14:34:39 +09:30
niftynei e549746578 features: match up feature names to feature bit
Aligns the feature bit number to the name-label. Is this correct?
2021-03-25 20:05:11 +10:30
Rusty Russell 6c9d9ee9a2 connect: return address we actually connected to.
Otherwise, we might find an address other than the one given and
the user might think that address worked.

Fixes: #4185
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `connect` returns `address` it actually connected to
2021-03-17 08:38:08 +10:30
Christian Decker 21355edc43 plugin: Do not send the internal framed message over the wire
Looks like #4394 treated a symptom but not the root cause. We were
actually sending the message framed with the WIRE_CUSTOMMSG_OUT and
the length prefix over the encrypted connection to the peer. It just
happened to be a valid custommsg...

This fixes the issue, and this time I made sure we actually send the
raw message over the wire. However for backward compatibility we
needed to imitate the faulty behavior which is 90% of this patch :-)

Changelog-Fixed: plugin: `dev-sendcustommsg` included the type and length prefix when sending a message.
2021-03-09 14:39:22 +10:30
Karol Hosiawa 7b72ea7061 Added listforwards test 2021-03-03 09:19:20 +10:30
Rusty Russell d0946b75bc common: support opt_shutdown_anysegwit checks (EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-02 14:42:34 +10:30
Christian Decker ebb1b19c65 plugin: Fix the custommsg hook not to include the internal prefix
We were always prefixing the `message` field with the internal type
prefix 0x0407, followed by the length prefix. Neither is needed since
the type being constant is of no interest to the plugin and the length
being implicit due to the JSON-encoding.

Reported-by: Ilya Evdokimov
Changelog-Fixed: plugin: The `custommsg` hook no longer includes the internal type prefix and length prefix in its `payload`
Changelog-Deprecated: plugin: The `message` field on the `custommsg` hook is deprecated in favor of the `payload` field, which skips the internal prefix.
2021-03-02 14:41:16 +10:30
Rusty Russell 4848c0022f plugins/bcli: fake minimum fee if we're in regtest mode.
Saves a great deal of confusion for regtest users.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: If bitcoind won't give a fee estimate in regtest, use minimum.
2021-03-02 13:34:55 +10:30
Rusty Russell f3159ec4ac pytest: detect warnings, too.
Since we turned many errors into warnings, we want our tests to fail
when they happen unexpectedly.  We make WARNING clear in the strings
we print, too, to help out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:52 +10:30
Rusty Russell f4ee41a989 common: remove peer_failed in favor of peer_failed_warn/peer_failed_err
And make all the callers choose which one.  In general, I prefer warn,
which lets them reconnect and try again, however some places are either
stated that they must be errors in the spec itself, or in openingd
where we abandon the channel when we close the connection anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we now send warning messages and close the connection, except on unrecoverable errors.
2021-02-04 12:02:52 +10:30
Michael Schmoock 67f2939540 pytest: custommsg chainable tests 2021-01-29 13:37:42 +10:30
Christian Decker 52e82b76b6 pytest: Stabilize test_bad_onion 2021-01-29 10:29:09 +10:30
Rusty Russell fc3e679c97 lightningd: control onion messages by `experimental-onion-messages` option.
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
2021-01-13 14:45:36 +01:00
niftynei 2c9ce2566c df-tests: test_lightningd_still_loading 2021-01-10 13:44:04 +01:00
niftynei 7eb6213fca df-tests: test_reconnect_sender_add1, others
We're still ignoring multifundchannel failure tests, but now we use the
'dual-fund' flag instead of a blanket true.
2021-01-10 13:44:04 +01:00
niftynei 09da171381 df-tests: test_io_logging
Move pid collection down to when dualopend is definitely dead
2021-01-10 13:44:04 +01:00
niftynei 0d45823b82 df-tests: test_funding_reorg_private 2021-01-10 13:44:04 +01:00
Rusty Russell 23af9d4972 onion_message: support variable-length onion messages.
Updated to the BOLT, and a few tweaks, and we can send giant onion_messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Karol Hosiawa e582c0ac0a Added listfunds test 2020-12-22 13:00:02 +01:00
ZmnSCPxj jxPCSnmZ a437936c78 lightningd/log.c: Fix up handling of SIGHUP.
Fixes: #4240

ChangeLog-Fixed: log: Do not terminate on the second received SIGHUP.
2020-12-02 16:48:48 +10:30
ZmnSCPxj jxPCSnmZ 67d8fdcc75 tests/test_misc.py: Add a test for issue #4240. 2020-12-02 16:48:48 +10:30
niftynei 26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
niftynei de34f08b82 tests,fundchannel: return the result from the fundchannel rpc call
We need this so we can verify the 'close_to' result
2020-10-16 13:51:57 +10:30
Rusty Russell 6195d953cc plugins: use "slow" feerate for mutual close negotiation.
We're rarely in a hurry here, and bitcoind is aggressive with fees.
You can always spend this output if you really have to, using CPFP.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: mutual closing feerate reduced to "slow" to avoid overpaying.
2020-10-13 20:53:34 +02:00