Commit Graph

8897 Commits

Author SHA1 Message Date
Rusty Russell 8db5fb7345 gossipd: new struct to hold scids and timestamps together.
It's not (yet?) compulsory to have the timestamps, but handing them around
together makes sense (a missing timestamp has the same effect as a zero
timestamp).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 17:37:10 -06:00
Rusty Russell 59f23bf23c gossipd: use straight counter instead of bitmap for query_channel_range replies.
The spec (since d4bafcb67dcf1e4de4d16224ea4de6b543ae73bf in March
2020) requires that reply_channel_range be in order (and all
implementations did this anyway).

But when I tried this, I found that LND doesn't (always) obey this,
since don't divide on block boundaries.  So we have to loosen the
constraints here a little.

We got rid of the old LND compat handling though, since everyone should
now be upgraded (there are CVEs out for older LNDs).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Support for receiving full gossip from ancient LND nodes.
2020-11-09 17:37:10 -06:00
Rusty Russell 4745e7e3d0 pytest: make test_hook_dep tests more stable.
I think this is what Travis is having an issue with, but it work
fine locally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Michael Schmoock f3804daabb pytest: simplify test_openchannel_hook_chaining now order is deterministic. 2020-11-09 15:22:33 -06:00
Rusty Russell fb295ffb51 plugin: sort topological candidates by specified order.
We previously registered hooks up in who-replies-to-getmanifest-first
order, but then if any had dependencies it would scatter that order.

This allows users to manually set dependencies developers have
forgotten by specifying the plugins manually in their configuration or
cmdline.  This was an excellent consideration by @mschmook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell d429e21db3 pytest: test that we maintain load order unless hook deps require a change.
Suggested-by: @mschmook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell 852e14c947 plugins: check order once all plugins have returned from getmanifest.
This means we need to stop at this stage even in the runtime-loaded
case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell a4f79e8e7e pytest: test that we don't complain about missing dependencies at startup.
We fail this, because we check dependencies as they come in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell ac93b780d5 plugins: deprecate old form of hooks.
Now both python and c libraries are updated, we can officially
deprecate the old form.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: plugins: hooks should now be specified using objects, not raw names.
2020-11-09 15:22:33 -06:00
Rusty Russell 62c52fe868 libplugin: add support for before and after deps on hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell e16ed0e207 pyln: add support for dependent hooks.
And use that to add simple tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell 6a55b4367e lightningd: actually order the hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell e2a31f42f2 plugins: allow 'before' and 'after' arrays for hooks.
The next patch will use these to order the hooks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: hooks can now specify that they must be called 'before' or 'after' other plugins.
2020-11-09 15:22:33 -06:00
Rusty Russell 7c3c0b1013 common: allow tal_arr_expand() to have an `n` argument.
It needs to use a non-clashing name for the internal var.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
ZmnSCPxj jxPCSnmZ 2604a81aba lightningd/opening_control.c: Remove predeclaration. 2020-11-07 17:18:49 +00:00
ZmnSCPxj jxPCSnmZ 0fd87b85da openingd/: Fail `fundchannel_start` if we already are, or will become, the fundee.
Fixes: #4108

Changelog-Fixed: Network: Fixed a race condition when us and a peer attempt to make channels to each other at nearly the same time.
2020-11-07 17:18:49 +00:00
ZmnSCPxj jxPCSnmZ 712e0796fc tests/test_connection.py: Test for #4108. 2020-11-07 17:18:49 +00:00
Michael Schmoock 86137852d2 doc: update channel_state_changed notification 2020-11-06 14:47:04 -06:00
Michael Schmoock 4a8722bf1d pytest: state_change history 2020-11-06 14:47:04 -06:00
Michael Schmoock 6cc96c07dc feat: adds state_changes to listpeers output
Changelog-Added: RCP: Added 'state_changes' history to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock 88c1dc56e8 db: adds a state_change history to database 2020-11-06 14:47:04 -06:00
Michael Schmoock 68ce25c92d feat: adds timestamp to state_change notification 2020-11-06 14:47:04 -06:00
Michael Schmoock 4d765003dd rpc: adds json_add_timeiso helper 2020-11-06 14:47:04 -06:00
Michael Schmoock 67f4970eb1 pytest: test opener and closer of listpeers 2020-11-06 14:47:04 -06:00
Michael Schmoock b7c18517df rpc: adds opener and closer to listpeers channels
Changelog-Added: RPC: Added 'opener' and 'closer' to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock 083a856c31 db: persist channel closer and state change cause 2020-11-06 14:47:04 -06:00
Michael Schmoock c52efe0f37 pytest: channel state change cause and message 2020-11-06 14:47:04 -06:00
Michael Schmoock 8a8dabaa58 feat: adds state change cause and message
This adds a `state_change` 'cause' to a channel.
A 'cause' is some initial 'reason' a channel was created or closed by:

  /* Anything other than the reasons below. Should not happen. */
  REASON_UNKNOWN,
  /* Unconscious internal reasons, e.g. dev fail of a channel. */
  REASON_LOCAL,
  /* The operator or a plugin opened or closed a channel by intention. */
  REASON_USER,
  /* The remote closed or funded a channel with us by intention. */
  REASON_REMOTE,
  /* E.g. We need to close a channel because of bad signatures and such. */
  REASON_PROTOCOL,
  /* A channel was closed onchain, while we were offline. */
  /* Note: This is very likely a conscious remote decision. */
  REASON_ONCHAIN

If a 'cause' is known and a subsequent state change is made with
`REASON_UNKNOWN` the preceding cause will be used as reason, since a lot
(all `REASON_UNKNOWN`) state changes are a subsequent consequences of a prior
cause: local, user, remote, protocol or onchain.

Changelog-Added: Plugins: Channel closure resaon/cause to channel_state_changed notification
2020-11-06 14:47:04 -06:00
Rusty Russell d5d9858b7b lightningd: fix similar race in `stop`.
Tested by putting a sleep in the rpc_command hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-06 14:05:44 -06:00
Rusty Russell 5e6b0f9445 lightningd: fix crash if we abort after enabling notifications.
The rpc_command hook means that we have a delay between receiving
a JSON command and actually calling the handler.  In this case, the
caller can go away:

```
==1348== Invalid write of size 1
==1348==    at 0x130EA6: json_notifications (jsonrpc.c:1350)
==1348==    by 0x12EE9E: command_exec (jsonrpc.c:636)
==1348==    by 0x12F3C6: rpc_command_hook_callback (jsonrpc.c:752)
==1348==    by 0x15AA08: plugin_hook_callback (plugin_hook.c:210)
==1348==    by 0x155C9D: plugin_response_handle (plugin.c:398)
==1348==    by 0x155E84: plugin_read_json_one (plugin.c:504)
==1348==    by 0x15603D: plugin_read_json (plugin.c:548)
==1348==    by 0x1D4AB3: next_plan (io.c:59)
==1348==    by 0x1D5630: do_plan (io.c:407)
==1348==    by 0x1D566E: io_ready (io.c:417)
==1348==    by 0x1D7834: io_loop (poll.c:445)
==1348==    by 0x12CFAC: io_loop_with_timers (io_loop_with_timers.c:24)
==1348==  Address 0x58 is not stack'd, malloc'd or (recently) free'd
==1348==
lightningd: FATAL SIGNAL 11 (version v0.9.1-266-ga4df315)
0x180f7e send_backtrace
	common/daemon.c:38
0x181024 crashdump
	common/daemon.c:51
0x5bd7fcf ???
	???:0
0x130ea6 json_notifications
	lightningd/jsonrpc.c:1350
0x12ee9e command_exec
	lightningd/jsonrpc.c:636
0x12f3c6 rpc_command_hook_callback
	lightningd/jsonrpc.c:752
0x15aa08 plugin_hook_callback
	lightningd/plugin_hook.c:210
0x155c9d plugin_response_handle
	lightningd/plugin.c:398
0x155e84 plugin_read_json_one
	lightningd/plugin.c:504
0x15603d plugin_read_json
	lightningd/plugin.c:548
0x1d4ab3 next_plan
	ccan/ccan/io/io.c:59
0x1d5630 do_plan
	ccan/ccan/io/io.c:407
0x1d566e io_ready
	ccan/ccan/io/io.c:417
0x1d7834 io_loop
	ccan/ccan/io/poll.c:445
0x12cfac io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x132825 main
	lightningd/lightningd.c:1016
0x5bbab96 ???
	???:0
0x1159e9 ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20201106001723
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-06 14:05:44 -06:00
fiatjaf 15b2546d45 doc: move "next_onion" and "shared_secret" to inside "onion". 2020-11-05 14:15:18 -06:00
Antoine Poinsot 8846c35ac6 dual_open: correct silent enum conversions
We were silently converting a side enum (3 variants) to a tx_role enum
(2 variants).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-04 14:29:22 -06:00
niftynei e99335e5b3 df: move tx_role functions (used in next patch) 2020-11-04 14:29:22 -06:00
Rusty Russell cce3e717d9 bitcoin/signature: wrap libwally call.
If a tx is larger than 2k, libwally will do an alloc:

```
lightning_hsmd: common/setup.c:11: wally_tal: Assertion `wally_tal_ctx' failed.
0x11c283 wally_tal
	common/setup.c:11
0x15ebd1 wally_malloc
	../../../libwally-core/src/internal.c:233
0x171e9e tx_to_bip143_bytes
	../../../libwally-core/src/transaction.c:1918
0x172cda tx_to_bytes
	../../../libwally-core/src/transaction.c:2086
0x1759df tx_get_signature_hash
	../../../libwally-core/src/transaction.c:2776
0x175afd wally_tx_get_signature_hash
	../../../libwally-core/src/transaction.c:2800
0x175b62 wally_tx_get_btc_signature_hash
	../../../libwally-core/src/transaction.c:2810
0x1297d9 bitcoin_tx_hash_for_sig
	bitcoin/signature.c:139
0x1298ca sign_tx_input
	bitcoin/signature.c:161
0x10e701 handle_sign_remote_commitment_tx
	hsmd/hsmd.c:1011
0x110f7f handle_client
	hsmd/hsmd.c:1968
0x147a71 next_plan
	ccan/ccan/io/io.c:59
0x1485ee do_plan
	ccan/ccan/io/io.c:407
0x14862c io_ready
	ccan/ccan/io/io.c:417
0x14a7f2 io_loop
	ccan/ccan/io/poll.c:445
0x111125 main
	hsmd/hsmd.c:2040
```

I reduced that constant in libwally to 200, and ran the entire
test suite, and found no other places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-04 14:28:06 -06:00
Michael Schmoock bde2806290 chore: fixes a cppcheck false positive
Just applied the same suppression as rusty in:
6635fe12e4 (Rusty Russell    2020-05-15 15:57:29 +0930 146)
/* cppcheck-suppress uninitvar - false positive on f1->bits */

My cppcheck was complaining about the same issue in the following functions.
I wonder why travis does not care though.

Changelog-None
2020-11-03 10:18:08 -06:00
YOSHIDA Masanori 9f893e3f6d wire: remove declaration of towire_tlvs since its implementation is already removed
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-11-02 10:59:05 -06:00
Christian Decker d151d554b3 travis: Add a pytest run with EXPERIMENTAL_FEATURES=1
Signed-off-by: Lisa <@niftynei>
2020-10-29 13:42:32 -05:00
Christian Decker 62f7e646ec pyln: Bump cryptography dependency to 3.2 due to upstream bug
https://cryptography.io/en/latest/changelog.html

Changelog-None
2020-10-29 11:11:48 +10:30
Christian Decker d487ec5c9b wallet: Remove orphan statements.po file 2020-10-29 11:11:48 +10:30
Christian Decker 9a0327cd25 pyln-proto: Add compactsize alias for varint_{encode,decode}
We were mistakenly calling it varint, while Bitcoin refers to it as
CompactSize.
2020-10-29 11:11:48 +10:30
Jan Sarenik 895722fd8a nit: lightningd.c: fix "Richard Stevens' advice"
See https://github.com/rustyrussell/ccan/pull/88

Changelog-None
2020-10-28 14:32:35 -05:00
Jan Sarenik 90ad541cd2 doc: STYLE.md fix "parameters get"
Changelog-None
2020-10-28 14:32:20 -05:00
Rusty Russell ae4139ba10 lightningd: note whether payment secret is used or not.
This will allow nodes (with log-level=debug) to gather how many payments
are made without payment_secrets.  We need to know this so we know when
we can make them compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 12:03:14 +01:00
Rusty Russell cd5a93d0bd gossmap: fix reutrn of gossmap_xxx_has_feature, rename.
1. One place returned false instead of -1.
2. The names implied it returned a bool, and it doesn't.

Fix both, and curse C's loose typing a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 15:32:12 +10:30
niftynei 1e173463c9 df: rename 'feerate_per_kw' to 'feerate_per_kw_commitment'
Differentiate btw feerate for commitment txs and feerate for funding
transaction.
2020-10-27 19:52:05 -05:00
niftynei 4ea9d9e928 df: Pass new feerate options through to plugin, set reasonable bounds
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.

We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.

In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
2020-10-27 19:52:05 -05:00
niftynei 97fd18f0b5 df: incorporate a few spec changes -- serial_id is now 64-bits
And we pass 3-params for feerate so it's a 'pick a range' conversation.
2020-10-27 19:52:05 -05:00
Jan Sarenik d46427234b nit: lightningd.c: Fix a couple of SATTS comments
SATTS stands for Shared Adventure Through The Code

 - fix "must declared"
 - fix "an zero-length"
 - fix redundant space before dot
2020-10-27 10:55:11 -05:00
niftynei 21122af3a8 dev-force-features: adds a second valid format for forcing features
The previous dev-force-features forced you to explicitly declare every
desired feature bit in an array, for each set.

Here, we allow you to also denote adding/subtracing a feature bit
by just passing in the number of the bit to flip and the direction to
turn it. e.g.

	'dev-force-features': '+223'

Will turn on opt_dual_fund/odd.

	'dev-force-features': '-16'

Will flag off opt_basic_mpp.
2020-10-26 21:31:24 -05:00
niftynei ddc9500a64 features: add 'feature_bit_sub', which will subtract/unset bits
Given a two sets of featurebits, unset the featurebits in the first set
that are set in the second set
2020-10-26 21:31:24 -05:00