Commit Graph

8869 Commits

Author SHA1 Message Date
Rusty Russell d5d9858b7b lightningd: fix similar race in `stop`.
Tested by putting a sleep in the rpc_command hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-06 14:05:44 -06:00
Rusty Russell 5e6b0f9445 lightningd: fix crash if we abort after enabling notifications.
The rpc_command hook means that we have a delay between receiving
a JSON command and actually calling the handler.  In this case, the
caller can go away:

```
==1348== Invalid write of size 1
==1348==    at 0x130EA6: json_notifications (jsonrpc.c:1350)
==1348==    by 0x12EE9E: command_exec (jsonrpc.c:636)
==1348==    by 0x12F3C6: rpc_command_hook_callback (jsonrpc.c:752)
==1348==    by 0x15AA08: plugin_hook_callback (plugin_hook.c:210)
==1348==    by 0x155C9D: plugin_response_handle (plugin.c:398)
==1348==    by 0x155E84: plugin_read_json_one (plugin.c:504)
==1348==    by 0x15603D: plugin_read_json (plugin.c:548)
==1348==    by 0x1D4AB3: next_plan (io.c:59)
==1348==    by 0x1D5630: do_plan (io.c:407)
==1348==    by 0x1D566E: io_ready (io.c:417)
==1348==    by 0x1D7834: io_loop (poll.c:445)
==1348==    by 0x12CFAC: io_loop_with_timers (io_loop_with_timers.c:24)
==1348==  Address 0x58 is not stack'd, malloc'd or (recently) free'd
==1348==
lightningd: FATAL SIGNAL 11 (version v0.9.1-266-ga4df315)
0x180f7e send_backtrace
	common/daemon.c:38
0x181024 crashdump
	common/daemon.c:51
0x5bd7fcf ???
	???:0
0x130ea6 json_notifications
	lightningd/jsonrpc.c:1350
0x12ee9e command_exec
	lightningd/jsonrpc.c:636
0x12f3c6 rpc_command_hook_callback
	lightningd/jsonrpc.c:752
0x15aa08 plugin_hook_callback
	lightningd/plugin_hook.c:210
0x155c9d plugin_response_handle
	lightningd/plugin.c:398
0x155e84 plugin_read_json_one
	lightningd/plugin.c:504
0x15603d plugin_read_json
	lightningd/plugin.c:548
0x1d4ab3 next_plan
	ccan/ccan/io/io.c:59
0x1d5630 do_plan
	ccan/ccan/io/io.c:407
0x1d566e io_ready
	ccan/ccan/io/io.c:417
0x1d7834 io_loop
	ccan/ccan/io/poll.c:445
0x12cfac io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x132825 main
	lightningd/lightningd.c:1016
0x5bbab96 ???
	???:0
0x1159e9 ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20201106001723
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-06 14:05:44 -06:00
fiatjaf 15b2546d45 doc: move "next_onion" and "shared_secret" to inside "onion". 2020-11-05 14:15:18 -06:00
Antoine Poinsot 8846c35ac6 dual_open: correct silent enum conversions
We were silently converting a side enum (3 variants) to a tx_role enum
(2 variants).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-04 14:29:22 -06:00
niftynei e99335e5b3 df: move tx_role functions (used in next patch) 2020-11-04 14:29:22 -06:00
Rusty Russell cce3e717d9 bitcoin/signature: wrap libwally call.
If a tx is larger than 2k, libwally will do an alloc:

```
lightning_hsmd: common/setup.c:11: wally_tal: Assertion `wally_tal_ctx' failed.
0x11c283 wally_tal
	common/setup.c:11
0x15ebd1 wally_malloc
	../../../libwally-core/src/internal.c:233
0x171e9e tx_to_bip143_bytes
	../../../libwally-core/src/transaction.c:1918
0x172cda tx_to_bytes
	../../../libwally-core/src/transaction.c:2086
0x1759df tx_get_signature_hash
	../../../libwally-core/src/transaction.c:2776
0x175afd wally_tx_get_signature_hash
	../../../libwally-core/src/transaction.c:2800
0x175b62 wally_tx_get_btc_signature_hash
	../../../libwally-core/src/transaction.c:2810
0x1297d9 bitcoin_tx_hash_for_sig
	bitcoin/signature.c:139
0x1298ca sign_tx_input
	bitcoin/signature.c:161
0x10e701 handle_sign_remote_commitment_tx
	hsmd/hsmd.c:1011
0x110f7f handle_client
	hsmd/hsmd.c:1968
0x147a71 next_plan
	ccan/ccan/io/io.c:59
0x1485ee do_plan
	ccan/ccan/io/io.c:407
0x14862c io_ready
	ccan/ccan/io/io.c:417
0x14a7f2 io_loop
	ccan/ccan/io/poll.c:445
0x111125 main
	hsmd/hsmd.c:2040
```

I reduced that constant in libwally to 200, and ran the entire
test suite, and found no other places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-04 14:28:06 -06:00
Michael Schmoock bde2806290 chore: fixes a cppcheck false positive
Just applied the same suppression as rusty in:
6635fe12e4 (Rusty Russell    2020-05-15 15:57:29 +0930 146)
/* cppcheck-suppress uninitvar - false positive on f1->bits */

My cppcheck was complaining about the same issue in the following functions.
I wonder why travis does not care though.

Changelog-None
2020-11-03 10:18:08 -06:00
YOSHIDA Masanori 9f893e3f6d wire: remove declaration of towire_tlvs since its implementation is already removed
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-11-02 10:59:05 -06:00
Christian Decker d151d554b3 travis: Add a pytest run with EXPERIMENTAL_FEATURES=1
Signed-off-by: Lisa <@niftynei>
2020-10-29 13:42:32 -05:00
Christian Decker 62f7e646ec pyln: Bump cryptography dependency to 3.2 due to upstream bug
https://cryptography.io/en/latest/changelog.html

Changelog-None
2020-10-29 11:11:48 +10:30
Christian Decker d487ec5c9b wallet: Remove orphan statements.po file 2020-10-29 11:11:48 +10:30
Christian Decker 9a0327cd25 pyln-proto: Add compactsize alias for varint_{encode,decode}
We were mistakenly calling it varint, while Bitcoin refers to it as
CompactSize.
2020-10-29 11:11:48 +10:30
Jan Sarenik 895722fd8a nit: lightningd.c: fix "Richard Stevens' advice"
See https://github.com/rustyrussell/ccan/pull/88

Changelog-None
2020-10-28 14:32:35 -05:00
Jan Sarenik 90ad541cd2 doc: STYLE.md fix "parameters get"
Changelog-None
2020-10-28 14:32:20 -05:00
Rusty Russell ae4139ba10 lightningd: note whether payment secret is used or not.
This will allow nodes (with log-level=debug) to gather how many payments
are made without payment_secrets.  We need to know this so we know when
we can make them compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 12:03:14 +01:00
Rusty Russell cd5a93d0bd gossmap: fix reutrn of gossmap_xxx_has_feature, rename.
1. One place returned false instead of -1.
2. The names implied it returned a bool, and it doesn't.

Fix both, and curse C's loose typing a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 15:32:12 +10:30
niftynei 1e173463c9 df: rename 'feerate_per_kw' to 'feerate_per_kw_commitment'
Differentiate btw feerate for commitment txs and feerate for funding
transaction.
2020-10-27 19:52:05 -05:00
niftynei 4ea9d9e928 df: Pass new feerate options through to plugin, set reasonable bounds
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.

We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.

In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
2020-10-27 19:52:05 -05:00
niftynei 97fd18f0b5 df: incorporate a few spec changes -- serial_id is now 64-bits
And we pass 3-params for feerate so it's a 'pick a range' conversation.
2020-10-27 19:52:05 -05:00
Jan Sarenik d46427234b nit: lightningd.c: Fix a couple of SATTS comments
SATTS stands for Shared Adventure Through The Code

 - fix "must declared"
 - fix "an zero-length"
 - fix redundant space before dot
2020-10-27 10:55:11 -05:00
niftynei 21122af3a8 dev-force-features: adds a second valid format for forcing features
The previous dev-force-features forced you to explicitly declare every
desired feature bit in an array, for each set.

Here, we allow you to also denote adding/subtracing a feature bit
by just passing in the number of the bit to flip and the direction to
turn it. e.g.

	'dev-force-features': '+223'

Will turn on opt_dual_fund/odd.

	'dev-force-features': '-16'

Will flag off opt_basic_mpp.
2020-10-26 21:31:24 -05:00
niftynei ddc9500a64 features: add 'feature_bit_sub', which will subtract/unset bits
Given a two sets of featurebits, unset the featurebits in the first set
that are set in the second set
2020-10-26 21:31:24 -05:00
niftynei 0871bf0999 features: have clear_feature_bit correctly resize bitfield
There's a spec rule about only ever sending a correctly sized
feature-bits, so as a precaution we have `clear_feature_bit` correctly
resize when a bit is cleared.
2020-10-26 21:31:24 -05:00
niftynei c7839dff76 features: reorg, going to use elsewhere soon 2020-10-26 21:31:24 -05:00
Rusty Russell 497e71ca13 Makefile: fix bashism in NO_PYTHON
== is a bash extension; in shell it's a single =:

```
/bin/sh: 1: [: unexpected operator
sql-rewrite wallet/db_postgres_sqlgen.c
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-26 21:28:36 -05:00
YOSHIDA Masanori b7ab8866b8 Makefile: fix typo and wrong path in external/Makefile to properly remove objects with "make clean"
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-10-27 10:42:48 +10:30
YOSHIDA Masanori 54729376d7 Makefile: add status_levels.c to $(COMMON_SRC_NOGEN)
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-10-27 10:42:48 +10:30
YOSHIDA Masanori 4f12ff0967 Makefile: add $(LIGHTNINGD_SRC_NOHDR) to $(ALL_C_SOURCES) for "make clean" to work properly
"make clean" removes all object files listed in $(ALL_OBJS), which is derived from $(ALL_C_SOURCES).
Source files in $(LIGHTNINGD_SRC_NOHDR) are missing from $(ALL_C_SOURCES) therefore some object files are not removed by "make clean".
This commit fixes this point.

Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-10-27 10:42:48 +10:30
Michael Schmoock 6907e85a39 chore: optimize postgres binary lookup
This will change the current hardwired PostgreSQL binary lookup mechanism to
utilizing the `pg_config --bindir` utility that should work better for most
distributions. The old method caused issues at least on Archlinux.

Changelog-None
2020-10-27 10:40:52 +10:30
niftynei 7a2e72f56c dualopend: handle dev-memleaks; fix placement of return on DEV options 2020-10-26 13:04:42 +10:30
niftynei 396c406983 dualopend: re-order 'handle_dev_memleak' occurrence 2020-10-26 13:04:42 +10:30
niftynei fe56f572ed df: finalize redeemscript at the same time as witness stack
libwally has a quirk where the finalize method will fail to 'completely'
finalize an input's parts if either the final_scriptsig or
final_redeemscript fields are set

since we manually set the final_witness stack here, we also need to
fully finalize the redeemscript -> final_scriptsig here as well.
2020-10-26 13:04:42 +10:30
niftynei 0ca6c3cc84 script: break out redeemscript->scriptsig function
We're going to finalze some redeemscripts here shortly, so break out the
ability to render an arbitrary redeemscript -> scriptsig
2020-10-26 13:04:42 +10:30
Michael Schmoock 6697ebcebb doc: update HACKING.md 2020-10-23 12:47:47 -05:00
Michael Schmoock 5b542195d3 doc: typo in lightning-setchannelfee.7
Changelog-None
2020-10-23 12:47:47 -05:00
Rusty Russell 1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell f395404a10 lightningd: infrastructure for internal notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 44f6b8a746 pytest: add notifications to tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 41290a436f lightning-cli: print notifications (with '# ' prefix) if received.
This can be suppressed with -N.

Note that we wull get an error with older lightningd, but we ignore it
anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: print notifications and progress bars if commands provide them.
2020-10-23 13:53:16 +10:30
Rusty Russell 806f208295 pyln: handle (ignore) notifications, and add notify_msg to send them.
We also sanity check that response id matches our request.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: pyln.client handles and can send progress notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell 572c849873 libplugin: ignore incoming notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 127811757f libplugin: support for sending notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: routines to send notification updates and progress.
2020-10-23 13:53:16 +10:30
Rusty Russell 9f687d60d9 lightningd: forward notifications from plugins if enabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell c732d8707a JSON-RPC: notifications command.
This lets callers enable notifications; we won't send any if they don't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `notifications` command to enable notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell a95205c25c common/json_stream: add generic double-cr helper.
And make caller of json_stream_forward_change_id use it, since
we're going to reuse that.

Also call json_out_finished here, so next object doesn't have a ","
prepended.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell d970addd4b ccan: update to latest version, get json_out_finished update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell becd4fe576 common: add routines for log level names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell 0dec593aa8 lightningd: clean up close code now force is always true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Sergi Delgado Segura 9de5f438c4 pyln: Adds type annotations to zbase32 functions 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura 3779c0b6ee pyln: Moves old zbase32 test from primitives 2020-10-22 13:56:57 +02:00