Commit Graph

11593 Commits

Author SHA1 Message Date
Christian Decker 50180e040c msggen: Update generated files to add Listpeers.peers[].remote_addr
Changelog-None
2022-07-15 15:27:45 +09:30
Christian Decker 1a9979784f contrib: Remove obsolete libhsmd_python class
This has been replaced with better rust bindings that can then be
consumed via pyo3, consolidating the C interface in a portable
wrapper.

Changelog-Removed: libhsmd: Removed the `libhsmd_python` wrapper as it was unused
2022-07-15 15:27:45 +09:30
Christian Decker 99367dbb32 make: Add msggen generated files to check-gen-updated target
Changelog-None
2022-07-15 15:27:45 +09:30
Igor Bubelov 5ef6729224 Use correct naming 2022-07-14 22:39:18 -05:00
Vincenzo Palazzo b624c53051 plugin: add check on the type json object during the IO message handling 2022-07-14 12:49:23 -05:00
niftynei d284b98911 notify: `channel_state_changed` now receives notice when channel opens
Previously we wouldn't notify when a channel moves into state
"CHANNELD_AWAITING_LOCKIN", as this is the original state (so there's
no movement btw states). This meant that it's impossible to track when a
channel's commitment txs have been exchanged and we're waiting for
onchain confirmation.

It's useful to have notice of this initialization though, all in one
place so that the `channel_state_changed` notification can successfully
track the entire lifecycle of a channel, from inception to close.

Note that for v2 "dual-funded" channels, we already notify at the same place, at
"DUALOPEND_AWAITING_LOCKIN" (the initial state for a dualopend channel
is "DUALOPEND_OPEN_INIT" -- this is the only state we don't get notified
at now...)

Changelog-Added: Plugins: `channel_state_changed` now triggers for a v1 channel's initial "CHANNELD_AWAITING_LOCKIN" state transition (from prior state "unknown")
2022-07-14 12:42:48 -05:00
Rusty Russell bdefbabbef lightningd: re-transmit all closing transactions on startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-14 12:40:57 -05:00
Rusty Russell d544ae075b pytest: test (failing) that we rexmit closing transactions on startup.
Usually bitcoind will have them, but it's best to re-xmit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-14 12:40:57 -05:00
adi2011 7b160b203a tests: Add tests for the RPCs
Changelog-Added: Static channel backup, to enable smooth fund recovery in case of complete data loss
2022-07-14 12:24:48 -05:00
adi2011 829fe09c13 doc: Add documentation for new RPCs and a FIXME: in fromschema.py 2022-07-14 12:24:48 -05:00
adi2011 1450f1758c Plugin: Add new plugin for SCB, Updated makefile and gitignore as well 2022-07-14 12:24:48 -05:00
adi2011 6ba8abb0de lightningd: Add RPC for populating DB with stub channels and set an error on reconnecting 2022-07-14 12:24:48 -05:00
adi2011 286d6c3165 tools/gen: Always return bool! 2022-07-14 12:24:48 -05:00
adi2011 1a1be6abd6 lightningd/peer_control: Add RPC for fetching scb for all the in-memory channels. 2022-07-14 12:24:48 -05:00
adi2011 eca844eb36 channel: Add struct scb_chan in channel and making last tx optional. 2022-07-14 12:24:48 -05:00
adi2011 e42ba8366b common: Add scb_wire for serializing the static_chan_backup 2022-07-14 12:24:48 -05:00
adi2011 64c03f8990 hsmd: Create derive_secret and makesecret RPC for deriving pseudorandom keys from HSM 2022-07-14 12:24:48 -05:00
niftynei ba7d4a8f6b make-schema: don't include tools/fromschema.py in SHASUMS
We were including the entire list of prerequisites when generating a
shastamp, which for schemas includes the `tools/fromschema.py` doc.

This meant all of our shasums were updating anytime this tool file
changed.

Instead, we just include the first prerequisite.

See: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
2022-07-14 12:24:48 -05:00
Alex Myers 27e5ddc7b7 gossipd: fix of gossip v10 channel removal handling
Node announcement indices should be reinitialized after removing the
referenced announcement.  Debugging output also corrected.

Changelog-None
2022-07-14 09:51:12 +09:30
niftynei 95ec897ac0 dual-fund: Fail if you try to buy a liquidity ad w/o dualfunding on
Fixes #5271

In-Collaboration-With: Base58 'n Coding Seminar Participants

Changelog-Changed: `fundchannel` now errors if you try to buy a liquidity ad but dont' have `experimental-dual-fund` enabled
2022-07-13 12:29:50 -05:00
Michael Schmoock f67ab2a86e test: run-gossmap update test store instructions
Changelog-None
2022-07-12 13:59:46 -05:00
Michael Schmoock b3177e945f doc: mention ip discovery only active when no addresses are announced
Changelog-None
2022-07-12 13:59:46 -05:00
Michael Schmoock 301acc9556 gossipd: only use IP discovery if no addresses are announced
This will only add the discovered `remote_addr` IPs if no other
addresses would be announced. Meaning whenever a public address was
found by autobind or an address was specified via commandline or config,
IP discovery will be disabled.

Addresses: #5305

Note from the author: We could/should also enable IP discovery when we only
have a TOR address (but without --always-use-proxy ofc). This will give
nodes an option to have a bootstrap way to be reached until IP discovery
can do the job in a more stable way.

Changelog-Changed: Only use IP discovery as fallback when no addresses would be announced
2022-07-12 13:59:46 -05:00
Michael Schmoock da85014fdf gossipd: nit: update a misleading comment
This commit got reduced to just changing a comment because
the stuff it initially did was already merged in before by
commit 7ff62b4a

So I just kept the changed comment as its more precise.

Changelog-None
2022-07-12 13:59:46 -05:00
Rusty Russell 1217f479df sendpay: allow route to contain both amount_msat and (deprecated) msatoshi.
Since it's a deprecation, we simply ignore one, rather than properly
checking they match etc.

Fixes: #5386
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Rusty Russell 1d78911b29 pytest: test that we allow both msatoshi and amount_msat is route for sendpay.
We deprecated msatoshi, but getroute() still returns both (unless
--allow-deprecated-apis=False), so we need to accept both.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Rusty Russell f98df63b75 Revert "pytest: fix test_gossip_no_backtalk flake."
This reverts commit 6cc4858847.

No longer needed now we don't flush gossip rcvd_filter as aggressively.
2022-07-12 21:41:19 +09:30
Rusty Russell 06e1e119aa pytest: fix test_gossip_no_empty_announcements flake.
This is a side-effect of fixing aging: sometimes, we age our
rcvd_filter cache too fast, and thus re-xmit.  This breaks
our test, since it used dev-disconnect on the channel_announce,
but that closes to l3, not l1!

```
>       assert l1.rpc.listchannels()['channels'] == []
E       AssertionError: assert [{'active': T...ags': 1, ...}] == []
E         Left contains 2 more items, first extra item: {'active': True, 'amount_msat': 100000000msat, 'base_fee_millisatoshi': 1, 'channel_flags': 0, ...}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5403
2022-07-12 21:41:19 +09:30
Alex Myers ddf8fbdb5d gossipd: fix crash from gossip_store v10 changes
routing.c fixed to properly remove rate-limited gossip_store entries
when channels are closed. This caused gossipd to crash on a subsequent
gossip_store_load. Also corrects an overzealous limit of one gossip_store
entry per message (should now allow one broadcastable and one
rate-limited). Addresses issues 5387, 5395.

Changelog-None
2022-07-12 14:18:06 +09:30
Rusty Russell 312751075c lightningd: save outgoing information for more forwards.
There's one case where we can present more infomation, so do that, and
fix documentation.

Changelog-Added: JSON-RPC: `listforwards` now shows `out_channel` in more cases: even if it couldn't actually send to it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5329
2022-07-12 06:38:11 +09:30
Rusty Russell 3a1a7eb93f wallet: allow saving forwarding scid even if we don't have amount.
They're not logically connected: we can know where they wanted to
go, but we didn't send it.

Where possible, it's the scid *they asked for*; otherwise, it's the
scid or fallback to the alias, but do this in the *caller*, not by
overriding inside wallet_forwarded_payment_add.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Rusty Russell d6afb0cd8d lightningd: allow outgoing_scid without outgoing amount.
This (will) happen if they ask is to forward to an unknown scid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Rusty Russell 980f3bda1f pytest: test that listforwards gives as much outgoing information as possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Christian Decker 4ae7b993de docs: Add reference to issue 5366 for stuck channel in awaiting 2022-07-11 14:26:32 -05:00
Christian Decker b8edd3478c doc: Add two entries to the FaQ about channel management 2022-07-11 14:26:32 -05:00
Christian Decker 9ff9d3d4e3 tests: Fix the canned gossmap
The gossip_store version was bumbed to 10 in PR #5239 and #5375 has
apparently not been rebased on top of that when it was merged, causing
the `run-gossmap_canned` test to fail when parsing the v9 gossip
store.

Changelog-None
2022-07-11 14:24:10 -05:00
Christian Decker 36ab0e5676 git: Ignore arm64-darwin external build directory 2022-07-11 14:06:39 -05:00
Christian Decker 74c3325208 mac: Ensure that we compile the configurator with the M1 libs 2022-07-11 14:06:39 -05:00
Christian Decker e3f53e072f make: Add macos M1 support
The M1 Macs support both x86_64 and arm64 architectures, which forced
homebrew to use a different path for its storage (`/opt/homebrew/`
instead of `/usr/local`). If we don't adjust the path we'd mix x86_64
and arm64 libraries which can lead to weird compiler and linker
errors.

This patch just introduces `CPATH` and `LIBRARY_PATH` as suggested by
the homebrew team, and detects the current architecture automatically.

Changelog-Added: macos: Added m1 architecture support for macos
2022-07-11 14:06:39 -05:00
Justin Moon 42783aaa92 cln_plugin: Configure "dynamic" field in "getmanifest" message 2022-07-11 11:05:50 +02:00
Simon Vrouwe 71cd07ea61 json: add "dynamic" field to plugin list
CHANGELOG: add "dynamic" field to plugin list
2022-07-10 21:09:41 -05:00
Simon Vrouwe 82a18813b3 doc: improve/update lightning-plugin, PLUGINS.md and lightning-listconfigs 2022-07-10 21:09:41 -05:00
Simon Vrouwe cdf12d06ba lightningd: Make sure plugins don't register the same option "name"
The extra entry in opt_table would never be called, leaving plugins
clueless why options keep defaulting.

Note that option registration outside startup does nothing.
Instead, dynamic plugins can use `plugin start [second_parameter]` to pass options.
2022-07-10 21:09:41 -05:00
Simon Vrouwe 7115611249 pytest: test plugin does not register same option "name" 2022-07-10 21:09:41 -05:00
Simon Vrouwe 2fddfe3ffc lightningd: don't increment plugin state to NEEDS_INIT when error in getmanifest
Otherwise we hangs forever in startup when it was the last plugin, we would
miss destroy_plugin --> check_plugins_manifests --> io_break

e.g. when a plugin tries to register a bool option with a string as default value.
2022-07-10 21:09:41 -05:00
Simon Vrouwe 981fd2326a lightningd: convert plugin->cmd to absolute path, fail plugin early when non-existent
Otherwise different relative paths (e.g. /dir/plugin and /dir/../dir/plugin) to same plugin
executable would not be recognized as the same plugin
2022-07-10 21:09:41 -05:00
Rusty Russell ae71a87c40 ccan: update to latest htable fixes, and update gossmap to meet new assertions.
Updating ccan to stricter htable revealed we were trying to put
(void *)1 in the htable, which is forbidden:

```
topology: ccan/ccan/htable/htable.c:382: htable_add_: Assertion `entry_is_valid((uintptr_t)p)' failed.
topology: FATAL SIGNAL 6 (version 1358d7f)
0x55f30c689c34 send_backtrace
	common/daemon.c:33
0x55f30c689ce0 crashdump
	common/daemon.c:46
0x7f5d150fe51f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f5d15152828 __pthread_kill_implementation
	./nptl/pthread_kill.c:44
0x7f5d15152828 __pthread_kill_internal
	./nptl/pthread_kill.c:80
0x7f5d15152828 __GI___pthread_kill
	./nptl/pthread_kill.c:91
0x7f5d150fe475 __GI_raise
	../sysdeps/posix/raise.c:26
0x7f5d150e47b6 __GI_abort
	./stdlib/abort.c:79
0x7f5d150e46da __assert_fail_base
	./assert/assert.c:92
0x7f5d150f5e25 __GI___assert_fail
	./assert/assert.c:101
0x55f30c6adbe4 htable_add_
	ccan/ccan/htable/htable.c:382
0x55f30c65f303 chanidx_htable_add
	common/gossmap.c:35
0x55f30c6605ed new_channel
	common/gossmap.c:337
0x55f30c6609cf add_channel
	common/gossmap.c:425
0x55f30c661101 map_catchup
	common/gossmap.c:607
0x55f30c66221e gossmap_refresh
	common/gossmap.c:927
0x55f30c66e3e9 get_gossmap
	plugins/topology.c:27
0x55f30c66f939 listpeers_done
	plugins/topology.c:369
0x55f30c671f46 handle_rpc_reply
	plugins/libplugin.c:558
0x55f30c672a19 rpc_read_response_one
	plugins/libplugin.c:726
0x55f30c672b4f rpc_conn_read_response
	plugins/libplugin.c:746
0x55f30c6ae35e next_plan
	ccan/ccan/io/io.c:59
0x55f30c6aef93 do_plan
	ccan/ccan/io/io.c:407
0x55f30c6aefd5 io_ready
	ccan/ccan/io/io.c:417
0x55f30c6b1371 io_loop
	ccan/ccan/io/poll.c:453
0x55f30c67587c plugin_main
	plugins/libplugin.c:1559
0x55f30c6708eb main
	plugins/topology.c:701
0x7f5d150e5fcf __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7f5d150e607c __libc_start_main_impl
	../csu/libc-start.c:409
0x55f30c65d894 ???
	???:0
0xffffffffffffffff ???
	???:0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-10 20:48:53 -05:00
grubles c589543f1c add generated man page 2022-07-10 20:33:41 -05:00
grubles d50c62cf41 Update lightning-newaddr.7.md
Added a sentence to explain which command to use to send an on-chain payment _from_ the CLN wallet.
2022-07-10 20:33:41 -05:00
Rusty Russell 6753675c31 lightningd: fire watches on blocks before telling lightningd.
We noticed bogus behavior where (with 200 blocks added at once)
lightningd didn't see a DF channel open: this is because we told
lightningd about the new blocks (and it timed out channel) before
the watches for the transaction are fired.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-10 20:31:30 -05:00