Commit Graph

11545 Commits

Author SHA1 Message Date
Rusty Russell dcf2916dcb devtools: add --print-timestamps to dump-gossipstore.
Allowed me to sanity check the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
Rusty Russell 92fe871467 connectd: optimize case where peer doesn't want gossip.
LND and us send 0xFFFFFFFF to turn off gossip.  LDK and Eclair don't
seem to turn off gossip at all, but that's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
Christian Decker 50180e040c msggen: Update generated files to add Listpeers.peers[].remote_addr
Changelog-None
2022-07-15 15:27:45 +09:30
Christian Decker 1a9979784f contrib: Remove obsolete libhsmd_python class
This has been replaced with better rust bindings that can then be
consumed via pyo3, consolidating the C interface in a portable
wrapper.

Changelog-Removed: libhsmd: Removed the `libhsmd_python` wrapper as it was unused
2022-07-15 15:27:45 +09:30
Christian Decker 99367dbb32 make: Add msggen generated files to check-gen-updated target
Changelog-None
2022-07-15 15:27:45 +09:30
Igor Bubelov 5ef6729224 Use correct naming 2022-07-14 22:39:18 -05:00
Vincenzo Palazzo b624c53051 plugin: add check on the type json object during the IO message handling 2022-07-14 12:49:23 -05:00
niftynei d284b98911 notify: `channel_state_changed` now receives notice when channel opens
Previously we wouldn't notify when a channel moves into state
"CHANNELD_AWAITING_LOCKIN", as this is the original state (so there's
no movement btw states). This meant that it's impossible to track when a
channel's commitment txs have been exchanged and we're waiting for
onchain confirmation.

It's useful to have notice of this initialization though, all in one
place so that the `channel_state_changed` notification can successfully
track the entire lifecycle of a channel, from inception to close.

Note that for v2 "dual-funded" channels, we already notify at the same place, at
"DUALOPEND_AWAITING_LOCKIN" (the initial state for a dualopend channel
is "DUALOPEND_OPEN_INIT" -- this is the only state we don't get notified
at now...)

Changelog-Added: Plugins: `channel_state_changed` now triggers for a v1 channel's initial "CHANNELD_AWAITING_LOCKIN" state transition (from prior state "unknown")
2022-07-14 12:42:48 -05:00
Rusty Russell bdefbabbef lightningd: re-transmit all closing transactions on startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-14 12:40:57 -05:00
Rusty Russell d544ae075b pytest: test (failing) that we rexmit closing transactions on startup.
Usually bitcoind will have them, but it's best to re-xmit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-14 12:40:57 -05:00
adi2011 7b160b203a tests: Add tests for the RPCs
Changelog-Added: Static channel backup, to enable smooth fund recovery in case of complete data loss
2022-07-14 12:24:48 -05:00
adi2011 829fe09c13 doc: Add documentation for new RPCs and a FIXME: in fromschema.py 2022-07-14 12:24:48 -05:00
adi2011 1450f1758c Plugin: Add new plugin for SCB, Updated makefile and gitignore as well 2022-07-14 12:24:48 -05:00
adi2011 6ba8abb0de lightningd: Add RPC for populating DB with stub channels and set an error on reconnecting 2022-07-14 12:24:48 -05:00
adi2011 286d6c3165 tools/gen: Always return bool! 2022-07-14 12:24:48 -05:00
adi2011 1a1be6abd6 lightningd/peer_control: Add RPC for fetching scb for all the in-memory channels. 2022-07-14 12:24:48 -05:00
adi2011 eca844eb36 channel: Add struct scb_chan in channel and making last tx optional. 2022-07-14 12:24:48 -05:00
adi2011 e42ba8366b common: Add scb_wire for serializing the static_chan_backup 2022-07-14 12:24:48 -05:00
adi2011 64c03f8990 hsmd: Create derive_secret and makesecret RPC for deriving pseudorandom keys from HSM 2022-07-14 12:24:48 -05:00
niftynei ba7d4a8f6b make-schema: don't include tools/fromschema.py in SHASUMS
We were including the entire list of prerequisites when generating a
shastamp, which for schemas includes the `tools/fromschema.py` doc.

This meant all of our shasums were updating anytime this tool file
changed.

Instead, we just include the first prerequisite.

See: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
2022-07-14 12:24:48 -05:00
Alex Myers 27e5ddc7b7 gossipd: fix of gossip v10 channel removal handling
Node announcement indices should be reinitialized after removing the
referenced announcement.  Debugging output also corrected.

Changelog-None
2022-07-14 09:51:12 +09:30
niftynei 95ec897ac0 dual-fund: Fail if you try to buy a liquidity ad w/o dualfunding on
Fixes #5271

In-Collaboration-With: Base58 'n Coding Seminar Participants

Changelog-Changed: `fundchannel` now errors if you try to buy a liquidity ad but dont' have `experimental-dual-fund` enabled
2022-07-13 12:29:50 -05:00
Michael Schmoock f67ab2a86e test: run-gossmap update test store instructions
Changelog-None
2022-07-12 13:59:46 -05:00
Michael Schmoock b3177e945f doc: mention ip discovery only active when no addresses are announced
Changelog-None
2022-07-12 13:59:46 -05:00
Michael Schmoock 301acc9556 gossipd: only use IP discovery if no addresses are announced
This will only add the discovered `remote_addr` IPs if no other
addresses would be announced. Meaning whenever a public address was
found by autobind or an address was specified via commandline or config,
IP discovery will be disabled.

Addresses: #5305

Note from the author: We could/should also enable IP discovery when we only
have a TOR address (but without --always-use-proxy ofc). This will give
nodes an option to have a bootstrap way to be reached until IP discovery
can do the job in a more stable way.

Changelog-Changed: Only use IP discovery as fallback when no addresses would be announced
2022-07-12 13:59:46 -05:00
Michael Schmoock da85014fdf gossipd: nit: update a misleading comment
This commit got reduced to just changing a comment because
the stuff it initially did was already merged in before by
commit 7ff62b4a

So I just kept the changed comment as its more precise.

Changelog-None
2022-07-12 13:59:46 -05:00
Rusty Russell 1217f479df sendpay: allow route to contain both amount_msat and (deprecated) msatoshi.
Since it's a deprecation, we simply ignore one, rather than properly
checking they match etc.

Fixes: #5386
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Rusty Russell 1d78911b29 pytest: test that we allow both msatoshi and amount_msat is route for sendpay.
We deprecated msatoshi, but getroute() still returns both (unless
--allow-deprecated-apis=False), so we need to accept both.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Rusty Russell f98df63b75 Revert "pytest: fix test_gossip_no_backtalk flake."
This reverts commit 6cc4858847.

No longer needed now we don't flush gossip rcvd_filter as aggressively.
2022-07-12 21:41:19 +09:30
Rusty Russell 06e1e119aa pytest: fix test_gossip_no_empty_announcements flake.
This is a side-effect of fixing aging: sometimes, we age our
rcvd_filter cache too fast, and thus re-xmit.  This breaks
our test, since it used dev-disconnect on the channel_announce,
but that closes to l3, not l1!

```
>       assert l1.rpc.listchannels()['channels'] == []
E       AssertionError: assert [{'active': T...ags': 1, ...}] == []
E         Left contains 2 more items, first extra item: {'active': True, 'amount_msat': 100000000msat, 'base_fee_millisatoshi': 1, 'channel_flags': 0, ...}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5403
2022-07-12 21:41:19 +09:30
Alex Myers ddf8fbdb5d gossipd: fix crash from gossip_store v10 changes
routing.c fixed to properly remove rate-limited gossip_store entries
when channels are closed. This caused gossipd to crash on a subsequent
gossip_store_load. Also corrects an overzealous limit of one gossip_store
entry per message (should now allow one broadcastable and one
rate-limited). Addresses issues 5387, 5395.

Changelog-None
2022-07-12 14:18:06 +09:30
Rusty Russell 312751075c lightningd: save outgoing information for more forwards.
There's one case where we can present more infomation, so do that, and
fix documentation.

Changelog-Added: JSON-RPC: `listforwards` now shows `out_channel` in more cases: even if it couldn't actually send to it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5329
2022-07-12 06:38:11 +09:30
Rusty Russell 3a1a7eb93f wallet: allow saving forwarding scid even if we don't have amount.
They're not logically connected: we can know where they wanted to
go, but we didn't send it.

Where possible, it's the scid *they asked for*; otherwise, it's the
scid or fallback to the alias, but do this in the *caller*, not by
overriding inside wallet_forwarded_payment_add.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Rusty Russell d6afb0cd8d lightningd: allow outgoing_scid without outgoing amount.
This (will) happen if they ask is to forward to an unknown scid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Rusty Russell 980f3bda1f pytest: test that listforwards gives as much outgoing information as possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Christian Decker 4ae7b993de docs: Add reference to issue 5366 for stuck channel in awaiting 2022-07-11 14:26:32 -05:00
Christian Decker b8edd3478c doc: Add two entries to the FaQ about channel management 2022-07-11 14:26:32 -05:00
Christian Decker 9ff9d3d4e3 tests: Fix the canned gossmap
The gossip_store version was bumbed to 10 in PR #5239 and #5375 has
apparently not been rebased on top of that when it was merged, causing
the `run-gossmap_canned` test to fail when parsing the v9 gossip
store.

Changelog-None
2022-07-11 14:24:10 -05:00
Christian Decker 36ab0e5676 git: Ignore arm64-darwin external build directory 2022-07-11 14:06:39 -05:00
Christian Decker 74c3325208 mac: Ensure that we compile the configurator with the M1 libs 2022-07-11 14:06:39 -05:00
Christian Decker e3f53e072f make: Add macos M1 support
The M1 Macs support both x86_64 and arm64 architectures, which forced
homebrew to use a different path for its storage (`/opt/homebrew/`
instead of `/usr/local`). If we don't adjust the path we'd mix x86_64
and arm64 libraries which can lead to weird compiler and linker
errors.

This patch just introduces `CPATH` and `LIBRARY_PATH` as suggested by
the homebrew team, and detects the current architecture automatically.

Changelog-Added: macos: Added m1 architecture support for macos
2022-07-11 14:06:39 -05:00
Justin Moon 42783aaa92 cln_plugin: Configure "dynamic" field in "getmanifest" message 2022-07-11 11:05:50 +02:00
Simon Vrouwe 71cd07ea61 json: add "dynamic" field to plugin list
CHANGELOG: add "dynamic" field to plugin list
2022-07-10 21:09:41 -05:00
Simon Vrouwe 82a18813b3 doc: improve/update lightning-plugin, PLUGINS.md and lightning-listconfigs 2022-07-10 21:09:41 -05:00
Simon Vrouwe cdf12d06ba lightningd: Make sure plugins don't register the same option "name"
The extra entry in opt_table would never be called, leaving plugins
clueless why options keep defaulting.

Note that option registration outside startup does nothing.
Instead, dynamic plugins can use `plugin start [second_parameter]` to pass options.
2022-07-10 21:09:41 -05:00
Simon Vrouwe 7115611249 pytest: test plugin does not register same option "name" 2022-07-10 21:09:41 -05:00
Simon Vrouwe 2fddfe3ffc lightningd: don't increment plugin state to NEEDS_INIT when error in getmanifest
Otherwise we hangs forever in startup when it was the last plugin, we would
miss destroy_plugin --> check_plugins_manifests --> io_break

e.g. when a plugin tries to register a bool option with a string as default value.
2022-07-10 21:09:41 -05:00
Simon Vrouwe 981fd2326a lightningd: convert plugin->cmd to absolute path, fail plugin early when non-existent
Otherwise different relative paths (e.g. /dir/plugin and /dir/../dir/plugin) to same plugin
executable would not be recognized as the same plugin
2022-07-10 21:09:41 -05:00
Rusty Russell ae71a87c40 ccan: update to latest htable fixes, and update gossmap to meet new assertions.
Updating ccan to stricter htable revealed we were trying to put
(void *)1 in the htable, which is forbidden:

```
topology: ccan/ccan/htable/htable.c:382: htable_add_: Assertion `entry_is_valid((uintptr_t)p)' failed.
topology: FATAL SIGNAL 6 (version 1358d7f)
0x55f30c689c34 send_backtrace
	common/daemon.c:33
0x55f30c689ce0 crashdump
	common/daemon.c:46
0x7f5d150fe51f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f5d15152828 __pthread_kill_implementation
	./nptl/pthread_kill.c:44
0x7f5d15152828 __pthread_kill_internal
	./nptl/pthread_kill.c:80
0x7f5d15152828 __GI___pthread_kill
	./nptl/pthread_kill.c:91
0x7f5d150fe475 __GI_raise
	../sysdeps/posix/raise.c:26
0x7f5d150e47b6 __GI_abort
	./stdlib/abort.c:79
0x7f5d150e46da __assert_fail_base
	./assert/assert.c:92
0x7f5d150f5e25 __GI___assert_fail
	./assert/assert.c:101
0x55f30c6adbe4 htable_add_
	ccan/ccan/htable/htable.c:382
0x55f30c65f303 chanidx_htable_add
	common/gossmap.c:35
0x55f30c6605ed new_channel
	common/gossmap.c:337
0x55f30c6609cf add_channel
	common/gossmap.c:425
0x55f30c661101 map_catchup
	common/gossmap.c:607
0x55f30c66221e gossmap_refresh
	common/gossmap.c:927
0x55f30c66e3e9 get_gossmap
	plugins/topology.c:27
0x55f30c66f939 listpeers_done
	plugins/topology.c:369
0x55f30c671f46 handle_rpc_reply
	plugins/libplugin.c:558
0x55f30c672a19 rpc_read_response_one
	plugins/libplugin.c:726
0x55f30c672b4f rpc_conn_read_response
	plugins/libplugin.c:746
0x55f30c6ae35e next_plan
	ccan/ccan/io/io.c:59
0x55f30c6aef93 do_plan
	ccan/ccan/io/io.c:407
0x55f30c6aefd5 io_ready
	ccan/ccan/io/io.c:417
0x55f30c6b1371 io_loop
	ccan/ccan/io/poll.c:453
0x55f30c67587c plugin_main
	plugins/libplugin.c:1559
0x55f30c6708eb main
	plugins/topology.c:701
0x7f5d150e5fcf __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7f5d150e607c __libc_start_main_impl
	../csu/libc-start.c:409
0x55f30c65d894 ???
	???:0
0xffffffffffffffff ???
	???:0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-10 20:48:53 -05:00
grubles c589543f1c add generated man page 2022-07-10 20:33:41 -05:00