Commit Graph

9807 Commits

Author SHA1 Message Date
Rusty Russell d1f8a5f0be gossmap: add gossmap_chan_get_capacity() helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell eb370ab8ff gossmap: remembers when a channel is private.
Steals one bit from the scid_off, and turns that into a relative
offset.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell cc198748d4 common/json_tok: hoist param_short_channel_id from inside lightningd/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 7ef6dd04d9 plugins/test/run-route-overlong: modern overlong routing test.
gossmap equivalent of gossipd/test/run-overlong.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 52e5ca3c11 plugins/libplugin: make headers update-mocks friendly.
In particular, it expects return type at start of line.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 6f71aae336 plugins/libplugin-pay: extreact route function so we can write unit tests.
New "route()" doesn't need a struct payment, so is easier to test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 088cfced18 common/test/run-route and run-route-specific: modern gossmap tests
Equivalent to gossipd/test/run-find_route.c and gossipd/test/run-find_route-specific.c
except they use gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 0682ad9c48 fetchinvoice: don't produce invalid JSON on sent_inv timeout.
An unrelated bug in the reverse path triggered this code, and boom.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 2e3e4a50ec update-mocks: prefer to find function defs from local includes.
This matters for:
	- common/wallet.h vs wallet/wallet.h
	- common/gossip_store.h vs gossipd/gossip_store.h
	- common/json.h vs lightningd/json.h
	- common/ping.h vs lightningd/ping.h

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell d868d4470a Makefile: don't let update-mocks run unless DEVELOPER and EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell cc6f1fd120 update-mocks: don't try to regenerate files.
If you do update-mocks in a dirty tree, the recursive make that it
uses will try to rebuild things!  Suppress that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell 33736b860a lightningd: attach HTLC timeout to htlc itself, fix gratuitous disconnect bug.
We set the timeout on first HTLC, but didn't clear it if that HTLC failed.

It's saner to have a per-HTLC timeout (since that's what it is!) and
also our timer infra is specially coded to scale approximately infinitely so
trying to optimize this is vastly premature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We would sometimes gratuitously disconnect 30 seconds after an HTLC failed.
2021-05-21 14:45:05 +09:30
Rusty Russell 11180e7aa1 pytest: add test for HTLC timeout on failed htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-21 14:45:05 +09:30
fiatjaf 0d68febd8d reorganize .gitignore entries across subdirs. 2021-05-18 09:43:50 +09:30
fiatjaf e17526f64c gitignore more things. 2021-05-18 09:43:50 +09:30
Rusty Russell e141b71675 pytest: fix flake in test_reconnect_no_update
If l2 didn't get FUNDING_LOCKED from l1 before it disconnected, it
won't be in state CHANNELD_NORMAL: it will be in DUALOPEND_AWAITING_LOCKIN.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-17 18:58:10 -05:00
Christian Decker 0f2009c7a3 make: Do not scan contrib for discouraged functions 2021-05-15 09:31:12 +09:30
Christian Decker 6c67afeb03 make: Add CPPCHECK_OPTS to allow excluding files from the check
With swig we now have C files that are generated with tools that are
not under our control, so provide an escape hatch for them.
2021-05-15 09:31:12 +09:30
Christian Decker a4c19ba6ba libhsmd: Clean up the build descriptors for libhsmd_python 2021-05-15 09:31:12 +09:30
Christian Decker 323b1a558b libhsmd: Add missing implementation of status functions 2021-05-15 09:31:12 +09:30
Christian Decker 7f15f86718 libhsmd: First working build script for libhsmd-python
Changelog-Added: libhsmd: Added python bindings for `libhsmd`
2021-05-15 09:31:12 +09:30
Rusty Russell 214fdcc9d7 plugin notifications: minor cleanups.
1. We don't need to check for NULL before tal_count(NULL).
2. Use of json_for_each_arr iterator is probably better.
3. Weird indent fixed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-14 10:24:05 +09:30
niftynei ce1e5bd388 df-test: make flakey test not as flakey
This would flake fairly regularly, what we really care about is
asserting that the l2 node is in CHANNELD_NORMAL state, while the l1
node hasn't progressed that far yet.
2021-05-12 11:25:41 +09:30
niftynei c75ca8c112 df-bugs: if we've already got the funding_locked, dont renotify
Peer sends funding locked, we tell lightningd who saves it to disk.
Then we restart/reconnect and they retransmit funding_locked. We were
re-notifying lightningd about their lock-in, which was crashing/breaking
things. Instead, we ignore duplicate lock-in messages from the peer.

lightningd-1: 2021-05-11T18:00:12.844Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Internal error DUALOPEND_AWAITING_LOCKIN: channel_got_funding_locked twice
2021-05-12 11:25:41 +09:30
niftynei e375932108 df-tests: test_coin_movement_notices, make work?
No idea how this slipped past the first time
2021-05-12 11:25:41 +09:30
niftynei 71a4a2e31c df: rework closing logic
Trying to put all the disconnect logic into the same path was a dumb
idea. If you asked to reconnect but passed in an 'unsaved' channel, we
would not call the 'reconnect' code.

Instead, we make a differentiation between "unsaved" channels
(ones that we haven't received commitment tx for) and handle the
disconnect for these separate from where we want to do a reconnect.
2021-05-12 11:25:41 +09:30
niftynei efdc36c8e9 tests: mark test as requiring developer
hangs with EXP_DF when developer=0
2021-05-12 11:25:41 +09:30
niftynei 5cac36724e df-test: v2 has reconnects 2021-05-12 11:25:41 +09:30
niftynei ef333d5cee df-callbacks: dont log as broken, just reconnect 2021-05-12 11:25:41 +09:30
niftynei 4432672300 df-bug: avoid referencing null channel->owner
If dualopend dies, we shouldn't reference it
2021-05-12 11:25:41 +09:30
niftynei 5ee4c9e46c df: patch for valgrind error
We were freeing the payload, which is then subsequently freed by the
plugin_hook caller. Whoops.

Now we pass through to the callback function and just clean up neatly.

------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.406602
==406602== Invalid read of size 8
==406602==    at 0x12AC93: openchannel2_hook_cb (dual_open_control.c:669)
==406602==    by 0x12AF0A: openchannel2_hook_deserialize (dual_open_control.c:721)
==406602==    by 0x16EF0E: plugin_hook_callback (plugin_hook.c:186)
==406602==    by 0x169746: plugin_response_handle (plugin.c:514)
==406602==    by 0x169959: plugin_read_json_one (plugin.c:620)
==406602==    by 0x169B23: plugin_read_json (plugin.c:665)
==406602==    by 0x1F4076: next_plan (io.c:59)
==406602==    by 0x1F4C5B: do_plan (io.c:407)
==406602==    by 0x1F4C9D: io_ready (io.c:417)
==406602==    by 0x1F6F35: io_loop (poll.c:445)
==406602==    by 0x13D48D: io_loop_with_timers (io_loop_with_timers.c:24)
==406602==    by 0x143388: main (lightningd.c:1111)
==406602==  Address 0x75e7418 is 56 bytes inside a block of size 3,520 free'd
==406602==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602==    by 0x204FB0: del_tree (tal.c:421)
==406602==    by 0x20527E: tal_free (tal.c:486)
==406602==    by 0x122D68: delete_channel (channel.c:124)
==406602==    by 0x129291: channel_disconnect (dual_open_control.c:63)
==406602==    by 0x129364: channel_close_conn (dual_open_control.c:82)
==406602==    by 0x131CF6: peer_please_disconnect (connect_control.c:304)
==406602==    by 0x131DEB: connectd_msg (connect_control.c:326)
==406602==    by 0x172023: sd_msg_read (subd.c:509)
==406602==    by 0x1F4076: next_plan (io.c:59)
==406602==    by 0x1F4C5B: do_plan (io.c:407)
==406602==    by 0x1F4C9D: io_ready (io.c:417)
==406602==  Block was alloc'd at
==406602==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602==    by 0x204A39: allocate (tal.c:250)
==406602==    by 0x204FFA: tal_alloc_ (tal.c:428)
==406602==    by 0x123165: new_unsaved_channel (channel.c:209)
==406602==    by 0x130D34: peer_start_dualopend (dual_open_control.c:2985)
==406602==    by 0x15BD2A: peer_connected_hook_final (peer_control.c:1105)
==406602==    by 0x16F2E5: plugin_hook_call_ (plugin_hook.c:275)
==406602==    by 0x15BF5C: plugin_hook_call_peer_connected (peer_control.c:1155)
==406602==    by 0x15C16C: peer_connected (peer_control.c:1208)
==406602==    by 0x131E3B: connectd_msg (connect_control.c:332)
==406602==    by 0x172023: sd_msg_read (subd.c:509)
==406602==    by 0x171842: read_fds (subd.c:310)
2021-05-12 11:25:41 +09:30
niftynei 6d3fb11bc6 df-tests: patch for state == AWAITING_UNILATERAL problem
Found on CI where DEVELOPER=0 EXPERIMENTAL_DUAL_FUND=1,
as we turn off automatic reconnects when DEVELOPER=1

This test has been modified to make the error happen every time, and
then fixed.

lightningd-2: 2021-05-07T20:12:03.790Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer has reconnected, state AWAITING_UNILATERAL
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: FATAL SIGNAL 6 (version e8b3f78)
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x56384ee072e9
lightningd-2: 2021-05-07T20:12:03.813Z **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x56384ee0733b

----------------------------- Captured stderr call -----------------------------
lightningd: lightningd/peer_control.c:1100: peer_connected_hook_final: Assertion `channel->state == DUALOPEND_OPEN_INIT || channel->state == DUALOPEND_AWAITING_LOCKIN' failed.
lightningd: FATAL SIGNAL 6 (version e8b3f78)
0x56384ee072a1 send_backtrace
	common/daemon.c:39
0x56384ee0733b crashdump
	common/daemon.c:52
0x7f88486a020f ???
	???:0
0x7f88486a018b ???
	???:0
0x7f884867f858 ???
	???:0
0x7f884867f728 ???
	???:0
0x7f8848690f35 ???
	???:0
0x56384eddc94e peer_connected_hook_final
	lightningd/peer_control.c:1100
0x56384edea2ed plugin_hook_call_
	lightningd/plugin_hook.c:275
0x56384eddfeb8 plugin_hook_call_peer_connected
	lightningd/peer_control.c:1156
0x56384eddfeb8 peer_connected
	lightningd/peer_control.c:1209
0x56384edc30cd connectd_msg
	lightningd/connect_control.c:332
0x56384edebe6f sd_msg_read
	lightningd/subd.c:509
0x56384edebfb1 read_fds
	lightningd/subd.c:310
0x56384ee483b0 next_plan
	ccan/ccan/io/io.c:59
0x56384ee4885b do_plan
	ccan/ccan/io/io.c:407
0x56384ee488f8 io_ready
	ccan/ccan/io/io.c:417
0x56384ee4a23c io_loop
	ccan/ccan/io/poll.c:445
0x56384edcabda io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x56384edce826 main
	lightningd/lightningd.c:1111
0x7f88486810b2 ???
	???:0
0x56384edb52ad ???
	???:0
0xffffffffffffffff ???
	???:0
2021-05-12 11:25:41 +09:30
niftynei 342f783f53 df-tests: Turn on v2 tests on CI
Let's run more tests with the v2 open protocol
2021-05-12 11:25:41 +09:30
niftynei 558abe288a tests: explicitly mark tests to run both as v1+v2
By default, tests only run as v1 unless marked as v2.

These tests we want to run as both v1+v2

Includes fixes to have tests pass
2021-05-12 11:25:41 +09:30
niftynei dc758f616b tests: mark as v1 or v2 openchannel required 2021-05-12 11:25:41 +09:30
niftynei 3a2d602922 tests: add marker for v1/v2 channel opens
Tests that will only run when !EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v1')
	def test_...()

Tests that will only run when EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v2')
	def test_...()
2021-05-12 11:25:41 +09:30
niftynei d0bbf07655 tests: not DEVELOPER -> mark.developer
Nicer syntaxtic sugar for marking pytests as 'developer required'
2021-05-12 11:25:41 +09:30
niftynei b4e24ac8ba df: anchor outputs are on if EXP_DF
So we should treat it the same as EXPERIMENTAL_FEATURES
2021-05-12 11:25:41 +09:30
niftynei 484d6bde83 tests: move EXP_DF into the testing utils 2021-05-12 11:25:41 +09:30
niftynei 7a488cc11e df-tests: remove df_accepter plugin
The `funder` plugin, does the same stuff as this temporary test plugin,
so we move over to using that instead.
2021-05-12 11:25:41 +09:30
niftynei de854fa9d4 psbt: really hacky patch over libwally's 0-input txs parse fail
Issue being tracked at https://github.com/ElementsProject/libwally-core/pull/273
2021-05-12 11:25:41 +09:30
niftynei ef9d8bcd5a dual-fund: reconnections were borked, this fixes them 2021-05-12 11:25:41 +09:30
niftynei 09b5c906fd funder, startupregtest: fixup default config to be 100
Fails on start otherwise
2021-05-12 11:25:41 +09:30
niftynei 6dc954bb91 df-bugs: rm duplicate call to channeld
This gets called from channel_set_owner, which both `delete_channel` and
the `channel_fail_reconnect` pathways call.

Fixes crash
------------------------------------------------------ Captured stderr teardown -------------------------------------------------------
lightning_connectd: peer_disconnected unknown peer: 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 (version v0.10.0-144-gfaf98c9)
0x560e90d59d08 send_backtrace
        common/daemon.c:39
0x560e90d648a5 status_failed
        common/status.c:214
0x560e90d50e8f peer_disconnected
        connectd/connectd.c:1606
0x560e90d510d5 recv_req
        connectd/connectd.c:1662
0x560e90d5a266 handle_read
        common/daemon_conn.c:31
0x560e90d98ccb next_plan
        ccan/ccan/io/io.c:59
0x560e90d998b0 do_plan
        ccan/ccan/io/io.c:407
0x560e90d998f2 io_ready
        ccan/ccan/io/io.c:417
0x560e90d9bb8a io_loop
        ccan/ccan/io/poll.c:445
0x560e90d512c8 main
        connectd/connectd.c:1735
0x7fbdb828b0b2 ???
        ???:0
0x560e90d4a6dd ???
        ???:0
0xffffffffffffffff ???
        ???:0
2021-05-11 15:37:24 +09:30
Rusty Russell e960b1015d !fixup
Log this so I can try to find dual fund connnectd race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell d8e68893f5 bcli: become less aggressive with onchain fee levels.
Users are more upset recently with the cost of unilateral closes
than they are the risk of being cheated.  While we complete our
anchor implementation so we can use low fees there, let's
get less aggressive (we already have 34 or 18 blocks to close
in the worst case).

The changes are:

- Commit transactions were "2 CONSERVATIVE" now "6 ECONOMICAL".
- HTLC resolution txs were "3 CONSERVATIVE" now "6 ECONOMICAL".
- Penalty txs were "3 CONSERVATIVE" now "12 ECONOMICAL".
- Normal txs were "4 ECONOMICAL" now "12 ECONOMICAL".

There can be no perfect levels, but we have had understandable
complaints recently about how high our default fee levels are.

Changelog-Changed: Protocol: channel feerates reduced to bitcoind's "6 block ECONOMICAL" rate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell 9a22e7b3a1 plugins/bcli: make feerate calls more changeable.
This make it clearer what we're doing, IMHO, so we can easily alter
the levels if we want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell 9825f32874 lightningd: implement --log-timestamps=false.
Fixes: #4494
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: config: New option `log-timestamps` allow disabling of timestamp prefix in logs.
2021-05-05 17:19:19 -05:00
Christian Decker 96acafcef3 libhsmd: Prefix check_client_capabilities
I wante to hide it inside the library, but it is good to have a single
place to verify that the client was permitted to send a message we are
handling, so make it officially part of the interface by prefixing it.
2021-05-04 11:18:52 +09:30