Commit Graph

10627 Commits

Author SHA1 Message Date
Rusty Russell f6a7a3f8a8 tests: test that successive closes do update parameters (particularly feerate!).
Noted by @t-bast, this is how it should work, and this tests that it does.

See also: #4777
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-15 18:23:30 +02:00
Rusty Russell f4f6e451b1 lightningd: fix leak report when close called multiple times.
```
Global errors:
 - Node /tmp/ltests-odpd7qtt/test_close_twice_1/lightning-1/ has memory leaks: [
    {
        "backtrace": [
            "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
            "ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)",
            "lightningd/peer_control.c:1673 (param_feerate_range)",
            "common/param.c:43 (make_callback)",
            "common/param.c:140 (parse_by_name)",
            "common/param.c:259 (param_arr)",
            "common/param.c:344 (param)",
            "lightningd/peer_control.c:1706 (json_close)",
            "lightningd/jsonrpc.c:643 (command_exec)",
            "lightningd/jsonrpc.c:778 (rpc_command_hook_final)",
            "lightningd/plugin_hook.c:280 (plugin_hook_call_)",
            "lightningd/jsonrpc.c:866 (plugin_hook_call_rpc_command)",
            "lightningd/jsonrpc.c:965 (parse_request)",
            "lightningd/jsonrpc.c:1056 (read_json)",
            "ccan/ccan/io/io.c:59 (next_plan)",
            "ccan/ccan/io/io.c:407 (do_plan)",
            "ccan/ccan/io/io.c:417 (io_ready)",
            "ccan/ccan/io/poll.c:445 (io_loop)",
            "lightningd/io_loop_with_timers.c:24 (io_loop_with_timers)",
            "lightningd/lightningd.c:1123 (main)"
        ],
        "label": "lightningd/peer_control.c:1673:u32[]",
        "parents": [
            "lightningd/channel.c:372:struct channel"
        ],
        "value": "0x56428b886208"
    }
]
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-15 18:23:30 +02:00
Vincenzo Palazzo 810fb7946a Remove lnprototest
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-13 15:54:09 +02:00
Rusty Russell 88adbbd20e openingd: clean up state properly when a hook says to reject opening.
This was just a minor leak, found by CI for
test_openchannel_hook_chaining.  We didn't call negotiation_aborted
which frees various fields: negotiation_failed() does that for us.

```
 MEMLEAK: 0x55b0f2d5f3c8
   label=common/channel_type.c:19:struct channel_type
   backtrace:
     ccan/ccan/tal/tal.c:442 (tal_alloc_)
     common/channel_type.c:19 (channel_type_none)
     common/channel_type.c:27 (channel_type_static_remotekey)
     common/channel_type.c:136 (channel_type_accept)
     openingd/openingd.c:844 (fundee_channel)
     openingd/openingd.c:1240 (handle_peer_in)
     openingd/openingd.c:1510 (main)
   parents:
     openingd/openingd.c:1414:struct state
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell acef45d02e common/channel_type: fix up bolt quotes now channel-types is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 027b11a5ab closingd: fix BOLT quotes to match final spec version.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 6e075d2dbb openingd: tell them channel_type if signature is bad.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 83e581b12a lightningd: print out support for channel_types in --list-features.
It's not a separate option, but lnprototest needs it to know to expect
the tlvs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell b985cd774d openingd: allow channel_type to *downgrade* from default.
e.g. you can negotiate anchor_outputs, but still ask for a
non-anchor-output channel.

If/when we make those features compulsory, downgrade will
not be allowed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 65290c02ba openingd: channel_type TLV no longer experimental.
It was merged (but this doesn't update the BOLT quotes, that's in another patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now send and support channel_type in channel open (not dual-funding though).
2021-09-13 15:53:23 +02:00
Rusty Russell c121fc9fcf openingd: EXPERIMENTAL_FEATURES provide and use channel_types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 12f298d830 openingd: tell lightningd what channel type we negotiated.
Currently we actually insist it's the default, but in future it could be
different.

We also need to tell openingd what the channel_type was, if we resume
via openingd_funder_complete().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 740afb822c common/initial_channel: use channel_type instead of individual option bools.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell 1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell d12a2ec76a openingd: don't hand redundant feature flags.
Openingd can query them itself (as dualopend already does).  And move
the two feature args next to each other on the wire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Dustin Dettmer cdb93eefec resolve() appears broken on Python 3.7+
On my machine it should have produced:
/tmp/ltests-1p0v5z1j/.locks/lock-1631090663.1066182
but instead it produced:
/private/tmp/ltests-1p0v5z1j/.locks/lock-1631090663.105848

The mismatch resulted in a hang inside flock.

Path .resolve() seems to be causing problems for others as well. It appears the library was not meant to handle complex path situations and isn't maintained as such (see link reference).

Since we have already built a full path here anyway, the call to .resolve() appears redundant.

Tested on python 3.9.6 on my Mac OS X 11.4 (20F71), Xcode 12.5.1 (12E507)

Relevant discussion: https://discuss.python.org/t/pathlib-absolute-vs-resolve/2573
2021-09-10 11:55:50 +02:00
Simon Vrouwe a308fb17b2 plugin: register `shutdown` instead of `plugin_shutdown` in notifications list 2021-09-10 07:10:05 +09:30
Rusty Russell 84957be410 close: spec is final, it's not experimental.
That was quick!

We remove the 50% test, since the default is now to use quickclose.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now perform quick-close if the peer supports it.
2021-09-09 12:04:48 +09:30
Rusty Russell 6ee8c40b29 closing: add option to set closing range.
This affects the range we offer even without quick-close, but it's
more critical for quick-close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSONRPC: `close` now takes a `feerange` parameter to set min/max fee rates for mutual close.
2021-09-09 12:04:48 +09:30
Rusty Russell e12f9f0872 pytest: add test for closing feerate greater than final commit tx.
This is now allowed for anchors (as per https://github.com/lightningnetwork/lightning-rfc/pull/847).

We need to play with feerates, since we don't put a discount on anchor
commitments yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell 1752616386 closingd: allow higher closing fee if anchor_outputs.
This follows https://github.com/lightningnetwork/lightning-rfc/pull/847.

For anchor_outputs, we pass down a max_feerate to closingd, and set the
fee ceiling to MAX.  It uses that to estimate the desired closing fee.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Anchor output mutual close allow a fee higher than the final commitment transaction (as per lightning-rfc #847)
2021-09-09 12:04:48 +09:30
Rusty Russell 79d7e83f51 --experimental-quick-close to enable quick-close negotiation
Based on a commit by @niftynei, but:
- Separated quickclose logic from main loop.
- I made it indep of anchor_outputs, use and option instead.
- Disable if they've specified how to negotiate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
niftynei 23a6c00496 Makefile: import latest spec, which includes quickclose.
It also gets rid of the requirement that close negotiation fee maximum
is the old commitment transaction.  We still do that, however, to
avoid surprising old peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell fb4edc2938 Makefile: update bolt version to include option_anchors_zero_fee_htlc_tx.
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell 1b2c6964fd Makefile: update spec version
This includes the new bolt11 test vectors, and also removes the
requirement that HTLCs be less than 2^32 msat.  We keep that for now
because Electrum enforced it on receive: in two releases we will stop
that too.

So no longer warn about needing mpp in that case either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Protocol: No longer restrict HTLCs to
2021-09-09 12:04:48 +09:30
Rusty Russell 88d55441c5 channeld: allow large HTLCs if peer offers option_support_large_channel
This check is going away anyway (only Electrum enforced it), but we
know that all wumbo peers expect large HTLCs to work today.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: Allow sending large HTLCs if peer offers option_support_large_channel (> 4294967295msat)
2021-09-09 12:04:48 +09:30
Rusty Russell d637680476 Makefile: Update BOLT version to cover bolt11 test vectors with payment secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell ad31a49455 lightningd/chaintopology: stop callbacks rearming once topology stopped.
Otherwise we get very upset when the plugins go away.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 31f439760f libplugin: make leaks log at LOG_BROKEN so they break CI.
Now we've fixed them, this makes sure CI notices if new leaks appear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 14002915a1 plugins/libplugin: mark timers as not-a-leak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 4f4c49c88a libplugin: fix leak of struct command when we don't read it all at once.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: C plugins would could leak memory on every command (esp. seen when hammering topology's listchannels).
2021-09-08 19:11:47 +02:00
Rusty Russell 127539a993 plugins/topology: use memleak annotation instead of global_gossmap hammer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell a0d531d515 pytest: annotate name_option against leak detection in test_libplugin
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 19413a821f wire/tlvstream: don't leak in tlvstream_set_tu64/tu32/short_channel_id.
Another tiny leak, which happened in keysend which uses these routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 68470c8f22 plugins/txprepare: annotate unreleased_txs list against false memleaks.
And fix utx leak in the withdraw case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 2e0179ffcc plugins/bcli: keep a list of current bitcoind requests.
This lets memleak track them, but makes sure they don't leak; using
notleak could cover up a leak here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 02553aa68a plugins/spender: don't use global tal context, use take() instead.
Otherwise it looks like a leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell f246896348 spender/openchannel.c: don't leave psbt parts owned by NULL.
They get grafted into clone, so have them parented there.  Otherwise
we get a small leak every time we RBF.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 665748236c plugins/funder: don't use global owner parent; mark policy for memleak.
And free the err string when policy turns out not to be a u64.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell af7ed03921 plugins/bcli: handle memleak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell b82e5be6e1 libplugin: let plugins add annotations to memleak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
mar
2021-09-08 19:11:47 +02:00
Rusty Russell dd65a9150a libplugin: if DEVELOPER set, do memleak detection on shutdown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell bab7778b5b libplugin: allow take() for commands, notif_subs, hook_subs, notif_topics.
Useful for plugins which dynamically generate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 952471e7a3 libplugin: allow stealing of feature sets, make keysend do that.
Otherwise the NULL parents look like a memleak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell 1f7fb33563 common/memleak: hoist strmap helper from out of lightningd/memdump.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell a5fee67b91 common/memleak: take over dump_memleak(), allow print pointer.
This will let plugins use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell b34953dd5e ccan: update to get version where tal_dump goes to stderr.
This is useful for plugins which can't send junk to stdout.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Dustin Dettmer a7d50ca3e7 Critical funder flaw
An explanation for noobs is needed here -- without it they may be confused by some of the documation.
2021-09-08 18:49:50 +02:00