Commit Graph

10899 Commits

Author SHA1 Message Date
niftynei 03c950bae8 db: decouple `fatal` reliance, have as impl defined function
`fatal` is defined in lightningd and has logfile dependencies etc.

Make it more generic by allowing declaration in the use file (wallet.c)
2022-03-05 15:03:34 +10:30
niftynei b0829fc52a lightningd/Make: cleanup lightningd+wallet headers
There is no "wallet_lib_headers" variable in wallet/Makefile

Likewise, there were two "lightningd_headers", a couple of unused
variables and some other nonsene in lightningd/Makefile
2022-03-05 15:03:34 +10:30
Rusty Russell cd37753849 plugins/bcli: neaten to fix spurious leak report.
```
plugin-bcli: MEMLEAK: 0x1d2a118
plugin-bcli:   label=plugins/bcli.c:636:const char *[]
plugin-bcli:   backtrace:
plugin-bcli:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
plugin-bcli:     ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)
plugin-bcli:     plugins/bcli.c:636 (estimatefees_next)
plugin-bcli:     plugins/bcli.c:684 (estimatefees)
plugin-bcli:     plugins/libplugin.c:1307 (ld_command_handle)
plugin-bcli:     plugins/libplugin.c:1348 (ld_read_json_one)
plugin-bcli:     plugins/libplugin.c:1368 (ld_read_json)
plugin-bcli:     ccan/ccan/io/io.c:59 (next_plan)
plugin-bcli:     ccan/ccan/io/io.c:407 (do_plan)
plugin-bcli:     ccan/ccan/io/io.c:417 (io_ready)
plugin-bcli:     ccan/ccan/io/poll.c:453 (io_loop)
plugin-bcli:     plugins/libplugin.c:1565 (plugin_main)
plugin-bcli:     plugins/bcli.c:965 (main)
plugin-bcli:   parents:
plugin-bcli:     plugins/libplugin.c:1235:struct command
```

It's neater to convert start_bitcoin_cli() to take varargs, and handle take:
the above "leak" is because we don't keep a pointer to the params[] array
which we use to pass to start_bitcoin_cli().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-03 10:23:11 +00:00
Dustin Dettmer 4386ccd842 bitcoin: Comment typo fix 2022-03-03 11:22:19 +10:30
niftynei 7bfbe1492a test_close: sync l1 to the blockchain, so that the events are there
Flake on CI coming from `penalty_in/outhtlc` due to the fact that all
the events haven't arrived for the check.

If we wait to sync `l1` as well as `l2`, this should resolve the flake.

        # We use a subset of tags in expected_2 that are used in expected_1
>       tags = check_utxos_channel(l1, [channel_id], expected_1)

tests/test_closing.py:726:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/utils.py:321: in check_utxos_channel
    txid = matchup_events(u_set, evs, chans, tag_list)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

u_set = [[{'account_id': 'external', 'blockheight': 104, 'coin_type': 'bcrt', 'credit': '100000000msat', ...}, None]]
evs = [('external', ['penalty'], None, None), ('external', ['penalty'], None, None), ('external', ['penalty'], None, None)]
chans = ['2722a5fe49a8b5fa4004c19828f3e903632ac02712c6fe78ebea418daad2691f']
tag_list = {'0': '892c64c7d8c8f15d7bdcdcde34b615817d273d2e33d9d775cc9ff38e8e3deeb2', 'A': '1e69d2aa8d41eaeb78fec61227c02a6303e9f32898c10440fab5a849fea52227', 'B': '0e11e2ca01bf5f30b4c54522af172af055dcff8e3810f80a5069a2394cad74b5'}

    def matchup_events(u_set, evs, chans, tag_list):
>       assert len(u_set) == len(evs) and len(u_set) > 0
E       AssertionError
2022-03-01 16:19:34 -06:00
Christian Decker 6332578070 rust: Use $CARGO_OPTS when building the example plugin 2022-02-28 23:25:37 +00:00
Christian Decker 75f0b8e916 cln-grpc: Add `listpeers` RPC method
This is pretty much the hardest to map, but we map it correctly, with
the exception of the state_changes[] array we truncated out in the
last commit.
2022-02-28 23:25:37 +00:00
Christian Decker b73405a4c2 cln-grpc: Allow fields to be mapped to None to ignore them
`listpeers` is a rather deeply nested structure which has a couple of
caveats, namely that we use the same enum multiple times, which causes
naming clashes. So we truncate the state_changes[]. We can later map
them if needed, but it'll get much easier once we have an abstract
model description that isn't JSON schema, which unrolls all types,
causing us to generate those enums multiple times.
2022-02-28 23:25:37 +00:00
Christian Decker fd2d126ec4 cln-grpc: Add conversion from pb Amount to json Amount 2022-02-28 23:25:37 +00:00
Christian Decker d95c9d2990 cln-grpc: Add conversion from pb Amount to json Amount 2022-02-28 23:25:37 +00:00
Christian Decker bb4946a6e6 msggen: Support renaming methods in GRPC
There is at least one clash with a built-in for the grpc server trait,
namely `connect` so we add support for renaming a method when
generating the scaffolding
2022-02-28 23:25:37 +00:00
Christian Decker efed7d8617 msggen: Support enums in requests too
They are sent as i32 over protobuf, so we need to convert them into
their enum representation.
2022-02-28 23:25:37 +00:00
Christian Decker 494243d41c msggen: Handle some more types in request conversions 2022-02-28 23:25:37 +00:00
Christian Decker 24e44ecbb6 cln-grpc: Add glue to get all pieces to work together 2022-02-28 23:25:37 +00:00
Christian Decker 62dc078271 cln-grpc: Generate server dispatcher
The server doesn't do much more than unwrapping the request from its
grpc envelope, convert it into the matching JSON-RPC binding struct,
initiate the RPC connection (until we have connection pooling), and
then forwards the converted request. The inverse then happens for the
result.
2022-02-28 23:25:37 +00:00
Christian Decker 5d6e9d6dae cln-grpc: Add generation of request conversion
This is taking protobuf requests on one side and converting them into
the JSON-RPC requests.
2022-02-28 23:25:37 +00:00
Christian Decker 8d3871d791 cln-grpc: Add result conversion generator to `msggen`
This takes the Rust bindings and converts them into the generated
protobuf bindings:

> JSON-RPC -> Rust bindings -> grpc bindings -> protobuf
2022-02-28 23:25:37 +00:00
Christian Decker 4c105d2424 cln-grpc: Generate grpc bindings from proto
This is the easy way, since there's already tooling for this.
2022-02-28 23:25:37 +00:00
Christian Decker d01b2c21a7 cln-grpc: Add generation of grpc protobuf file from schema 2022-02-28 23:25:37 +00:00
Christian Decker edf6832f8f rust: Actually check if we have cargo
Seems a temporary edit to check the detection slipped in during the
previous PR. This fixes that :-)
2022-02-28 23:25:37 +00:00
Simon Vrouwe 175e0b54b3 testing: fix flake in test_ping_timeout
[ Folded 2 fixups --RR ]
2022-03-01 09:38:40 +10:30
Simon Vrouwe c286eb053f pyln-client: Plugin, improve background comment 2022-03-01 09:38:40 +10:30
Simon Vrouwe ad22535d34 pyln-testing: improve description of wait_for_logs 2022-03-01 09:38:40 +10:30
Simon Vrouwe 84bead9396 pyln-testing: in LightningNode.openchannel, make wait_for_announce more reliable
it now waits for 'alias' in node_announcement, not just block confirms.
more cleanup
2022-03-01 09:38:40 +10:30
Simon Vrouwe f84e1a0536 pyln-testing: in utils, update method fundbalancedchannel, now it also works with dualfund
and some cleanup, fundchannel has been returning tx, txid and outnum for years
2022-03-01 09:38:40 +10:30
Simon Vrouwe 2d06c38997 plugin-funder: fix typos in option lease-fee-base-msat and funder-fuzz-percent
Changelog-Experimental: option `--lease-fee-base-msat` renamed to `--lease-fee-base-sat`

Changelog-Experimental: option `--lease-fee-base-msat` deprecated and will be removed next release
2022-03-01 09:38:40 +10:30
Simon Vrouwe 0173717b6a doc: add details about: log-level, experimental options and `features` field returned by listnodes,
listchannels and connect.
2022-03-01 09:38:40 +10:30
Simon Vrouwe 9bbb52097e lightningd: fix parsing of option dev-force-features
It did not pick-up the last (bolt11) bits set.
2022-03-01 09:38:40 +10:30
Simon Vrouwe 905a85dc99 JSON-RPC: getinfo, add field `our_features`
All build flags and (experimental) options make it hard to find
out what features are supported or enabled.

And the undocumented `--list-features-only`, does not account for all
our featurebits, for example bit 55 (keysend).

Changelog-Added: JSON-RPC: `getinfo` result now includes `our_features` (bits) for various Bolt #9 contexts
2022-03-01 09:38:40 +10:30
Simon Vrouwe 32fb8a42fd testing: cleanup in test_plugin_shutdown 2022-03-01 09:38:40 +10:30
Simon Vrouwe 760c271381 devtools: in decodemsg, add newline and fix decoding a node_announcement with tlvs
Terminal prompt got messed up because missing newline in case of empty fields.

printwire_addresses expected it to be the last field, which is not
the case of a node_announcement with tlv
2022-03-01 09:38:40 +10:30
Rusty Russell 05bd62fee4 configure: restore -Og default on non-developer builds.
Reverts change in 1ef77504b1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-28 14:42:41 -06:00
Rusty Russell dd495b2b21 memleak: fix handling of excluded pointers.
We often hand an exclude pointer (usually the current command) to
memleak.  But when we encountered this we would stop iterating, rather
than just ignore it: this means we would often ignore significant siblings.

In particular, fixing this (which has always been there) reveals many
previously-undetected leaks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell ea71e9fe5b memleak: fix handling of notleak pointers.
We were ignoring the *parent* of the notleak pointers, not the notleak
pointer itself!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell fe86395e7c memleak: restore functionality accidentally removed.
Accidentally removed in 6c9b752751.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 7081413e20 plugins/funder: actually free the pending_opens.
It's a small leak, but real.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell d5064ff56c libplugin: make memleak see current requests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 9bf2aa65cd common/param: don't keep around the temporary array off cmd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 3218e5332f common/param: fix leak with opt_param_def.
We allocate the default, then callback allocates over the top.  Mark
params with a default, so we can free that when it's called.

(We can't do this generally, since not all param args are actually
pointers to pointers, though opt_param_def has to be).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell f0ea4d60b9 onchaind: fix minor leaks.
Not actually leaks, but they do live longer than they need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell c5b424b6ce plugins/pay: make memleak happy.
1. Tell memleak about our linked-list of current payments.
2. Don't remove them from list until we actually free them (in destructor, naturally).
3. Decode invoices into tmpctx (we steal / copy what we want anyway).
4. Free params after we've used them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 246b724dbe libplugin-pay: fix spurious leak reports.
1. The dijkstra can be temporary, doesn't need to last as long as pay cmd.
2. We fail multiple times in several places, so don't leak old failreason.
3. Make payments findable by our memleak detector.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 4b7d2dc5b8 common/bolt11: make decoded routes hang off routes arr, not b11.
This causes weirdness in pay, which tal_steal's b11->routes and expects
to get it all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell b3338a174e plugins/keysend: prevent leak detecting getting upset.
1. We don't keep a pointer to payments (unlike pay, which keeps a
   linked list), so mark it notleak.
2. plugin->our_features is overloaded for "features we want to set" (used by keysend)
   and then "features we have".  Create a new field, which is cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 6c8ea95f6e hsmd: remove unnecessary memleak_remove_region.
The current client is in dbid_zero_clients (i.e. it's lightningd).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell b67329bec2 lightningd: don't leak address for printing.
Not really a leak, but it would last as long as the peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 899f54894f gossipd: move deferred_update list ptr to head to ease memleak checks.
They can only follow pointers (without help) if they point to the
*start* of a tal object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell a901402e66 plugins/pay: fix unnecessary code.
1. p is a child of cmd, so it's freed by command_failed.
2. cltv_budget is set a few lines up to the same thing already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell 2a6d3cad01 libbacktrace: update to latest.
Seeing if this helps my build box which previously gives:

```
/home/rusty/lightning-ltest/lightningd/lightning_connectd: libbacktrace: unrecognized DWARF version in .debug_info at 6
/home/rusty/lightning-ltest/lightningd/lightning_connectd: libbacktrace: no debug info in ELF executable
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Christian Decker 86b83e473b pytest: Remove 3 stress-tests
These tests have proven to be:

 a) very expensive, as they spin up many nodes, and perform long setup
 b) are not testing anything specific, they just fuzz functionality
    that is already tested otherwise
 c) have not helped pinpoint any issues in living memory
 d) are very flaky, making for really bad signal-to-noise, so much
    that devs usually just restart without even looking at the logs
 e) even if we were to look at the logs, we'd be unable to reproduce
    due to the inherent randomness involved in these tests
 f) are really noisy neighbors, causing other tests to flake as well,
    further muddying the water

All in all, these tests are a waste of time, and source of
frustration.

[ Cleaned up python unused imports --RR ]
Changelog-None
2022-02-26 14:36:50 +10:30