Commit Graph

10105 Commits

Author SHA1 Message Date
Vincenzo Palazzo 33b8a55bec Update lnprototest submodule
This is a fix propose of the ticket https://github.com/ElementsProject/lightning/issues/4516

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-06-25 09:51:29 +09:30
Rusty Russell 408342dc23 doc: schema for listconfigs, compare with documentation.
This made me document what are options are now in plugins, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell d7e8e220e6 lightningd: clean up listconfigs output.
1. Hide the now-deprecated enable-autotor-v2-mode option.
2. Really don't print dev- options.
3. Don't print true and false as strings in some cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `listconfigs` would list some boolean options as strings `"true"` or `"false"` instead of using JSON booleans.
2021-06-25 09:49:33 +09:30
Rusty Russell 2ddecdc95a doc: schemas for everything else.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: doc: Epic documentation rewrite: each now lists complete and accurate JSON output, tested against testsuite.
2021-06-25 09:49:33 +09:30
Rusty Russell 977cb62e12 doc: listpeers schema.
This is the most complex one.  It gets its own commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell 6e636a835f tools/fromschema.py: handle deprecated null field, don't create empty lists.
1. listpeers has a deprecated `"closer": null`, which we need
   to handle in the schema, while trying not to damage our
   documentation too much.

2. Don't print a condition if there are no fields to print.

3. Allow a special "untyped" marker for multifundchannel which returns
   arbitrary JSON in a field.

4. Allow a single field return (for 'stop').

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell 1a181a3240 doc/schemas: listfunds, listinvoices, listnodes, listoffers
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell 2296d4452f doc/schemas: listfunds
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell 74eeaa8213 doc/schemas: getsharedsecret, help, invoice, listchannels, listforwards
And a new "u8" type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell a9f49b5326 doc: document how to try to write JSON schemas.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell fe161ff962 doc/schemas: tighten requirements.
If we have conditional fields, we often set `additionalProperties` `true`
at the top-level, but then we have to make sure to cover all the possible
combinations in conditionals, so unintended additions don't appear.

This revealed missing fields in delinvoice, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell bf74be3348 plugins: add `command` field to subcommand output.
Makes it possible to write a decent JSON schema, but means we need to carry
additional data, so we create a `struct plugin_command`.

We remove the unused struct dynamic_plugin, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell 0a99b8c64c listtransactions: don't create a msat field called "satoshis".
That's a terrible, terrible idea.  (Documentation comes in later patch
which has the schema).

Also, blockheight is a u32, so simplify.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listtransactions` `outputs` `satoshis` field (use `msat` instead).
2021-06-25 09:49:33 +09:30
Rusty Russell d8136d856b listpeers: clean up JSON output.
In general, it's better to omit a field than put in a 'null', and
putting variable-named fields in an object is also a bad idea.

This is reflected in how hard this is to express in JSON schema, too.

Others:
1. Remove the obsolete "funding": "LOCAL" from unopened channels, but add
   "opener": "local" as used in normal channels.
2. htlc cltv_expiry is a u16.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `funding_allocation_msat` and `funding_msat`: use `funding`.
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `last_tx_fee`: use `last_tx_fee_msat`.
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `closer` is now omitted if it does not apply, not JSON `null`.
2021-06-25 09:49:33 +09:30
Rusty Russell edee0793d3 pyln.proto: fix receiving unknown fields in TLVs.
We can still convert these (e.g. k=1) to Python, by just using hex strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-24 19:46:38 +09:30
Rusty Russell ca3680246e pyln.proto: fix handling of subtypes in TLVs.
This was revealed by using lnprototest on channel_types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-24 19:46:38 +09:30
Rusty Russell eaa2ff23f1 EXPERIMENTAL_FEATURES: don't offer to downgrade anchor outputs!
We don't support this, even if other side allowed it (which we certainly
don't).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-24 19:46:38 +09:30
Rusty Russell 35c8247b5f cleanup: remove unused pyln-spec symlinks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-24 19:46:38 +09:30
nathanael 262f90d115 replace freenode with libera
Changelog-None
2021-06-21 12:42:09 +09:30
Christian Decker 2510bcad07 pay: Fix another instance of a crash when we can't find ourselves 2021-06-20 14:26:00 +09:30
Rusty Russell 50ebdfb5e8 invoice: allow creation of giant invoices.
https://github.com/lightningnetwork/lightning-rfc/pull/877 talks about
removing this restriction (only Electrum actually enforced it on
receive), so start by allowing creation of giant invoices, though
we mark them as requiring mpp.

Changelog-Changed: JSON-RPC: `invoice` now allows creation of giant invoices (>= 2^32 msat)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-20 14:25:40 +09:30
Antoine Poinsot 1643a61499 pyln-testing: check if process died after having read all logs
We would fail even if a process exited cleanly after having the line we
were looking for, because we checked if it died before reading the logs.

Co-Authored-by: Daniela Brozzoni <daniela@revault.dev>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-06-18 10:40:07 +09:30
Antoine Poinsot d6f441bdf4 hsm_encryption: merge getline() logic between stdin and non-stdin
Avoid duplicating it, to minimize the potential for divergence.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-06-18 10:40:07 +09:30
Antoine Poinsot e19b67f6da qa: test hsm encryption password provided from stdin
Test that it roundtrips with the non-stdin way, in order to make sure we
don't introduce a discrepancy between the two.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-06-18 10:40:07 +09:30
Antoine Poinsot 526e6026f5 hsmtool: this is 'success', not 'succes'
I tried to wait_for_log() on "successfully" without success :/

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-06-18 10:40:07 +09:30
Michael Schmoock a53aa1aaef pyln: adds a help text to start plugins manually
Changelog-None
2021-06-18 09:59:54 +09:30
Rusty Russell 40544b74f9 plugins/bcli: fix uninitialized variable.
We didn't set this to false on non-regtest!

```
==198363== Conditional jump or move depends on uninitialised value(s)
==198363==    at 0x10EF88: estimatefees_parse_feerate (bcli.c:443)
==198363==    by 0x10F3BF: estimatefees_second_step (bcli.c:550)
==198363==    by 0x10E720: bcli_finished (bcli.c:258)
==198363==    by 0x1438A7: destroy_conn (poll.c:244)
==198363==    by 0x1438CB: destroy_conn_close_fd (poll.c:250)
==198363==    by 0x151A7A: notify (tal.c:240)
==198363==    by 0x151F91: del_tree (tal.c:402)
==198363==    by 0x15232D: tal_free (tal.c:486)
==198363==    by 0x141EB8: io_close (io.c:450)
==198363==    by 0x14400B: io_loop (poll.c:449)
==198363==    by 0x114BB0: plugin_main (libplugin.c:1414)
==198363==    by 0x1105C4: main (bcli.c:973)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-17 11:47:02 +09:30
Rusty Russell 976a79094b connectd: fix advertizing for Tor ports.
We were accidentally using the port that the tor service was
connecting to, not the /torport the user said to use.

Fixes: #4597
Reported-by: @openoms
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Config: `addr` autotor and statictor /torport arguments now advertized correctly.
2021-06-17 11:46:48 +09:30
Rusty Russell 41767560d6 common/test: test wireaddr parsing.
In case this was the issue (it wasn't)!.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-17 11:46:48 +09:30
Rusty Russell 75720ad0e1 lightningd: wait for gossipd to finish initalizing before starting plugins.
This mainly helps our CI under valgrind, which starts a fresh instance
and immediately calls the invoice command.  This can cause the topology
plugin to try to access the gossmap file before it's created.

We can also move the gossmap reading in topology to init time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 7e7ab4cb3b gossipd: remove routing, listchannels and listnodes infrastructure.
This involves removing some fields from the now-misnamed routing.h
datastructures, and various internal messages.

One non-obvious change is to our "keepalive" logic which refreshes
channels every 13 days: instead of using the 'enabled' flag on the
last channel broadcast to decide whether to refresh it, we use the
local connected status directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 8810ce7241 gossipd/test: remove routing tests.
We're about to remove routing from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 50cb810bd0 pyln-client: revert old/new comparisons for getroute, listchannels, listnodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell b6aa9510a9 signmessage: use listnodes instead of gossipd_getnodes_request.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell cda8f8190b invoice: overhaul routehints to use topology.listincoming, cleanup.
This turned into a more extensive cleanup than intended.  The previous
warnings were overlapping and confusing, especially now MPP is the norm.

*warning_capacity* is now the "even under best circumstances, we don't
have enough incoming capacity", which is really what
warning_mpp_capacity was trying to say (no longer printed).

*warning_offline* and *warning_deadends* are only given if adding such
peers would have helped give capacity (i.e. not if *warning_capacity*
is set).  The new *warning_private_unused* tells you that we would
have sufficient capacity, but we refused to expose private channels.

The test cases have been enhanced to cover the new warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now gives `warning_private_unused` if unused unannounced channels could have provided sufficient capacity.
Changelog-Changed: JSON-RPC: `invoice` warnings are now better defined, and `warning_mpp_capacity` is no longer included (since `warning_capacity` covers that).
2021-06-16 10:29:17 +09:30
Rusty Russell 9e1b830191 plugins/topology: listincoming function to help invoices.
Invoices need to know what the peers' update contains, to create
routehints.  It also wants to know if a peer has no other public
channels (so-called "dead end" peers) to eliminate them from routehint
consideration.

This was previously done by a special function to ask gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 9a2d73891b pyln-client: hack in test that listnodes returns the same old/new.
It sometimes fails because the two race, and sometimes because there's
randomness, but it generally works (and doesn't fail systemically).

We remove this before the final merge.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 280f5a71a8 plugins/topology: add listnodes command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell e734097125 common/wireaddr: fromwire_wireaddr_array helper.
This was previously internal to gossipd and was called "read_addresses".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 79299a4780 pyln-client: hack in test that listchannels returns the same old/new.
It sometimes fails because the two race, and sometimes because there's
randomness, but it generally works (and doesn't fail systemically).

We remove this before the final merge.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 1a7731cfbf plugins/topology: make listchannels mark disconnected local channels active=false.
This is blurring the lines a bit, but it's closer to what gossipd did.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 63d97d2467 plugins/topology: listchannels command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell d1ab5e3872 pyln-client: hack in test that getroute returns the same old/new.
It sometimes fails because the two race, and sometimes because there's
randomness, but it generally works (and doesn't fail systemically).

We remove this before the final merge.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell e81e640e76 topology: plugin to implement getroute command.
Temporarily rename old getroute to getrouteold (we will remove this).

Changelog-Changed: JSON-RPC: `getroute` is now implemented in a plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell e784345b55 common: note that command_fail doesn't return NULL.
This suppresses some "may-be-uninitialized" warnings later.  It makes
gcc pickier about how we ignore the result though :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 38fad0f3e4 ccan: update to get RETURNS_NONNULL macro.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell 2910bb0235 pytest: fix flake in test_misc.py::test_funding_reorg_* xxremote_lags
If l2 doesn't have the funding locked in, the rest of the test fails
(we got a timeout on `wait_for(lambda: [c['active'] for c in l2.rpc.listchannels('103x1x0')['channels']] == [False, False])`)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-15 06:36:36 +09:30
Rusty Russell 11d24dea9b wire/Makefile: always generate non-exp files.
I wasn't regenerating these when I changed dependencies, because I was
configured with --enable-experimental-features.  Putting them in
ALL_C_HEADERS and ALL_C_SOURCES means they'll be regenerated, even
though nothing depends on them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-15 06:34:56 +09:30
Rusty Russell 3aafe8cacf Makefile: `pytest` target should depend on ALL_TEST_PROGRAMS.
In particular, this includes the test plugins!

Not sure why this breaks *now* though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:20 +09:30
Rusty Russell 064ad486e3 close: check that destination is going to be accepted.
Prior to this, sending a v1 address (or, in fact, any random crap!)
would cause the unsupporting node to unilaterally close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:20 +09:30