Commit Graph

9348 Commits

Author SHA1 Message Date
Rusty Russell 3776af4a35 common/sphinx: make TOTAL_PACKET_SIZE a macro.
This paves the way for using it on different-sized onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell dc67b60e28 common/sphinx: eliminate temporary buffer for prefill.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell b5ab7e3ce3 common/sphinx: don't use temporary to xor in cipher stream.
The chacha API makes this a bit awkward, to we use a helper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell 0701f74878 common/sphinx: don't make copy to compute packet hmac.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Christian Decker 5a69b94f9a pytest: Add test for keysend feature support 2020-12-11 15:50:09 +01:00
Christian Decker 3b599b846e keysend: Check that the destination supports keysend upon init
We were blindly initiating the keysend payment, which could lead to
confusing outcomes. This adds a very specific error message to the
error returned.

Changelog-Fixed: keysend: Keysend now checks whether the destination supports keysend before attempting a payment. If not a more informative error is returned.
2020-12-11 15:50:09 +01:00
niftynei 1ca2c66c54 make: have `git submodule update` be recursive
libwally added libsecp256k1 as a submodule, so we need to recursively
update the submodules now
2020-12-11 13:20:54 +01:00
Christian Decker 68d08fc7d7 pyln: Add TOR and SOCKS5 support in pyln.proto.wire.connect
I wanted to talk to TOR-based nodes, so here comes TOR support :-)
2020-12-11 11:39:18 +10:30
Rusty Russell 25b0dbe7e8 wallet: remove debugging fprintf.
Left over from e81d78ec4c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-10 20:32:18 +10:30
Rusty Russell 86176e8d0a hsmd: code to sign a bolt12 merkle root.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell f315f1c63f common/blindedpath: EXPERIMENTAL creation of blinded paths.
For inclusion in the onion as a reply path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 1d1804e318 devtools/bolt12-cli: cheap ripoff of bolt11-cli tool.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 3404914416 common/iso4217: define the currency codes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 4d086e9939 common/bolt12: encode/decode for bolt12 offer, invoice_request and invoice
Note the collapse of '+\s*' and the test cases from the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell d3ef871833 common/bech32_util: simple routines for checksumless bech32 charset usage.
Offers don't use the checksum, as they're signed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell f06ad0471c common/bolt12_merkle.h: experimental bolt 12 implementation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 28a903c917 channeld: hand up onionmessage fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell c1bdaa27bc sendonionmessage: add invoice, invoice_request and invoice_error fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 2cd93597cc wire.c: import bolt 12 offers CSV.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell a11edebb7c utf8: handle UTF-8 arrays.
BOLT 12 introduces this as a new fundamental type, which lets us easily
validate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 2734fd274e gossmap: helper to try to map x-only nodeid into 33-byte pubkey id.
This is a bridge for offers which use x-only pubkeys, and the current
code which doesn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 9d656464f5 bitcoin/pubkey: add pubkey32 primitive for xonly pubkey types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 094889f50f common/json_tok: minor cleanup to bitcoin_outpoint array parsing.
No reason to use a temp var here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 06:56:21 +10:30
Rusty Russell 6f205896aa bitcoin/tx: fix type of outpoint (n is a u32), simplify json_to_outpoint
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 06:56:21 +10:30
Christian Decker ad2f6cdabb cli: Add a JSON parsing sanity check before sending request
If some parameter is malformed on the command line we could end up
with a malformed JSON-RPC request, which would then result in very
unhelpful error messages.

Fixes #4238
Changelog-Changed: cli: `lightning-cli` now performs better sanity checks on the JSON-RPC requests it sends.
2020-12-09 06:56:21 +10:30
Christian Decker b2a5cf422f jsonrpc: Forward errors on malformed requests to cli
We were masquerading errors when parsing the request by reporting only
a bogus malformed `id` field in the response, when the real issue was
that we were unable to parse the request in the first place (which
caused the null-id error to be returned).

Fixes #4238
2020-12-09 06:56:21 +10:30
Christian Decker 71fafd23fd pytest: Add a test to reproduce #4258 2020-12-09 06:56:21 +10:30
Christian Decker e186b2620a txprepare: Verify that outputs arg is an array
We were not checking that outputs is indeed an array, and just going
ahead creating the array of outputs. Since `tok->size` for a string is
0 we ended up ignoring the argument altogether and thus the created
transaction would end up only with a single change output.

Fixes #4258
2020-12-09 06:56:21 +10:30
Christian Decker 47b8b46d54 cleanup: Remove stdio header from param.c 2020-12-09 06:56:21 +10:30
Christian Decker 55d9492d9e txprepare: Use param_outpoint_arr helper to validate input 2020-12-09 06:56:21 +10:30
Christian Decker 32000b6660 json: Add two param functions to parse string arrs and outpoint arrs
In a couple of places we accept arrays of strings and don't validate
them. If we forward them, e.g., call a JSON-RPC method from the
plugin, we end up embedding the unverified string in the JSON-RPC
call without escaping, which then leads to invalid JSON being passed
on.

This at least partially causes #4238
2020-12-09 06:56:21 +10:30
Christian Decker eacc54646f pyln: Remove deprecated txprepare variant
We promised to deprecate the old "destination satoshi feerate utxos"
variant some time on mid-2020 so let's do just that. `txprepare`
doesn't parse it either anymore.

Changelog-Changed: pyln: `txprepare` no longer supports the deprecated `destination satoshi feerate utxos` call format.
2020-12-09 06:56:21 +10:30
Michael Schmoock 362284981d plyn: use math.floor for msat mul and div 2020-12-08 13:48:14 +01:00
Michael Schmoock 565dc95a8e pyln: fix msat float multiplication 2020-12-08 13:48:14 +01:00
Michael Schmoock 2678cfb364 pyln: extend msat floating testcases
This adds two more xfail'ing testcases to show that the current way of
parsing Millisatoshi decimals is not yet optimal.

Changelog-None
2020-12-08 13:48:14 +01:00
Christian Decker c656cfe38d channeld: Fix the shutdown_sent billboard direction
While debugging a hanging channel with a user I noticed that they
called `close` on a channel, resulting in the channel showing
`CHANNELD_SHUTTING_DOWN`, but the billboard seemed to show the
information the wrong way around:

```json
{
   "peers": [
      {
         "connected": true,
	 // ...
         "channels": [
            {
               "state": "CHANNELD_SHUTTING_DOWN",
	       // ...
               "status": [
                  "CHANNELD_SHUTTING_DOWN:Reconnected, and reestablished.",
                  "CHANNELD_SHUTTING_DOWN:Funding transaction locked. They need our announcement signatures. They've sent shutdown, waiting for ours"
               ],
	       // ...
            }
         ]
      }
   ]
}
```

Aside from the hung channel, the switch in direction of the status
seemed weird. Checking the billboard code seems to have the status
switched as well:

ff8830876d/channeld/channeld.c (L223-L226)

We set `shutdown_sent[LOCAL]` when we send the shutdown:

ff8830876d/channeld/channeld.c (L823-L839)

And we set `shutdown_sent[REMOTE]` when we receive the shutdown:

ff8830876d/channeld/channeld.c (L1730-L1781)

So I think the billboard code just needs to be switched around.

Changelog-Fixed: JSON-RPC: The status of the shutdown meesages being exchanged is now displayed correctly.
2020-12-08 11:47:13 +10:30
Christian Decker eae49727b1 pytest: Disable flaky test_fundchannel_start_alternate
This test has been causing most CI runs to fail. See tracking issue #4265
for discussion on fixing and re-enabling it.

Changelog-None
2020-12-08 09:14:25 +10:30
Rusty Russell 23d4e53246 connectd: add annotation and fix up formatting on connectd/netaddress.c
This is stolen from the bitcoind source, so we use their style
(no tabs, 4-wide spaces).  Clean it up and add the emacs magic to
maintain it in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:26:52 +01:00
Rusty Russell 089ecfe418 connectd: fix valgrind complaint on FreeBSD
As reported by Wladimir J. van der Laan.  Valgrind will complain
about padding and unset fields, so memset the structs.

```
==42653== Syscall param socketcall.connect(serv_addr..sa_len) points to uninitialised byte(s)
==42653==    at 0x4C7D19A: _connect (in /lib/libc.so.7)
==42653==    by 0x4EE1F35: ??? (in /lib/libthr.so.3)
==42653==    by 0x249D57: get_local_sockname (netaddress.c:212)
==42653==    by 0x249CDB: guess_address (netaddress.c:242)
==42653==    by 0x2473D0: public_address (connectd.c:1003)
==42653==    by 0x246CE4: setup_listeners (connectd.c:0)
==42653==    by 0x246566: connect_init (connectd.c:1311)
==42653==    by 0x270CEB: next_plan (io.c:59)
==42653==    by 0x2713EE: io_ready (io.c:417)
==42653==    by 0x2726B1: io_loop (poll.c:445)
==42653==    by 0x24618A: main (connectd.c:1703)
==42653==  Address 0x7fc000690 is on thread 1's stack
==42653==  in frame #3, created by guess_address (netaddress.c:231)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:26:52 +01:00
Michael Schmoock 4507203a48 doc: fees on listpeers 2020-12-07 14:26:36 +01:00
Michael Schmoock ce1a2f134a pytest: tests fee_base and fee_ppm on listpeers 2020-12-07 14:26:36 +01:00
Michael Schmoock c4b7f4d9d2 feat: adds local halfchan fees to listpeers
This will add `fee_base` (msat) and `fee_ppm` (u32 num) to the RPC
`listpeers` output.

Changelog-Added: fee_base and fee_ppm to listpeers
2020-12-07 14:26:36 +01:00
Rusty Russell e13e3e90ec onchaind: cap max_possible_feerate using commitment transaction.
The commitment tx uses the same feerate as the HTLC txs (which we have
to grind to find), but we can't use it directly since the fee could be
increased by the presence of dust HTLCs.  We can still use it to cap
the maximum though.

Time before: 1m6.984s
Time after:  0m15.794s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: onchaind is much faster when unilaterally closing old channels.
2020-12-07 14:23:59 +01:00
Rusty Russell 5bc61e77cb onchaind: speed up HTLC matching by skipping identical CLTVs.
We try signatures to see which HTLC (we can have many) is the right one;
we can trivially match htlcs against commitment tx outputs, but the CTLV
can vary, and that's inside the htlc tx itself.

By sorting them, it's easy to skip comparing duplicates:

Time before: 2m32.547s
Time after: 1m6.984s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell e9e6f72d7c bitcoin_tx_check: don't rely on tmpctx.
We assert() this in onchaind while grinding fees; better to free newtx.

Before this we hit 530MB, after a mere 2.5MB.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: onchaind uses much less memory on unilateral closes for old channels.
2020-12-07 14:23:59 +01:00
Rusty Russell f22d719d2c onchaind: test stressful input.
In particular, this made valgrind OOM, and chewed much CPU.  I dumped the
input and output into a file to allow easy replay.

This will break as soon as we change onchaind's wire format, but it will
serve its purpose until then!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell af8eabb9a7 common/daemon: remove notifiers on root at cleanup.
Under some circumstances, valgrind complains.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell ff8830876d wire/tlvstream: add tlv_make_fields helper to populate ->fields array.
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!

Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell 5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire.
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00