Commit Graph

8657 Commits

Author SHA1 Message Date
Rusty Russell 924cc04bd2 bolt11: have caller supply preferred chain.
This lets us distinguish testnet from signet invoices, since they
have the same prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 09:24:14 +09:30
Rusty Russell 1356700e2b bitcoin/chainparams: signet uses the same P2PKH and P2SH version bytes as testnet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 09:24:14 +09:30
Rusty Russell d5cb0d85b5 utils: use a cleaner pattern to capture wally allocations.
We force use of tal_wally_start/tal_wally_end around every wally
allocation, and with "end" make the caller choose where to reparent
everything.

This is particularly powerful where we allocate a tx or a psbt: we
want that tx or psbt to be the parent of the other allocations, so
this way we can reparent the tx or psbt, then reparent everything
else onto it.

Implementing psbt_finalize (which uses a behavior flag antipattern)
was tricky, so I ended up splitting that into 'psbt_finalize' and
'psbt_final_tx', which I think also makes the callers clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:45:53 +02:00
Rusty Russell 66ffd7a73f bitcoin/: neaten use of libwally base58 and bas64 routines.
Now libwally uses tal, we don't need to make copies.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:45:53 +02:00
Rusty Russell 4e67b4fd03 Makefile: work with shasum -a as well as sha256sum.
At least on my Ubuntu box, they're compatible. If they're not, we need
to disable regeneration altogether.

Fixes: #4075
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:34:42 +02:00
Christian Decker d9ba39d2cb pytest: Add test for key-value style postgresql DSNs 2020-09-23 20:20:40 +09:30
Christian Decker 1d9e7cf079 db: Add support for key-value pair DSNs in postgresql
These are simple space-separated key-value pair sets of options instead of the
URI style DSNs, but they are also much more flexible allowing the user to
specify client SSL certificates, server certificates, compression and
encryption levels, and much more (see [1] for more information)

[1]: https://www.postgresql.org/docs/9.1/libpq-connect.html

Changelog-Added: db: Added support for key-value DSNs for postgresql, allowing for a wider variety of configurations and environments.
2020-09-23 20:20:40 +09:30
Antoine Poinsot 15adcc915f Remove varint typedef for bigsize
It's not part of the spec anymore

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-23 16:30:36 +09:30
Rusty Russell 7db8680530 lightningd: don't hand around partially-assembled channel_info.
wallet_commit_channel would fill in the old_remote_per_commit and
fee_states, which is weird since the caller doesn't care.

Make the caller set all the channel_info fields, so wallet_commit_channel
is a simple consumer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
Rusty Russell 58f6e316cc lightningd: separate fee_states out of struct channel_info.
It was always kind of weird in there anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
niftynei cf2a807d3c dual_open: don't release uc here, it gets freed elsewhere 2020-09-23 15:51:13 +09:30
niftynei db2b013990 dual-fund: call daemon_shutdown only
daemon_shutdown calls common_shutdown internally; calling both frees the
same context twice
2020-09-23 15:51:13 +09:30
niftynei 56b3711bbc dual_open: alignment nit 2020-09-23 15:51:13 +09:30
niftynei ed9f661e07 dual_open: memleak problem with fee_states 2020-09-23 15:51:13 +09:30
niftynei 7d836a5945 dual_open_control: nit, reword comment 2020-09-23 15:51:13 +09:30
niftynei aa6420cb58 dual_open: split logic for deserializing a changed vs signed PSBT
We need to run different checks on these, so split them out for finer
grained control
2020-09-23 15:51:13 +09:30
niftynei 1f165c00ae psbt_txid: it's possible a psbt may already have the finalized scriptsig
If we've already got a scriptSig field filled out for a PSBT input, we
use that instead of 'deriving' the scriptSig from the redeemscript
(finalizing a PSBT removes the redeemscript field)
2020-09-23 15:51:13 +09:30
niftynei c5f748e716 dualopend: tiny billboard capitalization change 2020-09-23 15:51:13 +09:30
niftynei 5f6a2c05bd dualopend: check that we're not exceeding max allowable capacity 2020-09-23 15:51:13 +09:30
niftynei 8761468f42 dualopend: use status failed instead of peer failed 2020-09-23 15:51:13 +09:30
Christian Decker 668debb74f pyln: Rename LightningNode.fund_channel to fundchannel
It is inconsistent with the RPC method, which was bothering me for some time.
2020-09-23 14:45:12 +09:30
Christian Decker 8ecb157af6 pyln: Add type annotations to lightning.py 2020-09-23 14:45:12 +09:30
Christian Decker 49ec800a07 pyln: Add type-annotations to plugin.py
This should help users that have type-checking enabled.
2020-09-23 14:45:12 +09:30
Christian Decker d27da4d152 pyln: Fix two formatting string not matching parameters 2020-09-23 14:45:12 +09:30
Christian Decker 748caf91d3 pyln-client: Add support to monkey patch the JSONEncoder
Several times we had issues with plugins not being able to re-encode an RPC
result because they forgot to use the custom encoder class. This allows us to
patch the JSONEncoder when we start the RPC or the plugin and automagically
support classes that provide a `to_json` method.
2020-09-23 14:45:12 +09:30
Christian Decker 382230509b pyln: Add stubs to generate documentation for pyln-client 2020-09-23 14:45:12 +09:30
Rusty Russell 75ce380e0e Travis: disable postgres test.
I don't have time to diagnose why it's failing all the time right now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:03:03 +09:30
Christian Decker 1cb527d46a topo: Do not keep txids in memory indefinitely
I mistakenly assumed the block would be freed after processing completed. That
is not true since chaintopology keeps headers and stubs around for reorgs. So
we need to remove the precomputed txids along with the full_txs.
2020-09-23 13:52:49 +09:30
Rusty Russell f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 0a50301d51 pytest: don't test for memleaks under valgrind.
The next patch perturbed things enough that we suddenly started
getting (with --track-origins=yes):

Valgrind error file: valgrind-errors.120470
==120470== Use of uninitialised value of size 8
==120470==    at 0x14EBD5: htable_val (htable.c:150)
==120470==    by 0x14EC3C: htable_firstval_ (htable.c:165)
==120470==    by 0x14F583: htable_del_ (htable.c:349)
==120470==    by 0x11825D: pointer_referenced (memleak.c:65)
==120470==    by 0x118485: scan_for_pointers (memleak.c:121)
==120470==    by 0x118500: memleak_remove_region (memleak.c:130)
==120470==    by 0x118A30: call_memleak_helpers (memleak.c:257)
==120470==    by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470==    by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470==    by 0x118B25: memleak_find_allocations (memleak.c:278)
==120470==    by 0x10EB12: closing_dev_memleak (closingd.c:584)
==120470==    by 0x10F3E2: main (closingd.c:783)
==120470==  Uninitialised value was created by a heap allocation
==120470==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==120470==    by 0x1604E8: allocate (tal.c:250)
==120470==    by 0x160AA9: tal_alloc_ (tal.c:428)
==120470==    by 0x119BE0: new_per_peer_state (per_peer_state.c:24)
==120470==    by 0x11A101: fromwire_per_peer_state (per_peer_state.c:95)
==120470==    by 0x10FB7C: fromwire_closingd_init (closingd_wiregen.c:103)
==120470==    by 0x10ED15: main (closingd.c:626)
==120470==

This is because there is uninitialized padding at the end of struct
peer_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell bba4fd0d70 common: don't suppress leak detection for libwally allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 161cc3e840 memleak: make "_notleak" names less powerful.
They previously prevented any child from being detected as leaks, now
they just mark the tal allocation itself as not being a leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 7b55d6cfe2 common: enforce that we have collected all wally allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell c23001d508 bitcoin: use tal_gather_wally() so we don't leave unattached allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell a316c517c7 common: add tal_gather_wally() function to reparent libwally objs.
This lets us reduce leaks, and ease their detection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 3c8049f32c bitcoin/psbt: psbt_input_add_unknown/psbt_output_add_unknown needs a tal ctx.
Since it allocates something, it needs a context (used in the next patch!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 77b62d9e42 bitcoin/psbt: psbt_finalize needs a tal ctx.
Since it returns a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 607075a3d4 bitcoin/psbt: wallt_tx_output needs a tal ctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 480f671e91 bitcoin/psbt: psbt_txid needs a tal ctx.
It returns a wally_tx; it's an anti-pattern not to hand in a tal context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 67fbb349b6 bitcoin/tx: trivial cleanups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell 11ebb84e7d bitcoin/psbt: attach destructors to wally allocations to avoid leaks.
This covers the obvious ones, but the later patches fix this more
seriously.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Some memory leaks in transaction and PSBT manipulate closed.
2020-09-23 13:52:49 +09:30
Rusty Russell 80c705d528 bitcoin/psbt: more const pointers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Antoine Poinsot 4b4fe2f4fb dual-openingd: msg_type should be dualopend_wire not u8
clang did the hard work here:

openingd/dualopend.c:958:42: error: result of comparison of constant 'WIRE_DUAL_OPEN_FAIL' (7003) with expression of type 'u8' (aka 'unsigned char') is always false
      [-Werror,-Wtautological-constant-out-of-range-compare]
        if ((msg_type = fromwire_peektype(msg)) == WIRE_DUAL_OPEN_FAIL) {
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~

Co-Authored-by: the clang compiler
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-23 13:46:02 +09:30
Christian Decker de7b41695f db: Fix size mismatch on postgres in a migration
Changelog-Fixed: db: Fixed a broken migration on postgres DBs that had really old channels.
2020-09-23 10:48:31 +09:30
Rusty Russell a08e83522b configure: clean up tmpfile if we fail.
Reported-by: @whitslack
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-21 13:07:01 +02:00
Rusty Russell d56f8b6c5d configure: refuse to create config.vars without any database.
We alter the script to do the move to config.vars right at the end,
which is probably a good idea anyway.

Fixes: #4054
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-19 12:38:54 +09:30
Rusty Russell 34a6294031 lightningd: really remove relative plugins dirs.
We promised this in 0.8.1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugin: Relative plugin paths are not relative to startup (deprecated v0.7.2.1)
2020-09-18 12:08:07 +09:30
Rusty Russell 288aa397fc lightningd: remove deprecated result for `plugin stop`.
Changelog-Removed: JSON API: `plugin stop` result with an empty ("") key (deprecated 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell 00115ee7f0 lightningd: remove deprecated rpc_command hook return.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON API: The hook `rpc_command` returning `{"continue": true}` (deprecated 0.8.1)
2020-09-18 12:08:07 +09:30
Rusty Russell 8aa660e938 lightningd: remove deprecated return for `db_write` hook.
Changelog-Removed: JSON API: The hook `db_write` can no longer return `true` (deprecated in 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30