Commit Graph

188 Commits

Author SHA1 Message Date
Rusty Russell a11edebb7c utf8: handle UTF-8 arrays.
BOLT 12 introduces this as a new fundamental type, which lets us easily
validate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 9d656464f5 bitcoin/pubkey: add pubkey32 primitive for xonly pubkey types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell 23d4e53246 connectd: add annotation and fix up formatting on connectd/netaddress.c
This is stolen from the bitcoind source, so we use their style
(no tabs, 4-wide spaces).  Clean it up and add the emacs magic to
maintain it in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:26:52 +01:00
Rusty Russell 089ecfe418 connectd: fix valgrind complaint on FreeBSD
As reported by Wladimir J. van der Laan.  Valgrind will complain
about padding and unset fields, so memset the structs.

```
==42653== Syscall param socketcall.connect(serv_addr..sa_len) points to uninitialised byte(s)
==42653==    at 0x4C7D19A: _connect (in /lib/libc.so.7)
==42653==    by 0x4EE1F35: ??? (in /lib/libthr.so.3)
==42653==    by 0x249D57: get_local_sockname (netaddress.c:212)
==42653==    by 0x249CDB: guess_address (netaddress.c:242)
==42653==    by 0x2473D0: public_address (connectd.c:1003)
==42653==    by 0x246CE4: setup_listeners (connectd.c:0)
==42653==    by 0x246566: connect_init (connectd.c:1311)
==42653==    by 0x270CEB: next_plan (io.c:59)
==42653==    by 0x2713EE: io_ready (io.c:417)
==42653==    by 0x2726B1: io_loop (poll.c:445)
==42653==    by 0x24618A: main (connectd.c:1703)
==42653==  Address 0x7fc000690 is on thread 1's stack
==42653==  in frame #3, created by guess_address (netaddress.c:231)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:26:52 +01:00
Rusty Russell ff8830876d wire/tlvstream: add tlv_make_fields helper to populate ->fields array.
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!

Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell 5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire.
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell ca2bd98082 unittest: use common_setup / common_shutdown almost everywhere.
Avoids much cut & paste.  Some tests don't need any of it, but most
want at least some of this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:55:09 +10:30
niftynei abad494fcf connectd: properly cleanup 'competing' outgoing connections
Coauthored-By: Rusty Russell @rustyrussell
2020-11-16 20:00:51 -06:00
niftynei 4a3ee19a22 connectd: Update connection list with new address
If we're already attempting to connect to a peer, we would ignore
new connection requests. This is problematic if your node has bad
connection details for the node -- you can't update it while inflight.

This patch appends new connection suggestions to the list of connections
to try.

Fixes #4154
2020-11-16 20:00:51 -06:00
niftynei 4508584b21 dualfund: rearrange things so that the wire-dependent calls are separate
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.

This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
2020-10-20 14:27:19 +10:30
niftynei 82c0b48215 wires: towire/fromwire for wally_tx
We're eventually moving away from 'bitcoin_tx
2020-10-20 12:50:31 +10:30
Rusty Russell f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Matt Whitlock eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
Rusty Russell f658dd0d78 pytest: test connection timeout.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-11 21:27:45 +09:30
Rusty Russell 7f2b332021 connectd: implement connection timeout (60 seconds).
This is simple, and we now can multifundchannel to every node on testnet
(one simply hangs once we connect).

Changelog-Fixed: Protocol: We now hang up if peer doesn't respond to init message after 60 seconds.
2020-09-11 21:27:45 +09:30
Rusty Russell c76cfe9c5c Makefile: fix typo which can cause build failures.
And add secp_recovery to headers, while we're at it.

```
./wire/wire.h:7:10: fatal error: secp256k1_recovery.h: No such file or directory
 #include <secp256k1_recovery.h>
          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:254: recipe for target 'connectd/test/run-initiator-success.o' failed
make: *** [connectd/test/run-initiator-success.o] Error 1
m
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 20:44:22 +09:30
niftynei 5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei 303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
niftynei 8d429ecd06 df: add needed info to any PSBT we produce
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
2020-09-09 19:54:20 +09:30
Rusty Russell 0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes.
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
Rusty Russell c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell d8e8426b52 Makefile: remove EXPERIMENTAL_FEATURES marker from generated files
We're going to make experimental versions of these completely separate files.

Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn.  We can always force-rebuild when we change a rule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell 82f2f43425 Generated files: more merge fallout.
We changed the generation templates!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 11:10:21 +09:30
Rusty Russell 3c6af3efb4 Makefile: commit and preserve all the wiregenerated files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: we no longer require extra Python modules to build.
2020-08-31 21:33:26 -05:00
Rusty Russell 1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell 3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 398b4806b9 connectd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell 007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell 168009c105 features: require dependent features at init handshake.
This simplifies our test matrix, as we never have to handle talking
to peers that specify one but not the other.

This is particularly important for option_anchor_outputs which
assumes option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell ffbb409b47 amount: use initializers everywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
niftynei bc5a817100 elements: convenience methods for dealing with assets
We don't preserve detailed asset information at the moment, so provide a
way to convert from a sat to an amount_asset struct.

We also need a way to convert from an 'amount_asset' to a 'value' for
elements, which for explicit (i.e. non-blinded) asssets is a 0x01 prefix
plus the big-endian encoded value.
2020-07-13 11:37:24 +09:30
Rusty Russell 731e037b36 change_amount: routine to determine if change output is worthwhile.
This can be used by plugins which create their own txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell a9427f1a8d bitcoin/feerate: new exposure for feerate parsing outside lightningd.
This exposes the numeric part of param_feerate() as param_feerate_val().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Saibato eea6b33154 connect/tor: Set errno, if connection failed we want to reflect this.
Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato 2b0aba13a8 connectd/connectd: Display an error hint for V3 tor onions when connect fail.
@thestick613 noticed that since tor version below 0.3.2.2-alpha
will not support V3 ed25519 address formats, the error handling
is not that helpful in the error message from cli.
So now we add an hint.

Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>

connectd/connectd.h; Add helper function to update conn error list

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
niftynei 2e9c387f45 coin_moves: update withdrawal logic to account for 'variable owner' txs
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)

Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
2020-06-29 16:10:05 +02:00
niftynei 559f88faa1 psbt: add serialize to/from wire for psbts 2020-05-21 18:45:07 +09:30
Rusty Russell b920d4d21b wire: make fromwire_fail return non-const, use it more
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case.  Use that more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-20 11:36:13 +02:00
niftynei fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
Rusty Russell cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 197d1bcef2 wire: move towire/fromwire_short_channel_id out to bitcoin/short_channel_id.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 4dbfce6057 wire: move towire/fromwire_preimage out to bitcoin/preimage.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell 27220646c3 common/wire: move bitcoin-specific marshalling functions into bitcoin files.
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those.  Now we do, so
there's less excuse for this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell b0c9059602 tools/generate-wire: no more lonely messages!
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward.  This changes it to directly
access those fields.

This is not only more elegant (60 fewer lines), it would also be
more cache friendly.  That's right: cache hot singles!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-06 14:56:09 -05:00
Sebastian Falbesoner 84ef0cf7fa connectd: fix '~' comment (s/pubkey/node_id/)
Usage of node_id was introduced with commit a2fa699e0e.

Changelog-None
2020-05-04 12:32:02 +02:00