Commit Graph

31 Commits

Author SHA1 Message Date
Christian Decker e4ab98459c wallet: Add tooling to extract SQL queries and generate driver info
This is the counterpart of the annotations we did in the last few commits. It
extracts queries, passes them through a driver-specific query rewriter and
dumps them into a driver-specific query-list, along with some metadata to
facilitate processing later on. The generated query list is then registered as
a `db_config` and will be loaded by the driver upon instantiation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Rusty Russell 0fdf22988d configure: absorb python3-mako check.
This is a better place for such checks than the Makefile, and we'll likely
have more in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell df298ba7e3 configure: suppress "maybe uninitialized" warnings altogether unless -O3.
We were just telling GCC not to treat them as errors: this suppresses them
entirely unless at -O3.  People keep trying to "fix" them, when in fact
they're false positives, as revealed with "./configure COPTFLAGS=-O3".

Fixes: #2856
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 54b66f0049 configure: fix --reconfigure, improve style, set defaults in one place.
I noticed that DEVELOPER was being reset to 0 by ./configure --reconfigure;
that's because we set the defaults first, then --reconfigure would only
override *unset* vars.  We now set up defaults last.

We unify all our "find the default" functions, to neaten them; in
particular find_pytest and our open-coded valgrind testing function.

We can figure out whether valgrind works using /bin/true instead of waiting
until configurator is built.

We're also more careful with ${FOO-default} vs ${FOO:-default}; the former
does not replace if FOO is set to the empty string, which is possible for
some of our settings (COPTFLAGS, CDEBUGFLAGS in particular).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-07 10:37:52 +02:00
Rusty Russell 7092d215fa configure: detect modern gcc, and only suppress necessary errors.
Drive-by fix of CONFIGURATOR_CC setting when CC is overridden on cmdline
(not via env var).

Note that clang defines __GNUC__ (to 4) :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell ed08ca6df3 configure: handle --reconfigure better.
`./configure CC=newcc --reconfigure` didn't set CC, because reconfigure
simply replaced all values: only make it replace undefined values.

Also, it didn't change COPTFLAGS or CWARNFLAGS, even if they were previously
the defaults (eg. --reconfigure --enable-developer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell 6b32b72dee configure: use "-Og" for non-developer builds, add COPTFLAGS variable.
Unfortuntely we get spurious uninitialized variable warnings with
anything but -O3 or no optimization, so set default CWARNFLAGS
appropriately.

MCP bench results without optimization:
   store_load_msec:28509-31001(29206.6+/-9.4e+02)
   vsz_kb:580004-580016(580006+/-4.8)
   store_rewrite_sec:11.640000-12.730000(11.908+/-0.41)
   listnodes_sec:1.790000-1.880000(1.83+/-0.032)
   listchannels_sec:21.180000-21.950000(21.476+/-0.27)
   routing_sec:2.210000-11.160000(7.126+/-3.1)
   peer_write_all_sec:36.270000-41.200000(38.168+/-1.9)

MCP bench with -Og: 22% speedup vs no optimization
   store_load_msec:21963-23645(22841+/-6.6e+02)
   vsz_kb:579916
   store_rewrite_sec:10.080000-10.960000(10.456+/-0.3)
   listnodes_sec:1.280000-1.390000(1.338+/-0.047)
   listchannels_sec:14.770000-16.080000(15.518+/-0.46)
   routing_sec:0.990000-6.660000(3.958+/-2.2)
   peer_write_all_sec:29.950000-32.950000(31.138+/-1)

MCP bench with -O2: 31% speedup vs no optimization
   store_load_msec:20713-22088(21505.6+/-4.8e+02)
   vsz_kb:579928
   store_rewrite_sec:9.570000-11.200000(10.192+/-0.54)
   listnodes_sec:0.960000-1.090000(1.028+/-0.045)
   listchannels_sec:10.400000-11.770000(11.012+/-0.48)
   routing_sec:0.300000-3.140000(1.978+/-1.1)
   peer_write_all_sec:28.980000-30.310000(29.572+/-0.44)

MCP bench with -O3 -flto: 36% speedup vs no optimization
   store_load_msec:19616-20191(19862.6+/-1.9e+02)
   vsz_kb:578452
   store_rewrite_sec:8.980000-9.960000(9.55+/-0.32)
   listnodes_sec:0.920000-1.910000(1.18+/-0.38)
   listchannels_sec:8.960000-9.450000(9.206+/-0.16)
   routing_sec:0.730000-1.850000(1.438+/-0.42)
   peer_write_all_sec:28.090000-29.410000(28.772+/-0.42)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-16 15:39:56 -04:00
GreenAddress fb07265663 remove libbase58, use base58 from libwally (#2594)
* remove libbase58, use base58 from libwally

This removes libbase58 and uses libwally instead.

It allocates and then frees some memory, we may want to
add a function in wally that doesn't or override
wally_operations to use tal.

Signed-off-by: Lawrence Nahum lawrence@greenaddress.it
2019-04-30 23:07:31 +02:00
Rusty Russell 4b91439c08 configure: check for sqlite3_expanded_sql (SQLite3 version 3.14, 2016-08-08)
Our Travis, for example, doesn't have 3.14.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell 102a67cbb2 configure: fix detection of builtin libs with -Werror.
We didn't include stdlib, so these tests "failed" due to implicit
declaration of exit().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Christian Decker 5518c7cba8 configure: Check for lower-case IETF acronym in sodium constants
The v1.0.9 release of libsodium added
crypto_aead_chacha20poly1305_ietf_NPUBBYTES which we use; before that it was
...IETF_NPUBBYTES.

Since that release was in April 2016, it seems fair to simply check for
ancient versions and use the internal one if found. The alternative would be
to use the older names (which are still in the header), but given we've never
tested with such old versions, this seems safer.

Reported-by: Zoltán Gálli <@gallizoltan>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-03-04 02:38:53 +00:00
Rusty Russell d98fb0fd5b configure: remove bashism.
== is bash-only; for other shells this gives an error (meaning that you won't
get the sanity check):

	./configure
	Compiling ccan/tools/configurator/configurator...done
	./configure: 148: [: gcc: unexpected operator

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-28 05:17:21 +00:00
Rusty Russell 17c8090a87 configure: relax --enable-address-sanitizer check a little.
If you use use CC='gcc <options>' this blocks ASAN.  Of course, no
check is perfect, but this catches the obvious misuse still.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-22 11:20:51 -08:00
Rusty Russell d413fc7e9b configure: use system libbase58 if available.
Also one less headache for reproducible builds.  But unlike
libsodium, this only seems common in Ubuntu.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-08 01:10:17 +00:00
Rusty Russell 21fd8f7eaa configure: use system libsodium if available and modern.
Also one less headache for reproducible builds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-08 01:10:17 +00:00
Christian Decker 7eaf5b55ff make: Add an option to compile with address sanitizer
Currently only works with `gcc` due to google/sanitizers#1028, so
configure makes sure we warn if clang with ASAN is attempted.

According to [my benchmarks][benchmarks] the performance degradation
is small enough to have it active always.

[benchmarks]: https://github.com/ElementsProject/lightning/issues/2277#issuecomment-455897417

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-03 23:50:18 +00:00
Rusty Russell 61420bf79b configure: (mostly) revert b7a56f0531
Plugins are a first-class citizen again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:52:13 +00:00
Christian Decker b7a56f0531 plugin: Gate the plugin subsystem with the --enable-plugins configure flag
Since we are planning to release a bug fix release, and the plugin
subsystem is not yet complete, it is better to make plugin support
opt-in while we continue testing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-10 17:15:10 -08:00
arowser c99aa0a21d update pytest check 2018-12-19 21:05:24 +01:00
arowser 7b043503e2 hide pytest check message 2018-12-19 21:05:24 +01:00
arowser 64677dcbe5 support static link 2018-12-18 00:10:23 +00:00
Rusty Russell 4b77d803ef configure: don't turn on EXPERIMENTAL_FEATURES just because DEVELOPER.
It's about to become even more bleeding-edge.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 22:25:32 +00:00
Rusty Russell fab5027d64 configure: add --experimental-features flag (default: == DEVELOPER).
This will be used for option_simplified_commitment and other
still-being-specified options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
lucash-dev b52fb14726 pytest: Fix configure to find pytest when installed using pip3.
Installing pytest through pip3 (at least sometimes) doesn't create a script.
This means calling `which` won't work.
Changed configure so that it can also test if the module is present by calling python/python3.
Change the error message for when pytest can't be found, so that it's clear to the user `configure` must be ran again after installing pytest.
2018-09-05 02:29:06 +00:00
Jan Sarenik 40ea74ba99 configure: Check for version of pytest
Addresses #486 and takes into account @rustyrussell 's
comment about PYTEST defined in the environment which
always overrides without checking the version.
2018-06-15 01:55:44 +02:00
Rusty Russell 6a73c7eeb1 configure: fix defaults in help.
Both "[ 1 ]" and "[ 0 ]" both evaluate true in shell; we really want
an explicit '1' test.

Before:
$ ./configure --help
...
  --enable/disable-compat (default disable)

After:
$ ./configure --help
...
  --enable/disable-compat (default enable)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:38:24 +02:00
Rusty Russell 46aaf08047 configure: add hack to rename ccan's sha256 routines to avoid libwally clash
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:26:25 +02:00
Rusty Russell 56046d470b configure: take PYTEST from env var or cmdline.
We should only autodetect if it's not set manually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-09 13:39:27 +02:00
Christian Decker 3cf1581d6d configure: Use the ./configure step to find pytest
Fixes #1544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Rusty Russell ca8671de0a configure: add --enable-valgrind/disable-valgrind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell f6d22ed0cf configure: new script.
You can use environment variables or the commandline to set defaults.

It looks very autoconf, but you don't need to learn m4.

Doesn't cover all the obscure flags, but it's easy to extend.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00