Commit Graph

615 Commits

Author SHA1 Message Date
darosior 70a79e3998 plugins/bcli: a new plugin for gathering Bitcoin data
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
  whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
  `getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
  `getfilteredblock` in `lightningd/bitcoind`.
2020-02-12 11:45:07 +10:30
Christian Decker 966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
darosior da476848d1 pytest: add a C testing plugin to test libplugin
We mark the test as xfail() as it exposes that libplugin's PLUGIN_RESTARTABLE
was not taken into account !
2020-02-10 09:49:15 +10:30
Niklas Claesson f9b3b96a63 external: Reorganize to support cross-compilation
On CI it is nice to cache the external dependencies. However if we
always compile them in the same folder we cannot cache for multiple
different architectures. After this commit native compile targets will
still live in `external` but cross compiled versions will live in
`external/<arch>`.
2020-02-04 11:16:02 +10:30
Vasil Dimov 1015e3774a build: avoid repetition of `LC_ALL=C sort`
Put `LC_ALL=C sort` in a variable and use it everywhere. It was
forgotten in one place.

Without `LC_ALL=C` it would order `a.b` before `a-c` even though
`. (0x2E)` > `- (0x2D)`.

Changelog-None
2020-02-03 15:38:11 +00:00
Christian Decker 24219b38f9 pyln: Add unit tests from pyln-* to `make check-python` 2020-02-01 16:50:58 +01:00
Rusty Russell f373cd45be Makefile: update BOLTVERSION to latest master.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell 6450a7e315 lightningd: update spec and remove allowance for final node to receive a fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Now correctly reject "fees" paid when we're the final hop (lightning-rfc#711)
2020-01-31 06:07:56 +00:00
Rusty Russell ff1e1dd487 Makefile: update BOLTVERSION to flatten features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell a0a2c257be Makefile: add a COMPAT_V080 flag.
We're going to need this in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Vasil Dimov 8adbd4d31e doc: also build docs when building everything else
Before this change `make` would build everything but the docs.
They would be built during `make install` which is unexpected.

Add a dependency to `doc-all` to the `default` target so that docs are
build together with the rest of the programs. Leave the `install-data`
as is, to still build the docs if they are not built yet during install.

Changelog-None
2020-01-15 12:34:57 +01:00
Vasil Dimov 899f5dee53 build: fix version extraction from non-git directory
https://github.com/ElementsProject/lightning/releases/download/v0.8.0/clightning-v0.8.0.zip
extracts into "clightning-v0.8.0".

https://github.com/ElementsProject/lightning/archive/v0.8.0.tar.gz
extracts into "lightning-0.8.0".

Tweak the regular expression in the Makefile to support both (extract
0.8.0 from either one). Also, downloading an arbitrary version, based on
git commit id:

https://codeload.github.com/ElementsProject/lightning/tar.gz/eb2b225c?dummy=/filename_this_is_irrelevant.tar.gz
extracts into "lightning-eb2b225c".

so, support extracting "eb2b225c" from that.

Changelog-None
2020-01-06 12:59:20 +01:00
Rusty Russell fb040bc619 Spec: trivial update to latest.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell d512bcb85f lightningd: automatically move files for existing deployments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell eaa72edbd6 pytest: fix `make pytest`
Travis adds the correct PYTHONPATH, but "make check" doesn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-13 10:59:05 +01:00
Christian Decker b31defdf96 make: Add in-tree pyln-testing to python path
We'll start relying on the in-tree version pretty soon, so we better make
sure we find it.
2019-11-12 21:23:55 +01:00
Christian Decker 81f7978cc8 pyln: Migrate implementation from pylightning to pyln-client
This should not affect any consumer of the API since we just shift the actual
implementation from one side to the other, and keep aliases in place so
scripts don't break.

We also bump the version number from 0.0.7.3 to 0.7.4 which allows us to be in
sync with c-lightning itself, and remove the superfluous `0` in front.
2019-11-12 21:23:55 +01:00
Rusty Russell c617f19a29 Makefile: fix static linking.
And update --enable-static, since it doesn't actually create a static
binary since 65fa1c4f39.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 14:04:43 -05:00
Christian Decker 357322d721 make: Enable the elements flag in the lightningd build
Without this we wouldn't see the elements specific fields and functions from
lightningd.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell 15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
Christian Decker 62dc8dc110 make: Add configuration detection and linking of libpq
If we have the client library for postgres configure will define HAVE_POSTGRES
the same way it already handled libsqlite3 an we start linking against it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell 895e552475 BOLT: update to master with gossip_queries_ex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 8fdfdcec74 BOLT: Update to include BOLT11 features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell e5564173e7 BOLT: update to cover minor changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Jacob Rapoport 6360ffb2ac fix macOS build & update docs 2019-09-16 05:08:42 +00:00
darosior 90fa2ae3fd plugins: load fundchannel by default
Reported by @darwin
2019-09-16 11:59:32 +08:00
Rusty Russell e467e6a5c6 Makefile: fix race if we run `make update-mocks` in a clean tree.
/bin/sh: 1: ccan/ccan/cdump/tools/cdump-enumstr: Text file busy
    make[1]: *** [common/Makefile:81: common/gen_htlc_state_names.h] Error 2
    make[1]: *** Waiting for unfinished jobs....

The fix is to make sure all generated headers are made first, and
thus cdump-enumstr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Christian Decker 7fafd934cf pytest: Remove --duration argument
We're not going to optimize travis any longer so let's no fill logs with
useless measurements.
2019-09-02 16:31:36 +02:00
Rusty Russell 6349222ea2 Spec: Update to latest BOLT, include our first global feature definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 394aef2391 Makefile: fix race with 'make check'
installcheck calls $(MAKE) under the covers, which can race with the
current builds, and we can try installing something which is still
being built:

    collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
    compilation terminated.
    /usr/bin/ld: can not read symbols: File truncated
    /usr/bin/ld: .eh_frame/.stab edit: File truncated
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.interp' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.ABI-tag' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.gnu.build-id' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.hash' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynsym' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynstr' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version_r' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.dyn' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.text' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rodata' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.eh_frame' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data.rel.ro' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynamic' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_json_command' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_hooks' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_type_to_string' not in segment
    Makefile:390: recipe for target 'lightningd/lightningd' failed
    make[1]: *** [lightningd/lightningd] Error 1
    make[1]: Leaving directory '/home/rusty/lightning'
    Makefile:553: recipe for target 'installcheck' failed
    make: *** [installcheck] Error 2

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell 9dfc854d94 tools/Makefile: put all tools/ stuff here.
Generally a Makefile should control make within that directory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell 0fdf22988d configure: absorb python3-mako check.
This is a better place for such checks than the Makefile, and we'll likely
have more in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell de361456d5 Makefile: detect version properly from dirname.
Our reproducibile builds use the dirname to get version, but they have
a v in them (the tools/repro-build.sh script gets this right, so I
copied that).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-19 03:43:13 +00:00
Rusty Russell bc23baf895 Makefile: fix make check so we can run in parallel.
And make travis use -j for `make check`

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-19 01:38:12 +00:00
Rusty Russell dbb15b0045 Makefile: fix occasional race where we fail to compile tools/tests.
This simplifies the dependencies:
1. Objs depend on headers, not other objs.
2. Programs depend on objs.
3. A .o file will generally implicitly depend on the .c file it's built from.
4. If a file has a build line, it's often better to list all deps there.
5. I spotted some missing 'make clean' files.

The particular problem in this case seems to be that make would use
tools/test/gen_test.c before it was ready.  It's probably confused by
the use of recursive make via update-mocks, so explicitly split that
into two stages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-17 04:20:05 +00:00
Rusty Russell 63103db3f3 pytest: disable some v. slow tests when !DEVELOPER.
We've gone over 50 minutes with valgrind, and my measurements show
these are the slowest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-14 05:53:36 +00:00
darosior f98ae2d1d3 Correct version regex in Makefile, add git as dependency 2019-08-09 02:49:45 +00:00
ZmnSCPxj 5dbff077c2 Makefile: Also install manpage for lightningd. 2019-08-09 02:36:59 +00:00
Rusty Russell 979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell c24573b0d2 Makefile: set PYTHONPATH to be absolute when running tests.
My test machine started failing on dynamic plugin tests, unable to
find the lightning module:

  ModuleNotFoundError: No module named 'lightning'

This is because plugins at startup are run from whatever directory
you're in, but on refresh are run from the lightning-dir.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell 913a1a9b59 bolt: update to 8b2cf0054660bece9e1004f42a500c6a1a77efd3
This contains only typo fixes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 8774070a31 bolt: Update to latest bolt, including TLV onion format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 00ca52ea07 bolt: update to aa33af0c4d7ae0180c04ef98e61af49c1f876a36
This introduces the TLV test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 9b88fd4c60 bolt: update to 950b2f5481c2a4b57ef1102e2374543e81c4aa88
Just a simple field renaming which only alters comments,
though I updated variable names too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 4f89d9be86 cppcheck: catch json_list_for_each
cppcheck on bionic doesn't like this it seems.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell af5b3203b5 Makefile: fix parallel `update-mocks` build.
We always rebuild headerversions to examine critical system headers,
however that stomps on parallel builds with:

make[1]: execvp: tools/headerversions: Text file busy

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 05:05:24 +00:00
Rusty Russell 928e3964f2 Makefile: fix `update-mocks`
Broken by 9e5b9a31c9517922052e4464aa7adc661bfe977a: it needs $MAKE set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 05:05:24 +00:00
Rusty Russell 9e5b9a31c9 FreeBSD: fix build.
This makes it build for me on FreeBSD 11:
1. $(MAKE) has to passed through into update-mocks.
2. FreeBSD sed doesn't turn \n into a newline on RHS.
3. Bash and mako dependencies were missing from INSTALL.md

Fixes: #2850
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-29 14:24:29 +02:00
Rusty Russell c5ae29d2b1 check: only check EXPERIMENTAL-tagged BOLT quotes if EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-24 06:35:57 +00:00
lisa neigut 068496298c bolt-gen: rm unused bolt-generator; rename new bolt generator
delete now unused wire-generator and replace it with the newer
version.
2019-07-24 06:31:46 +00:00
lisa neigut 32eaae0cb9 wire-gen: move in-house wire delcarations to new format
tidying things up!
2019-07-24 06:31:46 +00:00
lisa neigut 79f13fa429 bolt-gen: add compilation tests
Add a test for checking that the bolt-gens do the right thing
for a fairly exhaustive test case set (and that it compiles).

Note that this doesn't check that we've got the memory assignment
pieces worked out.

It's got a kind of exotic reliance on the update-mocks in that in
order to depend on as little of the wire/ code as possible (we
only import wire/wire.h), we include an AUTOGENERATE comment
in the test_cases CSV file, and then run update-mocks as part of
the build for that file.
2019-07-24 02:52:53 +00:00
lisa neigut eaa19f23fa bolt: update to latest RFC lightning-rfc:ffeece
Update to latest rfc checkpoint; no material source changes
2019-07-16 06:10:58 +00:00
lisa neigut 81c40da3b5 bolt: update to lightning-rfc:636b9f, change opcode reference format
Updates BOLT reference to match spec; replaces OP_CSV ->
OP_CHECKSEQUENCEVERIFY
2019-07-16 06:10:58 +00:00
lisa neigut 85e325cb1f bolt: update to lightning-rfc:6f6ea63233c new fundamental types
RFC tweaks to the types used to specify message fields
2019-07-16 06:10:58 +00:00
lisa neigut 5c07afac7d bolt: update to BOLT spec changes (extract format + type specifications)
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.

this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
2019-07-16 06:10:58 +00:00
lisa neigut 254ee8a5d3 bolt-spec: bring up to date at 309e86d471faf90c1f6c531701c16887684badb9 2019-07-16 06:10:58 +00:00
Nicola Squartini cbf240e110 makefiles: fix quotation of BINTOPKGLIBEXECDIR macro 2019-07-01 00:27:11 +00:00
lisa neigut 7046d0220c makefiles: move all unit tests under `make check-units`
Isolate unit tests under their own make directive.
2019-06-30 16:41:30 +09:30
Rusty Russell 220449e1cd ccan: import ccan/json_out and ccan/json_escape.
These are generalized from our internal implementations.

The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.

The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell 0d2a4830ed ccan: update to faster and correct crc32c implementation.
I decided to try a faster implementation, only to find our crc32c was
not correct!  Ouch.

I removed the crc32c functions from ccan/crc, and added a new crc32c
module which has the Mark Adler x86-64-optimized variants.

We bump gossip_store version again, since csums have changed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:40:10 +00:00
Rusty Russell 4370ffa978 autoclean: make this a plugin.
No change in behavior.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-22 00:18:29 +02:00
Rusty Russell 6b32b72dee configure: use "-Og" for non-developer builds, add COPTFLAGS variable.
Unfortuntely we get spurious uninitialized variable warnings with
anything but -O3 or no optimization, so set default CWARNFLAGS
appropriately.

MCP bench results without optimization:
   store_load_msec:28509-31001(29206.6+/-9.4e+02)
   vsz_kb:580004-580016(580006+/-4.8)
   store_rewrite_sec:11.640000-12.730000(11.908+/-0.41)
   listnodes_sec:1.790000-1.880000(1.83+/-0.032)
   listchannels_sec:21.180000-21.950000(21.476+/-0.27)
   routing_sec:2.210000-11.160000(7.126+/-3.1)
   peer_write_all_sec:36.270000-41.200000(38.168+/-1.9)

MCP bench with -Og: 22% speedup vs no optimization
   store_load_msec:21963-23645(22841+/-6.6e+02)
   vsz_kb:579916
   store_rewrite_sec:10.080000-10.960000(10.456+/-0.3)
   listnodes_sec:1.280000-1.390000(1.338+/-0.047)
   listchannels_sec:14.770000-16.080000(15.518+/-0.46)
   routing_sec:0.990000-6.660000(3.958+/-2.2)
   peer_write_all_sec:29.950000-32.950000(31.138+/-1)

MCP bench with -O2: 31% speedup vs no optimization
   store_load_msec:20713-22088(21505.6+/-4.8e+02)
   vsz_kb:579928
   store_rewrite_sec:9.570000-11.200000(10.192+/-0.54)
   listnodes_sec:0.960000-1.090000(1.028+/-0.045)
   listchannels_sec:10.400000-11.770000(11.012+/-0.48)
   routing_sec:0.300000-3.140000(1.978+/-1.1)
   peer_write_all_sec:28.980000-30.310000(29.572+/-0.44)

MCP bench with -O3 -flto: 36% speedup vs no optimization
   store_load_msec:19616-20191(19862.6+/-1.9e+02)
   vsz_kb:578452
   store_rewrite_sec:8.980000-9.960000(9.55+/-0.32)
   listnodes_sec:0.920000-1.910000(1.18+/-0.38)
   listchannels_sec:8.960000-9.450000(9.206+/-0.16)
   routing_sec:0.730000-1.850000(1.438+/-0.42)
   peer_write_all_sec:28.090000-29.410000(28.772+/-0.42)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-16 15:39:56 -04:00
Christian Decker 27afc804d5 json-rpc: Include received and resolved time to listforward result
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Rusty Russell dda000fe2d unittest: make sure VALGRIND env var set under valgrind.
In particular, `make unittest/bitcoin/test/run-secret_eq_consttime`
didn't set VALGRIND if it was set via config.vars, so runs for a *long*
time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-04 15:12:08 +02:00
Rusty Russell 6765423393 Documentation: Update to BOLT v1.0.
Mainly typo fixes, but we removed the INCORRECT_PAYMENT_AMOUNT error
altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-01 13:22:05 +02:00
Rusty Russell 8b74bcb0ad Makefile: don't always install in /usr/local: use configure's --prefix
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-22 11:20:51 -08:00
Christian Decker f622ffb087 pylightning: Add plugin dispatch tests to check-python and fix them
These weren't checked by CI yet, and they are really short so I just added
them to the check-python target.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Rusty Russell 38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Michael Schmoock 302a78f4eb fix: add inline exception for recent cppcheck false positive 2019-02-18 01:06:01 +00:00
Rusty Russell e5c80f63d7 lightningd: add code to search strmaps for memleak detection.
Didn't put this in common/memleak because only lightningd currently needs it
(as of next patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Christian Decker 7ac0f53da3 travis: Attempt to setup travis without docker and without sudo
This should speed up testing since it no longer requires virtualization.
2019-02-04 17:10:24 +01:00
Christian Decker 7eaf5b55ff make: Add an option to compile with address sanitizer
Currently only works with `gcc` due to google/sanitizers#1028, so
configure makes sure we warn if clang with ASAN is attempted.

According to [my benchmarks][benchmarks] the performance degradation
is small enough to have it active always.

[benchmarks]: https://github.com/ElementsProject/lightning/issues/2277#issuecomment-455897417

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-03 23:50:18 +00:00
Rusty Russell 0da4054045 Makefile: fix make install to depend on plugins.
Otherwise a straight "make install" gives:
	install: cannot stat 'plugins/pay': No such file or directory
	make: *** [Makefile:482: install-program] Error 1

Fixes: #2288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-31 15:30:56 +01:00
Rusty Russell 4a45caae32 plugins: install pay plugin when 'make install'
Reported-by: ctrlbreak on IRC
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-20 14:57:29 +01:00
Rusty Russell de4043a32a plugin/libplugin: API for C plugins.
Doesn't do logging yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Christian Decker 94eb2620dc bolt: Updated the BOLT specification to the latest version
This is mainly just copying over the copy-editing from the
lightning-rfc repository.

[ Split to just perform changes after the UNKNOWN_PAYMENT_HASH change --RR ]

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Rusty Russell c3e96e058e bolt: Updated the BOLT specification to unify UNKNOWN_PAYMENT_HASH & INCORRECT_PAYMENT_AMOUNT
This is based on Christian's change, but removes all trace of the old codes.

I've proposed another spec change which removes this code altogether:
	https://github.com/lightningnetwork/lightning-rfc/pull/544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Christian Decker 65054ae72e bolt: Updated the BOLT specification to a07dc3df3b4611989e3359f28f96c574f7822850
This is mainly just copying over the copy-editing from the
lightning-rfc repository.

[ Split to just perform changes prior to the UNKNOWN_PAYMENT_HASH change --RR ]

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Rusty Russell 0d5f0d79da build: allow building from source zip file.
Changes both how we construct it and how we deal with not having git.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:18:30 +00:00
William Casarin a01342a66a make: fail build if git isn't present
git is needed to generate version information

Add a sanity check so that the build don't continue with an empty VERSION. This
is useful for sandboxed build where we might have forgot to include git.

Signed-off-by: William Casarin <jb55@jb55.com>
2019-01-10 14:33:22 -08:00
Rusty Russell 90cdc47256 Makefile: add COMPAT_V062 flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
lucash-dev 3aecede214 Removed duplicate error message when running make without running configure first. 2019-01-02 14:09:34 +01:00
arowser c99aa0a21d update pytest check 2018-12-19 21:05:24 +01:00
arowser 65fa1c4f39 only static link sqlite/gmp/zlib 2018-12-18 00:10:23 +00:00
arowser 64677dcbe5 support static link 2018-12-18 00:10:23 +00:00
Rusty Russell 31a375af53 lightningd: add runtime checking for all system-provided libs.
And I tested this by rolling my own libz; make indeed detects
the change and fixes it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-14 05:36:34 +00:00
Rusty Russell 2a90325e80 Makefile: add dependency on external header versions.
There were a few reports that upgrading Ubuntu recently caused issues
because we assert that the sqlite3 library version matches the one we
were built with. 'make' doesn't fix this, because it doesn't know the
external libraries have changed.

Fix this harder, with a helper which updates a file every binary depends
on, which gets relinked every time so we detect link changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-14 05:36:34 +00:00
Rusty Russell 6da213be31 ccan: update to get updated pipecmd.
Note that this changes the order of arguments to pipecmd to match the
documentation, so we fix all the callers!

Also make configure re-run when configurator changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell eb03b33655 plugins: add and install built-in plugin dir, add clear and disable options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-05 01:22:55 +01:00
Christian Decker c3d2caafa2 pytest: Use the pytest-timeout plugin to kill tasks
This will kill a test that was running for 550 seconds, so that we
get a traceback before the travis inactivity timeout of 600 seconds
kicks in. The traceback should show us which test got stuck and where
it got stuck.

Signed-off-by: Christian Decker <@cdecker>
2018-11-26 22:53:37 +00:00
Rusty Russell fac68630b8 ccan: add ccan/bitmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 00:36:31 +00:00
Rusty Russell 40943c9867 Update for latest bolt version: bca814e270dcbee2fea51c0a26ca99efef261f2b
The only change is that the final_incorrect_htlc_amount field is now 64
bit.  Since no implementation yet parses that field, we just updated it
quietly in the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 23:51:05 +00:00
Rusty Russell c236361efd wireaddr: update bolt version, remove 'padding' from addresses.
Nobody used this, so it was removed from the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 23:51:05 +00:00
Rusty Russell 38e6aa66ff python: quieten modern flake8.
After Ubuntu 18.10 upgrade, lots of new flake8 warnings.

$ flake8 --version:
3.5.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 1.6.0) CPython 3.6.7rc1 on Linux

Note it seems that W503 warned about line breaks before binary
operators, and W504 complains about them after.  I prefer W504, so
disable W503.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 16:03:12 +01:00
Rusty Russell acc01e6436 ccan: update.
This was from a different series, so I just cherry-picked it.

It adds ccan/membuf as a depenency of ccan/rbuf, though we don't use
it directly yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 00:53:00 +00:00
lisa neigut b9331e5ac8 gossipd: parse and respect optional `htlc_maximum_msat`
If another channel has set the optional `htlc_maximum_msat` field,
we should correctly parse that field and respect it when drawing up
routes for payments.
2018-10-09 23:22:52 +00:00
Rusty Russell 0b1870908a Makefile: our first db-from-0.6.1 bugfix is coming.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
William Casarin f2488fbe14 make: fix broken error message in bolt-check
It looks like a printf was missing for the bolt-check error message

Signed-off-by: William Casarin <jb55@jb55.com>
2018-10-09 08:39:53 +00:00
lisa neigut 0ecbdf3421 makefile: don't print shellcheck to stdout 2018-10-09 00:08:41 +02:00
Rusty Russell 1b80cb4269 Makefile: don't rebuild all the time
The code to regenerate the local BOLT copy was causing eternal rebuild.

So only build if it's wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:55:51 +02:00
lisa neigut b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
lisa neigut b1f15c2605 BOLT updates: broken link fixes
See a9195a84d0
2018-09-21 00:24:12 +00:00
lisa neigut 73ea6d0038 BOLT 2 updates for fix placment of chain_hash req
See 4b62d26af9
2018-09-21 00:24:12 +00:00
lisa neigut b287f2f007 BOLT 11 human-readable formatting changes 2018-09-21 00:24:12 +00:00
Christian Decker 36eab5de26 pytest: Disable early abort if we run in parallel 2018-09-16 00:05:34 +02:00
Rusty Russell 30f129252d wallet: include Makefile from lightningd/Makefile so that lightning headers defined.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-14 22:11:48 +02:00
Rusty Russell e8a7a7addb wire/Makefile: generate CSVs from specs based on BOLTVERSION.
Not whatever happens to be lying around!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-13 17:38:13 -07:00
lucash-dev b52fb14726 pytest: Fix configure to find pytest when installed using pip3.
Installing pytest through pip3 (at least sometimes) doesn't create a script.
This means calling `which` won't work.
Changed configure so that it can also test if the module is present by calling python/python3.
Change the error message for when pytest can't be found, so that it's clear to the user `configure` must be ran again after installing pytest.
2018-09-05 02:29:06 +00:00
Rusty Russell cefb6925b2 db: save and restore last_sent_commit correctly.
It's an array: we were only saving the single element; if there was more than
one changed HTLC we'd get a bad signature!

The report in #1907 is probably caused by the other side re-requesting
something we considered already finalized; to avoid this particular error,
we should set the field to NULL if there's no last_sent_commit.

I'm increasingly of the opinion we want to just save all the update
packets to the db and blast them out, instead of doing this
second-guessing dance.

Fixes: #1907
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:43:27 +02:00
Rusty Russell a21f025a13 TAGS: add python files.
This is part of our new "Python programmers are people too" policy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-31 12:27:47 +02:00
Rusty Russell 77d3ca3ea3 v0.6.1-rc1
Reordered CHANGELOG.md, and added names and credits for all versions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 01:30:26 +00:00
Rusty Russell a409cc26d9 memleak: reenable CCAN_TAL_DEBUG in DEVELOPER mode.
It no longer has any effect on tal_len(), but it *does* give file and line
of allocations which is much nicer for tracking memory leaks!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
practicalswift a039630da1 lint: Add linter checking for use of discouraged standard library functions 2018-08-02 16:14:21 +09:30
Rusty Russell e3d95f3768 Update common on CCAN_TAL_DEBUG
We don't need this any more: every CCAN object has a valid tal_bytelen().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Mark Beckwith e8edecfbf7 Add lightningd-config(5) to install
The man5 section wasn't being added in the Makefile.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-26 05:41:48 +00:00
Rusty Russell eab83ca79b connectd: new daemon to handle connections.
This is just copying most of gossipd/gossip.c into connectd/connect.c.
It shares the same wire format as gossipd during transition, and changes
are deliberately minimal.

It also has an additional message 'connect_reconnected' which it sends
to the master daemon to tell it to kill a peer; gossipd relied on
closing the gossipfd to do this, but connectd doesn't maintain an fd
with remote peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 82ff891202 Update to latest BOLT version.
And remove the FIXMEs now that the gossip_query extension is merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 17:37:03 +02:00
Rusty Russell 2639b1e9a9 ccan: add UTF-8 module for checking alias fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
Corné Plooy 50efe34e3b ncc: we need to set variables like CC in a different way, to avoid them being overwritten by configure-specified values. 2018-06-26 10:34:17 +00:00
Christian Decker 2c56019b46 make: Specify the commit hash length for the bin-tarball name
Different versions of git have different default lengths. This uses a fixed
length hash prefix.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 23:04:10 +00:00
Rusty Russell 9071b853b6 Makefile: add bin-tarball rule to produce clightning-<version>-<distro>.tar.xz
Places everything under opt/clightning, as per LSB.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 14:02:31 +02:00
Rusty Russell 8c15e619a3 Makefile: update BOLTVERSION
This refers to the commit with query extensions.  Those should be merged
soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell 2180ff0a72 Release notes for 0.6 "I Accidentally The Smart Contract"
Written by Christian Decker wth feedback from Shannon Appelcline.
Special thanks to Fabrice Drouin of ACINQ fame for choosing the release
name!
2018-06-18 12:30:31 +02:00
Corné Plooy 94ef0a7fbf Add 'make ncc' target, for source code analysis. 2018-06-15 16:49:35 +02:00
Rusty Russell 361ed8675a Makefile: we should use COMPAT instead of NO_COMPAT.
That's what configure defines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:38:24 +02:00
Rusty Russell 0fff5038ff Makefile: we don't need to define DEVELOPER explicitly.
config.h does this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:38:24 +02:00
Christian Decker 3cf1581d6d configure: Use the ./configure step to find pytest
Fixes #1544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Rusty Russell c49765553f Makefile: not running configure first is so yesterday:
Make should only run configure if config.var already exists:

    $ make
    ./configure --reconfigure
    ./configure: 65: .: config.vars: not found
    ./configure --reconfigure
    ./configure: 65: .: config.vars: not found
    Makefile:179: recipe for target 'ccan/config.h' failed
    make: *** [ccan/config.h] Error 2
2018-06-08 17:56:01 +02:00
Rusty Russell 4bd27ec722 Makefile: clean should not delete configure results, distclean only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell 72d103d6bb Makefile: import config.vars.
We leave VALGRIND env var as an override for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell d395b58ffa ccan: update to have new configurator.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell 083a2cee07 zlib: add as a requirement.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker 573f1b58c8 make: Redirect unittest output to /dev/null
They are quite chatty and fail very rarely, so let's not have them spam our
logs. Failures should print to stderr anyway.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Jan Sarenik 16b7aab30b Makefile: Add NO_VALGRIND to pytest environment
Previously, by uncommenting NO_VALGRIND in Makefile, the variable
was not propagated and the pytest children processes would still
try to use Valgrind.
2018-05-30 09:05:15 -07:00
Christian Decker 9acb92103c ccan: Add CRC library for gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-29 12:16:00 +00:00
Christian Decker e117d25c1f pytest: Allow test parallelism to be configured via env var
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Rusty Russell d52564dc05 check-source: add check for tal_free(tmpctx).
My older rebased patches use this, and I want to catch it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell bd1c16d284 update-ccan: clean up the two unwanted imports.
We don't import these, though they're technically dependencies.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Christian Decker e604891910 pytest: Do not set test-groups unless explicitly requested
Setting these options could fail the `pytest` target if `pytest-test-groups` is
not installed.

Fixes #1508

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-16 19:42:44 -04:00
Rusty Russell 53c6ceeab9 ccan: import base32 module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell 57115f4914 ccan: update and import rbuf module.
This is a simple helper for dealing with buffered I/O.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Christian Decker 727d115296 pytest: Add py.test fixtures and migrate first example test
This is the first example of the py.test style fixtures which should allow us to
write much cleaner and nicer tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Rusty Russell e44f951b6c Makefile: use -modded instead of -with-local-modifications
Since we include the tail of the version in the default aliasname with
DEVELOPER, we want to see more of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Christian Decker 6139fee31d fixup! make: Disable DEVELOPER by default 2018-05-04 01:01:41 +02:00
Christian Decker 00e75fee0c make: Disable DEVELOPER by default
We had quite a few users running into issues because the `--dev-xyz` options and
`dev-xyz` RPC calls were available. Before a release we should make sure that
the default compilation flags are safe.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-04 01:01:41 +02:00
practicalswift abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
Rusty Russell 5ff0d40fed travis: don't retry failing tests.
Retrying gives spurious failures, since we see transactions from previous
runs.  That makes it near impossible to diagnose the actual problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell a1cf7897c0 shellcheck: restore the check.
Accidentally disabled in 9c3691340f.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-07 23:38:03 +02:00
Rusty Russell 9c3691340f ccan: update to more recent version.
In particular, this gets some MacOS fixes from #1327.
It also includes a major intmap update which fixes corner cases in traversals,
and requires ccan/bitops.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-06 13:34:27 +02:00
practicalswift bc4a099bff Add shell script linting: Check for shellcheck warnings in shell scripts 2018-04-04 15:06:30 +02:00
practicalswift 66fc0a047b Make version output simpler/more user-friendly 2018-04-04 02:32:44 +00:00
practicalswift 760e9f6993 Add cppcheck checking as part of check-source 2018-03-29 23:16:03 +02:00
Rusty Russell a54872063f check-cppcheck: autogen suppressions for all the list_for_each(_safe)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-29 23:16:03 +02:00
practicalswift e078fcefdd Add check-includes.sh: Find redundant includes. Check ifndef:s. 2018-03-25 23:54:21 +00:00
Zhen Zhang d1010673b5 contrib: Rewrite contrib/lightning-pay in Python, support bolt11
fixes #718
2018-03-25 23:19:35 +02:00
Rusty Russell f2adb76d04 test: remove obsolete directory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell 6da0861064 Makefile: add COMPAT_V052 define.
This lets us clearly mark transition features, in a way that they can
be removed after 0.6 is released.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Jan Sarenik 54b8562936 Fix libtool error
This is how the error before the fix looked like on Arch Linux with
GNU libtool 2.4.6.40-6ca5-dirty:

    $ make installcheck
    ...
    make[4]: Entering directory '/home/jasan/lightning/external/libwally-core/src/secp256k1'
     /usr/bin/mkdir -p 'testinstall/home/jasan/lightning/external/libwally-core/..'
     /bin/sh ./libtool   --mode=install /usr/bin/install -c   libsecp256k1.la 'testinstall/home/jasan/lightning/external/libwally-core/..'
    Usage: /home/jasan/lightning/external/libwally-core/src/secp256k1/libtool [OPTION]... [MODE-ARG]...
    Try 'libtool --help' for more information.
    libtool:   error: 'testinstall/home/jasan/lightning/external/libwally-core/..' must be an absolute directory name
    make[4]: *** [Makefile:910: install-libLTLIBRARIES] Error 1
    make[4]: Leaving directory '/home/jasan/lightning/external/libwally-core/src/secp256k1'
    make[3]: *** [Makefile:1253: install-exec-recursive] Error 1
    make[3]: Leaving directory '/home/jasan/lightning/external/libwally-core/src'
    make[2]: *** [Makefile:429: install-exec-recursive] Error 1
    make[2]: Leaving directory '/home/jasan/lightning/external/libwally-core'
    make[1]: *** [external/Makefile:41: external/libwallycore.a] Error 2
    rm external/libwally-core/src/secp256k1/libsecp256k1.la
    make[1]: Leaving directory '/home/jasan/lightning'
    make: *** [Makefile:430: installcheck] Error 2

Then there was linking error which needed to be fixed by correcting
the idea of installing external libraries to a particular directory.
2018-02-28 01:53:33 +00:00
Rusty Russell fba2bdb0ec ccan/config.h: make it depend on the top-level Makefile.
This way if the argument change, it will be regenerated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-26 15:01:02 +01:00
Rusty Russell 229f288eb5 Makefile: allow configurator to use a different CC, by setting CONFIGURATOR_CC
This should solve what @icota wanted in https://github.com/ElementsProject/lightning/pull/1035 in a much cleaner way.

In particular, this allows you to say what configurator should use, independent
of what other compilation should use, and reverts the '-static' which broke
MacOS.

Fixes: #1059
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-26 15:01:02 +01:00
practicalswift fd4a36e777 Add "make check-python" 2018-02-26 12:38:00 +01:00
Christian Decker da062df594 make: If running on travis allow 3 reruns of failed tests
This may mask a few flaky tests, but cuts down on the manual reruns.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-24 10:40:03 +01:00
arowser 73fab9e345 move library path to LDLIBS 2018-02-23 21:49:00 +00:00
Igor Cota d4d1c4acb0 Add PIE option to Makefile to build position independent executables
As of version 5.0 Android requires all dynamically linked executables to support PIE. This allows programs to be loaded at a different addresses, making it harder for attackers to target.
Enable with PIE=1
2018-02-20 03:55:53 +00:00
Igor Cota bd95aba7a5 Pass the compiler variable to the configurator executable when building ccan/config.h
When cross compiling it's important that the resulting config.h reflects the platform we are building for and not the one we are building on.
Otherwise we end up with a config.h that defines headers that are not there on the target platform, wrong endnianness and so on.
The -static flag is there to be able to easily run the configurator test executables on the build machine with qemu-*.
E.g. Without the -static flag the resulting dynamically linked ARM executables complain about the lack of linker (/lib/ld-linux-armhf.so or /system/bin/linker for Android), since these files are not usually available on the build machine building statically avoids this problem and results in a proper config.h for cross compiling.
2018-02-20 03:18:11 +00:00
Rusty Russell 237a65d000 ccan: add tal/link.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell c5cc5c5c1b ccan: import ccan/daemonize.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
practicalswift 3e40a48332 Add lightning spell check 2018-01-26 01:02:15 +00:00
Carl Dong f55adbdc9c build: Add necessary build/link flags for FreeBSD. 2018-01-23 16:10:19 +01:00
Carl Dong 75f4b56766 build: Correctly build and link ptr_valid. 2018-01-23 16:10:19 +01:00
Rusty Russell 2ff9e42b33 lightning-cli: fix amazingly slow getlogs speed.
real	0m0.380s
user	0m0.331s
sys	0m0.028s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-22 18:58:23 +01:00
Rusty Russell c22d2c8dae Makefile: rebuild all objects if Makefile changes.
This doesn't cover external libs in general (which *could* be effected by
CFLAGS), but at least all our own objects are rebuilt.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-22 18:58:23 +01:00
practicalswift a08b6fffc6 Add consistency check for dependencies listed in README.md and doc/INSTALL.md 2018-01-19 10:11:19 +00:00
ZmnSCPxj 72841056f5 Makefile: Install subdaemons into pkglibexecdir instead of bindir.
Fixes: #434
2018-01-09 14:52:48 +01:00
ZmnSCPxj 1946f0c288 Makefile: Add BINTOPKGLIBEXECDIR macro. 2018-01-09 14:52:48 +01:00
ZmnSCPxj bba8a3cab7 Makefile: Add installcheck target to check installation procedure. 2018-01-09 14:52:48 +01:00
ZmnSCPxj e559f2cb2e Implement "install" and "uninstall" targets for Makefile.
Fixes: #383
2018-01-09 14:52:48 +01:00
William Casarin a1d215a46d make: append contrib/pylightning to PYTHONPATH
Append contrib/pylightning instead of overriding PYTHONPATH for tests.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-03 22:25:00 +00:00
Christian Decker ab620c5068 travis: Enable pytest-test-groups to split tests across several runs
Adds two new environment variables TEST_GROUP_COUNT and TEST_GROUP to
split the integration tests into groups and run only a selected group.

This allows us to increase the TEST_GROUP_COUNT and add a new
TEST_GROUP to avoid bumping up against the time limit when running in
valgrind.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-21 22:43:24 +00:00
Rusty Russell 95df553813 lightningd: add dev-memdump command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
practicalswift 61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Rusty Russell af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Rusty Russell 1648eb548a devtools/bolt11-cli: simple helper to decode bolt11.
Can be extended to encode later, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell 88ec8df329 bitcoin/scripts: use htlckey instead of localkey.
Basically a namechange in the argument list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell 44e45348f2 option-data-loss-protect: fix generate-wire.py and update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 15:26:41 +01:00
Christian Decker fc176b6cc0 pytest: Use the py.test utility if available
The py.test unit test runner offers a number of more advanced features
than simply running using unittest.main. In particular it allows us to
capture a tests output and print it if it fails. This change checks
whether we have pytest available and if yes, enables verbose tests and
runs using pytest. This'll give the usual experience (with colors!)
and show us the stdout if a test fails making travis a lot easier to
handle.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-10 02:20:00 +00:00
Rusty Russell 1ae57fb72c Update to latest BOLT.
Nothing changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell 45b92dc2b0 pytest: pass DEVELOPER flag through.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 04:14:33 +00:00
Rusty Russell 3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell 9e869e641a take: turn labels on.
Gives us meaningful errors when there's a take() leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell ebba5f85a2 handshaked: remove.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell 7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell dfa062b06a Makefile: make default correctly build $(ALL_PROGRAMS) and $(ALL_TEST_PROGRAMS).
We can't reference the variables until after they're set, so hence the
two-step.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-11 14:29:20 +02:00
Christian Decker 47a7fac29f make: Add cli/lightning-cli to default make target
Closes #270

Reported-by: @sumBTC
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-11 14:29:20 +02:00
Rusty Russell 8a829ba9cb update-mocks: move mock generation into tools/, fix and generalize.
update-mocks was broken, since it assumed the daemon/ directory.

We now use "make" directly to build the test file and harvest errors,
and are more robust if it simply doesn't compile (ie. fails, but no
linker errors).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 739b163f8b Makefiles: simplify dependencies.
Gather all binaries and objects and make the depend on external
requirements and common headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell f42f34b82d external: new subdirectory for all external libraries and submodules.
You will want to 'make distclean' after this.

I also removed libsecp; we use the one in in libwally anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell c6976cd947 shachain: always build 48 bit version.
No more special Makefile hacks required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell a37c165cb9 common: move some files out of lightningd/
Basically all files shared by different daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8375857116 common: absorb remaining files from daemon/
Also, we split the more sophisticated json_add helpers to avoid pulling in
everything into lightning-cli, and unify the routines to print struct
short_channel_id (it's ':',  not '/' too).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 85ff95e829 common: new directory for any shared objects.
To avoid everything pulling in HTLCs stuff to the opening daemon, we
split the channel and commit_tx routines into initial_channel and
initial_commit_tx (no HTLC support) and move full HTLC supporting versions
into channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 660c051068 opt_bits: remove.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell e7e43cf9a5 remove_dust: remove.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 8088e5cadb protobuf: remove
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 2013906798 find_p2sh_out: remove.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 9ef34d01de irc: remove files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 88bb38f63b daemon/lightningd: remove building, and main files
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell 26ffa8fdec check-bolt: move to tools/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Christian Decker fa6e53bb08 Move short_channel_id primitive into bitcoin
Not really a bitcoin primitive but the place where we keep all the
small stuff currently.
2017-08-10 12:34:58 +09:30
Christian Decker df056e5973 bitcoin: Added chainparams grouping blockchain specific parameters 2017-07-12 11:30:23 +09:30
Rusty Russell 7dce850844 Makefile: wallet objects depend on sodium headers, too.
Noticed a build failure from a fresh git clone.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Christian Decker a53acb7877 make: Explicitly set cc compatibility
This is apparently necessary to build `lightningd` on debian jessie,
which uses an older gcc version.
2017-07-07 10:15:02 +02:00
Rusty Russell 122598dffc Makefile: use -f with pytest, to exit on first failure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-23 09:29:42 +09:30
Christian Decker 80436d06cb wallet: Create and manage a sqlite3 database for the wallet 2017-06-06 09:16:10 +09:30
Rusty Russell f61da7eb64 tests/test_lightningd.py: incorporate everything from old test-basic shell test.
This moves all the non-legacy blackbox testing into python.

Before:
	real	10m18.385s

After:
	real	9m54.877s

Note that this doesn't valgrind the subdaemons: that patch seems to cause
some issues in the python framework which I am still chasing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-29 10:30:10 +02:00
Rusty Russell 0e6667f3b2 pytest: fix dependencies
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-29 10:30:10 +02:00
Christian Decker 67315be673 travis: Simplified build script
The Dockerfile is now stored in contrib and built using the Docker
Hub. This allows us to simply pull in the finished image from the hub
instead of having to build it ourself. Should shave off about 2
minutes from the build time.

I also switched to running the individual build and check steps in
their own containers, but on the same volume, so travis can group the
commands and run them independently.
2017-04-29 10:29:44 +09:30
Christian Decker e603dba14a sphinx: Fixing makefile for test_sphinx and using lightningd version 2017-04-28 13:40:43 +09:30
Rusty Russell 27764b65f9 lightningd: fix shachain to be 48-bits, with hack for legacy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-01 23:59:46 +10:30
Rusty Russell 4bf398c4e7 status: move into lightningd/status.
It's really a lightningd-only thing, and we're about to do surgery on it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-20 07:50:53 +10:30
Rusty Russell c6f568dbde Makefile: fix distclean, clean targets.
libwally's tools/cleanup.sh doesn't actually remove files if it can't
run make, so do that manually.  Also clear some other cruft.

Also, we weren't deleting wire/gen_onion_wire.c in "make clean".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-02 22:51:39 +10:30
Christian Decker 7ac7a6a47e cleanup: Ignoring libwally artifacts and distclean cleans them 2017-02-27 23:14:43 +01:00
Rusty Russell fb93503e6d Makefile: fix libsodium build
eg:
	gcc: error: libsodium.a: No such file or directory

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-21 15:15:28 +10:30
Rusty Russell 1271ac8899 libwally-core: import version 3b025127cbf11912f8b95e7ff3c905d74e8433ce
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-21 15:15:28 +10:30
Rusty Russell c6997f15c7 bitcoin/preimage: struct preimage.
We had a hack for 'struct rval' in protobuf_convert.h; make an
explicit header and put it in bitcoin/preimage.h.  It's not really
bitcoin-specific, but it's better than having bitcoin/script depend on
an external header.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-02 14:48:00 +10:30
Rusty Russell a02f1e10c4 make check: run all the tests.
We can't run them in parallel, but we can at least have 'make check'
run them all.

Developers should be running "make check-source && make check".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-02 14:48:00 +10:30
Christian Decker 05cf3160d4 ccan: Added ccan/intmap 2017-02-02 11:29:47 +10:30
Christian Decker 7cc5e2fc7a pytest: Added pytest target to Makefile 2017-01-23 10:45:36 +10:30
Rusty Russell 69079e51a9 doc: move Makefile parts into doc/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-23 10:38:42 +10:30
Rusty Russell df59cef3af doc: rename waitinvoice manpage to waitanyinvoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-23 10:28:32 +10:30
Rusty Russell 73d07ce441 libsodium: use our local submodule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-11 10:04:26 +10:30
Rusty Russell 015eb072b8 Makefile: split CORE_SRC into CORE_SRC, CORE_TX_SRC and CORE_PROTOBUF_SRC
With the lightningd daemon split, we don't need them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell 8bf33c7839 lightningd/lightningd: daemon for controlling the other daemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:37:51 +10:30
Rusty Russell a0ac5c276e status: API for status reporting.
The API formalizes how daemons should report their statuses back to
the main lightningd.  It's a simple write API, which includes tracing
support (currently it always sends traces, later it could send iff
there's a failure, for example).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:24:20 +10:30
Christian Decker 07fbeee29d coverage: Added `coverage` makefile target
Added coverage files to `clean` and `coverage` target to generate HTML
report of coverage.
2017-01-10 09:27:55 +10:30
Christian Decker bd6b9c377e coverage: Enable compilation with `--coverage`
Setting the environment variable `COVERAGE=1` allows us to turn
coverage measurements on. This produces a number of auxiliary gcno and
gcda files which can then be converted into coverage reports.

I know line coverage and branch coverage is not a perfect metric, but
it might get us some more visibility into where more tests might be
needed.
2017-01-10 09:27:55 +10:30
Rusty Russell a6f98ad02b Makefile: wire objects also depend on CCAN headers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-05 12:11:18 +10:30
Rusty Russell 9df0aa7538 libbase58: add autosubmodule rule, move depends to bitcoin/Makefile
The object file should not be built inside the submodule, as that can
confuse git.

Not everything depends on the libbase58 header (CCAN doesn't), so
move that to the everything-else depends line.

The BITCOIN_SRC etc should also move to bitcoin/Makefile, but that's
a bigger change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-05 12:11:18 +10:30
Lucas Betschart 3e6bb958eb Add libbase58 as git submodule
Pointing to the latest release (0.1.4)
2017-01-05 12:11:18 +10:30
Rusty Russell a08a2105ea generate-wire.py: generalize, move to tools.
We're going to want to use this for inter-daemon comms, so generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell b7789bf065 Makefile: generalize whitespace check.
Spread to individual Makefiles, and include headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 751a0ae5cf check-whitespace: make it quieter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 0861ec33a5 check-source-bolt: generalize.
This way sub-Makefiles can add their own files to check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell b4f495fe99 check-source-bolt: don't try to check out if BOLTVERSION is empty or unset.
This lets you test against your local version, using:

	make check-source BOLTVERSION=

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 6dc7056625 check-bolt: use new BOLTs.
The structure is slightly different, so this requires some fixes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell 05feefbb8a test: move mockup script outside daemon/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell d30b67a3fe test: generalize update-mocks
So we can use it in other directories.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30