Commit Graph

435 Commits

Author SHA1 Message Date
ZmnSCPxj jxPCSnmZ 6fadd5aea2 Makefile: Remove gen_version.h from ALL_GEN_HEADERS.
This prevents recompiling everything when you are changing just a doc, or
touching only one file among hundreds of sources, just because the
`gen_version.h` is changed, especially since only one source actually
depends on that header.
2020-08-07 10:41:37 +09:30
ZmnSCPxj jxPCSnmZ 4ca2e49812 lightningd/plugin.c: Make builtin plugins important.
Changelog-Changed: plugin: Builtin plugins are now marked as important, and if they crash, will cause C-lightning to stop as well.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ 4a89819cdd plugins/Makefile: Change plugins build procedure.
Changelog-None
2020-07-07 21:32:31 +02:00
Christian Decker 6fbfe24e3e paymod: Add a COMPAT_090 flag
Since we end up consolidating some of the return values for `pay` and
`paystatus` and change the public interface we need to add the compatibility
flag and guard the switchover behind it.
2020-07-02 11:54:12 +02:00
ZmnSCPxj jxPCSnmZ 5d720536e2 Makefile: Install `tools/hsmtool` as `lightning-hsmtool`.
Changelog-Added: We now install `lightning-hsmtool` for your `hsm_secret` needs.

See: https://github.com/ElementsProject/lightning/issues/3717#issuecomment-644844594

It seems reasonable to add this to the standard install, and to document it properly as well, hopefully we can fill in the documentation better later on.
2020-07-01 09:24:03 +09:30
Rusty Russell 3882e8bdf7 pyln.spec.bolt*: make these separate packages, so versions can change indep.
They're almost entirely autogenerated, and we use symlinks into the
top directory to reduce replication.

They can't be under pyln.spec.message, because a package can't also
be a namespace.

We also add fulltext and desc fields, and exclude our "gen" files from
flake8, since the spec quotes contain weird whitespace.

Changelog-Added: Python: pyln.spec.bolt{1,2,4,7} packages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell 93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei 47c701cbe6 make: have `update-mocks` actually be quiet
Shhhh
2020-05-19 20:25:23 +09:30
Rusty Russell 7ed3e72b88 Makefile: don't print out what we're doing if --quiet/-s
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-19 20:25:23 +09:30
niftynei 66f59659a7 setup: add setup to make checks
- we've moved tmpctx management to setup.c from daemon.c, so we update
the `check-tmpctx`
- `common_setup(char *)` is now a valid analog for `setup_locale`, so we
check for either in check-setup_locale
2020-05-19 13:35:42 +09:30
William Casarin d6ede8db1b build: cleanup compiler output
Hide CFLAGS and LDFLAGS line noise each time an object file is compiled
or linked.

Also add a `make show-flags` command for displaying CC, LD, CFLAGS and
LDFLAG information. This is shown at the start of each build.

Use `V=1 make` to restore original output

make

  CC: gcc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall [..]
  LD: gcc   -Og    -Lexternal -lwallycore -lsecp256k1 -ljsmn  [..]
  ...
  cc wallet/test/run-db.c
  cc lightningd/test/run-jsonrpc.c
  cc lightningd/test/run-invoice-select-inchan.c
  cc lightningd/test/run-log-pruning.c
  cc lightningd/test/run-find_my_abspath.c
  cc cli/test/run-large-input.c
  cc cli/test/run-remove-hint.c
  ld lightningd/lightning_hsmd
  ld lightningd/lightning_gossipd
  ld lightningd/lightning_openingd
  ld lightningd/lightning_channeld
  ld lightningd/lightning_closingd
  ...

Signed-off-by: William Casarin <jb55@jb55.com>
Changelog-Changed: build: default compile output is prettier and much less verbose
2020-05-14 17:44:01 +02:00
Rusty Russell 1f2740efe8 Makefile: COMPAT_V082.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Christian Decker f1633de6e8 plugin: Add the keysend plugin to the list of installable plugins
Suggested-by: Matt Whitlock <@whitslack>
Signed-off-by: Christian Decker <@cdecker>
2020-04-20 07:29:07 +09:30
Christian Decker b68066e8e8 python: Consolidate requirements.txt files in a single place
We had them split according the separate use-cases:

 - testing
 - doc-gen
 - wire-gen

But that was causing new contributors to miss some dependencies when they
first got hacking. So this consolidates all of our own dependencies in a root
requirements.txt, with the notable exception of `pyln-client`, `pyln-testing`
and `pyln-proto` which are distributed as PyPI modules and therefore have
their own dependencies that need to be tracked in the module root.

Closes #3518
2020-04-09 15:14:06 +02:00
Rusty Russell d881a4bd66 BOLT: update to latest version.
This is all typo/clarity fixes, no substantive changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Christian Decker 3e3b05e1b2 pyln: Migrate remaining uses of the deprecated pylightning module
`pylightning` is not much more than an alias for `pyln-client`, so this
removes the need to install that as well just to run the tests.
2020-03-24 09:52:33 +10:30
Christian Decker 15ca3b615f make: Add compat flag for v0.8.1 2020-03-10 11:42:58 +10:30
Vasil Dimov 820f1b2f9d build: send cmp's stderr to /dev/null also
It was probably a typo to have 2>&2

Changelog-None
2020-03-05 14:58:57 -06:00
Vasil Dimov 79ba4866ec build: unbreak if CFLAGS is defined in make's env
Don't let make pollute subprojects' environment with our own `CFLAGS`,
which are quite strict because that breaks at least libwally-core:

```sh
$ ./configure ...
$ CFLAGS=whatever_this_is_irrelevant make
...
cd external/libwally-core-build && ../libwally-core/configure ...
...
  CFLAGS                  = -DBINTOPKGLIBEXECDIR="\"../libexec/c-lightning\"" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -std=gnu11 -g -fstack-protector  -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/libbacktrace-build -I . -I/usr/local/include   -DCCAN_TAKE_DEBUG=1 -DCCAN_TAL_DEBUG=1 -DCCAN_JSON_OUT_DEBUG=1 -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS   -DBUILD_ELEMENTS=1 -W -std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wno-unused-function -Wno-long-long -Wno-overlength-strings -fvisibility=hidden -O3
...
In file included from ../../libwally-core/src/base58.c:4:
../../libwally-core/src/ccan/ccan/endian/endian.h:71:24: error: unused function 'bswap_16'
      [-Werror,-Wunused-function]
static inline uint16_t bswap_16(uint16_t val)
                       ^

```

If `CFLAGS` is set in its environment, then `make` would export our own
`CFLAGS` to any subprocesses it starts, which means subprojects would
inherit our `CFLAGS="-Wall -Werror"` in their environments.

GNU Make's documentation:
https://www.gnu.org/software/make/manual/html_node/Variables_002fRecursion.html#Variables_002fRecursion
> make exports a variable only if it is either defined in the environment initially...

Example:
```make
A = x
default:
	echo $$A
```

then:

```sh
$ make # prints nothing, A is not exported to the subprocess
$ A=y make # prints "x", our A=x is exported to the subprocess
```

Changelog-None
2020-02-22 08:49:57 +10:30
Rusty Russell d478dc0e59 wallet: fix null column access for pre-0.7.3 dbs.
Added in d901304120, this column is null in old dbs like mine:

2020-02-15T00:08:41.444Z **BROKEN** database: Accessing a null column 12 in query SELECT  id, channel_htlc_id, msatoshi, cltv_expiry, hstate, payment_hash, payment_key, routing_onion, failuremsg, malformed_onion, origin_htlc, shared_secret, received_time FROM channel_htlcs WHERE direction= ? AND channel_id= ? AND hstate != ?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-18 10:22:58 +10:30
darosior 70a79e3998 plugins/bcli: a new plugin for gathering Bitcoin data
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
  whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
  `getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
  `getfilteredblock` in `lightningd/bitcoind`.
2020-02-12 11:45:07 +10:30
Christian Decker 966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
darosior da476848d1 pytest: add a C testing plugin to test libplugin
We mark the test as xfail() as it exposes that libplugin's PLUGIN_RESTARTABLE
was not taken into account !
2020-02-10 09:49:15 +10:30
Niklas Claesson f9b3b96a63 external: Reorganize to support cross-compilation
On CI it is nice to cache the external dependencies. However if we
always compile them in the same folder we cannot cache for multiple
different architectures. After this commit native compile targets will
still live in `external` but cross compiled versions will live in
`external/<arch>`.
2020-02-04 11:16:02 +10:30
Vasil Dimov 1015e3774a build: avoid repetition of `LC_ALL=C sort`
Put `LC_ALL=C sort` in a variable and use it everywhere. It was
forgotten in one place.

Without `LC_ALL=C` it would order `a.b` before `a-c` even though
`. (0x2E)` > `- (0x2D)`.

Changelog-None
2020-02-03 15:38:11 +00:00
Christian Decker 24219b38f9 pyln: Add unit tests from pyln-* to `make check-python` 2020-02-01 16:50:58 +01:00
Rusty Russell f373cd45be Makefile: update BOLTVERSION to latest master.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell 6450a7e315 lightningd: update spec and remove allowance for final node to receive a fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Now correctly reject "fees" paid when we're the final hop (lightning-rfc#711)
2020-01-31 06:07:56 +00:00
Rusty Russell ff1e1dd487 Makefile: update BOLTVERSION to flatten features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell a0a2c257be Makefile: add a COMPAT_V080 flag.
We're going to need this in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Vasil Dimov 8adbd4d31e doc: also build docs when building everything else
Before this change `make` would build everything but the docs.
They would be built during `make install` which is unexpected.

Add a dependency to `doc-all` to the `default` target so that docs are
build together with the rest of the programs. Leave the `install-data`
as is, to still build the docs if they are not built yet during install.

Changelog-None
2020-01-15 12:34:57 +01:00
Vasil Dimov 899f5dee53 build: fix version extraction from non-git directory
https://github.com/ElementsProject/lightning/releases/download/v0.8.0/clightning-v0.8.0.zip
extracts into "clightning-v0.8.0".

https://github.com/ElementsProject/lightning/archive/v0.8.0.tar.gz
extracts into "lightning-0.8.0".

Tweak the regular expression in the Makefile to support both (extract
0.8.0 from either one). Also, downloading an arbitrary version, based on
git commit id:

https://codeload.github.com/ElementsProject/lightning/tar.gz/eb2b225c?dummy=/filename_this_is_irrelevant.tar.gz
extracts into "lightning-eb2b225c".

so, support extracting "eb2b225c" from that.

Changelog-None
2020-01-06 12:59:20 +01:00
Rusty Russell fb040bc619 Spec: trivial update to latest.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell d512bcb85f lightningd: automatically move files for existing deployments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell eaa72edbd6 pytest: fix `make pytest`
Travis adds the correct PYTHONPATH, but "make check" doesn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-13 10:59:05 +01:00
Christian Decker b31defdf96 make: Add in-tree pyln-testing to python path
We'll start relying on the in-tree version pretty soon, so we better make
sure we find it.
2019-11-12 21:23:55 +01:00
Christian Decker 81f7978cc8 pyln: Migrate implementation from pylightning to pyln-client
This should not affect any consumer of the API since we just shift the actual
implementation from one side to the other, and keep aliases in place so
scripts don't break.

We also bump the version number from 0.0.7.3 to 0.7.4 which allows us to be in
sync with c-lightning itself, and remove the superfluous `0` in front.
2019-11-12 21:23:55 +01:00
Rusty Russell c617f19a29 Makefile: fix static linking.
And update --enable-static, since it doesn't actually create a static
binary since 65fa1c4f39.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 14:04:43 -05:00
Christian Decker 357322d721 make: Enable the elements flag in the lightningd build
Without this we wouldn't see the elements specific fields and functions from
lightningd.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell 15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
Christian Decker 62dc8dc110 make: Add configuration detection and linking of libpq
If we have the client library for postgres configure will define HAVE_POSTGRES
the same way it already handled libsqlite3 an we start linking against it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell 895e552475 BOLT: update to master with gossip_queries_ex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 8fdfdcec74 BOLT: Update to include BOLT11 features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell e5564173e7 BOLT: update to cover minor changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Jacob Rapoport 6360ffb2ac fix macOS build & update docs 2019-09-16 05:08:42 +00:00
darosior 90fa2ae3fd plugins: load fundchannel by default
Reported by @darwin
2019-09-16 11:59:32 +08:00
Rusty Russell e467e6a5c6 Makefile: fix race if we run `make update-mocks` in a clean tree.
/bin/sh: 1: ccan/ccan/cdump/tools/cdump-enumstr: Text file busy
    make[1]: *** [common/Makefile:81: common/gen_htlc_state_names.h] Error 2
    make[1]: *** Waiting for unfinished jobs....

The fix is to make sure all generated headers are made first, and
thus cdump-enumstr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Christian Decker 7fafd934cf pytest: Remove --duration argument
We're not going to optimize travis any longer so let's no fill logs with
useless measurements.
2019-09-02 16:31:36 +02:00
Rusty Russell 6349222ea2 Spec: Update to latest BOLT, include our first global feature definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell 394aef2391 Makefile: fix race with 'make check'
installcheck calls $(MAKE) under the covers, which can race with the
current builds, and we can try installing something which is still
being built:

    collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
    compilation terminated.
    /usr/bin/ld: can not read symbols: File truncated
    /usr/bin/ld: .eh_frame/.stab edit: File truncated
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.interp' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.ABI-tag' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.gnu.build-id' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.hash' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynsym' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynstr' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version_r' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.dyn' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.text' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rodata' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.eh_frame' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data.rel.ro' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynamic' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_json_command' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_hooks' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_type_to_string' not in segment
    Makefile:390: recipe for target 'lightningd/lightningd' failed
    make[1]: *** [lightningd/lightningd] Error 1
    make[1]: Leaving directory '/home/rusty/lightning'
    Makefile:553: recipe for target 'installcheck' failed
    make: *** [installcheck] Error 2

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00