Commit Graph

6859 Commits

Author SHA1 Message Date
Christian Decker 640e64cb73 db: Switch statement lookup to use the original query instead
Using a generated identifier with filename and line proved to be brittle since
compilers assign the __LINE__ macro differently on multi-line macro
invocations.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 3df8ccd3e0 db: Reorder migrations to reflect their relationship
This is dangerous but needed since postgres is not as forgiving about
unsatisfied foreign key constraints even while in a DB transaction.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker e2f7c86271 db: Move remainder of the sqlite3 into the apropriate file
We used to do some of the setup work in db.c, which is now free of any
sqlite3-specific code. In addition we also switch over to fully qualified DSNs
to specify the location of the wallet.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker bf613fa48a postgres: Add postgres statement rewriting support
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 5953a5051c cli: Add command line option to specify the wallet location
Will be demuxed into starting the selected DB backend in one of the next
commits. Defaults to the old database location.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 96a22b4003 pytest: Add db_provider and db instances for configurable backends
We will soon have a postgres backend as well, so we need a way to control the
postgres process and to provision DBs to the nodes. The two interfaces are the
dsn that we pass to the node, and the python query interface needed to query
from tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker 62dc8dc110 make: Add configuration detection and linking of libpq
If we have the client library for postgres configure will define HAVE_POSTGRES
the same way it already handled libsqlite3 an we start linking against it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell 27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 895e552475 BOLT: update to master with gossip_queries_ex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 6eb838ddda wire: update csv to current bolt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 8fdfdcec74 BOLT: Update to include BOLT11 features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell e5564173e7 BOLT: update to cover minor changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell 0da074cad3 BOLT: update bolt quotes now that TLVs are not experimental.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell ccbc46a476 check-bolt: escape { and }.
These are special in extended regexs, and so we fail to match once we fix
the BOLT comment in common/test/run-bigsize.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
fiatjaf 1d5bbc874a format JSON snippets with jq. 2019-09-21 19:41:43 +02:00
Rusty Russell 6a8d18c7e3 gossipd: naming cleanups.
Suggested-by: @cdecker.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 39c9dcbafc ratelimit: adjust based on --dev-fast-gossip, test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 8139164aa0 gossipd: disallow far future (+1 day) or far past (2 weeks) timestamps.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 76860683aa gossipd: only allow one channel_update per direction per day.
And similar for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell a92ead48bf gossipd: ignore redundant channel_update and node_announcement.
If you send a message which simply changes timestamp and signature, we
drop it.  You shouldn't be doing that, and the door to ignoring them
was opened by by option_gossip_query_ex, which would allow clients to
ignore updates with the same checksum.

This is more aggressive at reducing spam messages, but we allow refreshes
(to be conservative, we allow them even when 1/2 of the way through the
refresh period).

I dropped the now-unnecessary sleep from test_gossip_pruning, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 46e0f1efcc gossipd: refresh every 13 days, not every 7.
One day is plenty of time to propagate the update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 06afb408d8 gossipd: bias lower bit of timestamp to ensure alternation.
This is useful for various "partial timestamp" forms of propagation
in future, esp. minisketch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 21a6d502db gossipd: move gossip message generation into its own file.
gossipd.c is doing too many things: this is a start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 0bab2580fc gossipd: clean up local channel updates.
Make update_local_channel use a timer if it's too soon to make another
update.

1. Implement cupdate_different() which compares two updates.
2. make update_local_channel() take a single arg for timer usage.
3. Set timestamp of non-disable update back 5 minutes, so we can
   always generate a disable update if we need to.
4. Make update_local_channel() itself do the "unchanged update" suppression.
   gossipd: clean up local channel updates.
5. Keep pointer to the current timer so we override any old updates with
   a new one, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell e1c431d278 gossipd: use local_chan_map more.
We can look up local channels directly now, which offers simplifcations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 27d9b75456 gossipd: add shadow structure for local chans.
Normally we'd put a pointer into struct half_chan for local
information, but it would be NULL on 99.99% of nodes.  Instead, keep a
separate hash table.

This immediately subsumes the previous "map of local-disabled
channels", and will be enhanced further.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 70c4ac6d74 gossipd: suppress our own too-close node_announcement messages.
Never make them less than gossip_min_interval apart.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 178baeba6c gossipd: get gossip_min_interval from lightningd.
Default is 5 x gossip interval == 5 minutes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 4cfd0524eb gossipd: simplify duplicate node_announcement check.
Write helpers to split it into non-timestamp, non-signature parts,
and simply compare those.  We extract a helper to do channel_update, too.

This is more generic than our previous approach, and simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 5ddd7866e4 gossipd: make create_node_announcement const-correct.
sig is only non-const so we can override if NULL, but talz helps
us here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell 9578fb4035 lightningd: fix crash in cancel_channel
This path is not atomic, so we can't assume channel stays around.  It
could be the peer closes, it could be we get a parallel
fund_channel_cancel.  test_funding_cancel_race caused this crash:

    FATAL SIGNAL 6 (version 5f0a18e)
    backtrace: common/daemon.c:45 (send_backtrace) 0x55c7c373a429
    backtrace: common/daemon.c:53 (crashdump) 0x55c7c373a479
    backtrace: (null):0 ((null)) 0x7f88ee6ddf5f
    backtrace: (null):0 ((null)) 0x7f88ee6dded7
    backtrace: (null):0 ((null)) 0x7f88ee6bf534
    backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x55c7c379427c
    backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x55c7c3794444
    backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x55c7c3794483
    backtrace: ccan/ccan/tal/tal.c:186 (to_tal_hdr_or_null) 0x55c7c3794504
    backtrace: ccan/ccan/tal/tal.c:421 (tal_alloc_) 0x55c7c3794c10
    backtrace: ccan/ccan/tal/tal.c:466 (tal_alloc_arr_) 0x55c7c3794ded
    backtrace: ccan/ccan/tal/str/str.c:91 (tal_vfmt_) 0x55c7c3793560
    backtrace: common/wire_error.c:22 (towire_errorfmtv) 0x55c7c3747f7b
    backtrace: common/wire_error.c:39 (towire_errorfmt) 0x55c7c37480a1
    backtrace: lightningd/channel_control.c:635 (process_check_funding_broadcast) 0x55c7c37015bb
    backtrace: lightningd/bitcoind.c:558 (process_gettxout) 0x55c7c36f8e75
    backtrace: lightningd/bitcoind.c:227 (bcli_finished) 0x55c7c36f8090
    backtrace: ccan/ccan/io/poll.c:244 (destroy_conn) 0x55c7c37869fe
    backtrace: ccan/ccan/io/poll.c:250 (destroy_conn_close_fd) 0x55c7c3786a1e
    backtrace: ccan/ccan/tal/tal.c:235 (notify) 0x55c7c3794629
    backtrace: ccan/ccan/tal/tal.c:397 (del_tree) 0x55c7c3794b18
    backtrace: ccan/ccan/tal/tal.c:481 (tal_free) 0x55c7c3794ea4
    backtrace: ccan/ccan/io/io.c:450 (io_close) 0x55c7c378521d
    backtrace: ccan/ccan/io/poll.c:449 (io_loop) 0x55c7c3787139
    backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55c7c370b26d
    backtrace: lightningd/lightningd.c:837 (main) 0x55c7c3711661
    backtrace: (null):0 ((null)) 0x7f88ee6c0b6a
    backtrace: (null):0 ((null)) 0x55c7c36f70b9
    backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 18:41:12 +08:00
Rusty Russell 794bebcc81 lightningd: add find_channel_by_id() helper, use it.
This is overkill for the moment, but we'll need it in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 18:41:12 +08:00
trueptolemy 2d1a153975 plugins: Return `command_param_failed()` if `param()` fail 2019-09-19 01:07:11 +00:00
Rusty Russell fe66b53fb9 jsonrpc: add more fields to listfunds.
This avoids having to correlate with listpeers for the most pertinent
information.

This API predates plugins, otherwise we'd have listutxos and listpeers
and this would simply combine them appropriately.  Still, it exists so
there's little reason not to make it more friendly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 01:03:09 +00:00
Christian Decker ac1c894255 db: Get old sqlite3_trace based db_changes working again
Seems I accidentally broke the fallback mechanism while working on the DB
abstraction.
2019-09-18 18:35:11 +02:00
trueptolemy c5f4c9cefb hsmd: Fix a memory leak in `status.c` 2019-09-17 21:06:12 +02:00
trueptolemy c737fa6b91 pay: Fix logic of the intereface `find_worst_channel` 2019-09-17 21:06:12 +02:00
trueptolemy bdbfbf7e55 fundchannel: Remove the link of `lightningd/json.h` 2019-09-17 21:05:51 +02:00
trueptolemy b4c6a5c0f0 plugins/.gitignore: Add `fundchannel` entry 2019-09-17 21:05:51 +02:00
darosior 697b501132 doc: place lightningd manpages first in index.rst 2019-09-16 13:16:08 +02:00
lisa neigut aef5b30b81 wire: update how experimental csv patches are managed
We used to append new patches to a single file. This caused
some problems and is a lot harder to cleanup later.

This patch moves the experimental patches to their own, individual
patch files, that are named for the current BOLTVERSION, which they're
taken from.

Also moves the current patchfile over to a 'gossipqueries' one,
as it already exists.
2019-09-16 05:09:15 +00:00
Jacob Rapoport 6360ffb2ac fix macOS build & update docs 2019-09-16 05:08:42 +00:00
darosior 48fde4f0a5 wireaddr: correct the onion case in wireaddr_from_hostname
And, this time, add tests !
2019-09-16 05:05:24 +00:00
darosior fad133ed2e wireaddr: pass an array to wireaddr_from_hostname
'wireaddr_from_hostname' now expects a tal array.
2019-09-16 05:05:24 +00:00
trueptolemy ece0ec8bb9 CHANGELOG: Add the entry about `pay` can exclude error nodes 2019-09-16 12:22:06 +08:00
trueptolemy d4104c15aa pytest: Test `pay` can exclude error node 2019-09-16 12:22:06 +08:00
trueptolemy a3a0f651e3 pytest: Let `fail_htlcs.py` plugin generates `TEMPORARY_NODE_FAILURE` error 2019-09-16 12:22:06 +08:00
trueptolemy 22e8d242fc pay: Exclude nodes if the failcode of `sendpay` has the NODE bit set 2019-09-16 12:22:06 +08:00
trueptolemy 36852c36e6 CHANGELOG: Add the entry about `getroute` supports excluding nodes 2019-09-16 12:22:06 +08:00