Commit Graph

4129 Commits

Author SHA1 Message Date
Rusty Russell 8553edb7e2 common: fix up BOLT 4 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell dba756ef38 common: fix up BOLT 3 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell 0dc406c07a common/bolt11.c: fix up BOLT 11 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell f5d637059d common/cryptomsg.c: fix up bolt 8 references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell 8c15e619a3 Makefile: update BOLTVERSION
This refers to the commit with query extensions.  Those should be merged
soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell a5aaf82ed2 release-notes:
Feedback from @wythe and @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:30:31 +02:00
Rusty Russell 2180ff0a72 Release notes for 0.6 "I Accidentally The Smart Contract"
Written by Christian Decker wth feedback from Shannon Appelcline.
Special thanks to Fabrice Drouin of ACINQ fame for choosing the release
name!
2018-06-18 12:30:31 +02:00
Christian Decker 71ec8193b2
gossip: Avoid integer count overflow in gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-18 12:04:25 +02:00
Corné Plooy 94ef0a7fbf Add 'make ncc' target, for source code analysis. 2018-06-15 16:49:35 +02:00
Rusty Russell f5ebf8e231 gossipd: send correct channel_update in response to query_short_channel_ids
Cut & paste means we sometimes sent NULL:

```
2018-06-15T00:13:51.908Z lightningd(23653): lightning_closingd-03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f chan #436: Gossipd gave us bad send_gossip message 0bc80000
```

Fixes: #1581
Reported-by: @Xian001
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 15:39:30 +02:00
Rusty Russell 60b3f0e376 gossipd: remove oververbose logging when we uncompress short_channel_id array
Reported-by: Xian001 (#1581)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 15:39:30 +02:00
Rusty Russell 4990b6bbcc channeld: don't crash if remote asks for unnecessary retransmit.
We assume they haven't acknowledged it, but they might have (seems
like a bug though).  We could hang up on them, but we'll happily
continue.

lightning_channeld: channeld/channel.c:1702: resend_commitment: Assertion `peer->revocations_received == peer->next_index[REMOTE] - 2' failed.
lightning_channeld: Fatal signal 6
0x559919c35dcd crashdump
        common/daemon.c:37
0x7f76d5be27ef ???
        ???:0
0x7f76d5be277f ???
        ???:0
0x7f76d5be4379 ???
        ???:0
0x7f76d5bdab46 ???
        ???:0
0x7f76d5bdabf1 ???
        ???:0
0x559919c29337 resend_commitment
        channeld/channel.c:1702
0x559919c297cf peer_reconnect
        channeld/channel.c:1853
0x559919c2b3d5 init_channel
        channeld/channel.c:2518
0x559919c2b6f8 main
        channeld/channel.c:2580
0x7f76d5bcd3f0 ???
        ???:0
0x559919c244d9 ???
        ???:0
0xffffffffffffffff ???
        ???:0

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell 9d721ecb99 gossipd: add assertions to try to catch mysterious crash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell a80241ec7a bitcoind: fix spurious memleak reports.
Turn req_running into a pointer to the current bcli structure, which means
the leak detection can find it.

Also suppress leaks in the case where we're only attached to a timer

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell 5c19c55841 gossipd: fix take leak when peer is dying.
In this case, local and remote are *both* NULL; so if someone tries to
send a packet with take(), we need to free it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell a7e6cdb418 gossipd: peer->local->peer_out queue should have lifetime of peer->local.
The current code attaches it to peer, which is a slight leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell e098578731 gossipd: fix leak when we fail to dup fds.
In this case, peer would stay around, but conn would be freed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell f6ff89e596 gossipd: fix use-after-free when we fail to make connection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Christian Decker 4279e5cdbd gossip: Fix "already reaching" issue
I think this is what is causing #1536: getting disconnected causes gossipd to
attempt to reach the peer again, unconditionally setting the flag to tell the
master. At the same time the master also issues a reaching command (which is
allowed since it is its first), but then it clashes on the already set
flag. Setting this flag only when the master actually needs to be told should
fix this.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-15 01:06:42 +00:00
Jan Sarenik 40ea74ba99 configure: Check for version of pytest
Addresses #486 and takes into account @rustyrussell 's
comment about PYTEST defined in the environment which
always overrides without checking the version.
2018-06-15 01:55:44 +02:00
Benoit Verret f8da37fff0 Increase cltv_final
During a meeting earlier this week we agreed with Eclair to temporarily
increase the final CLTV delta in our invoices to establish
compatibility with the already deployed Eclair wallets. They in turn
agreed to remove the enforcement of higher final CLTV deltas, or bump
it locally should it not match their expectations as allowed by
BOLT 11. This has since been implemented in ACINQ/eclair#627.
2018-06-14 15:03:56 +02:00
Rusty Russell 6a73c7eeb1 configure: fix defaults in help.
Both "[ 1 ]" and "[ 0 ]" both evaluate true in shell; we really want
an explicit '1' test.

Before:
$ ./configure --help
...
  --enable/disable-compat (default disable)

After:
$ ./configure --help
...
  --enable/disable-compat (default enable)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:38:24 +02:00
Rusty Russell 361ed8675a Makefile: we should use COMPAT instead of NO_COMPAT.
That's what configure defines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:38:24 +02:00
Rusty Russell 0fff5038ff Makefile: we don't need to define DEVELOPER explicitly.
config.h does this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:38:24 +02:00
Jan Sarenik e5809cf268 external/libwally-core: Update to master after release_0.6.1
The extra patch after release_0.6.1 fixes another set of
FORTIFY warnings.

See https://github.com/ElementsProject/libwally-core/pull/36
2018-06-14 14:26:25 +02:00
Rusty Russell 46aaf08047 configure: add hack to rename ccan's sha256 routines to avoid libwally clash
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 14:26:25 +02:00
Rusty Russell 20e5bc938e wireaddr: zero entire sockaddr structure.
2018-06-14T01:09:03.495Z lightningd(23766): HSM: created new hsm_secret file
==23785== Syscall param socketcall.bind(my_addr.sin6_flowinfo) points to uninitialised byte(s)
==23785==    at 0x5731877: bind (syscall-template.S:78)
==23785==    by 0x11767C: make_listen_fd (gossip.c:2405)
==23785==    by 0x117DA2: handle_wireaddr_listen (gossip.c:2558)
==23785==    by 0x1183B7: setup_listeners (gossip.c:2653)
==23785==    by 0x118E86: gossip_activate (gossip.c:2871)
==23785==    by 0x11AC42: recv_req (gossip.c:3543)
==23785==    by 0x143FF1: next_plan (io.c:59)
==23785==    by 0x144AEE: do_plan (io.c:387)
==23785==    by 0x144B2C: io_ready (io.c:397)
==23785==    by 0x146719: io_loop (poll.c:310)
==23785==    by 0x11B0B0: main (gossip.c:3687)
==23785==  Address 0x1ffeffffa4 is on thread 1's stack
==23785==  in frame #2, created by handle_wireaddr_listen (gossip.c:2539)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 13:08:10 +02:00
Rusty Russell d9a672ab02 listinvoice: speed up single-invoice case.
satoshis.place was slowing to a crawl, c-lightning was unresponsive.
Logs revealed charged doing many, many listinvoice <label> RPCs.

We were iterating the entire db every time: stop that!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-14 12:46:42 +02:00
Christian Decker 2848103841 opts: Bump max_fee_multiplier to 10x
The fee range can sometimes cause channels to be closed when the estimator
jumps. This has been the case a few times in the last months, and causes a
number of channels to be closed, and issue reports to be filed.

Increasing this from 5x to 10x should get rid of 84%+ of these
closures (measured based on 1h windows over the last 6 months and assuming
worst case situations).

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Christian Decker 583c1c50f8 pytest: Explicitly state the max fee multiplier we assumed in tests
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Christian Decker 0b427b4c3c opts: Add the max_fee_multiplier to specify acceptable fee ranges
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Christian Decker 4dca6daf34 opts: Bump locktime_max so we don't disagree as much with lnd
I still believe that 2 weeks is way too much, but we were promised that these
defaults would be slowly reduced to saner values as the stability increases.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Rusty Russell 8183590cdb pytest: test_onchaind_replay needs to wait for tx broadcast.
Waiting for 'Disabling channel' is not enough, since it's async to the
actual tx broadcast: I caught a case where the unilateral close wasn't
in the block, and so failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-13 16:22:23 +02:00
Rusty Russell 965c20caae tests: reenable developer tests.
72d103d6bb deprecated DEVELOPER env var
in favor of config.vars, but didn't update test_closing.py or test_gossip.py.

5d0a54b7f0 then removed the explicit
DEVELOPER= setting from Travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-13 16:22:23 +02:00
Rusty Russell 02b55e35b8 daemon: print version in crash backtrace.
Sometimes people only send us the crash, not the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-12 23:18:53 +02:00
Rusty Russell 56046d470b configure: take PYTEST from env var or cmdline.
We should only autodetect if it's not set manually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-09 13:39:27 +02:00
Christian Decker 3cf1581d6d configure: Use the ./configure step to find pytest
Fixes #1544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker 135bd962f7 pytest: Mark test_closing_different_fees as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker 37327d31de topo: Remove obsolete FIXME marker
This was addressed in bdb87aa994

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker 985af483cf gossip: Wrap insert_broadcast and gossip_store_add in persistent_broadcast
They should sync up nicely otherwise we may be overestimating the stale rate.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 6632f44133 gossip: Disable gossip_store temporarily while replaying messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 2b5e1ee65f gossip: Enable the consistency check only when really pedantic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 8a5bebed59 gossip: Disable future compactions if we fail a compaction
A failed compaction shouldn't be deadly, but we should also not attempt to do
one on every gossip message after the first one fails.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 74a1cbd877 gossip: Implement gossip_store compaction
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker b9a2400a5f gossip: Simplify message handling in gossip_store
`gossip_store_add` is the entry point for messages from the network, so it
should do the bookkeeping and disable on failures. `gossip_store_append` is the
shared function that wraps messages and writes it to the given file. This is
shared between the from network path and the compaction path, so we don't
directly use the `gossip_store` instance, but `fd`s.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 60efa314fe gossip: Separate writing to gossip_store fd from append
We write both when coming from outside, as well as when compacting, so we
extract the write functionality to use it in both cases.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker e6ab594904 gossip: Have gossip_store annotate gossip messages
This makes the exposed interface much smaller, cleaner and will allow us to just
replay gossip messages from the broadcast.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 0546ca446d gossip: Pass routing_state to the gossip_store
We'll need it later to annotate the raw gossip messages, e.g., the capacity of a
channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker eaba5a249a gossip: Introduce bookkeeping into gossip_store for rewrite
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker 552ddb8dfd gossip: Pass broadcast_state to gossip_store
We'll be sourcing messages from this `broadcast_state` when rewriting the
`gossip_store`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00