Commit Graph

4252 Commits

Author SHA1 Message Date
Christian Decker 60e0eec967 wallet: Store payment description in the database 2018-07-30 03:04:45 +00:00
Christian Decker ff5ca37f3c wallet: Add descriptions to invoice and payment structs 2018-07-30 03:04:45 +00:00
Christian Decker 7119a02932 db: Add description fields to invoices and payments 2018-07-30 03:04:45 +00:00
Christian Decker 55d450ff00 pay: Exempt a fee from the maxfeepercent rule it is still tiny
Several users have noticed that they cannot pay satoshis.place or similar places
that have tiny payment amounts if they are not directly connected. This is due
to the forwarding fee dominating the transferred amount.

This commit adds a new option, exempting tiny fees (up to 5 satoshis by default)
from having to pass the maxfeepercent flag. While we could have told users to
tweak maxfeepercent I think it is usefull to have a default exemption.

[Squashed --RR]
2018-07-30 01:05:43 +00:00
Simon Vrouwe a235c1fa67 add more detail to log messages about feerate estimates 2018-07-29 20:12:21 +02:00
Mark Beckwith 1851ebbc56 programer -> developer
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith 428da65fb1 renamed error code
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith 381c32af75 Remove sigaction from run-param
Developer errors result in command_fail being called
just like other errors.  The bad_programmer() Test is now updated
and passing.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith 2ca1786931 Added check_fail function to run-param
Just a utility function to check that command_fail was called.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Mark Beckwith b7203b0c0c Remove Developer Asserts from param.c
They now just call command_fail() and cause param() to return false.
Temporarily disabled all the run-param.c tests that redirect
asserts so CI would still pass.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-29 16:27:38 +02:00
Rusty Russell 65103ac426 pytest: give more time for test_closing_torture.
The logs in various Travis failures show that it takes 20 seconds just for
closingd to read the init message.  As a result, the close times out (default
is 30 seconds).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-29 16:23:27 +02:00
Rusty Russell 9e14d6cf04 Remove all JSON commands and fields deprecated before 0.6.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-29 16:22:01 +02:00
Rusty Russell 8ee21bc352 Remove deprecated closing support from closingd.
This was changed in the spec in January, and bd1aa935b added it 2018-02-02.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-29 16:22:01 +02:00
Rusty Russell ab331f79a0 Add and fill in CHANGELOG.md
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-29 16:22:01 +02:00
Jordan Baczuk 0b597f671a added (or remotely) to README 2018-07-28 05:02:51 +00:00
Christian Decker 3783281fdd git: Ignore the connectd test binaries 2018-07-28 01:40:08 +00:00
Rusty Russell be3b782cb4 pytest: add test for update being decoded from onion error.
This actually tests that we correctly process an error packet, and
disable the channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 36730ddb6d gossipd: dev-suppress-gossip.
Useful for testing that we only get an update via the error message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 162879d6a2 channeld: use fulfilled_htlc and failed_htlc msgs in single htlc case.
We use these for receiving arrays at init time, we should also use them
for fulfull/fail of HTLCs in normal operation.  That we we benefit from all
those assertions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell f76203727d htlc_wire: be stricter in marshaling/unmarshaling struct failed_htlc.
There are three cases:
1. failcode is 0, scid is NULL, failreason is the onion to fwd.
2. failcode is non-zero, but UPDATE bit not set.  scid is NULL, failreason NULL.
3. failcode has UPDATE bit set.  scid is non-NULL, failreason is NULL.

Assert these on marshaling, and only send the parts we need so unmarshal is
always canonical.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 3c651f19a8 channeld: send correct channel_update when reporting failure.
The master tells us the short_channel_id of the outgoing channel, and
channeld is supposed to get the corresponding channel_update from gossipd.
Instead, it got the channel_update for the *local* channel and ignored
that one.
2018-07-27 14:12:00 +02:00
Rusty Russell 1119dd5577 channeld: always receive and maintain short_channel_id of failing channel.
The master tells us the short_channel_id of the outgoing channel when
failing an HTLC, but channeld didn't store it anywhere.  It also
didn't tell channeld the short_channel_id in the case where we're
reconnecting and it's feeding us an array of failed htlcs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 73b3782943 gossipd: send latest update in error message, even if delayed.
We delay internally to reduce broadcastig route flap, but errors are
a special case: we want to send the latest, otherwise we might send an
old (non-disabled) update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 3c66d5fa03 gossipd: add flag for locally disabling channel.
We used to just manually set ROUTING_FLAGS_DISABLED, but that means we
then suppressed the real channel_update because we thought it was a
duplicate!

So use a local flag: set it for the channel when the peer disconnects,
and clear it when channeld sends a local update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 93cf28553d devtools/decodemsg: add --onion option for decoding onion errors.
This requires a tweak to generate-wire.py too, since it always called the
top-level routine 'print_message'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell ef33dd2998 jsonrpc: reduce debug log spam for JSON commands.
Just log the failed ones, not every connection and successful commands.

Before (VALGRIND=0 -n10):
111 passed, 1 skipped in 175.78 seconds

After:
111 passed, 1 skipped in 173.92 seconds
111 passed, 1 skipped in 164.16 seconds
111 passed, 1 skipped in 171.30 seconds
111 passed, 1 skipped in 180.05 seconds
111 passed, 1 skipped in 180.04 seconds


Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell 2f78c90782 pytest: fix flakiness in test_penalty_outhtlc
We were *supposed* to be waiting for the next commitment tx so we
made sure the one we broadcast was old, *but* the 'revoke_and_ack'
we were waiting for could be matched by the completion of the previous
'revoke_and_ack'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 11:36:22 +02:00
Rusty Russell f73ea5d0ba pytest: test for correct wallet balances, re-enable penalty test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 11:36:22 +02:00
Rusty Russell ae8b459932 onchaind: tell wallet when we see our revoked commitment tx `to_remote` output
Without this, the wallet won't know how to spend it.

Fixes: #1738
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 11:36:22 +02:00
William Casarin 8869204730 test: add pytest-xdist to the python test requirements
Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-26 23:18:37 +02:00
William Casarin 6e4745b28a doc: update python test requirements in HACKING
Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-26 23:18:37 +02:00
Rene Pickhardt 88ef2246e8 exchanged code containing goto with a do while loop (#1758)
exchanged code containing goto with a do while loop
2018-07-26 23:17:45 +02:00
Christian Decker d6048de100 json-rpc: Shutdown the JSON-RPC in the context of a DB transaction
This needs to be done separately from the rest of the daemon since we can
otherwise not make sure that it happens before the DB is freed and we might
still need the DN, and be running in a DB transaction, for some destructors to
run.
2018-07-26 19:14:56 +02:00
Christian Decker ead0c8e17a pytest: Add test for issue 1737 that is expected to fail
This reproduces the issue described in issue #1737 and will be fixed by the next
commit.
2018-07-26 19:14:56 +02:00
Simon Vrouwe a80622edab at startup initialize smoothed feerate to polled feerate
fix indentation
2018-07-26 19:08:13 +02:00
Rusty Russell 378d73cd96 channeld: fix dev_disconnect doublefree crash.
We shouldn't unconditionally free msg in enqueue_peer_msg:

DEBUG: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: dev_disconnect: @WIRE_REVOKE_AND_ACK
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: FATAL SIGNAL 6 (version 8aae6a8)
...
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:98 (call_error) 0x80855d1
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:170 (check_bounds) 0x8085730
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:181 (to_tal_hdr) 0x8085791
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: ccan/ccan/tal/tal.c:504 (tal_free) 0x8085fe6
BROKEN: lightning_channeld-0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1: backtrace: channeld/channel.c:2651 (main) 0x8050639

For additional safety, handle each msg allocation separately, rather than
freeing at bottom of large branch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-26 14:33:23 +02:00
Mark Beckwith e8edecfbf7 Add lightningd-config(5) to install
The man5 section wasn't being added in the Makefile.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-26 05:41:48 +00:00
ueno 21e61a5232 fix: make devtools-clean 2018-07-25 15:44:25 +02:00
Rusty Russell 641b33349b peer_control: annotate feature bitfields as being taken.
Because tal_dup_arr takes, this does too.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell f2e0c35916 connectd: hack in a gossip flush as a workaround.
We were failing test_closing_torture, with gossipd complaining that it
received a malformed packet.  This makes it pass, but the real fix is
in the next series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 722dd05e9d lightningd: keep features arrays for connected peers.
As a side-effect, we only print them for connected peers (which avoids
an O(n^2) traversal).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell b5fcd54ef0 channeld: don't read from gossipfd while we're reconnecting.
That was the cause of the bad gossip order failures: gossipd thought our
channel was live, but the other end didn't receive message last time.

Now gossipd doesn't use fd to kill us (connectd tells master to do so), we
can implement read_peer_msg_nogossip().

Fixes: #1706
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell d241bd762c connectd: don't use gossip_getnodes_entry.
gossip_getnodes_entry was used by gossipd for reporting nodes, and for
reporting peers.  But the local_features field is only available for peers,
and most other fields are only available from node_announcement.

Note that the connectd change actually means we get less information
about peers: gossipd used to do the node lookup for peers and include the
node_announcement information if it had it.

Since generate_wire.py can't create arrays-of-arrays, we add a 'struct
peer_features' to encapsulate the two feature arrays for each peer, and
for convenience we add it to lightningd/gossip_msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell c9435787cb connectd: remove gossip-related fields.
This almost completes the removal of the last vestages of gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 7b2641ed0d gossipd: remove peer-related fields and wire messages.
This completes the removal of peer-related messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 0d442b5ff2 gossipd: move files into connectd.
These source files are only used for peer-related things, so move them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 5c066b9a25 connectd: wean off gossip_wire message types, use our own.
This involves much renaming of gossip -> channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell dba7f9002f gossipd: provide connectd with address resolution.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 3d3d2ef9af gossipd: remove connectd functionality, enable connectd.
This patch guts gossipd of all peer-related functionality, and hands
all the peer-related requests to channeld instead.

gossipd now gets the final announcable addresses in its init msg, since
it doesn't handle socket binding any more.

lightningd now actually starts connectd, and activates it.  The init
messages for both gossipd and connectd still contain redundant fields
which need cleaning up.

There are shims to handle the fact that connectd's wire messages are
still (mostly) gossipd messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell 92d66a5451 gossipd: take connectd fd on initialization.
connectd has a dedicated fd to gossipd, so it can ask for a new gossip_fd
for a peer.

gossipd has a standalone routine to create a remote peer (this will
eventually be the only way gossipd creates a new peer).

For now lightningd creates a socketpair but doesn't run connectd, so
gossipd never sees any requests on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00