Commit Graph

3082 Commits

Author SHA1 Message Date
Rusty Russell cc460095ca lightningd: make new_channel a proper constructor.
It's giant, but it's encapsulating at least.  It is called from the wallet
code when loading channels, or from the opening code when converting
an uncommitted_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 4ab7589427 sqlite3_column_arr: helper to allocate and read an array from a blob.
We use it on the secrets array for the moment, but it's also useful
for remote_shutdown_scriptpubkey, as used in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell e0603d7221 channel: now we're always complete, fields don't have to be optional.
Now any struct channel is a genuine channel, the following fields are
always valid:

1. funding_txid: doesn't need to be a pointer.
2. our_msatoshi: doesn't need to be a pointer.
3. last_sig: doesn't need to be a pointer.
4. channel_info: doesn't need to be a pointer.

In addition, 'last_tx' is always valid.

The main effect is to remove a whole heap of branches from the wallet code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 8db8c51201 lightningd: struct uncommitted_channel for opening channels.
Each peer can have one 'uncommitted' channel, which is in the process
of opening.  This is used for openingd, and then on return we convert
it into a full-fledged struct channel and commit it into the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell d2f691b288 subd: make functions more generic, don't assume 'struct channel'.
This means the caller needs to supply an explicit log to base the
subd log on, and also a callback for error handling.

The callback is kind of ugly, but it gets reworked towards the end
of this series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 1a78efaee5 peer_control: don't refer to peer in copy_to_parent_log
Once we rely on the logbook outlasting the peer, we can't refer to the
peer from the logbook function:

Valgrind error file: valgrind-errors.26567
==26567== Invalid read of size 8
==26567==    at 0x126297: copy_to_parent_log (peer_control.c:690)
==26567==    by 0x11C06B: maybe_print (log.c:253)
==26567==    by 0x11C145: logv (log.c:270)
==26567==    by 0x11C448: log_ (log.c:319)
==26567==    by 0x132951: destroy_subd (subd.c:537)
==26567==    by 0x179C19: notify (tal.c:240)
==26567==    by 0x17A0CE: del_tree (tal.c:400)
==26567==    by 0x17A120: del_tree (tal.c:410)
==26567==    by 0x17A4ED: tal_free (tal.c:509)
==26567==    by 0x16DEB5: io_close (io.c:443)
==26567==    by 0x1328BC: sd_msg_read (subd.c:516)
==26567==    by 0x1320AC: read_fds (subd.c:328)
==26567==  Address 0x6cf9ca0 is 48 bytes inside a block of size 216 free'd
==26567==    at 0x4C30D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26567==    by 0x17A1A9: del_tree (tal.c:421)
==26567==    by 0x17A4ED: tal_free (tal.c:509)
==26567==    by 0x124B6C: delete_peer (peer_control.c:180)
==26567==    by 0x12B369: destroy_uncommitted_channel (peer_control.c:2505)
==26567==    by 0x179C19: notify (tal.c:240)
==26567==    by 0x17A0CE: del_tree (tal.c:400)
==26567==    by 0x17A4ED: tal_free (tal.c:509)
==26567==    by 0x12B31E: opening_channel_errmsg (peer_control.c:2496)
==26567==    by 0x13243A: handle_peer_error (subd.c:407)
==26567==    by 0x1326E4: sd_msg_read (subd.c:472)
==26567==    by 0x1320AC: read_fds (subd.c:328)
==26567==  Block was alloc'd at
==26567==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26567==    by 0x179C83: allocate (tal.c:250)
==26567==    by 0x17A250: tal_alloc_ (tal.c:448)
==26567==    by 0x124950: new_peer (peer_control.c:151)
==26567==    by 0x12B3EC: new_uncommitted_channel (peer_control.c:2521)
==26567==    by 0x12B5C5: peer_accept_channel (peer_control.c:2569)
==26567==    by 0x126099: peer_sent_nongossip (peer_control.c:641)
==26567==    by 0x113B28: peer_nongossip (gossip_control.c:55)
==26567==    by 0x113D9D: gossip_msg (gossip_control.c:144)
==26567==    by 0x132783: sd_msg_read (subd.c:487)
==26567==    by 0x1320AC: read_fds (subd.c:328)
==26567==    by 0x16D1FE: next_plan (io.c:59)
==26567==

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 7c512f91ce memleak: ignore tal_link artifacts.
We use strends, because that works with or without CCAN_TAL_DEBUG (which
prepends file and line).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell a2c6ec6c9b lightningd: use tal_link for log_book.
BackgroundL Each log has a log_book: many logs can share the same one,
as each one can have a separate prefix.

Testing tickled a bug at the end of this series, where subd was
logging to the peer's log_book on shutdown, but the peer was already
freed.  We've already had issues with logging while lightningd is
shutting down.

There are times when reference counting really is the right answer,
this seems to be one of them: the 'struct log' share the 'struct
log_book' and the last 'struct log' cleans it up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 237a65d000 ccan: add tal/link.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 3c0be71d37 wallet: make static, not dynamic decision to insert for everything.
Since we create new entries from wallet_channel_insert(), there's no
need for the branches.  And indeed, many wallet functions can be
static.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 72108f0cb9 wallet: don't use rowid for the channel's DBID.
We derive the seed from this, so it needs to be unique, but using
rowid forced us to put the channel into the db early, before it
was ready.

Instead, use a counter to ensure uniqueness, initialized when we load
existing peers.  This doesn't need to touch the database at all.

As we now have only two places where the channel is committed (the
funder and fundee paths), so we create a new explicit
'wallet_channel_insert()' function: 'wallet_channel_save()' now just
updates.

Note that this also fixes some weirdness in
wallet_channels_load_active: we strangely avoided loading channels in
CLOSINGD_COMPLETE (which fortunately was a transient state, so
unlikely anyone hit this).  Note that since the lines above already
delete all the OPENINGD channels, we now simply load them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
ZmnSCPxj 9be4e159ae invoice: Disable overlong description.
See: https://github.com/ElementsProject/lightning/pull/1020#pullrequestreview-97372207

Fixes: #1014
2018-02-19 02:42:11 +00:00
John Barboza 7a77271922 lightningd: encode fallback address in 5 bits
The bolt11 specification requires that the version number of the
Fallback on-chain address should be 5 bits wide instead of 8
bits.
2018-02-19 02:01:37 +00:00
luca vaccaro c4590b6e60 Add dev-listaddrs option (#1001)
* Add dev-listaddrs option

* Fix whitespaces

* Check bip32 max derivation index

* Fix tal_tmpctx context

* Implicit tal_free

* Remove tal_free()

* Remove tmpctx
2018-02-18 13:52:46 +01:00
ZmnSCPxj 38535fc36c payalgo: Create a new failure for paying expired invoice. 2018-02-18 13:51:37 +01:00
Christian Decker b2819f9f97 wallet: Add check that wallet matches the network on startup
Adds a simple check that compares genesis-blockhashes from the
chainparams against the blockhash that the wallet was created
with. The wallet is network specific, so mixing is always a bad idea.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-17 11:36:17 +00:00
Anton Astafiev 8787766d44 Pylightning (#1003)
* Fix dev_setfees to set slow and normal fees correctly.

Due to a bug def_setfees(100, slow=100) would instead set immediate and
normal fees to 100. This behavior has been updated to set fees to
correct values, make the values truly optional as per documentation and
unit test this behavior.

* Fix pay() to set msatoshi, description and risk factor properly

Due to a bug pay(invoice, description='1000') resulted in payment of
1000 msatoshi instead. This was fixed and covered with tests.

* Fix named args in listpayments, listpeers and connect

* Do not pass None to methods where it is default value

* Make description on invoice and pay match.

Suggested-by: @ZmnSCPxj

* Fix dev_setfees to set slow and normal fees correctly.

Due to a bug def_setfees(100, slow=100) would instead set immediate and
normal fees to 100. This behavior has been updated to set fees to
correct values, make the values truly optional as per documentation and
unit test this behavior.

* Fix pay() to set msatoshi, description and risk factor properly

Due to a bug pay(invoice, description='1000') resulted in payment of
1000 msatoshi instead. This was fixed and covered with tests.

* Fix named args in listpayments, listpeers and connect

* Do not pass None to methods where it is default value

* Make description on invoice and pay match.

Suggested-by: @ZmnSCPxj
2018-02-16 22:40:55 +01:00
Rusty Russell 43ec3f0761 jsonrpc: allow multiple commands in-flight from single JSON connection.
We now keep a list of commands for the jcon instead of a simple
'current' pointer: the assertions become a bit more complex, but
the rest is fairly mechanical.

Fixes: #1007
Reported-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 17:51:04 +01:00
Igor Cota f7097c76bd Pass the HOST and BUILD environment vars to the external submodules configurators. Needed to cross-compile 2018-02-16 17:50:38 +01:00
ZmnSCPxj 0489c7eb93 test_lightningd: Change test_pay0 to use sendpay. 2018-02-16 13:08:29 +01:00
ZmnSCPxj d9163dbb4f test_lightningd: Modify test to remove repeated pay attempt. 2018-02-16 13:08:29 +01:00
ZmnSCPxj 4ad1021c2c payalgo: Repeat pay command if possible.
Fixes: #863
2018-02-16 13:08:29 +01:00
ZmnSCPxj fda26bdcda payalgo: New file for pay command. 2018-02-16 13:08:29 +01:00
ZmnSCPxj 7ee6ccfbd7 pay: Generalize internal interface of sendpay. 2018-02-16 13:08:29 +01:00
ZmnSCPxj 700dda7e60 pay: Rename `pay_command` to `sendpay_command`
In preparation for separating `pay` algorithm from
`sendpay`.
2018-02-16 13:08:29 +01:00
Rusty Russell bdd11e07fe chaintopology: fix 100 block subtraction.
We do a complicated dance because we don't know the current block
height before setting up the topology.

If we're starting at a particular block, we want to go back 100 blocks
before that to cover any reorgs.

If we're not (fresh startup), we still want to go back 100 blocks
because we don't bother handling a reorg which removes all the blocks
we know.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 37373f2c16 wallet: provide better comments on wallet_first_blocknum.
ZmnSCPxj queried the unilateral close case, so make that clearer.
Christian raise concerns about existing channels, so make it clear
what we're doing there too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell ef9c6cb8c3 wallet: don't scan from worst-case start on first use.
We only need to do that if it's possible there's something to find:
either we have an unspent output from a unilateral close, or we've
ever handed out an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell df4c669a60 test: test for funds sent while we were offline.
As described by @lvaccaro in #990.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 6620305606 wallet: use last_processed_block to determine scan start.
With fallback depending on chainparams: this means the first upgrade
will be slow, but after that it'll be fast.

Fixes: #990
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 21849329dd wallet: store last block number we searched for UTXOs.
We already go back 100 from this in case of reorgs, so the block number
itself is sufficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 256bdc12ff lightningd: activate crashlog later.
We error out for all kinds of reasons early on (eg. bitcoind down),
and printing a backtrace for them is pretty confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell eb17d6af71 lightningd: implement --daemon.
Includes closing off stdout and stderr.  We don't do it directly in the
arg parser, as we want to interact normally (eg with other errors) before
we turn off stdout/stderr.

Fixes: #986
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell 65593d4a98 chaintopology: don't start fee estimation loop until we're fully active.
This interacts badly with --daemon (next patch) which then tries to
reap a child it didn't create, which took me a couple of hours to
figure out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell c5cc5c5c1b ccan: import ccan/daemonize.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell ccd0e5db54 ccan: update so we get exposed path constants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell c8f9ea7bf7 fixup: test_multirpc does not terminate.
We need to set expiry, otherwise waitinvoice would take 1 hr, and we
can't read once for every cmd, since each read may consume more than
a single result, and we block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 12:56:48 +01:00
Christian Decker cab7e3e43a pytest: Add failing test with JSON-RPC requests crossing each other
Suggested-By: @ZmnSCPxj
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-16 12:56:48 +01:00
Rusty Russell a08bcfdbd3 jsonrpc: don't crash on multiple commands at once.
Once we read a command, we are supposed to io_wait until it finishes.
However, we are actually woken in two places: when it's complete
(which is correct), and when it's written out (which is wrong).

We don't care when it's written out, only when it's finished:
refactor to make json_done() free and NULL the old ->current,
rather than have the callers do it.  Now it's clear that it's
ready for both new output and new input.

Fixes: #934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 12:56:48 +01:00
Douglas Schilling Landgraf cb1a4a5180 pylightning: make flake8 happy
Trivial changes to make the code pythonic

Signed-off-by: Douglas Schilling Landgraf <dougsland@gmail.com>
2018-02-16 12:55:04 +01:00
Christian Decker e48a97ffd4 pylightning: Filter out None arguments in JSON-RPC calls
Passing them in doesn't hurt, but better to avoid ambivalences.

Suggested-by: @graphite
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-15 23:56:38 +00:00
Christian Decker db5c0aa6c1 pytest: Make pep8 happy(er)
A number of fixes to make our code more pythonic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Co-authored-by: Guido Dassori @gdassori
2018-02-15 23:56:38 +00:00
Christian Decker 0e40b9b08c pylightning: Switch to dict-based params instead of positional
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Co-authored-by: Guido Dassori @gdassori
2018-02-15 23:56:38 +00:00
Christian Decker ed2fee8977 pylightning: Added explicit logger to log to
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Co-authored-by: Guido Dassori @gdassori
2018-02-15 23:56:38 +00:00
ZmnSCPxj 33b25b6a9b travis: Correctly process $SOURCE_CHECK_ONLY.
The `!` outside of parentheses was being cut by travis.
2018-02-15 08:48:35 +00:00
Luca Vaccaro 4dac2da8fc Fix litecoin mainnet & testnet chainparams 2018-02-14 15:26:27 +01:00
Rusty Russell 8f7a19d1a3 onchain: handle case where multiple HTLCs exist for same payment_hash.
We will have probably failed the others, but either way, don't try to
fulfill an HTLC we've already failed.

Fixes: #394
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:57:42 +01:00
Rusty Russell 55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell 6a3ccafaf9 wallet: don't implicitly remove peers, but do it explicitly.
This provides a sanity check that we are in sync, and also keeps the
logic in the program and out of the SQL.

Since the destructor now doesn't clean up the peer, there are some
wider changes to be made when cleaning up.  Most notably we create
lots of channels in run-wallet.c and they previously freed the peer:
now we need free the peer explicitly, so we need to free them first.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell 98de10b842 channel: rename free_channel to delete_channel.
free_channel() sounds like a destructor.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00