Commit Graph

3102 Commits

Author SHA1 Message Date
Rusty Russell 4aeebe88f2 log: fix ltmp crash.
When we clear and recreate ltmp, we attach it to whatever logbook it's on.
This, of course, is fraught, since it may be freed.

We could make it NULL-parented, but that makes YA special-case to free
when we exit (we try to keep valgrind happy by freeing everything).  So
since the first log_book is the permanent one attached to lightningd,
just keep that parent when we re-build it after use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 12:58:13 +01:00
Igor Cota d4d1c4acb0 Add PIE option to Makefile to build position independent executables
As of version 5.0 Android requires all dynamically linked executables to support PIE. This allows programs to be loaded at a different addresses, making it harder for attackers to target.
Enable with PIE=1
2018-02-20 03:55:53 +00:00
libbitc c360cb7b1f Add option to specify pid file
Closes #969
2018-02-20 03:24:23 +00:00
Igor Cota bd95aba7a5 Pass the compiler variable to the configurator executable when building ccan/config.h
When cross compiling it's important that the resulting config.h reflects the platform we are building for and not the one we are building on.
Otherwise we end up with a config.h that defines headers that are not there on the target platform, wrong endnianness and so on.
The -static flag is there to be able to easily run the configurator test executables on the build machine with qemu-*.
E.g. Without the -static flag the resulting dynamically linked ARM executables complain about the lack of linker (/lib/ld-linux-armhf.so or /system/bin/linker for Android), since these files are not usually available on the build machine building statically avoids this problem and results in a proper config.h for cross compiling.
2018-02-20 03:18:11 +00:00
gabridome 3a5aa75fb6 A more detailed description of the "amount" field
The more frequent question I ask to myself when initiating channels regards the amount field.

In this document we have three different denominations and the modification proposed aims to address the possible doubts or errors.
The names <amount_in_bitcoins>, <amount_in_satoshis> or <amount_in_millisatoshis> (this last is omitted because the denomination is in the description) are not so elegant but they could serve the purpose.
2018-02-20 03:06:53 +00:00
Rusty Russell 1dc6f7575a lightningd: fix crash when we get onchain after old db.
lightningd(25923): lightning_onchaind(03cbf298b068300be33f06c947b9d3f00a0f0e8089da3233f5db37e81d3a596fe1): UPDATE WIRE_ONCHAIN_MISSING_HTLC_OUTPUT
lightningd(25923): FATAL SIGNAL 11 RECEIVED
lightningd(25923): backtrace: lightningd/log.c:498 (log_crash) 0x410ae8
lightningd(25923): backtrace: (null):0 ((null)) 0x7ff2ddb8e0df
lightningd(25923): backtrace: lightningd/pay.c:231 (local_routing_failure) 0x415b33
lightningd(25923): backtrace: lightningd/pay.c:383 (payment_failed) 0x41604c
lightningd(25923): backtrace: lightningd/peer_htlcs.c:783 (onchain_failed_our_htlc) 0x4225e5
lightningd(25923): backtrace: lightningd/peer_control.c:1209 (handle_missing_htlc_output) 0x41b31c
lightningd(25923): backtrace: lightningd/peer_control.c:1323 (onchain_msg) 0x41b759
lightningd(25923): backtrace: lightningd/subd.c:498 (sd_msg_read) 0x425b40

Fixes: #878
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
Rusty Russell d47d85fa30 wallet: add list of upgrades.
Useful for debugging a db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
Rusty Russell 097b9345eb wallet: load route_nodes and route_channels when seeking a particular entry.
This got missed when 299b280f7 got merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
practicalswift 4fccbbae69 Assert that we get the assumed return value from sqlite3_step(...) 2018-02-19 12:18:59 +01:00
practicalswift f351417905 Add assertions to clarify our assumptions about msg != NULL 2018-02-19 12:17:55 +01:00
ZmnSCPxj ea3ee01215 peer_control: Support xxx@yyy:zzz form for connecting. 2018-02-19 11:59:11 +01:00
Rusty Russell 4653493507 test_lightningd: make test_payment_failed_persistence more reliable.
We don't want the invoice to fail locally, which can happen if we're
slow enough.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell cfa50d393a openingd: use peer_failed like normal instead of boutique negotiation_failed.
Because peer_failed would previously drop the connection, we had a
special 'negotiation_failed' message which made the master hand it
back to gossipd.  We don't need that any more.

This also meant we no longer need a special hook in read_peer_msg
for openingd to send this message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell dba08f9d1b peer_failed: don't send error ourselves.
gossipd actually does that now, so we don't need this synchronous send
hack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 02d469b3d4 peer_failed: hand fds back to master when we fail.
master now hands it back to gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell f76ff90485 status: split off error messages into a new 'peer_status' type.
Several daemons (onchaind, hsm) want to use the status messages, but
don't communicate with peers.  The coming changes made them drag in
more code they didn't need, so instead we have a different
non-overlapping type.

We combine the status_received_errmsg and status_sent_errmsg
into a single status_peer_error, with the presence or not of the
'error_for_them' field indicating direction. 

We also rename status_fatal_connection_lost() to
peer_failed_connection_lost() to fit in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 65ff5f8bb1 read_peer_msg: ignore errors not destined for this channel.
We quoted the spec, but somehow the implementation disappeared.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 201d498e39 peer_failed: automatically hand PEER_FD, GOSSIP_FD; add gossip_index
We make it a macro, since everyone uses PEER_FD and GOSSIP_FD constants
(they're actually always the same, but this is slightly safer), and
add a gossip_index arg: this is groundwork for when we want to hand
the peer back to master for gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell e17b0ebcb4 channeld: map htlc add/remove errors to names.
I couldn't figure out what 'Bad peer_add_htlc: 7' meant!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 611ecc60ae lightningd: rename peer_state -> channel_state, remove OPENINGD.
And now we can finally do the db upgrade to remove any OPENINGD
channels once, since we never put them back.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell cc460095ca lightningd: make new_channel a proper constructor.
It's giant, but it's encapsulating at least.  It is called from the wallet
code when loading channels, or from the opening code when converting
an uncommitted_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 4ab7589427 sqlite3_column_arr: helper to allocate and read an array from a blob.
We use it on the secrets array for the moment, but it's also useful
for remote_shutdown_scriptpubkey, as used in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell e0603d7221 channel: now we're always complete, fields don't have to be optional.
Now any struct channel is a genuine channel, the following fields are
always valid:

1. funding_txid: doesn't need to be a pointer.
2. our_msatoshi: doesn't need to be a pointer.
3. last_sig: doesn't need to be a pointer.
4. channel_info: doesn't need to be a pointer.

In addition, 'last_tx' is always valid.

The main effect is to remove a whole heap of branches from the wallet code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 8db8c51201 lightningd: struct uncommitted_channel for opening channels.
Each peer can have one 'uncommitted' channel, which is in the process
of opening.  This is used for openingd, and then on return we convert
it into a full-fledged struct channel and commit it into the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell d2f691b288 subd: make functions more generic, don't assume 'struct channel'.
This means the caller needs to supply an explicit log to base the
subd log on, and also a callback for error handling.

The callback is kind of ugly, but it gets reworked towards the end
of this series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 1a78efaee5 peer_control: don't refer to peer in copy_to_parent_log
Once we rely on the logbook outlasting the peer, we can't refer to the
peer from the logbook function:

Valgrind error file: valgrind-errors.26567
==26567== Invalid read of size 8
==26567==    at 0x126297: copy_to_parent_log (peer_control.c:690)
==26567==    by 0x11C06B: maybe_print (log.c:253)
==26567==    by 0x11C145: logv (log.c:270)
==26567==    by 0x11C448: log_ (log.c:319)
==26567==    by 0x132951: destroy_subd (subd.c:537)
==26567==    by 0x179C19: notify (tal.c:240)
==26567==    by 0x17A0CE: del_tree (tal.c:400)
==26567==    by 0x17A120: del_tree (tal.c:410)
==26567==    by 0x17A4ED: tal_free (tal.c:509)
==26567==    by 0x16DEB5: io_close (io.c:443)
==26567==    by 0x1328BC: sd_msg_read (subd.c:516)
==26567==    by 0x1320AC: read_fds (subd.c:328)
==26567==  Address 0x6cf9ca0 is 48 bytes inside a block of size 216 free'd
==26567==    at 0x4C30D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26567==    by 0x17A1A9: del_tree (tal.c:421)
==26567==    by 0x17A4ED: tal_free (tal.c:509)
==26567==    by 0x124B6C: delete_peer (peer_control.c:180)
==26567==    by 0x12B369: destroy_uncommitted_channel (peer_control.c:2505)
==26567==    by 0x179C19: notify (tal.c:240)
==26567==    by 0x17A0CE: del_tree (tal.c:400)
==26567==    by 0x17A4ED: tal_free (tal.c:509)
==26567==    by 0x12B31E: opening_channel_errmsg (peer_control.c:2496)
==26567==    by 0x13243A: handle_peer_error (subd.c:407)
==26567==    by 0x1326E4: sd_msg_read (subd.c:472)
==26567==    by 0x1320AC: read_fds (subd.c:328)
==26567==  Block was alloc'd at
==26567==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26567==    by 0x179C83: allocate (tal.c:250)
==26567==    by 0x17A250: tal_alloc_ (tal.c:448)
==26567==    by 0x124950: new_peer (peer_control.c:151)
==26567==    by 0x12B3EC: new_uncommitted_channel (peer_control.c:2521)
==26567==    by 0x12B5C5: peer_accept_channel (peer_control.c:2569)
==26567==    by 0x126099: peer_sent_nongossip (peer_control.c:641)
==26567==    by 0x113B28: peer_nongossip (gossip_control.c:55)
==26567==    by 0x113D9D: gossip_msg (gossip_control.c:144)
==26567==    by 0x132783: sd_msg_read (subd.c:487)
==26567==    by 0x1320AC: read_fds (subd.c:328)
==26567==    by 0x16D1FE: next_plan (io.c:59)
==26567==

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 7c512f91ce memleak: ignore tal_link artifacts.
We use strends, because that works with or without CCAN_TAL_DEBUG (which
prepends file and line).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell a2c6ec6c9b lightningd: use tal_link for log_book.
BackgroundL Each log has a log_book: many logs can share the same one,
as each one can have a separate prefix.

Testing tickled a bug at the end of this series, where subd was
logging to the peer's log_book on shutdown, but the peer was already
freed.  We've already had issues with logging while lightningd is
shutting down.

There are times when reference counting really is the right answer,
this seems to be one of them: the 'struct log' share the 'struct
log_book' and the last 'struct log' cleans it up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 237a65d000 ccan: add tal/link.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 3c0be71d37 wallet: make static, not dynamic decision to insert for everything.
Since we create new entries from wallet_channel_insert(), there's no
need for the branches.  And indeed, many wallet functions can be
static.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell 72108f0cb9 wallet: don't use rowid for the channel's DBID.
We derive the seed from this, so it needs to be unique, but using
rowid forced us to put the channel into the db early, before it
was ready.

Instead, use a counter to ensure uniqueness, initialized when we load
existing peers.  This doesn't need to touch the database at all.

As we now have only two places where the channel is committed (the
funder and fundee paths), so we create a new explicit
'wallet_channel_insert()' function: 'wallet_channel_save()' now just
updates.

Note that this also fixes some weirdness in
wallet_channels_load_active: we strangely avoided loading channels in
CLOSINGD_COMPLETE (which fortunately was a transient state, so
unlikely anyone hit this).  Note that since the lines above already
delete all the OPENINGD channels, we now simply load them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
ZmnSCPxj 9be4e159ae invoice: Disable overlong description.
See: https://github.com/ElementsProject/lightning/pull/1020#pullrequestreview-97372207

Fixes: #1014
2018-02-19 02:42:11 +00:00
John Barboza 7a77271922 lightningd: encode fallback address in 5 bits
The bolt11 specification requires that the version number of the
Fallback on-chain address should be 5 bits wide instead of 8
bits.
2018-02-19 02:01:37 +00:00
luca vaccaro c4590b6e60 Add dev-listaddrs option (#1001)
* Add dev-listaddrs option

* Fix whitespaces

* Check bip32 max derivation index

* Fix tal_tmpctx context

* Implicit tal_free

* Remove tal_free()

* Remove tmpctx
2018-02-18 13:52:46 +01:00
ZmnSCPxj 38535fc36c payalgo: Create a new failure for paying expired invoice. 2018-02-18 13:51:37 +01:00
Christian Decker b2819f9f97 wallet: Add check that wallet matches the network on startup
Adds a simple check that compares genesis-blockhashes from the
chainparams against the blockhash that the wallet was created
with. The wallet is network specific, so mixing is always a bad idea.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-17 11:36:17 +00:00
Anton Astafiev 8787766d44 Pylightning (#1003)
* Fix dev_setfees to set slow and normal fees correctly.

Due to a bug def_setfees(100, slow=100) would instead set immediate and
normal fees to 100. This behavior has been updated to set fees to
correct values, make the values truly optional as per documentation and
unit test this behavior.

* Fix pay() to set msatoshi, description and risk factor properly

Due to a bug pay(invoice, description='1000') resulted in payment of
1000 msatoshi instead. This was fixed and covered with tests.

* Fix named args in listpayments, listpeers and connect

* Do not pass None to methods where it is default value

* Make description on invoice and pay match.

Suggested-by: @ZmnSCPxj

* Fix dev_setfees to set slow and normal fees correctly.

Due to a bug def_setfees(100, slow=100) would instead set immediate and
normal fees to 100. This behavior has been updated to set fees to
correct values, make the values truly optional as per documentation and
unit test this behavior.

* Fix pay() to set msatoshi, description and risk factor properly

Due to a bug pay(invoice, description='1000') resulted in payment of
1000 msatoshi instead. This was fixed and covered with tests.

* Fix named args in listpayments, listpeers and connect

* Do not pass None to methods where it is default value

* Make description on invoice and pay match.

Suggested-by: @ZmnSCPxj
2018-02-16 22:40:55 +01:00
Rusty Russell 43ec3f0761 jsonrpc: allow multiple commands in-flight from single JSON connection.
We now keep a list of commands for the jcon instead of a simple
'current' pointer: the assertions become a bit more complex, but
the rest is fairly mechanical.

Fixes: #1007
Reported-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 17:51:04 +01:00
Igor Cota f7097c76bd Pass the HOST and BUILD environment vars to the external submodules configurators. Needed to cross-compile 2018-02-16 17:50:38 +01:00
ZmnSCPxj 0489c7eb93 test_lightningd: Change test_pay0 to use sendpay. 2018-02-16 13:08:29 +01:00
ZmnSCPxj d9163dbb4f test_lightningd: Modify test to remove repeated pay attempt. 2018-02-16 13:08:29 +01:00
ZmnSCPxj 4ad1021c2c payalgo: Repeat pay command if possible.
Fixes: #863
2018-02-16 13:08:29 +01:00
ZmnSCPxj fda26bdcda payalgo: New file for pay command. 2018-02-16 13:08:29 +01:00
ZmnSCPxj 7ee6ccfbd7 pay: Generalize internal interface of sendpay. 2018-02-16 13:08:29 +01:00
ZmnSCPxj 700dda7e60 pay: Rename `pay_command` to `sendpay_command`
In preparation for separating `pay` algorithm from
`sendpay`.
2018-02-16 13:08:29 +01:00
Rusty Russell bdd11e07fe chaintopology: fix 100 block subtraction.
We do a complicated dance because we don't know the current block
height before setting up the topology.

If we're starting at a particular block, we want to go back 100 blocks
before that to cover any reorgs.

If we're not (fresh startup), we still want to go back 100 blocks
because we don't bother handling a reorg which removes all the blocks
we know.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 37373f2c16 wallet: provide better comments on wallet_first_blocknum.
ZmnSCPxj queried the unilateral close case, so make that clearer.
Christian raise concerns about existing channels, so make it clear
what we're doing there too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell ef9c6cb8c3 wallet: don't scan from worst-case start on first use.
We only need to do that if it's possible there's something to find:
either we have an unspent output from a unilateral close, or we've
ever handed out an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell df4c669a60 test: test for funds sent while we were offline.
As described by @lvaccaro in #990.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell 6620305606 wallet: use last_processed_block to determine scan start.
With fallback depending on chainparams: this means the first upgrade
will be slow, but after that it'll be fast.

Fixes: #990
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00