Commit Graph

5473 Commits

Author SHA1 Message Date
Michael Schmoock 4986d6b39d feat: block of newlines when attaching a logfile
This will make the logger write 4 newlines to re-attached logfiles.
The newlines wont appear on logfiles that are just created.
Additionally the server prints 50 '-' dashes before printing his
startup message, which also help increase readability on logfile.

This was inspired by the way Bitcoin Core handles logfiles.
2019-02-22 03:01:37 +00:00
Christian Decker f622ffb087 pylightning: Add plugin dispatch tests to check-python and fix them
These weren't checked by CI yet, and they are really short so I just added
them to the check-python target.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker 7f11b4854e pylightning: Split @method and @async_method decorators
Suggested-by: Rusty Russell <@rustyrussell>
Suggested-by: Conor Scott <@conscott>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker d9c3f5ec4b pylightning: Rename peer_id to node_id in getroute
Technically this is a node, not a direct peer, so this is correct.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker 571fb44d20 pylightning: Add a small test for async rpcmethods
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker cc4fbfca43 pylightning: Add the `background` keyword to hooks and methods
This indicates that the method or hook will accepts a request
parameter, and will use that to return the result or raise an
exception instead of returning the return value. This allows the hook
or method to stash the incomplete request or pass it around, without
blocking the JSON-RPC interface.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker 9670b4382f pylightning: Don't always use request ID 0
This isn't a problem for now since we don't support multithreading,
and only allow synchronous calls, but eventually this'll become
important.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker f452d00b77 pylightning: Add support for *args and **kwargs in plugin dispatch
These are a bit special and are handled separately.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker 8de1a85ac0 pylightning: Exception if we have unfulfilled positional arguments
This caused me to backtrack quite a bit, so this should help debugging
in the future.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker 81fa247d07 pylightning: Wrap request in an object
We well need this in the next commit to be able to return from an
asynchronous call. We also guard stdout access with a reentrant lock
since we are no longer guaranteed that all communication happens on
the same thread.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker 71795d4d6a pylightning: Wrap the plugin methods in a class
Sending around unnamed tuples is bound to cause some issues sooner or
later, so we just create a quick class that holds all the information
about a plugin method.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Conor Scott d96564ce3e pylightning: raise Error when missing rpc parameter 2019-02-21 14:23:01 +01:00
Rusty Russell 6ed54dd34e db: fix dangling peers on db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell 594f260de7 pytest: test starting with db corrupted by dangling peer.
db was taken from the failed test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell 7887e5c00a pytest: add support for starting nodes with a pre-canned db.
With xz, the db is only 9120 bytes, vs 163840.  And lzma is a builtin
in Python 3.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell 3b587a1c6d lightningd: fix db error where we can have detached peer.
An uncommitted channel should not keep the peer in the db, since the
uncommitted channel isn't in the db itself.

Fixes: #2367
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell 26df586fbf pytest: add test for db constraint failure
wallet_channel_insert: UNIQUE constraint failed: peers.node_id
lightningd: Fatal signal 6 (version v0.6.3rc1-202-g4c8cb98)
0x555625b14261 crashdump
	common/daemon.c:40
0x7f8f7400e0ff ???
	???:0
0x7f8f7400e077 ???
	???:0
0x7f8f73fef534 ???
	???:0
0x555625af34f2 fatal
	lightningd/log.c:624
0x555625b3dd96 db_exec_prepared_
	wallet/db.c:448
0x555625b44fcd wallet_channel_insert
	wallet/wallet.c:1067
0x555625af64f7 wallet_commit_channel
	lightningd/opening_control.c:229
0x555625af6e86 opening_funder_finished
	lightningd/opening_control.c:394
0x555625af7bfa openingd_msg
	lightningd/opening_control.c:714
0x555625b0d421 sd_msg_read
	lightningd/subd.c:474
0x555625b0cd3b read_fds
	lightningd/subd.c:302
0x555625b548d3 next_plan
	ccan/ccan/io/io.c:59
0x555625b553ef do_plan
	ccan/ccan/io/io.c:395
0x555625b5542d io_ready
	ccan/ccan/io/io.c:405
0x555625b5700e io_loop
	ccan/ccan/io/poll.c:310
0x555625af1562 main
	lightningd/lightningd.c:827
0x7f8f73ff109a ???
	???:0
0x555625adaad9 ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20190220020526

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Conor Scott af750d802d Switch to libsodium-dev for ubuntu install instructions 2019-02-21 09:14:39 +00:00
Rusty Russell 38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 28f5da7b2f tools/generate-wire: use amount_msat / amount_sat for peer protocol.
Basically we tell it that every field ending in '_msat' is a struct
amount_msat, and 'satoshis' is an amount_sat.  The exceptions are
channel_update's fee_base_msat which is a u32, and
final_incorrect_htlc_amount's incoming_htlc_amt which is also a
'struct amount_msat'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell bac9a594b8 wallet: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 3ac0e814d0 daemons: use amount_msat/amount_sat in all internal wire transfers.
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 0d30b89043 channeld: use amount_msat for struct htlc amount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 3412c5d392 commit_tx & htlc_tx: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell bb00deeea4 channeld: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 93dcd5fed7 channeld: avoid overflow in reloading of channel from db.
We used to just throw htlcs into the channel with a flag to tell it to
ignore overflow.  Instead, we can insert them in order (which is the same as
id order) which always must be valid.

This helps when we turn the balance into a struct amount_msat which will get
upset with overflows.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell b8e484b508 struct channel_config: use amount_sat / amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell 85b8b25749 bitcoin/chainparams: use amount_sat / amount_msat
Simple changes, but ripples through the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell cd341b34d6 plugins/pay: use struct amount_msat.
This is particularly interesting because we handle overflow during route
calculation now; this could happen in theory once we wumbo.

It fixes a thinko when we print out routehints, too: we want to print
them out literally, not print out the effect they have on fees (which
is in the route, which we also print).

This ABI change doesn't need a CHANGELOG, since paystatus is new since
release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell cc95a56544 pylightning: handle msat fields in JSON more appropriately.
Little point having users handle the postfixes manually, this
translates them, and also allows Millisatoshi to be used wherever an
'int' would be previously.

There are also helpers to create the formatting in a way c-lightning's
JSONRPC will accept.

All standard arithmetic operations with integers work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 52c843f708 CHANGELOG, documentation: update changelog to reflect suffix changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 7e3928359a listpeers: add all the alternate "msat" and "sat" fields for channels.
These are undocumented, unfortunately, but at least that means I don't
have to update the docs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell b5dcb93e5f wallet: use amount_msat / amount_sat.
We change struct utxo to use amount_sat, and paper over the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 83adb94583 lightningd and routing: use struct amount_msat.
We use it in route_hop, and paper over it in the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell feb92cf4e2 sendpay: allow 'amount_msat'
We're about to add 'amount_msat' to getroute, but it's common to feed
'getroute' back into 'sendpay', so sendpay should allow it.

If both are specified, make sure they're the same!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 2632cc3f34 lightningd/json: make wallet_tx functions take amount_sat.
Using param_tok is generally deprecated, as it doesn't give any sanity checking
for the JSON 'check' command.  So make param_wtx usable directly, and
also make it have a struct amount_sat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 3ba544bfde common/bolt11: use struct amount_msat
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 28ec65fd79 lightningd: add json_add_amount_msat and json_add_amount_sat helpers.
These create two fields, one old one which is purely numeric,
and a modern on with a suffix, eg "msatoshi" and "amount_msat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 177cfd9edc common: json_to_msat and json_to_sat helpers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell a25e22737c common/json_tok: add param_msat / param_sat.
The current param_sat accepts "any": rename and move that to invoice.c
where it's called.  We rename it to param_msat_or_any and invoice.c
is our first (trivial) amount_msat user.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell 023923e6a8 amount: minor comment and text improvements and remove unused function.
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 1d1fcc41b8 db: add amount functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 7fad7bccba common/amount: new types struct amount_msat and struct amount_sat.
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.

The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 269dbe585c wire: move short_channel formatting functions into bitcoin/short_channel_id
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell ebca6e1ea3 update-mocks: make sure we can find json_add functions.
These are on start of line, which is unexpected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 175c869b6b update-mocks: handle NO_NULL_ARGS and NON_NULL_ARGS functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 1c542bcd08 pylightning: prepare plugin for parameters to getmanifest.
We might still add these in future, so I think we should allow for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 53423e8a55 lightningd: re-enable IO logging for JSON output.
Hex format is terrible, but sometimes it's the only way to tell WTF is
going on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell 19d13f1781 type_to_string: return const char *.
Always be const if you can.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00