Commit Graph

6279 Commits

Author SHA1 Message Date
darosior 0cd3823c98 libplugin: pass configuration to plugin's 'init' callback
So that a plugin can, for example, know if it has been loaded on startup
2019-08-03 13:15:40 +02:00
darosior 017ee7c916 libplugin: Add 'getmanifest' 'dynamic' field 2019-08-03 13:15:40 +02:00
Rusty Russell f57f068592 gossipd: don't use O_APPEND on the gossip_store.
We always know the length, so we don't need it.  It causes much extra work
when we want to delete a record, which I suspect may cause issues amongst
some users who've been seeing gossip_store corruption.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 12:50:51 +02:00
Rusty Russell 4c9bfa351a lightningd: handle --version before trying to move to lightning-dir.
Otherwise it creates the lightning-dir.  This can't be helped for --help
(at least, if plugins are present), but --version simply prints and exits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell fd63b8bf53 lightningd: chdir as soon as we know lightning dir.
This is easy since we did the option parsing cleanup, but it has the
effect that plugins are launched from the lightning-dir.  Now
we have dynamic plugins, this means startup and post-startup plugins
experience the same environment.

This is absolutely a desirable thing: they can just drop files in
their cwd rather than having to move (including, I might note, core
files!).

We also highlight the change in various places (and a drive-up update
of PLUGINS.md which says you have to use --plugin).

The next patch adds a backwards compatibility wedge for old users of
relative plugin paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell 4fcd94a091 pytest: use absolute paths for plugin arguments.
We're going to change this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell c24573b0d2 Makefile: set PYTHONPATH to be absolute when running tests.
My test machine started failing on dynamic plugin tests, unable to
find the lightning module:

  ModuleNotFoundError: No module named 'lightning'

This is because plugins at startup are run from whatever directory
you're in, but on refresh are run from the lightning-dir.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
lisa neigut c7f3fa34b2 funding tx: include segwit marker + flag in fee calculation
Noticed an off by one error when running tests for dual-funding;
we're not including the two 'header' segwit bytes in our weight
calculations.
2019-08-03 05:19:24 +00:00
lisa neigut b0b6ddb66f devtools: rm repetitive field (it's included in remotesecrets) 2019-08-03 05:19:24 +00:00
lisa neigut abcde461df devtools: print witnesses along with signatures for mkfunding
we need the witnesses for dual-funding transactions
2019-08-03 05:19:24 +00:00
Rusty Russell 913a1a9b59 bolt: update to 8b2cf0054660bece9e1004f42a500c6a1a77efd3
This contains only typo fixes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 8774070a31 bolt: Update to latest bolt, including TLV onion format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell fafd5ed4b0 wire/test/run-tlvstream.c: use generated defs for test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 4b5cd5ff84 tools/generate-wire.py: only use 'struct node_id' in announcements.
Naturally, it's a struct pubkey.  However, those are large, and take
time to marshal, so gossipd treats them as node_id which is a simple
array.  It adds explicit checks at the right points to make sure
they're valid pubkeys.

However, the next commit adds TLV test vectors, which assumes we treat
node_id as a point (thus catch invalid values when parsing).  The best
solution is to restrain our types here to exactly those we've
optimized for.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 00ca52ea07 bolt: update to aa33af0c4d7ae0180c04ef98e61af49c1f876a36
This introduces the TLV test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell b17b4c3eb6 wire: include definition of struct node_id.
Next BOLT update puts it in a TLV, so we need the definition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell b10e0e08bb tools/generate-wire.py: add option to expose tlv_record_type
Next update adds TLV test vectors: without this, we get a warning
about them being unused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 9b88fd4c60 bolt: update to 950b2f5481c2a4b57ef1102e2374543e81c4aa88
Just a simple field renaming which only alters comments,
though I updated variable names too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell 2b3003f25b channeld: delay sending channel_announcement by 60 seconds.
We currently send channel_announcement as soon as we and our
peer agree it's 6 blocks deep.  In theory, our other peers might
not have seen that block yet though, so delay a little.

This is mitigated by two factors:
1. lnd will stash any "not ready yet" channel_announcements anyway.
2. c-lightning doesn't enforce the 6 depth minimum at all.

We should not rely on other nodes' generosity or laxity, however!

Next release, we can start enforcing the depth limit, and maybe stashing
ones which don't quite make it (or simply enforce depth 5, not 6).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 16:50:45 +02:00
ZmnSCPxj 3e74ca4b86 gossipd/routing.c: Correctly handle a duplicated entry in `exclude` of `getroute`. 2019-08-02 16:06:15 +02:00
ZmnSCPxj a5fb37298c tests/test_gossip.py: Add test to check that duplicated exclusions in `getroute` have no lasting effect. 2019-08-02 16:06:15 +02:00
Rusty Russell 359433f374 lightningd: convert the compiler-wanted-init FIXME.
I'm sure there are others, but this was the only one which showed up
in grep.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 79345cc0b9 CCAN: update to suppress path_readlink warning.
Reported-by: Chirimen-Jako
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 4f89d9be86 cppcheck: catch json_list_for_each
cppcheck on bionic doesn't like this it seems.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 1510ea5717 wallet/walletrpc.c: fix uninitialized warning.
The withdraw_tx function shouldn't use it, but GCC is right it's uninitialized:

wallet/walletrpc.c: In function ‘json_prepare_tx’:
wallet/walletrpc.c:202:15: error: ‘changekey’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 296cfe8d1b wire/tlvstream: suppress gcc -O3 warning about prev_type.
Use a pointer, so it's explicit and gcc is happy.  We avoid the
allocation by pointing it to another stack var.

./wire/tlvstream.c:81:22: error: ‘prev_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 02609773c0 lightningd: suppress gcc-7.4.0 error
In file included from wallet/test/run-wallet.c:15:0:
./lightningd/peer_htlcs.c: In function ‘htlcs_reconnect’:
./lightningd/peer_htlcs.c:2060:15: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   }  else if (failcode) {
               ^~~~~~~~
./lightningd/peer_htlcs.c:2056:19: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
          failcode != 0
          ~~~~~~~~~^~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 7835a25121 common: helper to suppress 'may be used uninitialized' warnings.
When gcc still warns at -O3 and you are sure it's not necessary,
this marks it appropriately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell df298ba7e3 configure: suppress "maybe uninitialized" warnings altogether unless -O3.
We were just telling GCC not to treat them as errors: this suppresses them
entirely unless at -O3.  People keep trying to "fix" them, when in fact
they're false positives, as revealed with "./configure COPTFLAGS=-O3".

Fixes: #2856
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 5a3ba1ce99 travis: don't run full testsuite with gcc-4.8
Build test (non-developer) is probably enough.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 6a2a118467 travis: upgrade to bionic
This means we'll start enforcing no "maybe uninitialized" warnings at
-O3, since xenial was using gcc 5.4 or gcc 4.8 which are too primitive.

Seems like `sudo: false` is deprecated (those deps weren't being
installed); you simply install in the `before_install` hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell 641876a20a ccan: update to latest.
configurator failed under clang:

checking for #pragma omp and -fopenmp support... ccan/tools/configurator/configurator: Test for HAVE_OPENMP failed with 32512:
./configurator.out: error while loading shared libraries: libomp.so: cannot open shared object file: No such file or directory

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
trueptolemy 5c153de81e CHANGELOG: Add the record for `forward_event` 2019-08-01 18:49:25 +08:00
trueptolemy 7b59431a6c doc: Add the description for 'forward_event' notification 2019-08-01 18:49:25 +08:00
trueptolemy 31e2e70f17 invoice: a cleanup for the json of struct sha256
Here should't be accessed directly to the underlying of struct sha256.
2019-08-01 18:49:25 +08:00
trueptolemy 5ac02907bb pytest: Add the test for 'forward_event' in test_plugin 2019-08-01 18:49:25 +08:00
trueptolemy d2423c6f9a pytest: Add a simple plugin to test 'forward_event' 2019-08-01 18:49:25 +08:00
trueptolemy 294f05dc98 Call notification 'forward_event' when forward payment status changes 2019-08-01 18:49:25 +08:00
trueptolemy e75d8e061b Plugin: New notification type, forward_event
`forward_event`

A notification for topic `forward_event` is sent every time the status
of a forward payment is set. The json format is same as the API
`listforwards`.

```json
{
  "forward_event": {
  "payment_hash": "f5a6a059a25d1e329d9b094aeeec8c2191ca037d3f5b0662e21ae850debe8ea2",
  "in_channel": "103x2x1",
  "out_channel": "103x1x1",
  "in_msatoshi": 100001001,
  "in_msat": "100001001msat",
  "out_msatoshi": 100000000,
  "out_msat": "100000000msat",
  "fee": 1001,
  "fee_msat": "1001msat",
  "status": "settled",
  "received_time": 1560696342.368,
  "resolved_time": 1560696342.556
  }
}
```
or

```json
{
  "forward_event": {
  "payment_hash": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "in_channel": "103x2x1",
  "out_channel": "110x1x0",
  "in_msatoshi": 100001001,
  "in_msat": "100001001msat",
  "out_msatoshi": 100000000,
  "out_msat": "100000000msat",
  "fee": 1001,
  "fee_msat": "1001msat",
  "status": "local_failed",
  "failcode": 16392,
  "failreason": "WIRE_PERMANENT_CHANNEL_FAILURE",
  "received_time": 1560696343.052
  }
}

```
 - The status includes `offered`, `settled`, `failed` and `local_failed`,
   and they are all string type in json.
   - When the forward payment is valid for us, we'll set `offered`
     and send the forward payment to next hop to resolve;
   - When the payment forwarded by us gets paid eventually, the forward
     payment will change the status from `offered` to `settled`;
   - If payment fails locally(like failing to resolve locally) or the
     corresponding htlc with next hop fails(like htlc timeout), we will
     set the status as `local_failed`. `local_failed` may be set before
     setting `offered` or after setting `offered`. In fact, from the
     time we receive the htlc of the previous hop, all we can know the
     cause of the failure is treated as `local_failed`. `local_failed`
     only occuors locally or happens in the htlc between us and next hop;
     - If `local_failed` is set before `offered`, this
       means we just received htlc from the previous hop and haven't
       generate htlc for next hop. In this case, the json of `forward_event`
       sets the fields of `out_msatoshi`, `out_msat`,`fee` and `out_channel`
       as 0;
       - Note: In fact, for this case we may be not sure if this incoming
         htlc represents a pay to us or a payment we need to forward.
         We just simply treat all incoming failed to resolve as
         `local_failed`.
     - Only in `local_failed` case, json includes `failcode` and
       `failreason` fields;
   - `failed` means the payment forwarded by us fails in the
     latter hops, and the failure isn't related to us, so we aren't
     accessed to the fail reason. `failed` must be set after
     `offered`.
     - `failed` case doesn't include `failcode` and `failreason`
       fields;
 - `received_time` means when we received the htlc of this payment from
   the previous peer. It will be contained into all status case;
 - `resolved_time` means when the htlc of this payment between us and the
   next peer was resolved. The resolved result may success or fail, so
   only `settled` and `failed` case contain `resolved_time`;
 - The `failcode` and `failreason` are defined in [BOLT 4][bolt4-failure-codes].
2019-08-01 18:49:25 +08:00
trueptolemy a449a91ae2 JSON: Warp the process of forward payment json object
Warp this process as a new function: 'void json_format_forwarding_object()'. This function will be used in 'forward_event' next, and can ensure the consistent json object structure for forward_payment between 'listforwards' API and 'forward_event' notification.
2019-08-01 18:49:25 +08:00
trueptolemy bcec6bb6cc API: 'listforwards' now include 'payment_hash' field
'payment_hash' can help users learn more about the forward payment.
2019-08-01 18:49:25 +08:00
trueptolemy ab60bcab7c wallet: Set the fee to 0 when forward doesn't have out channel with LOCAL_FAILED status 2019-08-01 18:49:25 +08:00
Rusty Russell 2255dd4dda
lightningd: avoid thundering herd on restart.
The reason lnd was sending sync error was that we were taking more than
30 seconds to send the channel_reestablish after connect.  That's
understandable on my test node under valgrind, but shouldn't happen normally.

However, it seems it has at least once,
(see https://github.com/ElementsProject/lightning/issues/2847)
: space out startup so it's less likely to happen.

Suggested-by: @cfromknecht
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-01 01:10:11 +00:00
Rusty Russell cc70b9c4ec wire: use common/bigsize routines
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell 2241e25cb4 test: move run-bigsize into common to match common/bigsize.
And make it use bigsize accessors directly, not via fromwire/towire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell 8349c1a0c2 bigsize: insist in minimal representation in bigsize_get().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell 3fa375881a bigsize: make it a proper first-class type.
It doesn't belong in bitcoin, and should not be confused with varint_t.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell e23f183468 common/sphinx: use bigsize_get not varint_get.
These are not the same if it's more than one byte.  Testing would have
caught this, I assume.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Simon Vrouwe 1ae8b73f0e doc: invoice manpage: add some details about route hint selection
Also add more detail to the warnings returned by the invoice command.
2019-07-31 23:25:37 +00:00