Commit Graph

7837 Commits

Author SHA1 Message Date
lisa neigut 964a3583c4 gitignore: ignore release directory
this is where built binaries are sent; we don't need it in git
2020-05-04 10:21:07 +09:30
lisa neigut 04117915fe gitignore: ignore external lib libbase58 2020-05-04 10:21:07 +09:30
lisa neigut 261a46449c wire: suppress printout from patch command
We were using patch's '--silent' flag, but that broke on the busybox
implementation of `patch`, since they don't support it. Instead, we use
the universally supported "pipe to /dev/null" approach

Suggested-By: @rustyrussell
Changelog-None
2020-05-04 10:21:07 +09:30
lisa neigut 3cfafa81eb bcli-bugfix: pass along entire script
sizeof doesn't give the correct length, instead use tal_hex which will
measure the script length correctly

Changelog-None
2020-05-02 12:39:16 +02:00
Christian Decker 64b0487228 pyln-testing: Just warn if we get an unexpected fee request
This was breaking a couple of tests if the pyln version was not synced up with
`lightningd`, so now we just warn (these are then collected when running in
pytest and highlighted).
2020-05-01 18:20:33 -05:00
Christian Decker e7b555dd90 pytest: Stabilize test_penalty_{in,out}htlc tests
They were looking for specific amounts which proved to be rather flaky. Now
they look for specific outputs being available in the `listfunds` result after
everything was settled.
2020-05-01 18:20:33 -05:00
Antoine Poinsot 3fc77730be bcli: don't log failed sendrawtransaction attempts twice
They are already logged both in bcli, and in lightningd.
This just adds a lot of noise to the logs. We keep successed attempts
though for the tests.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-04-30 19:27:54 -05:00
Antoine Poinsot 678591d851 lightningd/bitcoind: always die if the Bitcoin backend died
This in addition removes the init fixed timeout hack.

Changelog-fixed: We now *always* die if our Bitcoin backend failed unexpectedly.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-04-30 19:27:54 -05:00
Antoine Poinsot 7ea1a8a182 lightningd/bitcoind: remove an outdated comment
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-04-30 19:27:54 -05:00
Aiden McClelland c485cf7dd1 docs: Add documentation for Alpine to INSTALL.md
Changelog-Added: docs: Install documentation now has information about building for Alpine linux
2020-04-30 19:27:39 -05:00
lisa neigut 67703823c5 0.8.2 Changelog Update
0.8.2, we're doing it
2020-04-30 15:15:46 -05:00
lisa neigut f637b88932 v0.8.2rc3 changelog updates
We're doing an RC3 folks!
2020-04-28 15:08:46 -05:00
lisa neigut f2aa770a3e tests: since features are longer now, so bytes read is longer
Since the node announcements now include the 55th bit flag for keysends,
the total amount of bytes read from disk is now +8
2020-04-28 10:39:07 -05:00
lisa neigut d6d26dd056 features: split expected feature bits into node/peer sets
The new `keysend` plugin modifies the node features that we send to
peers. This commit breaks out the 'expected_features' we use for tests
to encompass this differentiation.
2020-04-28 10:39:07 -05:00
Christian Decker 9f298330b2 libplugin: It's featurebits, not features
The documentation was wrong, and I copied my mistake to `libplugin` where it
was then ignored instead of ORed into the node's featurebits. This fixes both.
2020-04-28 10:39:07 -05:00
Christian Decker 8c23f22591 pytest: Reproduce issue #3666
It seems we aren't registering the featurebits correctly.
2020-04-28 10:39:07 -05:00
Antoine Poinsot 6e5aa1af14 doc: bitcoin backend must return sat/kVB not btc/kVB on `estimatefees`
Reported-by: SimonVrouwe <s_github@protonmail.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-04-27 18:41:22 -05:00
lisa neigut 10f47b41fa changelog: update to rc2
Changelog-None
2020-04-24 14:23:42 -05:00
lisa neigut c7da7ca9f0 feerates: de-dupe min_ max_ printing twice when `feerates` called
We want to compare to `i`, the enum counter, not feerates[i], the
feerate value.

Without this fix, `feerates` appears as:
```
{
   "perkw": {
      "opening": 253,
      "mutual_close": 253,
      "unilateral_close": 253,
      "delayed_to_us": 253,
      "htlc_resolution": 253,
      "penalty": 253,
      "min_acceptable": 253,
      "max_acceptable": 2500,
      "min_acceptable": 253,
      "max_acceptable": 4294967295,
      "urgent": 253,
      "normal": 253,
      "slow": 506
   },
   "onchain_fee_estimates": {
      "opening_channel_satoshis": 177,
      "mutual_close_satoshis": 170,
      "unilateral_close_satoshis": 151,
      "htlc_timeout_satoshis": 167,
      "htlc_success_satoshis": 177
   }
}
```

bug introduced in "chaintopology: better feerate targets differentiation"
2020-04-24 14:21:27 -05:00
Rusty Russell fc8672061f lightningd: fix false positive on leak detection.
Commit 9aedb0c61f changed this from allocating off `c` to allocating
off NULL, knowing that it's tal_steal() in the callback.  But before
that, it can be detected as a mem leak:

```
    @pytest.fixture
    def teardown_checks(request):
        """A simple fixture to collect errors during teardown.

        We need to collect the errors and raise them as the very last step in the
        fixture tree, otherwise some fixtures may not be cleaned up
        correctly. Require this fixture in all other fixtures that need to either
        cleanup before reporting an error or want to add an error that is to be
        reported.

        """
        errors = TeardownErrors()
        yield errors

        if errors.has_errors():
            # Format a nice list of everything that went wrong and raise an exception
            request.node.has_errors = True
>           raise ValueError(str(errors))
E           ValueError:
E           Node errors:
E           Global errors:
E            - Node /tmp/ltests-iz9y1chb/test_hsmtool_secret_decryption_1/lightning-1/ has memory leaks: [
E               {
E                   "backtrace": [
E                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E                       "lightningd/jsonrpc.c:848 (parse_request)",
E                       "lightningd/jsonrpc.c:941 (read_json)",
E                       "ccan/ccan/io/io.c:59 (next_plan)",
E                       "ccan/ccan/io/io.c:407 (do_plan)",
E                  avis/build/ElementsProject/lightning/lightningd/../plugins/pay
```

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-21 13:02:42 -05:00
lisa neigut 6b2f6612e2 changelog: use correct format for label 2020-04-20 13:48:11 -05:00
Christian Decker fb8661714e wallet: Add a gap limit when checking for incoming transactions
Changelog-Added: wallet: The wallet now has a gap limit that is used to check for incoming transactions when scanning the blockchain.
2020-04-20 07:48:32 +09:30
Christian Decker 264b2d1975 pyln-testing: Add more meaningful error for mock_estimatefees
We only support a very limited number of argument combinations, and apparently
sometimes we trigger a case we aren't handling. This adds a more useful error
message, including the params we didn't match.
2020-04-20 07:48:32 +09:30
Christian Decker f1633de6e8 plugin: Add the keysend plugin to the list of installable plugins
Suggested-by: Matt Whitlock <@whitslack>
Signed-off-by: Christian Decker <@cdecker>
2020-04-20 07:29:07 +09:30
lisa neigut 9b42d9d8ba build: add mkrd to docker install? 2020-04-19 08:07:47 +09:30
lisa neigut 251bae55d7 build: Update build-release.sh to remove i386 build
fixup! docker: Remove Dockerfile for i386 builder
2020-04-19 08:07:47 +09:30
lisa neigut 9663d110d1 build: make non-parallel
There's a race condition with loading the submodules that's causes a
build failure on my machine, since the libwally 'includes' aren't on
disk yet when the gcc build step starts.
2020-04-19 08:07:47 +09:30
lisa neigut 1b2f4990ab tests: on reconnect, we need to send any fulfills before adds
The peer will fail the channel on reconnect if we send an 'add' we
don't have balance for yet; we can avoid this issue by always sending
fulfills (+) before sending adds (-)
2020-04-19 08:07:47 +09:30
lisa neigut 8189e9a64d v0.8.2-rc1 Changelog updates
Changelog-None
2020-04-19 08:07:47 +09:30
Rusty Russell bf053dcf7c plugins: simplify htlc_accepted hook payload-setting API.
As discussed with Christian, prepending the length to the payload returned
is awkward, but it's the only way to set a legacy payload.  As this will
be soon deprecated, simplify the external API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 18:03:35 +09:30
Christian Decker de0fd1f274 keysend: Change the plugin to add an invoice and then continue
So far we were relying on `lightningd` to create an ad-hoc invoice when
telling it to `resolve` with a given preimage. We now switch to having the
plugin create the invoice, remove the mandatory `keysend_preimage`
field (which would upset `lightningd` otherwise), and then return the modified
payload with the instructions to `continue` instead of resolving.

This ties back in with the existing payment/invoice handling code. Invoices
are created only if we don't have a label clash (unlikely since we have the
nano-time in the label), or the `payment_hash` was already used for another
invoice (at which point `lightningd` will automatically reject the payment and
we're a bit poorer for it, but meh :-)
2020-04-16 18:03:35 +09:30
Rusty Russell 63441075b5 lightningd: allow htlc_accepted hook to replace onion payload.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-added: `htlc_accepted` hook can now offer a replacement onion `payload`.
2020-04-16 18:03:35 +09:30
Christian Decker f3315ca110 pyln-proto: Add Tu32Field and Tu64Field helpers
These are necessary for the interim keysend plugin
2020-04-16 18:03:35 +09:30
Christian Decker 97fb27650d keysend: Announce keysend featurebit in node_announcement
The featurebit isn't quite settled.
2020-04-16 18:03:35 +09:30
Christian Decker e03acd9663 libplugin: Add features to plugin_main and getmanifest 2020-04-16 18:03:35 +09:30
Christian Decker 59567dc04b plugin: Move feature_place enum to features.h
Better not duplicate these, we might end up mixing them.
2020-04-16 18:03:35 +09:30
Christian Decker b5b11a3f67 keysend: Ignore keysends if the payload contains unknown even fields
We must really make sure that we understand the entire payload, not just the
fields we are interested in.
2020-04-16 18:03:35 +09:30
Christian Decker 568773daad pytest: Add a test for keysend
It currently uses a borrowed sending implementation from the noise plugin, but
we'll implement that functionality in the native keysend plugin next.
2020-04-16 18:03:35 +09:30
Christian Decker 1b32cc1c73 wire: Add a function to serialize a raw set of TLV fields
The generated wrappers will ignore the raw fields and will only consider the
shortcut fields. This function takes the raw fields and serializes them
instead.
2020-04-16 18:03:35 +09:30
Christian Decker ef0f28cd50 plugin: Implement the ability to receive keysend payments
This still uses the experimental TLV-type, but once the type is standardized
we can add detection for the new type quite easily.

Changelog-Added: pay: The `keysend` plugin implements the ability to receive spontaneous payments (keysend)
2020-04-16 18:03:35 +09:30
Christian Decker 5be59f4759 plugin: Add keysend skeleton 2020-04-16 18:03:35 +09:30
Christian Decker 5b11bab876 plugin: Check that the preimage returned as resolution matches hash
The plugin can basically return whatever it thinks the preimage is, but we
weren't handling the case in which it doesn't actually match the hash. If it
doesn't match now we just return an error claiming we don't have any matching
invoice.
2020-04-16 18:03:35 +09:30
Christian Decker 27ea47ae37 plugins: Fix undefined deallocation order in `struct plugins`
We use the new function `plugins_free` to define the correct deallocation
order on shutdown, since under normal operation the allocation tree is
organized to allow plugins to terminate and automatically free all dependent
resources. During shutdown the deallocation order is under-defined since
siblings may get freed in any order, but we implicitly rely on them staying
around.
2020-04-16 18:03:35 +09:30
Rusty Russell 284bd7de0c pytest: fix flaky test_excluded_adjacent_routehint
We weren't actually waiting until l3 got the channel_update from l2,
so it might not be able to create the routehint.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-15 20:52:13 -05:00
Rusty Russell deac09950a plugins: make chained hooks have two different callbacks.
One is called on every plugin return, and tells us whether to continue;
the other is only called if every plugin says ok.

This works for things like payload replacement, where we need to process
the results from each plugin, not just the final one!

We should probably turn everything into a chained callback next
release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 9aedb0c61f plugin: simplify hooks calling methods, and make lifetime requirements explicit.
They callback must take ownership of the payload (almost all do, but
now it's explicit).

And since the payload and cb_arg arguments to plugin_hook_call_() are
always identical, make them a single parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 1e34d8989d utils: add marker for functions which take ownership of pointers.
We have several of these, and they're not always called obvious things like
"delete" or "free".  `STEALS` provides a strong hint here.

I only added it to a couple I knew about off the top of my head.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:40:39 +09:30
Rusty Russell 2c3543e42d lightningd: fix crash when failing htlc once channeld dies.
We were reaching through it to get `ld`, but channeld is already dead.
Caught this on test_onchaind_replay (without valgrind) on my test machine:

    INFO 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
    **BROKEN** lightningd: FATAL SIGNAL 11 (version f6e1735)
    **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x5634dc83dc55
    **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x5634dc83dca9
    **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fd4b7c0b46f
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:285 (failmsg_incorrect_or_unknown) 0x5634dc82625a
    **BROKEN** lightningd: backtrace: lightningd/htlc_set.c:109 (htlc_set_add) 0x5634dc801e5c
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:496 (handle_localpay) 0x5634dc826997
    **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1008 (htlc_accepted_hook_callback) 0x5634dc827e60
    **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:197 (plugin_hook_callback) 0x5634dc831ea1
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:261 (plugin_response_handle) 0x5634dc82d2c3
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:359 (plugin_read_json_one) 0x5634dc82d46f
    **BROKEN** lightningd: backtrace: lightningd/plugin.c:391 (plugin_read_json) 0x5634dc82d5c6
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x5634dc896319
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x5634dc896efe
    **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x5634dc896f40
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 09:39:49 +09:30
Rusty Russell 1dc281c111 pytest: test for pay adjacent routehint crash.
When route returns a result which is too expensive, we try to figure out which
hop is most expensive to exclude it for next time.

If it's a single-hop route, we don't count it, since the first hop is free.
That's not usually a problem, since single-hop routes can't exceed our limits
(they're always "free"!).

But if we are using a routehint, the total cost could exceed our limits,
even if the start of the routehint is a single hop away.

This reproduces that test case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 14:13:26 -05:00
Rusty Russell df31af5221 invoice: ignore dead-end heuristic on explicitly specified channels.
This makes testing easier, and makes sense: lightningd might not
*know* about other connected channels, depending on gossip, but if the
user specifies it we should obey it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON: `invoice` `exposeprivatechannels` now includes explicitly named channels even if they seem like dead-ends.
2020-04-14 14:13:26 -05:00