Commit Graph

6875 Commits

Author SHA1 Message Date
Rusty Russell 83575f27a1 seeker: add code to check range of scids.
Once we've finished streaming gossip from the first peer, we ask a
random peer (maybe the same one) for all short_channel_ids in the last
6 blocks from the latest channel we know about.

If this reveals new channels we didn't know about, we expand the probe
by a factor of 2 each time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 521c7f7121 seeker: take over gossip control.
The seeker starts by asking a peer (the first peer!) for all gossip
since a minute before the modified time of the gossip store.

This algorithm is enhanced in successive patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 55323ec385 gossipd: move gossip seeking routines into new file seeker.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 79ca9bf998 gossipd: use per-peer information to make messages clearer.
We can (usually) indicate what peer caused the bad gossip error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 0091300ee3 gossipd: track what peer gave us gossip msgs so we can credit it.
Since we have to validate, there can be a delay (and peer might
vanish) between receiving the gossip and actually confirming it, hence
the use of softref.

We will use this information to check that the peers are making progress
as we start asking them for specific information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell d5eca470dc utils: add set_softref() / clear_softref().
We often want a pointer which will turn to NULL if the pointed-to thing is
freed.  This is possible with tal objects, so create it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 296868daf4 gossipd: have gossip_store_load() return a timestamp.
This is the modified-time of the file.  We have to store it internally
since we overwrite the gossip file with compaction on startup.

This means the "are we behind on gossip?" heuristic is no longer inside
gossip_store.c, which is cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell d75302deba gossipd: random_peer() selector.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 8a3abdd5ff pytest: fix test_restart_many_payments
It sometimes fail with a bad_gossip error because the sending node
might not have found out about the channel when it gets a
channel_update.  Make sure the whole network knows everything before
we start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 213263ca3f pytest: filter out gossip_timetamp_filter should we receive it.
We completely rework test_node_reannounce: it's assumes we always ask for
all gossip and that assumption will be broken in future patches too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell a3f6ce1f8f pytest: add filters arg to query_gossip()
Code changes mean we're going to get gossip_timestamp_filter messages from
peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 9d336763ff devtools/dump-gossipstore: get offets correct when we have DELETED entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 247fb6fa6a common/gossip_store: fix lseek argument order on gossip_store reload.
We don't currently use this, but it's badly wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell 2b7e5f7f5a dev_disconnect: make it more reliable.
I have seen some strange flakiness (under VALGRIND), which I have
traced down to dev-disconnect "+" not working as expected.  In
particular, the message is not sent out before closing the fd.

This seems to fix it on Linux, though it's so intermittant that it's hard
to be completely sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
lisa neigut 6e7cfcc76e walletrpc: explicitly note we discard utx on unsuccessful broadcast
If we can't broadcast the transaction, you're going to have to start
over regardless.  It's a tal child of cmd, which we free here.
2019-10-10 05:57:45 +00:00
lisa neigut af4ffe5fcd dev-forget-channel: accept passing in channel_id
This patch adds a channel_id parameter to allow for specifying
channels that are lacking a short_channel_id.

Useful in the case where a peer has 1) multiple channels (ONCHAIN etc)
and 2) a channel where the funding transaction hasn't been
broadcast/mined.
2019-10-10 05:57:45 +00:00
lisa neigut cbfa045f91 tx: remove input size assertion
this is unnecessary, and actually severely limits the functionality
of `wally_tx_add_input`, which will expand the allocated input
length if there's not enough room for the additional input

```external/libwally-core/src/transaction.c
    if (tx->num_inputs >= tx->inputs_allocation_len) {
        /* Expand the inputs array */
        struct wally_tx_input *p;
        p = realloc_array(tx->inputs, tx->inputs_allocation_len,
                          tx->num_inputs + 1, sizeof(*tx->inputs));
        ...
        tx->inputs = p;
        tx->inputs_allocation_len += 1;
```
2019-10-10 05:57:45 +00:00
lisa neigut 3458ee2a6e openingd: small formatting cleanups 2019-10-10 05:57:45 +00:00
lisa neigut c6338573d6 nits: align codes 2019-10-10 05:57:45 +00:00
lisa neigut 496d2cae5f varint: Add helper function for getting varlen size
Needed to calculate somethings for building dual funding txs
2019-10-10 05:57:45 +00:00
lisa neigut cb2cad8c94 script: add helper for checking if a 'known type'
To be used for verifying remote inputs' validity
2019-10-10 05:57:45 +00:00
lisa neigut 6f85088b2b tools: add handy debug error 2019-10-10 05:57:45 +00:00
lisa neigut 3f1f075421 tools: add ability to wrap wire messages with ifs
Makes it possible to hide wire messages behind EXPERIMENTAL_FEATURES
flag.
2019-10-10 05:57:45 +00:00
lisa neigut 9e29a47cf8 contrib: add aliases for logs
Add aliases 'l1-log' and 'l2-log' to startup_regtest script
2019-10-10 05:57:45 +00:00
darosior fac5faacc0 Document hsm_secret encryption 2019-10-09 22:00:38 -05:00
darosior 1c27545f9e pytest: test hsm_secret encryption 2019-10-09 22:00:38 -05:00
darosior c9982f9f49 pytest: allow to set stdin, stdout and stderr at lightningd startup
And also allow to not wait for it to be started.
Passing stderr=subprocess.STDOUT can be useful to wait_for_log() also on
stderr messages.
2019-10-09 22:00:38 -05:00
darosior 6f9030168e lightningd: prevent hsmd from interpretating an encrypted hsm_secret as not encrypted 2019-10-09 22:00:38 -05:00
darosior d393cda215 hsmd: encrypt hsm_secret if 'lightningd' pass an encryption key
This splits maybe_create_hsm_secret() in two parts (either encrypted
or in clear) for clarity, and adds an encryption detection in load_hsm().
There are actually three cases if an encryption key is passed:
- There is no hsm_secret => just create it and store the encrypted seed
- There is an encrypted hsm_secret => the provided key should be able to
decrypt the seed, if the wrong key is passed libsodium will nicely error
and hsmd will exit() to not throw a backtrace (using status_failed() as for
other errors) at the face of an user who mistyped its password.
- There is a non-encrypted hsm_secret => load the seed, delete the
hsm_secret, create the hsm_secret, store the encrypted seed.
2019-10-09 22:00:38 -05:00
darosior 62896566d0 lightningd: pass the hsm_secret encryption key in hsm_init 2019-10-09 22:00:38 -05:00
darosior a698395bf0 lightningd: '--encrypted-hsm', a new startup option
Add a new startup option which will, if set, prompt the user for a
password to derive a key from. This key will later be used to encrypt
and/or decrypt `hsm_secret`.

This was made a noarg option even if it would have been preferable to
let the user the choice of how to specify the password. Since we have
to chose, better to not let the password in the commands history.
2019-10-09 22:00:38 -05:00
darosior fe2543d8dc daemons: initialize libsodium at setup
According to the doc (https://download.libsodium.org/doc):
"sodium_init() initializes the library and should be called before
any other function provided by Sodium. [...]
the function ensures that the system's random number generator has
been properly seeded.".
2019-10-09 22:00:38 -05:00
lisa neigut 2c69ece7fe close: update short help text
Update short help text to match undeprecated close API
2019-10-09 21:04:16 -05:00
trueptolemy 9d4cccda19 pytest: Add the test for compact of deprecated close style 2019-10-09 21:04:16 -05:00
trueptolemy bb5aa774e2 CHANGELOG: Add the entry about `destination` in `close` 2019-10-09 21:04:16 -05:00
trueptolemy e53d065230 doc: Update the doc about `destination` of `close` 2019-10-09 21:04:16 -05:00
trueptolemy 63af8632b7 pytest: Test `close` to the specified destination 2019-10-09 21:04:16 -05:00
trueptolemy 019c052123 JSON-API: Allow `close` channel to specified address
Command format: close id [unilateraltimeout] [destination]

Close the channel with peer {id}, forcing a unilateral
close after {unilateraltimeout} seconds if non-zero, and
the to-local output will be sent to {destination}. If
{destination} isn't specified, the default is the address
of lightningd.

Also change the pylightning:
update the `close` API to support `destination` parameter
2019-10-09 21:04:16 -05:00
trueptolemy 71b606e050 lightningd: Add a new field `shutdown_scriptpubkey[NUM_SIDES]`
`shutdown_scriptpubkey[REMOTE]` is original remote_shutdown_scriptpubkey;
`shutdown_scriptpubkey[LOCAL]` is the script used for "to-local" output when `close`. Add the default is generated form `final_key_idx`;

Store `shutdown_scriptpubkey[LOCAL]` into wallet;
2019-10-09 21:04:16 -05:00
trueptolemy a1204a0b94 Closed: Cleanup for `wire_closing_init`
`final_scriptpubkey` is repeated with `funding_pubkey[LOCAL]`.
2019-10-09 21:04:16 -05:00
trueptolemy 75e946d256 json: Move `param_bitcoin_address` from wallet/walletrpc.c to lightningd/json.c
It's a useful helper, and it will be used to prase address in `close` command.
2019-10-09 21:04:16 -05:00
trueptolemy 8346d5c353 josn: Move `json_tok_address_scriptpubkey` from lightningd/jsonrpc.c to lightningd/json.c
Its declaration is in lightningd/json.h, so we should move it to lightningd/json.c.
2019-10-09 21:04:16 -05:00
darosior 7fd2f6db6d gossipd: query_messages: fail the connection if peer says it does not have up-to-date infos
It is most likely not on the same network, and in any case not a good peer to gossip with.
2019-10-09 16:54:39 -05:00
darosior 2638947ddc gossipd: query_scid: respond with complete to 0 on wrong chain_hash 2019-10-09 16:54:39 -05:00
darosior d3c8225968 gossipd: add a BOLT#7 comment when wrong chain_hash in 'query_channel_range'
And correct some typos
2019-10-09 16:54:39 -05:00
trueptolemy d59015d4ca CHANGELOG: Add the entries about the new parameter name `amount` 2019-10-09 16:51:42 -05:00
trueptolemy e4cd5df06c doc: Use `amount` as the parameter name for `fundchannel` 2019-10-09 16:51:42 -05:00
trueptolemy fdb90a765e pytest: Test compact of the old-style `fundchannel_start` 2019-10-09 16:51:42 -05:00
trueptolemy d149ba2f3a JSON-API: `fundchannel_start` uses `amount` fieldname to replace `satoshi` 2019-10-09 16:51:42 -05:00
trueptolemy 25583ffe37 doc: Use `amount` as the parameter name for `fundchannel` 2019-10-09 16:51:42 -05:00