Commit Graph

172 Commits

Author SHA1 Message Date
Nick Mathewson 2e3780affb Enforce (and obey) clippy lints about exhaustive enums, structs.
These lints force us to declare our exported enums and
exhaustive-looking structs as non-exhaustive (so that we can add to
them in the future without breaking our API) or to explicitly
disable the warning for a given enum/struct (to say that we _intend_
for additions to be a breaking change).
2021-04-27 14:17:49 -04:00
Nick Mathewson 975374beb5 Fix some broken rustdoc links. 2021-04-18 17:55:43 -04:00
Nick Mathewson 92de7c58f0 Move timer functions into an extension trait. 2021-04-17 09:37:46 -04:00
Nick Mathewson 6878fe8336 Move around the public modules in tor_rtcompat. 2021-04-17 09:21:09 -04:00
Nick Mathewson 1d07132787 Remove all non-runtime methods in tor_rtcompat. 2021-04-16 17:15:55 -04:00
Nick Mathewson 8532cef9aa Add a "Runtime" parameter to all the manager types.
This is a big change, but it is a step towards our goal of removing
tor_rtcompat:: calls directly.
2021-04-16 16:12:08 -04:00
Nick Mathewson e0f4c222dd Move the "sleep()" function from task to timer. 2021-04-16 15:44:36 -04:00
Nick Mathewson d60cf6ace6 Fix a compilation problem with async_std.
Thank you, CI!
2021-04-13 17:11:43 -04:00
Nick Mathewson 9efe458ad6 Merge branch 'dirstate' 2021-04-13 10:49:15 -04:00
Nick Mathewson bec5fa3e9c Major revision on DirMgr logic -- almost a complete rewrite.
The big idea of this revision is to separate the code that knows
about doing downloads from the code that decides what to download.
Later, we can make a similar change for database access.  With these
changes together, we can make our code much more testable, and
eventually enable more download types in parallel.
2021-04-13 10:48:52 -04:00
David Goulet 37c5ae6440 fallbackdir: New list from April 2021
Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-04-12 10:12:03 -04:00
Nick Mathewson 48bcde631f DocQuery: sort requests before splitting. 2021-04-09 08:03:09 -04:00
Nick Mathewson b2d43507b0 Start moving responsibility for building requests into dirmgr
This will help with my planned "directory state" refactoring.
2021-04-09 08:03:09 -04:00
Nick Mathewson 6cc8e5e028 Remove a few unused dependencies 2021-04-05 08:20:06 -04:00
Nick Mathewson f664a76504 Upgrade rusqlite; remove needless usage. 2021-04-05 08:12:10 -04:00
Nick Mathewson 98c0ef3dd5 DirMgr::netdir() is no longer async
Since we moved to a non-async mutex(*), we no longer need to worry that
this function might need to suspend.

(*) This is _not_ safe in general, but it's okay in this case, since
we never suspend while holding that mutex: see shared_ref.rs.
2021-04-02 14:08:07 -04:00
Nick Mathewson c0cb790873 missing file. 2021-04-02 10:58:28 -04:00
Nick Mathewson 6daccf89fc Turn the shared network directory reference into its own type
Doing this will make it easier to use it from other parts of the
crate, and will make it more obvious that it's safe to use a regular
(not async) rwlock here.
2021-04-02 09:15:42 -04:00
Nick Mathewson 51584f1a87 Make ConsensusRequest take a flavor. 2021-04-01 13:20:29 -04:00
Nick Mathewson 8e9abe425a Add accessors for members of different directory requests
This will let us use these types both as client and server-side
implementations.

Making this change required me to change the download code to take
requests by reference.  (Sorry, David)
2021-04-01 12:31:34 -04:00
Nick Mathewson ca7e1a0de9 DirMgr: add text() and texts() functions to ask for document text
Also, start on a larger refactoring where we ask for documents by
their ID.
2021-03-31 12:41:56 -04:00
Nick Mathewson beb9eff27b Refactor sqlite.rs to know about consensus flavors.
These were stored in the database before, but they were hardwired in
the API.
2021-03-31 08:23:51 -04:00
Nick Mathewson 4a438afbd5 Use Arc::make_mut() to simplify the code a bit
Previously we used NetDir::extend here, which used try_unwrap.  But
it's much simpler to use the intended object for this purpose.
2021-03-30 16:05:59 -04:00
Nick Mathewson 8cf4778c71 Look for RdDigest in the right location 2021-03-30 15:05:23 -04:00
Nick Mathewson 565e48ff8c small tweaks to routerdesc storage code. 2021-03-30 14:57:11 -04:00
David Goulet 2d43bdd561 dirmgr: Add functions to store/read/update router descriptors
Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-03-30 14:44:42 -04:00
David Goulet f7d1e515b9 dirmgr: Add SQLite schema for router descriptors
Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-03-30 14:23:49 -04:00
Nick Mathewson d6d78fdcc3 Make reasonable set of default fallback directories 2021-03-30 12:44:38 -04:00
Nick Mathewson 957701e912 Give the authorities value a reasonable default. 2021-03-30 12:20:39 -04:00
Nick Mathewson 95ea804636 Add a NetDirConfig::use_default_cache_path() function. 2021-03-30 12:08:03 -04:00
Nick Mathewson 6890f18094 resolve a clippy issue 2021-03-30 12:05:59 -04:00
Nick Mathewson d7fff75934 Declare a better default for DownloadScheduleConfig. 2021-03-30 11:48:43 -04:00
Nick Mathewson 3ad503866d Fix Rust-1.51 clippy warnings about acronyms in camel case.
This is painful, but we shouldn't have to do it again.
2021-03-29 16:33:48 -04:00
Nick Mathewson 3800c459f6 Rename Rsa{Identity,Signature} to fix clippy warning. 2021-03-29 16:14:43 -04:00
Nick Mathewson 50ba1cb85c DirMgr: cooperative multiprocess support.
A directory manager can now launch or run in read-only or read-write
mode, depending on whether it manages to acquire a lock on the
filesystem.

In read-write mode, it bootstraps and stores data into the database
as usual.

In read-only mode, it assumes that another process will be updating
the database, and so it only loads it periodically, looking for new
information.  However, it also tries to acquire the lock itself, so
it can enter read-write mode.
2021-03-29 15:37:43 -04:00
Nick Mathewson 87d191c4b8 Create a DirMgr::load_or_bootstrap_from_config. 2021-03-29 14:12:59 -04:00
Nick Mathewson 1bd110bf11 Add support for a lockfile to control concurrent access to a dirmgr. 2021-03-29 13:58:49 -04:00
Nick Mathewson 83ebd56f64 Add DirMgr::load_once.
This function loads a current bootstrapped directory from disk, if
it can.
2021-03-29 12:23:47 -04:00
Nick Mathewson e226a57869 Use the CircMgr in the DirMgr for all our downloading needs. 2021-03-29 12:13:31 -04:00
Nick Mathewson efe99633a5 Make CircMgr an optional element of DirMgr. Not yet used. 2021-03-29 12:00:33 -04:00
Nick Mathewson 603b0436e0 Add a compatibility layer so we can upgrade rand_core.
dalek-crypto is stuck on rand_core 0.5.1, so we've been stuck too.
This commit introduces a compatibility module so that we can wrap
new rand_core instances to make them backward compatible.
2021-03-18 13:16:32 -04:00
Nick Mathewson 6daf57957d Add the tor project as an author. 2021-03-17 14:43:40 -04:00
Nick Mathewson ba14d43be4 Add keywords to each Cargo.toml 2021-03-17 14:41:02 -04:00
Nick Mathewson c8481a9354 Add a description field to all our Cargo.toml files 2021-03-17 14:31:59 -04:00
Nick Mathewson 58d8472ea0 Give it a homepage everyplace. 2021-03-17 14:13:26 -04:00
Nick Mathewson 695268019e upgrade async-trait 2021-03-09 09:43:00 -05:00
Nick Mathewson 302653977d Bump dependencies with "cargo upgrade" 2021-03-06 08:55:13 -05:00
Nick Mathewson d7c1d4166a Avoid unwrap() in dirmgr. 2021-03-04 14:31:28 -05:00
Nick Mathewson 8bbf093e34 Avoid unwrap() in dirmgr 2021-03-04 13:24:00 -05:00
Nick Mathewson 8c31418500 Port to work with tokio or async-std.
This is fairly ugly and I think I'll need to mess around with the
feature configuration a while until we get something that's pleasant
to develop with.  This still seems like a good idea, though, since
we _will_ need to be executor-agnostic in the end, or we'll have no
way to handle wasm or embedded environments.

Later down the road, we'll probably want to use futures::Executor or
futures::Spawn more than having global entry points in
tor_rtcompat.  That would probably make our feature story simpler.

Tokio is the default now, since tokio seems to be more heavily used
for performance-critical stuff.

This patch breaks tests; the next one will fix them, albeit
questionably.
2021-03-02 11:55:09 -05:00
Nick Mathewson 568567c5e0 Update to latest futures, once_cell 2021-02-23 15:54:29 -05:00
Nick Mathewson ab9d4a2229 Update some dependencies. 2021-02-19 09:39:23 -05:00
Nick Mathewson afdcd69e03 Make first-level directory retry logic configurable. 2021-02-17 11:55:39 -05:00
Nick Mathewson d8d2c0f79c Make our initial bootstrap retry schedule configurarable 2021-02-17 11:25:57 -05:00
Nick Mathewson bd6625174e tor-dirmgr: simplify config.rs a bit. 2021-02-16 13:37:20 -05:00
Nick Mathewson ddf994acc5 Add support to override network parameters in the configuration. 2021-02-16 13:29:46 -05:00
Nick Mathewson 94901ec355 Apply "deny_unknown_fields" to configuration types. 2021-02-12 11:26:00 -05:00
Nick Mathewson 209d1df242 Remove fallback to .arti/cache for cache directory 2021-02-12 10:59:06 -05:00
Nick Mathewson aeb410b915 Make authorities and fallbacks configurable.
This commit adds configuration options for these values, with the
right defaults, and uses those options instead of built-in functions
to set them.

We also remove the function to extract information from chutney
directories: now that arti is configurable, it can be chutney's job
to make its own network configurations.
2021-02-11 13:08:46 -05:00
Nick Mathewson 243880473f bump serde and log 2021-02-01 16:39:07 -05:00
Nick Mathewson c5799e136f Check declared post-digest on consensus diffs. 2021-01-25 16:51:56 -05:00
Nick Mathewson e11bb47d5b Don't give up easily on initial bootstrap attempts. 2021-01-25 16:23:43 -05:00
Nick Mathewson 4ba7a89a3f Resolve a few more XXX-A1 comments in dirmgr. 2021-01-25 16:12:56 -05:00
Nick Mathewson ca8646bc0c Remove some not-necessary XXX-A1s in sqlite.rs 2021-01-25 14:23:33 -05:00
Nick Mathewson 88e9db7bc2 dirmgr: fix usage and meaning of `pending`.
The documentation and implementation diverged; the documentation was
correct.
2021-01-25 14:19:04 -05:00
Nick Mathewson c16412e372 Detect consensuses that are signed by the wrong authorities.
With this patch we don't consider a consensus to be even potentially
well-signed if the authorities that are listed as signing it don't
contain enough authorities we believe in.  Otherwise, we'd just try
fetching certs for them and failing forever.

(I found this by switching from a chutney network to the main
network without cleaning out my cache.)

Closes #44
2021-01-25 14:08:47 -05:00
Nick Mathewson 1a497b04db dirmgr: Extra explanatory comments.
Also replace some "disk" with "cache" in log messages
2021-01-25 10:32:25 -05:00
Nick Mathewson 3f455e6f30 require latest futures crate 2021-01-15 17:05:05 -05:00
Nick Mathewson fa57fb4995 Upgrade a few more dependencies. 2021-01-13 08:17:14 -05:00
Nick Mathewson 7e843de2c8 Upgrade a few dependencies. 2021-01-13 08:14:28 -05:00
Nick Mathewson f01d641ec8 cargo upgrade. 2020-12-30 15:59:33 -05:00
George Kadianakis 4962128b0e dirmgr: Handle another error in download_mds().
Co-authored-by: David Goulet <dgoulet@torproject.org>
2020-12-23 15:38:16 +02:00
George Kadianakis a02de2010e dirmgr: Handle some errors in download_mds().
Co-authored-by: David Goulet <dgoulet@torproject.org>
2020-12-23 15:37:44 +02:00
Nick Mathewson ccd6cbf627 upgrade to latest anyhow 2020-12-21 10:21:24 -05:00
Nick Mathewson f1eba83ebd Fix a few clippy warnings 2020-12-17 15:23:55 -05:00
Nick Mathewson 31aa73c022 Download and use consensus diffs.
Had to refactor a few methods for database access, and fix a bug
where we were storing valid_until times with the wrong information.
2020-12-17 15:08:53 -05:00
Nick Mathewson a8a35c9914 dirmgr: Refactor so it has no unusable state.
With this new API, if you have a DirMgr, it has a NetDir that was
bootstrapped, and was live at least once during your program's
execution.
2020-12-15 11:57:24 -05:00
Nick Mathewson 863f459389 Add a "RetryError" to capture the idea of multiple failed attempts.
When we try to do something a few times and it fails each time, it
can be a good idea to remember why the individual failures
happened.
2020-12-12 12:54:00 -05:00
Nick Mathewson cfdedb995f upgrade rusqlite 2020-12-08 17:26:49 -05:00
Nick Mathewson 17c84fd59e upgrade to require latest anyhow. 2020-12-08 17:26:19 -05:00
Nick Mathewson 0caee4718f Document a problem on network transition 2020-12-08 16:59:16 -05:00
Nick Mathewson 8d6087426e De-parameterize ChanMgr and everything that wraps it.
This makes a whole lot of our code simpler, and makes it so that
CircMgr and DirMgr no longer need to have anything parameterized
over transports, either.

Instead of boxing Transport inside of ChanMgr, I've made a new
Connection trait that goes from a ChanTarget* straight to a Channel.
This lets us avoid having to box the intermediate TLS object.

[*] Actually, a copy of the information from a ChanTarget. Ick, but
I had to make a copy to avoid parameterizing
Connecter::build_channel.
2020-12-08 16:29:18 -05:00
Nick Mathewson bdc7908c7f Fix a logic bug in the --chutney-dir option.
It was recording the dirport for each chutney authority as though it
were an orport.  That obviously won't work.

Closes arti#45.
2020-12-07 16:54:18 -05:00
Nick Mathewson a52585d9d6 Mark some XXXX issues as XXXX-A1. 2020-12-07 16:07:59 -05:00
Nick Mathewson 0f963774db Fix grammar in a log message 2020-12-07 10:21:01 -05:00
Nick Mathewson 6f1e05a1d3 Add a few log messages about download timing. 2020-12-07 09:24:51 -05:00
Nick Mathewson 57eea69f52 Teach updater to download microdescriptors for usable directory.
This follows a three-phase process: We are either fetching
microdescriptors, waiting for the time to download the next
consensus, or fetching the next consensus and making it usable.

We can stop fetching microdescriptors for two reasons: by having no
more mds that we need to download, or by running out of time in
which the current consensus is usable.
2020-12-07 08:29:35 -05:00
Nick Mathewson 915515442f Reuse microdescriptors from previous consensus
Previously, we'd re-parse them from the database every time we got a
new consensus.
2020-12-06 11:42:50 -05:00
Nick Mathewson 6ddf55f799 Enforce that the periods in a consensus lifetime are nonempty.
To be valid, a lifetime must have valid_after < fresh_until <
valid_until.
2020-12-04 15:00:06 -05:00
Nick Mathewson 402d501020 Update a few comments. 2020-12-04 14:03:05 -05:00
Nick Mathewson b56223f382 Dirmgr: Add a feature to keep the directory up-to-date.
This required me to move the sqlite connection from a rwlock into mutex,
since it isn't actually Sync.
2020-12-04 13:53:51 -05:00
Nick Mathewson a72437a1c7 Abandon directory circuits on most kinds of directory failure.
(There are more cases that this _doesn't_ handle, but it's a start
for now.)
2020-12-02 09:46:35 -05:00
Nick Mathewson db55c84c89 tor-dirmgr: retry up to 3 times on download failure. 2020-12-01 09:38:59 -05:00
Nick Mathewson 85cf6c09b3 Make mmap usage optional. 2020-12-01 07:56:36 -05:00
Nick Mathewson d7b79095c8 tor-dirmgr: tests for storage.rs 2020-12-01 07:44:47 -05:00
Nick Mathewson b0fc36c06a Add a decorrelated-jitter implementation.
This is the algorithm we use on download failure to back off before
we retry.
2020-11-30 16:43:36 -05:00
Nick Mathewson ffd4ed16a0 only expire dorectory info on a successful bootstrap. 2020-11-30 12:54:56 -05:00
Nick Mathewson a1d02d4154 Add tests for blob_fname failing cases 2020-11-30 12:35:33 -05:00
Nick Mathewson 39bc39ddd5 More tests for sqlite.rs 2020-11-30 12:26:11 -05:00
Nick Mathewson c3c3d24973 Initial tests for authcert storage 2020-11-30 11:59:41 -05:00