Commit Graph

547 Commits

Author SHA1 Message Date
Ian Jackson c6234428e3 bridge descriptor dormancy: Accept the dormancy value
But right now, don't do anything with it.  That will come in a future MR.
2022-11-11 15:34:35 +00:00
Ian Jackson eeb6d52678 bridge descriptor dormancy: Add a TODO for this enum 2022-11-11 15:26:42 +00:00
Nick Mathewson 936920b26e bump rust-version to 1.60 in every crate. 2022-11-10 10:57:33 -05:00
Ian Jackson eacb5dc2e3 bridge descriptor dormancy: Define an enum and semantics
This is part of #630.
2022-11-10 15:17:36 +00:00
Nick Mathewson d9788c16b3 Merge branch 'bridge-error' into 'main'
Use ErrorKind::TorAccessFailed for misbehaving bridges

See merge request tpo/core/arti!842
2022-11-10 13:50:47 +00:00
Nick Mathewson 3c29dfd99d Merge branch 'bdm-process-doc-test' into 'main'
bridge descs: Add tests for process_document

See merge request tpo/core/arti!841
2022-11-10 13:50:23 +00:00
Nick Mathewson 6454896d95 Merge branch 'bdm-params' into 'main'
Remove TODO re BridgeDescDownloadConfig defaults

See merge request tpo/core/arti!839
2022-11-10 13:49:54 +00:00
trinity-1686a 614cf34284 Check more targets 2022-11-10 13:34:17 +00:00
Ian Jackson 42dd5d7746 bridge desc: Drop a done TODO
We have a separate BridgeDescMgr, rather than reusing DirMgr, so we
will not be implementing BridgeDescProvider for DirMgr.
2022-11-08 18:54:31 +00:00
Ian Jackson 8fbff3a27f Use ErrorKind::TorAccessFailed for misbehaving bridges
(Use semantic linefeed for the doc comment.)
2022-11-08 18:52:57 +00:00
Ian Jackson d808f70a13 bridge descs: Add tests for process_document 2022-11-08 18:42:34 +00:00
Ian Jackson f1f4ca56a8 Remove TODO re BridgeDescDownloadConfig defaults 2022-11-08 17:34:25 +00:00
Ian Jackson 05cad7c0eb bridge desc: todo re config: weaken and reference a new ticket 2022-11-08 17:34:13 +00:00
Nick Mathewson ec9e9c86e7 Merge branch 'store' into 'main'
Persistently cache bridge descriptors

Closes #619

See merge request tpo/core/arti!831
2022-11-08 15:10:21 +00:00
Ian Jackson ea369e8385 bridge descs: Rename BridgeDescMgr from BridgeDescManager
This is more consistent with our naming elsewhere.

Suggested-by: Nick Mathewson <nickm@torproject.org>
2022-11-08 14:40:28 +00:00
eta 53f62eb8be Merge branch 'store-yak-misc' into 'main'
Small misc yaks for bridge descriptor persistence

See merge request tpo/core/arti!828
2022-11-08 14:28:14 +00:00
Ian Jackson d9d9cdeac9 bridge descs: Apply sensitive() to bridge identity in error log msg 2022-11-08 13:19:42 +00:00
Ian Jackson 93a150229a sqlite cache: Document reasoning about timestamps
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/831#note_2851764
2022-11-08 13:00:51 +00:00
Dimitris Apostolou 9c7d1802f5
Fix typos 2022-11-06 06:51:58 +02:00
Ian Jackson 5b66a5e199 bridge desc tests: Test caching and if-modified-since 2022-11-04 19:08:32 +00:00
Ian Jackson 78f7916a88 bridge desc tests: Break out queues_are_empty helper 2022-11-04 19:08:32 +00:00
Ian Jackson 40c21b7d19 bridge desc tests: Provide a sqlite connection
The current test case doesn't use this.
2022-11-04 19:08:32 +00:00
Ian Jackson 636bff3193 bridge desc: Implement storage functions at the DB layer 2022-11-04 19:08:32 +00:00
Ian Jackson 6bb2eb9902 bridge desc: Make Downloaded be Debug 2022-11-04 19:08:32 +00:00
Ian Jackson 5bc1d40a2b bridge desc: Implement cache reuse logic and if-modified-since
This introduces the new API functions to Store.
But currently they are all no-ops.
So all this machinery doesn't actually do anything.

Also, it changes the API to the mockable downloader, to allow it to
support if-modified-since.  So this isn't used either.  But it is more
convenient to do this all at once in BridgeDescManager, since care
needs to be taken about the intraction between if-modified-since and
the persistent cache.
2022-11-04 19:08:32 +00:00
Ian Jackson 279a169031 bridge desc: Rename one more variable to text 2022-11-04 19:02:23 +00:00
Ian Jackson eaaa61a8e2 bridge desc: Catch panics in the download functionality 2022-11-04 19:02:23 +00:00
Ian Jackson 3f3f27193c bridge desc: Provide a Store to the bridge desc manager
We use the one in the dirmgr.  That means that our constructor now has
to take a dirmgr.  And, the dirmgr must have a circmgr.

This is all rather odd, TBH.

Add .. to the binding of the return values from setup, as
future-proofing.

The tests now need to provide a Store too.  Make the sqlite::new_empty
function pub(crate) so we can use it.

We must retain the _db_tmp_dir, since when it goes away the tmp
directory is deleted and the db goes readonly.
2022-11-04 19:02:23 +00:00
Ian Jackson 1fa9dd62b3 bridge desc tests: Apply tracing_test::traced_test
This lets us see the log messages.  They look fine, as it happens.
2022-11-04 19:02:23 +00:00
Ian Jackson 9424b0970d tor-dirmgr: impl AsRef<dyn Error> for Error
This allows use with tor_error::Report.
2022-11-04 19:01:24 +00:00
Ian Jackson dcdd8053b4 dirmgr: Make store be in an Arc
This will let us share it with the bridge descriptor manager.
(As discussed with Nick.)
2022-11-04 19:01:24 +00:00
Ian Jackson 9a285ec4ad dirmgr sqlite store: Improve schema updates
* Centralise applying the schema updates, in a closure
 * Make the schema update SQL texts be in an array so we can loop
 * Make the version update statement generic, not cloned-and-hacked

This will make it possible to add another schema version without
error-prone conditions etc.
2022-11-04 19:01:24 +00:00
Ian Jackson bf563f3bdf bridgedesc: Rename "text" from "output"
This was confusing, because it's the output of the donwload, but
nowhere near the output of the whole process.  And it's going to
become even less so.
2022-11-03 19:00:47 +00:00
Ian Jackson 17ceae724e bridgedesc: Make process_document into a free function
It deserves this treatment, I think.
2022-11-03 18:59:50 +00:00
Ian Jackson c7a7ce5c20 bridgedesc: Introduce Downloaded
This gives names to things and makes things clearer, and this will be
even more true in a moemnt.
2022-11-03 18:59:50 +00:00
Ian Jackson d2b31f7796 bridgedesc: download: Break out process_document
We're going to want to reuse this for handling cached stuff.
2022-11-03 18:57:40 +00:00
Ian Jackson 6fbf3059cd bridgedesc: Prepare Mockable trait for if-modified-since 2022-11-03 18:57:40 +00:00
Nick Mathewson 932fe48eaf Run add_warnings. 2022-11-03 11:06:02 -04:00
Ian Jackson 70b420adf8 bridge descriptors tests: Fix flaky test 2022-11-03 11:46:20 +00:00
Ian Jackson daf7eac60a bridge descriptors tests: Test set_bridges more thoroughly (fix)
The RetryDelays are being triggered for the 2nd time here, so their
timeouts can be longer.  We must bump the sleep to make sure we don't
have a flaky test.
2022-11-03 11:37:30 +00:00
Ian Jackson 7a99601721 bridge descriptors: logging: Log addition and removal (fmt)
Worsify formatting as demanded by rustfmt.
2022-11-03 11:37:30 +00:00
Ian Jackson 269808336e bridge descriptors: logging: Log addition and removal
This involves breaking the calls to `new_bridges.remove` into a helper
that does the logging.
2022-11-03 11:37:30 +00:00
Ian Jackson 17c766fde4 bridge descriptors: logging: Log download start and outcome 2022-11-03 11:37:30 +00:00
Ian Jackson 1ccd9db9af bridge descriptors tests: Test set_bridges more thoroughly
Prior to the previous commit, set_bridges would malfunction if there
were bridges which where (i) in current (ii) in queued or running
(iii) in the new bridge set.

This test failed then and passes now.
2022-11-03 11:37:30 +00:00
Ian Jackson 4a830e1ac2 bridge descriptors: Rewrite set_bridges
This was fundamentally confused and irregular.  Now it is more
regular: it does the same things to all the elements of Tracked,
and a simple filtering on current.

This fixes a bug, for which I'm about to add a test case.
2022-11-03 11:37:30 +00:00
Ian Jackson 0571674a64 bridge descriptors tests: Add another couple of test cases 2022-11-03 11:37:30 +00:00
Ian Jackson 6db3403820 bridge descriptors tests: stream_drain_ready returns event count
We'll use this in a moment.
2022-11-03 11:37:30 +00:00
Ian Jackson 53c1cf0c6d bridge descriptors tests: Break out bad_bridge helper function 2022-11-03 11:37:30 +00:00
Ian Jackson 42b1b08fdf bridge descriptors: Change default max_refetch
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/820#note_2850269
2022-11-03 11:37:30 +00:00
Ian Jackson 2ae8a52371 bridge descriptors: Add clarification about bridge states 2022-11-03 11:37:30 +00:00
Ian Jackson ad39873782 bridge descriptors: Improve and fix docs for consider_launching 2022-11-03 11:37:30 +00:00
Ian Jackson d44bc1d0f8 bridge descriptors: Add comments where StateGuard is dropped 2022-11-03 11:37:30 +00:00
Ian Jackson 54e51cdba5 bridge descriptors: Document `changed` and `earliest_timeout` Options 2022-11-03 11:37:30 +00:00
Ian Jackson fb4ffb76fd bridge descriptors: Document updates argument to timeout_task 2022-11-03 11:37:30 +00:00
Ian Jackson 48f48841e1 Require derive_more 0.99.3
0.99.[012] have a bug https://github.com/JelteF/derive_more/issues/114
which makes the Deref derive for bridgedesc::StateGuard not work
and therefore breaks minimal-versions CI.

It seems simpler to require the newer version everywhere.
2022-11-03 11:37:23 +00:00
Ian Jackson 8eb7c731a2 bridge descriptors: Tests 2022-11-03 11:37:05 +00:00
Ian Jackson 3a185d61df bridge descriptors: Provide a consistency check function
This is really part of the tests, but I put it here so that it can be
compiled-in if someone wants it during debugging.
2022-11-03 11:37:05 +00:00
Ian Jackson 51b4588cc0 bridge descriptors: Provide an implementation of BridgeDescProvider
Tests will come in a moment.
2022-11-03 11:37:05 +00:00
Ian Jackson 42f9651b8e Copy tor-netdoc/testdata/routerdesc1.txt into tor-dirmgr
This seems better than having tor_netdoc export a `#[cfg(test)]`
public constant.
2022-11-03 11:37:05 +00:00
trinity-1686a e6e064ac8a remove unused dependancies 2022-10-29 13:29:55 +02:00
Ian Jackson 4a59d384b6 tor-dirmgr: impl Default for FlagPublisher 2022-10-27 15:57:57 +01:00
Ian Jackson 94325cd456 tor-dirmgr: impl FlagEvent for BridgeDescEvent 2022-10-27 15:57:57 +01:00
Ian Jackson 9c9d82f6b8 Fix clippy false positive 2022-10-25 19:42:41 +01:00
Ian Jackson 9a175ee323 FlagEvent trait: Implement using macros
The explicit list of variant names, that needs to be kept in sync, and
is a test failure semver break hazard, is now gone.

All the necessary code is now generated automatically, and cannot be
wrong.

I want this because I find myself wanting to add a second
implementation of FlagEvent, for another type.
2022-10-25 10:33:28 +01:00
Ian Jackson a30e42a740 tor-circmgr: Make get_or_launch_dir_specific feature-specific 2022-10-21 15:19:18 +01:00
Ian Jackson f68591b923 tor-dirmgr README: Remove some redundant/duplicated headings 2022-10-21 15:18:41 +01:00
Ian Jackson d9375b2b8d tor-dirclient: Make RequestFailed its own error type
We're going to have functions on Response that fail by returning only
one of these.

Sadly this diff is quite noisy.
2022-10-20 17:24:53 +01:00
Ian Jackson 3b1af05851 tor-dirclient Response: Rename output methods
These don't check errors and are therefore quite hazardous.
I'm going to introduce a more cooked version in a moment.
2022-10-20 17:24:50 +01:00
Nick Mathewson 5786637b4f Fix various typos (using typos tool and hand-inspection) 2022-10-18 15:39:33 -04:00
Ian Jackson f84d8777db cargo fmt to remove blank lines
Apparently cargo fmt doesn't like these, which my perl rune didn't
delete.

This commit is precisely the result of `cargo fmt`.
2022-10-12 15:29:04 +01:00
Ian Jackson d9910dba08 Replace all README copies in src/lib.rs with includes
The feature we want is `#[doc = include_str!("README.md")]`, which is
stable since 1.54 and our MSRV is now 1.56.

This commit is precisely the result of the following Perl rune:
  perl -i~ -0777 -pe 's{(^//!(?!.*\@\@).*\n)+}{#![doc = include_str!("../README.md")]\n}m' crates/*/src/lib.rs
2022-10-12 15:29:03 +01:00
Nick Mathewson 102e3c221d Bump minor version of tor-rtcompat and most of its dependents
(Since the APIs for the `Schedule::sleep*` functions changed, this
is a breaking change in tor-rtcompat.  Therefore, the Runtime trait
in tor-rtcompat is now a different trait.  Therefore, anything that
uses the Runtime trait in its APIs has also broken.)
2022-10-03 10:01:46 -04:00
Nick Mathewson 388e0cbab5 Bump crates that have had backward compatible API changes. 2022-10-03 09:52:34 -04:00
Nick Mathewson 4b61d778bd Update README.md files. 2022-09-30 15:19:48 -04:00
Nick Mathewson b146dacf1d Mark all bridge and pt features as experimental for now.
Also, document the features.

Closes #588.
2022-09-29 10:59:19 -04:00
Nick Mathewson 1196e1b680 GuardMgr/DirMgr: Add APIs for bridge descriptor lists 2022-09-26 11:23:21 -04:00
trinity-1686a 8e7a904260 fix clippy::needless_borrow 2022-09-10 14:39:47 +02:00
trinity-1686a dcbdd076eb fix clippy::unwrap_used and old TODO for msrv 1.56 2022-09-10 14:37:32 +02:00
Nick Mathewson 86e479ae13 `TaskSchedule`: give error on `sleep*()` if last handle is dropped
This fixes an busy-loop.

When the last `TaskHandle` on a `TaskSchedule` is dropped, the
schedule is permanently canceled: whatever operation it was
scheduling should no longer be performed.  But our code was broken:
the `sleep()` and `sleep_until_wallclock()` functions don't verify
whether the handles are dropped or not.

This breakage caused an CPU-eating busy-loop in
`sleep_until_wallclock`.

With this patch, we now return a `Result<(), SleepError>` from these
functions.

Fixes #572.
2022-09-07 09:22:38 -04:00
Nick Mathewson a5f704f443 Remove semver.md from arti-1.0.0 2022-09-07 09:17:00 -04:00
Nick Mathewson 96875ea208 Bump crate versions in preparation for Arti 1.0.0 release.
Because we want to work more on ensuring that our semver stability
story is solid, we are _not_ bumping arti-client to 1.0.0 right now.

Here are the bumps we _are_ doing.  Crates with "minor" bumps have
had API breaks; crates with "patch" bumps have had new APIs added.

Note that `tor-congestion` is not bumped here: it's a new crate, and
hasn't been published before.

```
tor-basic-utils         minor
fs-mistrust             minor
tor-config              minor
tor-rtcompat            minor
tor-rtmock              minor
tor-llcrypto            patch
tor-bytes               patch
tor-linkspec            minor
tor-cell                minor
tor-proto               minor
tor-netdoc              patch
tor-netdir              minor
tor-persist             patch
tor-chanmgr             minor
tor-guardmgr            minor
tor-circmgr             minor
tor-dirmgr              minor
arti-client             minor
arti-hyper              minor
arti                    major
arti-bench              minor
arti-testing            minor
```
2022-09-01 08:59:49 -04:00
Nick Mathewson a9f19b0045 Use anonymize_home() when displaying various messages.
Closes #555
2022-08-31 09:38:35 -04:00
trinity-1686a 426a59b2ba add feature annotation not added by doc_auto_cfg 2022-08-24 18:22:41 +02:00
trinity-1686a 7f939fa480 enable doc_auto_cfg feature on every crate when documenting for docs.rs 2022-08-24 18:22:41 +02:00
Nick Mathewson 8a2a0ec1a0 Decrease the lifetimes for storing descriptors.
These values were chosen experimentally, based on those from Tor, to
save disk space without wasting much bandwidth.

Closes #527.
2022-08-09 09:30:22 -04:00
Nick Mathewson 760e2f7e95 Finish implementation of params() for DirMgr.
Now it maintains an up-to-date set of default parameters to be
handed out if there is no directory.

Closes #528.
2022-08-02 10:20:13 -04:00
Nick Mathewson 9d2fb92c37 Add params() method to NetDirProvider, and partial implementation
This method will let users get the latest `NetParameters`, with
user-configured overrides, even if there is no current directory at
all.

Part of #528
2022-08-02 09:50:39 -04:00
Nick Mathewson b16c042004 Now that versions have bumped, remove semver.md files. 2022-08-01 10:07:12 -04:00
Nick Mathewson 859ae4acd0 Bump patch versions on crates that have new APIs.
Do _not_ bump the dependency versions on crates that have had no
changes since arti 0.0.5, since those crates do not depend on the
new APIs.

```
cargo set-version -p tor-basic-utils   --bump patch
cargo set-version -p tor-llcrypto      --bump patch
git restore crates/tor-checkable
git restore crates/tor-consdiff
git restore crates/tor-rtmock
```
2022-08-01 09:56:29 -04:00
Nick Mathewson 511cb46997 Bump minor version on crates with deps with breaking changes.
This performs the transitive closure of the last operation:
everything that depends on a crate with a breaking change gets the
version which it depends on bumped.

```
cargo set-version -p tor-proto         --bump minor
cargo set-version -p tor-netdoc        --bump minor
cargo set-version -p arti-hyper        --bump minor
cargo set-version -p arti-bench        --bump minor
cargo set-version -p arti-testing      --bump minor
cargo set-version -p tor-config        --bump minor
```
2022-08-01 09:49:13 -04:00
Nick Mathewson f5dcb98f06 Bump minor versions on all crates that have had breaking changes.
Done with these commands:

```
cargo set-version -p fs-mistrust       --bump minor
cargo set-version -p tor-bytes         --bump minor
cargo set-version -p tor-socksproto    --bump minor
cargo set-version -p tor-cert          --bump minor
cargo set-version -p tor-linkspec      --bump minor
cargo set-version -p tor-cell          --bump minor
cargo set-version -p tor-netdir        --bump minor
cargo set-version -p tor-persist       --bump minor
cargo set-version -p tor-chanmgr       --bump minor
cargo set-version -p tor-guardmgr      --bump minor
cargo set-version -p tor-circmgr       --bump minor
cargo set-version -p tor-dirclient     --bump minor
cargo set-version -p tor-dirmgr        --bump minor
cargo set-version -p arti-client       --bump minor
cargo set-version -p arti              --bump minor
```
2022-08-01 09:43:09 -04:00
Nick Mathewson 14cd423630 tor-dirmgr: Remove opt_netdir entirely.
Its existence tended to hide bugs, and was just asking for trouble.
2022-07-26 10:41:38 +01:00
Nick Mathewson de460b0668 Remove unused DirMgr::netdir method, make opt_netdir private. 2022-07-26 10:41:06 +01:00
Nick Mathewson c39aa5de73 Add new APIs to NetDirProvider to better support timeliness.
Over the years we've found that most callers who want a netdir want
what C Tor calls a "reasonably live" network directory: One that is
not expired by too much, or too far in the future. But a few want a
_strictly_ live directory: one that says it is valid now, with no
tolerances.  And a few want _any_ directory, no matter how expired
it is.

This commit adds net methods to NetDirProvider to provide these
directories.  I think that most use cases will want to explicitly
think about what kind of directory they want, so I've made `netdir`
the simplest method.  I might remove `timely_netdir` by the end of
this branch; see TODO comments.

Part of #518.
2022-07-26 10:40:53 +01:00
Nick Mathewson 41104137b1 Rename DirSkewTolerance to DirTolerance
This name is more accurate because we aren't only dealing with
clock skew here: we're also trying to tolerate the case where the
authorities fail to reach consensus for a while.
2022-07-22 10:04:25 -04:00
Nick Mathewson 052f3089ca Bump to rusqlite 0.28. 2022-07-19 16:49:44 -04:00
Nick Mathewson 8d66f8ecd0 Merge branch 'dirmgr_error_cleanup' into 'main'
Clean up some errors in tor-dirmgr

Closes #521

See merge request tpo/core/arti!628
2022-07-18 14:40:11 +00:00
Nick Mathewson 3c424c82d7 dirmgr: Add a helper to create LockFile errors. 2022-07-18 10:10:28 -04:00
Nick Mathewson 9d3beb34a3 Bump tor-dirmgr to version 0.5.1.
This does not require a change in any other crate, since
the change here does not affect tor-dirmgr's APIs.
2022-07-14 12:00:42 -04:00
Nick Mathewson c983987782 dirmgr: Make UnrecognizedSchema more helpful 2022-07-14 09:53:17 -04:00
Nick Mathewson 8cbc740d90 dirmgr: Add an instance of BadUtf8InCache 2022-07-14 09:32:01 -04:00
Nick Mathewson d1e9d4ce84 dirmgr: Clarify disk IO errors. 2022-07-14 09:27:29 -04:00
Nick Mathewson c1101f13f3 dirmgr: style edits to error messages. 2022-07-14 08:40:42 -04:00
feelingnothing 171dd5eb63 Fix illegal formatting in cache filenames 2022-07-14 15:27:52 +03:00
Nick Mathewson 1dda69af2b Remove semver.md files now that 0.5.0 is out 2022-06-24 13:31:38 -04:00
Nick Mathewson 20435aea75 Bump crate and dependency versions.
These were done with the following commands:

```
cargo set-version -p tor-basic-utils   --bump patch
cargo set-version -p fs-mistrust       --bump minor
cargo set-version -p tor-error         --bump patch
cargo set-version -p tor-config        --bump patch
cargo set-version -p tor-units         --bump patch
cargo set-version -p tor-rtcompat      --bump minor
cargo set-version -p tor-llcrypto      --bump patch
cargo set-version -p tor-bytes         --bump minor
cargo set-version -p tor-socksproto    --bump minor
cargo set-version -p tor-cert          --bump minor
cargo set-version -p tor-cell          --bump minor
cargo set-version -p tor-proto         --bump minor
cargo set-version -p tor-netdoc        --bump patch
cargo set-version -p tor-netdir        --bump minor
cargo set-version -p tor-persist       --bump patch
cargo set-version -p tor-chanmgr       --bump minor
cargo set-version -p tor-guardmgr      --bump minor
cargo set-version -p tor-circmgr       --bump patch
cargo set-version -p tor-dirclient     --bump patch
cargo set-version -p tor-dirmgr        --bump minor
cargo set-version -p arti-client       --bump patch
cargo set-version -p arti              --bump minor
cargo set-version -p arti-bench        --bump minor
cargo set-version -p arti-testing      --bump minor
```
2022-06-24 12:00:22 -04:00
Ian Jackson 4831f9d38c Merge branch 'clippy' into 'main'
Fix clippy nightly again

See merge request tpo/core/arti!603
2022-06-24 14:13:58 +00:00
Nick Mathewson 1f8f24dbcb Merge branch 'dirmgr' into 'main'
dirmgr: followups to !587, mostly using iterators

Closes #468

See merge request tpo/core/arti!599
2022-06-24 13:46:52 +00:00
Nick Mathewson ea3160737c Fix a couple of typos in rustdoc comments. 2022-06-24 08:18:15 -04:00
Ian Jackson 589c6e52bb Run maint/add_warning crates/*/src/{lib,main}.rs
Update all lint blocks
2022-06-23 19:15:42 +01:00
Ian Jackson 6c110a232c dirmgr: Use ... .find().is_none() rather than clippy's suggestion 2022-06-23 18:42:33 +01:00
Ian Jackson d2bd71c5f0 dirmgr: DirBootstrapStatus: Refactor mut_status_for using entries_mut
IMO this is rather less confusing.
2022-06-22 10:53:02 +01:00
Ian Jackson d36c433b36 dirmgr: Use new statuses() helper method
The two call sites become clearer, IMO.
2022-06-22 10:53:02 +01:00
Ian Jackson 24d43f83fd dirmgr: DirBootstrapStatus: Provide statuses and entries_mut
We're going to use these in a moment.

One returns entries and the other statuses simply because that's
what's actually going to be wanted.
2022-06-22 10:53:02 +01:00
Ian Jackson 5c51d03efb dirmgr: Remove a TODO comment
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/587#note_2815945
2022-06-22 10:53:02 +01:00
eta 4e3537746f Merge branch 'display_source_cleanup' into 'main'
Do not include error source() in display() format.

See merge request tpo/core/arti!598
2022-06-21 19:34:27 +00:00
Ian Jackson 9b4ba4893e Merge branch 'stalled_directory' into 'main'
Detect and report stalled directory downloads

Closes #468

See merge request tpo/core/arti!587
2022-06-21 19:13:40 +00:00
Nick Mathewson 08d9bbf33b Do not include error source() in display() format.
According to doc/Errors.md, and in keeping with current best
practices, we should not include display an error's `source()` as
part of that error's display method.  Instead, we should let the
caller decide to call source() and display that error in turn.

Part of #323.
2022-06-21 14:14:14 -04:00
Nick Mathewson 9b7849a293 Refactor DirStatus data structures.
AttemptId is now linked to DirStatus in StatusEntry.

DirBootstrapStatus now has an inner enum to make invalid status
unrepresentable.
2022-06-21 10:26:53 -04:00
Nick Mathewson 5509bd0269 Merge branch 'config-fix2' into 'main'
Use impl_standard_builder more and remove manual Default/builder impls

See merge request tpo/core/arti!594
2022-06-16 20:30:34 +00:00
Ian Jackson db51bdcbc1 Merge branch 'config-fix' into 'main'
Config handling and logging fixes

Closes #480

See merge request tpo/core/arti!589
2022-06-16 15:46:18 +00:00
Ian Jackson 2ca2c03059 impl_standard_builder: Use for tor_dirmgr::Authority 2022-06-16 15:42:06 +01:00
Ian Jackson 17caebd284 impl_standard_builder: Use for tor_dirmgr::DownloadSchedule 2022-06-16 15:42:06 +01:00
Ian Jackson 8b724d40f8 impl_standard_builder: Use for tor_dirmgr::DownloadScheduleConfig 2022-06-16 15:27:00 +01:00
Ian Jackson 2ab5c0861e impl_standard_builder: Use for tor_dirmgr::NetworkConfig 2022-06-16 15:27:00 +01:00
Ian Jackson e32210a398 impl_standard_builder: Use for tor_dirmgr::DirSkewTolerance 2022-06-16 15:27:00 +01:00
Nick Mathewson 0223c91b26 Merge branch 'high-level-features' into 'main'
Add "full" and "experimental" features to arti, arti-client, and below.

Closes #499

See merge request tpo/core/arti!584
2022-06-16 12:24:47 +00:00
Ian Jackson 4c65e9994d Add some missing `serde(default)` to uses of humantime_serde::option
Without this, if the substructure is provided, these fields are
mandatory within it.
2022-06-16 11:08:16 +01:00
Nick Mathewson 647d4410bb Fix clippy::significant_drop_in_scrutinee warnings
This is apparently a new warning from clippy nightly, documented in
https://rust-lang.github.io/rust-clippy/master/index.html#significant_drop_in_scrutinee .

I'm not in love with the temporary variables that this warning wants
me to introduce, but it does seem like a decent way to avoid some
kinds of deadlock.
2022-06-15 11:52:30 -04:00
Nick Mathewson 425af9db91 Work around an erroneous warning from nightly clippy 2022-06-15 11:52:30 -04:00
Nick Mathewson 331578d19c Make display formats work with older derive_more. 2022-06-14 14:10:18 -04:00
Nick Mathewson c74c268c8b tor-dirmgr: Make DirStatus non-public.
I believe this type was made public in error: No other crate uses it,
and no API exposes it.
2022-06-14 11:02:29 -04:00
Nick Mathewson e3dced360b Add a blockage-reporting mechanism to tor-dirmgr.
The `DirBootstrapStatus` type now exposes a blockage() method to
return an `Option<DirBlockage>`.

The blockage types reported are more low-level than I'd like, but
they are IMO good enough for now: we'll want to get experience with
actual vs hypothetical problems before we refine them.
2022-06-14 10:42:18 -04:00
Nick Mathewson 25bbdd5f25 Add a couple of TODO items. 2022-06-14 10:05:27 -04:00
Nick Mathewson 660b3104a9 Do not update_progress on load if nothing happened.
If there was no partial information, then calling update_progress
will now report a stall in this case, which is not what we want.
2022-06-14 09:58:58 -04:00
Nick Mathewson 13c152d2ac DirMgr: Track errors, stalls, and resets as part of status.
A "reset" happens whenever we have to start a download attempt over
-- either because we ran out of retries, or we found something wrong
with the consensus after fetching certificates.

An "error" happens when we have a recoverable error from one or more
directory sources.

A "stall" happens whenever a round of downloads or cache loads leads
to no change in the status.

We don't yet use this as part of our status reporting.
2022-06-13 13:58:31 -04:00
Nick Mathewson 2a2afaa7c2 Refactor our logic for distinguishing download attempts
Previously we used the "if-modified-since" time associated with the
consensus download, and/or the "valid-after" time in the consensus
attempt, to put multiple attempts into sequence, and to tell one
from another.  But that approach was always a kludge, and will soon
get more unreliable as the DirStatus logic gets a bit more complex.

With this commit, we change separate download attempts to be
identified with an AttemptId that increments whenever we decide to
get a different directory from the one we have.  IMO this new code
is _much_ cleaner.
2022-06-13 13:21:09 -04:00
Nick Mathewson 68ea880b84 Refactor most DirStatus users to use DirProgress.
This is all crate-internal APIs, fortunately.
2022-06-13 12:22:04 -04:00
Nick Mathewson 238f2b5637 Move `at_least_as_new_as` to DirProgress. 2022-06-13 12:02:30 -04:00
Nick Mathewson 6c9d30681f Turn the DirProgress in DirStatus into a named field. 2022-06-13 10:53:46 -04:00
Nick Mathewson f581feb0ac Rename DirStatusInner to DirProgress.
This is about to become only a _part_ of what defines a DirStatus: a
DirStatus will also include a reset count, and some kind of info
about how long we've gone without progress.
2022-06-13 10:49:09 -04:00
Nick Mathewson d44dd6f44f Merge branch 'dormant_directory' into 'main'
Use TaskSchedule to sleep in directory bootstrapping

Closes #497

See merge request tpo/core/arti!571
2022-06-13 13:59:26 +00:00
Nick Mathewson 2129aee6cd DirMgr: Use scopeguard crate in place of ad hoc RAII substitutes 2022-06-13 08:28:41 -04:00
Nick Mathewson a7b406e69f Put the task schedule back when the dl task exits.
This will allow somebody else to call bootstrap() if bootstrap() fails
the first time.
2022-06-13 08:28:41 -04:00
Nick Mathewson 430f4ffb86 New API to expose and use TaskHandle from DirMgr.
This change also means that we need to create the handle and scheduler
earlier in the process of creating the DirMgr.  If we don't, we won't
have a way to manage the task before bootstrap() returns.
2022-06-13 08:28:41 -04:00
Nick Mathewson 8a586a40fa Add "full" and "experimental" features to arti-client and below.
The "full" feature is a catch-all for all features, _except_:

   * Those that select a particular implementation (like
     tor-llcrypto/with-openssl) or build flag (like "static")
   * Those that are experimental or unstable (like "experimental-api")
   * Those that are testing-only.
2022-06-10 15:32:21 -04:00
Nick Mathewson 171b406b00 DirMgr: Refactor bootstrap to use a TaskSchedule for sleeping.
This change (not yet exposed as an API) will let the TorClient have
a `TaskHandle` corresponding to the directory task, letting it
make the directory task dormant as needed.
2022-06-10 13:59:30 -04:00
Nick Mathewson 5854e4bbd9 Merge branch 'use-testing-rng' 2022-06-07 19:02:12 -04:00
Nick Mathewson 7f785b66c2 Upgrade float_eq dev-dependency to 1.0.0 2022-06-07 08:03:55 -04:00
Nick Mathewson 967ea67b7d Use testing_rng() in tests throughout our crates.
This only affects uses of thread_rng(), and affects them all more or
less indiscriminately.  One test does not work with
ARTI_TEST_PRNG=deterministic; the next commit will fix it.
2022-06-02 14:56:42 -04:00