Commit Graph

472 Commits

Author SHA1 Message Date
Ian Jackson 279a169031 bridge desc: Rename one more variable to text 2022-11-04 19:02:23 +00:00
Ian Jackson eaaa61a8e2 bridge desc: Catch panics in the download functionality 2022-11-04 19:02:23 +00:00
Ian Jackson 3f3f27193c bridge desc: Provide a Store to the bridge desc manager
We use the one in the dirmgr.  That means that our constructor now has
to take a dirmgr.  And, the dirmgr must have a circmgr.

This is all rather odd, TBH.

Add .. to the binding of the return values from setup, as
future-proofing.

The tests now need to provide a Store too.  Make the sqlite::new_empty
function pub(crate) so we can use it.

We must retain the _db_tmp_dir, since when it goes away the tmp
directory is deleted and the db goes readonly.
2022-11-04 19:02:23 +00:00
Ian Jackson 1fa9dd62b3 bridge desc tests: Apply tracing_test::traced_test
This lets us see the log messages.  They look fine, as it happens.
2022-11-04 19:02:23 +00:00
Ian Jackson 9424b0970d tor-dirmgr: impl AsRef<dyn Error> for Error
This allows use with tor_error::Report.
2022-11-04 19:01:24 +00:00
Ian Jackson dcdd8053b4 dirmgr: Make store be in an Arc
This will let us share it with the bridge descriptor manager.
(As discussed with Nick.)
2022-11-04 19:01:24 +00:00
Ian Jackson 9a285ec4ad dirmgr sqlite store: Improve schema updates
* Centralise applying the schema updates, in a closure
 * Make the schema update SQL texts be in an array so we can loop
 * Make the version update statement generic, not cloned-and-hacked

This will make it possible to add another schema version without
error-prone conditions etc.
2022-11-04 19:01:24 +00:00
Ian Jackson bf563f3bdf bridgedesc: Rename "text" from "output"
This was confusing, because it's the output of the donwload, but
nowhere near the output of the whole process.  And it's going to
become even less so.
2022-11-03 19:00:47 +00:00
Ian Jackson 17ceae724e bridgedesc: Make process_document into a free function
It deserves this treatment, I think.
2022-11-03 18:59:50 +00:00
Ian Jackson c7a7ce5c20 bridgedesc: Introduce Downloaded
This gives names to things and makes things clearer, and this will be
even more true in a moemnt.
2022-11-03 18:59:50 +00:00
Ian Jackson d2b31f7796 bridgedesc: download: Break out process_document
We're going to want to reuse this for handling cached stuff.
2022-11-03 18:57:40 +00:00
Ian Jackson 6fbf3059cd bridgedesc: Prepare Mockable trait for if-modified-since 2022-11-03 18:57:40 +00:00
Nick Mathewson 932fe48eaf Run add_warnings. 2022-11-03 11:06:02 -04:00
Ian Jackson 70b420adf8 bridge descriptors tests: Fix flaky test 2022-11-03 11:46:20 +00:00
Ian Jackson daf7eac60a bridge descriptors tests: Test set_bridges more thoroughly (fix)
The RetryDelays are being triggered for the 2nd time here, so their
timeouts can be longer.  We must bump the sleep to make sure we don't
have a flaky test.
2022-11-03 11:37:30 +00:00
Ian Jackson 7a99601721 bridge descriptors: logging: Log addition and removal (fmt)
Worsify formatting as demanded by rustfmt.
2022-11-03 11:37:30 +00:00
Ian Jackson 269808336e bridge descriptors: logging: Log addition and removal
This involves breaking the calls to `new_bridges.remove` into a helper
that does the logging.
2022-11-03 11:37:30 +00:00
Ian Jackson 17c766fde4 bridge descriptors: logging: Log download start and outcome 2022-11-03 11:37:30 +00:00
Ian Jackson 1ccd9db9af bridge descriptors tests: Test set_bridges more thoroughly
Prior to the previous commit, set_bridges would malfunction if there
were bridges which where (i) in current (ii) in queued or running
(iii) in the new bridge set.

This test failed then and passes now.
2022-11-03 11:37:30 +00:00
Ian Jackson 4a830e1ac2 bridge descriptors: Rewrite set_bridges
This was fundamentally confused and irregular.  Now it is more
regular: it does the same things to all the elements of Tracked,
and a simple filtering on current.

This fixes a bug, for which I'm about to add a test case.
2022-11-03 11:37:30 +00:00
Ian Jackson 0571674a64 bridge descriptors tests: Add another couple of test cases 2022-11-03 11:37:30 +00:00
Ian Jackson 6db3403820 bridge descriptors tests: stream_drain_ready returns event count
We'll use this in a moment.
2022-11-03 11:37:30 +00:00
Ian Jackson 53c1cf0c6d bridge descriptors tests: Break out bad_bridge helper function 2022-11-03 11:37:30 +00:00
Ian Jackson 42b1b08fdf bridge descriptors: Change default max_refetch
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/820#note_2850269
2022-11-03 11:37:30 +00:00
Ian Jackson 2ae8a52371 bridge descriptors: Add clarification about bridge states 2022-11-03 11:37:30 +00:00
Ian Jackson ad39873782 bridge descriptors: Improve and fix docs for consider_launching 2022-11-03 11:37:30 +00:00
Ian Jackson d44bc1d0f8 bridge descriptors: Add comments where StateGuard is dropped 2022-11-03 11:37:30 +00:00
Ian Jackson 54e51cdba5 bridge descriptors: Document `changed` and `earliest_timeout` Options 2022-11-03 11:37:30 +00:00
Ian Jackson fb4ffb76fd bridge descriptors: Document updates argument to timeout_task 2022-11-03 11:37:30 +00:00
Ian Jackson 48f48841e1 Require derive_more 0.99.3
0.99.[012] have a bug https://github.com/JelteF/derive_more/issues/114
which makes the Deref derive for bridgedesc::StateGuard not work
and therefore breaks minimal-versions CI.

It seems simpler to require the newer version everywhere.
2022-11-03 11:37:23 +00:00
Ian Jackson 8eb7c731a2 bridge descriptors: Tests 2022-11-03 11:37:05 +00:00
Ian Jackson 3a185d61df bridge descriptors: Provide a consistency check function
This is really part of the tests, but I put it here so that it can be
compiled-in if someone wants it during debugging.
2022-11-03 11:37:05 +00:00
Ian Jackson 51b4588cc0 bridge descriptors: Provide an implementation of BridgeDescProvider
Tests will come in a moment.
2022-11-03 11:37:05 +00:00
Ian Jackson 42f9651b8e Copy tor-netdoc/testdata/routerdesc1.txt into tor-dirmgr
This seems better than having tor_netdoc export a `#[cfg(test)]`
public constant.
2022-11-03 11:37:05 +00:00
trinity-1686a e6e064ac8a remove unused dependancies 2022-10-29 13:29:55 +02:00
Ian Jackson 4a59d384b6 tor-dirmgr: impl Default for FlagPublisher 2022-10-27 15:57:57 +01:00
Ian Jackson 94325cd456 tor-dirmgr: impl FlagEvent for BridgeDescEvent 2022-10-27 15:57:57 +01:00
Ian Jackson 9c9d82f6b8 Fix clippy false positive 2022-10-25 19:42:41 +01:00
Ian Jackson 9a175ee323 FlagEvent trait: Implement using macros
The explicit list of variant names, that needs to be kept in sync, and
is a test failure semver break hazard, is now gone.

All the necessary code is now generated automatically, and cannot be
wrong.

I want this because I find myself wanting to add a second
implementation of FlagEvent, for another type.
2022-10-25 10:33:28 +01:00
Ian Jackson a30e42a740 tor-circmgr: Make get_or_launch_dir_specific feature-specific 2022-10-21 15:19:18 +01:00
Ian Jackson f68591b923 tor-dirmgr README: Remove some redundant/duplicated headings 2022-10-21 15:18:41 +01:00
Ian Jackson d9375b2b8d tor-dirclient: Make RequestFailed its own error type
We're going to have functions on Response that fail by returning only
one of these.

Sadly this diff is quite noisy.
2022-10-20 17:24:53 +01:00
Ian Jackson 3b1af05851 tor-dirclient Response: Rename output methods
These don't check errors and are therefore quite hazardous.
I'm going to introduce a more cooked version in a moment.
2022-10-20 17:24:50 +01:00
Nick Mathewson 5786637b4f Fix various typos (using typos tool and hand-inspection) 2022-10-18 15:39:33 -04:00
Ian Jackson f84d8777db cargo fmt to remove blank lines
Apparently cargo fmt doesn't like these, which my perl rune didn't
delete.

This commit is precisely the result of `cargo fmt`.
2022-10-12 15:29:04 +01:00
Ian Jackson d9910dba08 Replace all README copies in src/lib.rs with includes
The feature we want is `#[doc = include_str!("README.md")]`, which is
stable since 1.54 and our MSRV is now 1.56.

This commit is precisely the result of the following Perl rune:
  perl -i~ -0777 -pe 's{(^//!(?!.*\@\@).*\n)+}{#![doc = include_str!("../README.md")]\n}m' crates/*/src/lib.rs
2022-10-12 15:29:03 +01:00
Nick Mathewson 102e3c221d Bump minor version of tor-rtcompat and most of its dependents
(Since the APIs for the `Schedule::sleep*` functions changed, this
is a breaking change in tor-rtcompat.  Therefore, the Runtime trait
in tor-rtcompat is now a different trait.  Therefore, anything that
uses the Runtime trait in its APIs has also broken.)
2022-10-03 10:01:46 -04:00
Nick Mathewson 388e0cbab5 Bump crates that have had backward compatible API changes. 2022-10-03 09:52:34 -04:00
Nick Mathewson 4b61d778bd Update README.md files. 2022-09-30 15:19:48 -04:00
Nick Mathewson b146dacf1d Mark all bridge and pt features as experimental for now.
Also, document the features.

Closes #588.
2022-09-29 10:59:19 -04:00
Nick Mathewson 1196e1b680 GuardMgr/DirMgr: Add APIs for bridge descriptor lists 2022-09-26 11:23:21 -04:00
trinity-1686a 8e7a904260 fix clippy::needless_borrow 2022-09-10 14:39:47 +02:00
trinity-1686a dcbdd076eb fix clippy::unwrap_used and old TODO for msrv 1.56 2022-09-10 14:37:32 +02:00
Nick Mathewson 86e479ae13 `TaskSchedule`: give error on `sleep*()` if last handle is dropped
This fixes an busy-loop.

When the last `TaskHandle` on a `TaskSchedule` is dropped, the
schedule is permanently canceled: whatever operation it was
scheduling should no longer be performed.  But our code was broken:
the `sleep()` and `sleep_until_wallclock()` functions don't verify
whether the handles are dropped or not.

This breakage caused an CPU-eating busy-loop in
`sleep_until_wallclock`.

With this patch, we now return a `Result<(), SleepError>` from these
functions.

Fixes #572.
2022-09-07 09:22:38 -04:00
Nick Mathewson a5f704f443 Remove semver.md from arti-1.0.0 2022-09-07 09:17:00 -04:00
Nick Mathewson 96875ea208 Bump crate versions in preparation for Arti 1.0.0 release.
Because we want to work more on ensuring that our semver stability
story is solid, we are _not_ bumping arti-client to 1.0.0 right now.

Here are the bumps we _are_ doing.  Crates with "minor" bumps have
had API breaks; crates with "patch" bumps have had new APIs added.

Note that `tor-congestion` is not bumped here: it's a new crate, and
hasn't been published before.

```
tor-basic-utils         minor
fs-mistrust             minor
tor-config              minor
tor-rtcompat            minor
tor-rtmock              minor
tor-llcrypto            patch
tor-bytes               patch
tor-linkspec            minor
tor-cell                minor
tor-proto               minor
tor-netdoc              patch
tor-netdir              minor
tor-persist             patch
tor-chanmgr             minor
tor-guardmgr            minor
tor-circmgr             minor
tor-dirmgr              minor
arti-client             minor
arti-hyper              minor
arti                    major
arti-bench              minor
arti-testing            minor
```
2022-09-01 08:59:49 -04:00
Nick Mathewson a9f19b0045 Use anonymize_home() when displaying various messages.
Closes #555
2022-08-31 09:38:35 -04:00
trinity-1686a 426a59b2ba add feature annotation not added by doc_auto_cfg 2022-08-24 18:22:41 +02:00
trinity-1686a 7f939fa480 enable doc_auto_cfg feature on every crate when documenting for docs.rs 2022-08-24 18:22:41 +02:00
Nick Mathewson 8a2a0ec1a0 Decrease the lifetimes for storing descriptors.
These values were chosen experimentally, based on those from Tor, to
save disk space without wasting much bandwidth.

Closes #527.
2022-08-09 09:30:22 -04:00
Nick Mathewson 760e2f7e95 Finish implementation of params() for DirMgr.
Now it maintains an up-to-date set of default parameters to be
handed out if there is no directory.

Closes #528.
2022-08-02 10:20:13 -04:00
Nick Mathewson 9d2fb92c37 Add params() method to NetDirProvider, and partial implementation
This method will let users get the latest `NetParameters`, with
user-configured overrides, even if there is no current directory at
all.

Part of #528
2022-08-02 09:50:39 -04:00
Nick Mathewson b16c042004 Now that versions have bumped, remove semver.md files. 2022-08-01 10:07:12 -04:00
Nick Mathewson 859ae4acd0 Bump patch versions on crates that have new APIs.
Do _not_ bump the dependency versions on crates that have had no
changes since arti 0.0.5, since those crates do not depend on the
new APIs.

```
cargo set-version -p tor-basic-utils   --bump patch
cargo set-version -p tor-llcrypto      --bump patch
git restore crates/tor-checkable
git restore crates/tor-consdiff
git restore crates/tor-rtmock
```
2022-08-01 09:56:29 -04:00
Nick Mathewson 511cb46997 Bump minor version on crates with deps with breaking changes.
This performs the transitive closure of the last operation:
everything that depends on a crate with a breaking change gets the
version which it depends on bumped.

```
cargo set-version -p tor-proto         --bump minor
cargo set-version -p tor-netdoc        --bump minor
cargo set-version -p arti-hyper        --bump minor
cargo set-version -p arti-bench        --bump minor
cargo set-version -p arti-testing      --bump minor
cargo set-version -p tor-config        --bump minor
```
2022-08-01 09:49:13 -04:00
Nick Mathewson f5dcb98f06 Bump minor versions on all crates that have had breaking changes.
Done with these commands:

```
cargo set-version -p fs-mistrust       --bump minor
cargo set-version -p tor-bytes         --bump minor
cargo set-version -p tor-socksproto    --bump minor
cargo set-version -p tor-cert          --bump minor
cargo set-version -p tor-linkspec      --bump minor
cargo set-version -p tor-cell          --bump minor
cargo set-version -p tor-netdir        --bump minor
cargo set-version -p tor-persist       --bump minor
cargo set-version -p tor-chanmgr       --bump minor
cargo set-version -p tor-guardmgr      --bump minor
cargo set-version -p tor-circmgr       --bump minor
cargo set-version -p tor-dirclient     --bump minor
cargo set-version -p tor-dirmgr        --bump minor
cargo set-version -p arti-client       --bump minor
cargo set-version -p arti              --bump minor
```
2022-08-01 09:43:09 -04:00
Nick Mathewson 14cd423630 tor-dirmgr: Remove opt_netdir entirely.
Its existence tended to hide bugs, and was just asking for trouble.
2022-07-26 10:41:38 +01:00
Nick Mathewson de460b0668 Remove unused DirMgr::netdir method, make opt_netdir private. 2022-07-26 10:41:06 +01:00
Nick Mathewson c39aa5de73 Add new APIs to NetDirProvider to better support timeliness.
Over the years we've found that most callers who want a netdir want
what C Tor calls a "reasonably live" network directory: One that is
not expired by too much, or too far in the future. But a few want a
_strictly_ live directory: one that says it is valid now, with no
tolerances.  And a few want _any_ directory, no matter how expired
it is.

This commit adds net methods to NetDirProvider to provide these
directories.  I think that most use cases will want to explicitly
think about what kind of directory they want, so I've made `netdir`
the simplest method.  I might remove `timely_netdir` by the end of
this branch; see TODO comments.

Part of #518.
2022-07-26 10:40:53 +01:00
Nick Mathewson 41104137b1 Rename DirSkewTolerance to DirTolerance
This name is more accurate because we aren't only dealing with
clock skew here: we're also trying to tolerate the case where the
authorities fail to reach consensus for a while.
2022-07-22 10:04:25 -04:00
Nick Mathewson 052f3089ca Bump to rusqlite 0.28. 2022-07-19 16:49:44 -04:00
Nick Mathewson 8d66f8ecd0 Merge branch 'dirmgr_error_cleanup' into 'main'
Clean up some errors in tor-dirmgr

Closes #521

See merge request tpo/core/arti!628
2022-07-18 14:40:11 +00:00
Nick Mathewson 3c424c82d7 dirmgr: Add a helper to create LockFile errors. 2022-07-18 10:10:28 -04:00
Nick Mathewson 9d3beb34a3 Bump tor-dirmgr to version 0.5.1.
This does not require a change in any other crate, since
the change here does not affect tor-dirmgr's APIs.
2022-07-14 12:00:42 -04:00
Nick Mathewson c983987782 dirmgr: Make UnrecognizedSchema more helpful 2022-07-14 09:53:17 -04:00
Nick Mathewson 8cbc740d90 dirmgr: Add an instance of BadUtf8InCache 2022-07-14 09:32:01 -04:00
Nick Mathewson d1e9d4ce84 dirmgr: Clarify disk IO errors. 2022-07-14 09:27:29 -04:00
Nick Mathewson c1101f13f3 dirmgr: style edits to error messages. 2022-07-14 08:40:42 -04:00
feelingnothing 171dd5eb63 Fix illegal formatting in cache filenames 2022-07-14 15:27:52 +03:00
Nick Mathewson 1dda69af2b Remove semver.md files now that 0.5.0 is out 2022-06-24 13:31:38 -04:00
Nick Mathewson 20435aea75 Bump crate and dependency versions.
These were done with the following commands:

```
cargo set-version -p tor-basic-utils   --bump patch
cargo set-version -p fs-mistrust       --bump minor
cargo set-version -p tor-error         --bump patch
cargo set-version -p tor-config        --bump patch
cargo set-version -p tor-units         --bump patch
cargo set-version -p tor-rtcompat      --bump minor
cargo set-version -p tor-llcrypto      --bump patch
cargo set-version -p tor-bytes         --bump minor
cargo set-version -p tor-socksproto    --bump minor
cargo set-version -p tor-cert          --bump minor
cargo set-version -p tor-cell          --bump minor
cargo set-version -p tor-proto         --bump minor
cargo set-version -p tor-netdoc        --bump patch
cargo set-version -p tor-netdir        --bump minor
cargo set-version -p tor-persist       --bump patch
cargo set-version -p tor-chanmgr       --bump minor
cargo set-version -p tor-guardmgr      --bump minor
cargo set-version -p tor-circmgr       --bump patch
cargo set-version -p tor-dirclient     --bump patch
cargo set-version -p tor-dirmgr        --bump minor
cargo set-version -p arti-client       --bump patch
cargo set-version -p arti              --bump minor
cargo set-version -p arti-bench        --bump minor
cargo set-version -p arti-testing      --bump minor
```
2022-06-24 12:00:22 -04:00
Ian Jackson 4831f9d38c Merge branch 'clippy' into 'main'
Fix clippy nightly again

See merge request tpo/core/arti!603
2022-06-24 14:13:58 +00:00
Nick Mathewson 1f8f24dbcb Merge branch 'dirmgr' into 'main'
dirmgr: followups to !587, mostly using iterators

Closes #468

See merge request tpo/core/arti!599
2022-06-24 13:46:52 +00:00
Nick Mathewson ea3160737c Fix a couple of typos in rustdoc comments. 2022-06-24 08:18:15 -04:00
Ian Jackson 589c6e52bb Run maint/add_warning crates/*/src/{lib,main}.rs
Update all lint blocks
2022-06-23 19:15:42 +01:00
Ian Jackson 6c110a232c dirmgr: Use ... .find().is_none() rather than clippy's suggestion 2022-06-23 18:42:33 +01:00
Ian Jackson d2bd71c5f0 dirmgr: DirBootstrapStatus: Refactor mut_status_for using entries_mut
IMO this is rather less confusing.
2022-06-22 10:53:02 +01:00
Ian Jackson d36c433b36 dirmgr: Use new statuses() helper method
The two call sites become clearer, IMO.
2022-06-22 10:53:02 +01:00
Ian Jackson 24d43f83fd dirmgr: DirBootstrapStatus: Provide statuses and entries_mut
We're going to use these in a moment.

One returns entries and the other statuses simply because that's
what's actually going to be wanted.
2022-06-22 10:53:02 +01:00
Ian Jackson 5c51d03efb dirmgr: Remove a TODO comment
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/587#note_2815945
2022-06-22 10:53:02 +01:00
eta 4e3537746f Merge branch 'display_source_cleanup' into 'main'
Do not include error source() in display() format.

See merge request tpo/core/arti!598
2022-06-21 19:34:27 +00:00
Ian Jackson 9b4ba4893e Merge branch 'stalled_directory' into 'main'
Detect and report stalled directory downloads

Closes #468

See merge request tpo/core/arti!587
2022-06-21 19:13:40 +00:00
Nick Mathewson 08d9bbf33b Do not include error source() in display() format.
According to doc/Errors.md, and in keeping with current best
practices, we should not include display an error's `source()` as
part of that error's display method.  Instead, we should let the
caller decide to call source() and display that error in turn.

Part of #323.
2022-06-21 14:14:14 -04:00
Nick Mathewson 9b7849a293 Refactor DirStatus data structures.
AttemptId is now linked to DirStatus in StatusEntry.

DirBootstrapStatus now has an inner enum to make invalid status
unrepresentable.
2022-06-21 10:26:53 -04:00
Nick Mathewson 5509bd0269 Merge branch 'config-fix2' into 'main'
Use impl_standard_builder more and remove manual Default/builder impls

See merge request tpo/core/arti!594
2022-06-16 20:30:34 +00:00
Ian Jackson db51bdcbc1 Merge branch 'config-fix' into 'main'
Config handling and logging fixes

Closes #480

See merge request tpo/core/arti!589
2022-06-16 15:46:18 +00:00
Ian Jackson 2ca2c03059 impl_standard_builder: Use for tor_dirmgr::Authority 2022-06-16 15:42:06 +01:00
Ian Jackson 17caebd284 impl_standard_builder: Use for tor_dirmgr::DownloadSchedule 2022-06-16 15:42:06 +01:00
Ian Jackson 8b724d40f8 impl_standard_builder: Use for tor_dirmgr::DownloadScheduleConfig 2022-06-16 15:27:00 +01:00
Ian Jackson 2ab5c0861e impl_standard_builder: Use for tor_dirmgr::NetworkConfig 2022-06-16 15:27:00 +01:00