Commit Graph

514 Commits

Author SHA1 Message Date
Nick Mathewson 557a07677e guardmgr: Rename and clarify of non-persistent-state copy fns
These will need a bigger overhaul: see #612 and #611.
2022-10-27 16:40:35 -04:00
Ian Jackson 4f171acaf7 Clarify fix_consistency 2022-10-27 16:40:35 -04:00
Ian Jackson 7180079112 Clarify what it means to be "an identity" for a primary guard. 2022-10-27 16:40:35 -04:00
Ian Jackson 15d464a929 tor-guardmgr: Add big doc comment about guard id handling 2022-10-27 16:40:35 -04:00
Ian Jackson 26a7aa9aa6 BridgeDescProvider: Make BridgeDescEvent repr(u16) and derive traits
This will enable us to impl FlagEvent for it.
2022-10-27 15:57:54 +01:00
Ian Jackson a56efbfdf0 impl Hash for BridgeConfig and various PT information
The bridge descriptor manager wants to index data structures by the
BridgeConfig.
2022-10-27 15:57:54 +01:00
Ian Jackson a89daba1c8 tor-guardmgr: bridges: export BridgeDescError at toplevel 2022-10-27 15:57:54 +01:00
Nick Mathewson 20016df6fa guardmgr: Use new ambiguity-tolerant contains in state handling
This has subtleties; the comments try to explain them.
2022-10-25 11:15:53 -04:00
Nick Mathewson d65c351b33 guard sample: Change semantics of `contains` to handle ambiguity.
This is necessary for the (somewhat undesirable) lookup_ids function
to return an ID that the dirmgr can actually use to report successes
and failures.

As noted, lookup_ids will create problems down the road when we
implement relays. We should refactor it out before then.
2022-10-25 11:15:53 -04:00
Nick Mathewson 28f2e2997e Replace contains_relay with can_add_relay().
To see if we can add a relay as a guard, we need a conservative
test: any IDs in common with the given relay prevent us from adding
it.
2022-10-25 11:15:53 -04:00
Nick Mathewson 3a994c32dc Convert guard samples to use ByRelayIds.
This required a number of changes, which I've tried to document.
I've taken a conservative approach to modification, and I'm not
using any of the by_*_mut() functions (yet).  For cases which
potentially modify the whole set, I'm using into_values() and
collect() to ensure that it's re-indexed correctly, even though the
identities don't change.

I introduce some "TODO pt-client" comments here which I will resolve
in the next commit(s).
2022-10-25 11:15:53 -04:00
Ian Jackson f7e5892e40 BridgeDescEvent: Change the sole variant to SomethingChanged
And explain what this all means.
2022-10-25 15:32:56 +01:00
Nick Mathewson 8528d61a78 impl HasRelayIds for GuardId. 2022-10-25 09:23:28 -04:00
Nick Mathewson 40ec12b0cb Merge branch 'guards_as_bridges_part1' into 'main'
Allow GuardMgr to expose bridges as guards (part 1)

See merge request tpo/core/arti!785
2022-10-24 13:30:02 +00:00
Nick Mathewson bb117a4bd4 guardmgr: Refactor the interior of FirstHop.
Now it contains either an `OwnedChanTarget` or an `OwnedCircTarget`,
which will let `GuardMgr` return bridges that can be used to make
circuits.

As part of this change, it was necessary to revise some
address-modification functions that applied to filters and
`OwnedChanTarget`.  Now they do the smart thing, and remove only the
address that are in the `ChanMethod`.  This means that the addresses
from HasAddrs are still accurate about which addresses the relay
"has".
2022-10-24 08:59:12 -04:00
Ian Jackson 0924fb3623 tor-guardmgr: descs: Make BridgeDescError DynClone 2022-10-21 16:22:17 +01:00
Ian Jackson f4ec3e6cec tor-guardmgr: descs: Make BridgeDescError into a trait 2022-10-21 16:22:13 +01:00
Nick Mathewson fa1d2f453a guardmgr: Hold FallbackDir in fallback::set::Entry
This resolves an old TODO, and will simplify our work a little.
2022-10-21 09:21:43 -04:00
Nick Mathewson 19fdf196d8 guardmgr: Add bridges sample, encode sample ID in FirstHopId.
The most important part of this commit is to make sure that each
`FirstHopId` includes the `GuardSetSelector` from which the guard
was selected.  Doing this lets us be certain that when we report
that a guard has succeeded or failed, we're reporting it in the
right context.

Additionally, this commit uses strum to make an iterator over the
samples, so that we can make sure that our "for each sample" code is
robust against future changes, and we don't miss the bridge sample.
2022-10-21 09:21:43 -04:00
Ian Jackson 803131c1ea Merge branch 'bridge-internal-apis' into 'main'
Use BridgeConfig to identify bridges in two places

See merge request tpo/core/arti!781
2022-10-20 16:14:14 +00:00
Ian Jackson 9c7ea41322 impl ChanTarget etc. for BridgeConfig 2022-10-20 14:16:34 +01:00
Ian Jackson 11c61a3d7f bridge descriptors: Change types of the descriptor cache
Now keyed by Arc<BridgeConfig>, and the values can be errors.

Currently there is no implementation so there can't be any errors,
but the error enum will become nonempty.
2022-10-20 13:28:23 +01:00
Nick Mathewson 2845ee24bf fix a couple of spelling errors in guardmgr 2022-10-19 07:49:39 -04:00
Nick Mathewson b4cc98b21b Merge branch 'readmes' into 'main'
Abolish maint/readme and use doc include

Closes #603

See merge request tpo/core/arti!768
2022-10-12 15:16:18 +00:00
Ian Jackson f84d8777db cargo fmt to remove blank lines
Apparently cargo fmt doesn't like these, which my perl rune didn't
delete.

This commit is precisely the result of `cargo fmt`.
2022-10-12 15:29:04 +01:00
Ian Jackson d9910dba08 Replace all README copies in src/lib.rs with includes
The feature we want is `#[doc = include_str!("README.md")]`, which is
stable since 1.54 and our MSRV is now 1.56.

This commit is precisely the result of the following Perl rune:
  perl -i~ -0777 -pe 's{(^//!(?!.*\@\@).*\n)+}{#![doc = include_str!("../README.md")]\n}m' crates/*/src/lib.rs
2022-10-12 15:29:03 +01:00
Ian Jackson 19de971775 struct BridgeConfig: Rename from Bridge
Fixes #599
2022-10-12 13:54:25 +01:00
Ian Jackson 029a11c833 struct Bridge: add ref to ticket 2022-10-12 13:51:52 +01:00
Ian Jackson f9e841841a tor-guardmgr: Export BridgeParseError from toplevel
Callers could `use` it as `tor_guardmgr::config::BridgeParseError` but
it seems unecessary to force them to.
2022-10-12 13:50:29 +01:00
Ian Jackson 82ad9eea5f Allow "clippy::single_char_pattern" in tests.
This lint exists for perf reasons, and this is rarely relevant in
tests.

Using double quoted str is generally cognitively less burdensome.
2022-10-12 13:50:29 +01:00
Nick Mathewson 74d4c73d09 Change multiplicity of ChannelMethod and addresses
Now each `ChanTarget` has at most one `ChannelMethod`, and only
`Direct` `ChannelMethods` can have multiple addresses.

Closes #600.
2022-10-11 08:53:31 -04:00
Ian Jackson 85464bc770 tor-guardmgr: Fix build without pt-client feature 2022-10-11 12:22:46 +01:00
Nick Mathewson 299ebd729d tor-linkspec: Remove the old OwnedFoo::new() functions
These are now builders.
2022-10-06 15:13:05 -04:00
Nick Mathewson ce09b7c96f Begin revising HasAddr and its relationship to ChanTarget
HasAddr used to mean "Here are addresses that I have, at which I can
be contacted."  But "Where (and how) can I be contacted?" is now a
question for HasChannelMethod to answer.

(We still need to have "HasAddr", though, so we can answer things
like "what country is this relay in" and "are these relays in the
same /8?")

So this commit introduces:
  * A new trait for adding an implementation of HasChannelMethod in
    terms of HasAddr.
  * A requirement on ChanTarget that it needs to implement
    HasChannelMethod.

There is some temporary breakage here, marked with "TODO pt-client",
that I'll fix later in this branch.
2022-10-06 15:13:05 -04:00
Nick Mathewson 1d70bf4ddd Merge branch 'bridge_containers' into 'main'
Start implementing more data structures to hold Bridge descriptors.

See merge request tpo/core/arti!755
2022-10-06 17:16:44 +00:00
Nick Mathewson 3f419f7f65 Extend some comments about bridge descriptors 2022-10-06 12:56:28 -04:00
Nick Mathewson fac975ddfb Add a temporary HasAddrs for BridgeRelay
See comment for an explanation of the next issue here.
2022-10-04 17:53:40 -04:00
Nick Mathewson 23b3b0553f BridgeRelay: Implement more traits.
Also add a BridgeRelayWithDesc type (name tbd) to guarantee that
a bridge relay really does have a known descriptor before you
try to build a circuit with it.
2022-10-04 17:49:32 -04:00
Nick Mathewson d9e3d38bd6 Start on a new BridgeRelay type.
This is the one we'll actually use to connect to bridges. It
has a `Bridge` line, and an optional `BridgeDesc`.

Maybe this will turn into a `BridgeRelay<'a>` by analogy to `Relay`
some time; I'm not sure.
2022-10-04 17:49:32 -04:00
Nick Mathewson 013b9bff1b Implement HasRelayIds for Bridge. 2022-10-04 17:49:32 -04:00
Nick Mathewson d4f26581c0 Implement BridgeDesc and BridgeDescList.
BridgeDesc is a separate type to make sure that we do not confuse
bridges' descriptors with the descriptors from other routers down
the road.  (Bridges' descriptors need to be used differently, and
treated as more private.)

With this code, BridgerDescList is now just an alias for
`ByRelayIds<BridgeDesc>`, which is pretty keen.
2022-10-04 17:49:32 -04:00
Nick Mathewson 6341057129 Temporarily suppress a false positive from nightly.
Nightly rust gives a warning about this "pub use", but the warning
is a false positive.  Since it doesn't seem to be going away in a
hurry, let's suppress it for now.
2022-10-04 16:40:34 -04:00
Ian Jackson 9705ef3fea Merge branch 'bridge-parse' into 'main'
Parse (and format) bridge lines

See merge request tpo/core/arti!745
2022-10-03 20:13:02 +00:00
Ian Jackson 193d0f7007 pt: bridge line parsing: overhaul errors
Include the offending word in all the applicable errors.
Always print it with {word:?}.

As a consequence, there are no From impls any more and error
generation/conversion is by hand in all cases.

Clarify InvalidPtOrAddr vs InvalidIAddrorPt, and don't make the
attempted parse be a source error for those.

Where we still have source errors, don't print them in Display.
2022-10-03 20:45:54 +01:00
Ian Jackson 00fb96650b pt: Check PT key=value syntax
This is not perfect but it at least ensures that our own parsing and
printing code works correctly with all the values we accept.
2022-10-03 20:45:54 +01:00
Ian Jackson b67b5ac536 pt: Make PtTarget::push_setting fallible, and take Into
It has its own error type PtTargetInvalidSetting.

In check_doc_features, adjust suppression to new code.
2022-10-03 20:45:54 +01:00
Ian Jackson a237b80ad8 pt: bridge line parsing: Add commentary 2022-10-03 20:45:54 +01:00
Ian Jackson f6a6fe0d8e pt: Document Bridge line syntax 2022-10-03 20:45:54 +01:00
Nick Mathewson 88a06225b9 Remove semver.md files now that arti 1.0.1 is out. 2022-10-03 14:16:56 -04:00
Nick Mathewson 102e3c221d Bump minor version of tor-rtcompat and most of its dependents
(Since the APIs for the `Schedule::sleep*` functions changed, this
is a breaking change in tor-rtcompat.  Therefore, the Runtime trait
in tor-rtcompat is now a different trait.  Therefore, anything that
uses the Runtime trait in its APIs has also broken.)
2022-10-03 10:01:46 -04:00
Nick Mathewson 388e0cbab5 Bump crates that have had backward compatible API changes. 2022-10-03 09:52:34 -04:00
Ian Jackson 5be8830b3f pt: bridge parsing: Add an IPv6 test case 2022-09-30 17:08:14 +01:00
Ian Jackson 47aae0c27a pt: impl Display for Bridge and test it 2022-09-30 17:08:14 +01:00
Ian Jackson 0dfa69510e implement FromStr for Bridge, and test it 2022-09-30 17:08:14 +01:00
Ian Jackson 4b28c0e120 tor-guardmgr: Introduce pt-client feature
guardmgr needs to know how to parse Bridge lines into Bridge structs
so it needs to know about PTs specifically
2022-09-30 17:08:14 +01:00
Ian Jackson c41ea6e07b pt: Make some more types Eq
I wanted this for testing.  I think it's reasonable to promise this.
2022-09-30 17:08:14 +01:00
Nick Mathewson b146dacf1d Mark all bridge and pt features as experimental for now.
Also, document the features.

Closes #588.
2022-09-29 10:59:19 -04:00
Nick Mathewson 3272c6fb04 Add the necessary APIs for bridge-based circuit construction. 2022-09-26 11:23:21 -04:00
Nick Mathewson 1196e1b680 GuardMgr/DirMgr: Add APIs for bridge descriptor lists 2022-09-26 11:23:21 -04:00
Nick Mathewson ab5dd6cbc3 GuardMgr: Add TODOs for bridge support.
This covers only the most basic notions of working with bridges:
that we need a separate set of guards, and that they have to
come from the list of known bridges.
2022-09-26 11:23:21 -04:00
Nick Mathewson e097d64417 A basic configuration type for a bridge.
This type goes in tor-guardmgr, since that's where decisions about
circuits' first hops are made.

There are a lot of "todo"s here for us to resolve.
2022-09-26 11:23:21 -04:00
Nick Mathewson a5f704f443 Remove semver.md from arti-1.0.0 2022-09-07 09:17:00 -04:00
Nick Mathewson 96875ea208 Bump crate versions in preparation for Arti 1.0.0 release.
Because we want to work more on ensuring that our semver stability
story is solid, we are _not_ bumping arti-client to 1.0.0 right now.

Here are the bumps we _are_ doing.  Crates with "minor" bumps have
had API breaks; crates with "patch" bumps have had new APIs added.

Note that `tor-congestion` is not bumped here: it's a new crate, and
hasn't been published before.

```
tor-basic-utils         minor
fs-mistrust             minor
tor-config              minor
tor-rtcompat            minor
tor-rtmock              minor
tor-llcrypto            patch
tor-bytes               patch
tor-linkspec            minor
tor-cell                minor
tor-proto               minor
tor-netdoc              patch
tor-netdir              minor
tor-persist             patch
tor-chanmgr             minor
tor-guardmgr            minor
tor-circmgr             minor
tor-dirmgr              minor
arti-client             minor
arti-hyper              minor
arti                    major
arti-bench              minor
arti-testing            minor
```
2022-09-01 08:59:49 -04:00
Nick Mathewson 0f133de6b9 Downgrade "guard set loaded" messages
These aren't interesting to the user.
2022-08-25 11:03:19 -04:00
trinity-1686a 7f939fa480 enable doc_auto_cfg feature on every crate when documenting for docs.rs 2022-08-24 18:22:41 +02:00
Nick Mathewson 36a28a8587 Merge branch 'main' into 'linkspec_refactor_v3'
# Conflicts:
#   crates/tor-netdir/semver.md
2022-08-10 14:48:57 +00:00
Nick Mathewson 6dc8b1af20 Make sure all HasRelayIds constaints allow ?Sized. 2022-08-10 10:39:47 -04:00
Nick Mathewson 37b3daa11d tor-netdir: Collapse by_id and by_relay_id into a single fn.
There are some downstream changes required for this to work, but
they are all just unit tests that could no longer infer the type of
an Ed25519 key.
2022-08-10 10:39:46 -04:00
Nick Mathewson 2d4507ff35 Final (?) API revisions for tor-linkspec
With this change, each individual identity type becomes optional.
The functions that expose them unconditionally are now in a "legacy"
trait that only some downstream types are expected to implement.

There are new convenience APIs in HasRelayIds:
  * to return Option<&keytype>,
  * to see if one identity-set contains another.

This commit will break several downstream crates!  For the
reviewer's convenience, I will put the fixes for those crates into a
series of squash! commits on this one.

tor-netdir
----------

Revise tor-netdir to accept optional identities.  This required some
caveats and workarounds about the cases where we have to deal with a
key type that the tor-netdir code does not currently recognize at
all.  If we start to add more identity types in the future, we may
well want more internal indices in this code.

tor-proto
---------

In order to make tor-proto support optional identities, there were
fewer changes than I thought.  Some "check" functions needed to start
looking at "all the ids we want" rather than at "the two known IDs";
they also needed to accommodate that case where we don't have an ID
that we demand.

This change will also help with bridges, since we want to be able to
connect to a bridge without knowing all of its IDs up front.

The protocol currently _requires_ the two current ID types in some
places. To deal with that, I added a new `MissingId` error.

I also removed a couple of unconditional identity accessors for
chanmgr; code should use `target().identity(...)` instead.

tor-chanmgr
-----------

This is an incomplete conversion: it does not at all handle channel
targets without Ed25519 identities yet.  It still uses those
identities to index its internal map from identity to channel; but
it gives a new `MissingId` error type if it's given a channel target
that doesn't have one.

We'll want to revise the map type again down the road when we
implement bridges, but I'd rather not step on the channel-padding
work in progress right now.

tor-guardmgr
------------

This change is mostly a matter of constructing owned identity types
more sensibly, rather than unwrapping them directly.

There are some places marked with TODOs where we still depend on
particular identity types, because of how the directory protocol
works.  This will need revisiting when we add bridge support here.

tor-circmgr
-----------

These changes are just relatively simple API changes in the tests.
2022-08-10 10:39:37 -04:00
Nick Mathewson b56e1bf119 Introduce a RelayIdSet and use it in place of HashSet<RelayId>.
This makes lookups a little more efficient.  I do with that HashSet
just supported this use-case, but for now this seems our best option.
2022-08-10 10:39:36 -04:00
Nick Mathewson deaf8b657d Teach guard restrictions about RelayId.
This implementation is (sadly) too copy-heavy or now, because
HashSet<RelayId> can't be indexed with RelayIdRef.
2022-08-10 10:39:36 -04:00
Nick Mathewson 8af41bd65f Use accessors in two places where tests look at specific keys.
(It's safe to require these specific keys, since we constructed the
objects for testing, and we know for certain that they contain given
keys.)
2022-08-10 10:39:36 -04:00
Nick Mathewson 6ee9535126 guardmgr: Avoid a case of using HasRelayIds key methods. 2022-08-02 14:33:32 -04:00
Nick Mathewson 0bfc92ce8e netdir: Add and use a new ids_listed() function. 2022-08-02 14:32:57 -04:00
Nick Mathewson fdfc5f29e3 Introduce HasRelayIds::same_relay_ids.
This method tells if two HasRelayIds contain exactly the same set of
Relay identities, and is generally useful for debugging.
2022-08-02 14:19:00 -04:00
Ian Jackson d523ad495a Merge branch 'inc-rename' into 'main'
Rename `.inc` and other included files to end in `.rs`

Closes #381

See merge request tpo/core/arti!645
2022-08-02 16:48:49 +00:00
Nick Mathewson 0e41913a4c guardmgr: Change APIs that used to take IDs.
These are the other inspiration for #428.
2022-08-02 12:40:57 -04:00
Nick Mathewson 2d32a4602b guardmgr: Replace IdPair with RelayIds
I believe that this was the original motivation behind #428.
2022-08-02 12:40:57 -04:00
Nick Mathewson b68a3ed5e5 tor-linkspec: Refactor out traits to represent a relay's ID set.
We want the set of identities supported by a relay to be extensible
in the future with minimal fuss; we'd also like to make working
with these ID sets more convenient.  To handle that, this commit
adds a new trait for "Something that has the same IDs as a relay"
and a new object for "an owned representation of a relay's IDs."

This commit introduces a similar trait for "Something with a list of
SocketAddr, like a relay has."  There's no owned equivelent for
that, since Vec<SocketAddr> is already a thing.

Closes #428.
2022-08-02 12:40:23 -04:00
Nick Mathewson b16c042004 Now that versions have bumped, remove semver.md files. 2022-08-01 10:07:12 -04:00
Nick Mathewson 859ae4acd0 Bump patch versions on crates that have new APIs.
Do _not_ bump the dependency versions on crates that have had no
changes since arti 0.0.5, since those crates do not depend on the
new APIs.

```
cargo set-version -p tor-basic-utils   --bump patch
cargo set-version -p tor-llcrypto      --bump patch
git restore crates/tor-checkable
git restore crates/tor-consdiff
git restore crates/tor-rtmock
```
2022-08-01 09:56:29 -04:00
Nick Mathewson 511cb46997 Bump minor version on crates with deps with breaking changes.
This performs the transitive closure of the last operation:
everything that depends on a crate with a breaking change gets the
version which it depends on bumped.

```
cargo set-version -p tor-proto         --bump minor
cargo set-version -p tor-netdoc        --bump minor
cargo set-version -p arti-hyper        --bump minor
cargo set-version -p arti-bench        --bump minor
cargo set-version -p arti-testing      --bump minor
cargo set-version -p tor-config        --bump minor
```
2022-08-01 09:49:13 -04:00
Nick Mathewson f5dcb98f06 Bump minor versions on all crates that have had breaking changes.
Done with these commands:

```
cargo set-version -p fs-mistrust       --bump minor
cargo set-version -p tor-bytes         --bump minor
cargo set-version -p tor-socksproto    --bump minor
cargo set-version -p tor-cert          --bump minor
cargo set-version -p tor-linkspec      --bump minor
cargo set-version -p tor-cell          --bump minor
cargo set-version -p tor-netdir        --bump minor
cargo set-version -p tor-persist       --bump minor
cargo set-version -p tor-chanmgr       --bump minor
cargo set-version -p tor-guardmgr      --bump minor
cargo set-version -p tor-circmgr       --bump minor
cargo set-version -p tor-dirclient     --bump minor
cargo set-version -p tor-dirmgr        --bump minor
cargo set-version -p arti-client       --bump minor
cargo set-version -p arti              --bump minor
```
2022-08-01 09:43:09 -04:00
trinity-1686a 19a71534ec fix runtime issues on ios 2022-07-30 15:42:37 +02:00
eta f02d732b7f Rename `.inc` and other included files to end in `.rs`
In order to mitigate syntax highlighting issues and a rust-analyzer bug
(https://github.com/rust-analyzer/rust-analyzer/issues/10178), rename
files that are included with the `include!` macro to have a `.rs`
extension.

Make sure the included files are outside `src/`, in order to not confuse
humans and automated editing tools that might mistake them for valid
Rust modules.

fixes arti#381
2022-07-26 15:09:28 +01:00
Nick Mathewson 0bca4d825c GuardMgr: Update to use new NetDirProvider API. 2022-07-26 10:41:06 +01:00
Nick Mathewson 1735b70769 Note semver break in tor-guardmgr::Error. 2022-07-07 13:20:57 -04:00
Nick Mathewson 3a2631ff9e Remove now-unused NoGuardsUsable case. 2022-07-07 13:16:06 -04:00
Nick Mathewson 287fe915f9 Improve message for failure to select a guard.
This uses similar techniques to the commit I just did for Fallbacks.
2022-07-07 13:16:06 -04:00
Nick Mathewson d7a626aaa8 Add more information to failed-to-select fallback errors.
Also re-order the filters to be a little more logical.
2022-07-07 13:16:06 -04:00
Nick Mathewson 373b9e2ccf GuardMgr: tweak an error message. 2022-07-06 15:39:14 -04:00
Nick Mathewson 1dda69af2b Remove semver.md files now that 0.5.0 is out 2022-06-24 13:31:38 -04:00
Nick Mathewson 20435aea75 Bump crate and dependency versions.
These were done with the following commands:

```
cargo set-version -p tor-basic-utils   --bump patch
cargo set-version -p fs-mistrust       --bump minor
cargo set-version -p tor-error         --bump patch
cargo set-version -p tor-config        --bump patch
cargo set-version -p tor-units         --bump patch
cargo set-version -p tor-rtcompat      --bump minor
cargo set-version -p tor-llcrypto      --bump patch
cargo set-version -p tor-bytes         --bump minor
cargo set-version -p tor-socksproto    --bump minor
cargo set-version -p tor-cert          --bump minor
cargo set-version -p tor-cell          --bump minor
cargo set-version -p tor-proto         --bump minor
cargo set-version -p tor-netdoc        --bump patch
cargo set-version -p tor-netdir        --bump minor
cargo set-version -p tor-persist       --bump patch
cargo set-version -p tor-chanmgr       --bump minor
cargo set-version -p tor-guardmgr      --bump minor
cargo set-version -p tor-circmgr       --bump patch
cargo set-version -p tor-dirclient     --bump patch
cargo set-version -p tor-dirmgr        --bump minor
cargo set-version -p arti-client       --bump patch
cargo set-version -p arti              --bump minor
cargo set-version -p arti-bench        --bump minor
cargo set-version -p arti-testing      --bump minor
```
2022-06-24 12:00:22 -04:00
Ian Jackson 4831f9d38c Merge branch 'clippy' into 'main'
Fix clippy nightly again

See merge request tpo/core/arti!603
2022-06-24 14:13:58 +00:00
Ian Jackson 23c352b96b clippy: Consolidate many lints in maint/add_warning
Found these by disabling the nightly dbg macro special case.  Now, we
have a mechanism for globally adding suppressions to tests, we can use
that instead.
2022-06-24 14:23:38 +01:00
Nick Mathewson ce4d062c09 Update README.md files from rustdoc. 2022-06-24 08:02:56 -04:00
Ian Jackson 589c6e52bb Run maint/add_warning crates/*/src/{lib,main}.rs
Update all lint blocks
2022-06-23 19:15:42 +01:00
Nick Mathewson 9ae57e8bc4 Merge branch 'reachable_addrs_v2' into 'main'
Implement support for reachable_addrs

Closes #491 and #93

See merge request tpo/core/arti!583
2022-06-17 13:16:16 +00:00
Nick Mathewson f4c9b9cc1e Rename guardset-selection function.
Also, improve its documentation.
2022-06-17 08:47:57 -04:00
Nick Mathewson 03568cd4a5 Rename pick_guard, pick_guard_ext.
Now the primary exposed function is `pick_guard` again.

This commit is just function renaming.
2022-06-17 08:47:57 -04:00
Nick Mathewson 1be44891c9 Refactor and document issues with modify_hop.
At the site of modify_hop, we now have a comment explaining the
internal-error issue.

To make the internal error less likely, we lower the modify_hop call
in lib.rs into GuardSet, where it can make sure it's looking at the
same filter as was used to select the guard.

The function name "pick_guard_ext" is not permanent; I'm going to
rename it in the next commit.
2022-06-17 08:47:57 -04:00
Nick Mathewson 96dfa97473 API-fix for extend_sample_as_needed.
Previously, the API said "you need to call this in a loop till it
returns false".  We did that in one place, but not another.

With the introduction of filters, forgetting to loop here becomes a
bug: so instead, change the behavior of extend_sample_as_needed so
it handles looping itself.
2022-06-17 08:47:57 -04:00
Nick Mathewson a92dfa0b2a Remove some outdated comments.
These all say, in one form or another, "there is no guard filtering;
there is only one selection".  That's now false.
2022-06-17 08:47:57 -04:00
Nick Mathewson 89f9e1decb Tweak parameters in guardmgr tests to improve testnet behavior.
The fake network we construct for these tests is small in ways that
can sometimes cause weird behavior with guard filters.  We fix this
by adjusting the parameters of the guard selection algorithm
accordingly in the tests.

With these new parameters, #491 no longer occurs.

This commit also adds comments to explain why the parameters are set
as they are.

Closes #491.
2022-06-17 08:47:57 -04:00
Nick Mathewson 48a86506be GuardMgr: Support for multiple guard sets
guard-spec.txt specifies that we have multiple separate samples of
guards that we can use depending on whether the filter is
restrictive or not.  Here we implement the rules for switching
between samples.
2022-06-17 08:47:57 -04:00
Nick Mathewson e098a5a303 Move set_filter into GuardMgrInner.
Convert its argument type to Option<&NetDir> to better match the
rest of the API.
2022-06-17 08:47:57 -04:00
Nick Mathewson a720f4f667 GuardMgr: Function to tell how permissive a filter is.
When we're filtering guards, we have to check whether the filter is
"restrictive": if it forbids most of the guards (by bandwidth), we
keep its guards separated from the main set.  If it is
super-restrictive, we also warn.

This functionality is specified in guard-spec.txt.
2022-06-17 08:47:57 -04:00
Nick Mathewson e3c2a86195 GuardMgr: Also apply filters to fallback directories. 2022-06-17 08:46:16 -04:00
Nick Mathewson a189aaf1fb GuardMgr: remove disallowed addresses from returned FirstHops.
Since a guard can have a bunch of addresses, and the guard is
permitted if any one of those addresses is permitted, then we might
decide to use a guard with some non-permitted addresses.  Thus, we
need to filter those addresses before returning the view of the
guard as a FirstHop.
2022-06-17 08:46:16 -04:00
Nick Mathewson 23218fe037 GuardMgr: Remove old testing filter variant.
We don't need to restrict based on bits in the key id any longer,
since we have a real filter.
2022-06-17 08:46:16 -04:00
Nick Mathewson 9c4f31edc1 GuardMgr: Add a new filter type based on reachable addresses. 2022-06-17 08:46:16 -04:00
Nick Mathewson 801b6bec5f Guardmgr: Change the GuardFilter API.
The guard filter is now a set of restrictions that can be placed on
allowable guards.
2022-06-17 08:46:16 -04:00
Nick Mathewson 5509bd0269 Merge branch 'config-fix2' into 'main'
Use impl_standard_builder more and remove manual Default/builder impls

See merge request tpo/core/arti!594
2022-06-16 20:30:34 +00:00
Ian Jackson e004316942 impl_standard_builder: Use for tor_guardmgr::FallbackDir 2022-06-16 15:42:06 +01:00
Nick Mathewson 5e05573ec9 Merge branch 'clippy_20220614' into 'main'
Fix several clippy issues, most with nightly

See merge request tpo/core/arti!588
2022-06-15 17:34:53 +00:00
Nick Mathewson 647d4410bb Fix clippy::significant_drop_in_scrutinee warnings
This is apparently a new warning from clippy nightly, documented in
https://rust-lang.github.io/rust-clippy/master/index.html#significant_drop_in_scrutinee .

I'm not in love with the temporary variables that this warning wants
me to introduce, but it does seem like a decent way to avoid some
kinds of deadlock.
2022-06-15 11:52:30 -04:00
Ian Jackson 3e035927f2 tor-netdir: testnet: Make construct_netdir infallible (rustfmt)
Run rustfmt.  Separate commit to make review of the substantive commit
easier.
2022-06-13 14:26:32 +01:00
Ian Jackson 7d223ac9e6 tor-netdir: testnet: Make construct_netdir infallible
This is a *lot* of unwraps.  The function takes no parameters and
is used only for testing.  It ought to be infallible.
2022-06-13 14:25:45 +01:00
Nick Mathewson 5854e4bbd9 Merge branch 'use-testing-rng' 2022-06-07 19:02:12 -04:00
Nick Mathewson b1b7f30de0 Merge branch 'netdir_provider_in_guardmgr_v2' into 'main'
Use NetDirProvider in GuardMgr

Closes #93

See merge request tpo/core/arti!568
2022-06-07 20:46:27 +00:00
Nick Mathewson 20eeca135b GuardMgr: Use installed netdir provider when we want a netdir
Some of our existing code optionally takes a netdir from the
caller.  When it doesn't give us one, use the netdir from the
installed NetDirProvider.

(Possibly someday we should remove the NetDir arguments
entirely. I'm deferring that because there are only two APIs
affected, and because making this change would force us to rewrite a
pretty large mess of unit tests.)
2022-06-07 11:44:51 -04:00
Nick Mathewson dc0a4e3c3d Move responsibility for GuardMgr NetDir updates to GuardMgr.
Previously it was the job of a task in CircMgr to do this; but we're
going to want to give GuardMgr full access to the latest NetDir for
this, and for other code-simplification reasons.

With this change I'm deprecating a couple of functions in
tor-circmgr.  It's no longer necessary for us to have an artificial
external way for you to feed new NetDirs to a circmgr.  (I could
just remove them, but I want practice deprecating.)
2022-06-07 11:44:51 -04:00
Nick Mathewson 7f785b66c2 Upgrade float_eq dev-dependency to 1.0.0 2022-06-07 08:03:55 -04:00
Nick Mathewson b4c0febd97 Refactor fallback-set tests to work with deterministic PRNG.
The trouble was that one of the helper functions they used
did not take a PRNG as an argument.
2022-06-02 14:59:55 -04:00
Nick Mathewson 967ea67b7d Use testing_rng() in tests throughout our crates.
This only affects uses of thread_rng(), and affects them all more or
less indiscriminately.  One test does not work with
ARTI_TEST_PRNG=deterministic; the next commit will fix it.
2022-06-02 14:56:42 -04:00
Ian Jackson 4f42101554 lints: Add let_unit_value allow to all crates
From running add_warning, with manual picking of the right
hunks/lines.
2022-05-31 15:23:52 +01:00
Ian Jackson ba0843da4a lints: Add lint block delimiters to every crate
This was the result of:
  maint/add_warning crates/*/src/{lib,main}.rs
and then manually curating the results.
2022-05-31 13:00:31 +01:00
Orhun Parmaksız bfd41ddb5f
Lexically sort Cargo.toml dependencies
Utilize cargo-sort: https://github.com/DevinR528/cargo-sort

Signed-off-by: Orhun Parmaksız <orhunparmaksiz@gmail.com>
2022-05-28 20:05:51 +03:00
Nick Mathewson 4326aa1de9 Regenerate version bump from previous commit.
This commit was made by reverting the previous commit, then
re-running the script I used to generate it.  In theory there should
be no semantic changes: only changes due to improved formatting from
cargo edit.
2022-05-27 10:18:52 -04:00
Nick Mathewson b232365a75 Semantic version changes for Arti 0.4.0 release
I followed the following procedure to make these changes:

* I used maint/changed_crates to find out which crates had changed
  since 0.3.0.
* I used grep and maint/list_crates to sort those crates in
  topological (dependency) order.
* I looked through semver_status to find which crates were listed as
  having semver-relevant changes (new APIs and breaking changes).
* I scanned through the git logs of the crates with no
  semver-relevant changes listed to confirm that, indeed, they had
  no changes.  For those crates, I incremented their patch-level
  version _without_ changing the version that other crates depend on.
* I scanned through the git logs of the crates with no
  semver-relevant changes listed to confirm that, indeed, they had
  no obvious breaking changes.
* I treated all crates that depend on `arti` and/or `arti-client` as
  having breaking changes.
* I identified crates that depend on crates that have changed, even
  if they have not changed themselves, and identified them as having
  a non-breaking change.
* For all of the crates, I used `cargo set-version -p $CRATE --bump
  $STATUS` (where `STATUS` is `patch` or `minor`) to update the
  versions, and the depended-upon versions.
2022-05-27 09:01:20 -04:00
Nick Mathewson 460117f2b0 Allow use of RetainMut, since we do not require Rust 1.61 yet. 2022-05-24 15:09:00 -04:00
Ian Jackson 86a3e006d3 impl_standard_builder: Test the Deserialize impl
Test the Deserialize impl of every config struct.

This detects bugs like the one fixed in !502.

The macro now becomes more complex because it needs to take options.
Right now this tt-munching option parser is overkill, but this
leave space for further options in the future.
2022-05-12 18:50:26 +01:00
Ian Jackson c1c6f2b376 Rename impl_standard_builder from impl_default_via_builder
I have Plans for this macro.  In particular:

 * I have a wip branch which tests that the Builder can be
   deserialised from an empty config (ie, that config reading
   of a config with a blank section for this item works).

 * I think we should autogenerate $Config::builder(),
   and promote that, rather than $ConfigBuilder::default().
   This macro could do that.
2022-05-12 15:59:13 +01:00
Ian Jackson 7909a51ff4 No longer derive Default on three structs which derive Builder 2022-05-11 18:27:15 +01:00
Nick Mathewson 7b93091f57 Bump the version of every* crate to 0.3.0
* Except for safelog and fs-mistrust, which are new.
2022-05-06 10:03:15 -04:00
Nick Mathewson 9447e69355 Update README.md files (automated). 2022-05-06 09:51:11 -04:00
Ian Jackson 030289481f Switch to derive_builder_arti_fork
For reference, the git source for this crate (and the others in its
workspace) currently lives in my personal github account (ijackson).
If this fork turns out to be long-lived and gains features and/or
users, it would be good to move it to a gitlab somewhere.

I have granted Nick crate ownership on the crates.io system.
2022-05-06 13:36:40 +01:00
Ian Jackson d47e94b459 config derive attrs: Make builders serde, and validated structs not
* Builders additionally derive: Debug, Serialize, Deserialize.

 * Validated structs no longer derive: Serialize, Deserialize
   and all related attributes deleted.

 * As a consequence, all the `#[serde(deny_unknown_fields)]`
   are gone.  That means that right now unknown fields are totally
   ignored.  This is good for compatibility but poor for useability.
   Doing something better here is arti#417, in progress.

 * As a consequence, delete tor_dirmgr::retry::default_parallelism.
   (The default value was already duplicated into a builder attr.)
2022-05-05 10:35:52 +01:00
Ian Jackson a334f17262 Merge branch 'socket-addr-list-builder' into 'main'
FallbackDir: orports: Introduce and use VecBuilder

See merge request tpo/core/arti!474
2022-05-04 18:13:45 +00:00
Ian Jackson 4ad4cae418 FallbackDir: Use VecBuilder for orports
And drop the ad-hoc orport() method.  This brings FallbackDir's
orports field in line with our list builder API.

The general semver note in "configuation" seems to cover most of this.
2022-05-04 17:18:55 +01:00
Ian Jackson cc394ca9b9 FallbackDir: Do orport validation after autogenerated build
This avoids it having to recapitulate defaulting logic.
2022-05-04 17:18:55 +01:00
Ian Jackson 6791758d0b FallbackDir: Do not consider defaulted-empty orport list to be valid 2022-05-04 17:18:55 +01:00
Ian Jackson 015db3d78d GuardUsage: restrictions: Use list builder
Although these do not appear in the config, it does have a builder.
It seems sensible to get rid of this ad-hoc list manipulation site,
and replace it with our standard list builder API.

define_list_builder_helper requires that the builder element type be
Deserialize.  Currently GuardUsageRestriction is a transparent, public
enum, so we aren't really exposing anything.

We could introduce GuardUsageRestrictionBuilder now, but
since it's not in the config and thereofore only in the public API of
the lower crates, we can definitely put that off.
2022-05-04 16:16:38 +01:00
Ian Jackson 4bca912715 Change builder list API
The new API is (roughly) as discussed in
  https://gitlab.torproject.org/tpo/core/arti/-/issues/451

This is quite a large commit and it is not convenient to split it up.
It contains the following changes:

 * Redo the list builder and accessor macros implemnetation,
   including docs and tests.

 * Change uses of define_list_config_builder.  In each case:
   - Move the docs about the default value to the containing field.
   - Remove the other docs (which were just recapitulations, and
     are now not needed since the ListBuilder is no longer public).
   - Rewmove or replace `pub` in the define_list_builder_helper call,
     so that the builder is no longer public.
   - Change the main macro call site to use define_list_builder_helper.
   - Add a call to define_list_builder_accessors.

 * Make the module `list_builder` pub so that we have somewhere to
   put the overview documentation.

 * Consequential changes:
   - Change `outer.inner().replace(X)` to `outer.set_inner(X)`
   - Consequential changes to imports (`use` statements).
2022-05-04 13:50:10 +01:00
eta 6f787e1e77 Merge branch 'derive-builder-git-fixup' into 'main'
derive_builder: Use git dep everywhere, rather than cargo patch

See merge request tpo/core/arti!477
2022-04-27 14:31:08 +00:00
Ian Jackson a97ad69855 derive_builder: Use git dep everywhere, rather than cargo patch
The `[patch]` approach causes the tree not to build when used as a
dependency, unless the `[patch]` is replicated into the depending
project.

Instead, replace our `derive_builer =` dependencies with a reference
to a specific git commit:

  perl -i~ -pe 'next unless m/^derive_builder/; s#"(0\.11\.2)"#{ version = "$1", git = "https://github.com/ijackson/rust-derive-builder", rev = "ba0c1a5311bd9f93ddf5f5b8ec2a5f6f03b22fbe" }#' crates/*/Cargo.toml

Note that the commitid has changed.  This is because derive_builder is
in fact a workspace of 4 crates.  3 of them are of interest to arti
itself (the 4th exists only for testing).  So the same "add git
revision" treatment had to be done to the `derive_builder` and
`derive_builder_macro` crates.  Each dependency edge involves a new
commit in the derive_builder workspace, since we can't create a git
commit containing its own commitid.  (We want to use commits, rather
than a branch, so that what we are depending on is actually properly
defined, and not subject to the whims of my personal github
namespace.)

There are no actual code changes in derive_builder.
2022-04-27 14:57:59 +01:00
Samanta Navarro c53818d496 Fix grammar and typos 2022-04-27 13:52:13 +01:00
Nick Mathewson 5586f0d039 Upgrade to Postage 0.5.0 2022-04-26 12:16:22 -04:00
Nick Mathewson c1ea419477 Merge branch 'main' into 'msrv_1_56'
# Conflicts:
#   crates/tor-config/Cargo.toml
#   crates/tor-dirmgr/src/state.rs
#   doc/semver_status.md
2022-04-26 12:45:16 +00:00
Ian Jackson 83b9fbea99 list_builder: Use $crate namespaced imports
I don't think we need to bother with things in the prelude,
but doing it for serde and ConfigBuildError seems nice.

Noticed while writing a test case.
2022-04-25 18:23:02 +01:00
Ian Jackson ce877e4421 Document defaults for all the config lists
And add an imprecation in define_list_config_builder's doc comment do
do so in future for other invocations of the macro.

Add add the missing full stops.
2022-04-25 18:22:54 +01:00
Nick Mathewson 2f6bc6bdc4 squash! Bump every crate's edition to 2021.
Remove all `use` statements for `TryFrom` and `TryInto`.  These are
now redundant in Rust 2021.
2022-04-25 13:06:26 -04:00
Nick Mathewson b60b0a266a Bump every crate's edition to 2021.
This is an automated change made with a perl one-liner and verified
with grep -L and grep -l.

Some warnings are introduced with this change; they will be removed
in subsequent commits.

See arti#208 for older discussion on this issue.
2022-04-25 13:05:31 -04:00
Nick Mathewson ca05b0904c Add 'rust-version = "1.56"' to every Cargo.toml file.
This change was made automatically with a perl one-liner, and
confirmed with `grep -L`.

The `rust-version` field itself was introduced in 1.56.0.
2022-04-25 13:04:31 -04:00
Ian Jackson dca4f3ede1 Use better syntax for doc comment attribute
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/471#note_2798020
2022-04-25 17:05:30 +01:00
Ian Jackson 0036b91662 Introduce define_list_config_builder macro
This replaces two almost-identical sets of structs and impls.  More
are on the way, as per
  https://gitlab.torproject.org/tpo/core/arti/-/issues/447
2022-04-25 12:41:00 +01:00
trinity-1686a b9dd23de91 fix typo in doc 2022-04-25 00:27:29 +02:00
Ian Jackson 962b6c32e1 fallback list: Introduce and use FallbackListBuilder
Now the network fallbacks configuration wants to Deserialize
a Vec<FallbackDirBuilder>, rather than validated Vec<FallbackDir>.

Methods on FallbackListBuilder are as per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/462#note_2797697
mutatis mutandi for the fact that this struct has only fallbacks in it.
2022-04-22 17:48:03 +01:00
Ian Jackson 61425a96bd fallback list: Move default list into tor-guardmgr
This is where the FallbackList type is.  We are going to want to
provide a builder too, which ought to impl Default.

This means that the default value for the type must be next to the
type.  In any case, it was anomalous that it wasn't.

This commit is pure code motion.
2022-04-22 17:39:58 +01:00
Nick Mathewson 11a9091602 arti-client: Report clock skew when it is noteworthy
(Also, blame clock skew when it is an explanation of why we cannot
finish a connection.)
2022-04-12 08:03:49 -04:00
Nick Mathewson 72f00daf12 circmgr: re-export clock skew estimates. 2022-04-12 08:03:49 -04:00
Nick Mathewson 5f5cbdc08e GuardMgr: publish skew estimates.
Instead of just having a function that recalculates the latest clock
skew, instead recalculate the clock skew when it may have changed,
and notify other processes via a postage::watch.
2022-04-12 08:03:49 -04:00
Nick Mathewson 61080dcaec guardmgr: fix a unit test panic.
Apparently on OSX you are not allowed to construct an Instant that is a
long time before the time when the test is running.

Also, fix the length of a year in this test.
2022-04-11 14:13:17 -04:00
Nick Mathewson c909926302 Fold FallbackStatus into Entry.
This simplifies the code a lot.
2022-04-11 12:34:16 -04:00
Nick Mathewson 452e1c0935 Add a couple of TODO items to clock-skew estimator. 2022-04-11 12:34:16 -04:00
Nick Mathewson cf362fac9f Implement a better clock skew estimator.
This time, our estimator discards outliers, takes the mean of what's
left, and uses the standard deviation to try to figure out how
seriously to take our report of skew/not-skew.

These estimates are still not actually used.
2022-04-11 12:34:16 -04:00
Nick Mathewson eedee51899 Initial functions to determine and expose a clock skew estimate.
(This is just a placeholder; I'm going to make the functions
smarter in the next commit.)
2022-04-07 16:01:46 -04:00
Nick Mathewson ae92f626fb Reformat tor-guardmgr/Cargo.toml 2022-04-07 12:11:09 -04:00
Nick Mathewson 99146da2c2 GuardMgr: record clock skew information.
(It is not yet actually used.)
2022-04-07 11:33:34 -04:00
Nick Mathewson c3c43b088e Create and use API to report guard/fallback skew.
(The information is not yet recorded.)
2022-04-07 10:47:45 -04:00
Nick Mathewson 142e28cd96 Merge branch 'main' into 'separate_dir_status'
# Conflicts:
#   doc/semver_status.md
2022-04-06 13:33:01 +00:00
Nick Mathewson 430611ba3a Fix a rustdoc link 2022-04-05 12:37:36 -04:00
Nick Mathewson fe43f0ddb5 GuardMgr: Tests for copy_status_from. 2022-04-05 10:36:29 -04:00
Nick Mathewson 3b32e635cd GuardMgr: Tests for note_external_{success,failure}. 2022-04-05 10:01:30 -04:00
Nick Mathewson 3ba9b47000 Rewrite and fix Guard::copy_status_from.
The old version of this function was error-prone, and in fact had
errors: it was too easy to forget to add non-persistent fields, and
that's exactly what we forgot in a few cases
(`microdescriptor_missing`, `circ_history`, and
`suspicious_behavior_warned`).

The new version of this function consumes both of the incoming
Guards, and constructs every field explicitly so that we can't
forget to list any.

Closes #429.
2022-04-05 09:37:44 -04:00
Nick Mathewson b3e06b93b6 GuardMgr: Track directory status and circuit status separately.
Previously, we treated successfully building a circuit to a guard as
a "success", and any failure, including a directory cache failure,
as a failure.  With this change, guards now have separate
success/failure and retry status for circuit usage and directory
usage.

This change is needed for guard-as-directory retry to have
reasonable behavior.  Otherwise, when a guard succeeds at building a
circuit, that clears the directory-is-failing status and makes us
retry the guards to quickly.
2022-04-05 09:18:04 -04:00
Nick Mathewson 52c7a0e26a Allow DirStatus objects to have different timeout floors
This will help when we give Guards a DirStatus as well.
2022-04-04 12:32:19 -04:00
Nick Mathewson 75fd63a120 Rename fallback::status::Status to DirStatus.
It's about to be shared between fallbacks and guards.
2022-04-04 12:31:41 -04:00
Nick Mathewson 6872555302 guardmgr: implement HasRetryTime. 2022-04-04 11:15:18 -04:00
Nick Mathewson 5b2fc118df Bump all arti*, tor* crates to 0.2.0
Not all of these strictly need to be bumped to 0.2.0; many could go
to 0.1.1 instead.  But since everything at the tor-rtcompat and
higher layers has had breaking API changes, it seems not so useful
to distinguish.  (It seems unlikely that anybody at this stage is
depending on e.g. tor-protover but not arti-client.)
2022-04-01 09:15:18 -04:00
Nick Mathewson df3c51c8a0 Use derive_more to derive AsRef. 2022-03-30 10:41:40 -04:00
Nick Mathewson 5233d35223 Reformat tor-guardmgr/Cargo.toml. 2022-03-30 10:41:40 -04:00
Nick Mathewson 3e92d30e49 Fix some Rustdoc links. 2022-03-30 10:41:40 -04:00
Nick Mathewson 6282df34fb Refactor FirstHopId into type-differentiated form
The FirstHopId type now records an enum that stores whether the hop
is a guard or a fallback.  This change addresses concerns about
remembering to check the type or source of an Id before passing it
down to the FallbackState or GuardSet.

Making this change required an API change, so that dirmgr can
report success/failure status without actually knowing whether it's
using a fallback or a guard.
2022-03-30 10:41:39 -04:00
Nick Mathewson 13af6134f6 Rename Guard=>FirstHop, GuardId=>FirstHopId
This is preparation for having separate GuardId and FirstHopId types
that distinguish which back-end they index.
2022-03-30 10:40:59 -04:00
Nick Mathewson 9803b645c0 FallbackState: Use itertools::merge_join_by.
This replaces a hand-coded replacement that was probably a little
less efficient.
2022-03-30 10:40:14 -04:00
Nick Mathewson b752c6c771 Fold fallback::Status::reset() into its (only) caller. 2022-03-30 10:40:14 -04:00
Nick Mathewson 9bf2fe3aca Rename FallbackState::lookup_mut => get_mut. 2022-03-30 10:40:14 -04:00
Nick Mathewson b1ccd3c7dc Rename FallbackSet => FallbackState. 2022-03-30 10:40:14 -04:00
Nick Mathewson 2cf00e63de Refactor select_guard_with_expand to use match and log errors. 2022-03-30 10:40:14 -04:00
Nick Mathewson 578f2925d9 ListKind: Use an exhaustive match to future-proof. 2022-03-30 10:40:14 -04:00
Nick Mathewson c721c8ee53 Clarify documentation about GuardUsable constructors 2022-03-30 10:40:14 -04:00
Nick Mathewson 16ce29ef59 Add a TODO about an unslightly type. 2022-03-30 10:40:14 -04:00
Ian Jackson 82ea251f3c Clean up a rustdoc link 2022-03-30 10:40:14 -04:00
Nick Mathewson b88789d1dd guardmgr::fallback::set: basic unit tests. 2022-03-30 10:40:14 -04:00
Nick Mathewson bdd129f230 Rename ExternalFailure => ExternalActivity. 2022-03-30 10:40:12 -04:00
Nick Mathewson d88e9d676e Replace the fallback directories when they change in the config.
The code here uses a new iterator type, since I couldn't find one of
these on crates.io.  I tried writing the code without it, but it was
harder to follow and test.
2022-03-30 10:39:09 -04:00
Nick Mathewson bfb2353a8f Add status tracking to FallbackDir.
We do this by creating a new FallbackSet type that includes status
information, and updating the GuardMgr APIs to record success and
failure about it when appropriate.  We can use this to mark
FallbackDirs retriable (or not).

With this change, FallbackDir is now stored internally as a Guard in
the GuardMgr crate.  That's fine: the FallbackDir type really only
matters for configuration.
2022-03-30 10:39:09 -04:00
Nick Mathewson 40c8232426 GuardMgr: Return fallback directories when appropriate.
We only do this when we fail to get a regular guard (e.g., because
they're all down), and when we have been asked for a guard for a
one-hop directory.

Most of the change in this commit is plumbing to make all of the
types match up.

As before, compilation may still be broken.
2022-03-30 10:39:09 -04:00
Nick Mathewson 9e31ee0c9b Fix typos in guardmgr/lib.rs 2022-03-30 10:39:09 -04:00
Nick Mathewson a4a5f2163d GuardMgr: some prep work for returning fallbacks as guards.
We need to extend our notion of "the origin of a guard" to include
"somewhere outside the guard list"; we need the ability to return a
FallbackDir as a Guard; and we need to remember a few more pieces of
information in each pending request.

As before, this commit may break compilation; it will be restored soon.
2022-03-30 10:39:09 -04:00
Nick Mathewson 9da43189f3 Turn FallbackList into a real type, and store one in GuardMgr.
The guard manager is responsible for handing out the first hops of
tor circuits, keeping track of their successes and failures, and
remembering their states.  Given that, it makes sense to store this
information here.  It is not yet used; I'll be fixing that in
upcoming commits.

Arguably, this information no longer belongs in the directory
manager: I've added a todo about moving it.

This commit will break compilation on its own in a couple of places;
subsequent commits will fix it up.
2022-03-30 10:39:06 -04:00
Nick Mathewson 5b04e5a6b6 guardmgr: move error types into new err.rs module.
This is more in keeping with the rest of our code.
2022-03-30 10:34:50 -04:00
Nick Mathewson 80b65c3a4d Move fallback.rs into guardmgr.
This is the logical place for it, I think: the GuardMgr's job is to
pick the first hop for a circuit depending on remembered status for
possible first hops.  Making this change will let us streamline the
code that interacts with these objects.
2022-03-30 10:34:48 -04:00
Nick Mathewson 700e491813 Remove allow(clippy::disallowed_methods) lint. 2022-03-30 08:55:58 -04:00
eta fd081742fa Merge branch 'no-system-time' into 'main'
Don't use SystemTime::now()

Closes #306

See merge request tpo/core/arti!365
2022-03-30 12:44:25 +00:00
Nick Mathewson 8306a9cdcf GuardMgr:: generalize GuardId::from_relay. 2022-03-21 10:27:53 -04:00
Nick Mathewson 24e89a470f GuardMgr: new API to record guard problems from outside the crate.
We'll need this so that we can say "This guard behaved bogusly
as a directory cache; try somebody else."
2022-03-21 09:06:32 -04:00
Nick Mathewson eed1f06662 GuardMgr: revise handling of "all guards are down".
When all guards are down, we would previously mark them all as up,
and retry aggressively.  But that's far too aggressive: if there's
something wrong with our ability to connect to guards, it makes us
hammer the network over and over, ignoring all the guard retry
timeouts in practice.

Instead,
  * We now allow the `pick_guard()` function to fail without
    automatically retrying.
  * We give different errors in the cases when all our guards are
    down, and when all of the guards selected by our active usage
    are down.
  * Our "guards are down" error includes the time at which a guard
    will next be retriable.

This is part of #407.
2022-03-21 09:06:32 -04:00
Nick Mathewson cb103e04cf GuardMgr: use decorrelated-jitter backoff for retrying guards.
C tor used one schedule, and guard-spec specified another.  But in
reality we should probably use a randomized schedule to retry
guards, for the reasons explained in the documentation for
RetrySchedule.

I've chosen the minima to be not too far from our previous minima
for primary and non-primary guards.

This is part of #407.
2022-03-21 09:06:32 -04:00
Ian Jackson da787d074a derive_builder: Switch to upstream 0.11
This has the different syntax for builder field attributes than what I
originally proposed in my MR, and which therefore is in the pinned
branch.

My upstream MR for the field attributes feature was morged:
  https://github.com/colin-kiegel/rust-derive-builder/issues/239
2022-03-16 16:34:44 +00:00
Ian Jackson 8b8c2a426e humantime: Update to humantime-serde 1.1.1
This has the humantime_serde::option module, which we have upstreamed
and are about to switch to.

The remaining dependency with  version = "1"  is going to be removed
in a moment.
2022-03-14 10:33:59 +00:00
Ian Jackson 416b56d852 Use git source for derive_builder for now, for attrs feature
We are going to want to specify custom attributes on fields of the
builder struct.  This feature was missing from derive_builder.

This commitid is the current head of my MR branch
  https://github.com/colin-kiegel/rust-derive-builder/pull/237
  https://github.com/ijackson/rust-derive-builder/tree/builder-field-attrs
Using the commitid prevents surprises if that branch is updated.

We will require this newer version of derive_builder.  The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.
2022-03-07 15:56:57 +00:00
Ian Jackson ebfd734956 Move skip_fmt into tor-basic-utils
Code motion and the minimal mechanical changes.

As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/375#note_2783078
2022-03-04 11:45:24 +00:00
Ian Jackson daa44df221 Replace manual Default impls with educe in tor-guardmgr 2022-03-02 18:06:37 +00:00
Ian Jackson b2d8c14c90 Replace manual Debug impl with educe in tor-guardmgr 2022-03-02 18:03:00 +00:00
Nick Mathewson 83c8b11c2c Merge branch 'clippy-allow-arc-clone' into 'main'
Disable clippy::clone_on_ref_ptr

See merge request tpo/core/arti!352
2022-03-01 20:38:05 +00:00
Nick Mathewson e8e9791a97 Bump all crates to 0.1.0 2022-03-01 08:59:34 -05:00
eta 1276f71e9a Merge branch 'always-coarsetime' into 'main'
Make coarsetime dependency and traffic-timestamping non-optional.

See merge request tpo/core/arti!358
2022-02-28 13:39:10 +00:00
trinity-1686a f06b256010 use wallclock where possible in tests 2022-02-26 00:33:44 +01:00
Ian Jackson 6b615b4766 impl Debug for various internal types
I wanted this while debugging something.

The ad-hoc impl Debug with f.debug_struct is getting repetitive
and I've already perpetrated one copy-paste mistake.
We should consider using something like the `educe` crate's Clone.
2022-02-25 17:37:10 +00:00
Nick Mathewson 3d7d609922 Make coarsetime dependency and traffic-timestamping non-optional.
Previously coarsetime and the traffic-timestamp feature were
enabled, since they were only required for a small corner of the
guardmgr algorithm.

But in 1.0 and beyond we'll be adding a bunch of other features (eg,
netflow padding, DoS prevention) that will need coarsetime all over
the place.

And since we're going to be doing coarsetime all over the place, the
previous justification for making traffic-timestamping optional (the
tiny performance hit) is no longer relevant.
2022-02-25 10:31:01 -05:00
Ian Jackson afb50fe735 Disable clippy::clone_on_ref_ptr
This lint is IMO inherently ill-conceived.

I have looked for the reasons why this might be thought to be a good
idea and there were basically two (and they are sort of contradictory):

I. "Calling ‘.clone()` on an Rc, Arc, or Weak can obscure the fact
    that only the pointer is being cloned, not the underlying data."

This is the wording from
  https://rust-lang.github.io/rust-clippy/v0.0.212/#clone_on_ref_ptr

It is a bit terse; we are left to infer why it is a bad idea to
obscure this fact.  It seems to me that if it is bad to obscure some
fact, that must be because the fact is a hazard.  But why would it be
a hazard to not copy the underlying data ?

In other languages, faliing to copy the underlying data is a serious
correctness hazard.  There is a whose class of bugs where things were
not copied, and then mutated and/or reused in multiple places in ways
that were not what the programmer intended.  In my experience, this is
a very common bug when writing Python and Javascript.  I'm told it's
common in golang too.

But in Rust this bug is much much harder to write.  The data inside an
Arc is immutable.  To have this bug you'd have use interior mutability
- ie mess around with Mutex or RefCell.  That provides a good barrier
to these kind of accidents.

II. "The reason for writing Rc::clone and Arc::clone [is] to make it
     clear that only the pointer is being cloned, as opposed to the
     underlying data. The former is always fast, while the latter can
     be very expensive depending on what is being cloned."

This is the reasoning found here
  https://github.com/rust-lang/rust-clippy/issues/2048

This is saying that *not* using Arc::clone is hazardous.
Specifically, that a deep clone is a performance hazard.

But for this argument, the lint is precisely backwards.  It's linting
the "good" case and asking for it to be written in a more explicit
way; while the supposedly bad case can be written conveniently.

Also, many objects (in our codebase, and in all the libraries we use)
that are Clone are in fact simply handles.  They contain Arc(s) (or
similar) and are cheap to clone.  Indeed, that is the usual case.

It does not make sense to distinguish in the syntax we use to clone
such a handle, whether the handle is a transparent Arc, or an opaque
struct containing one or more other handles.

Forcing Arc::clone to be written as such makes for code churn when a
type is changed from Arc<Something> to Something: Clone, or vice
versa.
2022-02-24 18:15:44 +00:00
Nick Mathewson dd55f5ce2d Remove clippy::needless_borrow exception in CI.
This exception is no longer necessary now that the underlying CI bug
is fixed.
2022-02-20 09:09:38 -05:00
Nick Mathewson 1cecc7e45a Change deny(clippy::all) to warn(clippy::all).
Closes #338.
2022-02-14 09:24:06 -05:00
Nick Mathewson 3c342ae5d7 Add TODOs on uncertain points about time_since_last_traffic
This edge-case was there even before the migration of
595fe1ab88, but now it's more explicit and ought to be
revisited.
2022-02-09 10:06:53 -05:00
Yuan Lyu 595fe1ab88 Remove the use of Mutex in channel unused_since timestamp 2022-02-08 18:28:45 -05:00
Nick Mathewson 070e52653d Make SpawnError wrappers contain a 'spawning' string
(By our convention, these errors should say what we were trying to
spawn when the error occurred.)
2022-02-04 16:06:11 -05:00
Ian Jackson bdd58bb6b7 errors: impl HasKind for GuardMgrError 2022-02-04 14:42:37 +00:00
Ian Jackson a779c1f1dd spawn errors: tor-guardmgr: Use formulaic pattern
This makes this like all the others, and is marginally shorter
2022-02-04 14:42:37 +00:00
Ian Jackson 68d0ec437f spawn errors: impl HasKind for futures::SpawnError
This needs two kinds.  We have decided to treat a non-shutdown
SpawnError as "unexplained" rather than as an InternalError.

There are many crates whose
  From<futures::task::SpawnError> for Error
erroneously treat it as an internal error.  We will fix them in a moment.
2022-02-04 14:42:37 +00:00
Ian Jackson a623982197 tor_persist::Error: impl HasKind and adjust comments
And change the comments to slightly reinterpret these errors, to
relate to the circumstances rather than error generation site.
2022-02-04 14:42:37 +00:00
Nick Mathewson 03755a5c7b Merge branch 'dirclient-testing' into 'main'
dir-client: bug fix and more tests

See merge request tpo/core/arti!271
2022-02-03 15:09:06 +00:00
Ian Jackson 7be3bf6339 Temporarily disable some clippy lints on nightly 2022-02-02 21:57:30 +00:00
Nick Mathewson c8dd73d55f Upgrade required version of futures crate to 0.3.14
Earlier versions have a bug in UnboundedReceiver that make our new
dirclient tests fail.
2022-02-01 09:54:47 -05:00
Nick Mathewson 329bde58dd Bump tor-netdir and tor-guardmgr versions
tor-netdir needs to bump because tor-netdoc bumped, even though
there were no other changes in tor-netdir.  Whoops.

tor-guardmgr needs to bump because it already published, with the
older tor-netdir.
2022-01-31 11:05:34 -05:00
Nick Mathewson 01d9937308 Bump the patch version of every crate that changed since 0.0.3 2022-01-31 10:30:52 -05:00
Nick Mathewson 30b3818a9e Make the native-tls crate optional.
This commit puts the native-tls crate behind a feature.  The feature
is off-by-default in the tor-rtcompat crate, but can be enabled
either from arti or arti-client.

There is an included script that I used to test that tor-rtcompat
could build and run its tests with all subsets of its features.

Closes #300
2022-01-26 14:06:58 -05:00
Nick Mathewson 08d3ed978f Merge branch 'ticket_176_v2' into 'main'
guardmgr: Use a better persistent data format

Closes #176

See merge request tpo/core/arti!233
2022-01-11 17:52:00 +00:00
Nick Mathewson b89ce48490 Remove now-unused GuardSet::new(). 2022-01-11 12:00:09 -05:00
Nick Mathewson 70a2e2e751 guardmgr: Use a better persistent data format
Previously we stored only one guard sample, in a state file called
"default_guards".  That's not future-proof, since we want to have
multiple samples in the future.  (`guard-spec.txt` specifies
separate samples for highly restrictive filters, and for bridge
usage.)

This patch changes our behavior so that we can store multiple
samples in a new "guards" file.

I had thought about automatically migrating from the previous file
format and location, but I don't think that's necessary given our
current (lack of) stability guarantees.

Closes #176.
2022-01-11 12:00:09 -05:00
Nick Mathewson f89b0bc752 guardmgr::..::sample_test: Fix intermittent failure.
This test should only fail very rarely (around 1/2.4e8) when guards
are chosen from a list of 20 with uniform probability.  But that
wasn't what we were doing on the mock test network: we were choosing
from a list of 10 viable guards, with nonuniform probability.

As a fix, we change the test network probabilities so that the
guards _are_ chosen with a uniform probability for this test, and we
use a modified version of the test network where there are indeed 20
Guard-flagged relays with the required DirCache=2 protocol.

Closes #276.
2022-01-11 11:35:16 -05:00
Nick Mathewson 7d3482ca1a Bump all crate versions to 0.0.3. 2022-01-11 09:40:32 -05:00
eta da848a1b9c Merge branch 'ticket_178' into 'main'
Fix ticket 178: Don't use a NetDir until we have microdescriptors for all of our primary guards.

Closes #178

See merge request tpo/core/arti!220
2022-01-10 14:02:24 +00:00
Nick Mathewson 4841b50c9f Minimize the required version for each dependency.
I found these versions empirically, by using the following process:

First, I used `cargo tree --depth 1 --kind all` to get a list of
every immediate dependency we had.

Then, I used `cargo upgrade --workspace package@version` to change
each dependency to the earliest version with which (in theory) the
current version is semver-compatible.  IOW, if the current version
was 3.2.3, I picked "3".  If the current version was 0.12.8, I
picked "0.12".

Then, I used `cargo +nightly upgrade -Z minimal-versions` to
downgrade Cargo.lock to the minimal listed version for each
dependency.  (I had to override a few packages; see .gitlab-ci.yml
for details).

Finally, I repeatedly increased the version of each of our
dependencies until our code compiled and the tests passed.  Here's
what I found that we need:

anyhow >= 1.0.5: Earlier versions break our hyper example.

async-broadcast >= 0.3.2: Earlier versions fail our tests.

async-compression 0.3.5: Earlier versions handled futures and tokio
    differently.

async-trait >= 0.1.2: Earlier versions are too buggy to compile our
    code.

clap 2.33.0: For Arg::default_value_os().

coarsetime >= 0.1.20: exposed as_ticks() function.

curve25519-dalek >= 3.2: For is_identity().

generic-array 0.14.3: Earlier versions don't implement
    From<&[T; 32]>

httparse >= 1.2: Earlier versions didn't implement Error.

itertools at 0.10.1: For at_most_once.

rusqlite >= 0.26.3: for backward compatibility with older rustc.

serde 1.0.103: Older versions break our code.

serde_json >= 1.0.50: Since we need its Value type to implement Eq.

shellexpand >= 2.1: To avoid a broken dirs crate version.

tokio >= 1.4: For Handle::block_on().

tracing >= 0.1.18: Previously, tracing_core and tracing had separate
    LevelFilter types.

typenum >= 1.12: Compatibility with rust-crypto crates

x25519-dalek >= 1.2.0: For was_contributory().

Closes #275.
2022-01-07 19:08:58 -05:00
Nick Mathewson 723894b9ea Tests for new guardmgr functionality. 2022-01-06 15:58:36 -05:00
Nick Mathewson 5ac0fcb7ef Add API to check if primary MDs are missing.
We need this information to know if it's okay to migrate to a new
NetDir, or if we need to download more information first.

Part of #178.
2022-01-06 15:58:23 -05:00
Nick Mathewson db65fdd04e guardmgr: Don't use no-md guards for data circs.
If we don't know a current microdescriptor for a guard, we can't use it
for multihop circuits, since we don't know its onion keys.

This is part of a fix for #178.
2022-01-06 09:52:21 -05:00
Daniel Eades 592642a9e6 extend lints to include 'clippy::all' 2021-12-28 20:15:40 +00:00
Neel Chauhan ee16c69075 Remove unused started_at PendingRequest 2021-12-14 10:33:47 -08:00