The formats used here are backward-compatible with those used by C
tor and those used elsewhere in our code. We need a way to encode
_both_ current kinds of identities as a string that tells you what
kind of ID they are. Traditionally we have used hexadecimal,
sometimes with a $, for RSA ids, and we have used base64 for Ed25519
IDs.
We also introduce a new forward-compatible format for new identity
keys in the future. (The new format is the key identity type, a
colon, and the id encoded as base64.) We will use this new format
_only_ when we need to encode identities in a way where it would be
otherwise unclear what kind of key we are dealing with.
I wonder if these types are correct. I think it makes sense to have
a Ref type like this, rather than just using `&RelayId`, but it
doesn't seems that I can make `RelayId` and `RelayIdRef` implement
Borrow and ToOwned for one another, so maybe I've messed up.
Now we have bus>1 ownership of the crate name `shellexpand`. I have
made a release, and retired `shellexpand-fork`.
The new shellexpand release switches to a (quite similarly) unforked
version of `dirs`.
We want the set of identities supported by a relay to be extensible
in the future with minimal fuss; we'd also like to make working
with these ID sets more convenient. To handle that, this commit
adds a new trait for "Something that has the same IDs as a relay"
and a new object for "an owned representation of a relay's IDs."
This commit introduces a similar trait for "Something with a list of
SocketAddr, like a relay has." There's no owned equivelent for
that, since Vec<SocketAddr> is already a thing.
Closes#428.
Everything that is a secret encryption key, or an input that is used
to produce a secret encryption key, has to get zeroized. And that's
all!
Closes#254.
This does not yet make sure that `SecretBuf` is used where it
_should_ be, but at least it ensures that most uses of `SecretBytes`
will indeed act as intended, and make sure that whatever they
contain is zeroized.
It requires some corresponding changes to method calls for
correctness and type conformance.
This Writer is a simple wrapper around `Vec<u8>` that makes sure
that its contents are cleared whenever they are dropped _or
reallocated_.
The reallocation is the important part here: without that, we risk
not zeroizing the first allocation of the buffer.
Using `zeroize` here tells these crates that they should make
various structures zeroize-on-drop.
(This is not yet implemented in `aes` 0.8.1, but support has been
merged in the repository for `aes`, so it should go out in the next
release.)
No corresponding feature flag is needed to enable zeroize-on-drop
for `rsa` and `*25519-dalek` private keys.
Do _not_ bump the dependency versions on crates that have had no
changes since arti 0.0.5, since those crates do not depend on the
new APIs.
```
cargo set-version -p tor-basic-utils --bump patch
cargo set-version -p tor-llcrypto --bump patch
git restore crates/tor-checkable
git restore crates/tor-consdiff
git restore crates/tor-rtmock
```
This performs the transitive closure of the last operation:
everything that depends on a crate with a breaking change gets the
version which it depends on bumped.
```
cargo set-version -p tor-proto --bump minor
cargo set-version -p tor-netdoc --bump minor
cargo set-version -p arti-hyper --bump minor
cargo set-version -p arti-bench --bump minor
cargo set-version -p arti-testing --bump minor
cargo set-version -p tor-config --bump minor
```
In order to mitigate syntax highlighting issues and a rust-analyzer bug
(https://github.com/rust-analyzer/rust-analyzer/issues/10178), rename
files that are included with the `include!` macro to have a `.rs`
extension.
Make sure the included files are outside `src/`, in order to not confuse
humans and automated editing tools that might mistake them for valid
Rust modules.
fixes arti#381
The other tests wait for 100 milliseconds; this one waits for 100
*microseconds* for some reason, which meant it was understandably flaky
if run on anything less than perfect conditions (arti#515).
This is probably a typo, so just change it.
fixes arti#515
This allows us to give better errors in the case where bootstrapping
succeeds at first, but fails thereafter for long enough to make our
directory expire.
Over the years we've found that most callers who want a netdir want
what C Tor calls a "reasonably live" network directory: One that is
not expired by too much, or too far in the future. But a few want a
_strictly_ live directory: one that says it is valid now, with no
tolerances. And a few want _any_ directory, no matter how expired
it is.
This commit adds net methods to NetDirProvider to provide these
directories. I think that most use cases will want to explicitly
think about what kind of directory they want, so I've made `netdir`
the simplest method. I might remove `timely_netdir` by the end of
this branch; see TODO comments.
Part of #518.
This name is more accurate because we aren't only dealing with
clock skew here: we're also trying to tolerate the case where the
authorities fail to reach consensus for a while.
This logic can now be adjusted via the config object so that it does
its own overriding by looking at the environment as appropriate.
Removing these methods helps simplify the code a bit.
Enabled by #483.
Now that configuring the environment variables related to fs
permissions works properly, we don't need to use the "override"
feature any more: we can just add the option to the configuration
when appropriate.
With this design, `--disable-fs-permission-checks` is now mostly an
alias for
`--option storage.permissions.dangerously_trust_everyone=true`
Enabled by #483.
By default we look at `$FS_MISTRUST_DISABLE_PERMISSIONS_CHECKS`.
Optionally, the user can provide another variable as well, or
disable looking at the environment entirely.
There are going to be some tasks (well, right away, one task) which
will want to go away when the sender is dropped.
The docs in postage are silent, but postage:⌚:Sender does not
have a Drop impl so I don't think we can rely on the Receivers getting
None from their Stream impl.
So we're going to have the watch send Options, which are None only
when the sender is dropped.
We are going to want to be able to wake up other tasks elsewhere in
Arti, that need to know about dormancy. We will give them a postage
watch Receiver.
Right now there are no such things yet.
We need to replace the AtomicBool for dormant mode with something that
can wake up tasks. postage::watch is the right shape.
But we want to be able to update it but suppress no-op updates.
(There is going to be a call site where no-op updates can occur.)
In the absence of a suitable upstream method as requested here
https://github.com/austinjones/postage-rs/issues/56
we introduce this facility via an extension trait.
This interface allows using FilterCount with functions that expect
predicates rather than iterator chains.
I'm about to use it to get meaningful FilterCount results in the
path-selection code in circmgr.
This change was a bit annoying, since most of this code _can't_ fail,
and so the only reasonable response is to wrap the input in an
internal error... except for one case where we're actually encoding
a caller-provided message, so we _do_ want to wrap the EncodeError
from tor_bytes.
This comprises four renames:
```
write_onto -> write_onto_infallible
write_into -> write_into_infallible
write -> write_infallible
writer_and_consume -> write_and_consume_infallible.
```
The rest of this branch will be concerned with replacing these
`_infallible` methods with ones that return a `Result`. This is
part of #513.
This implements a higher-level API for the ntor v3 handshake, in line
with that exposed by the ntor handshake. It does not, however, use the
existing `ClientHandshake` trait, due to fundamental differences in the
handshakes (namely, that the v3 handshake can include some additional
extra extension data).
Currently, the higher-level API assumes circuit extension, and copies
the (undocumented!) magic verification string from c-tor that indicates
this usage.
A rudimentary set of functions for serializing and deserializing
extensions to be sent with the handshake is also included, implementing
the protocol in proposal 332 § A.2. Currently, it only implements the
congestion control extensions specified in proposal 324 § 10.3.
part of arti#88
These errors no longer use Debug to dump the `Option<SourceInfo>`, but
instead produce reasonable text. Also, I've fixed the SourceInfo
Display implementation so that it now says that it got the error
"from $source via $circuit" rather than the other way around.
This is a helper rather than a Display implementation because it
isn't the only logical way to display these values. (In fact,
without context, it isn't even the _most_ logical way)
This is going to make it simpler to write the code in guardmgr (and
later in circmgr) that keeps track of how many relays were rejected
for what reason. The latter, in turn, should improve error messages
when we're unable to pick a guard or a path.
Now each type has a peer. In some cases this peer is just an
address, whereas in others (where key is relevant or address isn't
yet narrowed down) it's a full OwnedChanTarget.
Every error now has an action (what we were trying to do), a
resource (what we were trying to do it to), and a source (what
problem we encountered).
Initially I tried to add "action" and "resource" fields to error
variants individually, but that led to a combinatorial explosion.
Part of #323.
This is behind a feature flag, since it isn't needed for pure
clients: only onion services and relays need this.
I've named the object that constructs these certs
`Ed25519CertConstructor` because it doesn't follow the builder
pattern exactly: mainly because you can't get an Ed25519Cert out of
it. _That_ part is necessary because we require that an Ed25519Cert
should only exist if the certificate was found to be well-signed
with the right public key.
Closes#511.
- arti#448 and arti!607 highlight an issue with upgrading `rsa`: namely,
the `x25519-dalek` version previously used has a hard dependency on
`zeroize` 1.3, which creates a dependency conflict.
- However, `x25519-dalek` version `2.0.0-pre.1` relaxes this dependency.
Reviewing the changelogs, it doesn't look like that version is
substantially different from the current one at all, so it should be
safe to use despite the "prerelease" tag.
- The new `x25519-dalek` version also bumps `rand_core`, which means we
don't have to use the RNG compat wrapper in `tor-llcrypto` as much.
closes arti#448
"Trivial" here includes stuff like cargo reformatting, comment
edits, error message string changes, and clippy warning changes.
Crates that depend on these do not need to increment.
Found these by disabling the nightly dbg macro special case. Now, we
have a mechanism for globally adding suppressions to tests, we can use
that instead.
Some of these were for decoding particular objects (we now say
what kind of objects), and some were unrelated tor_cert errors that
for some reason we had shoved into a tor_bytes::Error.
There is now a separate tor_cert::CertError type, independent from
tor_cert's use of `tor_bytes::Error` for parsing errors.
Since there is currently only one error type that can occur when
encoding, it doesn't make sense to use the full Error type here.
This split will help us downstream, as we no longer need to
categorize tor_bytes::Error as "an error in encoding or decoding".
I considered renaming Error to DecodeError, but that had pretty huge
downstream effects, and didn't seem to be worth it.
These changes make sure that the errors conform to our preferred
style, and include a description of what exactly we were doing when
something went wrong.
Note: the `base64ct` crate rejects invalid characters when the
decoding is done on padded strings. However, the `FromStr` impl
for `B64` can have both padded **and** unpadded inputs, so all
inputs are now padded first, before decoding.
This lets a caller map the inner value, eg to convert the type.
I don't provide `map` as well as `try_map` now, since I don't need it;
we could add it later if it is desirable (although try_map can always
be used instead).
I was hoping to provide a `TryFrom` instead, but that necesasrily
overlaps with the std conversion impl from IntegerMilliseconds<T> to
IntegerMilliseconds<U> where T == U.
It is semantically quite meaningful for these to contain something
that isn't `TryInto<u64>`. (Of course the `Duration` conversion won't
work without that.)
Indeed, this condition was only applied to two out of the three types.
Prompted by being near this code, but not actually necessary for
anything I'm doing here.
The main copy of the (global) configuration for the client's channels
is going to have to live here, inside this mutex. So this really
needs to become a struct with names fields.
According to doc/Errors.md, and in keeping with current best
practices, we should not include display an error's `source()` as
part of that error's display method. Instead, we should let the
caller decide to call source() and display that error in turn.
Part of #323.
At the site of modify_hop, we now have a comment explaining the
internal-error issue.
To make the internal error less likely, we lower the modify_hop call
in lib.rs into GuardSet, where it can make sure it's looking at the
same filter as was used to select the guard.
The function name "pick_guard_ext" is not permanent; I'm going to
rename it in the next commit.
Previously, the API said "you need to call this in a loop till it
returns false". We did that in one place, but not another.
With the introduction of filters, forgetting to loop here becomes a
bug: so instead, change the behavior of extend_sample_as_needed so
it handles looping itself.
The fake network we construct for these tests is small in ways that
can sometimes cause weird behavior with guard filters. We fix this
by adjusting the parameters of the guard selection algorithm
accordingly in the tests.
With these new parameters, #491 no longer occurs.
This commit also adds comments to explain why the parameters are set
as they are.
Closes#491.
guard-spec.txt specifies that we have multiple separate samples of
guards that we can use depending on whether the filter is
restrictive or not. Here we implement the rules for switching
between samples.
When we're filtering guards, we have to check whether the filter is
"restrictive": if it forbids most of the guards (by bandwidth), we
keep its guards separated from the main set. If it is
super-restrictive, we also warn.
This functionality is specified in guard-spec.txt.
Since a guard can have a bunch of addresses, and the guard is
permitted if any one of those addresses is permitted, then we might
decide to use a guard with some non-permitted addresses. Thus, we
need to filter those addresses before returning the view of the
guard as a FirstHop.
Rustls uses ring, which uses code from BoringSSL, which derived from
OpenSSL before OpenSSL changed their license. So ring is currently
under 3BSD/SSLEay licenses, which aren't GPL-compatible, which may
be a problem for some people.
See #493.
The `DirBootstrapStatus` type now exposes a blockage() method to
return an `Option<DirBlockage>`.
The blockage types reported are more low-level than I'd like, but
they are IMO good enough for now: we'll want to get experience with
actual vs hypothetical problems before we refine them.
A "reset" happens whenever we have to start a download attempt over
-- either because we ran out of retries, or we found something wrong
with the consensus after fetching certificates.
An "error" happens when we have a recoverable error from one or more
directory sources.
A "stall" happens whenever a round of downloads or cache loads leads
to no change in the status.
We don't yet use this as part of our status reporting.
Previously we used the "if-modified-since" time associated with the
consensus download, and/or the "valid-after" time in the consensus
attempt, to put multiple attempts into sequence, and to tell one
from another. But that approach was always a kludge, and will soon
get more unreliable as the DirStatus logic gets a bit more complex.
With this commit, we change separate download attempts to be
identified with an AttemptId that increments whenever we decide to
get a different directory from the one we have. IMO this new code
is _much_ cleaner.
This is about to become only a _part_ of what defines a DirStatus: a
DirStatus will also include a reset count, and some kind of info
about how long we've gone without progress.
These need to be optional: they improve performance by shifting to
asm implementations, which may not be everybody's idea of good practice.
These are not 'pure' features, since they select one implementation
but disable another. Therefore they don't go in `full`.
Closes#441.
This change also means that we need to create the handle and scheduler
earlier in the process of creating the DirMgr. If we don't, we won't
have a way to manage the task before bootstrap() returns.
The "full" feature is a catch-all for all features, _except_:
* Those that select a particular implementation (like
tor-llcrypto/with-openssl) or build flag (like "static")
* Those that are experimental or unstable (like "experimental-api")
* Those that are testing-only.
This change (not yet exposed as an API) will let the TorClient have
a `TaskHandle` corresponding to the directory task, letting it
make the directory task dormant as needed.
In reviewing !553 I noticed that the empty digest list error had to be
handled in two places. I filed #492 about the duplication.
In fact it turns out to have been quadruplication.
The new code also avoids cloning the underlying objects, instead
sorting a Vec of references.