This introduces the new API functions to Store.
But currently they are all no-ops.
So all this machinery doesn't actually do anything.
Also, it changes the API to the mockable downloader, to allow it to
support if-modified-since. So this isn't used either. But it is more
convenient to do this all at once in BridgeDescManager, since care
needs to be taken about the intraction between if-modified-since and
the persistent cache.
This removes the last cargo audit override (for the unmaintained
ansi_term).
Don't mark options as required when they have default values:
see <https://github.com/clap-rs/clap/pull/3793>.
The new `ipc` module inside `tor-ptmgr` implements the Pluggable
Transport Specification version 1 (`pt-spec.txt`,
https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/pt-spec.txt).
This enables module users to spawn pluggable transport binaries inside a
child process asynchronously, and receive structured information about
how to connect to the transports provided by said binaries.
Internally, this is structured as a pure set of serialisers and
deserialisers for the protocol in the specification (in the form of
environment variables, and the PT's stdout), a wrapper to run the PT
binary and enable accessing its output asynchronously, and a user-facing
wrapper that handles ensuring all the requested transports launched
properly.
The included `run-pt` example is an exceedingly minimal wrapper program
that was useful in testing. More tests can and should be added in a
further MR.
closes arti#394; part of arti#69
The explicit list of variant names, that needs to be kept in sync, and
is a test failure semver break hazard, is now gone.
All the necessary code is now generated automatically, and cannot be
wrong.
I want this because I find myself wanting to add a second
implementation of FlagEvent, for another type.
The most important part of this commit is to make sure that each
`FirstHopId` includes the `GuardSetSelector` from which the guard
was selected. Doing this lets us be certain that when we report
that a guard has succeeded or failed, we're reporting it in the
right context.
Additionally, this commit uses strum to make an iterator over the
samples, so that we can make sure that our "for each sample" code is
robust against future changes, and we don't miss the bridge sample.
This code is _not_ conditional, since we want to support running
with a proxy even if we don't support pluggable transports.
We may eventually want to refactor this into a new create.
Also, add a few tests for this and the other accessors.
We'll need this accessor to find whether we have any channels to
_any_ of the identities that we're trying to connect to.
Also add a BridgeRelayWithDesc type (name tbd) to guarantee that
a bridge relay really does have a known descriptor before you
try to build a circuit with it.
Every element in the set has up to N keys, each of which may have differnt
types. No value for any key may correspond to more than one element in
the set.
These properties can be provided, via a macro, for values of N between 1
and $BIG_ENOUGH.
We'll use this to implement a type that holds HasRelayIds.
(Since the APIs for the `Schedule::sleep*` functions changed, this
is a breaking change in tor-rtcompat. Therefore, the Runtime trait
in tor-rtcompat is now a different trait. Therefore, anything that
uses the Runtime trait in its APIs has also broken.)
This fixes an busy-loop.
When the last `TaskHandle` on a `TaskSchedule` is dropped, the
schedule is permanently canceled: whatever operation it was
scheduling should no longer be performed. But our code was broken:
the `sleep()` and `sleep_until_wallclock()` functions don't verify
whether the handles are dropped or not.
This breakage caused an CPU-eating busy-loop in
`sleep_until_wallclock`.
With this patch, we now return a `Result<(), SleepError>` from these
functions.
Fixes#572.
Because we want to work more on ensuring that our semver stability
story is solid, we are _not_ bumping arti-client to 1.0.0 right now.
Here are the bumps we _are_ doing. Crates with "minor" bumps have
had API breaks; crates with "patch" bumps have had new APIs added.
Note that `tor-congestion` is not bumped here: it's a new crate, and
hasn't been published before.
```
tor-basic-utils minor
fs-mistrust minor
tor-config minor
tor-rtcompat minor
tor-rtmock minor
tor-llcrypto patch
tor-bytes patch
tor-linkspec minor
tor-cell minor
tor-proto minor
tor-netdoc patch
tor-netdir minor
tor-persist patch
tor-chanmgr minor
tor-guardmgr minor
tor-circmgr minor
tor-dirmgr minor
arti-client minor
arti-hyper minor
arti major
arti-bench minor
arti-testing minor
```
Since our last round of releases, these crates have had either
trivial changes, or changes that did not affect their APIs.
Therefore we are bumping their versions, but not changing which
versions of them other crates depend on.
This commit largely follows the example for resolve_alternative_specs.
The difference is that there are two fields, so we use a macro to
avoid recapitulating the field names.
This is more standard. It also provides the ::build() method.
This isn't a config type, and build failures ought not to happen,
so we use Bug for the error.
This is a compile-time feature with an associated configuration
flag, both enabled by default.
When it's turned on, hardening prevents the arti process from
dumping core or being attached to by low-privileged processes.
(This is a defense-in-depth measure, not an absolute way to prevent
attacks. For more information, see
[`secmem_proc`](https://docs.rs/secmem-proc/0.1.1/secmem_proc/).)
Closes#364.
The remaining unconditionally public APIs are those related to our
configuration objects, and the main_main() API.
The rationale for making main_main() public is to have an actual
entry point.
The rationale for making the config APIs public is:
1. We really do intend for others to be able to read our
configuration files using this API.
2. The structure of our configuration files is already part of our
interface.
Closes#530.
This commit implements the round-trip-time estimation algorithm from Tor
proposal 324, validating the implementation against the test vectors
found in C tor. (Note that at the time of writing, the new test vectors
may not be committed to C tor yet, but they will be soon.)
This also adds the necessary consensus parameters to `NetParameters`.
Some of them have been renamed in order to (hopefully) make them more
understandable.
This had to become a new internal function, since at the point that
the handshake needs this code, it does not yet have a Channel to use.
This change made the error messages in the handshake code more
informative: and now they require a regex to check. Later, we might
want to defer formatting these strings, but I don't think we need
to do it now.
Doing this will make sure that we fix a correctness issue in netdir that
will be caused if we add more IDs.
(Also add RelayIdType::COUNT in tor-linkspec.)
The formats used here are backward-compatible with those used by C
tor and those used elsewhere in our code. We need a way to encode
_both_ current kinds of identities as a string that tells you what
kind of ID they are. Traditionally we have used hexadecimal,
sometimes with a $, for RSA ids, and we have used base64 for Ed25519
IDs.
We also introduce a new forward-compatible format for new identity
keys in the future. (The new format is the key identity type, a
colon, and the id encoded as base64.) We will use this new format
_only_ when we need to encode identities in a way where it would be
otherwise unclear what kind of key we are dealing with.
I wonder if these types are correct. I think it makes sense to have
a Ref type like this, rather than just using `&RelayId`, but it
doesn't seems that I can make `RelayId` and `RelayIdRef` implement
Borrow and ToOwned for one another, so maybe I've messed up.
Now we have bus>1 ownership of the crate name `shellexpand`. I have
made a release, and retired `shellexpand-fork`.
The new shellexpand release switches to a (quite similarly) unforked
version of `dirs`.
This Writer is a simple wrapper around `Vec<u8>` that makes sure
that its contents are cleared whenever they are dropped _or
reallocated_.
The reallocation is the important part here: without that, we risk
not zeroizing the first allocation of the buffer.
Using `zeroize` here tells these crates that they should make
various structures zeroize-on-drop.
(This is not yet implemented in `aes` 0.8.1, but support has been
merged in the repository for `aes`, so it should go out in the next
release.)
No corresponding feature flag is needed to enable zeroize-on-drop
for `rsa` and `*25519-dalek` private keys.
This allows us to give better errors in the case where bootstrapping
succeeds at first, but fails thereafter for long enough to make our
directory expire.
We need to replace the AtomicBool for dormant mode with something that
can wake up tasks. postage::watch is the right shape.
But we want to be able to update it but suppress no-op updates.
(There is going to be a call site where no-op updates can occur.)
In the absence of a suitable upstream method as requested here
https://github.com/austinjones/postage-rs/issues/56
we introduce this facility via an extension trait.
Every error now has an action (what we were trying to do), a
resource (what we were trying to do it to), and a source (what
problem we encountered).
Initially I tried to add "action" and "resource" fields to error
variants individually, but that led to a combinatorial explosion.
Part of #323.
This is behind a feature flag, since it isn't needed for pure
clients: only onion services and relays need this.
I've named the object that constructs these certs
`Ed25519CertConstructor` because it doesn't follow the builder
pattern exactly: mainly because you can't get an Ed25519Cert out of
it. _That_ part is necessary because we require that an Ed25519Cert
should only exist if the certificate was found to be well-signed
with the right public key.
Closes#511.
- arti#448 and arti!607 highlight an issue with upgrading `rsa`: namely,
the `x25519-dalek` version previously used has a hard dependency on
`zeroize` 1.3, which creates a dependency conflict.
- However, `x25519-dalek` version `2.0.0-pre.1` relaxes this dependency.
Reviewing the changelogs, it doesn't look like that version is
substantially different from the current one at all, so it should be
safe to use despite the "prerelease" tag.
- The new `x25519-dalek` version also bumps `rand_core`, which means we
don't have to use the RNG compat wrapper in `tor-llcrypto` as much.
closes arti#448
"Trivial" here includes stuff like cargo reformatting, comment
edits, error message string changes, and clippy warning changes.
Crates that depend on these do not need to increment.
Some of these were for decoding particular objects (we now say
what kind of objects), and some were unrelated tor_cert errors that
for some reason we had shoved into a tor_bytes::Error.
There is now a separate tor_cert::CertError type, independent from
tor_cert's use of `tor_bytes::Error` for parsing errors.
These changes make sure that the errors conform to our preferred
style, and include a description of what exactly we were doing when
something went wrong.
Note: the `base64ct` crate rejects invalid characters when the
decoding is done on padded strings. However, the `FromStr` impl
for `B64` can have both padded **and** unpadded inputs, so all
inputs are now padded first, before decoding.
These need to be optional: they improve performance by shifting to
asm implementations, which may not be everybody's idea of good practice.
These are not 'pure' features, since they select one implementation
but disable another. Therefore they don't go in `full`.
Closes#441.
The "full" feature is a catch-all for all features, _except_:
* Those that select a particular implementation (like
tor-llcrypto/with-openssl) or build flag (like "static")
* Those that are experimental or unstable (like "experimental-api")
* Those that are testing-only.
In reviewing !553 I noticed that the empty digest list error had to be
handled in two places. I filed #492 about the duplication.
In fact it turns out to have been quadruplication.
The new code also avoids cloning the underlying objects, instead
sorting a Vec of references.
The randomized tests in this crate take a lot of iterations to
converge, so they default to using a deterministic PRNG seed with
few iterations and higher tolerance, and they only randomize the
tests (with more iterations and tighter tolerances) when you
explicitly opt in to randomization.
(If you specify a seed explicitly, you're doing that to reproduce a
randomized case, so we use the same behavior.)
This only affects uses of thread_rng(), and affects them all more or
less indiscriminately. One test does not work with
ARTI_TEST_PRNG=deterministic; the next commit will fix it.
The new `testing_rng()` function is meant as a replacement for
thread_rng() for use in unit tests. By default, it uses a randomly
seeded RNG, but prints the seed before the test so that you can
reproduce any failures that occur. You can override this via the
environment to use a previous seed, or by using a deterministic
seed for all your tests.
Backend for #486.
I followed the following procedure to make these changes:
* I used maint/changed_crates to find out which crates had changed
since 0.3.0.
* I used grep and maint/list_crates to sort those crates in
topological (dependency) order.
* I looked through semver_status to find which crates were listed as
having semver-relevant changes (new APIs and breaking changes).
* I scanned through the git logs of the crates with no
semver-relevant changes listed to confirm that, indeed, they had
no changes. For those crates, I incremented their patch-level
version _without_ changing the version that other crates depend on.
* I scanned through the git logs of the crates with no
semver-relevant changes listed to confirm that, indeed, they had
no obvious breaking changes.
* I treated all crates that depend on `arti` and/or `arti-client` as
having breaking changes.
* I identified crates that depend on crates that have changed, even
if they have not changed themselves, and identified them as having
a non-breaking change.
* For all of the crates, I used `cargo set-version -p $CRATE --bump
$STATUS` (where `STATUS` is `patch` or `minor`) to update the
versions, and the depended-upon versions.
These crates had only clippy fixes that do not affect their
behavior:
tor-bytes
tor-cell
tor-events
tor-linkspec
tor-netdir
tor-socksproto
This crate only had the cargo-husky dependency removed, which
does not affect compatibility:
tor-llcrypto
Since these changes have no compatibility effects, it is not
necessary to bump the versions of these crates which other crates
depend on.
We support all of the following (in TOML notation):
```
user = "rose" # by name
user = 413 # by ID
user = false # no user
user = ":current" # A 'special' user.
user = { name: "rose" }
user = { id: 413 }
user = { special: ":none" }
user = { special: ":current" }
```
This is a general-purpose implementation of the ad-hoc approach
currently taken in (eg) crates/tor-proto/src/channel/reactor.rs,
with an API intended to defned against the more obvious mistakes.
This allows us to separate the two concerns: the channel reactor can
focus on handling channel cells and control messages and is over 2.5x
shorter.
The complexity of the manual sink implementation, and the machinery
needed to avoid having to suspend while holding an item, are dealt
with separately. That separate implemenation now has proper
documentation. (Tests are in the nest commit to avoid this one being
even more unwieldy.)
We use `extend` to define this as an extension trait. A competitor is
`ext` but in my personal projects I have found `extend` slightly
better.
Generally, change the paths that mention the crate name to go via a
module-level "use".
This involves adding tor-config as a direct dependency for a few
crates.
This is a benchmarking tool, and fs-mistrust doesn't like the
permissions in our CI. The env var ARTI_FS_DISABLE_PERMISSION_CHECKS
is (of course) specific to arti. Maybe it should be honoured here,
or this should be done via the config files.
But disabling this is fine for now.
A build script reaching into your .git/hooks/ and modifying them
nonconsensually was a bit of a horrifying concept, and also made it hard
to build arti with the feature disabled. Remove this crate, and replace
it with manual instructions on how to install the hooks in
CONTRIBUTING.md.
- The additional parameters passed to GetConsensusState are now passed
through all the states, and used as well.
- WriteNetDir doesn't have a now() or config() method any more, since
the states now get this from the runtime or the config parameters.
- This required modifying the tests to make a mocked runtime and custom
config directly, instead of using DirRcv for this purpose.
- Additionally, because we don't have to upgrade a weak reference for
DirState::dl_config(), that function no longer wraps its return value
in Result.
- (A bunch of the FIXMEs from the previous commit that introduced the
additional parameters have now been rectified as a result.)
Relay nicknames are always between 1 and 19 characters long, and
they're always ASCII: That means that storing them in a [u8;19] will
always be possible, and always use less resources than storing them
in a String.
Fortunately, the tinystr crate already helps us with this kind of
thing.
This is derived from the environment, not the configuration file: We
might not want to trust the configuration file until we've decided
whether we like its permissions.
For reference, the git source for this crate (and the others in its
workspace) currently lives in my personal github account (ijackson).
If this fork turns out to be long-lived and gains features and/or
users, it would be good to move it to a gitlab somewhere.
I have granted Nick crate ownership on the crates.io system.
This is a rough first-cut of an API that I think might help us with
keeping limited categories of sensitive information out of our logs.
I'll refine it based on experiences with using it.
This allows us to use this with an item builder type which doesn't
impl Default. (Obviously this only makes sense for items which aren't
actually builders.)
Document that this can contain either a string for expansion, or a
literal PathBuf not for expansion.
Rename the `from_path` method to `new_literal`: a very important
difference is whether it gets expanded - less important than the Rust
type. Also, now it takes `Into<PathBuf>`, which avoids a needless
clone.
(We don't change the API in `arti-client` because
`&tempfile::Tempdir()` doesn't implement `Into<PathBuf>`, so
`arti-client` has to have some new `as_ref` calls.)
Provide accessors `as_unexpanded_str` and `as_literal_path`. The
deserialisation already makes this part of the stable API,l so not
pvoding accessors seems just obstructive. They are useful for tests,
too.
Add tests for the new entrypoints, and for deserialisation of both
variants from TOML (via config, or directly) and JSON.
This required a bit of poking through the `users` crate, to mess
with the user and group dbs. The original goal was to "trust the
group with the same name as us", but it turned into a bit of a
production, since:
* We want to take our own name from $USER, assuming that matches
our uid. (Otherwise we want to ask getpwuid_r().)
* We only want to trust the group if we are actually a member of
that group.
* We want to cache this information.
* We want to test this code.
This crate is meant to solve #315 by giving a way to make sure that
a file or directory is only accessible by trusted users. I've tried
to explain carefully (in comments and documentation) what this crate
is doing and why, under the assumption that it will someday be read
by another person like me who does _not_ live and breathe unix file
permissions. The crate is still missing some key features, noted in
the TODO section.
It differs from the first version of the crate by taking a more
principled approach to directory checking: it emulates the path
lookup process (reading symlinks and all) one path change at a time,
thus ensuring that we check every directory which could enable
an untrusted user to get to our target file, _or_ which could
enable them to get to any symlink that would get them to the target
file.
The API is also slightly different: It separates the `Mistrust`
object (where you configure what you do or do not trust) from the
`Verifier` (where you set up a check that you want to perform on a
single object). Verifiers are set up to be a bit ephemeral,
so that it is hard to accidentally declare that _every_ object
is meant to be readable when you only mean that _some_ objects
may be readable.
The `[patch]` approach causes the tree not to build when used as a
dependency, unless the `[patch]` is replicated into the depending
project.
Instead, replace our `derive_builer =` dependencies with a reference
to a specific git commit:
perl -i~ -pe 'next unless m/^derive_builder/; s#"(0\.11\.2)"#{ version = "$1", git = "https://github.com/ijackson/rust-derive-builder", rev = "ba0c1a5311bd9f93ddf5f5b8ec2a5f6f03b22fbe" }#' crates/*/Cargo.toml
Note that the commitid has changed. This is because derive_builder is
in fact a workspace of 4 crates. 3 of them are of interest to arti
itself (the 4th exists only for testing). So the same "add git
revision" treatment had to be done to the `derive_builder` and
`derive_builder_macro` crates. Each dependency edge involves a new
commit in the derive_builder workspace, since we can't create a git
commit containing its own commitid. (We want to use commits, rather
than a branch, so that what we are depending on is actually properly
defined, and not subject to the whims of my personal github
namespace.)
There are no actual code changes in derive_builder.
Now that we require Rust 1.56, we can upgrade to AES 0.8. This
forces us to have some slight API changes.
We require cipher 0.4.1, not cipher 0.4.0, since 0.4.0 has
compatibility issues with Rust 1.56.
This is where the FallbackList type is. We are going to want to
provide a builder too, which ought to impl Default.
This means that the default value for the type must be next to the
type. In any case, it was anomalous that it wasn't.
This commit is pure code motion.
This commitid is the current head of my MR branch
https://github.com/colin-kiegel/rust-derive-builder/pull/253https://github.com/ijackson/rust-derive-builder/tree/field-builder
Using the commitid prevents surprises if that branch is updated.
We will require this newer version of derive_builder. The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.
We will need the new version of not only `derive_builder_core` (the
main macro implementation) but also`derive_builder` for a new error
type.
Instead of just having a function that recalculates the latest clock
skew, instead recalculate the clock skew when it may have changed,
and notify other processes via a postage::watch.
This time, our estimator discards outliers, takes the mean of what's
left, and uses the standard deviation to try to figure out how
seriously to take our report of skew/not-skew.
These estimates are still not actually used.
Upstream 0.8.2 has broken compilation with Rust 1.53; versions
0.8.{0,1} have been yanked.
Possibly by the time the next arti version comes out, they'll have
fixed this situation, or we'll have upgraded our MSRV.
Upstream issue at https://github.com/Nugine/rlimit/issues/42 .
Not all of these strictly need to be bumped to 0.2.0; many could go
to 0.1.1 instead. But since everything at the tor-rtcompat and
higher layers has had breaking API changes, it seems not so useful
to distinguish. (It seems unlikely that anybody at this stage is
depending on e.g. tor-protover but not arti-client.)
Unlike the rest of the crates, these don't have a "tor-" or "arti-"
prefix, and are potentially used by code outside arti. With that in
mind, it's probably for the best not to bump them to 0.2.0 along
with the rest of our crates.
They have had no changes since 0.1.0 other than refactoring and
changing of clippy lints. Therefore, I'm not bumping the
dependencies from other crates onto these: it's fine whether our
other crates use caret/retry-error 0.1.0 or 0.1.1.
This feature allows us to detect different failing cases for
arti#329 that would otherwise be hard to induce. It works by
filtering consensus directory objects and/or microdescriptor objects
before introducing them to the directory manager.
Closes#397.
This commit uses the `visibility` and `visible` crates to
conditionally make certain structs and their fields public
(respectively). This is incredibly dangerous to use for anything
besides testing, and I've tried to write the documentation for the
feature accordingly.
The guard manager is responsible for handing out the first hops of
tor circuits, keeping track of their successes and failures, and
remembering their states. Given that, it makes sense to store this
information here. It is not yet used; I'll be fixing that in
upcoming commits.
Arguably, this information no longer belongs in the directory
manager: I've added a todo about moving it.
This commit will break compilation on its own in a couple of places;
subsequent commits will fix it up.
This is the logical place for it, I think: the GuardMgr's job is to
pick the first hop for a circuit depending on remembered status for
possible first hops. Making this change will let us streamline the
code that interacts with these objects.
The various background daemon tasks that `arti-client` used to spawn are
now handled inside their respective crates instead, with functions
provided to spawn them that return `TaskHandle`s.
This required introducing a new trait, `NetDirProvider`, which steals
some functionality from the `DirProvider` trait to enable `tor-circmgr`
to depend on it (`tor-circmgr` is a dependency of `tor-dirmgr`, so it
can't depend on `DirProvider` directly).
While we're at it, we also make some of the tasks wait for events from
the `NetDirProvider` instead of sleeping, slightly increasing
efficiency.
Now we use NetParams. That implies making its constructor public,
which I think it fine.
This is related to #413 but is far from completing that ticket.
This has the different syntax for builder field attributes than what I
originally proposed in my MR, and which therefore is in the pinned
branch.
My upstream MR for the field attributes feature was morged:
https://github.com/colin-kiegel/rust-derive-builder/issues/239
This has the humantime_serde::option module, which we have upstreamed
and are about to switch to.
The remaining dependency with version = "1" is going to be removed
in a moment.
We are going to want to specify custom attributes on fields of the
builder struct. This feature was missing from derive_builder.
This commitid is the current head of my MR branch
https://github.com/colin-kiegel/rust-derive-builder/pull/237https://github.com/ijackson/rust-derive-builder/tree/builder-field-attrs
Using the commitid prevents surprises if that branch is updated.
We will require this newer version of derive_builder. The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.
This commit adds support for a BrokenTcp provider that can make
connection attempts fail or time out. It doesn't yet have a way to
turn on the failure.
This commit adds a new program to try to implement the ideas behind
experimentation in arti#329. In particular, it tries to implement
basic client "can I bootstrap and connect" functionality testing,
with a lot of instrumentation, and support for breaking things.
So far, the instrumentation is limited to counting TCP bytes and
connections, and counting events. Still, this is enough to measure
behavior on some of the incorrect-clock tests.
NOTE:
For now, you are _required_ to pass in an explicit configuration, in
hopes that this will lead you to override your storage directories
for doing specific experiments.
Add tls_conn field to ArtiHttpConnector (and argument to constructor).
Introduce MaybeHttpsStream and use it in ArtiHttpConnection.
Have the example program pass the native TLS connector.
Currently the TLS connector and the HTTPS variant are not used, but
this commit is very noisy and fomrulaic, so I have split out the code
to use them into a separate commit for easier preparation and review.
The new `BootstrapBehavior` enum controls whether an unbootstrapped
`TorClient` will bootstrap itself automatically (`Ondemand`) when an
attempt is made to use it, or whether the user must perform
bootstrapping themselves (`Manual`).
The `lazy-init` example shows how you could write a simple
`get_tor_client()` function that used a global `OnceCell` to share
a Tor client across an entire application with this API.
closes arti#278
Additionally, refactor the IoError out of tor_cell::Error:
nothing in TorCell created this; it was only used by tor_proto.
This required refactoring in tor_proto to use a new error type. Here I
decided to use a new CodecError for now, though we may refactor that
away soon too.
This error type doesn't impement HasKind, since the kind will depend
on context.
However, the existing implementation was pretty messy and inconsistent:
Some errors had positions, some didn't.
Some took messages as str, some as String.
Some had internal errors that were somewhat orthogonal to their actual
types.
This commit refactors tor_netdoc::Error to use a ParseErrorKind, and
adds a set of convenience functions to add positions and
messages to the errors that need them.
Since the only purpose of this function is to make sure that no
bootstrapping task is running, a simple futures:🔒:Mutex
should do the job just fine.
Closes#337.
This commit changes how the `TorClient` type works, enabling it to be
constructed synchronously without initiating the bootstrapping process.
Daemon tasks are still started on construction (although some of them
won't do anything if the client isn't bootstrapped).
The old bootstrap() methods are now reimplemented in terms of the new
create_unbootstrapped() and bootstrap_existing() methods.
This required refactoring how the `DirMgr` works to enable the same sort
of thing there.
closes#293
This needs two kinds. We have decided to treat a non-shutdown
SpawnError as "unexplained" rather than as an InternalError.
There are many crates whose
From<futures::task::SpawnError> for Error
erroneously treat it as an internal error. We will fix them in a moment.
tor-netdir needs to bump because tor-netdoc bumped, even though
there were no other changes in tor-netdir. Whoops.
tor-guardmgr needs to bump because it already published, with the
older tor-netdir.
This is based on @janimo's approach in !74, but diverges in a few
important ways.
1. It assumes that something like !251 will merge, so that we can
have separate implementations for native_tls and rustls compiled
at the same time.
2. It assumes that we can implement this for the futures::io traits
only with no real penalty.
3. It uses the `x509-signature` crate to work around the pickiness of
the `webpki` crate. If webpki eventually solves their
[bug 219](https://github.com/briansmith/webpki/issues/219), we
can remove a lot of that workaround.
Closes#86.
The information is pretty basic here: we use "have we been able to
connect/TLS-handshake/Tor-handshake" as a proxy for "are we on the
internet? Are we on a reasonably unfiltered part of the internet?"
Eventually we'll want to make the information gathered and exported
more detailed: I've noted a few places in the code. For now,
however, this is about as good as C Tor does today, and it should be
a good starting point.
This uses a slightly different design from tor-dirmgr. Instead of
exporting an entire state structure via `postage::watch`, it exports
only the parts of that structure which the user is supposed to
read. I think that's more reasonable in this case because most of
the possible internal transitions in the tor-chanmgr state don't
cause a change in the exposed status.
The interface is similar to the one exposed by `arti-client`: it
internally uses postage::watch to give a series of events showing
when a bootstrap status is changing.
Thanks to the existing state/driver separation in the DirMgr design
we don't need much new logic: each download state needs to expose
(internally) how far along it is in its download, which the
bootstrap code passes to the DirMgr if it has changed.
I believe that in the long run, we'll probably want to expose more
(or different) information here, and we'll want to process it
differently. With that in mind, I've made the API for
`DirBootstrapStatus` deliberately narrow, so that we can change its
of its internal later on without breaking code that depends on it.
(The information exposed by this commit is not yet summarized in
`arti-client`.)
Part of #96.
We now do multiple samples (configurable; default 3) per type of
`arti-bench` benchmark run, and take a mean and median average of all
data collected, in order to hopefully be a bit more resilient to random
outliers / variation.
This uses some `futures::stream::Stream` hacks, which might result in
more connections being made than required (and might impact the TTFB
metrics somewhat, at least for downloading).
Results now get collected into a `BenchmarkResults` struct per type of
benchmark, which will be in turn placed into a `BenchmarkSummary` in a
later commit; this will also add the ability to serialize the latter
struct out to disk, for future reference.
part of arti#292
The purpose of a this API is to tell the user how far along Arti is
in getting bootstrapped, and if it's stuck, what it's stuck on.
This API doesn't yet expose any useful information: by the time it's
observable to a client, it's always "100% bootstrapped." But I'm
putting it in a MR now so that we can review the basic idea, and to
avoid conflicts with later work on tickets like #293 and #278.
This is part of #96.
This test should only fail very rarely (around 1/2.4e8) when guards
are chosen from a list of 20 with uniform probability. But that
wasn't what we were doing on the mock test network: we were choosing
from a list of 10 viable guards, with nonuniform probability.
As a fix, we change the test network probabilities so that the
guards _are_ chosen with a uniform probability for this test, and we
use a modified version of the test network where there are indeed 20
Guard-flagged relays with the required DirCache=2 protocol.
Closes#276.
Previously we could only configure one global tracing filter that
applied to stdout and journald. There was no support for log files,
either.
This patch fixes both issues, by substantially revising the
configuration format: There are now separate filters for each log
file, for journald, and for the console log. Because we want to
allow multiple logfiles, they have to go into an array in the
configuration.
The configuration logic has grown a bit complicated in its types,
since the tracing_subscriber crate would prefer to have the complete
structure of tracing Layers known statically. That's fine when you
know how many you have, and which kinds there will be, but for
the runtime-configuration case we need to mess around with
`Box<dyn Layer ...>`.
I also had to switch from tracing_subscriber's EnvFilter to its
Targets filter. It seems "EnvFilter" can only be applied as a Layer
in itself, and won't work as a Filter on an individual Layer.
Closes#166.
Closes#170.
The new `arti-bench` crate does a simple end-to-end benchmark test
embedding Arti: it generates some random data (of configurable amount,
depending on command-line parameters), and then sends said data back and
forth via Arti (which should be configured to use a local Chutney
network).
Additionally, the benchmark can also be run via a local SOCKS5 server
(in order to benchmark the performance via a local Chutney node, for
comparison).
The `tests/chutney/arti-bench.sh` sets up and tears down Chutney as
required to make this work.
This is very much a first cut; there are many things that should
eventually get added, such as support for multiple connections, JSON
output capabilities, running multiple tests, ...
This approach tries to preserve the current interface, but uses a
counter-based event backend to implement a coalescing stream of
events that can be represented as small integers. The advantage
here is that publishing events no longer needs to be a blocking
operation, since there is no queue to fill up.
This patch doesn't actually make anything reconfigurable, but it
does create an API that will tell you "you can't change the value of
that!" If the API looks reasonable, I can start making it possible
to change the values of individual items.
The rand crate's documentation says it's not okay to rely on StdRng
having reproducible output. So instead, let's switch to ChaCha12Rng
instead (which is what StrRng currently uses).
This implements a basic typed event broadcast mechanism, as described in
arti#230: consumers of the new `tor-events` crate can emit `TorEvent`
events, which others can consume via the `TorEventReceiver`.
Under the hood, the crate uses the `async-broadcast`
(https://github.com/smol-rs/async-broadcast) crate, and a
`futures::mpsc::UnboundedSender` for the event emitters; these are glued
together in the `EventReactor`, which must be run in a background thread
for things to work. (This is done so event sending is always cheap and
non-blocking, since `async-broadcast` senders don't have this
functionality.)
Additionally, the `TorEventKind` type is used to implement selective
event reception / emission: receivers can subscribe to certain event
types (and in fact start out receiving nothing), which filters the set
of events they receive. Having no subscribers for a given event type
means it won't even be emitted in the first place, making things more
efficient.
To do this at all neatly, I had to split out `tor-config` from
`arti-config` again, and putting the lower level stuff (paths,
builder errors) into tor-config. I also changed our use of
derive_builder to always use a common error type, to avoid
error type proliferation.
Rather like e8e9699c3c ("Get rid of
tor-proto's ChannelImpl, and use the reactor more instead"), this
admittedly rather large commit refactors the way circuits in `tor-proto`
work, centralising all of the logic in one large nonblocking reactor
which other things send messages into and out of, instead of having a
bunch of `-Impl` types that are protected by mutexes.
Congestion control becomes a lot simpler with this refactor, since the
reactor can manage both stream- and circuit-level congestion control
unilaterally without having to share this information with consumers,
meaning we can get rid of some locks.
The way streams work also changes, in order to facilitate better
handling of backpressure / fairness between streams: each stream now has
a set of channels to send and receive messages over, instead of sending
relay cells directly onto the channel (now, the reactor pulls messages
off each stream in each map, and tries to avoid doing so if it won't be
able to forward them yet).
Additionally, a lot of "close this circuit / stream" messages aren't
required any more, since that state is simply indicated by one end of a
channel going away. This should make cleanup a lot less brittle.
Getting all of this to work involved writing a fair deal of intricate
nonblocking code in Reactor::run_once that tries very hard to be mindful
of making backpressure work correctly (and congestion control); the old
code could get away with having tasks .await on things, but the new
reactor can't really do this (as it'd lock the reactor up), so has to do
everything in a nonblocking manner.
It requires tracing-subscriber 0.2, which is a lower version than we
want, and which causes trouble with our minimal-versions CI test.
There is a pending issue to fix this; we can reinstate tracing-test
once it is merged: https://github.com/dbrgn/tracing-test/pull/11
Instead of awkwardly sharing the internals of a `tor-proto` `Channel`
between the reactor task and any other tasks, move most of the internals
into the reactor and have other tasks communicate with the reactor via
message-passing to allocate circuits and send cells.
This makes a lot of things simple, and has convenient properties like
not needing to wrap the `Channel` in an `Arc` (though some places in the
code still do this for now).
A lot of test code required tweaking in order to deal with the refactor;
in fact, fixing the tests probably took longer than writing the mainline
code (!). Importantly, we now use `tokio`'s `tokio::test` annotation
instead of `async_test`, so that we can run things in the background
(which is required to have reactors running for the circuit tests).
This is an instance of #205, and also kind of #217.
Nick Mathewson
Ian Jackson
Ian Jackson
Gabriel de Perthuis
trinity-1686a
eta
Reylaba
Arturo Marquez
Jim Newsome
Yuan Lyu
Trinity Pointard
Neel Chauhan