This adds a new tor-hspow crate with the first layers of support in
place for onion service client puzzles as described in Proposal 327.
The API here is experimental, and it's currently only implementing
the self-contained parts of the client puzzle. So, it can verify and
solve puzzles, but it has no event loop integration or nonce replay
tracking or prioritization code yet. These things seem like they would
eventually live in the same crate.
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
This is a new pure Rust implementation of the Equi-X algorithm
designed by tevador for Tor's onion service proof of work puzzle v1.
Equi-X is an asymmetric puzzle algorithm based on Equihash, with
N=60, K=3, the XOR replaced with modular addition, a 16-bit index
space, and HashX as the inner hash function.
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
This is a new pure Rust implementation of the HashX algorithm
designed by tevador for Tor's onion service proof of work puzzle v1.
HashX is a lightweight family of randomly generated hash functions.
A seed, via blake2 and siphash, drives a program generation model
which randomly selects opcodes and registers while following some
constraints that avoid timing stalls or insufficient hash mixing.
The execution of these hash funcions can be done using a pure Rust
interpreter, or about 20x faster using a very simple just in time
compiler based on the dynasm assembler crate. This has been
implemented for x86_64 and aarch64.
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
- This adds a new crate, `tor-geoip`, which can parse and perform
lookups in the GeoIP database C-tor already uses (generated by a
maintenance utility in the C-tor codebase).
- We embed a copy of C-tor's databases with the crate and use
`include_str!` to ship them with the binary, bloating its size
somewhat.
- This does, however, solve the problem of figuring out how to
distribute these.
- The plan is to gate this functionality behind a feature flag anyway,
so the cost should be nil unless explicitly opted into.
Part of tpo/core/onionmasq#47.
The HS `HsClientSpecifier` and `HsClientSecretKeySpecifier` are moved to
`tor-hsclient`. The HS service secret key specifier stubs are moved to
`tor-hsservice`.
"quicktest" is meant to build faster than our (tuned for size above
all) release profile, while having enough optimization to perform
reasonably well when used for testing.
Closes#639.
This module has types and operations needed in multiple places
for an onion service implementation. There are a bunch of
TODO hs-crypto comments that we'll need to fill in.
This commit implements the round-trip-time estimation algorithm from Tor
proposal 324, validating the implementation against the test vectors
found in C tor. (Note that at the time of writing, the new test vectors
may not be committed to C tor yet, but they will be soon.)
This also adds the necessary consensus parameters to `NetParameters`.
Some of them have been renamed in order to (hopefully) make them more
understandable.
This is a rough first-cut of an API that I think might help us with
keeping limited categories of sensitive information out of our logs.
I'll refine it based on experiences with using it.
This crate is meant to solve #315 by giving a way to make sure that
a file or directory is only accessible by trusted users. I've tried
to explain carefully (in comments and documentation) what this crate
is doing and why, under the assumption that it will someday be read
by another person like me who does _not_ live and breathe unix file
permissions. The crate is still missing some key features, noted in
the TODO section.
It differs from the first version of the crate by taking a more
principled approach to directory checking: it emulates the path
lookup process (reading symlinks and all) one path change at a time,
thus ensuring that we check every directory which could enable
an untrusted user to get to our target file, _or_ which could
enable them to get to any symlink that would get them to the target
file.
The API is also slightly different: It separates the `Mistrust`
object (where you configure what you do or do not trust) from the
`Verifier` (where you set up a check that you want to perform on a
single object). Verifiers are set up to be a bit ephemeral,
so that it is hard to accidentally declare that _every_ object
is meant to be readable when you only mean that _some_ objects
may be readable.
The `[patch]` approach causes the tree not to build when used as a
dependency, unless the `[patch]` is replicated into the depending
project.
Instead, replace our `derive_builer =` dependencies with a reference
to a specific git commit:
perl -i~ -pe 'next unless m/^derive_builder/; s#"(0\.11\.2)"#{ version = "$1", git = "https://github.com/ijackson/rust-derive-builder", rev = "ba0c1a5311bd9f93ddf5f5b8ec2a5f6f03b22fbe" }#' crates/*/Cargo.toml
Note that the commitid has changed. This is because derive_builder is
in fact a workspace of 4 crates. 3 of them are of interest to arti
itself (the 4th exists only for testing). So the same "add git
revision" treatment had to be done to the `derive_builder` and
`derive_builder_macro` crates. Each dependency edge involves a new
commit in the derive_builder workspace, since we can't create a git
commit containing its own commitid. (We want to use commits, rather
than a branch, so that what we are depending on is actually properly
defined, and not subject to the whims of my personal github
namespace.)
There are no actual code changes in derive_builder.
This commitid is the current head of my MR branch
https://github.com/colin-kiegel/rust-derive-builder/pull/253https://github.com/ijackson/rust-derive-builder/tree/field-builder
Using the commitid prevents surprises if that branch is updated.
We will require this newer version of derive_builder. The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.
We will need the new version of not only `derive_builder_core` (the
main macro implementation) but also`derive_builder` for a new error
type.
This has the different syntax for builder field attributes than what I
originally proposed in my MR, and which therefore is in the pinned
branch.
My upstream MR for the field attributes feature was morged:
https://github.com/colin-kiegel/rust-derive-builder/issues/239
We are going to want to specify custom attributes on fields of the
builder struct. This feature was missing from derive_builder.
This commitid is the current head of my MR branch
https://github.com/colin-kiegel/rust-derive-builder/pull/237https://github.com/ijackson/rust-derive-builder/tree/builder-field-attrs
Using the commitid prevents surprises if that branch is updated.
We will require this newer version of derive_builder. The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.
This commit adds a new program to try to implement the ideas behind
experimentation in arti#329. In particular, it tries to implement
basic client "can I bootstrap and connect" functionality testing,
with a lot of instrumentation, and support for breaking things.
So far, the instrumentation is limited to counting TCP bytes and
connections, and counting events. Still, this is enough to measure
behavior on some of the incorrect-clock tests.
NOTE:
For now, you are _required_ to pass in an explicit configuration, in
hopes that this will lead you to override your storage directories
for doing specific experiments.
The new `arti-bench` crate does a simple end-to-end benchmark test
embedding Arti: it generates some random data (of configurable amount,
depending on command-line parameters), and then sends said data back and
forth via Arti (which should be configured to use a local Chutney
network).
Additionally, the benchmark can also be run via a local SOCKS5 server
(in order to benchmark the performance via a local Chutney node, for
comparison).
The `tests/chutney/arti-bench.sh` sets up and tears down Chutney as
required to make this work.
This is very much a first cut; there are many things that should
eventually get added, such as support for multiple connections, JSON
output capabilities, running multiple tests, ...
This implements a basic typed event broadcast mechanism, as described in
arti#230: consumers of the new `tor-events` crate can emit `TorEvent`
events, which others can consume via the `TorEventReceiver`.
Under the hood, the crate uses the `async-broadcast`
(https://github.com/smol-rs/async-broadcast) crate, and a
`futures::mpsc::UnboundedSender` for the event emitters; these are glued
together in the `EventReactor`, which must be run in a background thread
for things to work. (This is done so event sending is always cheap and
non-blocking, since `async-broadcast` senders don't have this
functionality.)
Additionally, the `TorEventKind` type is used to implement selective
event reception / emission: receivers can subscribe to certain event
types (and in fact start out receiving nothing), which filters the set
of events they receive. Having no subscribers for a given event type
means it won't even be emitted in the first place, making things more
efficient.
To do this at all neatly, I had to split out `tor-config` from
`arti-config` again, and putting the lower level stuff (paths,
builder errors) into tor-config. I also changed our use of
derive_builder to always use a common error type, to avoid
error type proliferation.
By enabling link-time optimization, setting 'opt-level=s', and
setting compilation-units=1, we can get a much smaller download
size, which is one of our objectives.
Making these changes reduces the binary size for me (on x86_64) by about
42%. If you also run "strip --strip-debug" on the resulting binary,
the resulting size is 55% smaller than the original binary size.
These effects persist if you compress the binary. Supposing that we
use xz compression, these options make save 32% of compressed binary
size. If we also "strip --strip-debug" before compressing, the compressed
binary saves 43% from the original binary size.
With all of these options applied, on x86_64 linux with xz
compression, we're at a nice 1.5 MiB download. If we statically link
to openssl and sqlite, we're still only at a 2.8 MiB download.
There is a build time cost to these changes: for me, it comes to
a 10%-25% build time increase.
This is part of arti#172.
There are some missing parts here (like persistence and tests)
and some incorrect parts (I am 90% sure that the "exploratory
circuit" flag is bogus). Also it is not integrated with the circuit
manager code.
This commit defines a crate (tor-persist) that describes an abstract
key-value store using serde, plus an on-disk implementation using
Toml and fslock.
Micah Elizabeth Scott
Nick Mathewson
eta
Gabriela Moldovan
Dimitris Apostolou
Ian Jackson
Orhun Parmaksız
Samanta Navarro
Ian Jackson