arti/CHANGELOG.md

Notes

This file describes changes in Arti through the current release. Once Arti is more mature, and we start to version crates independently, we may switch to using a separate changelog for each crate.

Arti 0.2.0 — 1 Apr 2022

Arti 0.2.0 makes a large number of changes to Arti's code and infrastructure for better configurability, lower memory usage, support for running as a basic DNS resolver, improved stream isolation, better behavior under network failures, and API support for a "dormant mode" to suspend background activities.

Breaking changes

Here are the main breaking changes visible from the arti-client crate. Numerous other lower-level crates have breaking changes not noted here.

• Significant refactoring to our configuration handling logic and APIs. The goals here are: - To have the ConfigBuilder objects be the primary configuration objects, and simplify the handling of configuration at the TorClient and arti APIs. - To remove arti-config entirely, and fold its contents into arti or arti-client as appropriate. - To remove unnecessary ad-hoc accessor functions until they prove to be needed.

  This change is not done in this release; we expect to have more breakage in this area in our next release as well. (#314, #371, #372, #374, #396, #418, !391, !401, !417, !421, !423, !425, !427)

• The Runtime trait now includes (and requires) UDP support. (Part of !390's support for DNS.)

• Stream isolation support is completely revised; see notes on isolation below.

New features

• Experimental feature to allow the DirMgr to be replaced by a user-provided DirProvider. (#267, !318, !347)
• Arti now tolerates IPv6-only environments, by using a basic form of the RFC 8305 "happy eyeballs" algorithm to try connections to relays' IPv4 and IPv6 addresses in parallel. (!382)
• New experimental APIs for modifying consensus objects (!318, !402)
• The arti crate now exists as a library, to better expose features like its top-level configuration logic. (!403)
• Arti now supports a dns_port to relay A, AAAA, and PTR requests over the Tor network, like the C tor implementation's DnsPort. (!390, !408, !409)
• Arti has a new full-featured stream isolation API that supports more complicated isolation rules, including user-supplied rules. (#150, #414, !377, !418, !420, !429, !434)
• Channel and Circuit objects now remember the peers that they used when they were constructed, and allow queries of this information as part of their API. (#415)
• The logic for retrying failed guards has been revised to use the same decorrelated-jitter algorithm as directory requests, per proposal 336. (cb103e04cf4d9853, part of #407, !426)
• When all our guards have failed, we no longer retry them all aggressively, but rather assume that our net connection is down and wait a while. (eed1f06662366511, part of #407, !426)
• When running as a directory client, we now remember more information about the source of each request, so we can avoid caches that have failed. (87a3f6b58a5e75f7)
• Experimental feature to install a "filter" for modifying incoming directory objects. Used for testing, to observe client behavior when the directory is in an inconsistent or non-working state. (#397, !431)
• Arti now has initial support for a "Dormant Mode" where periodic events are suspended. Later, even more background tasks will be shut down. (#90, !429, !436)
• Fallback directory caches are now handled with logic similar to guards, so we can avoid ones that aren't working, and simplify our logic for path construction. As a fringe benefit, this unification means that we can now use our guards as directory caches even when we don't have an up-to-date consensus. (#220, #406, !433)

Infrastructure

• We have a new arti-testing crate (not published on crates.io) to perform various kinds of stress-testing on our implementation. It can simulate several kinds of failure and overload conditions; we've been using it to improve Arti's behavior when the network is broken or misbehaving. (#397, !378, !392, !442; see also #329)
• The arti-bench tool now constructs streams in parallel and supports isolated circuits, so we can stress-test the performance of a simulated busy client. (#380, !384)
• Reproducible build scripts now use Rust 1.59 and Alpine 3.15. (#376, !380)
• Improved messages from reproducible build script. (#378, !383)
• Scripts to launch chutney are now refactored and de-duplicated (!396)

Documentation and Examples

• Better documentation for default configuration paths. (!386)
• Instructions for using Tor Browser with Arti on Windows. (!388)
• Better instructions for building Arti on Windows. (!389, !393)
• Improved documentation for stress-testing Arti. (!407)

Cleanups, minor features, and minor bugfixes

• Use derive_more and educe (and simple built-in derive) in many places to simplify our code. (!374, !375)
• Use a forked version of shellexpand to provide correct behavior on Windows. (!274, !373)
• Avoid unnecessary Arc::clone()s in arti-client experimental APIs. (#369, !379)
• New tor-basic-utils crates for small pieces of low-level functionality.
• Small performance improvements to parsing and allocating directory objects, to improve start-up and download times. (#377, !381)
• Use significantly less memory (on the order of a few megabytes less per running client) to store directory objects. (#384, #385, #386, #387, #388, !389, !398, !415)
• Avoid allocating a backtrace object for each channel-creation attempt. (#383, !394)
• Always send an "If-Modified-Since" header on consensus requests, since we wouldn't want a consensus that was far too old. (#403, !412)
• Actually use "preemptive circuit" configuration. Previously, we missed a place where we needed to copy it. (Part of !417)
• Backend support for collecting clock skew information; not yet used. (#405, !410)
• Major refactoring for periodic events, to support an initial version of "dormant mode." (!429)
• Remove most uses of SystemTime::now, in favor of calling the equivalent function on SleepProvider. (#306, !365)
• Several bugs in the logic for retrying directory downloads have been fixed, and several parameters have been tuned, to lead to better behavior under certain network failure conditions. (!439)

Acknowledgments

Thanks to everybody who has contributed to this release, including Christian Grigis, Dimitris Apostolou, Lennart Kloock, Michael, solanav, Steven Murdoch, and Trinity Pointard.

Arti 0.1.0 — 1 Mar 2022

Arti 0.1.0 marks another important step towards stability, and the completion of our 0.1.0 milestone. With this milestone, we now consider Arti ready for experimental embedding within other applications.

Additionally with this release, we're now ready to declare the arti_client API more or less stable and supported. (We're not committing to never break it again in the future, but we'll try not to do so without pretty good reasons.) The 1.0.0 release, scheduled for this September, will represent an even stronger API commitment.

Breaking changes

• Our top-level Error type is now a mostly-opaque wrapper around an inner hidden ErrorDetail type. (You can access ErrorDetail by enabling a feature, but it breaks your semver guarantees.) To distinguish among different kinds of Errors, we provide a supported (and hopefully stable) ErrorKind API that developers can use. (!262, !291, !325, #322, #348)
• The interface to construct a TorClient instance has been completely replaced. The new API should be stable, and prevent the need for additional breaking changes in the future. (#350, !364, #326)
• Many smaller changes, too numerous to list. (Starting after this release, we will try be much more careful about breaking changes, and note them specifically here.)
• We no longer recommend the static feature flag; instead use static-native-tls or static-sqlite as appropriate. (#302)

New features

• The Arti client can now watch its configuration files to see if they change, and reconfigure itself when they do. This is controlled by a watch_configuration option, and is off-by-default. (#270, !280)
• Unused channels now expire after enough time has passed. (This is mostly not needed on the client side, since relays also expire unused channels.) (#41, !273)
• You can now create an unbootstrapped TorClient object, so that you can observe its bootstrapping progress and/or bootstrap it at a later time. (#293, !298)
• You can configure an unbootstrapped TorClient object to automatically bootstrap itself the first time it's used. (!322)
• Arti now returns a webpage with an error message if you try to use its SOCKS proxy as an HTTP proxy (!348)
• We now provide an arti-hyper crate for using Arti with the hyper HTTP library. This is also good example code for showing how to integrate Arti with other tools. (!342, !355]

Major bugfixes

• Fixed a number of problems in the circuit Reactor implementation that could result in cell reordering, leading to relays closing our circuits because of protocol violations. (!264, !282)
• Fixed bugs that could cause strange behavior on shutdown or failure during circuit construction. (#210, #365, !363, !366, !368)

Infrastructure

• Numerous CI improvements.
• Numerous coverage-testing improvements.
• We renamed our shell and python scripts to remove their ".sh" and ".py" suffixes, so that we can more freely change their implementations in the future (if needed). (#309)
• The DirMgr crate now uses an abstract Store trait to make it easier for us to implement new storage backends in the future. (!345, !317)

Documentation and Examples

• Provide better sample code for TorClient::connect. (!303)
• Provide an example for how to make a lazy-initialized TorClient object. (#278, !322)
• Provide an example for how to override the default TCP-connect implementation. (!341, !356)

Cleanups, minor features, and minor bugfixes

• Stop using : as a path character; it's reserved on Windows. (!277)
• Avoid returning junk data from over-long directory downloads (!271)
• Implement Debug and Display for many more types.
• We no longer deny(clippy::all); instead we only use warn(clippy::all) to prevent future clippy versions from breaking completely on our code. (#338)
• As part of our Error refactoring and implementation of ErrorKind, we improved the Error objects in many individual crates for better accuracy and specificity.
• Fix a bug that caused us to flush our persistent state to disk too aggressively. (#320, !321)
• The arti proxy now starts listening on its SOCKS port immediately, rather than waiting for bootstrapping to complete. (!333)

Acknowledgments

Thanks to everybody who has contributed to this release, including Daniel Schischkin, Dimitris Apostolou, Michael Prantl, tharvik, Trinity Pointard, and Yuan Lyu.

Arti 0.0.4 — 31 Jan 2022

This release adds support for bootstrap reporting and rustls, improves several APIs, fixes a few bugs, and adds numerous smaller features for future-proofing and correctness.

It breaks compatibility with previous releases, as is expected before release 0.1.0 (scheduled March 2022).

New features

• Add backends for exposing changes in bootstrap status, either to be queried by a function or read as a stream of events. These APIs will become more useful once there is a way to actually get an un-bootstrapped TorClient. (#96)
• TorClient now has a clone_with_prefs method to make a new client with a different set of default stream preferences. (7ff16fc252c0121f6607, #290])
• Add a feature for telling a TorClient that every stream should be isolated on its own circuit. Please use this sparingly; it can be inefficient. (!252)
• Convenience types for overriding parts of the behavior of an asynchronous Runtime. (!251)
• Optional support for rustls in place of native_tls. This is off by default; to turn it on, use the rustls feature, and construct your client using one of the Runtimes with Rustls in its name. (!260, #86)

Breaking changes

• Significant refactoring of exports and constructor functions in the arti-client crate. (!235)
• Change the persistence format used for guard information, to make it more future-proof. (#176)
• Functions and types that used to refer to "Connections" now refer to "Streams" for consistency. (!256)
• The types exported by the tor-rtcompat crate, and the functions used to create them, have been renamed for consistency. (!263)
• The Runtime API has changed slightly, to avoid a conflict with newer versions of async_executors. (bf8fa66d36298561cc86)

Major bugfixes

• Require authenticated SENDMEs when the relay supports them, and not otherwise. (#294)
• Fix the default location for the cache files. (Previously, they were put into the state directory.) (#297)

Infrastructure

• Numerous improvements to coverage tooling. (#248, !221, !269, !253)
• Improvements to arti-bench reliability and usefulness. (#292)
• Our CI now runs shellcheck on our shell scripts. ([#275])

Documentation

• Build instructions for iOS. (#132)
• Adopt a MSRV policy. (#283)
• More information about troubleshooting the build process. (#277)

Cleanups, minor features, and minor bugfixes

• The max_file_limit setting is now configurable. (#299)
• Fix an unreliable test. (#276)
• Fix a test that would always fail when run after January 27. (!268)
• Avoid possible incomplete reads and writes in Tor channel handshake. (1d5a480f79e7d878ff, !249])
• Refactor some types to expose Arc<> less often. (!236)
• Too many others to list!

Acknowledgments

Thanks to everybody who has contributed to this release, including Arturo Marquez, Daniel Eades, Daniel Schischkin, Jani Monoses, Neel Chauhan, and Trinity Pointard.

Arti 0.0.3 — 11 Jan 2022

This release adds support for preemptive circuit construction, refactors Arti's configuration code and behavior, and adds numerous smaller features needed for a correct Tor client implementation.

It breaks compatibility with previous releases, as is expected before release 0.1.0 (scheduled March 2022).

New features

• Arti now builds preemptive circuits in order to anticipate the user's predicted needs. This change matches Tor's behavior more closely, and should reduce latency for stream creation. (!154)
• The configuration for a TorClient object can be changed while the client is running. (!181)
• Guard selection now obeys family restrictions concerning exit nodes. (!139)
• Better support for overriding the TcpProvider on an Arti client and having this change affect the TlsProvider. This helps with testing support, with cases where TCP streams must be constructed specially, etc. (!166)
• We no longer consider a directory to be "complete" until we have microdescriptors for all of our primary guards. (!220)

Breaking changes

• Configuration files have been reorganized, and we have an all-new API for creating configuration objects. (!135, !137)
• A few unused types and functions have been removed. (214c251e etc)
• CircMgr now returns ClientCirc directly, not wrapped in an Arc. (ClientCirc instances are already cheap to clone.) (!224)
• TorClient now has separate connect and connect_with_prefs methods. (!229)
• Various other API refactorings and revisions. (Please remember that we plan to break backward compatibility with every release between now and 0.1.0 in early March.)

Major bugfixes

• We fixed a bug in handling stream-level SENDMEs that would sometimes result in an Arti client sending too much data, causing the exit relay to close the circuit. (!194)

Infrastructure

• We now have an experimental benchmarking tool to compare Arti's performance with Tor's, when running over a chutney network. So far, we seem competitive, but we'll probably find cases where we underperform. (!195)
• Our coverage tool now post-processes grcov's output to produce per-crate results. (!163)
• Our integration test scripts are more robust to cases where the user has already configured a CHUTNEY_PATH. (!168)
• We have lowered the required dependency versions in our Cargo.toml files so that each one is the lowest version that actually works with our code. (!227)

Cleanups, minor features, and minor bugfixes

• We store fewer needless fields from Tor directory documents. (!151, !165)
• We've gone through and converted every XXXX comment in our code (which indicated a must-fix issue) into a ticket, or a TODO. (#231)
• Our SOCKS code is much more careful about sending error messages if an error occurs before the SOCKS connection succeeds. (!189)
• We no longer build non-directory circuits when the consensus is super-old. (!90)
• We no longer consider timeouts to indicate that our circuits are all timing out unless we have seen some recent incoming network traffic. (!207)
• You can now configure logging to files, with support for rotating the files hourly or daily. You can have separate filters for each logging target. (!222)
• Too many others to list!

Acknowledgments

Thanks to everybody who has contributed to this release, including dagon, Daniel Eades, Muhammad Falak R Wani, Neel Chauhan, Trinity Pointard, and Yuan Lyu!

Arti 0.0.2 — 30 Nov 2021

This release tries to move us towards a more permanent API, and sets the stage for future work in performance evaluation and event reporting.

It breaks compatibility with previous releases, as is expected before release 0.1.0 (scheduled March 2022).

New features

• Warn if guard restrictions are too strict. (#242)
• Optimistic data is now supported on streams, and used by default on directory requests. (#23)
• Initial cut at a typed event framework. Not yet used, but will eventually take the role of Tor's "controller event" system. (#230)
• Large rewrite of configuration handling system, with more ergonomic builders for top-level configurations. (#84)

Breaking changes

• The ${APP_*} path variables have been renamed to ${ARTI_*}. (efdd3275)
• The configuration file format has been substantially revised. (#84)
• Most code that clients don't need is now behind a cargo feature. (#124)
• Revised APIs in many other high-level crates.

Documentation

• Many other improvements and rewrites.

Infrastructure

• Update our cargo-husky scripts to better match our CI. (!62)
• Use grcov, not tarpaulin. (!136)

Cleanups, minor features, and bugfixes

• Huge refactoring of the tor-proto crate to conform more closely to the reactor architecture, and reduce the need for locks. (#205, #217).
• By default, cargo build --release now chooses a more aggressive set of optimization flags. (!124)
  • Too many smaller fixes to list.

Acknowledgments

Thanks to everybody who's contributed to this release, including dagon, Daniel Eades, Dimitris Apostolou, Neel Chauhan, S0AndS0, Trinity Pointard, and Yuan Lyu!

Arti 0.0.1 — 29 Oct 2021

This release attempts to be "free of known privacy holes". That isn't to say that there are no remaining bugs, but rather that we've implemented the missing features that we think are essential for basic privacy.

New features

• Guard relay support... (#58)
  • ...with "Lightweight" path bias detection. (#185)
• Circuit isolation API. (#73, !104)
• Circuit build timeout inference. (#57)
• Persistent state on disk. (#59)
• Allow multiple Arti instances to share directories. (#194)
• Support for EnforceDistinctSubnets. (#43)
• Configurable logging (!68) to journald. (!73)
• Rejecting attempts to connect to internal addresses. (#85)
• Support for Tor RESOLVE and RESOLVE_PTR socks extensions. (#33)
• And too many others to list.

Breaking changes

• Switched from log to tracing. (#74)
• Renamed arti-tor-client to arti-client. (#130)
• Stopped exposing anyhow errors. (#165)
• CLI now uses clap, and uses subcommands. (!109)
• Too many others to list.

Documentation

• New top-level documentation for arti-client, with examples. (!111)
• Many other improvements and rewrites.

Infrastructure

• Reproducible builds for Linux (!69), Windows (!70), and OSX (!86).
• Support for static binaries. (!69)
• Simple integration tests, using chutney (!88).

Cleanups, minor features, and bugfixes

• Too many to list.

Acknowledgments

Thanks to everybody who's contributed to this release, including Ben Armstead, Daniel Eades, Dimitris Apostolou, Eugene Lomov, Felipe Lema, Jani Monoses, Lennart Kloock, Neel Chauhan, S0AndS0, Smitty, Trinity Pointard, Yuan Lyu, dagger, and rls!

Arti 0.0.0

Initial release, to reserve our crate names on crates.io.