This new function combines "sending a message" and "accepting
replies in a stream" into a single call, so that there is no gap
between when the message is sent and the replies are available.
There are a number of compromises here, in order to avoid API
proliferation. I've tried to contain them as best I can.
See comments for additional design discussion.
Now, the MetaCellHandler is responsible for consuming the messages
it gets, and reporting status to whatever task is waiting for a
status message.
Additionally, the MetaCellHandler can decide to remain installed or
shut down the circuit after a successful message. (Previously, it
could only uninstall itself on success and kill the circuit on
failure.)
These changes will enable MetaCellHandlers to be used as the basis
for handling more kinds of message.
(There is some moved and reformatted code here; you may want to
review it with `git {diff or show} --color-moved -b`.)
These crates have had small code changes, but no API additions:
tor-config
tor-socksproto
tor-cert
tor-chanmgr
tor-ptmgr
tor-guardmgr
tor-circmgr
tor-dirclient
tor-dirmgr
arti
tor-hsservice
tor-congestion
These crates have had API extensions:
fs-mistrust
tor-llcrypto
tor-bytes
tor-checkable
tor-linkspec
tor-netdoc
tor-persist
arti-client
This change makes sure that open streams and half-closed streams
have the same stream-type-dependent state machines with respect to
which cells are acceptable.
Fixes#774.
Fixes#769.
The role of CmdChecker is to verify that messages are arriving at
the appropriate sequence on a stream, with respect to the other
messages that have been received. Once the stream becomes
half-closed, the CmdChecker is also in charge of consuming incoming
messages on the stream and making sure that they are well-formed.
This includes a partial solution for #769, but also turned up
another bug (#774) while I was working on it. I'll close them both
once I have a real solution.
We now manipulate raw relay cell bodies as (an alias for)
`Box<[u8;509]>` rather than as (an alias for) `[u8;509]`. This
enables us to do much less copying. It will become more important
soon, as we defer parsing relay cell bodies even longer.
Related to #7.
We also use SliceWriter to avoid allocating a Vec<> for every relay
message we want to encode, and instead encode directly into the
cell.
Unlike C tor, we treat unrecognized commands as reason to kill off
the connection entirely. That's fine; if we need to add an
unrecognized command in the future, we can use VERSIONS to negotiate
it.
Also, if someday we want this code to support relay channels as
well, we can use some type trickery to have that work too.
Every FooMsg type now implements Into<AnyFooMsg>, and
TryFrom<FooMsg>.
Additionally, it now implements From<X> for every distinct type that
it supports. This last part lets us discard a bunch of code.
Unfortunately, I needed some downright hackish trickery in order to
get these macros to avoid generating `From<AnyFooMsg> for AnyFooMsg`
and conflicting with the blanket implementation.
The trickery to deal with RelayEarly and Relay being the same type
was not necessarily worth it; I will be separating them and removing
said trickery in the next commit.
In !948 we renamed a couple of accessor functions, which is a
breaking change in `tor-cell`'s API.
In retrospect, perhaps we should have deprecated the old names and
added the new ones, so we wouldn't have to break the API. (This is
the only API break AFAICT since 1.1.0.)
These changes influence behavior, but not effect compatibility.
(If I messed up, and any crate except for `arti` has non-breaking
API changes, that's still fine, since they are all version
0.x.)
This warning kind of snuck up on us! (See #748) For now, let's
disable it. (I've cleaned it up in a couple of examples, since
those are meant to be more idiomatic and user-facing.)
Closes#748.
This is a little tricky, but I think that we're not actually
exposing too much here. I expect we'll need to tweak this stuff
between now and our final version.
This is the hunks from running the rune in maint/adhoc-add-lint-blocks
but which require some subsequent manual fixup: usually, deleting
now-superfluous outer allows, but in some cases manually putting back
lints that the adhoc script deleted.
This commit renames the for_client and for_relay functions to
from_client and from_relay respectively, in order to indicate their
origin, as the term "for" is more likely to indicate a destination,
which is not true in that situation.
Even though channels are practically changeable, they use locks
internally so that you don't need a `&mut Channel` to send or
receive traffic. It makes sense for reparameterizing the channel to
also use a &self reference.
I'll need this so that I can store channels in an `ByRelayIds<>`
set, and still invoke their reparameterize methods.
The feature we want is `#[doc = include_str!("README.md")]`, which is
stable since 1.54 and our MSRV is now 1.56.
This commit is precisely the result of the following Perl rune:
perl -i~ -0777 -pe 's{(^//!(?!.*\@\@).*\n)+}{#![doc = include_str!("../README.md")]\n}m' crates/*/src/lib.rs
(Since the APIs for the `Schedule::sleep*` functions changed, this
is a breaking change in tor-rtcompat. Therefore, the Runtime trait
in tor-rtcompat is now a different trait. Therefore, anything that
uses the Runtime trait in its APIs has also broken.)
Because we want to work more on ensuring that our semver stability
story is solid, we are _not_ bumping arti-client to 1.0.0 right now.
Here are the bumps we _are_ doing. Crates with "minor" bumps have
had API breaks; crates with "patch" bumps have had new APIs added.
Note that `tor-congestion` is not bumped here: it's a new crate, and
hasn't been published before.
```
tor-basic-utils minor
fs-mistrust minor
tor-config minor
tor-rtcompat minor
tor-rtmock minor
tor-llcrypto patch
tor-bytes patch
tor-linkspec minor
tor-cell minor
tor-proto minor
tor-netdoc patch
tor-netdir minor
tor-persist patch
tor-chanmgr minor
tor-guardmgr minor
tor-circmgr minor
tor-dirmgr minor
arti-client minor
arti-hyper minor
arti major
arti-bench minor
arti-testing minor
```
Making ChannelPaddingInstructions::default() accurately reflect the
initial state of the reactor's padding timer simplifies the code
somewhat.
(When padding is wanted, parameters are computed and inserted
explicitly, so the only change is that if we start out dormant, we
defer setting the timer parameters until necessary.)
As per
https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/657#note_2827249
Get rid of unneeded constructor.
We never need to use hardcoded reduced padding parameters during
negotiation cell construction. If we are using reduced padding
parameters, the layers which decide this have netparams to use.
Prompted by
https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/657#note_2826092
This is actually a general facility for inserting locally-generated
cells into the outgoing stream.
It doesn't seem to be possible to do this without adding an additional
condition check to the reactor, since we need to insert it into the
right place in the stream, giving it priority over data, and only
using it up if there was room in the output.
We don't engage this machinery yet, because nothing sets
special_outgoing.
Change ChannelsParams::initial_update to compare fields with their
default values, and, if they're the same as the default, not to
include them in the returned update.
And if that update is then empty, return None.
The overall effect is to avoid the call to chan.reparameterize if
we're using the builtin default parameters, which is usual.
Channel padding depends on what the channel is being used for. We
therefore need to let the channel code know this information.
The implementation of the per-channel padding control logic will be in
the new note_usage function, which for now is simply a stub.
A future commit will introduce a `PaddingControlState` which lives in
the channel frontend; consult the doc comment for that type to see why
the plumbing through the channel manager terminates in the channel
frontend.
Unfortunately, because we don't have derive-adhoc here yet, rustfmt
didn't get to notice that this comma was needed.
We are going to add field(s), so add the comma now.
This is more standard. It also provides the ::build() method.
This isn't a config type, and build failures ought not to happen,
so we use Bug for the error.
We want to clarify that the tor-proto crate should only know _how_
its objects behave, not _why they behave that way_. (In other
words, we can have a "padding strategy" setting on a channel, but
not a "general usage" setting.)
Closes#531.
This had to become a new internal function, since at the point that
the handshake needs this code, it does not yet have a Channel to use.
This change made the error messages in the handshake code more
informative: and now they require a regex to check. Later, we might
want to defer formatting these strings, but I don't think we need
to do it now.
With this change, each individual identity type becomes optional.
The functions that expose them unconditionally are now in a "legacy"
trait that only some downstream types are expected to implement.
There are new convenience APIs in HasRelayIds:
* to return Option<&keytype>,
* to see if one identity-set contains another.
This commit will break several downstream crates! For the
reviewer's convenience, I will put the fixes for those crates into a
series of squash! commits on this one.
tor-netdir
----------
Revise tor-netdir to accept optional identities. This required some
caveats and workarounds about the cases where we have to deal with a
key type that the tor-netdir code does not currently recognize at
all. If we start to add more identity types in the future, we may
well want more internal indices in this code.
tor-proto
---------
In order to make tor-proto support optional identities, there were
fewer changes than I thought. Some "check" functions needed to start
looking at "all the ids we want" rather than at "the two known IDs";
they also needed to accommodate that case where we don't have an ID
that we demand.
This change will also help with bridges, since we want to be able to
connect to a bridge without knowing all of its IDs up front.
The protocol currently _requires_ the two current ID types in some
places. To deal with that, I added a new `MissingId` error.
I also removed a couple of unconditional identity accessors for
chanmgr; code should use `target().identity(...)` instead.
tor-chanmgr
-----------
This is an incomplete conversion: it does not at all handle channel
targets without Ed25519 identities yet. It still uses those
identities to index its internal map from identity to channel; but
it gives a new `MissingId` error type if it's given a channel target
that doesn't have one.
We'll want to revise the map type again down the road when we
implement bridges, but I'd rather not step on the channel-padding
work in progress right now.
tor-guardmgr
------------
This change is mostly a matter of constructing owned identity types
more sensibly, rather than unwrapping them directly.
There are some places marked with TODOs where we still depend on
particular identity types, because of how the directory protocol
works. This will need revisiting when we add bridge support here.
tor-circmgr
-----------
These changes are just relatively simple API changes in the tests.
We want the set of identities supported by a relay to be extensible
in the future with minimal fuss; we'd also like to make working
with these ID sets more convenient. To handle that, this commit
adds a new trait for "Something that has the same IDs as a relay"
and a new object for "an owned representation of a relay's IDs."
This commit introduces a similar trait for "Something with a list of
SocketAddr, like a relay has." There's no owned equivelent for
that, since Vec<SocketAddr> is already a thing.
Closes#428.
Everything that is a secret encryption key, or an input that is used
to produce a secret encryption key, has to get zeroized. And that's
all!
Closes#254.
This does not yet make sure that `SecretBuf` is used where it
_should_ be, but at least it ensures that most uses of `SecretBytes`
will indeed act as intended, and make sure that whatever they
contain is zeroized.
It requires some corresponding changes to method calls for
correctness and type conformance.
Using `zeroize` here tells these crates that they should make
various structures zeroize-on-drop.
(This is not yet implemented in `aes` 0.8.1, but support has been
merged in the repository for `aes`, so it should go out in the next
release.)
No corresponding feature flag is needed to enable zeroize-on-drop
for `rsa` and `*25519-dalek` private keys.
Do _not_ bump the dependency versions on crates that have had no
changes since arti 0.0.5, since those crates do not depend on the
new APIs.
```
cargo set-version -p tor-basic-utils --bump patch
cargo set-version -p tor-llcrypto --bump patch
git restore crates/tor-checkable
git restore crates/tor-consdiff
git restore crates/tor-rtmock
```
This performs the transitive closure of the last operation:
everything that depends on a crate with a breaking change gets the
version which it depends on bumped.
```
cargo set-version -p tor-proto --bump minor
cargo set-version -p tor-netdoc --bump minor
cargo set-version -p arti-hyper --bump minor
cargo set-version -p arti-bench --bump minor
cargo set-version -p arti-testing --bump minor
cargo set-version -p tor-config --bump minor
```
In order to mitigate syntax highlighting issues and a rust-analyzer bug
(https://github.com/rust-analyzer/rust-analyzer/issues/10178), rename
files that are included with the `include!` macro to have a `.rs`
extension.
Make sure the included files are outside `src/`, in order to not confuse
humans and automated editing tools that might mistake them for valid
Rust modules.
fixes arti#381
This change was a bit annoying, since most of this code _can't_ fail,
and so the only reasonable response is to wrap the input in an
internal error... except for one case where we're actually encoding
a caller-provided message, so we _do_ want to wrap the EncodeError
from tor_bytes.
This comprises four renames:
```
write_onto -> write_onto_infallible
write_into -> write_into_infallible
write -> write_infallible
writer_and_consume -> write_and_consume_infallible.
```
The rest of this branch will be concerned with replacing these
`_infallible` methods with ones that return a `Result`. This is
part of #513.
This implements a higher-level API for the ntor v3 handshake, in line
with that exposed by the ntor handshake. It does not, however, use the
existing `ClientHandshake` trait, due to fundamental differences in the
handshakes (namely, that the v3 handshake can include some additional
extra extension data).
Currently, the higher-level API assumes circuit extension, and copies
the (undocumented!) magic verification string from c-tor that indicates
this usage.
A rudimentary set of functions for serializing and deserializing
extensions to be sent with the handshake is also included, implementing
the protocol in proposal 332 § A.2. Currently, it only implements the
congestion control extensions specified in proposal 324 § 10.3.
part of arti#88
- arti#448 and arti!607 highlight an issue with upgrading `rsa`: namely,
the `x25519-dalek` version previously used has a hard dependency on
`zeroize` 1.3, which creates a dependency conflict.
- However, `x25519-dalek` version `2.0.0-pre.1` relaxes this dependency.
Reviewing the changelogs, it doesn't look like that version is
substantially different from the current one at all, so it should be
safe to use despite the "prerelease" tag.
- The new `x25519-dalek` version also bumps `rand_core`, which means we
don't have to use the RNG compat wrapper in `tor-llcrypto` as much.
closes arti#448
Found these by disabling the nightly dbg macro special case. Now, we
have a mechanism for globally adding suppressions to tests, we can use
that instead.
Some of these were for decoding particular objects (we now say
what kind of objects), and some were unrelated tor_cert errors that
for some reason we had shoved into a tor_bytes::Error.
There is now a separate tor_cert::CertError type, independent from
tor_cert's use of `tor_bytes::Error` for parsing errors.
According to doc/Errors.md, and in keeping with current best
practices, we should not include display an error's `source()` as
part of that error's display method. Instead, we should let the
caller decide to call source() and display that error in turn.
Part of #323.
The "full" feature is a catch-all for all features, _except_:
* Those that select a particular implementation (like
tor-llcrypto/with-openssl) or build flag (like "static")
* Those that are experimental or unstable (like "experimental-api")
* Those that are testing-only.
This only affects uses of thread_rng(), and affects them all more or
less indiscriminately. One test does not work with
ARTI_TEST_PRNG=deterministic; the next commit will fix it.
This commit was made by reverting the previous commit, then
re-running the script I used to generate it. In theory there should
be no semantic changes: only changes due to improved formatting from
cargo edit.
I followed the following procedure to make these changes:
* I used maint/changed_crates to find out which crates had changed
since 0.3.0.
* I used grep and maint/list_crates to sort those crates in
topological (dependency) order.
* I looked through semver_status to find which crates were listed as
having semver-relevant changes (new APIs and breaking changes).
* I scanned through the git logs of the crates with no
semver-relevant changes listed to confirm that, indeed, they had
no changes. For those crates, I incremented their patch-level
version _without_ changing the version that other crates depend on.
* I scanned through the git logs of the crates with no
semver-relevant changes listed to confirm that, indeed, they had
no obvious breaking changes.
* I treated all crates that depend on `arti` and/or `arti-client` as
having breaking changes.
* I identified crates that depend on crates that have changed, even
if they have not changed themselves, and identified them as having
a non-breaking change.
* For all of the crates, I used `cargo set-version -p $CRATE --bump
$STATUS` (where `STATUS` is `patch` or `minor`) to update the
versions, and the depended-upon versions.
This is a general-purpose implementation of the ad-hoc approach
currently taken in (eg) crates/tor-proto/src/channel/reactor.rs,
with an API intended to defned against the more obvious mistakes.
This allows us to separate the two concerns: the channel reactor can
focus on handling channel cells and control messages and is over 2.5x
shorter.
The complexity of the manual sink implementation, and the machinery
needed to avoid having to suspend while holding an item, are dealt
with separately. That separate implemenation now has proper
documentation. (Tests are in the nest commit to avoid this one being
even more unwieldy.)
We use `extend` to define this as an extension trait. A competitor is
`ext` but in my personal projects I have found `extend` slightly
better.
The type of ret.map_err(codec_err_to_chan)? is (). ISTM that
writing `let () = ` makes it clear that there is nothing there,
but the lint forbids this.
This lint is warn by default and trips here for me on current nightly.
It seems wrong to me. We should be able to make it clear to the
reader that there is nothing here - note how this differs from the
lines below where Ready contains msg. A let () binding is a good way
to do that.
I think the lint allow ought to be added everywhere, but that doesn't
seem easy right now - see this issue about maint/add_warning:
https://gitlab.torproject.org/tpo/core/arti/-/issues/469
Now that we require Rust 1.56, we can upgrade to AES 0.8. This
forces us to have some slight API changes.
We require cipher 0.4.1, not cipher 0.4.0, since 0.4.0 has
compatibility issues with Rust 1.56.
This is an automated change made with a perl one-liner and verified
with grep -L and grep -l.
Some warnings are introduced with this change; they will be removed
in subsequent commits.
See arti#208 for older discussion on this issue.
This time, our estimator discards outliers, takes the mean of what's
left, and uses the standard deviation to try to figure out how
seriously to take our report of skew/not-skew.
These estimates are still not actually used.
Fortunately, we don't need a separate type here: authenticated
clock skew can only come attached to a `tor_proto::Error`.
We also remove skew from `tor_proto::Error::HandshakeCertsExpired`,
since it would now be redundant.
of a channel.
At first I wanted to have this information not be a part of channels
at all, but it is a fairly tiny amount of data, and the alternatives
are pretty crufty.
Not all of these strictly need to be bumped to 0.2.0; many could go
to 0.1.1 instead. But since everything at the tor-rtcompat and
higher layers has had breaking API changes, it seems not so useful
to distinguish. (It seems unlikely that anybody at this stage is
depending on e.g. tor-protover but not arti-client.)
We now check the handshake certificates unconditionally, and only
report them as _expired_ as a last resort.
(Rationale: if somebody is presenting the wrong identity from a year
ago, it is more interesting that they are presenting the wrong ID
than it is that they are doing so with an expired cert.
We also now report a different error if the certificate is expired,
but its expiration is within the range of reported clock skew.
(Rationale: it's helpful to distinguish this case, so that we can
blame the failure on possible clock skew rather than definitely
attributing it to a misbehaving relay.)
Part of #405.
NETINFO cells, which are sent in every handshake, may contain
timestamps. This patch adds an accessor for the timestamp in the
Netinfo messages, and teaches the tor-proto code how to compute the
minimum clock skew in the code.
The computation isn't terribly precise, but it doesn't need to be:
Tor should work fine if your clock is accurate to within a few
hours.
This patch also notes a Y2038 problem in the protocol: see
torspec#80.
Part of #405.
Each channel now remembers an OwnedChanTarget.
Each circuit now remembers a vector of OwnedChanTarget to represent
the path that it was constructed for.
Part of #415.
Previously coarsetime and the traffic-timestamp feature were
enabled, since they were only required for a small corner of the
guardmgr algorithm.
But in 1.0 and beyond we'll be adding a bunch of other features (eg,
netflow padding, DoS prevention) that will need coarsetime all over
the place.
And since we're going to be doing coarsetime all over the place, the
previous justification for making traffic-timestamping optional (the
tiny performance hit) is no longer relevant.
Nick Mathewson
Ian Jackson
eta
Dimitris Apostolou
Ian Jackson
Neel Chauhan
coral
Emil Engler
trinity-1686a
David Goulet
Orhun Parmaksız