A "redactable" object is one that can be _partially_ scrubbed in
sensitive contexts. This can be very helpful for UX, but is not
risk-free: see comments.
Enums with variants conditional on cargo features must be
non-exhaustive, because cargo features are supposed to be additive,
meaning that enabling a feature (which might happen due to some random
distant thing) ought not to break things using that enum.
There were surprisingly few places to fix this.
SourceInfo primarily appears in errors, here and in (indirectly)
tor-dirmgr.
This
1. Makes the errors smaller
2. Redacts (currently, scrubs) the chantarget when safe logging is enabled
This commit therefore deals with both tor-dirmgr and tor-dirclient
errors, from the pov of error information that should become sensitive
as part of bridge support.
The error pyaloads directly in tor_dirmgr/src/err.rs don't seem to
contain anything in that category.
This
1. Makes the errors smaller
2. Redacts (currently, scrubs) the chantarget when safe logging is enabled
The other error variants in tor-circmgr::Error don't seem to contain
information that should become sensitive as part of bridge support.
This
1. Makes the errors smaller (in the case of Io's PtTargetAddr;
ChannelBuild's addresses are already indirected in a Vec).
2. Redacts (currently, scrubs) the PtTargetAddr and SocketAddr
when safe logging is enabled
These are the remaining error variants in tor-chanmgr that contain
information that should become sensitive as part of bridge support.
This
1. Makes the errors smaller
2. Redacts (currently, scrubs) the chantarget when safe logging is enabled
This commit doesn't treat other should-be-sensitive inforemation in
errors just yet. That will come in a moment.
`let _ignore =` isn't great because if the function called is
currently infallible, but becomes fallible, it suppresses the
detection of the error check.
A better pattern is to specify the type of the ignored value.
I'm about to do precisely this, here.
I did a git-grep and `let _ignore =` seems very common. Let's not
deal with all those now.
We will want the freedom to replace this, so it needs to go behind a
lock. We need to be able to Clone it cheaply now, so we're using an
Arc instead of a Box.
It doesn't make much sense to have the pluggable transport manager be a
registry, so replace its interface with a more narrowly defined, less generic
version.
Other changes:
- instead of returning a &-reference, it returns an owned Arc, which
should make the ptmgr easier to implement while allowing efficient
reuse
- provision for error handling is added, but will probably be revised in
a future commit pending discussion
- tor-ptmgr code that would generate warnings as a result of this change
is temporarily removed
This is a split out version of arti!886, intended so work on arti#659
can proceed.
We now parse the `bridges.enabled` BoolOrAuto, and the
`bridges.bridges` list.
The `bridges.bridges` list is Vec<()> in the builder, and Vec<Void> in
the built config. Ie, it is simply a count, and vanishes in the built
config.
But this count triggers us to try to call build(), to try to parse
bridges, and to try to set and honour the enablement boolean.
The result is that the type system now ensures that if bridges are
disabled, but specified (either by listing them in the config, or
writing `enabled=true`), we inevitably try to insist that we have a
non-empty Vec<Void>, which is of course impossible.
There will be a test case too for those who think this too abstract a
way to guarantee this property :-).
The dummy module is going to need an error type just like this but
with only the disabled variant. To avoid that dummy enum getting out
of step with the nontrivial one, we're going to make them the same.
So as a first step, break this out into its own file.
This test will start to fail if you compile without bridge support,
because we'll start rejecting the test configurations with bridges
specified.
We'll add a proper test for this later.
That the *de*serialisation works as expected will be tested properly
in just a moment, because when we plumb this all the way through, it
will be what parses the bridge lines in the example config file.
This struct is going to be the principal "dictionary-style" serde
representation for a bridge, and the builder, making this all in
keeping with our usual approach.
In this commit:
* Introduce the struct (defining the serialisation)
* Provide the setters (defining the Rust API)
* Add success test cases (not all of the data in which is used yet)
And the error too.
We need this for the API for BridgeConfigBuilder, where the user can
specify any kind of target "address", even a hostname.
It's already non-conditional, it's just that the name is too limiting.
In this commit:
* Change in tor-linkspec
* Export transitional aliases
* Add TODOs to remove the transitional aliases
Ticket #285 is closed and most of this is stable now we think.
(There are still a couple of stability warnings for specific types in
tor-config, which aren't exposed at the arti-client level.)
This commit replaces the
`impl From<&T> for OwnedChanTarget where T:ChanTarget`
with a new `IntoOwnedChanTarget` trait. This lets us be explicit
that we're constructing an owned object, and not just converting
something.
No semver change needed, since these APIs haven't been released.
----------- (new description)
We previously had a trace message on _every_ change. That's fine,
but we also want to log more important changes where the user
can see them. Namely:
* If we go from any other status to Reachable, we want to tell
the user. (We don't want to spam them if it was already
reachable.)
* If we go from Untried or Reachable to Unreachable, we want to
tell the user. (We don't tell them about changes from Retriable
to Unreachable, since that just means that a retry attempt
was not successful.)
Closes#627.
There are two cases here, and we will want to log them
differently.
(By removing the "Unknown" variant entirely, we ensure that we
didn't miss any code that formerly checked for Unknown.)
This change lets us display a ChanTarget's members without first
cloning them into an OwnedChanTarget.
It also resolves a reliability issue by outputting better info when
talking about connections via pluggable transports.
Closes#647
We're doing this because the type now holds "all the mutable state
in a ChanMgr", not just the map.
This is a pure renaming; no documentation has been updated.
Part of #606.
This lets us avoid calling `Instant::now()`, when `Runtime::now()`
is what we want.
Unfortunately, there are a bunch of functions that called `update()`
that needed to change. Fortunately, none of the changes were very
complicated.
Fixes a `TODO pt-client` comment.
If there are subkeys which are covered by also expected_missing
superkeys, delete them, since we expect the superkey to be missing, so
the subkey won't show up.
This brings the draft configuration mechanisms in tor-ptmgr in line with
the config in other crates, using builders. It also plumbs the config
type through into the main `arti-client` config, and adds some example
lines to `arti-example-config.toml`.
We already have `Pos::Unknown` for an error at an unknown
location and `Pos::None` for an error where no location is
sensible. There's no reason to have an additional `Option` here.
Additionally, the use of `Option` gave us a bug where our old
`ParseError::at_pos()` method didn't actually set the position
unless the position was already set. That's not what we want!
Fixes#640.
Finally, we have to fix a test that was a bit broken because it was
working around #640.
If we have a bridge guard that is using Direct connection and it
knows multiple addresses, our code to match it with a BridgeConfig
is wrong, because the BridgeConfig has only one address, and our
code looks for an exact match.
Fixes#642.
Without this fix, any attempt to actually use a circuit for a
`DirSpecificTarget` will fail, since supports() will say false, and
`restrict_mut()` will say `NotSupported`.
Having this done within circmgr was irregular - most of our other key
buildup functions are done in TorClient::create_inner.
It is also inconvenient, as it buries the guardmgr within the circmgr.
This seems tidier to me.
(It saved a lot of typing when I was trying to make some complicated
pub supertrait with sealed private supertrait, but apparently not
now.)
This is the only way I could find in which parameter interpretation
differs between bridge guards and relay guards; with it documented,
I can remove a TODO about identifying such ways.
Previously we would call extend_sample_as_needed in only two places,
one of which called the other unconditionally. That's obviously not
necessary.
I've selected just one of them (`update_guardset_internal`) since it
fits better with the theme if that function. I've added comments
explaining what is going on.
This commit also introduces a yes/no enum for "were any guards added
while extending this set". Formerly we had a boolean, but it got
passed around so many times that I think its intent became obscure.
What this function actually does is return the number of primary
guards whose presence (by identity) is ambiguous in a current
universe. The new name and documentation should help avoid
confusion.
The method's old name had led me astray when identifying whether it
should apply to bridges in one case. This commit also removes the
corresponding `TODO pt-client`.
Also, add a bunch of reminders around these implementations that
`HasAddrs` returns all the address associated with you for GeoIp or
family purposes, even if they are _not_ ones that we should actually
contact you at.
This explanation is slightly complicated by the fact that I think
that one of the calls to update_guardset_internal() is possibly
unnecessary, and that one of the calls that it makes is potentially
ill-advised.
I'm not going to make those changes right now, however, because they
are potentially a little destabilizing.
Now it is an Option, and is set to None if bridges aren't enabled.
This simplifies `replace_bridge_config` a bit, and forces us to
check for `None` in a few more places.
We do this by checking the FirstHops we're about to return, and when
they correspond to bridges, looking up an appropriate BridgeRelay
in the current BridgeSet (if we can).
The `GuardMgr` code has functionality to tell the DirMgr "Hey,
don't switch to the new NetDir yet: we still need more guard
information!" But we never want to do that if we're selecting
bridges, since they don't come from the NetDir.
Instead of duplicating the logic about which guard sample uses which
universe, we explicitly ask it, and then use that universe. This
will avoid trouble if/when we introduce more samples.
I'm using an Arc<[]> here though I think that there's a chance
that a simple Vec<> would suffice. Since it's an internal type,
nothing will break if we change it later.
Also, we now switch into and out of the Bridges guard sample
as needed. However, that selection is not (yet) built from the
list of bridges. That will come soon.
The first part changes which guard set is active based on based on
the parameters, which always come from a NetDir; the second changes
the contents of the active guard set, based on a Universe.
These arguments were used only for legacy (testing) purposes; the
tests now use `TestNetDirProvider`. This lets us simplify our
internal logic for passing a `NetDir` to our samples, and prepare
for having a `BridgeSet` to pass there instead.
This is a breaking change to `guardmgr` and `circmgr`.
This introduces the new API functions to Store.
But currently they are all no-ops.
So all this machinery doesn't actually do anything.
Also, it changes the API to the mockable downloader, to allow it to
support if-modified-since. So this isn't used either. But it is more
convenient to do this all at once in BridgeDescManager, since care
needs to be taken about the intraction between if-modified-since and
the persistent cache.
We use the one in the dirmgr. That means that our constructor now has
to take a dirmgr. And, the dirmgr must have a circmgr.
This is all rather odd, TBH.
Add .. to the binding of the return values from setup, as
future-proofing.
The tests now need to provide a Store too. Make the sqlite::new_empty
function pub(crate) so we can use it.
We must retain the _db_tmp_dir, since when it goes away the tmp
directory is deleted and the db goes readonly.
* Centralise applying the schema updates, in a closure
* Make the schema update SQL texts be in an array so we can loop
* Make the version update statement generic, not cloned-and-hacked
This will make it possible to add another schema version without
error-prone conditions etc.
Also replace Arg::takes_value with Arg::action to configure flags
that don't take parameters vs options that do.
This makes it easy to upgrade to clap 4.
Tested deprecations with:
cargo check --workspace --all-features --features clap/deprecated
This removes the last cargo audit override (for the unmaintained
ansi_term).
Don't mark options as required when they have default values:
see <https://github.com/clap-rs/clap/pull/3793>.
The RetryDelays are being triggered for the 2nd time here, so their
timeouts can be longer. We must bump the sleep to make sure we don't
have a flaky test.
Prior to the previous commit, set_bridges would malfunction if there
were bridges which where (i) in current (ii) in queued or running
(iii) in the new bridge set.
This test failed then and passes now.
This was fundamentally confused and irregular. Now it is more
regular: it does the same things to all the elements of Tracked,
and a simple filtering on current.
This fixes a bug, for which I'm about to add a test case.
0.99.[012] have a bug https://github.com/JelteF/derive_more/issues/114
which makes the Deref derive for bridgedesc::StateGuard not work
and therefore breaks minimal-versions CI.
It seems simpler to require the newer version everywhere.
Previously we always set `dir_info_missing` to `false` for new
guards, since new guards could only be taken from ones that were
present in the NetDir. But for bridges, we don't download their
info until _after_ we have chosen them as guards.
In this and the upcoming commits I'll be changing how guards related
to `NetDir` and to `Relay`. Previously, a guard could only come
from (or be updated from) a `Relay` in a `NetDir`. Soon it will be
able to be built from a bridge as well.
To do this, I'm defining a `Universe` trait (name negotiable) that
represents a set of things that may be guards. I'm going to
continue extending its functionality until there are no more
methods in guard.rs or sample.rs that take `NetDir`.
This commit removes most of the usage of `NetDir` and `Relay` in
`guard.rs`.
The new `ipc` module inside `tor-ptmgr` implements the Pluggable
Transport Specification version 1 (`pt-spec.txt`,
https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/pt-spec.txt).
This enables module users to spawn pluggable transport binaries inside a
child process asynchronously, and receive structured information about
how to connect to the transports provided by said binaries.
Internally, this is structured as a pure set of serialisers and
deserialisers for the protocol in the specification (in the form of
environment variables, and the PT's stdout), a wrapper to run the PT
binary and enable accessing its output asynchronously, and a user-facing
wrapper that handles ensuring all the requested transports launched
properly.
The included `run-pt` example is an exceedingly minimal wrapper program
that was useful in testing. More tests can and should be added in a
further MR.
closes arti#394; part of arti#69
This is necessary for the (somewhat undesirable) lookup_ids function
to return an ID that the dirmgr can actually use to report successes
and failures.
As noted, lookup_ids will create problems down the road when we
implement relays. We should refactor it out before then.
This required a number of changes, which I've tried to document.
I've taken a conservative approach to modification, and I'm not
using any of the by_*_mut() functions (yet). For cases which
potentially modify the whole set, I'm using into_values() and
collect() to ensure that it's re-indexed correctly, even though the
identities don't change.
I introduce some "TODO pt-client" comments here which I will resolve
in the next commit(s).
The explicit list of variant names, that needs to be kept in sync, and
is a test failure semver break hazard, is now gone.
All the necessary code is now generated automatically, and cannot be
wrong.
I want this because I find myself wanting to add a second
implementation of FlagEvent, for another type.
1. Import it, as we do elsewhere, and use that import.
2. Use thiserror to generate the Display impl, rather than
derive_more, as we do for errors elsewhere.
(tor_error still needs derive_more::Display for ErrorKind.)
Previously we could only use Relay for this case, which won't work
any more: a Bridge is not a `tor_netdir::Relay`. Instead we allow
the GuardMgr to give us something that knows how to convert itself
into an OwnedCircTarget.
This change required a far amount of follow-on revisions and
refactoring, but it should all be internal to the path-building
logic.
Now it contains either an `OwnedChanTarget` or an `OwnedCircTarget`,
which will let `GuardMgr` return bridges that can be used to make
circuits.
As part of this change, it was necessary to revise some
address-modification functions that applied to filters and
`OwnedChanTarget`. Now they do the smart thing, and remove only the
address that are in the `ChanMethod`. This means that the addresses
from HasAddrs are still accurate about which addresses the relay
"has".
This commit improves the overflow protection of one call to
Vec::write_u16(), by replacing the cast conversion from self.sig.len()
with a call to u16::try_from(), like it is already done in the rest of
the accompanying function.
This commit adds a `debug_assert!` macro into the `new_unchecked()`
function of the Data cell. Beside this, it also fixes a misleading
comment regarding that limit.
We could in the future provide a version of abs_retry_time which took
an &dyn closure if that turns out to be wanted.
I think this isn't a semver break because trait implementors are
allowed to relax bounds.
The most important part of this commit is to make sure that each
`FirstHopId` includes the `GuardSetSelector` from which the guard
was selected. Doing this lets us be certain that when we report
that a guard has succeeded or failed, we're reporting it in the
right context.
Additionally, this commit uses strum to make an iterator over the
samples, so that we can make sure that our "for each sample" code is
robust against future changes, and we don't miss the bridge sample.
This commit changes the way how a vector with a known sized gets
allocated, by using Vec::with_capacity() instead of Vec::new().
It will eventually avoid an allocation of more memory than required.
This commit renames the for_client and for_relay functions to
from_client and from_relay respectively, in order to indicate their
origin, as the term "for" is more likely to indicate a destination,
which is not true in that situation.
This commit renames the fixed_len! macro to fixed_len_handshake!, in
order to indicate, that this macro is only suited for cells with
commands related to handshaking.
The `PtTarget` type and its contents (`TransportName`,
`PtTargetAddr`, `PtSettings`) are now unconditionally compiled and
exposed. This will allow us to serialize and deserialize them in
our guard-state files even when we have been built without explicit
PT support.
The `pt-client` feature controls whether `TransportName` is a
variant of `TransportId`, and whether `PtTarget` is a variant of
`ChanMethod`: this in turn means that we'll still have simpler
binary code and smaller structures when we're building without PT
support (which is what we wanted when we initially made these types
conditional).
This lets us write functions which can either take an existing
owned OwnedChanTarget, or copy out of some other kind of ChanTarget
passed by reference.
Now keyed by Arc<BridgeConfig>, and the values can be errors.
Currently there is no implementation so there can't be any errors,
but the error enum will become nonempty.
This is a compromise between C tor (never escapes = signs) and the
spec (calls for escaping all = signs). In C tor there is no way to
actually construct a key with an = sign in it, so this difference
should be undetectable.
This code is _not_ conditional, since we want to support running
with a proxy even if we don't support pluggable transports.
We may eventually want to refactor this into a new create.
Now, instead of duplicate checks in various cases, we simply go
through the loop one last time.
This allows us to simplify some of our other logic around here.
This is necessary so that we can look up channels (open and pending)
by all of the Ids that we know about them.
The operations needed here are pretty complex: to get them right,
I've replaced most of the accessors on the inner `ChannelMap` with a
function that holds the lock while another `FnOnce` is called. This
still gets us the invariant that we can't accidentally await while
holding the lock on the `ChannelMap`.
I've removed the tests for the accessors that are no longer there.
There are some subtleties here. Now that we have more than one kind
of Id, it's possible to have a partial match. I've tried to explain
all these cases in the comments.
}
Even though channels are practically changeable, they use locks
internally so that you don't need a `&mut Channel` to send or
receive traffic. It makes sense for reparameterizing the channel to
also use a &self reference.
I'll need this so that I can store channels in an `ByRelayIds<>`
set, and still invoke their reparameterize methods.
This is mostly a testing only change for now too, but soon I'll use
it to deal with the fact that we need to know the IDs to actually
build a channel at all.
The `ByRelayIds` type doesn't have a type equivalent to
`hash_map::Entry`, since it's a set type rather than a map
type. Therefore, the only plausible way to do entry mutation will
be to remove the old entry and insert a new one. And so, we no
longer need a "poisoned" state.
Also, add a few tests for this and the other accessors.
We'll need this accessor to find whether we have any channels to
_any_ of the identities that we're trying to connect to.
We need to handle String, not just str, since some deserializers
have to handle escapes and generate new strings.
Found while writing tests; fixes#605.
In addition to the usual "You named that method wrong!" errors, we
have a new rustdoc error that complains about bogus "HTML tags" that
are actually unquoted usage of types like `Result<Foo>`.
This lets us build channels using different TransportHelpers,
including the (new) default TransportHelper, which just uses the old
connect_to_one() code.
The feature we want is `#[doc = include_str!("README.md")]`, which is
stable since 1.54 and our MSRV is now 1.56.
This commit is precisely the result of the following Perl rune:
perl -i~ -0777 -pe 's{(^//!(?!.*\@\@).*\n)+}{#![doc = include_str!("../README.md")]\n}m' crates/*/src/lib.rs
The traits that launch connections need to be async; the traits that
don't, shouldn't be async.
Additionally, we need a few more "Sync" annotations here for the
futures to work.
This is an internal type (distinct from factory::ChannelFactory)
that we use to make the code in `tor_chanmgr::mgr` agnostic about
what a channel actually is, and how it is actually launched.
Therefore, I'm renaming it and giving better documentation in a
couple of places, to prevent confusion.
Add ,ignore to ignore three examples that don't actually compile.
cargo readme would add these annotations to lib.rs, but the doc
include doesn't do stuff like that. pandoc seems to still render the
result just fine.
This demonstrates that:
* !bridge-client: uncommenting nondefault bridge config generates
urecognized config key warnings (but the config is still accepted)(
* bridge-client, !pt-client: uncommenting nondefault bridges generates
error due to attempting to use a PT. If that's filtered out,
everything is fine.
* pt-client: Everything is good (as before).
* Introduce filter_examples and resolve_examples helpers,
which will become more complex in a moment.
* Move the API test into a { } block to minimise subsequent diff.
It's going to become conditional.
* In subsequent comparisons, use the parsed version, since
the API built one might not exist.
No overall functional change.
We're going to want something that has the standard list builder
methods at the Rust API, but which has different serialisation.
Sadly the implementation is annoying, because macro_rules makes it
hard to parse a nice input syntax.
This is precisely the text from the original version of !744.
There is no implementation yet, so we must add a entry to the
exception list in the tests.
Section headings appear uncommented in the file, so if we have a
whole section which is completely unrecognized (ie, an entry with no
`.`, it will be spotted when we parse the not-uncommented file too.
Right now there aren't any but there will be in a moment.
This test depends on `Duration` having a granularity of 1
nanosecond, which is not the case on OSX, and is probably not the
case on other places too.
We can re-enable this once we have a set of test vectors that use
more realistic RTTs, and a set of testing code that tolerates some
divergence.
Temporary solution for #574.
I think this is quite an inconvenient way to be carrying on.
Maybe we should disable all dead code warnings unless all features are
also enabled, and just let the compiler get rid of unused stuff later.
`${PROGRAM_DIR}` expands to the equivalent of
`std::env::current_exe().parent()`, with appropriate unwrapping and
conversions.
It is expected to be useful for finding the locations of pluggable
transports in some kinds of bundles.
Closes#586.
HasAddr used to mean "Here are addresses that I have, at which I can
be contacted." But "Where (and how) can I be contacted?" is now a
question for HasChannelMethod to answer.
(We still need to have "HasAddr", though, so we can answer things
like "what country is this relay in" and "are these relays in the
same /8?")
So this commit introduces:
* A new trait for adding an implementation of HasChannelMethod in
terms of HasAddr.
* A requirement on ChanTarget that it needs to implement
HasChannelMethod.
There is some temporary breakage here, marked with "TODO pt-client",
that I'll fix later in this branch.
Nick Mathewson
trinity-1686a
Ian Jackson
Ian Jackson
eta
arnabanimesh
trinity-1686a
Dimitris Apostolou
Gabriel de Perthuis
Emil Engler
breezykermo
EliTheCoder