Now there's a module in `arti` that runs the loop for an RPC
listener. The part of the old `listener` module that made
the framed connections is now part of the `Session` object.
There is now yet another a temporary location for the pipe; we
should pick something better. At least now it's configurable.
In the future, this will probably hold more data as well, like a
TorClient and some configuration info.
The TorClient will present an issue; I've made comments about that.
Closes#820
Well, mostly correct. Our current serde implementation doesn't
tell us much about what went wrong with the object, so we can't
tell why we couldn't convert it into a Request.
Also, our output for the data field is not as the spec says:
we should bring them into conformance.
Part of #825.
It's experimental, and tokio-only. To enable it, build with
the "rpc" feature turned on, and connect to
`~/.arti-rpc-TESTING/PIPE`. (`nc -U` worked for me)
I'll add some instructions.
Per our design, every connection starts out unauthenticated, and
needs one authenticate command to become authenticated.
Right now the only authentication type is "This is a unix named
socket where everybody who can connect has permission."
I'm not sure about these APIs at all! They force us to use
`async_trait` for `tor_rpccmd::Context`, which bothers me. Should we
just have a function that returns
`Option<Box<dyn Sink<Item=X, Error=Y>>` or something? If so,
what's the correct Y?
This code uses some kludges (discussed with Ian previously and
hopefully well documented here) to get a type-identifier for each
type in a const context. It then defines a macro to declare a
type-erased versions of a concrete implementation functions, and
register those implementations to be called later.
We will probably want to tweak a bunch of this code as we move ahead.
Ordinarily you can cancel a future just by dropping it, but we'll
want the ability to cancel futures that we no longer own (because we
gave them to a `FuturesUnordered`).
We have a local alias of `HsDesc = String` which needs to be got rid
of.
But, right now the alternative would be to implement all the code for
signature checking and decryption of an `HsDesc`, before we can make a
test case for the downloader part.
These crates had no changes until just a moment ago. But since
we updated the versions on some of their dependents, they have now
changed themselves. Thus they get patchlevel bumps.
```
tor-rtmock
tor-protover
tor-socksproto
tor-consdiff
tor-chanmgr
tor-dirclient
tor-hsservice
```
For these crates, the changes are nontrivial, so we
_do_ bump the versions on which their dependent crates depend.
Fortunately, since they are all pre-1.0, we don't need to
distinguish semver-additions from other changes. (Except for arti,
which _is_ post-1.0, but gets a patchlevel bump anyway.)
These are unstable crates with breaking changes:
```
tor-hscrypto
tor-hsclient
```
These have new or extended APIs:
```
safelog
tor-bytes
tor-cell
tor-linkspec
tor-llcrypto
tor-proto
tor-cert
arti-client
```
These have new unstable APIs or features:
```
tor-netdoc
tor-circmgr (also broke some unstable APIs)
arti (is post-1.0)
```
These have bugfixes only:
```
caret
tor-dirmgr
```
Their dependents are _not_ updated to a more recent version.
These bumped the version of a dependency that they don't expose
```
tor-rtcompat
fs-mistrust
```
This one had internal refactoring:
```
tor-netdir
```
These had trivial changes only:
```
tor-checkable
tor-ptmgr
tor-guardmgr
arti-hyper
arti-bench
arti-testing
```
This introduces the `NetdocBuilder` trait described in
`netdoc-builder.md` and a new `tor-netdoc::doc::hsdesc::build` module,
which exports the `HsDescBuilder`. Hidden services will use
`HsDescBuilder` to build and encode hidden service descriptors.
There are several TODOs in the code that I'm planning to address
separately.
Partially addresses #745.
Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
This new function combines "sending a message" and "accepting
replies in a stream" into a single call, so that there is no gap
between when the message is sent and the replies are available.
There are a number of compromises here, in order to avoid API
proliferation. I've tried to contain them as best I can.
See comments for additional design discussion.
Explicit drops don't work. Instead, introduce a scope.
We need two scopes, actually: one where we do the initial table
wrangling, and one for the retries after relock.
So we must put the meat in a closure so we can reuse it.
And we must return the flow control as an enum. Bah, etc.
Avoid reformatting this for the moment. This makes the delta legible...
Provide a more cookied "secret keys for use to connecting to a
particular HS" type, with a builder.
This wants to use config stuff, so oughtn't to be in tor-*crypto.
The individual types remain there.
These crates didn't have any changes until now, when I bumped
the versions of some other crates they depend on:
tor-consdiff
arti-hyper
arti-bench
arti-testing
These crates have had small code changes, but no API additions:
tor-config
tor-socksproto
tor-cert
tor-chanmgr
tor-ptmgr
tor-guardmgr
tor-circmgr
tor-dirclient
tor-dirmgr
arti
tor-hsservice
tor-congestion
These crates have had API extensions:
fs-mistrust
tor-llcrypto
tor-bytes
tor-checkable
tor-linkspec
tor-netdoc
tor-persist
arti-client
These are API breaks, but the crates themselves are currently
100% experimental, so there's no need to bump the minor versions
according to our semver rules.
We now manipulate raw relay cell bodies as (an alias for)
`Box<[u8;509]>` rather than as (an alias for) `[u8;509]`. This
enables us to do much less copying. It will become more important
soon, as we defer parsing relay cell bodies even longer.
Related to #7.
We also use SliceWriter to avoid allocating a Vec<> for every relay
message we want to encode, and instead encode directly into the
cell.
Locally, the only functional effect is that now we refuse to handle
non-whole-number-of-minutes lengths - but since the consensus
parameter can't represent those, there's no overall functional change.
There are some places where I note certificates which are not
currently validated, because there is no cryptographic point in
doing so. We should either document that this is okay, or validate
the certificates anyway.
This code might benefit from refactoring to make it prettier.
In !948 we renamed a couple of accessor functions, which is a
breaking change in `tor-cell`'s API.
In retrospect, perhaps we should have deprecated the old names and
added the new ones, so we wouldn't have to break the API. (This is
the only API break AFAICT since 1.1.0.)
These changes influence behavior, but not effect compatibility.
(If I messed up, and any crate except for `arti` has non-breaking
API changes, that's still fine, since they are all version
0.x.)
This logic is a bit tricky, so I've tried to document it and add
fairly good tests. The silver lining is that the external API for
all of this logic will make it invisible and hidden.
There are some cases where I added functions that I think might
eventually get lowered into MdConsensus: But I don't want to lower
too much right now, since the convention for our netdoc accessors is
that they are fairly unsophisticated, and they show you the document
as it is.
Closes#686
This required rewriting some of our error handling code in
command-line processing, since the toml crate now displays and
reports errors differently. (Admittedly, this code still is kind of
ugly, but at least it is nicely hidden.)
This is in lieu of upgrading to the latest base64 crate, which has
a different API from the old one. Since we have to migrate either
way, we might as well use base64ct everywhere.
I don't think that most of these cases _require_ constant-time
base64, but it won't hurt.
These are a bit complex internally, but the API they present is
pretty simple. I've left some discussion of points where the design
isn't totally fleshed out, and where we need to look harder at the
spec.
Part of #716.
On 1.22, cargo audit is complaining about RUSTSEC-2023-0001.
We aren't affected, since we don't use windows named pipes (yet),
but let's make cargo audit happier.
This module has types and operations needed in multiple places
for an onion service implementation. There are a bunch of
TODO hs-crypto comments that we'll need to fill in.
This is the new upstream version (published by me, recently).
It has the same MSRV and one breaking change:
The caller who specifies a home dir function for substituting into
strings, must now supply a string, not Path. Previously shellexpand
would allow the caller to supply non-unicode data, and then simply not
substitute it. That was an infelicity in the shellexpand API.
Now this infelicity is pushed into our code. The overall behaviour of
Arti hasn't changed as a result. And it seems reasonable to me.
shellexpand 3.x also has a module for expanding Paths instead, in
response to requests for this filed as upstream tickets. We *could*
use that but I am not sanguine about that approach: the Pathness would
spread throughout much of our config and file handling code.
I think we should at the very least postpone trying to work with
invalid-unicode-paths as long as we can.
As near as I can tell, the rust-crypto SHA1 crate was called `sha-1`
for a while because of a conflict with a different SHA1 crate. Now
they apparently have the `sha1` name back and have deprecated the
`sha-1` name.
This makes a `pt_state` directory inside .local/share/arti (or the
local equivalent), right next to our existing `state` dir.
Ideally we would use a separate directory for each PT, but we have a
very fuzzy "what is a specific PT" notion.
Closes#667
These are implementation details and we don't want to mark them as
stable.
To do this, I needed to apply some additional options to the example
code's configuration in Cargo.toml.
This commit implements `PtMgr`, a pluggable transport manager
responsible for keeping track of spawned PTs and spawning them to
satisfy client requests on demand.
It does this in two parts: the `PtMgr` type exported to the rest of the
code, and the background `PtReactor` that actually does the spawning;
this design ensures that only one attempt to spawn a PT is active at a
time, and will prove useful later for implementing e.g. timeouts.
A few changes were necessary to the rest of the code in order to make
this all work out. Namely:
- `TransportRegistry`'s API didn't make any sense for two reasons:
- It wasn't feasible for implementors to implement `ChannelFactory`,
since that'd require constructing a `ChanBuilder` (which requires a
bootstrap reporting event sender).
- Treating the PT manager as a registry is over-general; it's only
necessary for it to spawn pluggable transports, so saddling it with
other concerns didn't make any sense.
- (It's possible to get extensibility for arbitrary user customization
by just letting the user swap in a new `ChannelFactory`, anyway.)
- Therefore, the `PtMgr` implements the new `AbstractPtMgr` trait, which
is far more narrowly focused; this only exists to solve a dependency
loop, and is documented as such.
- This provides a `TransportHelper` instead of a `ChannelFactory`.
This struct is going to be the principal "dictionary-style" serde
representation for a bridge, and the builder, making this all in
keeping with our usual approach.
In this commit:
* Introduce the struct (defining the serialisation)
* Provide the setters (defining the Rust API)
* Add success test cases (not all of the data in which is used yet)
We previously had a trace message on _every_ change. That's fine,
but we also want to log more important changes where the user
can see them. Namely:
* If we go from any other status to Reachable, we want to tell
the user. (We don't want to spam them if it was already
reachable.)
* If we go from Untried or Reachable to Unreachable, we want to
tell the user. (We don't tell them about changes from Retriable
to Unreachable, since that just means that a retry attempt
was not successful.)
Closes#627.
This brings the draft configuration mechanisms in tor-ptmgr in line with
the config in other crates, using builders. It also plumbs the config
type through into the main `arti-client` config, and adds some example
lines to `arti-example-config.toml`.
This introduces the new API functions to Store.
But currently they are all no-ops.
So all this machinery doesn't actually do anything.
Also, it changes the API to the mockable downloader, to allow it to
support if-modified-since. So this isn't used either. But it is more
convenient to do this all at once in BridgeDescManager, since care
needs to be taken about the intraction between if-modified-since and
the persistent cache.
This removes the last cargo audit override (for the unmaintained
ansi_term).
Don't mark options as required when they have default values:
see <https://github.com/clap-rs/clap/pull/3793>.
The new `ipc` module inside `tor-ptmgr` implements the Pluggable
Transport Specification version 1 (`pt-spec.txt`,
https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/pt-spec.txt).
This enables module users to spawn pluggable transport binaries inside a
child process asynchronously, and receive structured information about
how to connect to the transports provided by said binaries.
Internally, this is structured as a pure set of serialisers and
deserialisers for the protocol in the specification (in the form of
environment variables, and the PT's stdout), a wrapper to run the PT
binary and enable accessing its output asynchronously, and a user-facing
wrapper that handles ensuring all the requested transports launched
properly.
The included `run-pt` example is an exceedingly minimal wrapper program
that was useful in testing. More tests can and should be added in a
further MR.
closes arti#394; part of arti#69
The explicit list of variant names, that needs to be kept in sync, and
is a test failure semver break hazard, is now gone.
All the necessary code is now generated automatically, and cannot be
wrong.
I want this because I find myself wanting to add a second
implementation of FlagEvent, for another type.
The most important part of this commit is to make sure that each
`FirstHopId` includes the `GuardSetSelector` from which the guard
was selected. Doing this lets us be certain that when we report
that a guard has succeeded or failed, we're reporting it in the
right context.
Additionally, this commit uses strum to make an iterator over the
samples, so that we can make sure that our "for each sample" code is
robust against future changes, and we don't miss the bridge sample.
This code is _not_ conditional, since we want to support running
with a proxy even if we don't support pluggable transports.
We may eventually want to refactor this into a new create.
Also, add a few tests for this and the other accessors.
We'll need this accessor to find whether we have any channels to
_any_ of the identities that we're trying to connect to.
Also add a BridgeRelayWithDesc type (name tbd) to guarantee that
a bridge relay really does have a known descriptor before you
try to build a circuit with it.
Every element in the set has up to N keys, each of which may have differnt
types. No value for any key may correspond to more than one element in
the set.
These properties can be provided, via a macro, for values of N between 1
and $BIG_ENOUGH.
We'll use this to implement a type that holds HasRelayIds.
(Since the APIs for the `Schedule::sleep*` functions changed, this
is a breaking change in tor-rtcompat. Therefore, the Runtime trait
in tor-rtcompat is now a different trait. Therefore, anything that
uses the Runtime trait in its APIs has also broken.)
This fixes an busy-loop.
When the last `TaskHandle` on a `TaskSchedule` is dropped, the
schedule is permanently canceled: whatever operation it was
scheduling should no longer be performed. But our code was broken:
the `sleep()` and `sleep_until_wallclock()` functions don't verify
whether the handles are dropped or not.
This breakage caused an CPU-eating busy-loop in
`sleep_until_wallclock`.
With this patch, we now return a `Result<(), SleepError>` from these
functions.
Fixes#572.
Because we want to work more on ensuring that our semver stability
story is solid, we are _not_ bumping arti-client to 1.0.0 right now.
Here are the bumps we _are_ doing. Crates with "minor" bumps have
had API breaks; crates with "patch" bumps have had new APIs added.
Note that `tor-congestion` is not bumped here: it's a new crate, and
hasn't been published before.
```
tor-basic-utils minor
fs-mistrust minor
tor-config minor
tor-rtcompat minor
tor-rtmock minor
tor-llcrypto patch
tor-bytes patch
tor-linkspec minor
tor-cell minor
tor-proto minor
tor-netdoc patch
tor-netdir minor
tor-persist patch
tor-chanmgr minor
tor-guardmgr minor
tor-circmgr minor
tor-dirmgr minor
arti-client minor
arti-hyper minor
arti major
arti-bench minor
arti-testing minor
```
Since our last round of releases, these crates have had either
trivial changes, or changes that did not affect their APIs.
Therefore we are bumping their versions, but not changing which
versions of them other crates depend on.
Nick Mathewson
Gabriela Moldovan
Ian Jackson
Ian Jackson
Gabi Moldovan
Alexander Færøy
eta
Michael van Straten
Neel Chauhan
coral
trinity-1686a
Gabriel de Perthuis
Reylaba