This new function combines "sending a message" and "accepting
replies in a stream" into a single call, so that there is no gap
between when the message is sent and the replies are available.
There are a number of compromises here, in order to avoid API
proliferation. I've tried to contain them as best I can.
See comments for additional design discussion.
Now, the MetaCellHandler is responsible for consuming the messages
it gets, and reporting status to whatever task is waiting for a
status message.
Additionally, the MetaCellHandler can decide to remain installed or
shut down the circuit after a successful message. (Previously, it
could only uninstall itself on success and kill the circuit on
failure.)
These changes will enable MetaCellHandlers to be used as the basis
for handling more kinds of message.
(There is some moved and reformatted code here; you may want to
review it with `git {diff or show} --color-moved -b`.)
When I was trying to add HS support to these layers, I found I could
add a new variant to the `Host` enum but everything would still
compile even though I hadn't written the necessary implementation!
This method is a liability: when using it, one inevitably writes such
latent bugs.
Doing so doesn't seem like a good idea. It might even be some kind of
leak?
Found because I added a variant to `address::Host` for hidden
services, and noticed that the resolve code still compiled.
Doing this means that any attempt to use a read-only store would
crash as soon as it found that the consensus was usable.
It seems that this bug was introduced at some point doing all the
dirmgr refactors we did over the past year. Perhaps there should be
a test for running with a read-only store.
Fixes#779
Previously, if somebody wrote this code, an attacker could easily
use it to cause an OOM panic:
```
let n = r.take_u64();
let items: Vec<Foo> = r.extract_n(n as usize)?;
```
The first line of defense here is not to write protocols like that:
we don't actually _have_ any 32-bit counters in our protocol
AFAICT.
The second line of defense is to pre-check `n` for reasonableness
before calling `extract_n`.
Here we add a third line of defense: whereas previously we would do
`Vec::with_capacity(n)` in `extract_n`, we now allocate an initial
capacity of `min(n, r.remaining())`. This ensures that the size of
the allocation can't exceed the remaining length of the message,
which (for our cell types at least) should prevent it from
overflowing or running OOM.
These Arcs are all "downward", referencing items from layers lower in
the stack. So they don't cause cycles.
There was going to be a cycle involving the `OnionConnector` upcall
trait, but we have just abolished that.
Abolish CircMgr::get_or_launch_onion_client and everything to support
it. We have decided that `.onion` diversion ccan't/shouldn't occur in
tor-circmgr. Probably, it should occur much higher up - arti-client
maybe - since it will sometimes need ambient authority (KS_hsc_*).
Now all knowledge of HS connections is in tor-hsclient. This
gets rid of a layering inversion and the trait needed for tor-circmgr
to do the upcall to tor-hsclient.
This is getting rather confusing; simply reformtting it won't do, I
think. Also there would be much rightward drift.
So move the meat out into the new function.
(And introduce a convenience alias for its captures.)
Docs and reformatting will follow in a moment.
Explicit drops don't work. Instead, introduce a scope.
We need two scopes, actually: one where we do the initial table
wrangling, and one for the retries after relock.
So we must put the meat in a closure so we can reuse it.
And we must return the flow control as an enum. Bah, etc.
Avoid reformatting this for the moment. This makes the delta legible...
This is still correct from a lock hierarchy pov. It moves the guard
relock to the end, which is going to be necessary since it is going to
have to move right outside the loop.
Provide a more cookied "secret keys for use to connecting to a
particular HS" type, with a builder.
This wants to use config stuff, so oughtn't to be in tor-*crypto.
The individual types remain there.
*If* we're going to retain any HS knowledge in circmgr, it definitely
doesn't need to know about per-operation client secrets.
(Maybe there might be ambient secrets, used for .onion diversion, but
they don't need to be in this API.)
Otherwise there has to be a state entry in the circmgr *and* a state
entry in the hs connector, for every HS. This division of
responsibilit will be confusing.
The HS code will then be more completely just a layer on top of circmgr.
This makes our implementation behave the same as the C tor
implementation, by validating all of the expiration and signatures
on the certificates in the inner document.
(It is still not semantically necessary to check these certs: the
document in which they appear is already signed by the key with
which they are allegedly signed.)
Closes#744
The old code parsed and encoded a signature and a mac... but there
was no way to actually set them properly. Now EstablishIntro is
built around an EstablishIntroBody, and has the ability to check
signatures and macs.
Because there is no way to handle one of these messages if we can't
check the signature, we no longer accept unrecognized `auth_key` types
in this message.
I've added a test to make sure that we can validate a message from the
C tor implementation, and a test to make sure we can validate our
own cells. I also had to modify the previous tests so that their
keys were well-formed.
These crates didn't have any changes until now, when I bumped
the versions of some other crates they depend on:
tor-consdiff
arti-hyper
arti-bench
arti-testing
These crates have had small code changes, but no API additions:
tor-config
tor-socksproto
tor-cert
tor-chanmgr
tor-ptmgr
tor-guardmgr
tor-circmgr
tor-dirclient
tor-dirmgr
arti
tor-hsservice
tor-congestion
These crates have had API extensions:
fs-mistrust
tor-llcrypto
tor-bytes
tor-checkable
tor-linkspec
tor-netdoc
tor-persist
arti-client
These are API breaks, but the crates themselves are currently
100% experimental, so there's no need to bump the minor versions
according to our semver rules.
The four values of times taken in a particular test were changed to both
be human readable and have comments explaining their significance (they
are all important moments after the Unix Epoch for freedom)
Unlike linkspec, this doesn't validate the actual contents of the
specifiers. We'll use this so we can handle the linkspec list for an
introduction point in an HsDesc, and just pass it on when
constructing our circuits.
I haven't added any accessor or constructor functions, because I
don't expect to need them.
It said to check whether C enforces an absence of extraneous bytes
at the end of the link specifiers. It does, in
`hs_desc.c:decode_link_specifiers()`, where it says:
```
if (link_specifier_list_parse(&specs, decoded,
(size_t) decoded_len) < decoded_len) {
goto err;
}
```
The comparison with "decoded_len" checks whether all the bytes were
decoded.
Some code in our tests that worked fine with time 0.3.17 no
longer works with 0.3.19, despite the semver.
See https://github.com/time-rs/time/issues/552 for the upstream bug.
This change makes sure that open streams and half-closed streams
have the same stream-type-dependent state machines with respect to
which cells are acceptable.
Fixes#774.
Fixes#769.
The role of CmdChecker is to verify that messages are arriving at
the appropriate sequence on a stream, with respect to the other
messages that have been received. Once the stream becomes
half-closed, the CmdChecker is also in charge of consuming incoming
messages on the stream and making sure that they are well-formed.
There's a blanket impl of Itertor for &mut impl Iterator, so this
isn't necessary, and it prevents us passing iterators by value
producing syntactic vinegar.
This includes a partial solution for #769, but also turned up
another bug (#774) while I was working on it. I'll close them both
once I have a real solution.
We'll use this to router relay messages on a circuit to the
appropriate stream, and hand them to that stream, without parsing
the message until the stream has been determined.
We now manipulate raw relay cell bodies as (an alias for)
`Box<[u8;509]>` rather than as (an alias for) `[u8;509]`. This
enables us to do much less copying. It will become more important
soon, as we defer parsing relay cell bodies even longer.
Related to #7.
We also use SliceWriter to avoid allocating a Vec<> for every relay
message we want to encode, and instead encode directly into the
cell.
Because the API assumes that many writes are infallible, this writer
takes ownership of the backing object, and will only return it to
you if you didn't run over the end.
I'm going to use this to save some allocations in relay cell bodies
Several HS message types have an extension list type. They all use
the same framing for extensions, but each of them has separate
extension types and separate extension namespaces.
This commit simplifies establish_intro a little, and adds support
for maintaining unrecognized extension types--at the expense of some
new internal code.
Some of the HS message types have a lot of dependent types, like
extensions and options for those extensions, and so on. Except when
those extensions are portable across cell types, it makes sense
to put them in their own modules.
We want to fuzz these parsers, but there's no currently way to get
at the parsers for inner documents without going through a lot of
encryption. (Coverage-guided fuzzers are powerful, but they
can't find SHA3 preimages.)
There will be two call sites to demonstrate it.
Eventually maybe this will want to be in tor-basic-utils, since it
doesn't depend on any of the tor-netdoc types. But it would be
sensible to wait until the situation with PeekableIterator and
Itertools is improved.
For now we make this #[doc(hidden)] to avoid it becoming part of our
stable API.
This one was generated (by dgoulet) using `ClientDescEncKey`
encryption. Its information is:
```
Address: paozpdhgz2okvc6kgbxvh2bnfsmt4xergrtcl4obkhopyvwxkpjzvoad.onion
Time period: 19397
Client:
paozpdhgz2okvc6kgbxvh2bnfsmt4xergrtcl4obkhopyvwxkpjzvoad:descriptor:x25519:SDZNMD4RP4SCH4EYTTUZPFRZINNFWAOPPKZ6BINZAC7LREV24RBQ
Service:
descriptor:x25519:SACGOAEODFGCYY22NYZV45ZESFPFLDGLMBWFACKEO34XGHASSAMQ
```
Fixes an instance of #768.
The problem with the test vectors is that I used a random time
period number (1234) and the default-in-tor period length (1440)
without checking whether 1440 _meant_ minutes or seconds. I'll add
another test to Tor to make sure that the time period matches now.
With this change, I can test Tor-generated hsdescs with encryption,
so I'm fairly confident that the new behavior is correct.
Nick Mathewson
Ian Jackson
Ian Jackson
Dimitris Apostolou
Saksham Mittal