Commit Graph

5150 Commits

Author SHA1 Message Date
Gabriela Moldovan ed5a051ebe
tor-proto: Add ClientCirc::start_conversation().
This will enable hidden services to send `RENDEZVOUS1` messages to the
`N`th hop of the circuit rather than the `N + 1`th virtual one (which
can only used after the client and service have completed the
introduction handshake).

This also deprecates `start_conversation_last_hop`.

Closes #959
2023-08-04 13:08:31 +01:00
Gabriela Moldovan c23e85270e
tor-proto: Add method for getting the HopNum of the last hop. 2023-08-04 13:07:52 +01:00
Gabriela Moldovan eee3bb8822
tor-proto: Make HopNum public.
`HopNum` will be used in `ClientCirc`'s public API when we refactor
`ClientCirc::start_conversation_last_hop` to use the provided hop rather
than always using the last one.
2023-08-04 10:51:05 +01:00
gabi-250 ef5fdd0647 Merge branch 'proto-incoming-stream' into 'main'
tor-proto: Let onion services accept incoming BEGIN cells and make streams

Closes #864

See merge request tpo/core/arti!1451
2023-08-03 18:15:50 +00:00
Gabriela Moldovan 71d99ce709
tor-proto: Bump test sleep to 200ms for now.
This `sleep` is to give the reactor task a chance to process the
`AwaitIncomingStream` message. With an 100ms, this test sometimes fails
because for some reason the reactor doesn't get a chance to process the
`AwaitIncomingStream` control command before the BEGIN cell from the
client task is received. This bumps the sleep time to 200ms for now
(TODO: follow-up with an MR with a less flaky approach).

While trying to repro the issue, I found another corner case for which
I've added a TODO HSS.
2023-08-03 18:50:34 +01:00
Gabriela Moldovan 0ec1303b9a
tor-proto: Handle new BEGIN cells for rejected stream.
This updates the reactor to call the incoming stream handler even for
streams for which we have a stream map entry of `EndSent`. If we've
sent an END message for a stream but have not yet received an END
message back from the other party, but we later receive a BEGIN from
them, it is safe to assume we cam remove the stream from the stream map
and handle the new incoming stream request.
2023-08-03 16:46:28 +01:00
Gabriela Moldovan 296abfddcc
tor-proto: Clarify what we mean by "stream" in allow_stream_requests. 2023-08-03 16:44:45 +01:00
Gabriela Moldovan 23f1f2ce47
tor-proto: Add note about making hop_num optional. 2023-08-03 16:43:26 +01:00
Gabriela Moldovan 2f21dc5a87
tor-proto: Add a TODO about checking if a hop is allowed to create streams. 2023-08-03 16:42:12 +01:00
Gabriela Moldovan ffe4771a83
tor-proto: Remove unnecessary else-branch.
We return early if `message_closes_stream == true`, so we can get rid of
the `else` to remove one level of indentation.
2023-08-03 16:40:57 +01:00
Gabriela Moldovan 4edff7e670
tor-proto: Update the stream map if we get an END message.
This handles the previously not handled `message_closes_stream == true`
case.
2023-08-03 16:39:42 +01:00
Gabriela Moldovan 907d8cf255
tor-proto: Add TODO regarding allow_stream_requests corner case. 2023-08-03 16:38:19 +01:00
Gabriela Moldovan f30c5897cc
tor-proto: Remove outdated comments. 2023-08-03 16:36:58 +01:00
Gabriela Moldovan 0f9bed5160
tor-proto: Add a test for allow_stream_requests. 2023-08-03 16:35:43 +01:00
Gabriela Moldovan 37154dca95
tor-proto: Implement allow_stream_requests. 2023-08-03 16:34:09 +01:00
Gabriela Moldovan ffa8056437
tor-proto: Handle RELAY_BEGIN cells if we have an incoming req handler. 2023-08-03 16:33:58 +01:00
Gabriela Moldovan f06e0e2df0
tor-proto: Implement IncomingStream::{accept_data, request, reject}. 2023-08-03 16:32:29 +01:00
Gabriela Moldovan 445c052420
tor-proto: Add StreamTarget function for sending CtrlMsg::CloseStream .
This will be used by `IncomingStream` to implement `reject()`.
2023-08-03 16:31:05 +01:00
Gabriela Moldovan b9daaa245c
tor-proto: Add reactor control commands for accepting/closing incoming streams.
This adds a new `AwaitIncomingStream` control message for registering an
interest in an incoming stream.

This also adds a `ClosePendingStream` control message for explicitly
closing a stream with a given END message (needed for implementing
`IncomingStream::reject`).
2023-08-03 16:29:26 +01:00
Gabriela Moldovan 950d0da0b5
tor-proto: Add types for sharing stream request info with the reactor. 2023-08-03 16:27:48 +01:00
Gabriela Moldovan 18b01f94cc
tor-proto: Add a helper for adding a stream entry with a specific stream ID.
This adds a new `add_ent_with_id` function for adding a new entry to the
`StreamMap`. The existing `add_ent` function auto-generates a new stream
ID, which is not good if we're a hidden service, as stream IDs are
supposed to be chosen by the OP (client). When accepting a new stream,
services, exit relays, and dir auths need to use the stream ID received
in the BEGIN cell (instead of generating a new stream ID).
2023-08-03 16:25:46 +01:00
Gabriela Moldovan 4abfe30f41
tor-proto: Remove extraneous space. 2023-08-03 16:24:30 +01:00
Gabriela Moldovan f6745d31a9
tor-proto: Add constructors for pre-connected DataStreams.
When accepting a new stream, hidden services, exit relays and dirauths
don't wait for a `CONNECTED` cell from the initiator.

This commit adds constructors for building `DataStream`s and
`DataCmdChecker`s that can immediately receive data cells (and don't
expect to receive `CONNECTED` cells at all).
2023-08-03 16:23:11 +01:00
Gabriela Moldovan 4133a9464e
tor-proto: Add a constructor for IncomingStream. 2023-08-03 16:21:55 +01:00
Gabriela Moldovan dc0b904492
tor-proto: Add imports for commonly used types. 2023-08-03 16:20:45 +01:00
Gabriela Moldovan 3d5c712c55
tor-proto: Add IncomingCmdChecker for IncomingStream. 2023-08-03 16:18:35 +01:00
Nick Mathewson 5a84da3e1b Fix and expand the description of OnionAddressDisabled
Previously there was a missing "not", and it didn't say what to do
about the error.
2023-08-03 09:43:03 -04:00
Micah Elizabeth Scott 943960e4d3 equix/bench: Update Cargo.lock
New dependency from 82eb88bd74
2023-08-02 19:29:23 -07:00
Nick Mathewson 5c607e8cf6 Merge branch 'ticket889_fuzz' into 'main'
Fuzzers for Equi-X and HashX

See merge request tpo/core/arti!1459
2023-08-02 22:12:20 +00:00
Micah Elizabeth Scott 21dfc6d58c equix/fuzz: CI and doc fixes
The build found a stale private doc comment as well as an exception
that needed to be made in check_doc_features.

The check_doc_features change solidifies a decision that things marked
with cfg(fuzzing) aren't part of the documented API.
2023-08-02 14:46:46 -07:00
Nick Mathewson c7feae494e Merge branch 'ticket991_mr' into 'main'
Reduce runtime of hspow_vectors and split it up

See merge request tpo/core/arti!1462
2023-08-02 15:49:03 +00:00
Ian Jackson 3ca2870123 Merge branch 'encodable-key-rename' into 'main'
keymgr: Replace EncodableKey::to_bytes() with SSH-specific function.

Closes #965

See merge request tpo/core/arti!1464
2023-08-02 14:59:42 +00:00
Gabriela Moldovan b57a60d7b6
keymgr: Add TODO regarding SshEncodableKey impl for x25519. 2023-08-02 14:21:41 +01:00
Gabriela Moldovan 682d20110e
keymgr: Implement SshEncodableKey for ed25519::Keypair. 2023-08-02 14:21:06 +01:00
Gabriela Moldovan e89e76f974
keymgr: Remove KeyType::to_ssh_format.
This function isn't actually needed (it's not the responsibility of
`KeyType` to encode keys).

This commit also rewrites `ArtiNativeKeystore::insert` to use the new
`as_ssh_keypair_data` function instead of `to_ssh_format`.
2023-08-02 14:21:02 +01:00
Gabriela Moldovan e70be1353c
keymgr: Replace EncodableKey::to_bytes() with SSH-specific function.
The `EncodableKey::to_bytes` function didn't make much sense, because
not all keys have a canonical byte representation.

This commit replaces `EncodableKey::to_bytes` with
`EncodableKey::as_ssh_keypair_data`. In the future, `EncodableKey` will
grow functions for encoding keys in other storage formats too.

Closes #965
2023-08-02 14:13:18 +01:00
Micah Elizabeth Scott bf0119fbfe hashx/fuzz, equix/fuzz: use arti-corpora
Remove corpus from .gitignore and add a symlink to the corpora
submodule.
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott d7c2e7996e hashx/fuzz: update tor-c-equix dependency
my cargo_hashx_rng branch was just merged into main (thanks dgoulet!)
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 0ffb2849e1 equix/fuzz: rustfmt 2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 82eb88bd74 equix/fuzz: Fixes to get nick's prototype running
There are some places we might improve this, maybe testing more data
types and shapes. This patch just makes the minimal changes necessary
to get it working: adds allocation logic to the fuzzer itself,
and adds visibility for the bucket_array::mem interface.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 810adcc50d hashx/fuzz: Comments, explain our 'seed' input
In response to review feedback, explain that 'seed' here is more
for compatibility and convenience and not central to our goal of
fuzzing the program generator.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 5a85749e48 hashx/fuzz: Simplify, remove rayon dependency
Review feedback is that we don't want parallelism here.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:03 -07:00
Micah Elizabeth Scott ea2716595e hashx/fuzz: Start a cross-implementation fuzzer for HashX
Fuzz testing for HashX. Uses a hook into the pseudorandom number
stream to test the program generator deeply on input that can
be mutated by the fuzzer. Confirms program generation by running
a small number of arbitrary test hashes, so we don't need to
understand the implementation-specific program format to test the
program generator.

We test four implementations in parallel this way, the compiled and
interpreted implementations included in both this crate and c-tor.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:03 -07:00
Nick Mathewson 89fb338fd5 WIP: Fuzzer for BucketArray. 2023-08-01 19:34:03 -07:00
Micah Elizabeth Scott f40255cb95 tor-hspow: Shorten the solve_effort1k_aa_41_01 test
The solve tests are all tunable so that we can balance execution time
with test coverage. A longer solve will test more random programs and it
will test more of the nonce increment function, minor benefits at the
cost of much more CPU.

The starting nonce in solve_effort1k_aa_41_01 was set so that we would
exercise a rollover in bit 7 of the nonce increment before the full
width rollover, but this wasn't a particularly helpful place to test
and certainly not worth the 13+ seconds it takes on my machine.

This patch bumps the starting nonce to a value much closer to the
target, and still including the full-width rollover.

Brings solve_effort1k_aa_41_01 down from 13.2 seconds to 0.5 sec for me.

For ticket #991

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:31:23 -07:00
Micah Elizabeth Scott f84af7c360 tor-hspow: Split up hspow_vectors tests
This test had one large function for all the verifies and one for all
the solves. The solve test was slow enough to be a bottleneck,
documented in ticket #991.

This patch splits the existing tests up in to one labeled function per
solve or verify configuration.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:31:23 -07:00
Micah Elizabeth Scott 550d89fb57 hashx/bench: Shared generate wrapper for u64-hash and full-hash
Code cleanup from review feedback

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:46 -07:00
Micah Elizabeth Scott 315122f159 hashx/bench, equix/bench: Enable debug symbols
Propagates this setting from the outer Cargo.toml to the new
benchmark crates, since they no longer get the setting by
being included in the main workspace.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:46 -07:00
Micah Elizabeth Scott 256e5de9e5 hashx/bench, equix/bench: check in matching Cargo.lock files
It might be useful to keep these locked down for benchmark
reproducibility. Currently the hashx and equix crates are
fully separate.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:46 -07:00
Micah Elizabeth Scott ae58ea0697 equix, hashx: Benchmark against C implementation
This is a small batch of improvements for the equix and hashx
benchmarks. The headline feature is that we are now including
the C implementations (slightly modified from tevador's, hosted
as part of c-tor) and using them in apples-to-apples comparisons.

Minor features:
- Benchmarks moved to new nested crates, preventing their
  dependencies from spilling into the main workspace build.
- Tests are now grouped
- We also test the performance of memory reuse where possible
- Code cleanup for per-runtime options

These benchmark builds will now automatically pull in the c-tor
git repo and build portions of it with a Rust wrapper. This uses
the 'cc' and 'bindgen' crates, so it requires a C compiler and
libclang on the host system.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:43 -07:00