Commit Graph

7920 Commits

Author SHA1 Message Date
Gabriela Moldovan 445c052420
tor-proto: Add StreamTarget function for sending CtrlMsg::CloseStream .
This will be used by `IncomingStream` to implement `reject()`.
2023-08-03 16:31:05 +01:00
Gabriela Moldovan b9daaa245c
tor-proto: Add reactor control commands for accepting/closing incoming streams.
This adds a new `AwaitIncomingStream` control message for registering an
interest in an incoming stream.

This also adds a `ClosePendingStream` control message for explicitly
closing a stream with a given END message (needed for implementing
`IncomingStream::reject`).
2023-08-03 16:29:26 +01:00
Gabriela Moldovan 950d0da0b5
tor-proto: Add types for sharing stream request info with the reactor. 2023-08-03 16:27:48 +01:00
Gabriela Moldovan 18b01f94cc
tor-proto: Add a helper for adding a stream entry with a specific stream ID.
This adds a new `add_ent_with_id` function for adding a new entry to the
`StreamMap`. The existing `add_ent` function auto-generates a new stream
ID, which is not good if we're a hidden service, as stream IDs are
supposed to be chosen by the OP (client). When accepting a new stream,
services, exit relays, and dir auths need to use the stream ID received
in the BEGIN cell (instead of generating a new stream ID).
2023-08-03 16:25:46 +01:00
Gabriela Moldovan 4abfe30f41
tor-proto: Remove extraneous space. 2023-08-03 16:24:30 +01:00
Gabriela Moldovan f6745d31a9
tor-proto: Add constructors for pre-connected DataStreams.
When accepting a new stream, hidden services, exit relays and dirauths
don't wait for a `CONNECTED` cell from the initiator.

This commit adds constructors for building `DataStream`s and
`DataCmdChecker`s that can immediately receive data cells (and don't
expect to receive `CONNECTED` cells at all).
2023-08-03 16:23:11 +01:00
Gabriela Moldovan 4133a9464e
tor-proto: Add a constructor for IncomingStream. 2023-08-03 16:21:55 +01:00
Gabriela Moldovan dc0b904492
tor-proto: Add imports for commonly used types. 2023-08-03 16:20:45 +01:00
Gabriela Moldovan 3d5c712c55
tor-proto: Add IncomingCmdChecker for IncomingStream. 2023-08-03 16:18:35 +01:00
Nick Mathewson 5a84da3e1b Fix and expand the description of OnionAddressDisabled
Previously there was a missing "not", and it didn't say what to do
about the error.
2023-08-03 09:43:03 -04:00
Ian Jackson 5353a02138 Merge branch 'equix_bench_cargo_lock' into 'main'
equix/bench: Update Cargo.lock

See merge request tpo/core/arti!1466
2023-08-03 12:04:54 +00:00
Micah Elizabeth Scott 943960e4d3 equix/bench: Update Cargo.lock
New dependency from 82eb88bd74
2023-08-02 19:29:23 -07:00
Nick Mathewson 5c607e8cf6 Merge branch 'ticket889_fuzz' into 'main'
Fuzzers for Equi-X and HashX

See merge request tpo/core/arti!1459
2023-08-02 22:12:20 +00:00
Nick Mathewson 211ef5d26b Bump to new version of arti-corpora
This is the one with hashx/equix support.
2023-08-02 18:01:57 -04:00
Micah Elizabeth Scott 21dfc6d58c equix/fuzz: CI and doc fixes
The build found a stale private doc comment as well as an exception
that needed to be made in check_doc_features.

The check_doc_features change solidifies a decision that things marked
with cfg(fuzzing) aren't part of the documented API.
2023-08-02 14:46:46 -07:00
Nick Mathewson c7feae494e Merge branch 'ticket991_mr' into 'main'
Reduce runtime of hspow_vectors and split it up

See merge request tpo/core/arti!1462
2023-08-02 15:49:03 +00:00
Ian Jackson 3ca2870123 Merge branch 'encodable-key-rename' into 'main'
keymgr: Replace EncodableKey::to_bytes() with SSH-specific function.

Closes #965

See merge request tpo/core/arti!1464
2023-08-02 14:59:42 +00:00
Ian Jackson 43cbd80f23 Merge branch 'ticket889_bench' into 'main'
equix, hashx: Benchmark against C implementation

See merge request tpo/core/arti!1446
2023-08-02 14:58:20 +00:00
Gabriela Moldovan b57a60d7b6
keymgr: Add TODO regarding SshEncodableKey impl for x25519. 2023-08-02 14:21:41 +01:00
Gabriela Moldovan 682d20110e
keymgr: Implement SshEncodableKey for ed25519::Keypair. 2023-08-02 14:21:06 +01:00
Gabriela Moldovan e89e76f974
keymgr: Remove KeyType::to_ssh_format.
This function isn't actually needed (it's not the responsibility of
`KeyType` to encode keys).

This commit also rewrites `ArtiNativeKeystore::insert` to use the new
`as_ssh_keypair_data` function instead of `to_ssh_format`.
2023-08-02 14:21:02 +01:00
Gabriela Moldovan e70be1353c
keymgr: Replace EncodableKey::to_bytes() with SSH-specific function.
The `EncodableKey::to_bytes` function didn't make much sense, because
not all keys have a canonical byte representation.

This commit replaces `EncodableKey::to_bytes` with
`EncodableKey::as_ssh_keypair_data`. In the future, `EncodableKey` will
grow functions for encoding keys in other storage formats too.

Closes #965
2023-08-02 14:13:18 +01:00
Micah Elizabeth Scott bf0119fbfe hashx/fuzz, equix/fuzz: use arti-corpora
Remove corpus from .gitignore and add a symlink to the corpora
submodule.
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott d7c2e7996e hashx/fuzz: update tor-c-equix dependency
my cargo_hashx_rng branch was just merged into main (thanks dgoulet!)
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 0ffb2849e1 equix/fuzz: rustfmt 2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 82eb88bd74 equix/fuzz: Fixes to get nick's prototype running
There are some places we might improve this, maybe testing more data
types and shapes. This patch just makes the minimal changes necessary
to get it working: adds allocation logic to the fuzzer itself,
and adds visibility for the bucket_array::mem interface.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 810adcc50d hashx/fuzz: Comments, explain our 'seed' input
In response to review feedback, explain that 'seed' here is more
for compatibility and convenience and not central to our goal of
fuzzing the program generator.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:04 -07:00
Micah Elizabeth Scott 5a85749e48 hashx/fuzz: Simplify, remove rayon dependency
Review feedback is that we don't want parallelism here.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:03 -07:00
Micah Elizabeth Scott ea2716595e hashx/fuzz: Start a cross-implementation fuzzer for HashX
Fuzz testing for HashX. Uses a hook into the pseudorandom number
stream to test the program generator deeply on input that can
be mutated by the fuzzer. Confirms program generation by running
a small number of arbitrary test hashes, so we don't need to
understand the implementation-specific program format to test the
program generator.

We test four implementations in parallel this way, the compiled and
interpreted implementations included in both this crate and c-tor.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:34:03 -07:00
Nick Mathewson 89fb338fd5 WIP: Fuzzer for BucketArray. 2023-08-01 19:34:03 -07:00
Micah Elizabeth Scott f40255cb95 tor-hspow: Shorten the solve_effort1k_aa_41_01 test
The solve tests are all tunable so that we can balance execution time
with test coverage. A longer solve will test more random programs and it
will test more of the nonce increment function, minor benefits at the
cost of much more CPU.

The starting nonce in solve_effort1k_aa_41_01 was set so that we would
exercise a rollover in bit 7 of the nonce increment before the full
width rollover, but this wasn't a particularly helpful place to test
and certainly not worth the 13+ seconds it takes on my machine.

This patch bumps the starting nonce to a value much closer to the
target, and still including the full-width rollover.

Brings solve_effort1k_aa_41_01 down from 13.2 seconds to 0.5 sec for me.

For ticket #991

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:31:23 -07:00
Micah Elizabeth Scott f84af7c360 tor-hspow: Split up hspow_vectors tests
This test had one large function for all the verifies and one for all
the solves. The solve test was slow enough to be a bottleneck,
documented in ticket #991.

This patch splits the existing tests up in to one labeled function per
solve or verify configuration.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:31:23 -07:00
Micah Elizabeth Scott 550d89fb57 hashx/bench: Shared generate wrapper for u64-hash and full-hash
Code cleanup from review feedback

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:46 -07:00
Micah Elizabeth Scott 315122f159 hashx/bench, equix/bench: Enable debug symbols
Propagates this setting from the outer Cargo.toml to the new
benchmark crates, since they no longer get the setting by
being included in the main workspace.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:46 -07:00
Micah Elizabeth Scott 256e5de9e5 hashx/bench, equix/bench: check in matching Cargo.lock files
It might be useful to keep these locked down for benchmark
reproducibility. Currently the hashx and equix crates are
fully separate.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:46 -07:00
Micah Elizabeth Scott ae58ea0697 equix, hashx: Benchmark against C implementation
This is a small batch of improvements for the equix and hashx
benchmarks. The headline feature is that we are now including
the C implementations (slightly modified from tevador's, hosted
as part of c-tor) and using them in apples-to-apples comparisons.

Minor features:
- Benchmarks moved to new nested crates, preventing their
  dependencies from spilling into the main workspace build.
- Tests are now grouped
- We also test the performance of memory reuse where possible
- Code cleanup for per-runtime options

These benchmark builds will now automatically pull in the c-tor
git repo and build portions of it with a Rust wrapper. This uses
the 'cc' and 'bindgen' crates, so it requires a C compiler and
libclang on the host system.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-08-01 19:28:43 -07:00
Nick Mathewson b0b6e47d05 Merge branch 'misc_upgrades' into 'main'
Upgrade to sanitize-filename 0.5.0

See merge request tpo/core/arti!1453
2023-08-02 00:11:42 +00:00
Nick Mathewson 1dc47a65cf Upgrade to sanitize-filename 0.5.0 2023-08-01 19:47:39 -04:00
Nick Mathewson 4abad7b6fd Merge branch 'update-hss-dep' into 'main'
hsservice: Upgrade tor-keymgr dep

See merge request tpo/core/arti!1463
2023-08-01 23:28:58 +00:00
Nick Mathewson f7c0350e7f hsservice: Upgrade tor-keymgr dep
This breakage was caused by increasing the version of tor-keymgr
and independently merging !1452, which added a dependency on the
old version.
2023-08-01 18:51:55 -04:00
Ian Jackson f365b41c13 Merge branch 'remove_semver' into 'main'
Remove semver.md files now that 1.1.7 is out.

See merge request tpo/core/arti!1460
2023-08-01 17:42:58 +00:00
Ian Jackson d799a7c5a7 Merge branch 'ipts2bis' into 'main'
dev notes: Draft IPT algorithm: Overhaul for latest version of torspec!154

See merge request tpo/core/arti!1444
2023-08-01 17:42:18 +00:00
gabi-250 3a64c5b79c Merge branch 'bump_version' into 'main'
Script to bump patchlevel without touching other crates

Closes #945

See merge request tpo/core/arti!1461
2023-08-01 17:35:29 +00:00
Nick Mathewson dee95436f5 Merge branch 'hss_apis' into 'main'
hsservice: Initial data structures and APIs

Closes #972, #971, and #970

See merge request tpo/core/arti!1452
2023-08-01 17:06:10 +00:00
Nick Mathewson b71b04eb84 Script to bump patchlevel without touching other crates
Closes #945.

Based on @diziet's comment in #945, with a little extra safety and
paranoia.  We use this for cases when there are trivial changes only
in one of our crates.
2023-08-01 12:57:58 -04:00
Nick Mathewson 1e96d1b95a Remove semver.md files now that 1.1.7 is out. 2023-08-01 12:55:52 -04:00
Ian Jackson bcde97624b Merge branch 'pt-not-found-err' into 'main'
Add error if [[bridges.transports]] isn't written in config file

Closes #880

See merge request tpo/core/arti!1229
2023-08-01 16:43:46 +00:00
Saksham Mittal 752f748910 Add error if [[bridges.transports]] isn't written in config file 2023-08-01 16:43:46 +00:00
Nick Mathewson 3422169ff4 Merge branch 'bump_versions_117' into 'main'
Vesion bumps for 1.1.7.

See merge request tpo/core/arti!1458
2023-08-01 15:40:26 +00:00
Nick Mathewson 9e96079b6d Merge branch 'changelog_117_part2' into 'main'
Changelog 117 part2

See merge request tpo/core/arti!1457
2023-08-01 15:13:20 +00:00