Commit Graph

396 Commits

Author SHA1 Message Date
Gabriela Moldovan c166c878fe Create a new `IntroPointDesc` type.
Unlike `hsdesc::IntroPointDesc`, `hsdesc::build::IntroPointDesc`
represents link specifiers as `LinkSpec`s rather than
`UnparsedLinkSpec`s.

Since this is a general-purpose representation of an introduction point
and not merely an intermediate representation for decoding/encoding, it
will probably need to be factored out of `tor-netdoc` at some point.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-24 18:58:33 +00:00
Gabriela Moldovan 1e3214a8bd Use the less ambiguous inner/middle/outer terminology. 2023-03-24 18:57:53 +00:00
Gabi Moldovan a1074c0027 Add builder for encoding hidden service descriptors.
This introduces the `NetdocBuilder` trait described in
`netdoc-builder.md` and a new `tor-netdoc::doc::hsdesc::build` module,
which exports the `HsDescBuilder`. Hidden services will use
`HsDescBuilder` to build and encode hidden service descriptors.

There are several TODOs in the code that I'm planning to address
separately.

Partially addresses #745.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-24 18:57:49 +00:00
Gabriela Moldovan 89ca965d2a Define constants for `AuthClient` field lengths.
These are used in multiple places (and will also be used by the HS
descriptor encoder later on), so let's make them named constants.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-24 18:54:35 +00:00
Gabriela Moldovan 1394a81142 Implement `ItemArgument` for `String`.
This makes some code a bit more concise, as it allows us to make
`Strings` into `ItemArgument`s without calling `.as_str()`.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-24 18:54:35 +00:00
Ian Jackson 70b099fd0f Turn two blocking TODOs in hsdesc into "TODO HS"
TODO HS is blocking for HS release, but not for CI.
2023-03-21 13:08:14 +00:00
Nick Mathewson d6bb8fee0b Upgrade to bitflags 2.0
This version of bitflags now requires us to explicitly list the
traits that we want to derive for our bit-flags structs.
2023-03-13 08:47:14 -04:00
Dimitris Apostolou dedae15446
Fix typos 2023-03-03 13:00:33 +02:00
Nick Mathewson 87d9dd6423 tor-netdoc: Clarify that we must indeed check cert expiration. 2023-03-01 09:49:17 -05:00
Nick Mathewson a4a305c1cf tor-netdoc: Validate inner certs in HsDesc
This makes our implementation behave the same as the C tor
implementation, by validating all of the expiration and signatures
on the certificates in the inner document.

(It is still not semantically necessary to check these certs: the
document in which they appear is already signed by the key with
which they are allegedly signed.)

Closes #744
2023-03-01 09:48:51 -05:00
Nick Mathewson f41449d0bd Patchlevel bumps for remaining changed crates.
These crates have had small code changes, but no API additions:

tor-config
tor-socksproto
tor-cert
tor-chanmgr
tor-ptmgr
tor-guardmgr
tor-circmgr
tor-dirclient
tor-dirmgr
arti
tor-hsservice
tor-congestion

These crates have had API extensions:

fs-mistrust
tor-llcrypto
tor-bytes
tor-checkable
tor-linkspec
tor-netdoc
tor-persist
arti-client
2023-02-28 07:13:27 -05:00
Nick Mathewson 3e1ae65212 Bump tor-units version for breaking change.
(The breaking change was removing `as_days()` from IntegerMinutes.)

We are _not_ calling this a downstream-api breaking change, per
discussion at
https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1012?commit_id=bb2ab7c2a3e0994bb438188511688b5b039cae29#note_2876819
2023-02-28 07:10:00 -05:00
Nick Mathewson 4479d52a88 Bump patch-versions for breaking changes in tor-hs{crypto,client}
These are API breaks, but the crates themselves are currently
100% experimental, so there's no need to bump the minor versions
according to our semver rules.
2023-02-28 07:06:15 -05:00
Ian Jackson f63c7c48af Rename onion-* cargo features to hs-* in tor-* crates
Fixes #756
2023-02-28 11:10:17 +00:00
Nick Mathewson d09273a6e3 Apply a missing cfg_attr(docsrs...) in netdoc 2023-02-21 15:18:46 +00:00
Ian Jackson 750c512264 Merge branch 'unparsed_linkspec' into 'main'
HsDesc: Use a new UnparsedLinkSpecifier to avoid leaking which linkspec types we know

See merge request tpo/core/arti!1029
2023-02-21 15:16:45 +00:00
Nick Mathewson 3233c25943 tor-netdoc: Use UnparsedLinkSpec in IntroPointDesc. 2023-02-17 15:29:37 -05:00
Nick Mathewson 757c6d333f netdoc: Remove a TODO hs comment.
It said to check whether C enforces an absence of extraneous bytes
at the end of the link specifiers.  It does, in
`hs_desc.c:decode_link_specifiers()`, where it says:

```
  if (link_specifier_list_parse(&specs, decoded,
                                (size_t) decoded_len) < decoded_len) {
    goto err;
  }
```

The comparison with "decoded_len" checks whether all the bytes were
decoded.
2023-02-17 15:06:09 -05:00
Nick Mathewson 1297dfb5e3 Merge branch 'doc-iter' into 'main'
tor-netdoc: Abolish PauseAt in favour of using itertools

Closes #760

See merge request tpo/core/arti!1021
2023-02-17 12:34:32 +00:00
Ian Jackson 66beab6c9e tor-netdoc: Fix doc comment following removal of PauseAt 2023-02-16 11:53:32 +00:00
Ian Jackson ca127b013f Expire routerdescs as soon as any of their expiries expire
Fixes #772
2023-02-15 17:41:28 +00:00
Ian Jackson 676241c349 tor-netdoc: Run rustfmt (separately from the refactoring) 2023-02-15 17:17:01 +00:00
Ian Jackson 1993165f22 tor-netdoc: Abolish NetDocReader::iter and into_iter
Nothing uses these now; the NetDocReader is simply an iterator, itself.
2023-02-15 17:17:01 +00:00
Ian Jackson 4a91c732fe tor-netdoc: Remove many needless calls to .iter() and .into_iter() 2023-02-15 17:17:01 +00:00
Ian Jackson fa2e164939 tor-netdoc: Use impl Iterator for NetdocReader in .pause_at
This simplifies the return type!
2023-02-15 17:17:01 +00:00
Ian Jackson db79e3f369 tor-netdoc: impl Iterator etc. for NetdocReader
This will simplify things at many call sites.
2023-02-15 17:17:01 +00:00
Ian Jackson b58977b428 tor-netdoc: move PeekableIterator to util.rs
We're going to want this a bit more widely.
2023-02-15 17:17:01 +00:00
Ian Jackson 11660f84b0 tor-netdoc: Delete now-unused PauseAt type 2023-02-15 17:17:01 +00:00
Ian Jackson 15d93bd642 tor-netdoc: Use itertools to implement NetDocReader::pause_at
This is the last use of PauseAt.
2023-02-15 17:17:01 +00:00
Ian Jackson 089ebb7d11 tor-netdoc: Replace use of PauseAt::new_pred()
We can just make a new PauseAt, since it can take the inner peekable
iterator by mutable reference.

This seems to tidy the code up a bit too.
2023-02-15 17:16:57 +00:00
Ian Jackson 7491424ee9 tor-netdoc: Have sectionrules.parse take I, not &mut I
There's a blanket impl of Itertor for &mut impl Iterator, so this
isn't necessary, and it prevents us passing iterators by value
producing syntactic vinegar.
2023-02-15 17:16:57 +00:00
Ian Jackson 80d8d94c96 tor-netdoc: Import Peekable rather than referring to it by path
This makes things easier to read.  We're about to add another
site (albeit, temporarily).
2023-02-15 17:16:57 +00:00
Nick Mathewson e00dfcf41e netdoc: Add a basic fuzzer for the hsdesc layers.
I've been running this for 5 minutes and haven't found any
crashes. I'll run it for an hour or two, then add the results to the
corpus repository.
2023-02-14 07:49:34 -05:00
Nick Mathewson 79dd2ef995 netdoc: Add a feature to expose hsdesc inner doc parsers
We want to fuzz these parsers, but there's no currently way to get
at the parsers for inner documents without going through a lot of
encryption.  (Coverage-guided fuzzers are powerful, but they
can't find SHA3 preimages.)
2023-02-14 07:44:18 -05:00
Nick Mathewson 74ef8de86f Merge branch 'doc-iter' into 'main'
Introduce new batching iterator and use it in two places

See merge request tpo/core/arti!1016
2023-02-13 18:59:42 +00:00
Ian Jackson 297019bb23 batching_split_before: Whitespace error fix 2023-02-13 17:34:04 +00:00
Ian Jackson b4d0b585c2 batching_split_before: Better xref for doc hidden
And another UNSTABLE warning.
2023-02-13 17:32:54 +00:00
Ian Jackson 1232bb6661 batching_split_before: Clarify API sequential restriction 2023-02-13 17:29:17 +00:00
Ian Jackson 5be6db006c batching_split_before: Fix and clarify Batches/LendingIterator doc 2023-02-13 17:28:39 +00:00
Ian Jackson 15f33ce13c batching_split_before: Introduce unit token types
And explain why we're not using booleans.
2023-02-13 17:19:13 +00:00
Ian Jackson 73a20cdc36 batching_split_before: Expand on the remaining use of "meta-iterator" 2023-02-13 17:13:55 +00:00
Ian Jackson 04e3a22b82 batching_split_before: Fix link to BatchesWithHeader::subsequent 2023-02-13 17:12:17 +00:00
Ian Jackson 53ef0d65ea batching_split_before: Provide description for BatchesWithHeader 2023-02-13 17:09:08 +00:00
Ian Jackson 6799fb6e3e batching_split_before: Document generics on our non-private types 2023-02-13 17:04:29 +00:00
Ian Jackson e44be77f52 batching_split_before: Rename "prefix" to "header"
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1016#note_2877759

"_prefixed" becomes "_with_header".  That mirrors the returned struct
which is nice.
2023-02-13 16:55:36 +00:00
Ian Jackson 72b1cb797d batching_split_before: Rename "start" to "batch-starting"
Roughly as per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1016#note_2877822
2023-02-13 16:52:07 +00:00
Ian Jackson 09060b4748 batching_split_before: Remove false assertions about batch contents 2023-02-13 16:44:47 +00:00
Ian Jackson 4333a1ef1c batching_split_before: Clarify heading of BatchesWithPrefix 2023-02-13 16:37:13 +00:00
Nick Mathewson 5c1cbd5486 batching_split_before: docs improvements. 2023-02-13 16:12:26 +00:00
Nick Mathewson 7363bee6a4 netdoc: give hsdesc test info in its original form. 2023-02-13 10:30:27 -05:00
Ian Jackson 8703a969f4 tor-netdoc: Use batching iterator in authcert.rs
This gets rid of a lot of manual boolean state variable iterator
stuff.
2023-02-10 19:48:26 +00:00
Ian Jackson d35f7990e8 tor-netdoc: Use batching_split_before in HsDescInner 2023-02-10 19:47:45 +00:00
Ian Jackson fd35f05b26 tor-netdoc: Provide new batching_split_before
There will be two call sites to demonstrate it.

Eventually maybe this will want to be in tor-basic-utils, since it
doesn't depend on any of the tor-netdoc types.  But it would be
sensible to wait until the situation with PeekableIterator and
Itertools is improved.

For now we make this #[doc(hidden)] to avoid it becoming part of our
stable API.
2023-02-10 19:47:45 +00:00
Ian Jackson 1b5a5eb57a tor-netdoc: Provide NetDocReader::into_iter
This will avoids some hairy borrowck disasters when we try to use
iterator adapters.
2023-02-10 19:47:45 +00:00
Ian Jackson 5f0ff78c2d tor-netdoc: util: use super::* in tests
Not xure how my MR to try to do this everyone missed this one.
2023-02-10 19:47:45 +00:00
Ian Jackson a5be14ed5e tor-netdoc: util: Add standard lint block to tests
Not xure how my MR to try to do this everyone missed this one.
2023-02-10 12:47:02 +00:00
Nick Mathewson 3f5502c5bc netdoc: Add a test for HSDesc decryption. 2023-02-10 07:36:40 -05:00
Nick Mathewson ced33029f6 Add another test hsdesc to parse.
This one was generated (by dgoulet) using `ClientDescEncKey`
encryption.  Its information is:

```
Address: paozpdhgz2okvc6kgbxvh2bnfsmt4xergrtcl4obkhopyvwxkpjzvoad.onion

Time period: 19397

Client:

  paozpdhgz2okvc6kgbxvh2bnfsmt4xergrtcl4obkhopyvwxkpjzvoad:descriptor:x25519:SDZNMD4RP4SCH4EYTTUZPFRZINNFWAOPPKZ6BINZAC7LREV24RBQ

Service:

  descriptor:x25519:SACGOAEODFGCYY22NYZV45ZESFPFLDGLMBWFACKEO34XGHASSAMQ
```
2023-02-10 07:26:31 -05:00
Ian Jackson 62fa652fcd Merge branch 'hsdir' into 'main'
Compute the HS directory hash ring

See merge request tpo/core/arti!1012
2023-02-09 16:35:42 +00:00
Nick Mathewson 73ce68ac21 Rename DescEncNonce => HsDescEncNonce 2023-02-09 08:57:09 -05:00
Ian Jackson f5c3c1f277 hsdir: Be able to enumerate the hidden service directories
* Provide an accessor for the HSDIR flag
* Provide a function for testing a relay for hsdir inclusion
* Provide an iterator on NetDir that returns the hsdirs
2023-02-09 13:01:37 +00:00
Ian Jackson 81d4e6971b tor-netdoc: Discuss that relays are ordered by rsa id in Consensus 2023-02-09 12:56:18 +00:00
Nick Mathewson 0a80c468df hsdesc: Use an IntegerMinutes<u16> to hold lifetime. 2023-02-08 13:33:07 -05:00
Nick Mathewson 39f5f90f3f netdoc: Fix most "TODO HS rename" comments 2023-02-08 13:23:21 -05:00
Nick Mathewson 8c3bd3626b netdoc::hsdesc: Distinguish layer from document. 2023-02-08 13:16:44 -05:00
Nick Mathewson 966ddeb092 netdoc::hsdesc: Rename "layer" modules. 2023-02-08 13:08:57 -05:00
Nick Mathewson 964ca511c4 netdoc::hsdesc: Excise reference to "password" authentication
As far as we can tell, this never existed.
2023-02-08 12:57:52 -05:00
Nick Mathewson cce56c793b hscrypto,netdoc: Add and use a type for KP_hss_desc_enc. 2023-02-08 12:50:34 -05:00
Nick Mathewson 99ec1c9818 tor-hscrypto: Rename key types to correspond to new spec names. 2023-02-08 12:27:36 -05:00
Nick Mathewson d1ee5807ed Rename DescriptorCookie to DescEncNonce
(I think it's okay to omit the Hs here, since this type is not
visible outside the hsdesc parsing code. I'll rename if others
disagree.)
2023-02-08 12:10:24 -05:00
Nick Mathewson 39e1116ece Renaming cleanup for and around now-renamed "KP_hs_desc_ephem". 2023-02-08 12:06:42 -05:00
Nick Mathewson 18e8ac7cb6 Rename key identifiers that have changed in the spec
Generated with perl:

s/K([PS])_hs_intro_tid/K$1_hs_ipt_sid/g;
s/K([PS])_onion_ntor/K$1_ntor/g;
s/K([PS])_hs_intro_ntor/K$1_hss_ntor/g;
s/K([PS])_hs_desc_ephem/K$1_hss_desc_enc/g;
2023-02-08 11:57:51 -05:00
Dimitris Apostolou d1eeffbf42
Fix typo 2023-02-08 00:04:25 +02:00
Nick Mathewson f537a7ad48 Update hsdesc parser code to new sectionrules::builder code. 2023-02-07 15:28:10 -05:00
eta c7c4fff66e Merge branch 'authcert_bug' into 'main'
Fix AuthCert behavior on unrecognized tokens, and prevent bug from recurring elsewhere.

Closes #752

See merge request tpo/core/arti!1006
2023-02-07 16:06:20 +00:00
Nick Mathewson a1c36b2087 netdoc: Add a comment about renaming layer to document. 2023-02-07 09:02:15 -05:00
Nick Mathewson 257aaf9e5b netdoc::hsdesc: rename Passwd to Password. 2023-02-07 09:02:15 -05:00
Nick Mathewson 2ba28aa289 netdoc: Rename/comment objects from hsdesc. 2023-02-07 09:02:15 -05:00
Nick Mathewson d10ccf0dc5 netdoc: Rename/comment objects from inner_layer. 2023-02-07 09:02:15 -05:00
Nick Mathewson 1cdd36f511 netdoc: Rename/comment objects from middle_layer. 2023-02-07 09:02:15 -05:00
Nick Mathewson 1a59005ae0 netdoc: Rename/comment objects from desc_enc 2023-02-07 09:02:15 -05:00
Nick Mathewson 7701e02aad netdoc: Renaming and comments in outer_layer. 2023-02-07 09:02:15 -05:00
Nick Mathewson 68d2ccf5fe netdoc: Use Itertools::exactly_once in hsdesc parsing 2023-02-07 09:02:15 -05:00
Nick Mathewson 3d53a30c94 netdoc: Remove useless should_be_exhausted calls. 2023-02-07 09:02:15 -05:00
Nick Mathewson cff2326f6f Remove a needless line. 2023-02-07 09:02:15 -05:00
Nick Mathewson fbffd3168c Even more clarifying comments. 2023-02-07 09:02:15 -05:00
Nick Mathewson 328535855a netdoc: Try to add a bunch of clarifying documentation.
In the process I found a couple of keys without identifiers in the
spec.
2023-02-07 09:02:15 -05:00
Nick Mathewson 6954c6b488 netdoc: Use Signature::from to construct ed25519 sigs. 2023-02-07 09:02:15 -05:00
Nick Mathewson 4c24f8e8a2 netdoc: Clear up a few typos in hsdesc comments and strings. 2023-02-07 09:02:15 -05:00
Nick Mathewson d699742400 netdoc: Implement onion service descriptor parsers. 2023-02-07 09:02:15 -05:00
Nick Mathewson 23757d90c1 netdoc: Parse the inner layer of an onion service descriptor.
There are some places where I note certificates which are not
currently validated, because there is no cryptographic point in
doing so.  We should either document that this is okay, or validate
the certificates anyway.

This code might benefit from refactoring to make it prettier.
2023-02-07 09:02:15 -05:00
Nick Mathewson 6817f61641 netdoc: Add a workaround for C Tor's lack of mid-layer NL
It turns out that C Tor doesn't add a newline at the end of the
middle layer of an onion service descriptor.  I've made a spec MR
(torspec!109) to document this: here, it's time to work around the
issue.
2023-02-07 09:02:15 -05:00
Nick Mathewson 4be4d178fa netdoc: Parse the middle layer of a hsdesc, and decrypt it. 2023-02-07 09:02:15 -05:00
Nick Mathewson 321c8582ce netdoc: Use correct size for descriptor encryption cookie 2023-02-07 09:02:15 -05:00
Nick Mathewson d8849f583e netdoc: move test descriptor to a higher level test module
We're going to make use of it in all of our tests, so we may as well
expose it to them from hsdesc::test.
2023-02-07 09:02:15 -05:00
Nick Mathewson 02fa682bc0 netdoc: implement onion service descryptor encryption
This is tested via a round-trip check, and via a successful
decryption of our example descriptor's outer layer.
2023-02-07 09:02:13 -05:00
Nick Mathewson 6c4e9c8f1d netdoc: Parser for outer layer of onion service descriptors. 2023-02-07 09:00:19 -05:00
Nick Mathewson d35e8a751d netdoc: Add an example onion service descriptor to test our parsing.
I generated this using C tor (latest main) and a Chutney network
about a week ago.

The subcredential is:
   78210A0D2C72BB7A0CAF606BCD938B9A3696894FDDDBC3B87D424753A7E3DF37
The HS_blind_id is:
   43CC0D62FC6252F578705CA645A46109E265290343B1137E90189744B20B3F2D
2023-02-07 09:00:19 -05:00
Nick Mathewson 0f1cfc308a netdoc: Derive Debug and Clone on HsDesc and its kin. 2023-02-07 09:00:19 -05:00
Nick Mathewson 73199ebf68 netdoc: Require that unrecognized tokens be handled explicitly.
Now we require that, for all `SectionRules`, either the caller say
how to handle unrecognized tokens (using `.add(UNRECOGNIZED...)`),
or that they explicitly reject unrecognized tokens (using
`reject_unrecognized`()`.)

This solution uses an assert!() rather than an Error to indicate
failure.  I say that's fine, since
  1. This is a crate-internal API.
  2. We never dynamically construct SectionRules according to
     different behavior: they are always prefabricated in a fixed
     code block.  Thus, if we test a parser at all, we will make
     sure that its SectionRules are well-formed.

I considered and explicitly rejected a solution where the builder
had to be finalized with separate methods `build_strict()` or
`build_tolerant()`: It's too easy IMO for the caller to forget what
these call means.

Prevents further recurrences of #752.

Closes #752.
2023-02-03 11:33:30 -05:00