Commit Graph

213 Commits

Author SHA1 Message Date
trinity-1686a ef162655af switch from awk to python 2022-08-24 18:22:41 +02:00
trinity-1686a 426a59b2ba add feature annotation not added by doc_auto_cfg 2022-08-24 18:22:41 +02:00
Ian Jackson eb93e05552 maint/cargo_audit: Tolerate RUSTSEC-2021-0139 (ansi_term unmaintained) 2022-08-22 10:37:00 +01:00
Ian Jackson ff962e5109 maint/cargo_audit: Honour $CARGO
This makes it easier to for me to test this script.  (I would like to
do this to all of them but it's easier on a case-by-case basis.)
2022-08-22 10:36:53 +01:00
FAMASoon 4cfdeca0f1 format python script in maint 2022-08-17 02:15:35 +09:00
trinity-1686a 2339f637c5 fix error running check_licenses
--version was removed in a recent update

we use that to detected whether cargo-license is installed,

now use --help instead
2022-08-13 09:02:17 +02:00
FAMASoon 817b811560 Delete unused Enum and fileinput in maint file 2022-08-04 22:58:58 +09:00
Nick Mathewson dfdcc97858 Several typo fixes from `typos`. 2022-07-27 10:12:32 -04:00
Nick Mathewson 275c98940a Teach check_licenses to accept license on `unicode-ident`.
The license there is `(MIT OR Apache-2.0) AND Unicode-DFS-2016`,
which fine, but the existing shell script doesn't actually handle
"AND" correctly.  This commit adds a workaround for licenses that
are "AND", and some comments about weaknesses in our (lack of)
boolean expression parsing.

This should fix CI.
2022-07-20 09:40:12 -04:00
eta c2e2da5dc3 Update `rsa` dependency (and use `x25519-dalek` prerelease)
- arti#448 and arti!607 highlight an issue with upgrading `rsa`: namely,
  the `x25519-dalek` version previously used has a hard dependency on
  `zeroize` 1.3, which creates a dependency conflict.
- However, `x25519-dalek` version `2.0.0-pre.1` relaxes this dependency.
  Reviewing the changelogs, it doesn't look like that version is
  substantially different from the current one at all, so it should be
  safe to use despite the "prerelease" tag.
- The new `x25519-dalek` version also bumps `rand_core`, which means we
  don't have to use the RNG compat wrapper in `tor-llcrypto` as much.

closes arti#448
2022-07-06 14:57:45 +01:00
Ian Jackson b7c03943f1 maint/add_warning: Fix parsing of --check
Previously this would eat the first filename!
2022-06-24 14:23:38 +01:00
Ian Jackson 23c352b96b clippy: Consolidate many lints in maint/add_warning
Found these by disabling the nightly dbg macro special case.  Now, we
have a mechanism for globally adding suppressions to tests, we can use
that instead.
2022-06-24 14:23:38 +01:00
Ian Jackson bebf013bef maint/add_warning: Add two lints via add_warning as a test case 2022-06-24 14:23:38 +01:00
Ian Jackson 72515abf91 maint/add_warning: Add ability maintain test lint blocks 2022-06-24 14:23:38 +01:00
Ian Jackson 76ff330610 maint/add_warning: Scan all files by default, but insist only in some
This will allow us to have add_warning manage test lint blocks.

We have to stop printing all the filenames because there are too
many.  Filenames still come out on error of course.
2022-06-24 14:23:34 +01:00
Ian Jackson 13e27919c5 clippy: Supresss clippy::significant_drop_in_scrutinee
As per the linked discussion.
2022-06-23 19:14:54 +01:00
Ian Jackson 28dea73523 lints: Scheme for allowing us to name nightly lints
As per
  https://gitlab.torproject.org/tpo/core/arti/-/issues/501
2022-06-23 19:14:11 +01:00
Nick Mathewson bf15abbb97 Avoid matching crate names by prefix. 2022-06-17 08:41:38 -04:00
Nick Mathewson 677f1b8af0 Update check_tree to better detect errors in cargo-tree. 2022-06-16 16:13:11 -04:00
Nick Mathewson c16b32f1d4 Add script and CI to make sure `ring` doesn't show up in arti/full
See arti#493.
2022-06-16 09:00:59 -04:00
Ian Jackson 86c4f42c49 CI: Break out maint/via-cargo-install 2022-06-08 19:32:59 +01:00
Ian Jackson d00ba995be maint/cargo_sort: Tolerate toplevel Cargo.toml not being sorted 2022-06-08 18:27:45 +01:00
Ian Jackson ba016053de maint/toposort: Make shebang be python3, not python
Otherwise it doesn't work on sane systems where /usr/bin/python is
never an incompatible version from previously.
2022-06-08 17:17:20 +01:00
Ian Jackson 6f96736278 Merge branch 'check-license-fixes' into 'main'
Update check_licenses and add it to CI.

Closes #462

See merge request tpo/core/arti!559
2022-06-02 13:59:35 +00:00
Ian Jackson 2730bed5bd maint/add_warning: Convert remaining asserts 2022-05-31 19:16:09 +01:00
Ian Jackson 6c52fa675b maint/add_warning: Introduce ImproperFile and convert one assert 2022-05-31 19:14:19 +01:00
Ian Jackson 7e96664140 maint/add_warning: Use python3 argparse 2022-05-31 19:03:38 +01:00
Nick Mathewson 842e6a697d check_licenses: list new crates with manual license verification
Closes #462.

Note that the license on `ring` is slightly problematic for some
users, including as it does old the openssl license[^1], with
advertising clause and all.  That's not a blocker for us now, since
`ring` is not a required dependency.  But we wouldn't want `ring` to
become a mandatory dependency because of this.

[^1]: To make the situation even more complicated, modern openssl
      has relicensed under apache-2.0, but that doesn't necessarily
      help us, since ring took its code from boringssll, which
      forked from an older version of openssl.
2022-05-31 13:58:40 -04:00
Ian Jackson 47a10fa84c maint/add_warning: Add "how to fix" to error message
As per
  https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/557#note_2808879
2022-05-31 18:54:08 +01:00
Ian Jackson 89b4a59048 maint/add_warning: Provide --check mode 2022-05-31 18:54:08 +01:00
Ian Jackson c2939b5f74 maint/add_warning: Change shebang to python3
"python" sometimes means Python 2.
2022-05-31 18:53:49 +01:00
Ian Jackson 00f82a7f4d maint/add_warning: Use .tmp~ for temp filename, not .bak
"bak" ought to be a backup file, with the old contents, which this
most definitely isn't.
2022-05-31 18:53:47 +01:00
Ian Jackson 2814fd99c1 lints: Add let_unit_value allow to add_warning
add_warning can now add allows.  This one was agreed in some other MR
for one crate.  Allow it everywhere.
2022-05-31 14:52:09 +01:00
Ian Jackson a8f3423107 lints: Add two missing ones to add_warning
These were in some crates, but not all.  Put them in add_warning
where they will propagate everywhere.
2022-05-31 14:52:09 +01:00
Ian Jackson 47a209fb31 lints: maint/add_warning: New marker-based mode
We don't process the in-script lint block, just paste it in, now.
This is less complicated.

This reverts/replaces
  "maint/add_warnings: Make it add the markers for its future self"
2022-05-31 13:00:31 +01:00
Ian Jackson 3ae7244f91 maint/add_warnings: Make it add the markers for its future self
We'll run this in a moment.  Then we'll revert it.
2022-05-31 13:00:31 +01:00
Ian Jackson ea5caf9ac2 lints: maint/add_warning: Improve ordering
The crates we have aren't consistent.  But changing this means the
deltas resulting from the script are smaller.
2022-05-31 13:00:31 +01:00
Ian Jackson a8331c588e maint/add_warnings: Tidy up, and add some commentary
This puts the actually-used lint list at the top.

No functional change.
2022-05-31 13:00:31 +01:00
Orhun Parmaksız bfd41ddb5f
Lexically sort Cargo.toml dependencies
Utilize cargo-sort: https://github.com/DevinR528/cargo-sort

Signed-off-by: Orhun Parmaksız <orhunparmaksiz@gmail.com>
2022-05-28 20:05:51 +03:00
Steven Murdoch 235fd78602 Fix "coverage -h" command
$SCRIPT_NAME is not defined so causes an error. Use $(basename "$0") instead.
2022-05-26 15:12:36 +00:00
trinity-1686a 545313a899 fix docker_reproducible_build not passing its parameters properly 2022-05-26 09:44:06 +02:00
trinity-1686a 0f14847023 update comments in coverage_fuzz_corpora 2022-05-25 22:29:04 +02:00
trinity-1686a 8eb843f2af make almost all scripts use bash and set euo pipefail 2022-05-25 22:25:58 +02:00
eta 6fc7a864c7 Remove cargo-husky, and replace with manual instructions
A build script reaching into your .git/hooks/ and modifying them
nonconsensually was a bit of a horrifying concept, and also made it hard
to build arti with the feature disabled. Remove this crate, and replace
it with manual instructions on how to install the hooks in
CONTRIBUTING.md.
2022-05-10 13:36:49 +01:00
trinity-1686a 3464cae490 use true instead of /bin/true
see https://gitlab.torproject.org/tpo/core/arti/-/issues/400#note_2800763
2022-05-05 21:05:42 +02:00
trinity-1686a 5bd38ab6dc generate aggregated coverage reports 2022-05-05 18:35:34 +02:00
eta 78ca87314c Merge branch 'explain_downgrade' into 'main'
Add a comment explaining what downgrade_dependencies is for.

See merge request tpo/core/arti!480
2022-05-03 10:57:48 +00:00
trinity-1686a 0fea8535df remove some unecessary crates from downgrade_dependencies
nix:0.4.2 is no longer a dependency
synstructure:0.12.0 is no longer required to build
2022-04-28 20:50:50 +02:00
Nick Mathewson 6fecbc721a Add a comment explaining what downgrade_dependencies is for. 2022-04-28 14:23:50 -04:00
trinity-1686a d1bd9e209f update tls-api to 0.8.0
and remove corresponding RUSTSEC from ignored list
2022-04-28 17:43:32 +02:00
Nick Mathewson 0425de104c downgrade_dependencies: Remove "quote" exception.
This is no longer necessary. :)
2022-04-26 12:16:22 -04:00
Ian Jackson 6da7a2e3e2 Use git source for derive_builder for now, for sub_builder feature
This commitid is the current head of my MR branch
  https://github.com/colin-kiegel/rust-derive-builder/pull/253
  https://github.com/ijackson/rust-derive-builder/tree/field-builder
Using the commitid prevents surprises if that branch is updated.

We will require this newer version of derive_builder.  The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.

We will need the new version of not only `derive_builder_core` (the
main macro implementation) but also`derive_builder` for a new error
type.
2022-04-21 14:07:28 +01:00
Dimitris Apostolou ea283584f5
Fix typos 2022-04-11 21:21:17 +03:00
Nick Mathewson adb94f8047 Merge branch 'cargo-audit' into 'main'
fix maint/cargo_audit not propagating error and add temporary exceptions

See merge request tpo/core/arti!452
2022-04-08 15:55:05 +00:00
trinity-1686a 777eb8a5fc fix maint/cargo_audit not propagating error and add temporary exceptions 2022-04-08 16:51:35 +02:00
trinity-1686a 7c4ad15fc3 use default toolchain from rustup instead of stable for coverage
using stable doesn't work for CI because a specific version like 1.60 is installed
2022-04-08 16:19:28 +02:00
trinity-1686a 98ed22ba55 bring back RUST_COVERAGE_TOOLCHAIN 2022-04-08 15:46:46 +02:00
trinity-1686a c113c607ac use rustc 1.60 (stable) for coverage 2022-04-07 23:04:10 +02:00
trinity-1686a 80ba587937 fix with_coverage script to exclude pub(crate) mod test from results 2022-03-24 19:53:53 +01:00
Ian Jackson 8a5d98f716 downgrade_dependencies: Upgrade darling
The new version of derive_builder requires a newer darling.
Ooops.  Upstream MR here:
  https://github.com/colin-kiegel/rust-derive-builder/pull/242

For now, upgrade darling.  When there's a new upstream derive_builder,
we can use it and drop this.
2022-03-16 16:34:44 +00:00
Nick Mathewson dcd74e7a64 tests/chutney: Merge chutney-launching scripts
Formerly, the "launch a chutney network" code was duplicated in
"setup" and "arti-bench", since "setup" always launched an arti
proxy, while "arti-bench" didn't want that functionality.

Now the "setup" script launches an arti proxy conditionally,
depending on whether the "proxy" argument is given.
2022-03-10 12:29:28 -05:00
Nick Mathewson 64fa8ea77d Omit arti-testing from coverage. 2022-03-04 15:24:38 -05:00
trinity-1686a 5e5864802d fix reproducible build summary giving invalid branch and commit id
fix #378
also fix unrelated error in nightly rustdoc CI
2022-03-04 19:25:40 +01:00
Nick Mathewson e15c47f762 Reproducible build: Bump to Rust 1.59 and Alpine 3.15
The Rust upgrade is necessary since our Cargo.lock file now requires
`ed25519` 1.4.0, which requires edition2021, which requires Rust
1.56 or later.

The Alpine upgrade is opportunistic: we might as well.

I've also added comments to remind us to keep the .gitlab-ci.yml
and docker_reproducible_build files in sync, since my first version
of this commit messed that up.

Closes #376.
2022-03-04 10:09:31 -05:00
Nick Mathewson f7009e238a add_warning: remove clone_on_ref_ptr
See !352
2022-03-01 15:38:46 -05:00
Nick Mathewson c646d3adca Add nix exception to downgrade_dependencies. 2022-02-28 13:25:14 -05:00
Ian Jackson 9618f85237 maint/downgrade_dependencies: Upgrade zeroize_derive, env_logger
The *earlier* versions of these crates pull in *dependencies* that
violate our MSRV policy by requiring a *later* version of Rust.

Empirically, env_logger 0.5.4 would be enough, but practice here seems
to be to just say "cargo update".
2022-02-28 17:11:59 +00:00
Nick Mathewson 6755648f6b Merge branch 'upgrade-dependencies' into 'main'
Upgrade a few dependencies to newer versions

See merge request tpo/core/arti!357
2022-02-25 16:30:00 +00:00
Nick Mathewson d3661927da Remove lexical-core from downgrade_dependencies exceptions. 2022-02-25 10:56:34 -05:00
trinity-1686a 9a95bac10a better config for fuzz 2022-02-23 17:49:28 +01:00
trinity-1686a 6ce6e400c7 add script for coverage on fuzzing 2022-02-22 20:45:55 +01:00
trinity-1686a 74e3f26b88 use libfuzz capabilities instead of scripting 2022-02-22 20:45:55 +01:00
Ian Jackson d3ab2bb636 maint/add_warning: Remove duplicate argv0 from usage message 2022-02-18 11:46:06 +00:00
Ian Jackson 894944ba5f maint/gen_md_links: Add missing +x bit
This is a program, with a main function, etc.
2022-02-18 11:46:06 +00:00
Ian Jackson 1c301230e4 rename *.py scripts to bare name
Update all references.  There are three remaining hits for
  git-grep '\.py\b'
all of which are scripts in "little-t-tor", not here in arti.
2022-02-18 11:46:06 +00:00
Ian Jackson 9aefbf4227 rename *.sh scripts to bare name
As per #309

Update all references.  There is one remaining hit for
  git-grep '\.sh\b'
namely "build.sh" in maint/reproducible_build, which I think is
the build script for osxcross - ie, an external project.
2022-02-18 11:46:06 +00:00
Nick Mathewson 1cecc7e45a Change deny(clippy::all) to warn(clippy::all).
Closes #338.
2022-02-14 09:24:06 -05:00
Daniel Schischkin 4509f8b9d4 Remove file ending of shellcheck_all and downgrade_dependencies script 2022-02-03 18:14:57 +00:00
Dimitris Apostolou 6526321851
Fix typos 2022-02-02 20:18:22 +02:00
Nick Mathewson 81a49fb6ec Merge branch 'dedup-cobertura' into 'main'
add postprocessing script to deduplicate cobertura reports

See merge request tpo/core/arti!272
2022-01-31 17:19:01 +00:00
Nick Mathewson d3039d805d Update changed_crates to be more useful. 2022-01-31 10:30:50 -05:00
Ian Jackson 7f83b9fb30 maint/shellcheck_all.sh: add missing final newline 2022-01-31 12:44:27 +00:00
Daniel Schischkin 42fb7667b4 Change which command to command -v 2022-01-29 00:12:36 +01:00
Daniel Schischkin 21a9ad848c Add shellcheck script into maint directory 2022-01-29 00:09:06 +01:00
trinity-1686a daa1eba969 add postprocessing script to deduplicate cobertura reports 2022-01-28 20:52:50 +01:00
Daniel Schischkin 0fed438a58 Add correct file ending to maint/downgrade_dependencies 2022-01-28 15:20:54 +01:00
Nick Mathewson aa9e5515d8 with_coverage: resolve a shellcheck warning
Shellcheck doesn't like it when you do `rm -r "$A/$B"` : it worries
that the path might accidentally expand to /.

That shouldn't actually be possible in this case, but let's avoid
being the people who accidentally remove somebody's filesystem.
2022-01-27 12:19:35 -05:00
Nick Mathewson 5b6b0d6a7d Once more exclude arti-bench from coverage.
This was added in !210; looks like it got removed by mistake with
!221.
2022-01-27 12:14:32 -05:00
Nick Mathewson 40c67e5e20 with_coverage.sh: split grcov options into separate lines.
This makes it easier to see when they're changing, and will make
future patches easier to review.
2022-01-27 12:09:20 -05:00
eta 785ad605d7 Merge branch 'coverage-tracking' into 'main'
Add test coverage tracking with CI/CD

Closes #250

See merge request tpo/core/arti!221
2022-01-27 12:56:22 +00:00
Arturo Marquez 3213b89390
Add changes in `maint`
Follows up on changes from https://gitlab.torproject.org/trinity-1686a/arti/-/merge_requests/2/diffs#bc2cc73a8917fa6fed183148df69119665eb4e55
2022-01-26 22:46:45 -06:00
Nick Mathewson ef5dc3cb3e with_coverage: Allow toolchain selection.
There's some kind of bug in nightly that (for me) keeps grcov from
generating correct results. Instead, it misses some crates entirely.

No trouble, though: I can just use an older nightly until they get
this issue fixed!

This patch makes using a different version of nightly possible with
a new RUST_COVERAGE_TOOLCHAIN environment variable.
2022-01-20 09:01:36 -05:00
Nick Mathewson 24ec0bbb76 maint/changed_crates.sh: Add a missing space.
found by shellcheck.
2022-01-19 08:11:12 -05:00
Nick Mathewson 446ace637d coverage.sh: Treat no-tests-selected as an error. 2022-01-19 08:04:46 -05:00
Nick Mathewson 58db731759 Rewrite coverage.sh to use with_coverage.
`coverage.sh` is now a more elaborate script to before, thanks to
moving from tarpaulin to grcov via `with_coverage.sh`.

This commit also adds support for the minimalistic
chutney+curl-based integration test from our gitlab-ci.yml file.

See the message from "coverage.sh -h" for information about running
these tests.

Closes #248.
2022-01-13 09:48:25 -05:00
Nick Mathewson aee32b4beb Add the python script I use to generate changelog links. 2022-01-11 13:19:52 -05:00
Nick Mathewson 06ddb56541 Add more names to exclude_contributors.txt
(We don't put Tor Project employees in the acknowledgments lists.
And we probably shouldn't put bots either.)
2022-01-11 09:22:49 -05:00
Nick Mathewson c84c547bf4 Move our "minimal versions" code to a script.
Previously this code was in .gitlab-ci.yml, but for the purposes of
testing my fix for #275, it turned out to be much better to have it
in a script of its own.

Also, we now need to update our "quote" second-order dependency,
since otherwise our minimal first-order dependencies won't build
correctly.
2022-01-07 19:10:07 -05:00
Daniel Eades 592642a9e6 extend lints to include 'clippy::all' 2021-12-28 20:15:40 +00:00
Nick Mathewson 3f85e69a76 Exclude arti-bench from coverage.
This program is itself for testing; we don't strongly care how much
coverage it has on its own.
2021-12-21 12:29:40 -05:00