Commit Graph

163 Commits

Author SHA1 Message Date
Nick Mathewson 0425de104c downgrade_dependencies: Remove "quote" exception.
This is no longer necessary. :)
2022-04-26 12:16:22 -04:00
Ian Jackson 6da7a2e3e2 Use git source for derive_builder for now, for sub_builder feature
This commitid is the current head of my MR branch
  https://github.com/colin-kiegel/rust-derive-builder/pull/253
  https://github.com/ijackson/rust-derive-builder/tree/field-builder
Using the commitid prevents surprises if that branch is updated.

We will require this newer version of derive_builder.  The version
will need to be bumped again later, assuming the upstream MR is merged
and upstream do a release containing the needed changes.

We will need the new version of not only `derive_builder_core` (the
main macro implementation) but also`derive_builder` for a new error
type.
2022-04-21 14:07:28 +01:00
Dimitris Apostolou ea283584f5
Fix typos 2022-04-11 21:21:17 +03:00
Nick Mathewson adb94f8047 Merge branch 'cargo-audit' into 'main'
fix maint/cargo_audit not propagating error and add temporary exceptions

See merge request tpo/core/arti!452
2022-04-08 15:55:05 +00:00
trinity-1686a 777eb8a5fc fix maint/cargo_audit not propagating error and add temporary exceptions 2022-04-08 16:51:35 +02:00
trinity-1686a 7c4ad15fc3 use default toolchain from rustup instead of stable for coverage
using stable doesn't work for CI because a specific version like 1.60 is installed
2022-04-08 16:19:28 +02:00
trinity-1686a 98ed22ba55 bring back RUST_COVERAGE_TOOLCHAIN 2022-04-08 15:46:46 +02:00
trinity-1686a c113c607ac use rustc 1.60 (stable) for coverage 2022-04-07 23:04:10 +02:00
trinity-1686a 80ba587937 fix with_coverage script to exclude pub(crate) mod test from results 2022-03-24 19:53:53 +01:00
Ian Jackson 8a5d98f716 downgrade_dependencies: Upgrade darling
The new version of derive_builder requires a newer darling.
Ooops.  Upstream MR here:
  https://github.com/colin-kiegel/rust-derive-builder/pull/242

For now, upgrade darling.  When there's a new upstream derive_builder,
we can use it and drop this.
2022-03-16 16:34:44 +00:00
Nick Mathewson dcd74e7a64 tests/chutney: Merge chutney-launching scripts
Formerly, the "launch a chutney network" code was duplicated in
"setup" and "arti-bench", since "setup" always launched an arti
proxy, while "arti-bench" didn't want that functionality.

Now the "setup" script launches an arti proxy conditionally,
depending on whether the "proxy" argument is given.
2022-03-10 12:29:28 -05:00
Nick Mathewson 64fa8ea77d Omit arti-testing from coverage. 2022-03-04 15:24:38 -05:00
trinity-1686a 5e5864802d fix reproducible build summary giving invalid branch and commit id
fix #378
also fix unrelated error in nightly rustdoc CI
2022-03-04 19:25:40 +01:00
Nick Mathewson e15c47f762 Reproducible build: Bump to Rust 1.59 and Alpine 3.15
The Rust upgrade is necessary since our Cargo.lock file now requires
`ed25519` 1.4.0, which requires edition2021, which requires Rust
1.56 or later.

The Alpine upgrade is opportunistic: we might as well.

I've also added comments to remind us to keep the .gitlab-ci.yml
and docker_reproducible_build files in sync, since my first version
of this commit messed that up.

Closes #376.
2022-03-04 10:09:31 -05:00
Nick Mathewson f7009e238a add_warning: remove clone_on_ref_ptr
See !352
2022-03-01 15:38:46 -05:00
Nick Mathewson c646d3adca Add nix exception to downgrade_dependencies. 2022-02-28 13:25:14 -05:00
Ian Jackson 9618f85237 maint/downgrade_dependencies: Upgrade zeroize_derive, env_logger
The *earlier* versions of these crates pull in *dependencies* that
violate our MSRV policy by requiring a *later* version of Rust.

Empirically, env_logger 0.5.4 would be enough, but practice here seems
to be to just say "cargo update".
2022-02-28 17:11:59 +00:00
Nick Mathewson 6755648f6b Merge branch 'upgrade-dependencies' into 'main'
Upgrade a few dependencies to newer versions

See merge request tpo/core/arti!357
2022-02-25 16:30:00 +00:00
Nick Mathewson d3661927da Remove lexical-core from downgrade_dependencies exceptions. 2022-02-25 10:56:34 -05:00
trinity-1686a 9a95bac10a better config for fuzz 2022-02-23 17:49:28 +01:00
trinity-1686a 6ce6e400c7 add script for coverage on fuzzing 2022-02-22 20:45:55 +01:00
trinity-1686a 74e3f26b88 use libfuzz capabilities instead of scripting 2022-02-22 20:45:55 +01:00
Ian Jackson d3ab2bb636 maint/add_warning: Remove duplicate argv0 from usage message 2022-02-18 11:46:06 +00:00
Ian Jackson 894944ba5f maint/gen_md_links: Add missing +x bit
This is a program, with a main function, etc.
2022-02-18 11:46:06 +00:00
Ian Jackson 1c301230e4 rename *.py scripts to bare name
Update all references.  There are three remaining hits for
  git-grep '\.py\b'
all of which are scripts in "little-t-tor", not here in arti.
2022-02-18 11:46:06 +00:00
Ian Jackson 9aefbf4227 rename *.sh scripts to bare name
As per #309

Update all references.  There is one remaining hit for
  git-grep '\.sh\b'
namely "build.sh" in maint/reproducible_build, which I think is
the build script for osxcross - ie, an external project.
2022-02-18 11:46:06 +00:00
Nick Mathewson 1cecc7e45a Change deny(clippy::all) to warn(clippy::all).
Closes #338.
2022-02-14 09:24:06 -05:00
Daniel Schischkin 4509f8b9d4 Remove file ending of shellcheck_all and downgrade_dependencies script 2022-02-03 18:14:57 +00:00
Dimitris Apostolou 6526321851
Fix typos 2022-02-02 20:18:22 +02:00
Nick Mathewson 81a49fb6ec Merge branch 'dedup-cobertura' into 'main'
add postprocessing script to deduplicate cobertura reports

See merge request tpo/core/arti!272
2022-01-31 17:19:01 +00:00
Nick Mathewson d3039d805d Update changed_crates to be more useful. 2022-01-31 10:30:50 -05:00
Ian Jackson 7f83b9fb30 maint/shellcheck_all.sh: add missing final newline 2022-01-31 12:44:27 +00:00
Daniel Schischkin 42fb7667b4 Change which command to command -v 2022-01-29 00:12:36 +01:00
Daniel Schischkin 21a9ad848c Add shellcheck script into maint directory 2022-01-29 00:09:06 +01:00
trinity-1686a daa1eba969 add postprocessing script to deduplicate cobertura reports 2022-01-28 20:52:50 +01:00
Daniel Schischkin 0fed438a58 Add correct file ending to maint/downgrade_dependencies 2022-01-28 15:20:54 +01:00
Nick Mathewson aa9e5515d8 with_coverage: resolve a shellcheck warning
Shellcheck doesn't like it when you do `rm -r "$A/$B"` : it worries
that the path might accidentally expand to /.

That shouldn't actually be possible in this case, but let's avoid
being the people who accidentally remove somebody's filesystem.
2022-01-27 12:19:35 -05:00
Nick Mathewson 5b6b0d6a7d Once more exclude arti-bench from coverage.
This was added in !210; looks like it got removed by mistake with
!221.
2022-01-27 12:14:32 -05:00
Nick Mathewson 40c67e5e20 with_coverage.sh: split grcov options into separate lines.
This makes it easier to see when they're changing, and will make
future patches easier to review.
2022-01-27 12:09:20 -05:00
eta 785ad605d7 Merge branch 'coverage-tracking' into 'main'
Add test coverage tracking with CI/CD

Closes #250

See merge request tpo/core/arti!221
2022-01-27 12:56:22 +00:00
Arturo Marquez 3213b89390
Add changes in `maint`
Follows up on changes from https://gitlab.torproject.org/trinity-1686a/arti/-/merge_requests/2/diffs#bc2cc73a8917fa6fed183148df69119665eb4e55
2022-01-26 22:46:45 -06:00
Nick Mathewson ef5dc3cb3e with_coverage: Allow toolchain selection.
There's some kind of bug in nightly that (for me) keeps grcov from
generating correct results. Instead, it misses some crates entirely.

No trouble, though: I can just use an older nightly until they get
this issue fixed!

This patch makes using a different version of nightly possible with
a new RUST_COVERAGE_TOOLCHAIN environment variable.
2022-01-20 09:01:36 -05:00
Nick Mathewson 24ec0bbb76 maint/changed_crates.sh: Add a missing space.
found by shellcheck.
2022-01-19 08:11:12 -05:00
Nick Mathewson 446ace637d coverage.sh: Treat no-tests-selected as an error. 2022-01-19 08:04:46 -05:00
Nick Mathewson 58db731759 Rewrite coverage.sh to use with_coverage.
`coverage.sh` is now a more elaborate script to before, thanks to
moving from tarpaulin to grcov via `with_coverage.sh`.

This commit also adds support for the minimalistic
chutney+curl-based integration test from our gitlab-ci.yml file.

See the message from "coverage.sh -h" for information about running
these tests.

Closes #248.
2022-01-13 09:48:25 -05:00
Nick Mathewson aee32b4beb Add the python script I use to generate changelog links. 2022-01-11 13:19:52 -05:00
Nick Mathewson 06ddb56541 Add more names to exclude_contributors.txt
(We don't put Tor Project employees in the acknowledgments lists.
And we probably shouldn't put bots either.)
2022-01-11 09:22:49 -05:00
Nick Mathewson c84c547bf4 Move our "minimal versions" code to a script.
Previously this code was in .gitlab-ci.yml, but for the purposes of
testing my fix for #275, it turned out to be much better to have it
in a script of its own.

Also, we now need to update our "quote" second-order dependency,
since otherwise our minimal first-order dependencies won't build
correctly.
2022-01-07 19:10:07 -05:00
Daniel Eades 592642a9e6 extend lints to include 'clippy::all' 2021-12-28 20:15:40 +00:00
Nick Mathewson 3f85e69a76 Exclude arti-bench from coverage.
This program is itself for testing; we don't strongly care how much
coverage it has on its own.
2021-12-21 12:29:40 -05:00
Nick Mathewson d582b1c439 Python does not have quadruple-quote.
Fix a bug in postprocess_coverage.py that stuck a big row of '
characters in its output.
2021-12-02 17:38:27 -05:00
Nick Mathewson 14eeff00ef Postprocess coverage index.html with python instead
This change jettisons the awk and ed dependencies and instead uses a
real HTML parser, via the BeautifulSoup library in python.

Using BeautifulSoup lets us do trickier stuff, like actually
extracting the coverage totals and adding our own table, with
per-crate coverage.

The script only does this post-processing when it finds python3; the
script exits with an error if BeautifulSoup isn't installed.
2021-12-01 12:02:53 -05:00
Nick Mathewson f0260e7475 script to tell which crates have changed since last release. 2021-11-29 15:17:51 -05:00
eta b786f6bb54 Merge branch 'coverage-ux' into 'main'
UX improvements to with_coverage script

See merge request tpo/core/arti!143
2021-11-29 16:55:30 +00:00
Daniel Eades db16d13df4 add semicolons if nothing returned 2021-11-25 13:20:37 +00:00
Nick Mathewson eac440010d with_coverage: port to work with mawk
The syntax is kinda minimal, but it seems to work okay.
2021-11-24 12:55:38 -05:00
Nick Mathewson af60e0bc46 with_coverage: detect missing llvm-tools-preview
Applies a suggestion from @trinity-1686a
2021-11-24 12:33:31 -05:00
Nick Mathewson fac2be6783 with_coverage: include command log in index.html
This is a slight kludge, but it helps a lot to keep track of what
exactly was tested for coverage.
2021-11-23 17:09:27 -05:00
Nick Mathewson b6cefc14e8 with_coverage: Add flags to accumulate data over runs (or not)
If the -c flag is provided, we _continue_ with the previous
consensus_meta data (without deleting it).

If the -s flag is provided, we _skip_ generating the final report
(since we're going to be doing more profiling.)
2021-11-23 16:48:45 -05:00
Nick Mathewson 2a3717ef73 with_coverage: Don't try to delete absent directories
If the directories are absent, then deleting them gives you a line
in stderr, even if you do "rm -f foo || true".
2021-11-23 16:41:22 -05:00
Nick Mathewson 9c040e85b4 Let with_coverage take command-line flags.
Now the shell is controlled by the presence of an -i flag, and can
be combined with a command.

This commit also adds usage messages and better reports for a couple
of kinds of setup errors.
2021-11-23 16:37:59 -05:00
Nick Mathewson 5bd841003d with_coverage: Wipe coverage_meta at the start of each run.
Also improve comments and stdout slightly...
2021-11-23 16:03:02 -05:00
Nick Mathewson 9e617fac43 Add usage string from commit message to with_coverage. 2021-11-23 09:04:59 -05:00
Trinity Pointard 06aa55cfe2 add with_coverage script
usage: with_coverage <command> [args...]: run <command> with [args...]
       with_coverage : run bash
2021-11-23 09:04:59 -05:00
Nick Mathewson 3715e0db3c Add a shell script to find binary size and download size.
It formats them in a nice little json object, which it writes to
stdout.

Part of arti#172
2021-11-09 12:11:20 -05:00
Nick Mathewson 3ff1561c25 Update reproducible_build.sh to use OSX SDK 10.12
We need this for clock_gettime_nsec_np().

The source is not canonical, but neither was the one we were using
before. To mitigate this, I've added a digest check on the SDK after
downloading it.

A 100% real canonical reproducible build process will probably
require Apple hardware anyway.

Closes #221
2021-11-04 11:23:59 -04:00
Nick Mathewson ad2b4e300a Remove alternative-name handling from exclude_contributors.txt
This doesn't actually do the right thing: it will make somebody get
ignored completely for a release if they _only_ use their
alternative name.
2021-10-29 10:39:05 -04:00
Nick Mathewson ba799d6335 thanks.sh: Remove tempdir on exit. 2021-10-29 10:38:08 -04:00
Nick Mathewson b9f23fbf98 Fix shellcheck warnings in thanks.sh 2021-10-29 10:35:37 -04:00
Nick Mathewson f399a7ecb5 Merge remote-tracking branch 'origin/mr/113' 2021-10-29 10:32:50 -04:00
Nick Mathewson a29378bf92 Add a list_crates.py script, for help with foreach scripting. 2021-10-29 10:32:16 -04:00
eta 71e87fc1ab Add an attempt at autogenerating contributor lists for releases
Disclaimer: I have no idea what I'm doing when it comes to shell
scripts.

maint/thanks.sh autogenerates a list of contributors since a revision
(that you pass it on the command line), intended to make the work of
going through the shortlog a bit less manual. It can exclude
contributors' names given in maint/exclude_contributors.txt, in order to
filter out people who work for Tor, or who commit under more than one
name.
2021-10-29 15:30:01 +01:00
Nick Mathewson b90e83781d Add a script to enforce topological sorting in workspace Cargo.toml 2021-10-29 10:27:05 -04:00
Dimitris Apostolou c26dc07fbb
Fix typos and cleanup 2021-10-25 19:58:42 +03:00
Nick Mathewson c4710ce929 Remove chrono exception from cargo_audit.sh 2021-10-24 11:33:12 -04:00
Nick Mathewson 36353aacd8 Fix some typos in comments.
Also, tell the "typos" tool to ignore Cargo.lock.
2021-10-19 10:40:32 -04:00
Nick Mathewson 5b6a862096 Upgrade to latest chrono; update cargo_audit.sh
Thanks to the chrono update, we no longer include an
obsolete/vulnerable version of the `time` crate.  Unfortunately, it
turns out that chrono has the same trouble as `time`: it, too, looks
at the environment via localtime_r, and the environment isn't
threadsafe.

One step forward, one step back.  At least the underlying issue is
one that lots of people seem to care about; let's hope they come up
with a solution.
2021-10-19 08:03:41 -04:00
Nick Mathewson a1758644a6 Add in first cargo-audit exception, as an example. 2021-10-17 11:11:54 -04:00
Nick Mathewson 342dec3872 Add an ignored cargo-audit warning, with appropriate documentation 2021-10-17 11:02:11 -04:00
Nick Mathewson 290fe83671 Move "cargo audit" invocation into a script. 2021-10-17 10:41:52 -04:00
Nick Mathewson af7c9d5a0b enable checked_conversions lint. 2021-10-09 16:53:13 -04:00
Nick Mathewson a5bfc3c963 Remove some planned items in add_warnings. 2021-10-09 16:52:54 -04:00
Nick Mathewson 4a7c563b79 Fix some typos (via the "typos" tool) 2021-10-08 09:42:54 -04:00
Trinity Pointard 1a157f5d60 refactor a bit reproducible_build.sh 2021-09-20 17:48:13 +02:00
Trinity Pointard 78b6a36671 add reproducible builds for MacOS 2021-09-20 17:48:12 +02:00
Nick Mathewson 8bf394b0d5 Move version-controlled Cargo.lock to /.
Previously I'd hoped to have it only apply to reproducible builds,
but the work of keeping it up-to-date and checking it for drift
seems to have been tripping us up.  Let's try doing it the other way
for now, and we'll see how that goes.
2021-09-20 11:37:11 -04:00
Nick Mathewson 1c97918d67 Typo fixes 2021-09-09 13:06:05 -04:00
Trinity Pointard b2f4b3d38e use /dev/shm for reproducible builds 2021-08-30 21:58:38 +02:00
Trinity Pointard 33474537ab make reproducible Windows build
Linux hash   : c024778da162cda77869c46d162729d012a7758bee973bc83fa78c964d080903
Windows hash : e97e171740b889b7e3722b79c3763177174baac7fa34a5c3615c999298e1bc48
2021-08-29 13:48:33 +02:00
Nick Mathewson 557a0ff40b Move all crates into a `crates` subdirectory.
This will cause some pain for now, but now is really the best time
to do this kind of thing.
2021-08-27 09:53:09 -04:00
Nick Mathewson 28cc12be25 Merge remote-tracking branch 'origin/mr/67' into unwrap_used 2021-08-27 08:36:59 -04:00
Nick Mathewson aa912a7656 Updates for reproducible build docs 2021-08-26 13:22:41 -04:00
Trinity Pointard 46f30aad4d address review comments
sha256: 7f190568fa05f71ad356dc56a7a9725a4be18e5e40d6880f469e22981a18edd7
2021-08-26 08:38:45 +02:00
S0AndS0 b95513c5e6 Move "use_unwrap" lint from some to want list
And convert some `add_or_port` calls to use `SocketAddr` instead of
parse + unwrap on strings.

> Note, this last change is only for IPv4 addresses, for now, and only
> within tests.
2021-08-25 12:23:57 -07:00
Trinity Pointard eb0522fe63 fix segfault on static bin
see https://github.com/rusqlite/rusqlite/issues/914

sha256: 684ebc4b8c270fc63beba185f6c54ceeb98734f13aa7aeca9b64acb33432a21c
2021-08-25 20:37:58 +02:00
Trinity Pointard 540302851e make arti availlable as build artifact
and make builds reproducible
sha256sum: f141c54929a43a31b9ed6b529f6f863aace87f7406818b2f8ffe2b7a5e2803fb
2021-08-25 18:16:45 +02:00
Nick Mathewson ffc27999ef add_warning.py: explain what to do when no args are given 2021-08-18 08:56:44 -04:00
Nick Mathewson 0b6f638517 Add clippy::missing_panics_doc to add_warnings.py. 2021-08-18 08:53:58 -04:00
S0AndS0 484cc24856 Add the "implicit_clone" lint.
> Check `implicit_clone` section of Clippy documentation for details;
>
>   https://rust-lang.github.io/rust-clippy/master/index.html#implicit_clone

This adds, and addresses, the following Clippy configuration to crates;

    #![deny(clippy::implicit_clone)]

And moves related line within `maint/add_warning.py` file. My intent is to
mitigate extra edits after merging, so please let me know if I need to do this
last bit differently.
2021-08-17 11:49:39 -07:00
Nick Mathewson 9c6dd27ca1 Use add_warnings.py for clippy::cast_lossless 2021-08-13 11:36:14 -04:00