Commit Graph

283 Commits

Author SHA1 Message Date
Nick Mathewson d3039d805d Update changed_crates to be more useful. 2022-01-31 10:30:50 -05:00
Ian Jackson 7f83b9fb30 maint/shellcheck_all.sh: add missing final newline 2022-01-31 12:44:27 +00:00
Daniel Schischkin 42fb7667b4 Change which command to command -v 2022-01-29 00:12:36 +01:00
Daniel Schischkin 21a9ad848c Add shellcheck script into maint directory 2022-01-29 00:09:06 +01:00
trinity-1686a daa1eba969 add postprocessing script to deduplicate cobertura reports 2022-01-28 20:52:50 +01:00
Daniel Schischkin 0fed438a58 Add correct file ending to maint/downgrade_dependencies 2022-01-28 15:20:54 +01:00
Nick Mathewson aa9e5515d8 with_coverage: resolve a shellcheck warning
Shellcheck doesn't like it when you do `rm -r "$A/$B"` : it worries
that the path might accidentally expand to /.

That shouldn't actually be possible in this case, but let's avoid
being the people who accidentally remove somebody's filesystem.
2022-01-27 12:19:35 -05:00
Nick Mathewson 5b6b0d6a7d Once more exclude arti-bench from coverage.
This was added in !210; looks like it got removed by mistake with
!221.
2022-01-27 12:14:32 -05:00
Nick Mathewson 40c67e5e20 with_coverage.sh: split grcov options into separate lines.
This makes it easier to see when they're changing, and will make
future patches easier to review.
2022-01-27 12:09:20 -05:00
eta 785ad605d7 Merge branch 'coverage-tracking' into 'main'
Add test coverage tracking with CI/CD

Closes #250

See merge request tpo/core/arti!221
2022-01-27 12:56:22 +00:00
Arturo Marquez 3213b89390
Add changes in `maint`
Follows up on changes from https://gitlab.torproject.org/trinity-1686a/arti/-/merge_requests/2/diffs#bc2cc73a8917fa6fed183148df69119665eb4e55
2022-01-26 22:46:45 -06:00
Nick Mathewson ef5dc3cb3e with_coverage: Allow toolchain selection.
There's some kind of bug in nightly that (for me) keeps grcov from
generating correct results. Instead, it misses some crates entirely.

No trouble, though: I can just use an older nightly until they get
this issue fixed!

This patch makes using a different version of nightly possible with
a new RUST_COVERAGE_TOOLCHAIN environment variable.
2022-01-20 09:01:36 -05:00
Nick Mathewson 24ec0bbb76 maint/changed_crates.sh: Add a missing space.
found by shellcheck.
2022-01-19 08:11:12 -05:00
Nick Mathewson 446ace637d coverage.sh: Treat no-tests-selected as an error. 2022-01-19 08:04:46 -05:00
Nick Mathewson 58db731759 Rewrite coverage.sh to use with_coverage.
`coverage.sh` is now a more elaborate script to before, thanks to
moving from tarpaulin to grcov via `with_coverage.sh`.

This commit also adds support for the minimalistic
chutney+curl-based integration test from our gitlab-ci.yml file.

See the message from "coverage.sh -h" for information about running
these tests.

Closes #248.
2022-01-13 09:48:25 -05:00
Nick Mathewson aee32b4beb Add the python script I use to generate changelog links. 2022-01-11 13:19:52 -05:00
Nick Mathewson 06ddb56541 Add more names to exclude_contributors.txt
(We don't put Tor Project employees in the acknowledgments lists.
And we probably shouldn't put bots either.)
2022-01-11 09:22:49 -05:00
Nick Mathewson c84c547bf4 Move our "minimal versions" code to a script.
Previously this code was in .gitlab-ci.yml, but for the purposes of
testing my fix for #275, it turned out to be much better to have it
in a script of its own.

Also, we now need to update our "quote" second-order dependency,
since otherwise our minimal first-order dependencies won't build
correctly.
2022-01-07 19:10:07 -05:00
Daniel Eades 592642a9e6 extend lints to include 'clippy::all' 2021-12-28 20:15:40 +00:00
Nick Mathewson 3f85e69a76 Exclude arti-bench from coverage.
This program is itself for testing; we don't strongly care how much
coverage it has on its own.
2021-12-21 12:29:40 -05:00
Nick Mathewson d582b1c439 Python does not have quadruple-quote.
Fix a bug in postprocess_coverage.py that stuck a big row of '
characters in its output.
2021-12-02 17:38:27 -05:00
Nick Mathewson 14eeff00ef Postprocess coverage index.html with python instead
This change jettisons the awk and ed dependencies and instead uses a
real HTML parser, via the BeautifulSoup library in python.

Using BeautifulSoup lets us do trickier stuff, like actually
extracting the coverage totals and adding our own table, with
per-crate coverage.

The script only does this post-processing when it finds python3; the
script exits with an error if BeautifulSoup isn't installed.
2021-12-01 12:02:53 -05:00
Nick Mathewson f0260e7475 script to tell which crates have changed since last release. 2021-11-29 15:17:51 -05:00
eta b786f6bb54 Merge branch 'coverage-ux' into 'main'
UX improvements to with_coverage script

See merge request tpo/core/arti!143
2021-11-29 16:55:30 +00:00
Daniel Eades db16d13df4 add semicolons if nothing returned 2021-11-25 13:20:37 +00:00
Nick Mathewson eac440010d with_coverage: port to work with mawk
The syntax is kinda minimal, but it seems to work okay.
2021-11-24 12:55:38 -05:00
Nick Mathewson af60e0bc46 with_coverage: detect missing llvm-tools-preview
Applies a suggestion from @trinity-1686a
2021-11-24 12:33:31 -05:00
Nick Mathewson fac2be6783 with_coverage: include command log in index.html
This is a slight kludge, but it helps a lot to keep track of what
exactly was tested for coverage.
2021-11-23 17:09:27 -05:00
Nick Mathewson b6cefc14e8 with_coverage: Add flags to accumulate data over runs (or not)
If the -c flag is provided, we _continue_ with the previous
consensus_meta data (without deleting it).

If the -s flag is provided, we _skip_ generating the final report
(since we're going to be doing more profiling.)
2021-11-23 16:48:45 -05:00
Nick Mathewson 2a3717ef73 with_coverage: Don't try to delete absent directories
If the directories are absent, then deleting them gives you a line
in stderr, even if you do "rm -f foo || true".
2021-11-23 16:41:22 -05:00
Nick Mathewson 9c040e85b4 Let with_coverage take command-line flags.
Now the shell is controlled by the presence of an -i flag, and can
be combined with a command.

This commit also adds usage messages and better reports for a couple
of kinds of setup errors.
2021-11-23 16:37:59 -05:00
Nick Mathewson 5bd841003d with_coverage: Wipe coverage_meta at the start of each run.
Also improve comments and stdout slightly...
2021-11-23 16:03:02 -05:00
Nick Mathewson 9e617fac43 Add usage string from commit message to with_coverage. 2021-11-23 09:04:59 -05:00
Trinity Pointard 06aa55cfe2 add with_coverage script
usage: with_coverage <command> [args...]: run <command> with [args...]
       with_coverage : run bash
2021-11-23 09:04:59 -05:00
Nick Mathewson 3715e0db3c Add a shell script to find binary size and download size.
It formats them in a nice little json object, which it writes to
stdout.

Part of arti#172
2021-11-09 12:11:20 -05:00
Nick Mathewson 3ff1561c25 Update reproducible_build.sh to use OSX SDK 10.12
We need this for clock_gettime_nsec_np().

The source is not canonical, but neither was the one we were using
before. To mitigate this, I've added a digest check on the SDK after
downloading it.

A 100% real canonical reproducible build process will probably
require Apple hardware anyway.

Closes #221
2021-11-04 11:23:59 -04:00
Nick Mathewson ad2b4e300a Remove alternative-name handling from exclude_contributors.txt
This doesn't actually do the right thing: it will make somebody get
ignored completely for a release if they _only_ use their
alternative name.
2021-10-29 10:39:05 -04:00
Nick Mathewson ba799d6335 thanks.sh: Remove tempdir on exit. 2021-10-29 10:38:08 -04:00
Nick Mathewson b9f23fbf98 Fix shellcheck warnings in thanks.sh 2021-10-29 10:35:37 -04:00
Nick Mathewson f399a7ecb5 Merge remote-tracking branch 'origin/mr/113' 2021-10-29 10:32:50 -04:00
Nick Mathewson a29378bf92 Add a list_crates.py script, for help with foreach scripting. 2021-10-29 10:32:16 -04:00
eta 71e87fc1ab Add an attempt at autogenerating contributor lists for releases
Disclaimer: I have no idea what I'm doing when it comes to shell
scripts.

maint/thanks.sh autogenerates a list of contributors since a revision
(that you pass it on the command line), intended to make the work of
going through the shortlog a bit less manual. It can exclude
contributors' names given in maint/exclude_contributors.txt, in order to
filter out people who work for Tor, or who commit under more than one
name.
2021-10-29 15:30:01 +01:00
Nick Mathewson b90e83781d Add a script to enforce topological sorting in workspace Cargo.toml 2021-10-29 10:27:05 -04:00
Dimitris Apostolou c26dc07fbb
Fix typos and cleanup 2021-10-25 19:58:42 +03:00
Nick Mathewson c4710ce929 Remove chrono exception from cargo_audit.sh 2021-10-24 11:33:12 -04:00
Nick Mathewson 36353aacd8 Fix some typos in comments.
Also, tell the "typos" tool to ignore Cargo.lock.
2021-10-19 10:40:32 -04:00
Nick Mathewson 5b6a862096 Upgrade to latest chrono; update cargo_audit.sh
Thanks to the chrono update, we no longer include an
obsolete/vulnerable version of the `time` crate.  Unfortunately, it
turns out that chrono has the same trouble as `time`: it, too, looks
at the environment via localtime_r, and the environment isn't
threadsafe.

One step forward, one step back.  At least the underlying issue is
one that lots of people seem to care about; let's hope they come up
with a solution.
2021-10-19 08:03:41 -04:00
Nick Mathewson a1758644a6 Add in first cargo-audit exception, as an example. 2021-10-17 11:11:54 -04:00
Nick Mathewson 342dec3872 Add an ignored cargo-audit warning, with appropriate documentation 2021-10-17 11:02:11 -04:00
Nick Mathewson 290fe83671 Move "cargo audit" invocation into a script. 2021-10-17 10:41:52 -04:00
Nick Mathewson af7c9d5a0b enable checked_conversions lint. 2021-10-09 16:53:13 -04:00
Nick Mathewson a5bfc3c963 Remove some planned items in add_warnings. 2021-10-09 16:52:54 -04:00
Nick Mathewson 4a7c563b79 Fix some typos (via the "typos" tool) 2021-10-08 09:42:54 -04:00
Trinity Pointard 1a157f5d60 refactor a bit reproducible_build.sh 2021-09-20 17:48:13 +02:00
Trinity Pointard 78b6a36671 add reproducible builds for MacOS 2021-09-20 17:48:12 +02:00
Nick Mathewson 8bf394b0d5 Move version-controlled Cargo.lock to /.
Previously I'd hoped to have it only apply to reproducible builds,
but the work of keeping it up-to-date and checking it for drift
seems to have been tripping us up.  Let's try doing it the other way
for now, and we'll see how that goes.
2021-09-20 11:37:11 -04:00
Nick Mathewson 1c97918d67 Typo fixes 2021-09-09 13:06:05 -04:00
Trinity Pointard b2f4b3d38e use /dev/shm for reproducible builds 2021-08-30 21:58:38 +02:00
Trinity Pointard 33474537ab make reproducible Windows build
Linux hash   : c024778da162cda77869c46d162729d012a7758bee973bc83fa78c964d080903
Windows hash : e97e171740b889b7e3722b79c3763177174baac7fa34a5c3615c999298e1bc48
2021-08-29 13:48:33 +02:00
Nick Mathewson 557a0ff40b Move all crates into a `crates` subdirectory.
This will cause some pain for now, but now is really the best time
to do this kind of thing.
2021-08-27 09:53:09 -04:00
Nick Mathewson 28cc12be25 Merge remote-tracking branch 'origin/mr/67' into unwrap_used 2021-08-27 08:36:59 -04:00
Nick Mathewson aa912a7656 Updates for reproducible build docs 2021-08-26 13:22:41 -04:00
Trinity Pointard 46f30aad4d address review comments
sha256: 7f190568fa05f71ad356dc56a7a9725a4be18e5e40d6880f469e22981a18edd7
2021-08-26 08:38:45 +02:00
S0AndS0 b95513c5e6 Move "use_unwrap" lint from some to want list
And convert some `add_or_port` calls to use `SocketAddr` instead of
parse + unwrap on strings.

> Note, this last change is only for IPv4 addresses, for now, and only
> within tests.
2021-08-25 12:23:57 -07:00
Trinity Pointard eb0522fe63 fix segfault on static bin
see https://github.com/rusqlite/rusqlite/issues/914

sha256: 684ebc4b8c270fc63beba185f6c54ceeb98734f13aa7aeca9b64acb33432a21c
2021-08-25 20:37:58 +02:00
Trinity Pointard 540302851e make arti availlable as build artifact
and make builds reproducible
sha256sum: f141c54929a43a31b9ed6b529f6f863aace87f7406818b2f8ffe2b7a5e2803fb
2021-08-25 18:16:45 +02:00
Nick Mathewson ffc27999ef add_warning.py: explain what to do when no args are given 2021-08-18 08:56:44 -04:00
Nick Mathewson 0b6f638517 Add clippy::missing_panics_doc to add_warnings.py. 2021-08-18 08:53:58 -04:00
S0AndS0 484cc24856 Add the "implicit_clone" lint.
> Check `implicit_clone` section of Clippy documentation for details;
>
>   https://rust-lang.github.io/rust-clippy/master/index.html#implicit_clone

This adds, and addresses, the following Clippy configuration to crates;

    #![deny(clippy::implicit_clone)]

And moves related line within `maint/add_warning.py` file. My intent is to
mitigate extra edits after merging, so please let me know if I need to do this
last bit differently.
2021-08-17 11:49:39 -07:00
Nick Mathewson 9c6dd27ca1 Use add_warnings.py for clippy::cast_lossless 2021-08-13 11:36:14 -04:00
Nick Mathewson 79d9eac58c Add unnecessary_wraps to the big warning list. 2021-08-06 16:20:37 -04:00
Nick Mathewson ac962a7665 Remove spurious "exit 0" 2021-08-04 16:04:24 -04:00
Nick Mathewson 7c9373a32f Add my "fuzz everything" script that I like to run overnight 2021-08-04 15:59:36 -04:00
Nick Mathewson 99e0c1789b Add a shell script to verify that every package has an okay license.
We implement this by giving a list of permitted licenses, and then
using cargo-license to dump everything's actual license.  Since
packages list their licenses as "x OR y OR z", we permit any package
that is available under at least one license on the allow-list.
2021-07-06 15:03:44 -04:00
YUAN LYU 92dbf20a76 Add clippy warn needless pass by value 2021-06-21 23:27:19 -04:00
YUAN LYU b830e81b7f Add clippy warn needless borrow 2021-06-21 21:34:40 -04:00
Nick Mathewson 62ee7f4acd Note a couple of more desired warnings. 2021-06-17 08:28:55 -04:00
Nick Mathewson 2102d2e2b5 Add noop_method_call warning.
This would have saved ahf and me a lot of confusion in debugging a
situation where we were cloning a reference of a type that didn't
implement Clone.
2021-05-27 14:32:29 -04:00
Nick Mathewson ac4786a6c2 Add a script with the command I've been using for coverage 2021-05-26 10:46:17 -04:00
Nick Mathewson 745c90e92c Enable cargo_common_metadata warning. 2021-05-25 15:42:47 -04:00
Nick Mathewson 3558af4321 Add a script to use cargo-readme to update all per-crate README.md files. 2021-05-21 08:59:36 -04:00
Nick Mathewson 411e1b35e0 Add the "unreachable_pub" lint.
This is a somewhat obnoxious change in its scope and requirements,
but it makes it easier to understand what the real public and
private parts of our APIs are.
2021-05-18 13:45:39 -04:00
Nick Mathewson 4507f44b54 Add the little script I use to add warnings to crates. 2021-05-18 09:31:49 -04:00