skip to content

Verification and Benchmarking

Rust builds tests into the language and toolchain: unit tests beside the code, integration tests in tests/, doc tests in comments. Interviewers ask how to test private code and mock dependencies.

on this pageshow

explore

questions

19

In a Rust package, what does Cargo build from each file in the tests/ directory, and which library items can that code call?

level: juniorimportance: must knowfreq 55%

answer

  1. outside the crate, looking in
  2. one executable per file
  3. use the library by its name
  4. pub items, nothing configured out

basics

~20 s

Cargo compiles each top-level .rs file in tests/ as its own crate and test executable, linked against the package's library. That code can call only the library's public API, exactly as a downstream crate would.

solid answer

~40 s

Each file such as `tests/api.rs` becomes a separate crate with its own test executable, built only by `cargo test`, so it needs no `#[cfg(test)]`. It brings the library in by its crate name — `use logfilter::{parse, Level};` — and sees only `pub` items reachable from the crate root: calling a private or `pub(crate)` function fails with E0603. The library is compiled for it as an ordinary build, so anything behind `#[cfg(test)]` in `src/` is configured out (E0425). Each file gets its own `Running tests/...` section, its `#[test]` functions run under the same libtest harness as unit tests, and it can use the package's `[dev-dependencies]`.

code

rust · 17 lines
rust
// tests/api.rs: its own crate, linked against the logfilter library
use logfilter::{keep, parse, Level};

#[test]
fn parse_ignores_case() {
    assert_eq!(parse("Info"), Ok(Level::Info));
}

#[test]
fn keep_drops_lines_below_min() {
    let lines = ["[DEBUG] tick", "[WARN] slow disk", "[ERROR] disk full"];
    let kept: Vec<&str> = keep(&lines, Level::Warn).collect();
    assert_eq!(kept, ["[WARN] slow disk", "[ERROR] disk full"]);
}

// Does not compile: normalize is private to the library (E0603).
// let tag = logfilter::normalize(" WARN ");

go deeper

for a junior

Recall that each file in tests/ is its own crate, imports the library by name, and can only use pub items.

for a middle

Explain the mechanics: a separate test executable per file, the library built without cfg(test), and the E0603 and E0425 errors that follow.

for a senior

Show that integration tests validate the published API, catch missing re-exports, and that many small test files cost link time you may want to consolidate.

for a principal

Frame the split as a design check: what is worth proving through the public API versus inside the module, and how test layout affects build time.

## What the tests/ directory is A Cargo package can hold a directory called `tests/` next to `src/`. Cargo treats the `.rs` files directly inside it as **integration tests**: code that exercises the package's **library crate** (`src/lib.rs`) from the outside, the way a user of the library would. The name contrasts with **unit tests**, which live inside `src/` in a `#[cfg(test)]` module and can reach private items; that side of testing is covered with in-module unit tests. ## Each file is its own crate The defining rule is that **every top-level file in `tests/` is compiled as a separate crate**, and each one becomes its own test executable linked against the library. - **Built only by `cargo test`.** Cargo compiles these files only when tests are built, so they need no `#[cfg(test)]` attribute anywhere. - **Separate scope.** `tests/api.rs` and `tests/cli.rs` cannot see each other's functions; each starts with an empty module tree of its own. - **The library is a dependency.** Because the test is another crate, it must name the library: `use logfilter::{keep, parse, Level};`. Nothing is in scope by default. - **Its own section in the output.** `cargo test` prints a `Running tests/api.rs (...)` block per file, followed by that file's results. - **Dev-dependencies are available.** Integration tests are linked with both `[dependencies]` and `[dev-dependencies]`; the manifest side of that belongs to Cargo's dependency material. Inside one executable, the libtest harness collects the `#[test]` functions and runs them on several threads, exactly as it does for unit tests. Cargo then runs the separate test executables one after another; the flags that change this are Cargo's. ## Only the public API Because an integration test is a different crate, Rust's visibility rules apply to it in full: 1. **Private items are rejected.** Calling a function the library did not mark `pub` fails with `error[E0603]: function `normalize` is private`. 2. **`pub(crate)` is still private here.** That visibility means "inside the library crate", and the test file is not inside it, so the same E0603 appears. 3. **`#[cfg(test)]` items do not exist.** The library is compiled for integration tests as a normal build, without `cfg(test)`, so a helper declared `#[cfg(test)] pub fn fixture()` in `src/` is configured out. rustc reports E0425 and adds the note `found an item that was configured out`. That restriction is the point. A test that compiles against the public API proves the API is usable as published: that the needed items are `pub`, re-exported where callers expect them, and composable without internal shortcuts. | | Unit test | Integration test | |---|---|---| | Location | a `#[cfg(test)] mod tests` inside `src/` | a `.rs` file directly in `tests/` | | Crate | part of the library or binary crate | a separate crate per file | | Reaches private items | yes, through `use super::*` | no, public API only | | Needs `#[cfg(test)]` | yes | no | | Sees `#[cfg(test)]` items in `src/` | yes | no | ## A worked example The `logfilter` package has a library with `pub fn parse`, `pub fn keep` and `pub enum Level`, plus a private `fn normalize`. The file `tests/api.rs` imports the public items and tests them together; un-commenting the last line breaks the build with E0603 rather than reaching into the library. ## Common mistakes - **Expecting `use super::*` to work.** A file in `tests/` has no parent module inside the library; it must import the library by name. - **Testing through internals.** If a behaviour can only be reached through a private function, either the public API is missing something or the check belongs in a unit test. - **Forgetting re-exports.** An item that is `pub` deep inside a private module is not reachable by path from outside; the integration test fails exactly as a user's code would, which is the signal to add a `pub use` at the crate root. - **Assuming a separate package.** Integration tests belong to the same package and share its `Cargo.toml`; they are separate crates, not separate packages, so they need no dependency entry for the library. ## What interviewers listen for - That **each file is a crate**, which explains both the explicit `use` line and why helpers need special handling. - That integration tests **see the crate as a user does**, so a missing `pub` or a missing re-export shows up here first. - That `#[cfg(test)]` code in the library is **invisible** to them, a common surprise when someone tries to share a fixture from `src/`. - That many small files cost a link step each; grouping related tests as modules of one test target is the usual fix when build time grows.

  • Can a Rust integration test call a library function marked pub(crate)?
    No. `pub(crate)` makes the item visible anywhere inside the library crate, and each file in `tests/` is a different crate, so the call fails with E0603 exactly like a private item. Test it from a unit test inside `src/`, or make it part of the public API if callers genuinely need it.
  • A helper in src/lib.rs is marked #[cfg(test)] pub fn fixture(); why can tests/api.rs not find it?
    For integration tests Cargo builds the library as an ordinary crate without `cfg(test)`, so the helper is configured out. rustc reports E0425 with a note that it found an item that was configured out. Shared integration helpers live under `tests/` instead, for example in `tests/common/mod.rs`.
  • Why do some Rust projects keep integration tests in one test target instead of many files?
    Every top-level file in `tests/` links its own executable, and Cargo runs those executables one after another. With dozens of files, linking dominates build time. Putting the tests in modules of a single target, such as `tests/integration/main.rs` with `mod` files beside it, links once and lets the harness run all of them in parallel.

An integration test is a customer at the counter: it can order only what is on the menu, the library's pub items. A unit test works in the kitchen and can reach the private shelves.

  • Files in tests/ need #[cfg(test)] around their code, just like unit test modules.
  • Integration tests can reach private functions with use super::*.
  • All files in tests/ are compiled together into one test crate.
  • A #[cfg(test)] helper in src/ is visible to integration tests because cargo test sets cfg(test).
  • pub(crate) items are callable from tests/ because the tests belong to the same package.
open as a page

In a Rust unit test, why does `assert_eq!(to_cents(135), Cents(14))` fail to compile until `Cents` derives `PartialEq` and `Debug`?

level: juniorimportance: must knowfreq 70%

basics

~10 s

assert_eq! compares its arguments with ==, which needs PartialEq, and on failure prints both with {:?}, which needs Debug. A bare struct has neither, so rustc reports E0369 and E0277; #[derive(PartialEq, Debug)] fixes both.

open as a page

In Rust, what do `#[test]` and a `#[cfg(test)] mod tests` block do, and how do those tests reach a private helper function?

level: juniorimportance: must knowfreq 80%

basics

~20 s

#[test] marks a no-argument function for the test harness; #[cfg(test)] compiles the tests module only in test builds. A child module may use its ancestors' private items, so use super::* lets tests call private helpers.

open as a page

In a Rust criterion benchmark, why does b.iter(|| { fnv1a(&buf); }) report about 225 picoseconds, and how does std::hint::black_box fix it?

level: middleimportance: must knowfreq 35%

basics

~20 s

The closure throws away fnv1a's result, so the optimiser deletes the pure call and criterion times an empty loop. Return the value, which Bencher::iter passes to black_box, and wrap inputs in std::hint::black_box so they stay opaque.

open as a page

A Rust doc example using ? fails cargo test with E0277; how does rustdoc wrap it, and how do hidden # lines fix it?

level: middleimportance: must knowfreq 45%

basics

~20 s

Rustdoc wraps an example lacking fn main in fn main() { ... }, which returns (), so ? cannot return an error. Add a hidden # fn main() -> Result<(), E> { with # Ok(()) and # }, or end with a hidden Ok::<(), E>(()).

open as a page

A Rust rate limiter calls `Instant::now()` directly and a reviewer asks for a mockall test: why can't the call be mocked in place, and should the seam be generic or `Box<dyn Clock>`?

level: seniorimportance: must knowfreq 36%

basics

~20 s

Rust cannot swap a directly named function at run time, and mockall only generates a new type, so the limiter must take a Clock trait. A generic C: Clock costs nothing at run time; a Box<dyn Clock> keeps one concrete type.

open as a page

In Rust on the stable toolchain, how do you write a benchmark with the criterion crate rather than the nightly-only #[bench] attribute?

level: juniorimportance: should knowfreq 30%

basics

~10 s

Add criterion as a dev-dependency, write a function taking &mut Criterion in a benches/ file that calls bench_function with b.iter, and register it with criterion_group! and criterion_main!. The built-in #[bench] needs nightly's #![feature(test)].

open as a page

In a Rust criterion benchmark run with --baseline main, what do the change interval and p-value mean, and when is a change only noise?

level: middleimportance: should knowfreq 25%

basics

~20 s

Criterion runs a t-test against the saved baseline and bootstraps a 95% interval for the relative change in mean time; by default, improved or regressed needs p below 0.05 and the whole interval beyond ±1%.

open as a page

Why can't tests/cli.rs import functions from a Rust package that has only src/main.rs, and how do you make the tool testable?

level: middleimportance: should knowfreq 50%

basics

~20 s

A binary crate exposes no items to other crates, so use logfilter::parse; in tests/cli.rs fails with E0432. Move the logic into src/lib.rs, keep src/main.rs a thin caller, and test the library; tests can still run the binary via CARGO_BIN_EXE_<name>.

open as a page

Why does a Rust helper file tests/common.rs run as its own integration-test target, and how does tests/common/mod.rs avoid that?

level: middleimportance: should knowfreq 40%

basics

~20 s

Cargo builds every top-level .rs file in tests/ as an integration-test crate, so common.rs runs alone and reports 0 tests. tests/common/mod.rs sits in a subdirectory, so it is only a module that test files include with mod common;.

open as a page

In Rust, what does mockall's `#[automock]` on a `Clock` trait generate, and what do `expect_now_ms()`, `times`, `returning` and `with` configure?

level: middleimportance: should knowfreq 30%

basics

~10 s

#[automock] generates MockClock, a struct implementing Clock. expect_now_ms() adds an expectation; returning or return_const sets its result, with or withf matches arguments, and times sets a count checked at each call and at drop.

open as a page

In Rust's proptest crate, what does a `proptest!` test over strategies like `1u32..=20` and `prop::collection::vec` run, and why use `prop_assert!`?

level: middleimportance: should knowfreq 32%

basics

~20 s

proptest! rewrites each fn name(x in strategy) into a zero-argument test that draws inputs from the strategies, runs the body until 256 cases pass by default and shrinks any failure; prop_assert! fails a case by returning an error, not panicking.

open as a page

In Rust, how does a unit test returning `Result<(), E>` use `?`, and what does the harness print when it returns `Err`?

level: middleimportance: should knowfreq 55%

basics

~10 s

A test returning Result<(), E> with E: Debug can use ?; Ok(()) passes and Err fails, printing Error: and the error's Debug form without a line number. It cannot carry #[should_panic].

open as a page

A Rust library uses compile_fail doctests to prove misuse of its API is rejected; how do they differ from no_run, ignore and should_panic, and why are they fragile?

level: seniorimportance: should knowfreq 25%

basics

~20 s

compile_fail passes when the block fails to compile; no_run compiles without running, ignore skips the block, and should_panic must compile and then panic. Any compile error satisfies compile_fail, a typo included, and stable rustdoc does not check an error code.

open as a page

A Rust proptest run reports `minimal failing input: capacity = 1, refill_ms = 1, idle_ms = 1` and writes `proptest-regressions/lib.txt`: how did it shrink, and what does that file hold?

level: seniorimportance: should knowfreq 22%

basics

~20 s

proptest shrank by simplifying each input's value tree, binary-searching integers toward each range's low end while the case kept failing. The file stores that case's RNG seed, replayed before new cases on every run, so commit it.

open as a page

A Rust unit test marked `#[should_panic]` for `split_evenly(total, 0)` still passes after its setup `unwrap()` started panicking; why, and how does `expected = "..."` catch it?

level: seniorimportance: should knowfreq 40%

basics

~20 s

A bare #[should_panic] passes on any panic in the test body, including a setup unwrap(). With expected = "parts must be positive" the harness also requires that substring in the panic message, so the wrong panic fails the test.

open as a page

Should a Rust project let criterion benchmarks block merges in CI, and how would you keep runner noise from failing good pull requests?

level: principalimportance: should knowfreq 20%

basics

~20 s

Gate only a few critical benchmarks, on a quiet dedicated machine, comparing base and head in one job against thresholds above measured noise; run every benchmark once per pull request. Criterion itself never fails a regressed run.

open as a page

As the lead of a Rust service team, what rule would you set for test seams: generic parameters, trait objects, hand-written fakes or mockall?

level: principalimportance: should knowfreq 18%

basics

~20 s

Put traits only at boundaries tests must control: time, I/O, randomness. Default to a generic seam with a production default type and a shared fake, use mockall where the call itself is the behaviour, and decide which crates see the mocks.

open as a page

In a Rust criterion benchmark of seal(frame: Vec<u8>), which consumes a 64 KiB buffer, why use iter_batched, and why did BatchSize::SmallInput use 3 GB of memory?

level: seniorimportance: nice to knowfreq 15%

basics

~20 s

seal consumes its buffer, so each iteration needs a fresh copy, which iter_batched builds in untimed setup. SmallInput builds a tenth of each timing call's iterations at once, and warm-up keeps doubling that count, so 64 KiB inputs pile up.

open as a page