After a Rust workspace's CI moves from `cargo test` to `cargo nextest run`, doctests vanish and one test starts failing; why, and how do you set the job up?
answer
- threads versus processes
- state one test leaves for another
- doctests need a second step
- retries, partitions and profiles
basics
~20 scargo-nextest runs every test in its own process, so a test relying on state another test left behind now fails, and it skips doctests. Remove the hidden dependency and add cargo test --doc as a separate step.
solid answer
~40 s`cargo test` runs each test binary once and executes its tests as threads inside that one process, so a test can silently depend on a `static` another test set first. `cargo nextest run` builds the same test binaries but launches **one process per test**, which isolates them: shared in-process state is gone, so the hidden dependency fails every time instead of passing by luck. Nextest does not run doctests, because of limitations in stable Rust, so the job needs a separate `cargo test --doc`. For CI, install it with `cargo install --locked cargo-nextest` (its README says builds without `--locked` are broken) or a prebuilt binary, put settings in a `ci` profile in `.config/nextest.toml` selected with `-P ci`, use `--retries N` to spot flaky tests, and split runs across machines with `--partition hash:1/3`.
code
rust · 28 linesuse std::sync::atomic::{AtomicBool, Ordering};
static CONFIG_LOADED: AtomicBool = AtomicBool::new(false);
pub fn load_config() {
CONFIG_LOADED.store(true, Ordering::SeqCst);
}
pub fn config_loaded() -> bool {
CONFIG_LOADED.load(Ordering::SeqCst)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_loads_config() {
load_config();
assert!(config_loaded());
}
// Relies on a_loads_config having run earlier in the same process.
#[test]
fn b_reads_config() {
assert!(config_loaded());
}
}go deeper
Recall that cargo nextest run is a third-party test runner and that doctests still need cargo test --doc.
Explain the difference between threads inside one test binary and one process per test, and what each means for shared state.
Diagnose a test that only fails under nextest as a hidden ordering dependency, and set up a CI job with a profile, retries, partitions and a doctest step.
Weigh isolation and sharding against per-process cost, and set a team rule on flaky tests: retried and reported, or failed outright.
## Two execution models **`cargo test`** compiles each test target (the library's unit tests, every file in `tests/`, and so on) into a **test binary** built on the standard test harness, then runs each binary once. Inside that one process the harness runs the binary's `#[test]` functions **in parallel on threads**, capturing their output. The tests share everything a process has: `static` items, a global logger, environment variables, the current directory. **`cargo nextest run`** (crate `cargo-nextest`, current release 0.9.150) builds the same test binaries with Cargo, works out which tests each binary contains, and then launches **a separate process for each test**, running many such processes in parallel. Nextest's own documentation calls this the process-per-test model and presents per-test isolation as a feature: a test that crashes, leaks a handle or sets global state cannot affect its neighbours. Its changelog notes that, because of this model, it is generally safe to change environment variables at the start of a test. ## Why one test started failing A test that passed under `cargo test` and fails every time under nextest usually depends on **state another test created in the shared process**. In the example below, `b_reads_config` asserts on a flag that only `a_loads_config` sets. With `cargo test -- --test-threads=1` the harness happens to run `a_` first in the same process, so both pass; with several threads the outcome depends on scheduling. Run `b_reads_config` on its own, as `cargo test --lib b_reads_config` does, and it fails: that is what nextest does to every test. The fix belongs in the test: each test sets up what it needs, or the shared setup goes behind a `std::sync::OnceLock` that any test can trigger. Nextest did not break the test; it exposed an ordering dependency that was already a latent flake. The opposite case also exists: a test suite that relies on process-wide one-time work (an expensive global initialisation, a database pool) gets slower under nextest, because that work now happens once per test process. ## Why doctests vanished Doctests are not compiled into the test binaries nextest runs; rustdoc builds and runs them separately. Nextest's documentation states that doctests are **not supported** because of limitations in stable Rust and recommends running them in a separate step with **`cargo test --doc`**. A CI job that only runs `cargo nextest run` silently stops checking every example in your documentation. ## Setting up the CI job 1. **Install it reproducibly.** Use the prebuilt binaries or `cargo install --locked cargo-nextest`; the README says builds without `--locked` are, and will remain, broken. 2. **Run both steps:** `cargo nextest run --workspace` and `cargo test --doc --workspace`. 3. **Use a profile.** `.config/nextest.toml` holds named profiles; `-P ci` (or `NEXTEST_PROFILE=ci`) selects one with CI settings such as retries, slow-test timeouts and JUnit XML output for the CI system. 4. **Handle flaky tests deliberately.** `--retries N` (or `retries` in the profile) reruns failures; a test that passes on retry is reported as **flaky** and counts as passing by default (`flaky-result = "pass"`), which you can change to `"fail"`. 5. **Shard large suites.** `--partition slice:1/3` or `hash:1/3` splits the tests across machines, each running its share; `slice:` spreads tests evenly, `hash:` keeps assignments stable as tests are added or removed, and the older `count:` mode is deprecated in favour of `slice:`. 6. **Control failure behaviour** with `--no-fail-fast` to see every failure, or `--max-fail` to stop after a number of failures. | | `cargo test` | `cargo nextest run` | |---|---|---| | Unit of execution | one process per test binary, tests on threads | one process per test | | Doctests | yes | no; run `cargo test --doc` | | Shared `static` state between tests | possible | impossible | | Retries and flaky detection | none | `--retries`, flaky reporting | | Sharding | none | `--partition` | | Configuration | command-line flags after `--` | `.config/nextest.toml` profiles | ## What to say in the interview - **The model:** one process per test versus threads inside one process per test binary. - **The consequence:** hidden shared state fails every time instead of sometimes, and a crashing or hanging test is contained. - **The gap:** doctests, filled by a separate `cargo test --doc`. - **The CI features:** profiles, retries with flaky reporting, partitions and JUnit output. How to write well-isolated tests in Rust is covered with unit testing; nextest is the runner that enforces that isolation.
- Why can a test suite get slower after moving to nextest even though nextest runs tests in parallel?Each test now pays process start-up and any process-wide one-time work, such as global initialisation behind a `OnceLock`, again in every process. Suites dominated by many tiny tests or heavy shared setup can lose time; suites with a few slow tests usually gain because nextest schedules each test independently.
- What does `flaky-result` control in nextest, and what is its default?It decides whether a test that failed and then passed on a retry counts as passing or failing. The default is `"pass"`: the test is reported as flaky but the run succeeds. Setting `flaky-result = "fail"` in a profile, or `--flaky-result fail`, makes such tests fail the run.
- How do you split a nextest run across three CI machines?Give each machine the same command with a different partition, for example `--partition hash:1/3`, `hash:2/3` and `hash:3/3`. Hash partitioning keeps each test's shard stable as tests are added or removed; `slice:` spreads tests more evenly, and the older `count:` mode is deprecated in favour of it.
- nextest runs a test binary's tests on threads, just faster than cargo test
- cargo nextest run also runs doctests, so cargo test can be dropped
- A test that fails only under nextest is a nextest bug
- Retries in nextest make flaky tests invisible in the report
- cargo install cargo-nextest without --locked is the recommended install