skip to content

A Cargo build script generates a CRC-32 lookup table at build time; where must it write the file, and how does the crate include it?

level: middleimportance: should knowfreq 30%

answer

  1. a directory Cargo hands the script
  2. read at run time, not with env!
  3. include! plus concat! plus env!
  4. never write into src/
  5. Cargo does not empty it

basics

~10 s

Into OUT_DIR, a per-package directory Cargo passes to the script as an environment variable; the crate then pastes the file in with include!(concat!(env!("OUT_DIR"), "/crc_table.rs")). Writing into src/ instead breaks cargo package verification.

solid answer

~40 s

The build script reads `OUT_DIR` at run time with `std::env::var_os("OUT_DIR")`, computes the 256-entry table and writes `crc_table.rs` there. It cannot use `env!("OUT_DIR")`, because the variable is set when the script runs, not when it is compiled; rustc rejects that with ``environment variable `OUT_DIR` not defined at compile time``. In the crate the opposite holds: Cargo sets `OUT_DIR` while compiling a package that has a build script, so `include!(concat!(env!("OUT_DIR"), "/crc_table.rs"))` pastes the generated items in. `OUT_DIR` is unique to the package inside the build directory, so generated files stay out of the source tree, which may be a read-only registry copy; `cargo package` fails verification if `build.rs` modifies the source directory. Cargo does not empty `OUT_DIR` between runs. With no input files, `cargo::rerun-if-changed=build.rs` stops needless reruns.

code

rust · 20 lines
rust
// build.rs
use std::{env, fs, path::Path};

fn main() {
    // CRC-32 (IEEE, reflected) lookup table, computed once per build.
    let mut table = [0u32; 256];
    for (i, slot) in table.iter_mut().enumerate() {
        let mut c = i as u32;
        for _ in 0..8 {
            c = if c & 1 != 0 { 0xEDB8_8320 ^ (c >> 1) } else { c >> 1 };
        }
        *slot = c;
    }
    let body: Vec<String> = table.iter().map(|v| format!("{v:#010x}")).collect();
    let code = format!("pub static CRC_TABLE: [u32; 256] = [{}];\n", body.join(", "));

    let out_dir = env::var_os("OUT_DIR").unwrap(); // set when the script runs
    fs::write(Path::new(&out_dir).join("crc_table.rs"), code).unwrap();
    println!("cargo::rerun-if-changed=build.rs");
}

go deeper

for a junior

Recall that generated files go in OUT_DIR and that the crate includes them with include!, concat! and env!.

for a middle

Explain why env! fails inside build.rs but works in the crate, and why each macro in the include line must see compile-time literals.

for a senior

Defend the OUT_DIR-only rule with its real consequences: immutable registry sources, cargo package verification, and separate output directories per profile and target.

for a principal

Choose between build-time generation and a checked-in generated file guarded by a test, weighing build cost and reviewability for the team.

## Two halves of build-time code generation Generating code at build time takes two cooperating pieces: a **build script** (`build.rs`) that writes a Rust source file, and the **crate** that pulls that file into its own compilation. Take a compression crate, `tinyz`, that needs a CRC-32 lookup table: 256 `u32` values computed from a polynomial. Writing them out by hand is error-prone, and computing them at start-up costs every run, so the build script computes them once per build and emits a `static` array. The file has to go somewhere both halves agree on. That place is **`OUT_DIR`**. ## Writing the file: OUT_DIR When Cargo runs a build script it sets the environment variable `OUT_DIR` to a directory reserved for that package's generated output. The Cargo reference describes it this way: - it lives inside the build directory (by default under `target/debug/build/<pkg>-<hash>/out`); - it is **unique to the package** being built, so two packages never collide; - it is **not cleaned or reset** between builds, so a script must not assume it starts empty and must tidy any files it no longer produces; - build scripts should place all their output and intermediate artifacts there and modify nothing outside it. The script reads the variable the way any program reads its environment, at run time: ```rust let out_dir = std::env::var_os("OUT_DIR").unwrap(); std::fs::write(std::path::Path::new(&out_dir).join("crc_table.rs"), code).unwrap(); ``` A common slip is writing `env!("OUT_DIR")` in `build.rs`. `env!` is expanded when the script itself is **compiled**, and at that moment Cargo has not set `OUT_DIR` for it, so rustc stops with ``environment variable `OUT_DIR` not defined at compile time``. ## Including the file in the crate Inside the package's own crates the timing is reversed. Cargo sets `OUT_DIR` in rustc's environment while compiling a package that has a build script, so `env!` works there, and three built-in macros combine: | Macro | Role in `include!(concat!(env!("OUT_DIR"), "/crc_table.rs"))` | |---|---| | `env!("OUT_DIR")` | expands to the directory as a string literal at compile time | | `concat!(..., "/crc_table.rs")` | joins literals into one path literal | | `include!(...)` | parses that file as Rust and pastes its items in place | Each piece needs compile-time literals. Swapping `env!` for `std::env::var("OUT_DIR").unwrap()` fails with `expected a literal`, because a function call has no value until the program runs. A `mod crc_table;` declaration fails too (E0583, file not found for module), since module files are looked up beside the source, not in `OUT_DIR`. `include_str!` would compile, but it yields a `&'static str` holding the text rather than the items. ## Why never write into src/ It is tempting to drop `crc_table.rs` into `src/` and use a normal module. Cargo's documentation and tooling push back for concrete reasons: 1. A dependency's sources live in Cargo's registry cache or a vendored directory, and Cargo relies on those sources being **immutable**; a script that edits them breaks that invariant the moment the crate is used as a dependency. 2. `cargo package` and `cargo publish` build the packaged crate as a check and compare its source directory afterwards. A script that added `src/generated.rs` makes that verification fail with `Source directory was modified by build.rs during cargo publish. Build scripts should not modify anything outside of OUT_DIR.` 3. Debug and release builds, or builds for different targets, get **separate output directories**; one shared file in `src/` would be overwritten by whichever ran last. When a team wants generated code reviewed in the repository, the Cargo book's advice is not to produce it from `build.rs` at all, but to check the file in and have a test regenerate it into a temporary file and fail when the two differ. ## Keeping reruns cheap This table depends on nothing but the script's own code, so the script prints `cargo::rerun-if-changed=build.rs`. Without any rerun-if line, Cargo would rerun the script whenever any file in the package changed. If the table were read from an input file, the script would list that file instead, so editing it regenerates the table. The same pattern scales beyond one table: - a script may write several files into `OUT_DIR` and the crate may `include!` each one where it is needed; - generated code should be deterministic, so the same inputs produce the same file on every machine and builds stay reproducible; - generators that are expensive to run, such as a parser generator, are exactly where a precise `rerun-if-changed` list pays off.

  • Why does the generated table not disappear when the build script changes what it writes?
    Cargo does not clean or reset `OUT_DIR` between builds, so files from an earlier run stay until `cargo clean` or until the script deletes them. A script that stops producing a file, or that needs a fresh directory, has to remove the old contents itself.
  • How would you keep a generated file reviewed in the repository without generating it from build.rs?
    Check the generated file in as ordinary source and add a test that regenerates it into a temporary file and fails if the two differ, run in CI. Updating means replacing the checked-in file with the fresh output; `build.rs` never touches the source tree.

  • The build script can read OUT_DIR with env! like the crate does.
  • Writing generated code into src/ is fine as long as it is gitignored.
  • Cargo empties OUT_DIR before every run of the build script.
  • A mod declaration can load a generated file straight from OUT_DIR.
  • include_str! pastes the generated Rust items into the crate.