skip to content

A Rust job reads 2 million files and hashes them with `par_iter` on Rayon's global pool; why does it stall on slow storage, and how do you reconfigure Rayon?

level: seniorimportance: should knowfreq 30%

answer

  1. one pool, sized to logical CPUs
  2. a waiting worker does no hashing
  3. build_global succeeds only once
  4. a separate pool via install

basics

~20 s

Rayon's global pool has one worker per logical CPU and adds none when one blocks, so file reads park workers and idle the CPUs. Read files outside Rayon, or run that stage in its own ThreadPool via install.

solid answer

~40 s

Rayon's global pool is created lazily with `RAYON_NUM_THREADS` workers or, by default, one per logical CPU from `available_parallelism`, and it is shared by every Rayon call in the process. Rayon assumes CPU-bound closures: a worker blocked in `fs::read` does no hashing and no replacement thread starts, so on slow storage most workers wait and throughput collapses. `ThreadPoolBuilder::num_threads(n).build_global()` can set the global size, but only once, before any Rayon use, otherwise it returns `Err`. The better fix is to separate stages: read with dedicated threads or async I/O and give Rayon only the hashing, or run the blocking stage inside its own `ThreadPool` with `pool.install(|| ...)` so it cannot starve the global pool.

code

rust · 21 lines
rust
use rayon::prelude::*;
use std::time::{Duration, Instant};

fn main() {
    rayon::ThreadPoolBuilder::new().num_threads(4).build_global().unwrap();
    let again = rayon::ThreadPoolBuilder::new().num_threads(8).build_global();
    println!("global threads {}, second build_global is_err {}", rayon::current_num_threads(), again.is_err());

    // 16 "downloads" that each block for 100 ms, on the 4-thread global pool
    let t = Instant::now();
    (0..16).into_par_iter().for_each(|_| std::thread::sleep(Duration::from_millis(100)));
    println!("blocking on the global pool: {} ms", t.elapsed().as_millis());

    // a separate pool for the blocking stage; par_iter inside install runs there
    let io_pool = rayon::ThreadPoolBuilder::new().num_threads(16).build().unwrap();
    let t = Instant::now();
    io_pool.install(|| {
        (0..16).into_par_iter().for_each(|_| std::thread::sleep(Duration::from_millis(100)))
    });
    println!("same work inside a 16-thread pool: {} ms, pool size {}", t.elapsed().as_millis(), io_pool.current_num_threads());
}

go deeper

for a junior

Recall that Rayon's global pool has about one worker per logical CPU and that Rayon is meant for CPU-heavy work, not waiting on disks or networks.

for a middle

Explain how the default size is chosen, why a blocked worker leaves a CPU idle, and the once-only rule for build_global.

for a senior

Diagnose the stall from idle CPUs during a parallel read, then split reading from hashing or isolate the blocking stage in its own ThreadPool via install.

for a principal

Own where Rayon may be used in a codebase: CPU-bound stages only, the application configuring the global pool, and libraries never calling build_global.

## How big Rayon's pool is and who shares it Every `par_iter`, `join` and `scope` call that is not inside a custom pool runs on Rayon's **global thread pool**. It is created lazily the first time any Rayon API needs it. Its size comes from `ThreadPoolBuilder`'s default, which rayon-core 1.13 computes as: 1. the `RAYON_NUM_THREADS` environment variable, if it parses to a number above zero; 2. otherwise `std::thread::available_parallelism()`, the number of logical CPUs the process may use (the deprecated `RAYON_RS_NUM_CPUS` variable is still checked just before it); 3. otherwise 1. The documentation adds a future-compatibility warning: the default may one day add or remove threads dynamically, so code that relies on a fixed count should set it explicitly with `num_threads`. The pool is **process-wide**. A library that calls `par_iter` and the application that calls it share the same workers. That is the point (one pool sized to the cores, no oversubscription) and also the trap: anything that blocks a worker blocks it for everyone. ## Why blocking work stalls the hashing job The scenario: a batch job reads 2 million files from slow network storage and hashes each file's contents in one chain, `paths.par_iter().map(|p| hash(&fs::read(p)?))`. On a 16-core machine the pool has 16 workers. While a worker waits in `fs::read`, it is **parked in the kernel** and does no hashing; Rayon does not know it is waiting and does not start another thread. With slow storage most workers are waiting most of the time, CPUs sit idle, and throughput is bounded by `pool size / read latency`, not by the hash. Rayon's own documentation for `join` states the assumption: closures are expected to be **CPU-bound** tasks without I/O or other blocking operations; if they block, overall performance may be poor, and making one closure wait on another (for example through a channel) can deadlock. The example makes it concrete: sixteen 100 ms sleeps on a 4-thread pool take about 400 ms, four rounds of four. The symptom in production is distinctive: during the parallel stage, CPU utilisation sits far below the core count, and stack samples of the Rayon workers show them inside file-read system calls rather than in the hash function. A CPU-bound Rayon stage looks the opposite way, with every core busy. ## Configuring pools | API | What it does | Notes | |---|---|---| | `ThreadPoolBuilder::new().num_threads(n).build_global()` | configures the global pool | succeeds once; returns `Err` if the global pool already exists | | `ThreadPoolBuilder::new().num_threads(n).build()` | creates a separate `ThreadPool` | returns `Result<ThreadPool, ThreadPoolBuildError>` | | `pool.install(op)` | runs `op` inside that pool | `par_iter`, `join` and `scope` inside `op` use this pool | | `RAYON_NUM_THREADS=n` | environment default | read only when `num_threads` is not set | The global pool is initialised **exactly once**. Calling `build_global` a second time, or after any `par_iter` has already created the pool implicitly, returns an error whose message is "The global thread pool has already been initialized." Call it at the very start of `main`, before any Rayon use, and handle the `Result`. Its documentation recommends calling it only to change the default configuration or to warm the pool up for a benchmark; in a library it takes a process-wide decision away from the application, so leave it to `main`. `install` is how a stage gets its own workers: build a `ThreadPool` with an explicit size and run the parallel chain inside `pool.install(|| ...)`. The sixteen sleeps finish in about 100 ms inside a 16-thread pool, and the global pool's workers stay free for CPU work. ## Fixing the hashing job - **Separate the stages.** Read files with a mechanism designed for waiting (a few dedicated threads feeding a bounded channel, or an async runtime), and hand the bytes to Rayon only for hashing, which is the CPU-bound part. - **If reads must stay inside Rayon**, run that stage in a dedicated `ThreadPool` via `install`, sized for the storage's concurrency rather than the core count, so it cannot starve the global pool. How to size it is a capacity question covered with thread-pool sizing. - **Keep hashing on the global pool**, sized to the cores, where work stealing balances uneven file sizes. - **Name the dedicated pool's threads.** `ThreadPoolBuilder::thread_name(|i| format!("io-{i}"))` makes those workers recognisable in a profiler or a stack dump. - **Never block one Rayon task on a result another Rayon task produces** through a channel or a lock held across Rayon calls; with every worker waiting, nothing runs. Calling Rayon from async code has the mirror-image problem (a CPU-heavy closure stalling an executor thread), covered with async runtimes and blocking.

  • Why should a library crate avoid calling `ThreadPoolBuilder::build_global`?
    The global pool is initialised exactly once per process, and every later `build_global` returns an error. A library that configures it takes that decision away from the application, and if the application or another library initialised it first, the library's call fails. Libraries should use the ambient pool, or accept a `ThreadPool` from the caller, and leave global configuration to `main`.
  • Inside `pool.install(|| v.par_iter().map(f).sum())`, which pool runs the `par_iter`?
    The custom `pool`. `install` runs its closure on one of that pool's workers, and Rayon calls made from a worker use the pool that worker belongs to, so `par_iter`, `join` and `scope` inside the closure all schedule on `pool`, not on the global pool. The calling thread blocks until the closure returns its result.

  • Rayon spawns extra threads when a worker blocks on I/O
  • Each par_iter call gets its own fresh thread pool
  • build_global can resize the global pool at any time
  • par_iter over file paths is the fastest way to read files
  • RAYON_NUM_THREADS overrides an explicit num_threads call