skip to content

In Rust, how does a request thread block on `std::sync::Condvar` until a rate-limit bucket has tokens, and why must the wait sit in a loop?

level: seniorimportance: should knowfreq 40%

answer

  1. sleep until state changes
  2. wait takes and returns the guard
  3. unlock, sleep, relock atomically
  4. spurious wakeups exist
  5. wait_while is the loop

basics

~20 s

Lock the token count, then call Condvar::wait with the guard: it unlocks, sleeps and relocks before returning. Because std allows spurious wakeups and other waiters may take the token, recheck the count in a while loop or use wait_while.

solid answer

~40 s

The bucket pairs a `Mutex<u32>` with a `Condvar`. The request thread locks the mutex and, while the count is zero, calls `tokens = cvar.wait(tokens).unwrap();`: `wait` atomically unlocks the mutex and blocks, then re-acquires it before handing the guard back. The refiller increments the count under the same mutex and calls `notify_one` or `notify_all`. The loop is required because std documents that `wait` is susceptible to spurious wakeups, and because another waiter can take the token before this thread relocks. `Condvar::wait_while(guard, |t| *t == 0)` packages that loop. Notifications are not buffered, so the predicate check under the lock is what makes a notify before the wait harmless. `wait_timeout_while` adds a deadline.

code

rust · 29 lines
rust
use std::sync::{Arc, Condvar, Mutex};
use std::thread;
use std::time::Duration;

struct Bucket {
    tokens: Mutex<u32>,
    refilled: Condvar,
}

fn main() {
    let bucket = Arc::new(Bucket { tokens: Mutex::new(0), refilled: Condvar::new() });

    let b = Arc::clone(&bucket);
    let refiller = thread::spawn(move || {
        thread::sleep(Duration::from_millis(50));
        *b.tokens.lock().unwrap() += 2;
        b.refilled.notify_all();
    });

    // Request thread: block until a token exists, then take one.
    let mut tokens = bucket
        .refilled
        .wait_while(bucket.tokens.lock().unwrap(), |t| *t == 0)
        .unwrap();
    *tokens -= 1;
    println!("took a token, {} left", *tokens);
    drop(tokens);
    refiller.join().unwrap();
}

go deeper

for a junior

Recall that a Condvar lets a thread sleep until another thread changes mutex-protected state and calls notify_one or notify_all.

for a middle

Explain that wait takes the guard, unlocks and relocks atomically, and why the predicate goes in a while loop or wait_while.

for a senior

Show the notifier's discipline, explain why notifications are not buffered, choose notify_one or notify_all, and bound the wait with wait_timeout_while.

for a principal

Judge when a Condvar is clearer than a channel or a semaphore-like design for back-pressure, and how a waiting limiter affects latency under load.

## What a `Condvar` is for A rate limiter that *rejects* a request when a client is out of tokens needs only a `Mutex`. A limiter that makes the request *wait* until a refill arrives needs a way to sleep until another thread changes the shared state. Spinning on `lock()` wastes a core, and sleeping for a fixed time adds latency. `std::sync::Condvar` is the std tool for blocking until a condition on mutex-protected data becomes true. A `Condvar` is always used together with: - a `Mutex<T>` that protects the state (here, the token count), and - a **predicate** on that state (here, "tokens > 0"). ## How `wait` works `Condvar::wait` takes the `MutexGuard` by value and returns `LockResult<MutexGuard<'_, T>>`. In one step it: 1. **atomically** unlocks the mutex and blocks the thread, so no notification sent after the unlock can be missed; 2. sleeps until woken; 3. **re-acquires** the mutex before returning the guard. Because the guard is moved in and handed back, the usual form is `tokens = cvar.wait(tokens).unwrap();`. The `Result` is about poisoning: `wait` returns `Err` if the mutex is poisoned when the thread re-acquires it. ## Why the wait must be a loop The std docs state that `wait` is **susceptible to spurious wakeups**: it may return although nobody called `notify_one` or `notify_all`, and the predicate must be checked every time it returns. There is a second reason as well: with several waiters, another thread can grab the token between the notification and this thread re-acquiring the lock. So: ```rust let mut tokens = bucket.tokens.lock().unwrap(); while *tokens == 0 { tokens = bucket.refilled.wait(tokens).unwrap(); } *tokens -= 1; ``` `Condvar::wait_while(guard, |t| *t == 0)`, stable since Rust 1.42, is that loop packaged up: it checks the closure first and keeps waiting **while it returns `true`**, then returns the guard. Writing `if` instead of `while` compiles and usually works in testing, which is why it survives code review — until a wakeup arrives with the count still zero and the decrement underflows (a panic in a debug build) or a token is handed out that does not exist. ## The notifier's side The thread that changes the state must follow the same discipline: - Change the state **while holding the same mutex**, then call `notify_one()` or `notify_all()`. - Notifications are **not buffered**: the docs say calls to `notify_one` and `notify_all` are not buffered in any way. A notify with no thread waiting is simply lost — harmless only because the waiter checks the predicate under the lock before it ever sleeps. - Use `notify_all` when one change may satisfy several waiters (a refill of many tokens); `notify_one` when exactly one waiter can proceed. A `Condvar` must stay paired with one mutex: std says `wait` may panic if the condvar is used with more than one mutex over time. ## Timeouts | Method | Returns | Use | |---|---|---| | `wait(guard)` | `LockResult<MutexGuard>` | wait for one notification | | `wait_while(guard, cond)` | `LockResult<MutexGuard>` | loop until `cond` is false | | `wait_timeout(guard, dur)` | `LockResult<(MutexGuard, WaitTimeoutResult)>` | give up after `dur` | | `wait_timeout_while(guard, dur, cond)` | the same pair | loop with a deadline | `WaitTimeoutResult::timed_out()` tells you whether the deadline passed. A request handler that must answer within a budget uses `wait_timeout_while`, then checks `timed_out()` and the count, and returns a rate-limit error if no token appeared. ## Mistakes that survive testing - **`if` instead of `while`** around `wait`: rare spurious or stolen wakeups make it pass every test and fail under production load. - **Notifying without changing state under the lock**: a waiter that has just checked the predicate can miss the notification, because it is not buffered. - **Doing slow work while holding the guard** after taking the token: release it before processing the request, or every waiter and the refiller queue behind you. - **Sharing one `Condvar` between two mutexes**: std allows `wait` to panic in that case; give each mutex its own condvar. ## `parking_lot`'s variant `parking_lot::Condvar` takes `&mut MutexGuard` instead of moving the guard and documents **no spurious wakeups**, so its own example uses `if`. The loop is still needed whenever several threads compete for what the notification announced, so keeping `while` (or `wait_while`) is the portable habit.

  • What goes wrong if the refiller increments the count without holding the bucket's mutex and then calls `notify_one`?
    With a `Mutex<u32>` it cannot, since the count is only reachable through the guard. If the state lived outside the mutex, say in an atomic, the waiter could check the predicate, the refiller could update and notify, and only then would the waiter block — missing a notification that is not buffered. Updating under the mutex closes that gap because `wait` unlocks and sleeps atomically.
  • How does a handler give up after 50 ms instead of waiting forever?
    Call `cvar.wait_timeout_while(guard, Duration::from_millis(50), |t| *t == 0)`. It returns `LockResult<(MutexGuard<u32>, WaitTimeoutResult)>`; if `timed_out()` is true and the count is still zero, release the guard and return a rate-limit error. The deadline covers the whole loop, not each wakeup.

  • Condvar::wait keeps the mutex locked while the thread sleeps
  • An if check around wait is enough because wait only returns after a notify
  • A notify sent before anyone waits is stored and delivered later
  • The refiller can update the count without the mutex as long as it notifies
  • wait_while returns once the closure returns true