Under what conditions does busy-wait spinning outperform parking a waiting thread? Give the break-even reasoning, not just a rule of thumb.

middleshould knowfreq 48%

answer

spin if remaining hold H < park+wake cost C
spin budget ≈ C ⇒ regret bounded at ~2C
needs a spare core: holder must be running
never spin around I/O, faults or allocation
high contention lengthens H via cache-line ping-pong

basics

~20 s

Spin when the expected remaining hold time is shorter than the round trip of parking and waking, and when a spare core exists so the holder can still run. Long, unpredictable or blocking critical sections, and oversubscribed CPUs, favour parking.

solid answer

~60 s

The break-even is a comparison of two costs.

Spinning costs H — the expected remaining hold time — of wasted CPU, and delivers the lock after roughly that same H. Parking costs a park plus a wake, call it C (single-digit microseconds including the cold-cache aftermath), and delivers the lock after max(H, C).

So spinning wins when H < C, and the sensible policy is to spin for about C and then park: you never waste more CPU than the switch you avoided, and you cap the regret at a factor of two.

Three preconditions matter as much as the arithmetic:

A spare core. With more runnable threads than CPUs, the spinner may be occupying the CPU the holder needs, so H becomes unbounded.
A non-blocking critical section. If the holder can do I/O, allocate, page-fault or be preempted, H is not short and not predictable.
Low-to-moderate contention. Many spinners on one lock generate cache-line traffic that slows the holder, lengthening the very H you are betting on.

On top of that, spinning trades throughput for latency, so it is worth more in a latency-critical path than in a batch pipeline.

The break-even is a comparison of two costs. Spinning costs `H` — the expected *remaining* hold time — of wasted CPU, and delivers the lock after roughly that same H. Parking costs a park plus a wake, call it `C` (single-digit microseconds including the cold-cache aftermath), and delivers the lock after `max(H, C)`. So spinning wins when **H < C**, and the sensible policy is to spin for about C and then park: you never waste more CPU than the switch you avoided, and you cap the regret at a factor of two. Three preconditions matter as much as the arithmetic: 1. **A spare core.** With more runnable threads than CPUs, the spinner may be occupying the CPU the holder needs, so H becomes unbounded. 2. **A non-blocking critical section.** If the holder can do I/O, allocate, page-fault or be preempted, H is not short and not predictable. 3. **Low-to-moderate contention.** Many spinners on one lock generate cache-line traffic that slows the holder, lengthening the very H you are betting on. On top of that, spinning trades throughput for latency, so it is worth more in a latency-critical path than in a batch pipeline.

code

text · 10 lines

text

acquire():
    budget = SPIN_LIMIT              # ~ cost of one park+wake round trip
    while budget > 0:
        if try_lock(): return        # common case: short hold, no syscall
        cpu_pause_hint()
        budget -= 1
    enqueue(self); park()            # long hold: give the CPU back

# worst case cost ~ 2C (spun a full budget, then parked anyway)
# best case  ~ H  (H << C, switch avoided entirely)

go deeper

for a junior

Say that spinning pays off only for very short waits, because otherwise you burn CPU for longer than a context switch would have cost.

for a middle

State the H < C comparison explicitly, give the spin-then-park policy with a budget near the switch cost, and name the spare-core precondition.

for a senior

Discuss hold-time distributions rather than means, contention-driven feedback that lengthens the hold, NUMA and container-quota effects, and when latency justifies the CPU spend.

for a principal

Frame it as allocating a scarce resource: spinning converts CPU into latency at a known exchange rate, so the decision belongs to the workload's objective (tail latency versus throughput) and to the deployment's oversubscription level, and it should be revisited when either changes.

Stating the trade formally

Let:

H = the expected remaining time the current holder will keep the lock (not the average hold time from the start — you arrive part-way through).
C = the full round-trip cost of parking and being woken: the park path, the wake path, the scheduler work, and the performance loss from resuming with cold caches, TLB and branch predictors. In practice single-digit microseconds on a commodity system, and larger than a naïve context-switch benchmark suggests.

A spinner obtains the lock after about H and consumes H of CPU. A parker obtains the lock after about max(H, C) and consumes ~C of CPU while releasing the core for other work in between.

Spinning is the better bet when H < C. Since you rarely know H, the standard policy is spin for a bounded budget of about C, then park. The worst case is that you spin the full budget and then park anyway, paying roughly 2C instead of C — bounded regret — while the common case of a very short hold avoids the switch entirely.

Why the arithmetic is not enough

Three structural preconditions can make H effectively infinite regardless of the numbers.

1. There must be a spare CPU. Spinning assumes the holder is executing right now on another core. If runnable threads outnumber cores, the scheduler may have preempted the holder, and the spinner is burning the very time slice the holder needs. On a single CPU with a non-preemptive scheduler this is a hard livelock. This precondition — not the nanosecond arithmetic — is why kernels spin freely and application runtimes do not.

2. The critical section must not block or fault. Any operation that can suspend the holder — I/O, a lock acquisition, memory allocation that triggers reclamation, a page fault, a garbage-collection safepoint — turns a nanosecond-scale H into a millisecond-scale one. A spinlock's correctness argument depends on the section being short and non-blocking, which is why interrupt-context code uses spinlocks and everything else generally does not.

3. Contention must be low enough. Spinning is self-defeating at high contention. Each naïve spin iteration performs an atomic read-modify-write, which acquires the cache line exclusively and invalidates every other copy, including the holder's. With k spinners you get k-way cache-line ping-pong that measurably slows the holder, increasing H — the mechanism you chose because H was small makes H larger. This is why real spin loops read (test) before they attempt (test-and-set), and add backoff.

Latency versus throughput

Even when H > C, spinning can be the right call if what you care about is tail latency and you have CPU to spare — a trading path, a low-latency RPC hop, a real-time audio thread. You are explicitly buying predictability with cores. Conversely, on a throughput-oriented, fully-loaded batch machine, spinning is nearly always wrong: every wasted cycle is a cycle of real work not done, and nobody is measuring the microsecond.

This is the honest framing of the trade: spinning converts CPU time into latency reduction at a poor exchange rate that is nonetheless worth it when latency is the scarce resource.

Practical inputs to the decision

Measured hold-time distribution. The mean matters less than the tail: a section that is 100 ns at p50 and 40 µs at p99 will make spinners pay dearly exactly when the system is busy. Size any spin budget against the distribution, not the mean.
Contention level. Uncontended locks never spin at all — the fast path succeeds immediately — so the spin policy only affects the contended minority. Measure that minority's frequency before optimising it.
Core count versus runnable threads. Track oversubscription. In a container with a CPU quota this is subtler: you may have eight visible cores but a quota of two, so spinning consumes quota and can trigger throttling that stalls the whole container.
NUMA topology. A cross-socket cache-line transfer is several times a same-socket one, so the effective H for a remote holder is larger and spinning is correspondingly worse.

The shape of a good answer

Name the two costs, state the H < C condition, propose the spin-for-about-C-then-park policy with its bounded regret, and then insist on the preconditions — spare core, non-blocking section, moderate contention — because those are what turn a reasonable estimate into a hang. Finish by noting that you would measure the hold-time distribution rather than assume it.

## Stating the trade formally Let: - **H** = the expected *remaining* time the current holder will keep the lock (not the average hold time from the start — you arrive part-way through). - **C** = the full round-trip cost of parking and being woken: the park path, the wake path, the scheduler work, and the performance loss from resuming with cold caches, TLB and branch predictors. In practice single-digit microseconds on a commodity system, and larger than a naïve context-switch benchmark suggests. A spinner obtains the lock after about H and consumes H of CPU. A parker obtains the lock after about max(H, C) and consumes ~C of CPU while releasing the core for other work in between. **Spinning is the better bet when H < C.** Since you rarely know H, the standard policy is *spin for a bounded budget of about C, then park*. The worst case is that you spin the full budget and then park anyway, paying roughly 2C instead of C — bounded regret — while the common case of a very short hold avoids the switch entirely. ## Why the arithmetic is not enough Three structural preconditions can make H effectively infinite regardless of the numbers. **1. There must be a spare CPU.** Spinning assumes the holder is executing *right now on another core*. If runnable threads outnumber cores, the scheduler may have preempted the holder, and the spinner is burning the very time slice the holder needs. On a single CPU with a non-preemptive scheduler this is a hard livelock. This precondition — not the nanosecond arithmetic — is why kernels spin freely and application runtimes do not. **2. The critical section must not block or fault.** Any operation that can suspend the holder — I/O, a lock acquisition, memory allocation that triggers reclamation, a page fault, a garbage-collection safepoint — turns a nanosecond-scale H into a millisecond-scale one. A spinlock's correctness argument depends on the section being short and non-blocking, which is why interrupt-context code uses spinlocks and everything else generally does not. **3. Contention must be low enough.** Spinning is self-defeating at high contention. Each naïve spin iteration performs an atomic read-modify-write, which acquires the cache line exclusively and invalidates every other copy, including the holder's. With k spinners you get k-way cache-line ping-pong that measurably slows the holder, increasing H — the mechanism you chose because H was small makes H larger. This is why real spin loops read (test) before they attempt (test-and-set), and add backoff. ## Latency versus throughput Even when H > C, spinning can be the right call if what you care about is tail latency and you have CPU to spare — a trading path, a low-latency RPC hop, a real-time audio thread. You are explicitly buying predictability with cores. Conversely, on a throughput-oriented, fully-loaded batch machine, spinning is nearly always wrong: every wasted cycle is a cycle of real work not done, and nobody is measuring the microsecond. This is the honest framing of the trade: spinning converts CPU time into latency reduction at a poor exchange rate that is nonetheless worth it when latency is the scarce resource. ## Practical inputs to the decision - **Measured hold-time distribution.** The mean matters less than the tail: a section that is 100 ns at p50 and 40 µs at p99 will make spinners pay dearly exactly when the system is busy. Size any spin budget against the distribution, not the mean. - **Contention level.** Uncontended locks never spin at all — the fast path succeeds immediately — so the spin policy only affects the contended minority. Measure that minority's frequency before optimising it. - **Core count versus runnable threads.** Track oversubscription. In a container with a CPU quota this is subtler: you may have eight visible cores but a quota of two, so spinning consumes quota and can trigger throttling that stalls the whole container. - **NUMA topology.** A cross-socket cache-line transfer is several times a same-socket one, so the effective H for a remote holder is larger and spinning is correspondingly worse. ## The shape of a good answer Name the two costs, state the H < C condition, propose the spin-for-about-C-then-park policy with its bounded regret, and then insist on the preconditions — spare core, non-blocking section, moderate contention — because those are what turn a reasonable estimate into a hang. Finish by noting that you would measure the hold-time distribution rather than assume it.

Why is the relevant quantity the *remaining* hold time rather than the average hold time?
A waiter arrives at a random point inside the critical section, so what it must wait for is whatever is left, not the whole thing. For highly variable hold times this matters a lot: with a heavy-tailed distribution, arriving during a long hold means the expected remaining time can be longer than the mean hold time, which is the inspection-paradox effect. That is exactly when a fixed spin budget performs worst.
How does running inside a container with a CPU quota change the calculation?
Spinning consumes quota without doing work, so a busy-wait can exhaust the container's CPU allowance for the period and get the whole container throttled — every thread stops, including the lock holder, for the remainder of the period. The visible core count is also misleading, because the quota, not the core count, determines how many threads can genuinely run in parallel. Under a tight quota, parking is usually the correct default.
If a lock is almost never contended, does the spin-versus-park choice matter at all?
Barely, and that is the point. An uncontended acquire takes the fast path — a single successful atomic operation — and never reaches the waiting policy. The policy only governs the rare contended case, so it should be optimised only after measuring how often contention actually occurs; premature spin tuning on a cold lock is wasted effort.

saying these in an interview costs you the question

Giving a fixed spin count with no reference to the cost of parking
Ignoring the requirement that a spare core exists for the holder to run on
Assuming more spinning helps under heavy contention
Comparing against average hold time instead of remaining hold time
Treating spinning as free because "the thread had nothing else to do"

Under what conditions does busy-wait spinning outperform parking a waiting thread? Give the break-even reasoning, not just a rule of thumb.

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Stating the trade formally

Why the arithmetic is not enough

Latency versus throughput

Practical inputs to the decision

The shape of a good answer

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions5 questions

▾related topics4 topics

Stating the trade formally

Why the arithmetic is not enough

Latency versus throughput

Practical inputs to the decision

The shape of a good answer

answer

go deeper

context