A service that uses busy-wait spinning performs well on dedicated hardware but collapses when moved onto shared or virtualised machines. Explain the mechanism behind that collapse and what mitigations exist.

## The precondition spinning depends on Busy-waiting is a bet: *the holder is running right now on another CPU and will release within a time shorter than a context switch.* Everything about spinlock performance follows from whether that bet is true. On a dedicated machine with more cores than runnable threads and short, non-blocking critical sections, it is. Under oversubscription it is not, and the failure is not gradual — it is a cliff. ## Lock-holder preemption Suppose N threads run on M CPUs with N > M. Thread H acquires a spinlock and, two instructions into a 100 ns critical section, exhausts its time slice and is descheduled. Now: - Every other thread that wants the lock spins. - Each spinner is runnable and consumes a CPU for its entire quantum — typically milliseconds. - H is on the run queue behind those spinners. It cannot release until it is rescheduled. - Because the spinners are burning the CPUs, H is scheduled later than it otherwise would be. A critical section measured in nanoseconds has become a stall measured in milliseconds, and the amplification factor is roughly (number of spinners × quantum). Adding threads makes it worse, which is why the symptom is often described as "it got slower when we gave it more concurrency". ## Where oversubscription comes from in practice - **Virtual machines.** vCPUs are themselves scheduled by a hypervisor. A guest thread can be "running" from the guest's point of view while its vCPU has no physical CPU — visible as steal time. The guest scheduler has no idea, so it happily lets threads spin for a holder whose vCPU is descheduled. - **Containers with CPU quota.** The cgroup may see 64 cores but be allowed 2 CPU-seconds per 100 ms period. Spinning consumes quota, and when the quota is exhausted **the entire cgroup is frozen until the next period** — including the lock holder. Spinners actively cause the stall they are waiting through. - **Co-tenanted hosts and CI machines.** Neighbours' load reduces effective parallelism unpredictably. - **Thread-count misconfiguration.** Pools sized from the host's core count rather than the container's allowance, or several such pools inside one process, easily produce 5–10× oversubscription. ## Priority inversion A sharper variant: a low-priority thread L holds the lock; a high-priority thread H spins for it. The scheduler, doing exactly its job, keeps H on the CPU because it has higher priority — so L never runs, never releases, and H spins forever. On a single CPU this is an unconditional livelock; on multiple CPUs it is a severe stall whenever enough high-priority spinners exist to occupy all of them. A blocking lock avoids it because H's park makes the CPU available to L; priority inheritance fixes it properly by boosting L to H's priority, but inheritance requires a lock that records an owner, which a bare spinlock does not. ## Mitigations **1. Spin-then-park (adaptive spinning).** The universal answer. Spin for a bounded budget roughly equal to the cost of a park-and-wake round trip, then block. The bet is still taken when it is cheap, and the loss is capped when it is wrong. This alone converts the cliff into a modest slowdown. **2. Spin only while the owner is running.** If the runtime can observe whether the owning thread is currently on a CPU, spinning becomes evidence-based rather than hopeful: park immediately when the owner is descheduled. This directly tests the precondition and is the highest-value adaptive signal available. **3. Paravirtualised spinlocks.** In a virtualised guest, the guest kernel can tell the hypervisor "I am spinning for the holder of this lock; stop running me and run that vCPU instead." Directed yield converts wasted physical time into progress for the holder. This is why paravirtual lock support is a standard guest kernel feature and why disabling it on a busy hypervisor is measurable. **4. Remove the oversubscription.** Pin threads to CPUs, size pools from the *effective* CPU allowance rather than the host's core count, and give latency-critical spinning services dedicated cores. This is the only mitigation that restores the original precondition rather than damping the consequences. **5. Use an ownership-bearing mutex where priorities differ.** Priority inheritance or a priority ceiling protocol needs to know who holds the lock. This is a design-level argument for mutexes over bare spinlocks in any system with mixed thread priorities, and it is standard practice in real-time systems. **6. Shorten or remove the critical section.** The exposure window scales with hold time. Per-CPU or sharded data, read-mostly structures with optimistic reads, or lock-free algorithms remove the window entirely — no holder means no holder preemption. ## What to measure Steal time in virtualised environments; cgroup throttled-periods and throttled-time counters in containers; runnable-thread count against effective CPU allowance; context-switch and involuntary-preemption rates; and time spent in lock acquisition. The signature of this failure is high CPU utilisation with flat or falling throughput and a latency distribution with a long tail clustered near multiples of the scheduling quantum. ## The judgement to state Spinning is a bet on parallelism you may not control. When the deployment target is a shared, virtualised or quota-limited environment — which is most of them — the default should be a bounded spin followed by parking, and any pure-spin fast path should be justified by a measurement on the actual deployment target rather than on a developer laptop with idle cores.