Why are compilers and CPUs allowed to execute memory reads and writes in an order different from the source code, and what rule limits how far they can go?
answer
- two reorderers: compiler + CPU
- as-if-serial = single-thread only
- no dependency → free to swap
- store buffer, register caching, speculation
- order across threads must be requested
basics
~20 sBoth only promise that a single thread running alone produces the same result — the as-if-serial rule. Within that they hoist, sink, cache values in registers, buffer stores and execute out of order. Another thread watching memory can observe a different order.
solid answer
~50 sReordering happens at two independent layers. The **compiler** moves, merges, deletes and register-caches loads and stores. The **CPU** buffers stores, executes out of order and speculates past branches. Both are constrained only by *as-if-serial* semantics: one thread, running alone, must produce the same visible result as the source order. That rule is defined over a single thread's data and control dependencies, so it says nothing about what a *second* thread sees. So writing `data = 42` then `ready = true` can become visible to a reader as `ready` first, and the reader may itself have hoisted its load of `data` above its load of `ready`. Cross-thread ordering is never free and never implied: you obtain it by using synchronization — a lock, an ordered atomic, or an explicit fence. Those are the only points where the memory model promises anything about the order other threads observe.
code
text · 8 linesProducer Consumer
data = 42 while (!ready) {}
ready = true use(data)
legal executions include:
ready=true committed before data=42 -> use(0)
consumer loads data before loading ready -> use(0)
consumer caches ready in a register -> spins forevergo deeper
Recall that compilers and CPUs may reorder memory operations and that only single-threaded results are preserved; conclude that shared data needs synchronization.
Name both layers, give concrete mechanisms (register caching, hoisting, store buffers, out-of-order execution), and walk the data/ready publication example end to end.
Frame it as a contract: the memory model defines the few points where ordering is promised, everything else is optimizable; explain why per-thread dependency analysis is blind to other observers.
Discuss the engineering consequence — ordering must be expressed in code, not discovered by testing — and how you enforce that in a codebase through review rules, race detectors and portability targets across strong and weak architectures.
## The core idea The order in which memory operations appear in your source is not the order in which they execute, nor the order in which other threads observe them. That is not a toolchain bug; it is the bargain every mainstream language and CPU makes in exchange for speed. ## Two independent reorderers **The compiler.** An optimizer treats memory access as data flow, not as a script. It hoists a load out of a loop into a register (so a later change by another thread is never re-read), sinks a store past other work, merges two stores into one, deletes a store that is overwritten later, and reassociates independent expressions. Every one of these changes when — or whether — a value hits memory. **The hardware.** Modern cores post stores into a *store buffer* and retire the instruction before the value is visible to other cores; they execute independent instructions out of order; they prefetch and speculate past branches; some architectures let two loads complete out of order. The commit order of memory operations is not the program order. ## The only rule they honour: as-if-serial Both layers preserve the illusion for *one* thread in isolation: a thread always sees its own writes, and dependencies within the thread (this load feeds that add) are respected. Formally, they may reorder any two operations that are not related by a dependency the single-thread semantics can detect. The crucial gap: dependency analysis is per-thread. Two stores to *different* locations have no dependency, so their order is arbitrary from outside. Two loads of different locations, likewise. The optimizer literally cannot see the other thread that cares. ## What this breaks The classic publication idiom: ``` Producer: Consumer: data = 42 while (!ready) {} ready = true use(data) ``` Without synchronization, four things can go wrong: the producer's two stores commit in the opposite order; the consumer's `use(data)` load is hoisted above the loop by the compiler or executed early by speculation; the consumer's `ready` load is hoisted out of the loop into a register and spins forever; or `data` is read from a stale cached value. Any of these yields `use(0)` or a hang. The code is *racy*, and a racy program has no defined ordering at all. ## Where ordering comes from A memory model is a contract: the program marks the few places where cross-thread order matters, and the implementation is free everywhere else. The markers are synchronization — acquiring and releasing a lock, an ordered atomic read/write, a fence instruction, thread start/join. At those points the compiler suppresses motion across the marker and emits whatever hardware barrier the target architecture needs. Everything between markers stays fully optimizable, which is exactly why the model is worth having. ## Why 'it works on my machine' proves nothing Strong hardware (x86-family, TSO) hides most reordering, and an unoptimized debug build hides most compiler motion. The same source on a weakly ordered CPU, or at a higher optimization level, or after a JIT recompiles a hot loop, can expose it immediately. Testing cannot establish ordering; only the synchronization you wrote can. ## What to say in an interview Name both layers, state as-if-serial as the single constraint, point out that it is per-thread and therefore blind to other observers, and conclude that cross-thread order must be requested explicitly. That framing is the whole answer.
- If reordering is legal, why does most unsynchronized code still appear to work?Because the common desktop/server architecture is strongly ordered and hides nearly all hardware reordering, and because the racy window is often nanoseconds wide, so tests rarely hit it. Debug builds also suppress much compiler motion. None of that is a guarantee — the same source on a weakly ordered CPU or in an optimized build can fail immediately.
- Does marking one variable as an ordered/atomic write only affect that variable?No. An ordered write acts as a barrier for the operations around it: the writes that preceded it in program order are also made visible before it. That is what makes the publication idiom work — the ordinary payload writes ride along with the one ordered write, and the reader's matching ordered read exposes them.
An editor may reorder the paragraphs of a memo as long as the memo still reads correctly to one reader. If a second reader is peeking at the pages as they are typed, that reader sees whatever order the typist chose.
saying these in an interview costs you the question
- Believing reordering is only a hardware phenomenon and that the compiler preserves source order
- Thinking a thread might fail to see its own earlier writes
- Assuming that because two statements are adjacent, no other thread can observe them out of order
- Claiming a sleep, a print statement, or a longer loop 'fixes' the ordering
- Saying x86 has no reordering at all (store-to-load reordering is permitted there)