A limiter loads an atomic.Int64 count and an atomic.Pointer[Config] separately — why can the pair disagree?

## The shape of the bug An admission check on a hot path holds two pieces of state: an `atomic.Int64` counting in-flight work, and an `atomic.Pointer[Config]` holding settings that an operator can reload at run time. The check reads both: ```go n := inFlight.Load() c := cfg.Load() if n >= c.MaxInFlight { return errBusy } ``` Every access here is atomic and none of it is a data race. Yet the decision can be wrong. Between the two `Load` calls a reload goroutine can `Store` a new `*Config` whose `MaxInFlight` is half the old one. The check then compares a count taken under the old regime against a limit from the new one — a pair of values that never described the system at any single instant. Reverse the order of the reads and you get the mirror bug. This is the general truth about atomics, and it is the thing candidates most often miss: **an atomic operation protects one word, not an invariant.** Two atomic operations are two atomic operations. There is no implicit bracket around them. ## Why it is so hard to catch Three properties make this class of bug expensive: - **`-race` is silent.** The race detector reports unsynchronised access to the same memory. Here every access went through `sync/atomic`, so there is nothing for it to report. Passing `-race` is often taken as proof of concurrency correctness; it is proof of the absence of *data races* only. - **The window is tiny and the trigger is rare.** The two loads are adjacent instructions; the config reload happens once an hour. The bug fires when a preemption or a core migration lands exactly there, which is to say in production, at load, occasionally. - **The symptom is plausible.** Slightly too much traffic admitted just after a config change looks like an operator error, a caching delay, or an off-by-one — not like a memory-ordering question. ## The fix: one pointer, one immutable struct When several values must be read together, make them one value. Put them in a struct, treat that struct as immutable once published, and swap the whole thing with a single `atomic.Pointer[T]`: ```go type settings struct { limit int64 burst int64 } var cur atomic.Pointer[settings] func reload(limit, burst int64) { cur.Store(&settings{limit: limit, burst: burst}) } func allowed(n int64) bool { s := cur.Load() // one load: limit and burst are guaranteed to belong together return n < s.limit+s.burst } ``` One `Load` yields a pointer to one struct, and the fields inside it were fixed before the pointer was ever visible. Readers on the hot path pay a single pointer load and no lock. The discipline that makes it work is **write-once**. A published `*settings` is never mutated — not by the writer, not by a reader. A reload builds a fresh struct and stores the new pointer; goroutines still holding the old pointer keep using a fully valid older version and drop it when they return, and the garbage collector reclaims it. The moment somebody "just updates one field" on a published struct, the pattern is gone and you have a data race instead. ## The field that cannot live in the snapshot The counter is different from the config. A snapshot works for the config because it changes rarely and wholesale; the in-flight count changes constantly and must be a live shared value, so it cannot be frozen into an immutable struct. That leaves three honest options: 1. **Accept the skew deliberately.** For an advisory limiter, comparing a fresh count against a limit that may be a few nanoseconds stale is fine — but say so in a comment, so the next reader knows it was a decision and not an oversight. 2. **Reorder to make the skew safe.** Load the config *first*, then the count, so the count is always at least as new as the limit it is judged against. It does not make the pair atomic, but it makes the error direction predictable, which is sometimes all a limiter needs. 3. **Take a `sync.Mutex` across both.** If the pair really is an invariant somebody depends on, hold a lock over the read and over the update. Reaching for a lock after showing you understand why the atomics were not enough is a strong answer, not a retreat. ## `atomic.Pointer[T]` versus `atomic.Value` `atomic.Value` is the older way to publish an arbitrary value atomically. It stores an `any`, so it is checked at run time rather than compile time: `Store` panics if the value is nil, and it panics if the value's concrete type is inconsistent with what was stored before. It also boxes the value into an interface. `atomic.Pointer[T]`, added with the typed atomics in **Go 1.19**, is type-safe at compile time, cannot be given the wrong type, and reads as `*T` with no assertion at the call site. For new code publishing a struct pointer, `atomic.Pointer[T]` is the default and `atomic.Value` is what you meet in older code. ## The review heuristic When a change replaces a mutex with atomics, the question to ask is not "is each operation atomic?" but **"how many variables does one decision read, and can they change between the reads?"** One variable: atomics are a good fit. Two or more that must agree: either fold them into one value published through a single pointer, or keep the lock.