A single shared atomic counter incremented on every request becomes a throughput bottleneck on a many-core server, and adding cores makes it worse rather than better. Explain the mechanism and describe the techniques that relieve it, with their costs.

## The mechanism: one line, exclusive ownership An atomic read-modify-write is not magic; it is implemented by the cache-coherence protocol. To perform it, a core must hold the target cache line in an exclusive/modified state, which requires invalidating that line in every other core's cache and pulling it across the interconnect. That is an operation measured in tens to hundreds of nanoseconds, versus about a nanosecond for a local cache hit. When many cores update the same counter, the line can only be in one place at a time, so it bounces from core to core — the classic ping-pong. Every update waits for a transfer, and each transfer invalidates the line for everyone else. The result is that a workload with no logical dependency between requests becomes serialised on a single memory location, and adding cores adds contenders rather than throughput: the curve rises, flattens, and then declines, because coherence traffic grows faster than useful work. If the counter uses a compare-and-swap loop rather than an unconditional add, it is worse still: losing attempts also acquire the line exclusively, so wasted work actively slows the eventual winner. ## Fix 1 — stop sharing the line (striping) The structural answer is to convert one contended location into many uncontended ones. Keep an array of cells; each thread maps to a cell by an identity-derived index; each update touches only that cell. Reading the total sums all cells. Three details make or break it: - **Pad each cell to its own cache line.** If two cells share a line, the cores updating them still fight over it even though the variables are logically independent — you have kept all the contention while adding memory. - **Choose the stripe count sensibly.** Roughly the number of cores that update concurrently; more stripes cost memory and make reads slower. - **Accept the read semantics.** Summing N cells that are being mutated concurrently gives a value that was never simultaneously true — it is consistent with *some* recent interleaving, not with any single instant. For metrics and statistics this is fine. For anything enforcing a limit, it is not. This is the idea behind striped-counter abstractions that platforms provide for high-contention statistics. ## Fix 2 — don't share at all If each worker keeps a plain, non-atomic local count and publishes it periodically or at shutdown, the hot path has zero atomic operations and zero coherence traffic. This is the cheapest option whenever a slightly delayed total is acceptable, which covers most counting. The cost is staleness proportional to the publish interval and the need to handle threads that die without publishing. ## Fix 3 — batch Accumulate locally and perform one atomic add per N events. Atomic traffic falls by a factor of N; the shared total lags by at most N per thread. This composes with striping and is often the simplest change to an existing hot loop. ## Fix 4 — reduce the damage of contention you keep When the shared update must stay: - **Prefer an unconditional primitive** (fetch-and-add) over a compare-and-swap loop, since it cannot fail and needs one operation per update. - **Back off** between retries — exponential with jitter — so losers stop stealing the line while a winner completes. - **Read before writing.** If the update is frequently a no-op (setting a flag already set, raising a maximum already higher), a cheap shared read avoids the exclusive acquisition entirely; shared reads can be replicated in many caches at once. - **Separate hot fields onto their own lines** so unrelated variables do not drag the line around. ## Fix 5 — relax the requirement Often the cheapest fix is to question the requirement. Do you need an exact count, or a rate? Sampling one in K events and multiplying, or using an approximate counting scheme, removes most of the traffic. Similarly, a limit enforced approximately per shard (each shard gets a quota) removes the global counter altogether at the price of imprecise global enforcement. ## What you give up Every technique here trades the same thing: a single exact, instantaneous, globally visible number. Striping and batching give you an eventually-accurate total; per-thread counters give you a delayed one; sampling gives you an estimate. So the design question is always "what is this number for?" A metric or a statistic tolerates all of it. A semaphore, a quota, or an admission limit does not — those need a genuinely shared decision point, and there the honest answer is to shard the *resource* so each shard has its own limit, rather than to weaken the counter. And measure first: a counter that is only touched a few thousand times a second is not a bottleneck, and striping it is complexity for nothing.