You measure a service's throughput at increasing concurrency levels and find it rises to a peak around 24 in-flight workers, then declines as you push higher. How would you use scalability modelling to decide between buying more hardware, partitioning the system, or redesigning the hot path?

## Read the curve before spending anything A throughput-versus-concurrency curve has three regimes, and the regime dictates the remedy: - **Rising, near-linear**: you are under-provisioned. Adding capacity works. Buy hardware. - **Flat plateau**: something serializes. Extra workers queue behind a shared resource. Hardware buys nothing; splitting the resource does. - **Rising then falling (retrograde)**: coordination cost is growing faster than capacity. Extra workers make throughput *worse*. A peak at 24 followed by decline is the third case, and that single observation eliminates the most commonly proposed fix. Buying machines when the curve is retrograde funds the decline. This is the judgment the question is really testing: recognizing that the shape of the curve, not the size of the incident, chooses the strategy. ## Make the measurement trustworthy first Before acting, verify the axis: - Sweep concurrency, not offered load, and measure at steady state per level with warm caches and pools. - Keep the workload mix constant. A different read/write ratio is a different system with different coefficients. - Cross-check with **Little's law**: `L = lambda x W` — average in-flight requests equals arrival rate times average residence time. If your client says 24 in flight but rate x latency says 60, you are measuring queueing somewhere you did not intend, and the whole curve is suspect. - Confirm the decline is not an artifact: timeouts and retries can manufacture a fake retrograde by amplifying load, and thermal throttling or a noisy neighbour can fake one too. Retries especially create a feedback loop that mimics coherence collapse; rule them out before concluding anything about architecture. ## Separate the two loss terms Fit the sweep to a model with both penalty terms — contention, whose cost grows linearly with worker count, and coherence, whose cost grows quadratically because keeping N workers' view of shared state consistent is a pairwise problem. The relative size of the two coefficients is the diagnosis: **Contention-dominated (plateau).** One resource serializes everything: a global lock, one connection pool, a single-threaded coordinator, one hot partition, one disk. Remedies split the resource: finer-grained locking, per-shard queues, more connections, removing the coordinator from the request path, replacing a global counter with striped counters. These are usually contained changes with predictable payoff. **Coherence-dominated (retrograde).** Workers are reconciling shared mutable state: cache-line ping-pong on a hot object, cross-node invalidation, distributed locking, consensus per request, chatty gossip. Remedies must reduce *sharing itself*: - **Partition by key** so each item has exactly one owner and cross-worker coordination disappears rather than getting faster. - **Batch or amortize** coordination — one reconciliation per hundred operations instead of per operation. - **Per-worker accumulators** merged once at the end, converting continuous crosstalk into a single reduction. - **Relax consistency** where the domain tolerates it — eventual consistency, conflict-free merge, read-your-writes only where required. - **Independent replicas** with no shared state, so capacity scales by replication instead of by concurrency. Note the asymmetry that should drive the decision: contention fixes make the plateau higher, while coherence fixes change the *shape* of the curve. Because the peak concurrency scales with the inverse square root of the coherence coefficient, halving that coefficient buys roughly 40% more peak concurrency — and driving it to zero removes the decline entirely, leaving only a plateau. Structural sharing removal has a qualitatively different payoff from tuning. ## Sequence the work by time-to-value 1. **Hours — cap concurrency.** Enforce an admission limit at or slightly below the measured peak. Bounded queues, a semaphore at the entry point, load shedding with a clear rejection response. This converts a collapsing system into a merely saturated one, and a saturated system with correct backpressure degrades predictably instead of falling over. Do this even while you plan the real fix. 2. **Days — attack contention.** Usually localized, low-risk, measurable. Often raises the peak enough to buy planning time. 3. **Weeks to quarters — remove sharing.** Partitioning is an architectural change with data-migration, routing, and rebalancing consequences. Justify it with the fitted coefficient, not with intuition, and state the expected new peak so the outcome is falsifiable. ## Frame the decision economically Each option has a cost and an expected effect on the curve: - **More hardware**: cheap to try, zero or negative effect on a retrograde curve. Rejected on evidence, not on principle — and worth saying explicitly, because "we measured it and it makes things worse" is a far stronger argument to stakeholders than "scaling out won't help." - **Splitting the bottleneck**: moderate effort, raises the plateau, does not remove the decline if coherence is present. - **Partitioning / removing sharing**: high effort, changes the shape of the curve, and is the only thing that lifts a hard ceiling. - **Reducing per-request work**: always helps, orthogonal to all of the above, and sometimes moves the deadline far enough that no restructuring is needed this year. The strongest answer names the measurement that would change the recommendation, commits to a predicted post-fix peak, and re-runs the same sweep to confirm. Success is a demonstrably higher peak concurrency and a flatter decline beyond it — not a single faster benchmark run.