Why can an LLM's emergent ability vanish when the benchmark metric changes?

## What the emergence claim says The standard framing of an emergent ability is: a capability that is not present in smaller models, is present in larger ones, and cannot be predicted by extrapolating the smaller models' performance. The plots that made this famous show near-chance accuracy across several model sizes and then a steep rise — the visual signature of something appearing rather than improving. The claim matters because it implies you cannot forecast: you must build the bigger model to discover what it can do. ## The metric-choice critique The critique is that many of these curves are artifacts of how the task is scored. Two ingredients produce a fake discontinuity. **Nonlinear scoring.** Suppose a task requires four sub-decisions and is scored by exact match — all four must be right. If per-field accuracy across three model sizes is 0.5, 0.7 and 0.9, the exact-match scores are 0.5^4 = 0.06, 0.7^4 = 0.24 and 0.9^4 = 0.66. The underlying competence rose linearly; the reported metric rose 4x then 3x and looks like an ignition. Nothing appeared. A scoring rule that multiplies probabilities together will always compress low competence towards zero and expand high competence, and the compression is what reads as "absent in smaller models". **Insufficient resolution.** If the true accuracy of the smaller model is 1% and your eval set has 100 items, you will usually record zero. Zero looks like inability. Enlarge the set or use a continuous score — the probability the model assigns to the correct completion, for instance — and the smaller model turns out to be measurably, if weakly, on the right track. The distinction between "cannot" and "can, rarely" is exactly the distinction the emergence claim depends on, and a small test set erases it. Apply either fix — partial credit per field, token edit distance, Brier score, log-likelihood of the target — and a substantial share of published emergent curves flatten into ordinary smooth trends. ## What survives the critique Three things, and a candidate who states only the debunking is answering half the question. 1. **Some abilities are genuinely non-smooth in the ways that matter.** The critique shows that many reported discontinuities are metric-induced; it does not show that none are real, and it does not cover every observed jump. 2. **All-or-nothing is often the metric the product needs.** If your pipeline routes a claim only when all four extracted fields are correct, exact match *is* your business metric. The smooth underlying curve is scientifically truer and operationally irrelevant. The honest statement is: the capability improves smoothly, and the usable threshold is crossed abruptly. 3. **Unpredictability is still partly real.** Knowing that the underlying curve is smooth lets you extrapolate the underlying quantity, but the mapping from that quantity to "does the feature work" still contains a threshold you have to locate empirically. ## How to interrogate an emergence claim When someone shows you a step-change plot, work through this: - **What is the y-axis?** Exact match, all-of-N, or any thresholded score is a warning sign. Ask for a graded score on the same runs. - **How many items?** Near-zero accuracy on a small set carries no information about whether the ability exists at low levels. - **Was the prompt held constant?** Different shot counts, formats or instructions across model sizes confound the comparison entirely. - **Is there a continuous proxy?** The probability assigned to the correct answer is available even when the sampled answer is wrong, and it usually reveals the smooth trend hiding under the step. - **Does the jump reproduce?** A single run at high sampling temperature on a small set produces plenty of accidental steps. ## Why an interviewer asks this They want to hear you read a benchmark number sceptically. The weak answer treats a plot as a fact about the model. The strong answer separates three things: the model's underlying competence, the scoring rule applied to it, and the product threshold that decides whether a feature ships. Confusing any two of those is how teams end up planning a roadmap around a capability that either was already there or never arrived.