Why can Counter.most_common(20) reorder tied segments between runs of a translation-memory updater?

## What most_common actually promises `Counter.most_common(n)` returns a list of `(element, count)` pairs, highest count first; with no argument it returns all of them. The documented tiebreak is the one people forget: **elements with equal counts are ordered in the order first encountered**. That guarantee arrived in Python 3.7 (before that, the ordering among ties was unspecified), and it holds both for `most_common()` and for `most_common(n)`. "First encountered" means insertion order in the underlying dict — `Counter` is a `dict` subclass, so it inherits the language-level ordering guarantee that dicts preserve insertion order. ```python from collections import Counter Counter("mississippi").most_common(2) # [('i', 4), ('s', 4)] ``` Both `i` and `s` occur four times; `i` wins the tie only because it appears earlier in the input string. ## Why that becomes a production bug A translation-memory updater rebuilds a segment-frequency Counter on boot — a 45-second cold start that walks a corpus directory, reads each file and counts segments. The top-20 list feeds a cache warmer, a report and a golden-file test. Nothing about the *data* changed between two deployments, but the ranking did. The candidates: 1. **Ingestion order changed.** Directory iteration order is not sorted unless you sort it. Add a file, restore from a different snapshot, or move to a filesystem that enumerates differently, and the first-encounter order of every segment shifts. All counts are identical; the tie order is not. 2. **The shards were merged differently.** `a + b` builds its result by walking `a`'s items first (keeping those that stay positive) and then appending keys that occur only in `b`. So the merge order of shards determines key order in the merged Counter, and therefore the tie order out of `most_common()`. 3. **Parallel ingestion.** If workers hand back partial Counters that arrive in completion order, the merge order is effectively random per run. The failure is nasty because it is silent and intermittent: a test that has passed a hundred times fails once on a machine whose directory listing differs, and the counts in the failure message look identical to the expected ones. ## The fix: make the tiebreak explicit If a ranking crosses a boundary — persisted, diffed, asserted on, shown to a user — the total order must be a function of the data alone: ```python top = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))[:20] ``` Negating the count sorts descending while keeping `sorted` ascending overall, and the element itself supplies a deterministic secondary key. Because `sorted` is stable and the key is now a total order over distinct elements, the result depends only on the counts and the element values. Two smaller variants are worth knowing. If elements are not comparable with each other (mixed types), sort on a derived key that is — a string form, or an id. And if you genuinely only need *a* top-20 and reproducibility does not matter, `most_common(20)` is faster than sorting everything, since it does a partial selection rather than a full sort. ## Sorting is not the whole answer Making the order deterministic does not make it *meaningful*. If ranks 18 through 25 all have the same count, position 20 is a cut through a plateau: your list is stable but the boundary is arbitrary, and one extra occurrence tomorrow reshuffles it. A ranking that feeds a human decision usually wants either the count displayed alongside, so the tie is visible, or a real secondary signal — recency, segment length, source weight — instead of an alphabetical fallback chosen only to be deterministic. ## Related ordering facts worth having ready - `Counter.elements()` yields in the same first-encountered key order, repeating each element by its count. - The Counter `repr` is built via `most_common()`, so it, too, shows counts descending — which is why a Counter *looks* sorted even though the mapping is in insertion order. - Iterating the Counter directly (`for k in c`) gives insertion order, not count order. Code that assumes iteration is ranked is wrong. ## What to say in an interview State the rule (count descending, ties by first encounter, guaranteed since 3.7), name ingestion order and merge order as the two things that change it, and show the explicit-key sort as the fix. Then add the judgement line: determinism and meaningfulness are different problems, and a plateau at the cut line needs a real secondary signal rather than an alphabetical one.