Why does a Python dict raise RuntimeError when a payroll import deletes stale keys mid-iteration, and how do you fix it?

The error is `RuntimeError: dictionary changed size during iteration`, and it is a deliberate guard rather than an accident. **The mechanism.** When you write `for key in d`, Python calls `iter(d)` and gets a `dict_keyiterator`. That iterator stores a reference to the dict and records the dict's current size. On every `__next__` call it re-reads the dict's size and compares it with the recorded one; if they differ, it raises `RuntimeError` instead of continuing. The same guard sits in the value and item iterators, so `for k in d.keys()`, `for v in d.values()` and `for k, v in d.items()` all behave identically — obtaining a view first does not help, because a view is a live window on the same dict, not a copy. The guard exists because a dict is a hash table. Deleting an entry leaves a tombstone; inserting may push the table past its load factor and trigger a resize that reallocates the entries array and moves every entry. An iterator holds a position into that array. If the array is rebuilt underneath it, the position now points somewhere meaningless, and without the check you would get silently duplicated keys, silently skipped keys, or memory unsafety. Fail-fast is the cheaper contract. **What does *not* raise.** Reassigning the value of a key that already exists does not change the size and does not resize the table, so this is legal and safe: ```python for key in totals: totals[key] = round(totals[key], 2) ``` Only adding or removing keys trips the guard. **The scenario.** A nightly payroll CSV import loads each row into a dict keyed by employee id, and the same dict is then served to lookups by the payroll API, which peaks around 1,200 requests per minute. A cleanup pass was written as `for emp_id, row in rows.items(): if row.stale: del rows[emp_id]`, and it dies on the first stale row. The team's first instinct — wrapping the loop body in `try` / `except RuntimeError: continue` — is the wrong fix twice over: the exception comes from the iterator, not the body, so the loop simply ends early and leaves stale rows in place, and the ones it processed are whatever happened to come first. **The fixes, in order of preference.** 1. **Rebuild.** `rows = {k: v for k, v in rows.items() if not v.stale}` — one pass, no mutation of the dict being read, and because dicts are insertion-ordered (guaranteed since Python 3.7) the survivors keep their original CSV order. This is the fix to reach for when other code depends on the order, which it did here: the export step wrote columns in `rows` order. 2. **Snapshot the keys.** `for emp_id in list(rows):` materializes a list of keys first, so the loop iterates the list while mutating the dict. Slightly cheaper in memory than rebuilding for a large dict with few deletions, and the right choice when you must mutate in place because other references point at the same dict object. 3. **Collect then delete.** `stale = [k for k, v in rows.items() if v.stale]` followed by `for k in stale: del rows[k]`. Two passes, but the intent is explicit and the predicate is testable on its own. 4. **Drain.** `while rows: k, v = rows.popitem()` never creates an iterator at all, so the guard cannot fire. Use it when you want to consume the dict entirely. **The check is not a correctness guarantee.** It compares sizes only. A loop that deletes one key and inserts another on the same pass leaves the size unchanged, slips past the guard, and iterates a rearranged table with no error at all: ```python rows = {"a": 1, "b": 2, "c": 3} for key in rows: del rows[key] rows[key.upper()] = 1 ``` That runs clean and produces a dict of entirely different keys. Do not treat the absence of a `RuntimeError` as proof that mutation during iteration was safe. For the same reason, the guard is a debugging aid, not a synchronization mechanism: it says nothing about correctness when a second thread is mutating the dict. **Neighbouring behaviours worth knowing.** A `set` raises the parallel `RuntimeError: Set changed size during iteration`. A `list` raises nothing at all — `for x in L: L.remove(x)` walks by index, so removing an element shifts the rest down and the loop silently skips every other item, which is the quieter and nastier bug. Given the choice, the dict's loud failure is the friendlier design.