Why does a CPython dict still hold a 2.4 GB table after a document-conversion queue deletes every finished job?

### The signature A conversion service keeps per-job state in a long-lived dict keyed by job id. A backlog pushes concurrency to millions of in-flight jobs, the backlog drains, every finished job is deleted — and the process still sits on a 2.4 GB working set with a dict whose `len()` is in the hundreds. Nothing is leaking in the reference-counting sense: the job objects really were freed. What did not go away is the dict's own table. ### What deletion actually does A CPython dict is a sparse index array over a dense, append-only entries array. `del d[key]` does three things: it clears the entry's key and value slots in the entries array, it writes a *deleted* marker into the index slot, and it decrements the live count. It does **not** move later entries down, it does **not** rewind the append cursor, and it does **not** reallocate either array. The hole stays. The deleted marker is necessary rather than sloppy: an open-addressed probe sequence that hit a truly empty slot would stop early and fail to find keys inserted after the deleted one. Marking preserves probe chains. The practical consequence is stark: ```python import sys d = {i: i for i in range(100_000)} peak = sys.getsizeof(d) for k in list(d): del d[k] print(len(d), peak, sys.getsizeof(d)) # 0 5242960 5242960 ``` Zero entries, five megabytes of table. ### Why an insertion does not necessarily fix it The intuitive repair — "put something back in and it will right-size itself" — usually does nothing: ```python d["next"] = 1 print(sys.getsizeof(d)) # still 5242960 ``` A resize is triggered only when an insertion finds no usable slot left. Filling 100,000 entries into a table of 262,144 slots leaves tens of thousands of usable slots unconsumed, and deletion does not hand any back — the append cursor only ever moves forward. So one insert simply takes the next unused slot. When the usable slots finally do run out, the rebuild is sized from the number of **live** entries, roughly three times that count rounded up to a power of two. That is the moment a dict can get smaller, and it genuinely does: ```python for i in range(200_000): d[i] = i del d[i] print(len(d), sys.getsizeof(d)) # 1 224 ``` This is the important nuance for diagnosis. A dict under *steady churn* — add and delete at similar rates — repeatedly exhausts its usable slots, compacts on the resulting resize, and stays roughly proportional to its live size. The failure mode is the opposite shape: a **burst then quiet**. Peak, drain, and no further insert pressure means no resize, and the peak-sized table is retained for the life of the dict. ### Confirming it Compare the reported size against the length. `sys.getsizeof(d)` reports the dict object plus the table it owns (not the keys and values themselves), so a dict with `len(d) == 300` reporting tens of megabytes is conclusive on its own. To find *which* mapping, walk your long-lived registries and print `len` alongside `sys.getsizeof`, or take `tracemalloc` snapshots either side of a drain and diff them: a retained table shows as an allocation that never goes away rather than as growth. One honest caveat before blaming the dict for the whole 2.4 GB: freeing objects returns memory to the interpreter's allocator, which only returns an arena to the operating system once that arena is entirely empty. Fragmentation alone can keep RSS high after a burst even with no retained table. Measure the container before you conclude. ### The fixes **Rebuild.** `d = dict(d)` builds a fresh, right-sized table from the live entries and drops the old one. Cheap, obvious in review, and the standard repair for a mapping that has passed its peak. **Clear when draining.** `d.clear()` releases the table outright and resets the dict to its empty state — a 50,000-entry dict goes from 2.6 MB to 64 bytes. Use it when the whole generation of state is done, not per key. **Bound the mapping.** The design fix. If the dict tracks in-flight work, its size is a capacity decision: cap concurrency, evict on completion into a bounded structure, or move the state out of process. A per-job registry that can reach millions of entries is an unbounded queue with extra steps. **Do not reach for `gc.collect()`.** The table is reachable and refcounted; there is no cycle to break and collection will not shrink it.