A daily aggregation job deletes every entry from its Go map, yet the heap never drops — why?

## The behaviour A Go map allocates storage as it fills and **never releases it**. There is no shrink threshold, no compaction pass, no automatic downsizing when the entry count falls. The map's memory footprint tracks its **high-water mark**, not its current `len`. This is a deliberate design choice, not an oversight. Shrinking would mean rehashing every surviving entry into smaller storage — an unpredictable pause — and would punish the extremely common pattern of a map that is emptied and refilled repeatedly. ## What delete and clear actually do `delete(m, k)` removes the entry and clears the memory of the slot's key and value. That part matters: if the key or value contained pointers — strings, slices, structs with references — those references are dropped, so the objects they pointed at become collectable. What is *not* released is the table storage itself: the slots and their bookkeeping stay allocated, marked available for reuse. `clear(m)` empties the map in one operation, with the same outcome for capacity: entries gone, capacity kept. That is the right behaviour when you refill the same map every request or every loop iteration — no reallocation, no rehashing. It is the wrong tool when the goal is to give the memory back. So an emptied map is a strange object: `len(m) == 0`, and it is still hundreds of megabytes. ## The batch-job shape of this bug A job that aggregates daily log files into a map keyed by a `(tenant, region)` struct is the classic case. On an ordinary day the key cardinality is modest. Then one day a misbehaving producer emits a region field derived from something unbounded — a request id, a hostname with a random suffix — and the map explodes to millions of distinct keys. The job finishes, empties the map, and moves to the next file. Memory does not come back, and every subsequent run in that process carries the peak. If the process is long-lived, the spike is permanent for its lifetime; the next spike ratchets it higher. ## Confirming it is the map You want evidence, not a hunch: - **Check the shape of the claim first.** `len(m)` near zero while the process's in-use heap stays flat across an explicit collection is already suggestive. For a batch job, `runtime.GC()` followed by `runtime.ReadMemStats` and a look at `MemStats.HeapAlloc` before and after emptying the map is a two-line experiment. - **Reproduce it in a benchmark.** Fill one map to N entries, empty it, and repeat across a growing N, run with `-benchmem`. The bytes-per-operation figure tracks the peak N rather than the final size, which is the behaviour stated back to you as a number. - **Check key cardinality, not just size.** The interesting question is usually *why* the map got that big. Logging the distinct key count per run turns a memory mystery into a data-quality bug. ## The fixes, in order of preference **1. Give the map a shorter life.** Build a new map per run, per file, or per window, and let the previous one become garbage: ```go counts = make(map[key]int, expected) ``` Once nothing references the old map, the collector reclaims it whole. This is nearly always the right answer for a batch job, and it costs one allocation per run. **2. Bound the cardinality.** If the key can be unbounded, make it bounded on purpose: validate the region field, or fold rare pairs into an explicit `other` key once the distinct count crosses a threshold. This turns an unbounded growth risk into a known, capped one, and it usually improves the output too — a report with two million one-row groups was not useful anyway. **3. Rebuild a long-lived map periodically.** If the map genuinely must stay live as a cache, copy the survivors into a right-sized new map when the live count falls far below the peak, and swap it in. You pay one copy and get the footprint back. **4. Shard by time.** Keep several maps, one per window, and drop whole windows. Deleting a map is free in a way that deleting its entries is not. ## What not to do - Calling `runtime.GC()` does not shrink a live map. The map is reachable; the collector is doing exactly its job. - Looping `delete` over every key is strictly worse than `clear`, and neither returns capacity. - Setting the map variable to `nil` *does* work — it makes the map unreachable — but only if no other reference remains, and you then have to remake it before the next write, since writing to a nil map panics. ## The one-sentence version **Maps grow and never shrink; to get the memory back you must throw the map away, not empty it.** Say that, then say how you would verify it, and the answer is complete.