Why does a tracing collector's cost rise faster than linearly as the heap approaches its live set?
answer
- the curve has a knee
- free space is the denominator
- the last gigabyte costs most
- live set over free space
- collector loses the allocation race
basics
~20 sCollector work per allocated byte is the live set divided by the free space, and free space is the quantity being removed. As the denominator shrinks toward zero the ratio grows without bound, so the last gigabyte taken away costs far more than the first.
solid answer
~40 sWork per allocated byte is about `L / (H - L)` for a live set `L` in a heap `H`. Shrinking `H` attacks the denominator, so the curve is hyperbolic rather than straight. With a 2 GB live set: at 20 GB the ratio is `0.11`; at 8 GB it is `0.33`; at 4 GB it is `1.00`; at 2.5 GB it is `4.00`. Removing 12 GB tripled the cost; removing the next 1.5 GB quadrupled it again. The practical consequence is a knee, not a slope - a heap can be trimmed repeatedly with barely visible effect and then fall off sharply. A collector doing concurrent work degrades more abruptly still, because past some point it loses the race against allocation and falls back to a stopping collection.
go deeper
Remember that taking memory away from a collected service does not slow it down evenly - the first cut may cost almost nothing and a later, smaller cut may cost a great deal.
Explain why: the cost is live set divided by free space, and shrinking the heap attacks the denominator, so the curve bends upward rather than sloping.
Show the operating consequence - a knee rather than a slope, a peak live set rather than an average as the basis, and a concurrent collector's fallback to a long stopping collection when it loses the race.
Set the fleet policy in terms of a headroom ratio with a defined margin, and state what evidence would justify moving the ratio in either direction.
## The denominator is what you are taking away The amortized cost model for a tracing collector is `work per allocated byte = L / (H - L)`, where `L` is the live set and `H` the heap. Tracing cost per cycle follows `L`; what a cycle yields is the free space `H - L`. Adding memory grows the denominator and the ratio falls gently. **Removing** memory shrinks the denominator toward zero, and a function with a shrinking denominator does not fall off linearly - it climbs hyperbolically. ## The numbers, for a 2 GB live set | heap | free space per cycle | work per allocated byte | |---|---|---| | 20 GB | 18 GB | 0.11 | | 8 GB | 6 GB | 0.33 | | 4 GB | 2 GB | 1.00 | | 2.5 GB | 0.5 GB | 4.00 | | 2.1 GB | 0.1 GB | 20.00 | Read the steps in the order an operator would actually take them: 1. Trimming 20 GB to 8 GB gives back **12 GB** and triples collector cost from `0.11` to `0.33` - still a small number in absolute terms, easy to miss on a dashboard. 2. Trimming 8 GB to 4 GB gives back **4 GB** and triples it again. 3. Trimming 4 GB to 2.5 GB gives back only **1.5 GB** and quadruples it. Each step returns less memory than the one before and costs more than the one before. That is the signature of a knee: several successful reductions are not evidence that the next one is safe. ## Why a concurrent collector falls further A stopping collector meets the squeeze by running more often - unpleasant, but continuous. A collector doing its tracing alongside the application has a deadline: it must finish before the program exhausts the free space that existed when the cycle began. Squeezing the heap shortens that deadline from both ends, because there is less free space and the same allocation rate consumes it sooner. When the deadline is missed, there is no graceful option. The collector stops the application and does a full, usually compacting, collection - a stall potentially orders of magnitude longer than its normal ones. So the degradation is a slope followed by a step, and the step lands exactly on the service that was squeezed because latency mattered. ## The measurement traps - **The live set is not a constant.** It varies with traffic, with in-flight work, and with caches that fill over hours. Free space is the difference between two numbers, so a 10% error in `L` becomes a much larger error in `H - L` once the heap is tight. - **A quiet-hour reading flatters you.** Size from the peak live set measured **after** a collection has completed, not from an average, and not from a sawtooth peak that includes uncollected garbage. - **Occupancy percentage reads backwards.** A heap reported at 95% occupied is not 'efficient'; it is a service sitting at the steep end of this curve with almost no margin for a traffic change. - **Successful past reductions are not a safety argument.** The curve is shallow for a long way and then is not; three reductions that worked tell you nothing about the fourth. ## What to do with it Treat headroom as a ratio to the live set rather than an absolute number of gigabytes, because that is what the arithmetic depends on. Reduce it in steps, measuring collector processor share and stall behaviour under peak load at each step, and stop while the measured ratio is still comfortably on the flat part of the curve. When the live set itself grows - a new cache, a larger working set, a traffic pattern that retains more - remember that the heap did not change but the position on this curve did.
- What does this curve imply for a heap sized from a measurement taken at a quiet hour?The quiet-hour live set is the trough. At peak the live set is larger, so free space is smaller than planned and the service sits further up the knee than the sizing suggested. Size from the peak live set measured after a collection, and express headroom as a multiple of it.
- Why does a concurrent collector degrade more abruptly here than a stopping one?A stopping collector simply runs more often as free space shrinks - a slope. A concurrent one must finish tracing before allocation exhausts the free space that existed at the start of the cycle; when it cannot, it falls back to a stopping, compacting collection, which is a step change in stall length.
- Does reducing the allocation rate move the service along the same curve?It does not change the curve's shape, but it changes how fast the service travels it: the cost per allocated byte is unchanged while the bytes allocated per second fall, so collector processor share drops proportionally. It also widens the deadline a concurrent cycle has to meet.
saying these in an interview costs you the question
- Expects memory reductions to trade against throughput linearly.
- Sizes a heap from an average live set rather than the peak.
- Thinks a heap at 95 percent occupancy is merely slightly worse.
- Assumes a collector that kept up before will keep up after.
- Reads high heap occupancy as efficient use of memory.