HotSpot's Concurrent Mark-Sweep collector had to begin its concurrent old-generation cycle well before the old generation was actually full. Why could it not simply wait until the space ran out, and what were the costs of starting too early or too late?

## The race a concurrent collector runs A stop-the-world collector has a simple contract: when space runs out, stop everyone, reclaim, resume. It can afford to wait until the last moment because nothing is allocating while it works. A concurrent collector cannot. Its marking and sweeping overlap with application execution, and during that window application threads keep allocating and keep promoting survivors out of the young generation into the old one. The cycle therefore has to be *started early enough that the space remaining when it starts covers everything allocated before it finishes*. That is the whole reason an initiating threshold exists. The required headroom is roughly the promotion rate multiplied by the cycle duration, plus a margin. Both inputs move with load: a traffic spike raises the promotion rate, and CPU contention lengthens the cycle, so the two errors compound exactly when the system is busiest. This is why concurrent-collector incidents cluster at peak. ## How the threshold was chosen By default CMS ran an adaptive heuristic, estimating from recent history how long a cycle takes and how fast the old generation grows, and triggering when the projection said it should. Operators frequently overrode it because the estimate reacted too slowly to bursty workloads: - `-XX:CMSInitiatingOccupancyFraction=<percent>` set the old-generation occupancy at which a cycle begins; - `-XX:+UseCMSInitiatingOccupancyOnly` told the JVM to use that number every time instead of only as the first cycle's seed. Tuning meant watching the GC log: if concurrent-mode failures appeared, lower the fraction (start earlier) or add heap; if the collector was cycling constantly on a heap that never came close to filling, raise it. ## Floating garbage and why headroom is structural Marking concurrently means marking a moving target. An object that is reachable when the marker visits it, and becomes garbage a millisecond later, is still marked live for this cycle — the collector cannot un-mark it safely once the marking wavefront has passed. That object is **floating garbage**: real garbage that survives until the next cycle collects it. The practical consequence is that a concurrent collector's effective live set is always larger than the application's true live set, by an amount proportional to allocation rate and cycle duration. Heap sizing must include it. A heap sized to the true live set plus a thin margin will start failing cycles under load even with a perfectly tuned threshold, because floating garbage eats the margin. ## The two failure directions **Starting too late** is the dangerous one. The old generation fills mid-cycle, the concurrent attempt is abandoned, and the JVM performs a stop-the-world mark-sweep-compact full collection — in older JDKs single-threaded, so its duration scales with the entire live set. A service with 30 ms pauses suddenly shows a multi-second freeze. Because the trigger is load-correlated, one failure often begets more: the long pause backs up queues, the backlog raises allocation rate, and the next cycle is even more likely to lose its race. **Starting too early** is merely wasteful, but the waste is real. Each cycle costs CPU for the marking threads and produces two stop-the-world phases of its own. On a machine where the application already uses most of the cores, that CPU comes directly out of application throughput. Continuous cycling — sometimes visible as back-to-back cycles in the log with no idle gap — means the collector is consuming a constant slice of the machine to keep pace. ## Fragmentation interacts with the timing question Even a perfectly timed cycle can be defeated by the collector's other weakness. Because CMS swept in place and never compacted, the old generation's free space fragmented over time, so a promotion could fail for lack of a *contiguous* chunk at an occupancy level the threshold considered comfortable. Timing tuning cannot fix that; only compaction can, and CMS compacted only in the fallback full collection. ## The generalisable lesson Every concurrent collector after CMS inherits this same structure — pick a trigger point, keep enough headroom to cover a cycle, and fail over to something drastic if the race is lost. The parameters differ and the modern collectors do far more of it adaptively, but the reasoning transfers: **a concurrent collector converts heap headroom and CPU into pause-time reduction**. If you refuse to pay in either currency, you get the long pause back at the worst possible moment. When capacity-planning a latency-sensitive JVM service, the question is not "what is the average pause" but "how much headroom does the collector need to keep the fallback from ever firing under peak load".