Modern JVM collectors perform most of their work concurrently with the running application but still take short stop-the-world pauses. What work is placed in those pauses, and what does moving work out of them cost the system?

## The question behind the question Asking why any pause remains is really asking which parts of collection are *inherently* global, and what price the runtime pays to shrink them. A principal-level answer names the residual work, explains why it resists concurrency, and then prices the alternative honestly. ## What genuinely resists being made concurrent ### 1. Capturing roots Marking starts from roots — references in thread stacks and registers, static fields, JNI handles. Reading a thread's stack requires that thread's state to be describable and stable, which means the thread must be at a safepoint. Historically this was done for all threads inside one global pause, and its cost grew with thread count and stack depth. The modern move is to scan each thread's roots via a **per-thread handshake**: the thread stops at its own next poll, its roots are processed, and it resumes — no global stop. This removes root scanning from the pause budget almost entirely and is a large part of why some collectors advertise pauses independent of heap size. What remains is a much cheaper global agreement. ### 2. Phase and barrier transitions A concurrent collector's correctness depends on all threads agreeing about which phase is active, because the barrier behaviour differs by phase — what a write barrier must record while marking is not what it must do while idle, and a relocation phase changes what a read must do with a reference. Switching that state needs an instant that all threads observe consistently. A short global safepoint is the simplest and cheapest way to establish it. ### 3. Terminating marking While marking runs concurrently, the application keeps mutating references, so the collector must consume whatever the barriers recorded. Because the application can generate new work while the collector drains it, marking needs a termination step where mutation is halted briefly to drain the residue and prove completion. Collectors keep this short by bounding how much can be outstanding, but a final short pause is the standard mechanism. ### 4. Relocation bookkeeping Moving collectors must ensure no thread uses a stale address. Concurrent relocation is possible using read barriers that forward accesses to the new copy, but installing and retiring the forwarding state is again a phase transition, and reference updating must be coordinated with the mutator. ## The costs you pay for concurrency ### Barrier overhead on the application This is the biggest and most persistent cost. To trace concurrently, the collector needs the application to report reference changes; to relocate concurrently, it needs reads to be corrected. Either way, ordinary field accesses in application code carry extra instructions. The cost is distributed across all execution rather than concentrated in a pause — which is exactly the point, but it means a fully stop-the-world collector can achieve higher raw throughput than a low-pause one on the same hardware. ### CPU contention Concurrent work runs on cores. On a saturated host, giving the collector threads means taking them from request handling, so latency can worsen even as pauses shrink. It also feeds back into safepoint arrival times: a starved application thread is slow to reach its next poll. Concurrency assumes spare CPU; without headroom it degrades. ### Floating garbage and the need for headroom Concurrent marking works against a logical snapshot. Objects that become unreachable after the snapshot is taken are not identified in that cycle — they *float* and survive to the next one. So a concurrent collector needs a larger heap for the same workload. Additionally, the cycle must complete before allocation exhausts the remaining space, which requires the collector to start early enough (pacing based on observed allocation rate) and to be given headroom to absorb prediction error. ### A worse failure mode A stop-the-world collector's bad day is a long pause. A concurrent collector's bad day is that the cycle loses the race: threads are throttled or stalled waiting for memory, or the collector falls back to a degenerate or full stop-the-world collection — which is both long *and* unexpected. Failure is less frequent but sharper, which matters for how you alarm and how much headroom you keep. ### Complexity and observability More moving parts, more heuristics, more failure modes. The relevant metric shifts from "pause length" to "did the cycle keep up", and monitoring must follow — allocation rate versus reclamation rate, cycle start time, stall counts. ## How to reason about the tradeoff The honest framing is a three-way budget between **pause time, throughput and memory**, at a given CPU allocation: - A batch job that cares only about total completion time is often best served by putting all the work in pauses: no barriers, no floating garbage, minimal heap. - A latency-sensitive service with tail-latency objectives buys short pauses with barrier overhead, extra heap and CPU headroom, and accepts lower peak throughput. - Under-provisioning CPU or heap while selecting a concurrent collector is the common mistake: it produces the worst of both — reduced throughput *and* stalls when the cycle cannot finish. And the residual pause is rarely the binding constraint once it is sub-millisecond. At that point the practical limits on stopped time are usually elsewhere: arrival at safepoints under CPU pressure, non-GC VM operations, and host-level effects such as page faults. A principal answer says so, because it directs effort where the remaining latency actually lives. ## Summary Pauses retain the work needing global agreement or stable per-thread state: root capture, phase and barrier transitions, and marking termination — with handshakes moving root scanning out of the global pause. The concurrency is paid for in barrier overhead, CPU, floating garbage and headroom, and it changes the failure mode from a long pause to an allocation stall. Choose the point on that curve deliberately, and provision for it.