skip to content

You are choosing between a moving collection strategy (copying or compacting) and a non-moving one (mark-sweep) for a runtime. Lay out the trade-offs that decide it, including effects on the allocation path, fragmentation risk, and code that holds raw object addresses.

level: principalimportance: nice to knowfreq 34%

answer

  1. moving -> bump allocation + no fragmentation
  2. non-moving -> stable addresses + free lists
  3. moving cost ~ survivors; sweep cost ~ heap size
  4. moving needs precision, safepoints, pinning/handles
  5. identity hash and lock state must survive relocation

basics

~20 s

Moving buys bump-pointer allocation, no fragmentation, and better locality, paid for with relocation cost proportional to live data, precise reference information, and address instability that native code must tolerate. Non-moving keeps addresses stable and collection cheap per cycle, paid for with free-list allocation and fragmentation that can eventually block large allocations.

solid answer

~60 s

Frame it as four coupled decisions. **Allocation path.** Moving strategies leave contiguous free space, so allocation is a bump pointer - a few instructions, trivially thread-local via allocation buffers. Non-moving strategies require a free list with size classes and coalescing; allocation is a search and is harder to make cheap. **Fragmentation risk.** Non-moving heaps fragment monotonically under mixed object sizes and long uptime, so large contiguous allocations can fail while total free memory looks fine. Something must eventually compact, which usually means an occasional very expensive pass. **Cost model.** Moving costs scale with *surviving* data, so it wins where survival is low (young generations) and loses where the live set is large and dense. Non-moving sweep costs scale with heap size and are indifferent to survival rate. **External constraints.** Moving requires precise identification of every reference, safepoints where stack maps are valid, and a story for native code holding raw pointers (pinning or handle indirection); header state such as identity hash and lock information must survive relocation. Most real designs refuse the binary choice: move where survival is low, avoid moving where the live set is dense, and keep compaction as a fallback.

go deeper

for a junior

Know the headline: moving collectors compact the heap and make allocation a simple pointer bump; non-moving ones leave addresses alone but fragment.

for a middle

Add the cost asymmetry - relocation scales with surviving data, sweeping with heap size - and explain why fragmentation makes large allocations fail.

for a senior

Bring in the runtime obligations of relocation (precise reference maps, safepoints, pinning or handles for native code, identity hash and lock state) and the fragmentation contingency plan a non-moving design needs.

for a principal

Give a decision rule tied to workload evidence - survival rate, allocation size distribution, interop surface, uptime, latency budget - and note that the choice is made per heap area rather than once globally.

## Why this is a real design fork "Does the collector relocate surviving objects?" is the single decision that most shapes a memory manager. It determines the allocation instruction sequence, the failure modes under long uptime, the constraints on the rest of the runtime, and the shape of pauses. ## The allocation path After a moving collection, free memory is contiguous. Allocation reduces to: read a pointer, add the size, compare against a limit, store. It parallelises trivially by giving each thread its own buffer carved from the contiguous space, so the common path needs no synchronisation. Initialisation is also cheaper because freshly bumped memory is contiguous and cache-friendly. A non-moving heap cannot offer this. Free memory is a set of holes, so the allocator maintains size-class lists, must choose a fitting hole, and must coalesce neighbours to fight fragmentation. Every one of those is more instructions on the hottest path in a managed runtime, and each is harder to make lock-free. ## Fragmentation as a long-run failure mode Fragmentation is not merely wasted bytes. Its practical signature is an allocation failure for a large contiguous object while total free memory is comfortable, arriving after days of uptime and after all the load testing passed. A non-moving design must have an answer: either a compaction fallback (correct but a long pause exactly when the system is already under pressure), size-segregated allocation that bounds worst-case waste, or an application-level guarantee about object size distribution - which is rarely available. ## The cost model Moving costs are proportional to what survives. Copying an area where 2% survives is nearly free; compacting an old generation that is 85% live is close to worst case, because almost everything is read, written, and re-pointed to reclaim little. Non-moving sweeping is proportional to heap size and is indifferent to survival rate. This asymmetry is why generational designs apply copying where survival is low and something else where it is high, rather than picking one strategy for the whole heap. ## The constraints moving imposes on the rest of the runtime 1. **Precision.** A non-moving collector can be conservative: if a stack word might be a reference, treat it as one; the only harm is retaining an object slightly too long. A moving collector cannot guess - rewriting a word that was actually an integer corrupts data. So it needs precise stack maps and object layout maps, and it can only relocate at points where those maps are valid, which is what safepoints are for. 2. **Native code.** Code outside the managed world may hold raw addresses. Options are pinning (the object cannot move while a native reference exists), handle indirection (native code holds a slot that the collector updates), or copying data across the boundary. All of them impose API design constraints. 3. **Header state.** Identity hash codes must remain stable even though addresses change (typically by materialising and storing a hash on first use), and lock or monitor state must travel with the object. 4. **Reference-holding structures elsewhere.** Anything caching an address rather than a reference - certain off-heap indexes, some JIT-compiled constants - needs an update protocol. ## Reading the workload - Very high allocation rate with very low survival: moving is strongly favoured; the collection is cheap and the allocation path is the hottest code in the system. - Large, dense, long-lived live sets: the reward for moving shrinks while its cost grows; incremental or region-wise relocation becomes preferable to whole-space compaction. - Heavy native interop with long-lived direct pointers: the pinning/handle burden pushes toward non-moving, or toward a design with an explicit pinning region. - Mixed and unpredictable object sizes with multi-month uptime: fragmentation risk pushes toward having compaction available, whether continuous or as a fallback. ## The answer to actually give A principal-level answer refuses to declare a winner and instead states the decision rule: relocate where survival is low and the allocation path dominates cost; avoid relocating where the live set is dense or where external code depends on address stability; and never ship a non-moving design without an answer for the day fragmentation blocks a large allocation. Note also that the choice is not global - real systems apply different strategies to different parts of the heap, which is exactly why generational and region-based designs exist.

  • Why can a non-moving collector be conservative about what is a reference while a moving one cannot?
    A conservative collector may mistake an integer for a reference; the only consequence is retaining an object that was actually dead, which is safe. A moving collector would rewrite that word with a new address, corrupting whatever value it really held. Relocation therefore demands precise maps of where references live, and it may only happen at points where those maps are known valid.
  • How do runtimes preserve object identity hash codes when objects move?
    The hash cannot be derived from the address, because the address changes. The usual approach is to compute a value on first request and then store it - in the object header, or in a side table - so subsequent requests return the same value regardless of relocation. The cost is header space or an extra word for objects whose identity hash is ever taken, which is why implementations compute it lazily rather than for every object.

Renumbering desks in an office: doing it keeps seating tidy and makes assigning a new hire trivial, but every directory, sign, and phone list that referenced the old numbers must be updated - and anyone outside the company who wrote a number down is now wrong unless you gave them a name to look up instead.

saying these in an interview costs you the question

  • Declaring moving collection universally superior without acknowledging that its cost scales with live data.
  • Ignoring that relocation requires precise reference information and safepoints rather than being a pure implementation detail.
  • Assuming native code can keep raw object pointers across a moving collection with no pinning or handle mechanism.
  • Treating fragmentation as merely wasted memory rather than a source of allocation failures under long uptime.
  • Presenting the choice as one global decision, when real runtimes apply different strategies to different parts of the heap.

context