You operate several hundred small JVM services on shared nodes, each with a heap under 512 MB. How would you decide whether to standardize the fleet on the single-threaded stop-the-world collector, and what would you measure to defend the decision?

## The trap in the question "Standardize the fleet on one collector" sounds like an efficiency win, and at the level of *operational simplicity* it can be. But a collector is not a style choice; it is a resource-allocation policy. Answering with a single global verdict is the failure mode an interviewer is probing for. The strong answer sets a **default**, defines the **conditions under which it is wrong**, and specifies the **measurements** that decide. ## Start by observing what is already true With heaps under 512 MB, most of these containers are likely CPU-limited to one or two cores. HotSpot's ergonomics classify anything below roughly two available processors and ~1792 MB of usable memory as non-server-class and fall back to the single-threaded collector automatically. So the first task is not to choose but to **audit**: log the selected collector and detected limits for every service (`-Xlog:gc` at startup, `-Xlog:os+container=trace`), and find out how much of the fleet is already there by accident. Making an accidental behaviour explicit has real value: a capacity team raising a CPU limit from 1 to 2 can flip a service from a single-threaded to a region-based concurrent collector overnight, changing pause profile, thread count, and native footprint, with no code change and no reviewer. ## The measurements that decide **Live set.** Heap occupancy immediately after a full GC. A single-threaded compacting collection is proportional to it, so this number, more than heap size, predicts the pause. Collect it per service over a week under real traffic, not from a synthetic load test. **Pause distribution measured at the caller.** GC logs report the collector's own duration. Under a cgroup quota, GC work can exhaust the period's CPU budget and leave the container throttled afterwards, so the stall a client observes may exceed the logged pause. Compare server-side p99.9 with client-side p99.9. **RSS per instance.** Density on shared nodes is the point of running hundreds of small JVMs. Concurrent and region-based collectors carry heap-proportional marking structures, remembered sets, and more worker thread stacks — all off-heap and all charged to the container. Quantify the delta; on small limits it can be tens of megabytes per instance, which multiplies across the fleet into whole nodes. **CPU-seconds per request.** Concurrent collectors insert barriers into application code and run collector threads alongside it. On a fleet, that tax shows up as aggregate CPU cost, i.e. money, even where latency is fine. Measure it per request, not as a percentage of a busy loop. **Process lifetime and restart rate.** Short-lived and frequently redeployed processes favour the collector with the cheapest startup and no background cycle to complete. ## Shaping the policy Tier by SLO, not by team or language version: - **Background / batch / sidecars** — default to the single-threaded collector with an explicitly pinned heap. Their latency budget is minutes; density and footprint dominate. This is where standardization genuinely pays. - **Normal request-serving** — default the same way, but hold a gate: if measured full-GC pauses exceed some fraction of the p99 budget, the service must either shrink its live set or move up a resource tier. - **Latency-critical** — require a floor of at least two cores and enough heap headroom before a concurrent collector is allowed, because below that floor concurrency is fictional and merely steals mutator CPU. Approve on measured p99.9, not on collector reputation. Encode the default in the base image or platform manifest so it is versioned and reviewable, and make overrides explicit and annotated with the measurement that justified them. ## Guardrails to ship with the policy - Alert on **full-GC frequency and duration** per service; a rising live set is the leading indicator that a service has outgrown its tier. - Alert on **container RSS approaching the limit**, since a kernel OOM-kill produces no Java diagnostics. - Re-run the audit whenever CPU or memory limits change, because ergonomics may silently disagree with your pinned intent if the flag is not actually set. - Keep a small canary set per tier where you can A/B collectors with real traffic; fleet-wide changes should be validated on live workloads, not benchmarks. ## What "defending the decision" looks like A defensible answer is a table: per SLO tier, the chosen collector, the measured p99.9 and full-GC distribution, the RSS and CPU-per-request delta, and the resulting node density and cost. The judgment is not "single-threaded collectors are good for small heaps" — it is "for these services, the pause cost is inside budget and buys measurable density, and here is the trigger that moves a service out of this tier."