skip to content

A sync.Pool of output buffers accepts rare 30 MB buffers alongside typical 4 KB ones — why does memory stay high, and what guard fixes it?

level: seniorimportance: should knowfreq 33%

answer

  1. the pool never says no
  2. capacity comes back, not length
  3. one cache per P, not one per pool
  4. check the capacity before you Put

basics

~20 s

sync.Pool has no size limit: it retains whatever you Put until a collection clears it, and it caches per P, so several oversized buffers stay live at once. Guard the release path — drop any buffer whose capacity exceeds a threshold.

solid answer

~50 s

A reused buffer remembers its capacity, not the size of the last thing written into it, so a buffer that grew to 30 MB for one large document is still a 30 MB buffer after a reset. Put it back and the pool holds 30 MB; the next 4 KB request gets that buffer, resets it and puts it back — nothing in the cycle ever shrinks it, so one outlier promotes itself into the steady state. It also multiplies, because `sync.Pool` caches per P, so you can accumulate roughly one oversized buffer per P rather than one in total. The collector does clear pooled entries, so an idle service releases them, but under continuous traffic each buffer is fetched and returned before it can age out. The fix is a comparison on release: `if b.Cap() > maxPooled { return }`, letting the rare huge buffer become ordinary garbage. Everything in one pool should cost roughly the same.

code

go · 13 lines
go
const maxPooled = 64 << 10 // keep only ordinary-sized buffers

var bufPool = sync.Pool{
	New: func() any { return new(bytes.Buffer) },
}

func release(b *bytes.Buffer) {
	if b.Cap() > maxPooled {
		return // rare huge render: let the collector take it
	}
	b.Reset()
	bufPool.Put(b)
}

go deeper

for a junior

Know what sync.Pool is for — handing objects back so they can be reused instead of reallocated — and that anything you Put stays in memory until a garbage collection clears it out.

for a middle

Explain that the pool retains an entry's capacity rather than its used length, and that it keeps caches per P, so what you Put decides the footprint rather than what a typical request needs.

for a senior

Show the diagnosis and the fix together: memory tracking the largest documents rather than the typical ones, and a capacity check on the release path that drops rare oversized buffers for the collector.

for a principal

Own the sizing policy — which threshold, derived from which measured distribution, and whether the service should bound input size upstream or stream the output rather than absorbing 30 MB renders at all.

## What `sync.Pool` will and will not do for you A `sync.Pool` is a free list of reusable objects. `Put` offers an object for reuse; `Get` returns some previously offered object, or calls `New` (or returns nil, if `New` is unset) when it has nothing to hand back. It keeps its entries in caches attached to each **P** — the runtime's scheduling contexts, of which there are `GOMAXPROCS` — so `Get` and `Put` are cheap and scale with cores. What it explicitly does **not** have is a size limit. There is no maximum entry count, no byte budget, no eviction policy you can configure, and no accounting of how much memory an entry costs. Whatever you `Put` is held, alive and reachable, until a garbage collection drops the pool's contents (a victim cache gives entries one extra cycle of grace before they go). ## Why one huge buffer becomes a permanent floor A reusable buffer remembers its **capacity**, not the size of the last thing written into it. A buffer that grew to 30 MB while rendering one unusually large document is, after a reset, still a 30 MB buffer. `Put` it back and the pool now holds 30 MB. The next request needs 4 KB, gets the 30 MB buffer, uses a fraction of it, resets it, and puts it back — still 30 MB. Nothing in the cycle ever shrinks it, so a single outlier promotes itself into the steady state. It also multiplies. The pool caches per P, and different Ps handle different large documents, so over time you can accumulate roughly one oversized buffer per P rather than one in total. The footprint is closer to *peak buffer size x number of Ps* than to *typical buffer size*. The collector does clear pooled entries, so an **idle** service releases the memory. Under continuous traffic it does not help: every big buffer is fetched and put back well before it can age out, so resident memory sits at the high-water mark for as long as the load lasts. And it does not look like a leak — the memory is genuinely reachable from the pool, so it is live, not lost. ## The fix: guard the `Put` The remedy is one comparison on the release path. Anything above a threshold is simply not returned to the pool; it becomes ordinary garbage and the collector handles it well, since it is one large object with no scanning work if it is a byte buffer. ``` const maxPooled = 64 << 10 func release(b *bytes.Buffer) { if b.Cap() > maxPooled { return } b.Reset() bufPool.Put(b) } ``` The rule of thumb behind it: **everything in one pool should cost roughly the same**, because the pool cannot distinguish entries and will happily keep the most expensive one. If you genuinely need reuse at several scales, run separate pools per size class and pick one by the requested size — but that bookkeeping is rarely worth it compared with letting the rare large allocation happen. ## Choosing the threshold Take it from the real size distribution rather than a round number: pick a value comfortably above the ordinary high-water mark, so the common path is always served from the pool, and below what you are willing to hold multiplied by `GOMAXPROCS`, which is the worst case the pool can retain. Note that on modern Go the default `GOMAXPROCS` in a container follows the cgroup CPU limit, so the multiplier is the one the container actually gets, not the host's core count. ## Related levers, and when to reach for them instead - **Bound the work upstream.** If 30 MB renders are legitimate, decide whether the service should accept them at all, or stream the output through an `io.Writer` in fixed-size chunks so no single buffer ever reaches that size. A streaming design removes the problem rather than capping it. - **Do not pool at all.** If large buffers are the common case rather than the outlier, a pool is holding your peak permanently; allocating per request and letting the collector reclaim is often the smaller footprint. - **Pool a pointer type**, such as `*bytes.Buffer` or `*[]byte`, rather than a bare slice value — handing a slice to `Put` is not free. ## What an interviewer is listening for That you identify capacity, not length, as what comes back; that you know the pool has no bound and is cleared only by the collector; that you say per-P caching multiplies the retention; and that your fix is a guard on the release path rather than a call to `runtime.GC` or a `GOGC` change, which would empty the whole pool to solve a problem caused by one entry.

  • How would you choose the threshold?
    From the real size distribution rather than a round number: high enough that ordinary work is always served from the pool — above the everyday high-water mark — and low enough that the threshold multiplied by GOMAXPROCS, which is the worst case the pool can hold, is memory you are willing to spend. In a container the multiplier is the CPU limit the container actually gets.
  • Does the garbage collector free those big buffers eventually anyway?
    Yes. A pool's contents are dropped at collection, with a victim cache giving entries one extra cycle of grace, so an idle service releases them. The problem is steady state: under continuous traffic each oversized buffer is fetched and put back long before it can age out, so the footprint persists for as long as the load does.
  • Why not force a collection or lower GOGC instead?
    Both are blunt: they empty the whole pool, throwing away the ordinary entries that were doing useful work, and they pay a global cost to solve a problem caused by one entry. A capacity check on the release path costs a comparison, keeps the common path allocation-free, and bounds the retention deterministically.
  • When should you not use a pool for these buffers at all?
    When large buffers are the common case rather than the outlier — the pool would then hold your peak permanently. It is also the wrong tool if the output can be streamed: writing through an io.Writer in fixed-size chunks means no single buffer ever reaches 30 MB, which removes the problem instead of capping it.

It is a coat check with no shelf limit: someone hands in a wardrobe trunk once, and from then on every customer who wants a coat hook is given the trunk back, resets it, and hands it in again.

saying these in an interview costs you the question

  • Says the collector shrinks pooled buffers back to their used size
  • Thinks sync.Pool has a maximum size you configure
  • Believes the New function controls what Get returns when the pool is non-empty
  • Claims GOGC tuning will keep pooled buffers small
  • Calls it a leak rather than retained, reachable memory