skip to content

What does Go's tiny allocator do, and which objects qualify for it?

level: middleimportance: nice to knowfreq 20%

answer

  1. some objects share one allocation
  2. there is a size ceiling and a type rule
  3. under 16 bytes, and nothing scannable
  4. the block is bump-allocated as it fills
  5. all its occupants must die together

basics

~20 s

Go's tiny allocator packs several heap objects that are smaller than 16 bytes and contain no pointers into one shared 16-byte block. It cuts allocation count for things like small ids and counters; anything holding a pointer never qualifies.

solid answer

~50 s

The runtime has a special path for objects that are **under 16 bytes and pointer-free**. Instead of giving each one its own slot, the P's cache keeps a current 16-byte block plus an offset and bump-allocates the next tiny object into whatever space remains, aligning it to its size; when it no longer fits, a fresh 16-byte block is taken from the ordinary 16-byte pointer-free size class. So three 4-byte ids can cost one allocation rather than three. The pointer-free requirement is not arbitrary: the collector's pointer information is per slot, and a block shared by several objects is a single object as far as it is concerned. That is also the catch -- the block is reclaimed only when *every* object carved from it is unreachable, so one retained id can keep its block-mates resident.

code

go · 8 lines
go
type seq uint32                       // 4 bytes, pointer-free
type datagramID [8]byte               // 8 bytes, pointer-free
type stamp struct{ sec, nsec uint32 } // 8 bytes, pointer-free

type labelled struct {
	id   datagramID
	name *string // a pointer: never tiny-allocated
}

go deeper

for a junior

Remember there is a special path for very small pointer-free objects that packs several of them into one 16-byte block, saving allocations.

for a middle

State both entry conditions -- under 16 bytes and no pointer fields -- and explain why the shared block is a single object as far as the collector is concerned.

for a senior

Be ready to say when it bites: retaining one tiny object keeps its block-mates resident, which reads as a slow, small leak in a service allocating ids per request.

for a principal

The point worth owning is that a runtime optimisation nobody can configure changes the retention profile of your data, so lifetime discipline has to live in the design rather than in a flag.

## The tiny allocator Go's heap allocator has three paths, not two: large objects (over 32 KB) straight from the page allocator, normal small objects into a size-class slot, and a third path for **tiny** objects. ### The entry conditions An allocation takes the tiny path when both hold: 1. Its size is **strictly less than 16 bytes**, and 2. Its type contains **no pointers** (the runtime calls this *noscan*). Both conditions are checked on the allocation fast path from information the compiler already computed for the type. A `[8]byte` datagram id, a bare `uint32` sequence number, a `struct{ sec, nsec uint32 }` timestamp all qualify. A 12-byte struct holding a `*string` does not, because of the pointer. A 16-byte struct of two `int64` fields does not, because 16 is not less than 16 -- it gets its own slot in the 16-byte class. ### The mechanism Each per-P cache holds a **current tiny block** -- one 16-byte object taken from the pointer-free 16-byte size class -- and an offset into it. A qualifying allocation is aligned according to its size (an 8-byte value gets 8-byte alignment, a 4-byte value 4-byte alignment) and carved out of the remaining space by bumping the offset. When the next request does not fit, the runtime allocates a fresh 16-byte block and starts over, keeping whichever of the old and new block has more room. So up to four 4-byte objects, or two 8-byte objects, can share a single allocation. In a service that allocates a small fixed-size id per incoming datagram, that is a direct cut in allocation count and in size-class bookkeeping, and it shows up as fewer bytes and fewer allocations per operation in a benchmark. ### Why pointer-free is mandatory The garbage collector tracks, for each span, whether its objects contain pointers and where. That information is per slot. If two objects shared a slot and one held a pointer, the collector would have to know which bytes within the shared block are pointers at what offsets -- exactly the per-object metadata the size-class design exists to avoid. Restricting the path to pointer-free objects means the shared block is simply a noscan object: the collector never looks inside it. ### The catch: shared lifetime Because the block is one object to the collector, it is freed only when **none** of the objects carved from it is reachable. If a service allocates a stream of small ids and stashes one of them in a long-lived index, the two or three unrelated ids sharing its block stay resident too. The waste is small per block, but it is proportional to how often you retain one tiny object out of many, and it makes memory look mildly, persistently higher than the live set suggests. This is not a bug to work around so much as a shape to know about. If you are retaining a value long-term, it is usually worth copying it into a structure you own -- a slice of ids you appended to, say -- rather than keeping the pointer to the freshly allocated tiny object, because the copy releases the whole block. ### What it is not It is not an API. There is nothing to enable, tune or call; the compiler and runtime decide from the type and the size. It is also not stack allocation: values the compiler can keep in a frame never reach the allocator at all, and the tiny path is specifically about things that did end up on the heap. ### How to reason about it in review When you see a per-request allocation of something small, ask two questions: does the type contain a pointer (a string field, a slice field, a map, an interface, a `*T`), and is it under 16 bytes? If both answers are favourable, the allocation is already cheaper than the allocation count suggests. If the type has a pointer field, it does not qualify and the object sits in its own slot with its own scanning cost -- which is one of the quieter arguments for keeping hot little value types free of pointer fields.

  • Why does a struct with a single pointer field never take the tiny path?
    Because the collector's pointer metadata is per slot, not per object. A block shared by several objects is one object to the collector, so it can only be treated as containing no pointers at all. Admitting a pointer-bearing object would require per-object pointer maps inside the block, which is exactly the metadata the size-class design avoids.
  • How can the tiny allocator make a program hold more memory, not less?
    The 16-byte block is reclaimed only when every object carved from it is unreachable. Retain one tiny object -- an id stashed in a long-lived map, say -- and the unrelated objects packed alongside it stay alive too. It is bounded and small per block, but in a service that retains one out of every few thousand tiny allocations it is a visible, permanent overhead.
  • Does a 16-byte pointer-free struct qualify?
    No. The threshold is strictly under 16 bytes, so a 16-byte struct gets its own slot in the pointer-free 16-byte size class. It still benefits from being pointer-free -- the collector skips its span when scanning -- but it is never packed with a neighbour.

It is carpooling for small objects. Several passengers share one car, which is cheap while they all get out at the same stop -- but if one stays aboard indefinitely, the car cannot be returned.

saying these in an interview costs you the question

  • Thinks any object under 16 bytes qualifies, pointers included
  • Believes tiny objects are reclaimed individually
  • Describes it as an API you can call or configure
  • Confuses the tiny path with stack allocation
  • Assumes it applies to small strings or slices