skip to content

How do Go's mcache, mcentral and mheap cooperate on a small heap allocation?

level: middleimportance: must knowfreq 50%

answer

  1. three tiers, one lock-free
  2. the fast path belongs to a P
  3. refill when the current span fills up
  4. one shared structure per size class
  5. the global tier owns pages and arenas

basics

~20 s

Each P owns an mcache holding one span per size class, so the usual allocation is a lock-free pop. An exhausted span is refilled from that class's shared mcentral, which takes fresh spans from the global mheap.

solid answer

~50 s

Go's allocator is a three-tier thread-caching design. The fast path is the **mcache**: every P has one, holding a current span for each size class, and because a P is run by one thread at a time the mcache needs no lock -- compute the size class, pop the next free slot, done. When that span has no free slots, the mcache refills from the **mcentral** for that class, a process-wide structure with lists of partly and fully used spans, behind a lock that is contended only when several Ps want the same class at once. If the mcentral has nothing free, it asks the **mheap**, which owns the page allocator and the address space reserved from the OS, for a fresh span. Objects over 32 KB skip both caches and come straight from the mheap as their own run of pages.

code

go · 9 lines
go
var sink *[24]byte

func BenchmarkAllocParallel(b *testing.B) {
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			sink = new([24]byte)
		}
	})
}

go deeper

for a junior

Know the order of the three tiers by name: an allocation tries the per-P mcache first, then the mcentral for its size class, then the global mheap.

for a middle

Explain why the fast path needs no lock, what a refill actually does, and where the per-class lock is the one that can be contended.

for a senior

Reason about the allocator under load: which tier serialises, why a 1 MB buffer takes a different path from a 32-byte struct, and what GOMAXPROCS costs in cached spans.

for a principal

Own the framing that allocation throughput is a design property rather than a tuning knob -- the tiering is not configurable, so the only lever teams have is allocating less and allocating uniformly.

## The three tiers Go's heap allocator is a thread-caching allocator in the tradition of tcmalloc, adapted to the runtime's own scheduling units. It has three levels, and the whole design exists so that the overwhelmingly common case touches no lock. ### mcache -- per P, no lock Every P (the scheduling context; there are `GOMAXPROCS` of them) owns an **mcache**. The mcache holds, for each size class, a pointer to a span it is currently allocating out of. The fast path for a small object is: 1. Round the requested size to its size class. 2. Take that class's span from the current P's mcache. 3. Pop the next free slot from the span's free bitmap and return its address. 4. Zero it, unless the type does not require zeroing. There is **no lock and no atomic** on this path. The reason is structural rather than clever: a P is held by exactly one OS thread at a time, only one goroutine runs on a P at a time, and the allocation fast path runs with preemption disabled. The mcache therefore has exactly one user for the duration, so ordinary loads and stores are safe. A consequence worth stating in an interview: cached spans are per P, so the runtime's idle allocator memory scales with `GOMAXPROCS`. On a machine with many cores, raising it raises the floor of memory sitting in caches, in exchange for less contention. ### mcentral -- per size class, locked When an mcache's span for a class has no free slots left, the mcache must **refill**. It goes to the **mcentral** for that class -- there is one mcentral per class, shared by all Ps -- and asks for a span with free slots. The mcentral keeps lists of spans that have free slots and spans that are full, and hands one over under a lock. This is the first point where two Ps can serialise, and only when they want the same size class at the same moment. Refills are rare relative to allocations (a 24-byte span serves 341 objects before it needs replacing), so the lock is cheap in aggregate. Sweeping also runs here: as the garbage collector sweeps a span and frees slots, the span moves back onto the mcentral's list with free slots. Each size class actually has two variants -- one for objects containing pointers and one for pointer-free ("noscan") objects -- so the collector can skip whole spans it never needs to scan. That doubles the number of mcentrals and the number of spans an mcache holds. ### mheap -- global, owns pages and address space If the mcentral has nothing with free slots, it asks the **mheap**. The mheap owns the page allocator: a map of which 8 KB pages of the heap's address space are in use. It finds a free run of pages, marks the span as belonging to that size class, records the metadata, and hands it up. If there are no free pages at all, the mheap reserves more address space from the operating system in large arenas and maps it. The mheap is also where **large objects** -- anything over 32 KB -- are served from directly. They bypass the mcache and mcentral entirely, get their own span of whole pages, and are swept individually. That is why a single 1 MB buffer and thirty thousand 32-byte structs behave nothing alike from the allocator's point of view even though they consume similar memory. ### Why the design looks like this The shape follows from Go's concurrency model. Goroutines are cheap and numerous, so a per-goroutine cache would be absurd; threads come and go, so a per-thread cache would be awkward; but Ps are a small, bounded, runtime-controlled set that already serialises execution. Attaching the cache to the P gives lock-free allocation with bounded memory overhead for free. The practical implication for a service is that allocation throughput scales with cores until refill traffic starts hammering one mcentral, which happens when every P is churning objects of the same size class as fast as it can. Even then the tier you are contending on is one lock per class, not a global heap lock -- the thing many engineers coming from a hand-rolled `malloc` mental model expect. ### What you can actually control Almost nothing about the tiering itself; it is not configurable. The levers are on your side of it: allocate fewer objects, allocate them in uniform sizes so spans fill and drain together, and keep large buffers large rather than splitting them into many small ones that spread across classes.

  • What exactly happens when the mcache's span for a size class has no free slots?
    The mcache refills. It hands the full span back and asks that class's mcentral for a span with free slots, taking the mcentral's lock; the mcentral may have to sweep a span first. If the mcentral has none, it asks the mheap for a fresh run of pages, which grows the heap from the OS if the page allocator has nothing free. Only then does the original allocation complete.
  • Why can a P allocate from its mcache with no lock and no atomic operation?
    Because the mcache has exactly one user at a time by construction. A P is held by one OS thread, only one goroutine runs on a P at once, and the fast path runs with preemption disabled, so no other execution context can touch that mcache concurrently. The synchronisation comes from the scheduler's ownership model, not from instructions in the allocator.
  • How does GOMAXPROCS affect the allocator's memory overhead?
    There is one mcache per P, and each can hold a span for every size class in both its pointer and pointer-free variants. Raising `GOMAXPROCS` therefore raises the amount of memory sitting in caches rather than in use, which is a real floor on a many-core machine. It is a throughput-for-footprint trade, not free.

A chef's own mise en place bowl (mcache), the kitchen's shared tray of that ingredient (mcentral), and the walk-in store room (mheap). Reaching into your own bowl needs no coordination; opening the store room needs the most.

saying these in an interview costs you the question

  • Says every Go allocation takes a global heap lock
  • Thinks the mcache is per goroutine rather than per P
  • Believes the runtime calls libc malloc underneath
  • Cannot say where objects over 32 KB come from
  • Claims the mcentral is also per P