skip to content

Why is a reflect-based struct mapper slower than hand-written field assignment, and where does the allocation come from?

level: middleimportance: must knowfreq 48%

answer

  1. compile time versus run time
  2. the parameter is an interface
  3. boxing allocates, per value
  4. the same type walked ten million times
  5. cost per field times rows

basics

~20 s

Reflection moves work from compile time to run time. Every call boxes values into interfaces, re-inspects the type, and reaches fields through calls that cannot inline. The boxing is what allocates, once per value converted.

solid answer

~50 s

Hand-written `dst.Name = src.Name` compiles to a copy at a byte offset the compiler already knows, and it inlines. The reflective version derives that at run time. `reflect.ValueOf` takes an `any`, so the argument is boxed, and because that parameter escapes, the boxed copy is heap-allocated. Each `Value.Field(i)` is a real function call that checks a kind and a set of flags before building a new `reflect.Value`. Every field handed back out through `Value.Interface()` is boxed again. On top of that, a naive mapper re-walks the destination type on every call, rebuilding descriptors identical to the previous row's. Per call the difference looks small; on a nightly batch of ten million rows with a dozen fields each it is a hundred million-plus dynamic accesses and the garbage behind them, so the collector becomes part of the bill.

code

go · 9 lines
go
func columns(row any) []any {
	v := reflect.ValueOf(row)
	t := v.Type()
	out := make([]any, t.NumField())
	for i := 0; i < t.NumField(); i++ {
		out[i] = v.Field(i).Interface() // boxed again on the way out
	}
	return out
}

go deeper

for a junior

Be ready to say that reflection does at run time what the compiler normally does at compile time, and that this costs both speed and compile-time type checking. Knowing it is slower, and roughly why, is enough here.

for a middle

You are expected to name the mechanics: the any parameter that boxes and escapes, field access through non-inlinable calls, and a type walk repeated on every call. Say which of those allocates.

for a senior

Show that you size the cost against the workload. The same mapper is fine in a start-up config decoder and ruinous over ten million rows, and you should be able to say what evidence would settle it.

for a principal

Frame reflection as buying flexibility with run-time cost and lost compile-time checking, and be ready to say where in a codebase that trade is worth making versus where it should never have been allowed on a hot path.

## The compile-time version When you write `dst.Name = src.Name`, the compiler knows both types completely. It knows `Name` sits at a fixed byte offset in each struct, it knows the field is a `string`, and it emits a few instructions that copy the string header from one offset to another. There is no lookup, no run-time check, and nothing to allocate. The whole assignment can be inlined into its caller, so there is not even a call. ## What `reflect` has to rebuild at run time A reflective mapper recreates that knowledge on every call, and there are four separate costs hiding in it. **1. Boxing at the door.** `reflect.ValueOf` has the signature `func ValueOf(i any) Value`. Passing a concrete value converts it to an interface, and the interface's data word has to point somewhere, so a non-pointer-shaped value is copied to a fresh allocation. Because `reflect.ValueOf`'s parameter escapes, escape analysis cannot keep that copy in the caller's frame; `go build -gcflags=-m` will report the argument as escaping. Passing a pointer instead (`reflect.ValueOf(&row)`) avoids that particular copy, because a pointer already fits in the data word — but the struct it points at is now reachable from a `reflect.Value` and generally escapes too. **2. Dynamic field access.** `v.Field(i)` is an ordinary, non-inlinable function call. It bounds-checks the index, reads the field descriptor out of the type, folds in flags recording whether the result is addressable and whether it came from an unexported field, and constructs a new `reflect.Value` describing the field. That is tens of instructions and a couple of loads where the static version had one move. **3. Boxing on the way out.** `Value.Interface()` converts the field back into an `any`. Anything not already pointer-shaped allocates again — so an `int` column, a `time.Time`, a small struct each cost a heap object per row. **4. Re-deriving the same plan.** The mapper written the obvious way calls `v.Type()` and loops over `t.NumField()` on every single call, producing exactly the descriptors it produced for the previous row and throwing them away. ## Why this sinks a batch and not a config loader Cost per operation only means something multiplied by the operation count on the hot path. A configuration file decoded once at start-up pays all four costs and nobody can measure it — reflection is the right tool there, and its flexibility is worth real money. A nightly job mapping ten million database rows onto typed structs multiplies every one of those costs by rows times fields. Twelve fields per row is 120 million dynamic accesses; if two of the steps allocate, that is a few hundred million heap objects for a non-generational collector to trace, and the job becomes GC-bound rather than database-bound. The tell in a `-benchmem` run is that `allocs/op` scales with fields per row, not with rows alone. ## The order in which the costs come off They do not all yield to the same fix, and knowing which is which is the point of this topic: - Cost 4 — the repeated type walk — disappears entirely if you build a plan once per `reflect.Type` and cache it. This is cheap, local, and needs no change to the build. - Costs 1 and 3 — the interface conversions — survive the cache. They are inherent to crossing the `any` boundary, so they only go away if the code stops being dynamic: generated per-type mappers, or type parameters where the shape allows. - Cost 2 — dynamic field access — shrinks with a cached index path but never reaches a static field offset. So the honest framing in an interview is: caching removes the repeated *discovery*, code generation removes the *dynamism*. Reach for the first before the second, and only reach for the second with a measurement in hand. ## Answers that sound confident and are wrong "Reflection is resolved at compile time so it costs nothing" — no, that is the definition of what it is not. "It is slow because `reflect` takes a lock" — there is no lock on the read path. "It reads the original value in place, so there is no copy" — a value passed as `any` is copied into the interface. "Escape analysis keeps it on the stack anyway" — it is precisely where escape analysis gives up.

  • Does passing a pointer, `reflect.ValueOf(&row)`, get rid of the allocation?
    It removes one of them. A pointer already fits in an interface's data word, so converting `*Row` to `any` needs no heap copy of the struct. What it does not remove is the escape — the struct is now reachable from a `reflect.Value`, so it generally cannot stay in the frame — nor the dynamic field access, nor any boxing you do on the way back out with `Value.Interface()`.
  • The mapper's author says it is only about a hundred nanoseconds per call, so it cannot matter. Why can that still sink a nightly batch?
    Because the cost is per field per row, not per call site. Ten million rows with a dozen fields is well over a hundred million dynamic accesses; at a hundred nanoseconds that is minutes of CPU, and the allocations behind them pull the garbage collector in on top. Per-operation cost only means something multiplied by the operation count on the hot path.
  • How would you show a reviewer that the mapper, and not the database driver, is doing the allocating?
    Exercise the mapper alone over rows already held in memory and read `allocs/op` and `B/op` from a `-benchmem` run, then compare the same rows through a hand-written mapper for that struct. If allocations track fields per row rather than rows alone, the boxing inside the mapper is the source, and the driver is not implicated.

Static field access is a warehouse worker who has memorised the shelf number. Reflection is one who re-reads the whole floor plan for every single item, and photocopies each item before handing it over.

saying these in an interview costs you the question

  • Says reflection is resolved at compile time, so it is free
  • Claims reflect reads the original value with no copy
  • Blames a lock inside the reflect package for the slowdown
  • Thinks the cost is one lookup per call, not per field
  • Assumes escape analysis keeps reflected values on the stack
  • Cannot separate the repeated type walk from the interface boxing