skip to content

Inside one shared generic body that reaches its type's operations through a passed-in table, what does each operation cost?

level: middleimportance: should knowfreq 45%

answer

  1. one more hop to reach the code
  2. target unknown while the body compiles
  3. inlining is what is really lost
  4. uniform representation adds a data hop
  5. hot loops feel it, setup calls do not

basics

~10 s

Each operation becomes an indirect call: the body loads the entry from the table and jumps to it, so the optimiser cannot inline the target, fold its result, or specialise the loop around it.

solid answer

~50 s

One extra level of indirection per operation. The body holds a table, loads the entry for the operation and calls whatever address it finds, so the target is unknown while the body is being compiled. The jump itself is usually cheap; what costs is everything that follows from knowing a target — inlining, folding a width or a mask into a constant, hoisting an invariant check out of the loop, unrolling or vectorising. Values often travel in a uniform representation as well, adding a second hop to reach the data. A copy generated for one argument turns each of those calls into a direct, inlinable one. Some run-time optimisers close part of the gap by noticing that a hot shared body always receives the same table and emitting a guarded specialised version — a bet made while running, not a guarantee made at build time.

code

pseudocode · 13 lines
pseudocode
// shared body: both operations go through the table, every sample
function sumWindow(window, ops):
    total = ops.zero()
    for each slot in window:
        total = ops.add(total, ops.read(slot))   // two indirect calls per sample
    return total

// copy generated for one sample type: the same two operations, inlined
function sumWindow_for_reading(window):
    total = 0
    for each slot in window:
        total = total + (slot.high * 256 + slot.low)   // no call at all
    return total

go deeper

for a junior

Remember that reaching an operation through a table means the code being called is decided while the program runs, so the compiler has less to work with than when the target is known.

for a middle

Name the blocked optimisations, not just the extra hop: inlining, constant folding, hoisting and loop transformation all need a known target, and the call boundary also forces values out of registers.

for a senior

Size the cost before acting on it. Count operations per unit of real work, read a profile, and separate a genuinely hot per-sample loop from setup calls where the indirection never mattered.

for a principal

Decide how much of your platform's performance you are willing to leave to a run-time optimiser's speculation, given that it costs warm-up and disappears wherever a call site really is polymorphic.

Take the decoder's inner loop on a field controller: for every slot in a telemetry window it calls two operations of the placeholder sample type — read the slot, then combine the result into a running total. How much that loop costs depends entirely on whether the body was compiled as a copy for one concrete sample type or as one shared body that receives the type's operations in a table. ## What the body executes in each case Compiled as **one shared body**, neither call has a known target. The body holds a table of the argument type's operations; for each call it loads the relevant entry and jumps to the address it finds. Compiled as **a copy for one sample type**, both calls name code the compiler can see, and after inlining there may be no call left at all. ## The jump is the small part Engineers often reduce this to "an extra pointer hop", and that part is genuinely minor: the table entry can frequently be loaded once and hoisted out of the loop, and an indirect jump that lands on the same target every iteration predicts well on most machines. The real bill is what an unknown target forbids: - **Inlining.** The operation's body cannot be pulled into the loop, so the loop stays a sequence of opaque calls. - **Constant folding.** A width, a mask or a bound that would have become a literal inside a generated copy stays a value fetched at run time. - **Hoisting.** A check that is invariant across iterations cannot be proven invariant through an unknown call. - **Loop transformation.** Unrolling and vectorising need to see the work; a call boundary hides it. - **Register pressure.** A call through a table is still a call, so live values obey the calling convention and get spilled around it. An inlined operation imposes no such boundary. ## A second indirection: reaching the data A body compiled without knowing its argument's layout also cannot address fields directly. Platforms solve this in one of two ways: hold every value behind a uniform, fixed-size handle the body can move blindly — which costs a load before any field is touched — or pass the size and layout in alongside the operations, which keeps the data inline but makes every offset a computed value rather than a constant. Either way, the shared body works from information it fetches; the generated copy works from information the compiler already had. | | shared body plus table | copy generated per argument | |---|---|---| | target known while compiling | no | yes | | inlining of the operation | not before run-time specialisation | available | | loop transformations around it | limited | available | | reaching a field of the value | through a handle or a computed offset | a constant offset | ## Where it shows up, and where it does not The honest way to size this is to count how many times the operation runs per unit of real work. Two operations per sample across a window of ten thousand samples, executed for every frame, is twenty thousand opaque calls where a generated copy would have had a tight arithmetic loop; that difference is measurable and sometimes dramatic. The same two operations called once per frame to set up a decoder are noise beside the decoding itself. "A shared body is slower" is not a useful claim; "a shared body puts an unoptimisable call in my innermost loop" is. ## Recovering part of the gap while running A run-time optimiser can watch a hot shared body, notice that the same table arrives on every entry, and emit a copy specialised to that table behind a guard: if the table matches, run the fast copy; if not, fall back to the general one. This recovers inlining and much of what follows from it, but it is a different thing from build-time generation in three ways — it costs warm-up before it happens, it is lost at call sites that genuinely see several tables, and it cannot be relied upon when writing the code, only observed afterwards in a profile. ## What an interviewer is checking That you can separate the mechanism from the folklore. A weak answer says the shared strategy is slow because of pointer chasing; a strong one says the target is unknown at compile time, names the optimisations that unknown target blocks, distinguishes the per-operation cost from the once-per-call setup, and adds that the gap is partly recoverable while running but not promised.

  • Does the indirection cost the same in a loop over ten thousand samples as in a call made once per frame?
    No, and the difference is the whole story. Once per frame the cost disappears into the work around it. Ten thousand times per frame it compounds: every iteration keeps a call boundary the optimiser cannot see through, so the loop never gets unrolled, folded or vectorised. Size the cost by how many times the operation runs per unit of real work.
  • How can a run-time optimiser recover part of the gap?
    By speculating. It observes that a hot shared body keeps receiving the same operations table, emits a copy specialised to that table, and guards it with a check; matching entries take the fast copy, anything else falls back to the general body. It costs warm-up, and it is lost at a call site that genuinely sees several different tables.

saying these in an interview costs you the question

  • Says the indirect jump itself is the whole cost
  • Claims a shared body cannot be optimised at all
  • Believes the indirection is paid once on entry, not per operation
  • Says the body can inline an operation it reaches through the table
  • Treats run-time re-specialisation as something the build guarantees