skip to content

A profiler flags a C++ physics loop calling virtual update() on 100,000 bodies each frame; what does each call really cost, and how do you cut it?

level: seniorimportance: should knowfreq 55%

answer

  1. the first load touches the object
  2. indirect branch prediction
  3. no inlining, no vectorisation
  4. group the bodies by type
  5. move dispatch out of the loop

basics

~20 s

Each call loads the object's vptr, often a cache miss when bodies are scattered on the heap, then makes an indirect branch that mispredicts when types interleave, and blocks inlining. Group bodies by type and dispatch once per batch.

solid answer

~50 s

A virtual call is two dependent loads and an **indirect branch**, and in a loop over 100,000 heap-allocated bodies each part has a cost. Loading the **vptr** is the first touch of each object, so with `std::vector<std::unique_ptr<Body>>` it is often a **cache miss** that the profiler charges to the call. The vtable load is cheap because a few vtables stay hot. The indirect branch is predicted from history: if particles, rigid bodies and cloth interleave, the target keeps changing and **mispredicts**. And the call is opaque, so `update` cannot be **inlined** or vectorised across bodies. Remedies, cheapest first: sort the pointer array by type; store each concrete type in its own contiguous container and loop with a `final` static type, so the calls become direct and inlinable; or make the virtual call once per batch, as in `virtual void updateAll(float dt)` on a per-type system. Measure branch and cache misses before and after.

code

cpp · 26 lines
cpp
#include <cstdio>
#include <memory>
#include <vector>

struct Body {
    virtual ~Body() = default;
    virtual void update(float dt) = 0;
    float x = 0.0f, v = 1.0f;
};
struct Particle final : Body { void update(float dt) override { x += v * dt; } };
struct Rigid final : Body { void update(float dt) override { x += v * dt; v *= 0.99f; } };

int main() {
    // Before: one heap object and one indirect call per body, types interleaved.
    std::vector<std::unique_ptr<Body>> mixed;
    for (int i = 0; i < 100000; ++i)
        mixed.push_back(i % 2 ? std::unique_ptr<Body>(new Particle) : std::make_unique<Rigid>());
    for (auto& b : mixed) b->update(0.016f);

    // After: one contiguous array per final type; every call is direct and inlinable.
    std::vector<Particle> particles(50000);
    std::vector<Rigid> rigids(50000);
    for (auto& p : particles) p.update(0.016f);
    for (auto& r : rigids) r.update(0.016f);
    std::printf("%g %g\n", particles[0].x, rigids[0].v);
}

go deeper

for a junior

Know that a virtual call goes through a pointer to a table, so it costs a little more than a direct call and cannot be inlined.

for a middle

Break the call into its parts: vptr load, slot load, indirect branch, and explain which of those is cheap and which depends on data.

for a senior

Diagnose from counters, not intuition: cache misses on scattered objects versus branch mispredictions on interleaved types, then pick sorting, batching or lifted dispatch.

for a principal

Set where dynamic dispatch is allowed: per system or per batch freely, inside per-object inner loops only with profiling evidence.

## What one virtual call does The engine keeps every simulated object behind a base pointer and updates them each frame: ```cpp std::vector<std::unique_ptr<Body>> bodies; // 100,000 mixed types for (auto& b : bodies) b->update(dt); ``` In common ABIs each `b->update(dt)` is three steps: load the object's **vptr**, load the `update` slot from the class's **vtable**, and branch through that pointer. The standard requires none of this, but every mainstream compiler does it, so this is where the time goes: | Cost | When it hurts | Remedy | |---|---|---| | loading the vptr from the object | objects scattered across the heap | contiguous storage per type | | loading the vtable slot | rarely: a handful of vtables stay cached | none needed | | the indirect branch | types interleaved in the array | sort or group by type | | the opaque call | small `update` bodies | make the static type known so it can inline | ## Reading the profile correctly The loads are dependent: the CPU can run ahead on a predicted target, but it cannot confirm the branch until both loads complete, so a cache miss on the first load combined with a wrong prediction is the expensive case. With `std::unique_ptr<Body>`, each body was allocated separately, and the vptr load is the **first access to that object** in the frame. When the objects are spread across the heap, that load misses the cache, and a sampling profiler attributes the stall to the call instruction. The loop looks dispatch-bound, but much of the time is **memory latency**. Before changing the design, check hardware counters for cache misses and branch mispredictions; the material on benchmarks and profiling covers the tools. ## The branch predictor and type order Modern CPUs predict indirect branches from recent history. If the array holds particles, then rigid bodies, then cloth, in long runs, each call goes where the previous one went and the prediction is almost always right. If the types are interleaved, as they are when objects are created in gameplay order, the target changes unpredictably and each **misprediction flushes the pipeline**. Sorting the pointer array by dynamic type, for example once per frame or when objects are added, can recover most of that without touching the call. ## Lost inlining An indirect call is a wall for the optimiser. `Particle::update` might be two multiply-adds that the compiler could inline into the loop and, with a suitable data layout, vectorise across particles; through a virtual call it is a separate function entered 100,000 times, with the arguments and the return address handled each time. This is often the largest cost for tiny functions, and only removing the dynamic dispatch from the inner loop fixes it. ## Remedies, cheapest first 1. **Sort by type.** Keep the design, reorder the pointers. Helps branch prediction, not cache misses or inlining. 2. **Batch by type.** Store each concrete type in its own `std::vector<Particle>`, `std::vector<Rigid>` and loop over each. The objects are contiguous, and because the static type is a `final` class, every call is direct and can be inlined. 3. **Lift the dispatch.** Give each subsystem one virtual entry point that loops over its own concrete objects. The engine makes a handful of virtual calls per frame instead of 100,000: ```cpp struct System { virtual ~System() = default; virtual void updateAll(float dt) = 0; }; struct ParticleSystem final : System { std::vector<Particle> particles; void updateAll(float dt) override { for (auto& p : particles) p.update(dt); } }; ``` The inner loop sees `Particle`, a final type, so its calls are direct, while the engine keeps an open list of systems. 4. **Mark leaf classes `final`.** Costs nothing and lets the compiler devirtualize wherever the static type is the leaf; it does nothing for calls through `Body*`. 5. **Pool allocations** of one type together if the pointer-based design must stay; the wider topic of data layout belongs with cache-friendly layout. ## What not to conclude The fix is rarely "remove virtual functions everywhere". Virtual dispatch at the level of systems, tools or once-per-frame callbacks costs nothing measurable. It hurts in **inner loops over many small objects of mixed types**, and the cure is to change what the loop iterates over, then confirm the change with the same profile that raised the alarm.

  • Why does sorting a std::vector<Body*> by type help even though every call is still virtual?
    Because the CPU predicts an indirect branch from its recent targets. Long runs of one type make each call go where the previous one went, so predictions are right and the pipeline is not flushed. Sorting does nothing for cache misses on scattered objects or for lost inlining; it only fixes the branch part of the cost.
  • How do you tell whether the time is in the dispatch or in memory access?
    Read hardware counters for the loop: branch mispredictions point at the indirect branch, cache misses at the object loads. A quick experiment helps too: run the same loop over objects allocated contiguously, or over a type-sorted array, and see which change moves the time.

  • The vtable lookup searches for update() by name on each call.
  • Marking update() final in Particle speeds up calls through Body*.
  • The cost is the vtable load, since vtables are never in cache.
  • A virtual call costs the same whether types are sorted or interleaved.
  • If the profiler blames the call instruction, memory cannot be the problem.