In a C++ audio plugin framework whose virtual Effect::process() runs per sample, how do you decide between keeping it virtual and moving to std::variant or templates?
answer
- open set or closed set
- who adds new effect types, and when
- per sample versus per block
- binary plugin boundary
- measure before redesigning
basics
~20 sDecide by who extends the set and where the cost really is. Third-party effects loaded at run time need a virtual interface; a closed built-in set can be a std::variant; a fixed chain can be templates. Often process() per block fixes it.
solid answer
~50 sFirst question the granularity: a virtual call **per sample** is the problem far more often than virtual itself, and changing `process()` to take a block of 64 to 512 samples amortises one call over the whole block. Then ask who extends the set. **Third-party effects loaded from shared libraries at run time** need runtime polymorphism across a binary boundary, so that interface stays virtual. If the set is **closed and known when the host is compiled**, `std::variant` (C++17) with `std::visit` gives value storage, no per-effect allocation and calls the compiler can inline, at the price that every new type touches the variant and every visitor. A chain **fixed at compile time** can be templates: full inlining and vectorisation, paid for in compile time, code size and no runtime rebuilding. Many products mix them: virtual at the plugin boundary, static inside built-in kernels. Decide with a profile.
code
cpp · 25 lines#include <cstdio>
#include <variant>
#include <vector>
struct Gain {
float g;
void process(float* buf, int n) const { for (int i = 0; i < n; ++i) buf[i] *= g; }
};
struct Clip {
float limit;
void process(float* buf, int n) const {
for (int i = 0; i < n; ++i)
buf[i] = buf[i] > limit ? limit : (buf[i] < -limit ? -limit : buf[i]);
}
};
using BuiltInEffect = std::variant<Gain, Clip>; // closed set, stored by value
int main() {
std::vector<BuiltInEffect> chain{Gain{0.5f}, Clip{0.3f}};
float block[4] = {1.0f, -1.0f, 0.4f, 0.2f};
for (const auto& fx : chain)
std::visit([&](const auto& e) { e.process(block, 4); }, fx);
for (float s : block) std::printf("%g ", s);
std::printf("\n");
}go deeper
Know the three tools exist: virtual functions for runtime polymorphism, std::variant for a closed set of types, templates for types fixed at compile time.
Explain what each costs: indirect calls and allocation for virtual, size and recompiles for variant, code size and compile time for templates.
Show the diagnosis first: per-sample calls versus per-block calls, cache misses hidden behind the call, and a realistic benchmark before any redesign.
Own the policy: which layer stays open for third-party plugins and binary compatibility, which built-in kernels close the set, and the measured threshold for moving between them.
## The decision in front of the lead An audio framework's `Effect` hierarchy usually starts as a virtual interface: ```cpp struct Effect { virtual ~Effect() = default; virtual float process(float sample) = 0; // called 48,000 times per second per channel }; ``` Then a profile of a dense session shows time in effect dispatch, and someone proposes replacing the hierarchy with `std::variant` or templates. That is a design decision with consequences for plugin authors, build times and the binary interface, so it deserves more than a micro-benchmark. There is no single right answer; there are questions that narrow it down. ## Question 1: is the problem the granularity, not virtual? A virtual call per **sample** costs an indirect call, prevents inlining the effect into the loop, and blocks vectorising the loop across samples. The same interface called per **block** costs one indirect call per 64 to 512 samples, and the body of each effect becomes a tight loop the compiler can optimise freely: ```cpp virtual void process(float* buf, std::size_t n) = 0; // one call per block ``` This change keeps the extensibility model and often removes the hotspot entirely. Check it first; the vtable-level cost of a single indirect call is covered in the material on dispatch cost, and it is small once amortised. ## Question 2: is the set of effect types open or closed? | Situation | Fits | Why | |---|---|---| | third parties ship effects as shared libraries loaded at run time | virtual interface | only runtime polymorphism crosses a binary boundary the host was compiled without | | the team owns every effect type, and the set changes with releases | `std::variant` (C++17) | closed set, value storage, the compiler sees every alternative | | a fixed processing chain known at compile time, such as one product's channel strip | templates | full inlining and vectorisation across the chain | | user-built chains inside the host, with built-in effects only | either virtual per block or a variant | depends on how often types are added | A `std::variant<Gain, Clip, Reverb>` cannot hold a type that did not exist when the host was compiled, which rules it out for a plugin ecosystem. A virtual interface handles that case, often with the C-compatible boundary kept deliberately narrow. ## Question 3: what does each option cost the team? - **Virtual interface:** easy to extend, each effect usually heap-allocated and held through `std::unique_ptr<Effect>`, calls opaque to the optimiser unless devirtualised. Changing the base class layout or adding virtual functions is a binary-compatibility event for plugin authors. - **`std::variant`:** objects stored by value and contiguously, no allocation per effect, and `std::visit` dispatch that implementations can turn into a jump table or inline. Every new alternative is a source change in the host and recompiles everything that visits the variant; the variant is at least as large as its largest alternative, so one big reverb inflates every slot. - **Templates:** the fastest code when the composition is static, but each composition is a separate instantiation, compile times and binary size grow, errors move to template diagnostics, and runtime reconfiguration needs a type-erased layer on top anyway. ## Question 4: what does the measurement actually say? A profiler attributing time to a virtual call is often really showing a **cache miss on the object**, because the call is the first touch of a heap-allocated effect. Moving to a variant can help because of contiguous storage, not because the dispatch is cheaper. Measure the realistic case, a full session with mixed effects at the target buffer size, and compare: 1. the current code; 2. the same interface called per block; 3. a prototype of the variant or template design for the hottest built-in effects only. ## A defensible policy Many teams end up with a **hybrid**: a virtual, block-based interface at the plugin boundary, because extensibility across binaries is a product requirement; and a closed `std::variant` or template-composed kernel for built-in effects, because the team owns those types and controls the release. Write the rule down: which layer is open, which is closed, and the measurement that would justify moving a component from one to the other. Revisit it when the plugin ecosystem or the target buffer sizes change, since those are the facts the decision rests on.
- Why can a std::variant-based design not replace the interface for third-party plugins?A `std::variant` lists its alternatives in the host's source, so every type must be known when the host is compiled. A plugin built later as a shared library introduces a type the host has never seen; only a runtime-polymorphic interface, a virtual base or a C function table, lets the host call code it was not compiled with.
- When does templating the whole processing chain stop paying off?When the chain must change at run time, because each composition is a separate type and switching needs a type-erased layer again; and when the number of compositions grows, because each instantiation adds compile time and binary size. Templates fit a fixed, product-defined chain, not a user-editable graph.
- Virtual functions are always too slow for audio, so replace them everywhere.
- std::variant can hold plugin types loaded from shared libraries at run time.
- Marking every override final removes the cost of calls through Effect*.
- Templates are free: they cost nothing beyond the faster code they produce.
- The profiler shows time in the virtual call, so dispatch must be the cost.