A C++ particle update reads 24 bytes of a 64-byte struct that also holds a std::string debug name; how would you hot/cold split it, and what does the split cost?
answer
- bytes fetched versus bytes used
- std::string is not small
- two arrays, one index
- removal must touch both
- cold access becomes a second miss
basics
~20 sMove the fields the per-frame loop never reads into a parallel cold array with the same index, leaving 24-byte hot elements. The cost: both arrays must change in step, and code needing both halves takes an extra cache miss.
solid answer
~50 sWith `x, y, z, vx, vy, vz` plus a `std::string debugName`, a spawn frame and an emitter id, the struct is 64 bytes with libc++ (whose `std::string` is 24 bytes) and 72 with libstdc++ (32), so the update fetches 64 or more bytes to use 24. **Hot/cold splitting** keeps `std::vector<ParticleHot>` with the six floats and a parallel `std::vector<ParticleCold>` where `cold[i]` describes `hot[i]`. The loop now streams 24-byte elements, roughly 2.7 per cache line instead of one. What it costs: every add and remove must update both vectors in lock-step, code that needs hot and cold data together takes two cache misses, and the type no longer documents that the fields belong together, so the split should sit behind one owning class. Decide from measurement: which fields the hot loop reads, how often cold code runs, and whether the gain shows in the frame time.
code
cpp · 31 lines#include <cstdint>
#include <cstdio>
#include <string>
#include <vector>
// Before: every frame drags the editor-only fields through the cache.
struct ParticleFat {
float x, y, z, vx, vy, vz; // hot: read and written every frame
std::string debugName; // cold: editor and logs only
std::uint64_t spawnFrame; // cold
std::uint32_t emitterId; // cold
};
// After: hot/cold split into two arrays that share an index.
struct ParticleHot { float x, y, z, vx, vy, vz; };
struct ParticleCold { std::string debugName; std::uint64_t spawnFrame; std::uint32_t emitterId; };
struct ParticleSystem {
std::vector<ParticleHot> hot; // the update loop touches only this
std::vector<ParticleCold> cold; // cold[i] describes hot[i]
};
int main() {
std::printf("sizeof(ParticleFat) = %zu\n", sizeof(ParticleFat));
std::printf("sizeof(ParticleHot) = %zu\n", sizeof(ParticleHot));
ParticleSystem ps;
ps.hot.push_back({0, 0, 0, 1, 0, 0});
ps.cold.push_back({"spark", 120, 7});
for (ParticleHot& p : ps.hot) p.x += p.vx;
std::printf("%s at x=%g\n", ps.cold[0].debugName.c_str(), ps.hot[0].x);
}go deeper
Understand that a loop pays for every byte of the struct it walks, including fields it never reads.
Explain cache-line density, why std::string's size is a library choice, and the parallel-array split with a shared index.
Drive the split from a profile: identify hot fields, encapsulate the two arrays, handle removal and sorting, and guard the layout with static_assert.
Decide which teams own hot and cold data, and whether debug-only data should exist in shipping builds at all.
## Why the fat struct hurts The per-frame update is a stream: for each particle, read position and velocity, write position. The CPU does not fetch fields, it fetches **cache lines**, so the cost of the loop is set by the bytes per element, not by the bytes the loop names. ```cpp struct ParticleFat { float x, y, z, vx, vy, vz; // 24 bytes the update uses std::string debugName; // 24 bytes in libc++, 32 in libstdc++ std::uint64_t spawnFrame; // 8 std::uint32_t emitterId; // 4, plus padding }; ``` `sizeof(ParticleFat)` is 64 with libc++ on a 64-bit target and 72 with libstdc++; the size of `std::string` is each library's choice, not the standard's. Either way the loop drags at least 64 bytes through the cache for every 24 it uses, and a `std::string` member also gives the struct a non-trivial copy and destructor. ## The split ```cpp struct ParticleHot { float x, y, z, vx, vy, vz; }; // 24 bytes struct ParticleCold { std::string debugName; std::uint64_t spawnFrame; std::uint32_t emitterId; }; class ParticleSystem { std::vector<ParticleHot> hot_; // the update loop touches only this std::vector<ParticleCold> cold_; // cold_[i] describes hot_[i] }; ``` Effects on the hot loop: - **Density.** 24-byte elements mean about 2.7 particles per 64-byte line instead of one, so the same loop moves well under half the memory. - **Trivial type.** `ParticleHot` is trivially copyable, so moving it, sorting it or reallocating the vector is a memory copy rather than a loop of string moves. - **Vectorisation.** A dense array of 24-byte, all-float elements gives the compiler a short, regular stride to vectorise; a 64-byte stride with a string in every element rarely vectorises well. Hot/cold splitting is the light version of a full structure of arrays: it separates rarely used fields, while SoA separates every field. ## What it costs | Cost | Where it bites | Mitigation | |---|---|---| | Two arrays to keep in step | spawn, kill, sort, compaction | one owning class whose methods update both | | Second miss when both halves are needed | spawning, editor tools, logging | acceptable when that code is rare | | Index coupling | anything storing an index across a removal | swap-and-pop both arrays and patch handles | | Less obvious data model | readers no longer see one Particle | name the types and document the pairing | The design questions a senior answer works through, in order: 1. **Which fields does the hot loop read?** Profile, or read the loop; do not guess from field names. 2. **How often does cold code run?** If every frame also logs or reads `emitterId`, it is not cold. 3. **Where does the cold data live?** A parallel vector keeps indices aligned; a separate map keyed by particle id suits sparse data that most particles lack, such as a debug name only set in editor builds. 4. **Can the cold data shrink instead?** A `std::string` can often become an interned id, a fixed `char` buffer or nothing in release builds, which may make the split unnecessary. ## Why not a pointer to the cold part? A tempting shortcut is to keep one struct and move the cold fields behind `std::unique_ptr<ParticleCold> cold;`. That shrinks the hot element to 32 bytes, 24 for the floats plus an 8-byte pointer, so the loop does improve, but: - every particle still carries the pointer through the cache on every frame; - every particle now costs a separate heap allocation at spawn and a free at death; - the cold objects are scattered, so any pass over cold data becomes a pointer chase. A parallel `std::vector<ParticleCold>` keeps both halves contiguous, needs no per-particle allocation and leaves the hot element at exactly the 24 bytes the loop uses. ## Checking that it worked - Assert the hot layout: `static_assert(sizeof(ParticleHot) == 24)` and `static_assert(std::is_trivially_copyable_v<ParticleHot>)`, so a later field addition fails the build instead of quietly slowing every frame. - Compare the frame's update time before and after under the real particle count; a split that helps a microbenchmark but not the frame was not on the critical path. - Watch the cold side: if spawning or tooling became noticeably slower, the split moved cost rather than removing it. Field ordering and padding rules inside one struct belong to the material on object layout; the decision here is which data the hot loop should carry at all.
- The debug name is only set in editor builds. How would you store it so release builds pay nothing?Keep it out of the per-particle arrays entirely: a `std::unordered_map<ParticleId, std::string>` owned by the editor tooling, filled only when a name is set, or a conditional member compiled in only for editor configurations. Release builds then carry neither the bytes nor the non-trivial destructor, and the hot type stays trivially copyable.
- After the split, sorting particles by depth for rendering became slower. Why, and what would you do?Sorting must now permute both arrays in the same order, so a naive approach sorts an index array and then applies the permutation to hot and cold data, touching the cold array that the split was meant to avoid. Sort only what rendering needs, for example a separate array of depth and index pairs, and leave the particle storage in place.
- Reordering the members by size is enough to fix a loop that reads 24 of 64 bytes.
- The compiler only loads the fields a loop names, so unused members cost nothing.
- std::string is always 32 bytes because the standard fixes its small-buffer size.
- A split is free: the hot loop gets faster and nothing else changes.
- Making the cold part a std::unique_ptr member keeps the loop just as fast as a parallel array.