Which ordinary C++ operations in a market-data tick path allocate on the heap with no visible new, and how do you catch them?
answer
- no new keyword in sight
- small buffers have limits
- node containers allocate per insert
- count calls to operator new
- the counter misses malloc
basics
~20 sLong std::string values, std::function holding a large callable, vector growth, make_shared, new-key inserts into std::map or std::unordered_map, and string formatting all allocate. Catch them by counting global operator new in a test or with a heap profiler.
solid answer
~40 sMost tick-path allocations hide inside library types. A `std::string` allocates once its text outgrows the implementation's small buffer (22 characters in libc++, 15 in libstdc++); `std::function` allocates when the callable does not fit its internal buffer, which the standard only *recommends* for small objects; `push_back` past `capacity()` reallocates; `std::make_shared` allocates the object and its control block; inserting a new key into `std::map` or `std::unordered_map` allocates a node; `std::to_string`, `std::format` and `std::ostringstream` build heap strings. Creating a lambda is not itself an allocation — putting it into `std::function` is. To catch them, a test binary replaces the global `operator new` with a counting version, replays recorded ticks and asserts the count stays at zero after warm-up; a heap profiler covers what that counter cannot see, such as direct `malloc` calls from C libraries.
code
cpp · 29 lines#include <cstdio>
#include <cstdlib>
#include <functional>
#include <memory>
#include <new>
#include <string>
#include <vector>
static long g_allocs = 0; // test-only: count every call to the global operator new
void* operator new(std::size_t n) { ++g_allocs; if (void* p = std::malloc(n)) return p; throw std::bad_alloc{}; }
void operator delete(void* p) noexcept { std::free(p); }
void operator delete(void* p, std::size_t) noexcept { std::free(p); }
template <class F> void probe(const char* what, F f) {
long before = g_allocs;
f();
std::printf("%-32s %ld\n", what, g_allocs - before);
}
int main() {
double px = 101.25, qty = 300, fee = 0.1, tick = 0.01;
probe("std::string, 4 chars", [] { std::string s = "AAPL"; });
probe("std::string, 24 chars", [] { std::string s = "AAPL.NASDAQ.L1.BID.QUOTE"; });
probe("std::function, 8-byte capture", [&] { std::function<double()> f = [&px] { return px; }; });
probe("std::function, 32-byte capture", [=] { std::function<double()> f = [=] { return px * qty + fee + tick; }; });
probe("std::make_shared<double>", [] { auto p = std::make_shared<double>(1.0); });
std::vector<double> v(64);
probe("push_back past capacity", [&] { v.push_back(64); });
}go deeper
Recall the common culprits: long strings, std::function, vector growth, make_shared and map inserts. Know that creating a lambda does not allocate by itself.
Explain why each allocates, separate the standard's recommendation from libc++ and libstdc++ buffer sizes, and describe a counting operator new test with its warm-up step.
Show you know the counter's blind spots, such as direct malloc calls and runtime-allocated exceptions, and pair it with a heap profiler and per-thread counting in staging.
Treat hidden allocations as a review and testing policy: which types are banned on the hot path, and which toolchain the zero-allocation proof must run on.
## No new expression does not mean no allocation A tick path can be free of `new`, `make_unique` and `malloc` and still allocate thousands of times a second. The allocations live inside standard library types that own memory: they call the allocator on your behalf, usually through `std::allocator`, which obtains storage from the global `operator new`. Knowing which everyday operations do that is the first skill an interviewer probes for low-latency C++. ## The usual suspects | Operation | Allocates when | Allocation-free alternative | |---|---|---| | `std::string` construction, `+`, `append` | the text outgrows the small-string buffer | `std::string_view`, a reused string with reserved capacity, a fixed `char` array | | `std::function<Sig>` from a lambda | the callable is larger than the wrapper's internal buffer | a template parameter, a function pointer plus context pointer, capturing one pointer | | `std::vector::push_back` | the new size exceeds `capacity()` | `reserve` at startup and reuse the vector | | `std::make_shared<T>` | always: one block for the object and its control block | a long-lived object passed by reference, or a pool | | `std::map` / `std::unordered_map` insert | always for a new key: one node; `unordered_map` also on rehash | a pre-sized flat array indexed by symbol id | | `std::to_string`, `std::format`, `std::ostringstream` | they return or hold a `std::string` | `std::to_chars` into a caller-provided buffer | | `throw` | the exception object needs storage | error codes or `std::expected` on the hot path | Two traps sit beside the table: - **A lambda is not an allocation by itself.** A closure is an ordinary object; capturing four doubles makes it 32 bytes, stored wherever the lambda variable lives. Only what its captures do can allocate (a `std::string` captured by value copies its characters), and the classic hidden allocation happens when the closure is put into a `std::function`. - **Copying a `std::shared_ptr` does not allocate.** It increments an atomic count in the existing control block; the allocation is in `std::make_shared` or in constructing a `shared_ptr` from a raw pointer. ## Small buffers are an implementation's choice The **small-string optimization** and `std::function`'s small-object buffer are not required by the standard. For `std::function`, the standard says only that implementations *should avoid* dynamic allocation for small callable objects, giving an object holding a pointer and a member function pointer as the example; it promises no throw only when the target is a function pointer or a `std::reference_wrapper`. The real limits differ by library: - **libc++** stores up to 22 characters in a 64-bit `std::string` without allocating. Its `std::function` has a buffer of three pointers (24 bytes on a 64-bit target) that must also hold a vtable pointer, so a callable stays inline only up to 16 bytes and only if it is nothrow copy-constructible; on this machine a lambda capturing two doubles did not allocate and one capturing three did. - **libstdc++** stores up to 15 characters, and keeps a callable inside `std::function` only when it is trivially copyable and no larger than 16 bytes on common 64-bit targets. So the same source can be allocation-free on one toolchain and not on another: a test on the toolchain you ship is the only proof. ## Catching them with a counting operator new The cheapest detector is a test binary that replaces the global allocation functions (the material on new and delete covers the replacement rules): 1. Define `operator new(std::size_t)` to increment a counter and call `std::malloc`, and the matching `operator delete` overloads to call `std::free`. 2. Warm the handler up: let it reserve, fill pools and build its tables. 3. Snapshot the counter, replay a recorded stretch of ticks, and assert the counter did not move. 4. When it moves, break in the replacement with a debugger to get the call stack. The counter has blind spots, and saying so is part of a good answer: - It sees only allocations that go through the global `operator new`. A C library calling `malloc` directly bypasses it, and on this toolchain `throw 42;` did not call `operator new` at all, because the C++ runtime allocates the exception object itself. - It counts every thread unless you make it per-thread with a `thread_local` flag for the tick thread. - The standard leaves unspecified when or how often `std::allocator` calls `::operator new`, so an optimizer may merge or remove allocations; count in the build configuration you ship. A heap profiler or an allocator with statistics covers those gaps in staging; the counting test guards the tick path in continuous integration. ## Fixing what you find - Replace string building with `std::string_view` and `std::to_chars`. - Keep callbacks as template parameters, or capture a single pointer so the closure fits the buffer. - Reserve vectors and reuse them; replace node-based maps on the hot path with pre-sized arrays. - Hold long-lived objects by reference instead of creating `std::shared_ptr` per message.
- Why can a 20-character std::string allocate on one standard library and not on another?The small-string optimization is an implementation choice, not a requirement. libc++ keeps up to 22 characters inside a 64-bit `std::string`, while libstdc++ keeps up to 15, so a 20-character symbol is inline on libc++ and heap-allocated on libstdc++. Any allocation budget that relies on short strings has to be verified on the shipping toolchain.
- How do you keep a per-tick callback without std::function's allocation?Take the callable as a template parameter so it is stored and called directly, store a plain function pointer plus a context pointer, or make the lambda capture only one pointer to a state object so it fits the wrapper's small buffer. Each keeps the call path free of the heap; which one fits depends on whether the callback must be stored type-erased.
- A lambda allocates on the heap as soon as it captures anything
- The standard requires std::function to store small lambdas without allocating
- Short strings never allocate because the standard mandates a small-string buffer
- Copying a std::shared_ptr allocates a new control block
- If the code has no new expression, the hot path is allocation-free