A C# async cache lookup usually finds its value already loaded: what does each call allocate on the hit path versus the miss path, and why?
answer
- IsCompleted true means no suspension
- struct machine stays on the stack
- result Task: cached or new
- first suspension boxes the machine once
- Debug builds allocate every call
basics
~20 sOn a hit the awaited Task is already complete, so nothing suspends: the struct state machine stays on the stack and only the returned Task<T> may allocate. A miss boxes the machine once, and that box is the returned Task.
solid answer
~40 sIn a Release build the state machine is a `struct`. On a **hit**, `await` sees `IsCompleted == true`, calls `GetResult` inline and the method runs to its end without suspending, so the machine never leaves the stack. The builder's `SetResult` then calls `Task.FromResult`, which hands back a cached task for `true`/`false`, `int` values from -1 to 8, `null` and some `default` values, and otherwise allocates a new `Task<T>`, 80 bytes for a `Task<decimal>` on 64-bit .NET 10. On a **miss**, the first unfinished await makes the builder copy the machine into an `AsyncStateMachineBox<TStateMachine>`, which derives from `Task<TResult>` and is the `Task` the caller holds. That is one allocation per call, reused by later suspensions, sized by the hoisted fields, plus whatever the awaited operation allocates.
code
csharp · 36 linesusing System;
using System.Collections.Generic;
using System.Threading.Tasks;
var cache = new PriceCache();
await cache.GetPriceAsync("A1"); // miss: starts the load and caches its Task
for (int i = 0; i < 100; i++) { await cache.GetPriceAsync("A1"); await cache.InStockAsync("A1"); }
long a = GC.GetAllocatedBytesForCurrentThread();
Task<decimal> price = cache.GetPriceAsync("A1");
long b = GC.GetAllocatedBytesForCurrentThread();
Task<bool> inStock = cache.InStockAsync("A1");
long c = GC.GetAllocatedBytesForCurrentThread();
Console.WriteLine($"hit, Task<decimal>: completed = {price.IsCompleted}, {b - a} bytes");
Console.WriteLine($"hit, Task<bool>: completed = {inStock.IsCompleted}, {c - b} bytes");
class PriceCache
{
private readonly Dictionary<string, Task<decimal>> _loads = new();
public async Task<decimal> GetPriceAsync(string sku)
{
if (!_loads.TryGetValue(sku, out Task<decimal>? load))
_loads[sku] = load = LoadFromServiceAsync(sku);
return await load; // on a hit the Task is already complete
}
public async Task<bool> InStockAsync(string sku) => await GetPriceAsync(sku) > 0;
private static async Task<decimal> LoadFromServiceAsync(string sku)
{
await Task.Delay(50); // stands in for a network call
return 9.99m;
}
}go deeper
Recall that awaiting an already finished Task does not suspend the method, so a cache hit runs straight through.
Explain that the Release state machine is a struct, that the first suspension boxes it, and that the returned Task<T> may come from a cache.
Show the diagnosis: warmed-up Release measurements, the result Task as the remaining hit-path allocation, one box per suspended call, and hoisted fields sizing that box.
Decide where allocation on hot async paths is worth engineering effort, and set the expectation that such claims are backed by Release measurements, not Debug numbers.
## The scenario A price cache keeps a `Dictionary<string, Task<decimal>>`. `GetPriceAsync` looks the key up, starts a load on a miss, and does `return await load;`. Almost every call is a hit, so the question is what the hit costs, and why it differs from the miss. ## The hit path: no suspension On a hit the cached `Task<decimal>` has already completed. The compiled `await`: 1. calls `GetAwaiter()` on the task, which returns a `TaskAwaiter<decimal>` struct; 2. reads `IsCompleted`, which is `true`; 3. calls `GetResult()` and keeps running in the same `MoveNext` call. The method reaches its end without suspending. In a **Release** build the state machine is a `struct` living in the stub's stack frame, so it is never copied to the heap. The only remaining cost is the `Task<decimal>` the method must return. The builder's `SetResult` sees that no task exists yet and calls `Task.FromResult(result)`, which returns a **cached completed task** when it can: - `true` and `false` for `Task<bool>`; - `int` values from -1 to 8 for `Task<int>`; - `null`, and `default` for small value types without references (sizes 1, 2, 4, 8 or 16 bytes). Any other result allocates a new `Task<T>`. The cache ranges are the runtime's current implementation, not a documented contract. Measured on 64-bit .NET 10, a hit returning `9.99m` allocates 80 bytes (one `Task<decimal>`); an async method returning `Task<bool>` that awaits it adds nothing for its own result. ## The miss path: one box per call On a miss the load is unfinished, so the method must suspend. The builder's `AwaitUnsafeOnCompleted` needs the state machine to outlive the stack frame, so on the **first** suspension it creates an `AsyncStateMachineBox<TStateMachine>` and copies the struct into it. Three details matter: - the box **derives from `Task<TResult>`** and is stored as the builder's task, so it is the very `Task` the caller receives; there is no separate result task; - later suspensions in the same call **reuse** the box: a method that suspends five times allocates the same single box as one that suspends once; - the box's size grows with the **hoisted fields**: every local, parameter and awaiter that crosses an await is copied into it. The awaited operation adds its own allocations: the load's own state machine, a timer for `Task.Delay`, a socket operation for real I/O. ## Reading the example’s numbers The example measures with `GC.GetAllocatedBytesForCurrentThread()` around single, warmed-up calls in a Release build: - `GetPriceAsync` on a hit reports **80 bytes**: the `Task<decimal>` holding `9.99m`, because only `default(decimal)` is cached; - `InStockAsync` on a hit also reports **80 bytes**, yet its own `Task<bool>` is cached. The bytes belong to the inner `GetPriceAsync` call it awaits. Every async layer on a hot path pays for its own result unless that result is cacheable; - neither call allocated a state-machine box, because neither suspended. Warm-up matters: the first calls JIT-compile code and fill runtime caches, which shows up as noise in the counter. ## Debug builds lie about this | Build | State machine | Hit path, `Task<bool>` result | |---|---|---| | Release | `struct`, boxed only on suspension | no allocation of its own | | Debug | `class`, allocated on every call | allocates the machine every time | In a Debug build the machine is a `class`, so every call allocates it, hit or miss. Allocation numbers taken from a Debug run, or from a test that ran without optimizations, overstate the hit path. ## What to do with this in production - **Measure in Release** with an allocation profiler or `GC.GetAllocatedBytesForCurrentThread` around a warmed-up call, not in Debug. - **Cache the work, not just the value.** Storing the `Task<decimal>` also lets callers that arrive while a load is still running share that one load. - **Remember the result task.** When a hot method returns a value outside the cached ranges, its result `Task<T>` is the remaining allocation; avoiding it means returning a different type, which is the ValueTask lesson's subject, or returning the cached task without an `async` method around it, which the task-based methods lesson discusses. - **Keep hoisted state small** on paths that do suspend: large structs held across an await make every box bigger.
- How many state-machine boxes does a C# async method allocate if one call suspends at five different awaits?One. The builder creates the `AsyncStateMachineBox<TStateMachine>` on the first real suspension and stores it as the method's `Task`; later suspensions find the box already there and reuse it. Five `await Task.Yield()` calls in a Release build allocate the same 96 bytes as one. The awaited operations may still allocate on their own.
- Why does a C# async method returning Task<int> allocate on a hit that returns 42 but not on one that returns 3?On a synchronous completion the builder calls `Task.FromResult`, which in the current runtime returns cached completed tasks for `int` values from -1 to 8. The value 3 is in that range; 42 is not, so a new `Task<int>` is allocated. It is an implementation detail, so measure rather than rely on the exact range.
- Why can a hot C# async method show more allocations in a unit test than in production?Tests often run a Debug build, where the compiler emits the state machine as a class and allocates it on every call, hit or miss. A Release build uses a struct that is boxed only when the method actually suspends. Measure allocations in Release, after warm-up.
A restaurant only opens a ticket on the rail when your dish is not ready yet; if it is already plated, you get it straight across the counter, and the only cost is the plate.
- Every await allocates a new heap object for the state machine.
- An async method that completes synchronously allocates nothing at all.
- Allocation numbers from a Debug build match production.
- The state machine box and the returned Task are two separate allocations.
- Awaiting an already completed Task still schedules a continuation.