skip to content

A C# async cache lookup usually finds its value already loaded: what does each call allocate on the hit path versus the miss path, and why?

level: seniorimportance: should knowfreq 30%

answer

  1. IsCompleted true means no suspension
  2. struct machine stays on the stack
  3. result Task: cached or new
  4. first suspension boxes the machine once
  5. Debug builds allocate every call

basics

~20 s

On a hit the awaited Task is already complete, so nothing suspends: the struct state machine stays on the stack and only the returned Task<T> may allocate. A miss boxes the machine once, and that box is the returned Task.

solid answer

~40 s

In a Release build the state machine is a `struct`. On a **hit**, `await` sees `IsCompleted == true`, calls `GetResult` inline and the method runs to its end without suspending, so the machine never leaves the stack. The builder's `SetResult` then calls `Task.FromResult`, which hands back a cached task for `true`/`false`, `int` values from -1 to 8, `null` and some `default` values, and otherwise allocates a new `Task<T>`, 80 bytes for a `Task<decimal>` on 64-bit .NET 10. On a **miss**, the first unfinished await makes the builder copy the machine into an `AsyncStateMachineBox<TStateMachine>`, which derives from `Task<TResult>` and is the `Task` the caller holds. That is one allocation per call, reused by later suspensions, sized by the hoisted fields, plus whatever the awaited operation allocates.

code

csharp · 36 lines
csharp
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

var cache = new PriceCache();
await cache.GetPriceAsync("A1");                    // miss: starts the load and caches its Task
for (int i = 0; i < 100; i++) { await cache.GetPriceAsync("A1"); await cache.InStockAsync("A1"); }

long a = GC.GetAllocatedBytesForCurrentThread();
Task<decimal> price = cache.GetPriceAsync("A1");
long b = GC.GetAllocatedBytesForCurrentThread();
Task<bool> inStock = cache.InStockAsync("A1");
long c = GC.GetAllocatedBytesForCurrentThread();

Console.WriteLine($"hit, Task<decimal>: completed = {price.IsCompleted}, {b - a} bytes");
Console.WriteLine($"hit, Task<bool>:    completed = {inStock.IsCompleted}, {c - b} bytes");

class PriceCache
{
    private readonly Dictionary<string, Task<decimal>> _loads = new();

    public async Task<decimal> GetPriceAsync(string sku)
    {
        if (!_loads.TryGetValue(sku, out Task<decimal>? load))
            _loads[sku] = load = LoadFromServiceAsync(sku);
        return await load;                          // on a hit the Task is already complete
    }

    public async Task<bool> InStockAsync(string sku) => await GetPriceAsync(sku) > 0;

    private static async Task<decimal> LoadFromServiceAsync(string sku)
    {
        await Task.Delay(50);                       // stands in for a network call
        return 9.99m;
    }
}

go deeper

for a junior

Recall that awaiting an already finished Task does not suspend the method, so a cache hit runs straight through.

for a middle

Explain that the Release state machine is a struct, that the first suspension boxes it, and that the returned Task<T> may come from a cache.

for a senior

Show the diagnosis: warmed-up Release measurements, the result Task as the remaining hit-path allocation, one box per suspended call, and hoisted fields sizing that box.

for a principal

Decide where allocation on hot async paths is worth engineering effort, and set the expectation that such claims are backed by Release measurements, not Debug numbers.

## The scenario A price cache keeps a `Dictionary<string, Task<decimal>>`. `GetPriceAsync` looks the key up, starts a load on a miss, and does `return await load;`. Almost every call is a hit, so the question is what the hit costs, and why it differs from the miss. ## The hit path: no suspension On a hit the cached `Task<decimal>` has already completed. The compiled `await`: 1. calls `GetAwaiter()` on the task, which returns a `TaskAwaiter<decimal>` struct; 2. reads `IsCompleted`, which is `true`; 3. calls `GetResult()` and keeps running in the same `MoveNext` call. The method reaches its end without suspending. In a **Release** build the state machine is a `struct` living in the stub's stack frame, so it is never copied to the heap. The only remaining cost is the `Task<decimal>` the method must return. The builder's `SetResult` sees that no task exists yet and calls `Task.FromResult(result)`, which returns a **cached completed task** when it can: - `true` and `false` for `Task<bool>`; - `int` values from -1 to 8 for `Task<int>`; - `null`, and `default` for small value types without references (sizes 1, 2, 4, 8 or 16 bytes). Any other result allocates a new `Task<T>`. The cache ranges are the runtime's current implementation, not a documented contract. Measured on 64-bit .NET 10, a hit returning `9.99m` allocates 80 bytes (one `Task<decimal>`); an async method returning `Task<bool>` that awaits it adds nothing for its own result. ## The miss path: one box per call On a miss the load is unfinished, so the method must suspend. The builder's `AwaitUnsafeOnCompleted` needs the state machine to outlive the stack frame, so on the **first** suspension it creates an `AsyncStateMachineBox<TStateMachine>` and copies the struct into it. Three details matter: - the box **derives from `Task<TResult>`** and is stored as the builder's task, so it is the very `Task` the caller receives; there is no separate result task; - later suspensions in the same call **reuse** the box: a method that suspends five times allocates the same single box as one that suspends once; - the box's size grows with the **hoisted fields**: every local, parameter and awaiter that crosses an await is copied into it. The awaited operation adds its own allocations: the load's own state machine, a timer for `Task.Delay`, a socket operation for real I/O. ## Reading the example’s numbers The example measures with `GC.GetAllocatedBytesForCurrentThread()` around single, warmed-up calls in a Release build: - `GetPriceAsync` on a hit reports **80 bytes**: the `Task<decimal>` holding `9.99m`, because only `default(decimal)` is cached; - `InStockAsync` on a hit also reports **80 bytes**, yet its own `Task<bool>` is cached. The bytes belong to the inner `GetPriceAsync` call it awaits. Every async layer on a hot path pays for its own result unless that result is cacheable; - neither call allocated a state-machine box, because neither suspended. Warm-up matters: the first calls JIT-compile code and fill runtime caches, which shows up as noise in the counter. ## Debug builds lie about this | Build | State machine | Hit path, `Task<bool>` result | |---|---|---| | Release | `struct`, boxed only on suspension | no allocation of its own | | Debug | `class`, allocated on every call | allocates the machine every time | In a Debug build the machine is a `class`, so every call allocates it, hit or miss. Allocation numbers taken from a Debug run, or from a test that ran without optimizations, overstate the hit path. ## What to do with this in production - **Measure in Release** with an allocation profiler or `GC.GetAllocatedBytesForCurrentThread` around a warmed-up call, not in Debug. - **Cache the work, not just the value.** Storing the `Task<decimal>` also lets callers that arrive while a load is still running share that one load. - **Remember the result task.** When a hot method returns a value outside the cached ranges, its result `Task<T>` is the remaining allocation; avoiding it means returning a different type, which is the ValueTask lesson's subject, or returning the cached task without an `async` method around it, which the task-based methods lesson discusses. - **Keep hoisted state small** on paths that do suspend: large structs held across an await make every box bigger.

  • How many state-machine boxes does a C# async method allocate if one call suspends at five different awaits?
    One. The builder creates the `AsyncStateMachineBox<TStateMachine>` on the first real suspension and stores it as the method's `Task`; later suspensions find the box already there and reuse it. Five `await Task.Yield()` calls in a Release build allocate the same 96 bytes as one. The awaited operations may still allocate on their own.
  • Why does a C# async method returning Task<int> allocate on a hit that returns 42 but not on one that returns 3?
    On a synchronous completion the builder calls `Task.FromResult`, which in the current runtime returns cached completed tasks for `int` values from -1 to 8. The value 3 is in that range; 42 is not, so a new `Task<int>` is allocated. It is an implementation detail, so measure rather than rely on the exact range.
  • Why can a hot C# async method show more allocations in a unit test than in production?
    Tests often run a Debug build, where the compiler emits the state machine as a class and allocates it on every call, hit or miss. A Release build uses a struct that is boxed only when the method actually suspends. Measure allocations in Release, after warm-up.

A restaurant only opens a ticket on the rail when your dish is not ready yet; if it is already plated, you get it straight across the counter, and the only cost is the plate.

  • Every await allocates a new heap object for the state machine.
  • An async method that completes synchronously allocates nothing at all.
  • Allocation numbers from a Debug build match production.
  • The state machine box and the returned Task are two separate allocations.
  • Awaiting an already completed Task still schedules a continuation.