skip to content

Memory and Pooled Buffers

Memory<T> carries a buffer across awaits where Span cannot, and ArrayPool<T> rents arrays that must be returned and may be larger. Interviewers ask about pool misuse.

on this pageshow

explore

questions

5

In a C# async method that reads a stream into a buffer, why must the buffer be a Memory<byte> rather than a Span<byte>?

level: middleimportance: must knowfreq 50%

answer

  1. stack-only type versus storable struct
  2. await hoists live locals
  3. state machine moves to the heap
  4. take .Span at the point of use

basics

~20 s

Span<byte> is a stack-only ref struct, so it cannot be kept across an await, which hoists live locals into a heap state machine. Memory<byte> is an ordinary struct that can live there; take its .Span only between awaits.

solid answer

~50 s

`Span<T>` is a `ref struct`: it can hold a reference into the middle of an array or point at `stackalloc` memory, so the compiler only lets it live on the stack. An `await` turns every local still needed afterwards into a field of the async state machine, which moves to the heap when the method suspends, so a span there would break that rule. The compiler reports CS4012 for a span parameter on an async method and CS4007 for a span local used after an `await`. `Memory<T>` is a plain `readonly struct` holding an object (array, string or `MemoryManager<T>`), an index and a length, so it can be stored or cross an `await`. You pass `Memory<byte>` to `Stream.ReadAsync(Memory<byte>, CancellationToken)` and take `memory.Span` only between awaits; since C# 13 such a span local is allowed in an async method as long as it is not used after an `await`.

code

csharp · 24 lines
csharp
using System.Buffers;
using System.Security.Cryptography;

async Task<int> ForwardChunkAsync(Stream source, Memory<byte> buffer, IncrementalHash hash)
{
    int read = await source.ReadAsync(buffer);
    hash.AppendData(buffer.Span[..read]);
    return read;
}

var source = new MemoryStream(new byte[100_000]);
byte[] rented = ArrayPool<byte>.Shared.Rent(64 * 1024);
using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
try
{
    int total = 0, read;
    while ((read = await ForwardChunkAsync(source, rented.AsMemory(0, 64 * 1024), hash)) > 0)
        total += read;
    Console.WriteLine($"rented {rented.Length}, read {total}");
}
finally
{
    ArrayPool<byte>.Shared.Return(rented);
}

go deeper

for a junior

Know that Span<T> is stack-only and Memory<T> is the version you can store or use in async methods, and that .Span converts one way only.

for a middle

Explain the mechanism: await hoists live locals into a state machine that moves to the heap, which a ref struct cannot do, and name CS4007 and CS4012.

for a senior

Show the production shape: a Memory<byte> over a pooled array passed to ReadAsync, span taken only for the bytes read, and no span kept between awaits.

for a principal

Set the API convention: span parameters for synchronous code, Memory<T> for async or stored buffers, read-only variants by default, with ownership kept separate from the view.

## Two buffer types, two storage rules `Span<T>` and `Memory<T>` both describe a contiguous run of elements without copying it, but the compiler treats them very differently. - **`Span<T>`** (and `ReadOnlySpan<T>`) is a `ref struct`. It holds a managed reference that can point into the middle of an array, at a string's characters, at memory from `stackalloc`, or at native memory. Because such a reference must never outlive what it points at, the compiler only lets a span live on the stack: as a local, a parameter or a field of another `ref struct`. - **`Memory<T>`** (and `ReadOnlyMemory<T>`) is an ordinary `readonly struct`. It stores an object (a `T[]`, a `string` for `ReadOnlyMemory<char>`, or a `MemoryManager<T>`), a start index and a length. Nothing in it is a raw interior reference, so it can be a class field, a captured variable, a dictionary value or a local in an async method. `Memory<T>` cannot point at stack memory, which is exactly why it is allowed to go where the stack frame does not. ## What an await does to a local The compiler rewrites an `async` method into a **state machine**. Every local that is still needed after an `await` becomes a field of that state machine, and when the awaited operation has not finished, the state machine is moved to the heap so the method can resume later, possibly on another thread. A span stored there would be a stack-only value on the heap, and a span over `stackalloc` memory would point at a stack frame that no longer exists. So the compiler refuses. ## The compiler errors you meet | Code shape | C# 12 | C# 13 and later | |---|---|---| | `Span<byte>` parameter on an `async` method | error CS4012 | error CS4012 | | `Span<byte>` local used only between awaits | rejected (the C# 14 compiler set to `LangVersion` 12 reports CS9202, asking for 13) | allowed | | `Span<byte>` local used after an `await` | rejected | error CS4007 | | `scoped Span<byte>` or `ReadOnlySpan<byte>` local used after an `await` | rejected | error CS4007 | | `Memory<byte>` local or parameter across an `await` | allowed | allowed | CS4007 reads: "Instance of type 'System.Span<byte>' cannot be preserved across 'await' or 'yield' boundary." `scoped` does not help, because it narrows where a span may escape to; it does not let the value be hoisted. ## The Memory<T> pattern in an async pipeline In an upload service that reads 64 KB chunks from a network stream, hashes them and forwards them, the shape is: 1. Hold the buffer as `Memory<byte>` (often `rented.AsMemory(0, 64 * 1024)` over a pooled array). 2. Pass it to `Stream.ReadAsync(Memory<byte>, CancellationToken)`, which fills it and returns the count. 3. After the `await`, take `buffer.Span[..read]` for the synchronous work (hashing, parsing), and let that span die before the next `await`. The `.Span` property is the bridge: you take it at the point of use, in synchronous code, and never keep it. ## Choosing between them in an API The .NET Memory<T> usage guidelines give the rule of thumb: - **Synchronous API: take `Span<T>` or `ReadOnlySpan<T>`.** It accepts more kinds of memory (arrays, strings, stack and native memory), it is slightly cheaper to use, and the compiler enforces that the callee cannot keep the buffer after it returns. A caller with a `Memory<T>` passes `memory.Span`. - **Asynchronous API, or a type that stores the buffer: take `Memory<T>` or `ReadOnlyMemory<T>`.** A method that accepts one and returns a `Task` may use it only until that task completes; a constructor that accepts one makes every instance method a consumer of it. - **Read-only access: use the `ReadOnly` variant** so the signature says the callee will not write. There is no conversion from `Span<T>` back to `Memory<T>`; the direction is always memory to span. ## Costs and caveats - `Memory<T>.Span` is not free: the getter checks what kind of object backs the memory and validates the index and length, so take it once per synchronous section rather than in an inner loop. - `Memory<T>` keeps the array reachable for the garbage collector, but it does not own the buffer's lifetime. If the array was rented from a pool and returned, a `Memory<byte>` that still points at it reads whatever the next renter writes. - A `Memory<T>` field holds no lock and no lease; if two components use it at once, they need their own synchronisation.

  • If Memory<T> works in more places, why not use it for synchronous parameters too?
    Because `Span<T>` is the more general and cheaper parameter. It accepts arrays, strings, `stackalloc` and native memory; `Memory<T>.Span` has to inspect the backing object on every call; and a span parameter lets the compiler prove the callee cannot keep the buffer after returning. A caller holding a `Memory<T>` simply passes `memory.Span`.
  • Can a Span<byte> local appear in an async method at all?
    Since C# 13, yes, provided no `await` separates its uses: declare it after the `await`, use it, and let it go out of use before the next one. A span used after an `await` is still error CS4007, and a span parameter on an async method is still error CS4012. Under C# 12 any span local in an async method was rejected.
  • Does a Memory<byte> field keep its buffer safe from reuse?
    It keeps the array reachable, so the garbage collector will not reclaim it, but it does not reserve it. If the array came from `ArrayPool<byte>` and was returned, the `Memory<byte>` still points at it and sees the next renter's bytes. Lifetime is the owner's job, not the view's.

  • Span<T> is banned across await only because it is slower than Memory<T>.
  • Memory<T> is a ref struct that the compiler makes an exception for in async methods.
  • Marking the span local scoped lets it survive an await.
  • Memory<T> copies the data to the heap, so the source array can be reused at once.
  • On current C#, an async method can never declare a Span<T> local, even between awaits.

Assumes C# 14; since C# 13 a span local may appear in an async method if no await separates its uses, while C# 12 rejected any span local there.

open as a page

With .NET's ArrayPool<byte>.Shared, why can Rent(50_000) return a 65,536-byte array holding old data, and what must the caller track?

level: middleimportance: should knowfreq 42%

basics

~20 s

ArrayPool<byte>.Shared rounds requests up to power-of-two buckets and reuses returned arrays without clearing them, so Rent(50_000) can return 65,536 bytes with a previous renter's data. The caller must track how many bytes it actually uses.

open as a page

A C# upload service pools 64 KB buffers with ArrayPool<byte>.Shared, and forwarded chunks sometimes carry another upload's bytes; which rental bugs cause this?

level: seniorimportance: should knowfreq 32%

basics

~20 s

Two code paths share one array: a double Return let the pool hand it to two renters, or a stored Memory<byte> or unawaited task still used it after Return. ArrayPool<T>.Shared detects neither; return exactly once, after the last use.

open as a page

As a lead on a C# codebase, when would you allow ArrayPool<T> pooling, and how would your team catch pool misuse before it reaches production?

level: principalimportance: should knowfreq 18%

basics

~20 s

Allow pooling only on hot paths where measured buffer allocation matters, keep each rental under one owner (try/finally or an IMemoryOwner<T>), and catch misuse with review rules, an injected checking pool in tests and the pool's event source in production.

open as a page

In .NET, what does IMemoryOwner<T> from MemoryPool<T>.Shared.Rent add over a raw ArrayPool<T> rental, and who must dispose it?

level: seniorimportance: nice to knowfreq 22%

basics

~10 s

IMemoryOwner<T> turns ownership into an object: whoever holds it must either call Dispose, which returns the pooled array, or hand it on and stop using it. Consumers receive only Memory<T>, which carries no ownership.

open as a page