skip to content

In C#, why does a ref struct PacketReader compile in a synchronous method but fail when used after an await or a yield return?

level: middleimportance: should knowfreq 40%

answer

  1. the method is rewritten
  2. hoisted locals become fields
  3. state machine may live on heap
  4. C# 13 allowed the declaration
  5. live across the suspension point

basics

~20 s

Locals used after an await or yield return are hoisted into a state machine that can live on the heap, where a ref struct may not go. Since C# 13 a ref struct local compiles only if it is never used after one (CS4007).

solid answer

~50 s

The compiler rewrites an async method or an iterator into a state machine, and every local needed after a suspension point becomes one of its fields. An iterator's state machine is a class, and an async method's is moved to the heap when it first suspends, so a hoisted `PacketReader` would put its span on the heap. Up to C# 12 the compiler rejected a ref struct local anywhere in an async method. C# 13 narrowed the rule for async methods and iterators alike: the local is allowed, but using it after an `await` or `yield return` is CS4007, "Instance of type 'PacketReader' cannot be preserved across 'await' or 'yield' boundary." Compiler-generated uses count, so a `foreach` over a span with an `await` in its body fails too. The repair is to finish with the reader before the `await`, in a block or a synchronous helper, and carry plain values or a `Memory<T>` across it.

code

csharp · 28 lines
csharp
using System;
using System.Threading.Tasks;

byte[] packet = { 2, 0x10, 0x20 };
Console.WriteLine(await HandleAsync(packet));

static async Task<int> HandleAsync(byte[] packet)
{
    int kind;
    int length;
    {
        var reader = new PacketReader(packet);   // fine in an async method since C# 13
        kind = reader.ReadByte();
        length = reader.Remaining;               // last use before the await
    }
    await Task.Delay(10);                        // no PacketReader is live here
    // a reader declared above the block and read here: CS4007
    return kind * 100 + length;
}

ref struct PacketReader
{
    private readonly ReadOnlySpan<byte> _buffer;
    private int _pos;
    public PacketReader(ReadOnlySpan<byte> buffer) => _buffer = buffer;
    public int Remaining => _buffer.Length - _pos;
    public byte ReadByte() => _buffer[_pos++];
}

go deeper

for a junior

Remember that a span or a ref struct reader cannot be used after an await or a yield return, and that the compiler, not the runtime, enforces it.

for a middle

Explain hoisting: locals needed after a suspension point become state machine fields, which can live on the heap, so a ref struct may not be among them.

for a senior

Spot the hidden uses, a foreach over a span or a using declaration spanning an await, and restructure with a synchronous helper or a Memory<T> handle.

for a principal

Know which code relies on the C# 13 relaxation, so a library that must also build with an older language version does not depend on it.

## What the compiler does to async methods and iterators An `async` method and an **iterator** (a method containing `yield return`) cannot keep their locals in an ordinary stack frame, because they return to the caller in the middle of their body and resume later. The compiler rewrites each into a **state machine**: a type with a `MoveNext` method that runs the body piece by piece, and fields that hold whatever must survive between pieces. Moving a local into such a field is called **hoisting**. - An iterator's state machine is a compiler-generated **class**, so its fields are on the managed heap from the start. - An async method's state machine is a struct while the method runs synchronously, and is **moved to the heap** the first time it awaits something that has not completed. Either way, a hoisted local can end up in a heap object. ## Why a ref struct cannot be hoisted A ref struct such as `Span<T>` or a `PacketReader` that wraps a `ReadOnlySpan<byte>` may never be a field of a heap object. Hoisting it would do exactly that, and the span could also point at stack memory that disappeared when the method returned to its caller at the `await`. So the compiler forbids any ref struct value whose lifetime crosses a suspension point. ## The rule before and after C# 13 | Situation | C# 12 | C# 13 and later | |---|---|---| | Ref struct local in an async method, not used after any `await` | error (CS9202 on the SDK 10 compiler with LangVersion 12) | allowed | | Ref struct local used after an `await` | error | CS4007 | | Ref struct local in an iterator, not used after any `yield return` | see below | allowed | | Ref struct local used after a `yield return` | error | CS4007 | | Ref struct captured by a lambda or local function | CS8175 | CS8175 | The C# 13 specification states the mechanism precisely: after an `await` or a `yield return`, every ref local and every ref struct variable in scope is considered definitely unassigned, so any read of it is rejected. A variable that is in scope but never read again compiles fine; the error comes from a **use** on the far side. For iterators the documentation says ref struct locals were barred entirely before C# 13, but the SDK 10 compiler run at LangVersion 12 accepts one that is not used across a `yield return`. The rule to rely on is the C# 13 one, which treats `await` and `yield return` the same way. ## Hidden uses the compiler counts The rule also applies to code the compiler writes for you: - `foreach (byte b in data.AsSpan()) { await SendAsync(b); }` fails with CS4007, because the loop keeps the span alive across the `await` in its body. - `using var reader = new PacketReader(data);` followed by an `await` fails with CS4007, because the generated `Dispose` call at the end of the scope is a use after the `await`. - A ref local such as `ref int slot = ref counts[0];` follows the same rule and reports CS9217 when used after an `await` or `yield return`. ## Repairs 1. **Finish before the await.** Do the span work first, copy the results into ordinary locals (`int kind`, `int length`), and only then await. A nested block makes the end of the reader's life visible. 2. **Move the span work into a synchronous helper.** `static int ParseHeader(byte[] packet)` can create and use a `PacketReader` freely; the async method calls it and awaits afterwards. 3. **Carry storable handles across the await.** Keep the bytes as an array with an offset, or as a `Memory<byte>`, and create a fresh reader over `.Span` after each await (the material on Memory<T> covers that type). 4. **Pass, don't capture.** Where a lambda or local function needs the data, make it a parameter instead of a captured variable. ## Why the error is a good one CS4007 is a **compile-time** diagnostic. Nothing fails at run time, and no thread or `SynchronizationContext` setting changes the outcome, because the problem is where the variable would be stored, not which thread resumes the method. Returning `ValueTask` or calling `ConfigureAwait(false)` therefore does not help either: the state machine still has to hold the value across the suspension.

  • Does the same rule cover ref locals in an async method?
    Yes. C# 13 also allowed ref locals such as `ref int slot = ref counts[0];` in async methods and iterators under the same condition. Reading `slot` after an `await` or `yield return` is CS9217, "A 'ref' local cannot be preserved across 'await' or 'yield' boundary." Before C# 13 the declaration itself was an error.
  • How do you keep parsing a packet that arrives in pieces across several awaits?
    Keep the state outside the ref struct. Hold the received bytes as an array or a `Memory<byte>` plus an `int` position in the async method, and after each await create a new `PacketReader` over the unread part, read what is complete, and store the new position. The reader never crosses an await; only the array, the handle and the integer do.

  • A ref struct can still never appear anywhere in an async method.
  • Declaring the span outside a block lets you read it again after the await.
  • Returning ValueTask keeps the state machine on the stack, so spans survive awaits.
  • ConfigureAwait(false) avoids the error because the method resumes on the same thread.
  • Using a span after an await compiles and fails only when the method resumes.

Assumes C# 14; C# 13 first allowed ref struct locals in async methods and iterators, provided none is used across an await or yield return.