skip to content

In C#, why can a token passed with WithCancellation fail to reach an async iterator, and what does [EnumeratorCancellation] change?

level: seniorimportance: should knowfreq 45%

answer

  1. GetAsyncEnumerator takes the token
  2. the compiler needs to know which parameter
  3. warning CS8425 on an undecorated token
  4. both tokens become one combined token

basics

~20 s

WithCancellation passes its token to GetAsyncEnumerator, and an async iterator ignores that token unless one CancellationToken parameter carries [EnumeratorCancellation]. With the attribute, the body sees that token, combined with any token passed as an argument.

solid answer

~40 s

`IAsyncEnumerable<T>` has one place to receive a token: `GetAsyncEnumerator(CancellationToken)`; `MoveNextAsync()` takes none. `WithCancellation(token)` and the `CancellationToken` parameter of the `AsyncEnumerable` operators feed that method. For an async iterator the compiler generates `GetAsyncEnumerator`, and it only knows where to put the token if one `CancellationToken` parameter is marked `[EnumeratorCancellation]` (from `System.Runtime.CompilerServices`). Without it the token is dropped, the loop runs to the end, and the compiler warns with CS8425. With it, a token passed to `GetAsyncEnumerator` reaches the body through that parameter; if the caller also passed a token as an argument, the body sees a combined token that is cancelled when either one is. Cancellation stays cooperative: the body must pass the token to its awaits or check it.

code

csharp · 14 lines
csharp
using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(250));
int count = 0;
await foreach (int id in TailAsync().WithCancellation(cts.Token))
    count++;
Console.WriteLine($"loop finished normally after {count} items");

static async IAsyncEnumerable<int> TailAsync(CancellationToken token = default)
{
    for (int id = 1; id <= 10; id++)
    {
        await Task.Delay(100, token);
        yield return id;
    }
}

go deeper

for a junior

Recall that an async iterator should take a CancellationToken parameter marked [EnumeratorCancellation] and pass it to the awaits inside.

for a middle

Explain the single entry point, GetAsyncEnumerator, and how WithCancellation reaches it, so you can say why an undecorated parameter never sees the token.

for a senior

Diagnose the silent no-cancel bug from warning CS8425, know that both tokens are combined, and make the build fail on that warning in shared code.

for a principal

Argue for a team convention that every public async iterator takes a decorated token and that CS8425 is an error, since the failure is otherwise invisible.

## Where a token can enter an async sequence Cancelling a loop over an `IAsyncEnumerable<T>` is harder than cancelling a single `Task`, because the interface gives a token exactly one entry point: ```csharp public interface IAsyncEnumerable<out T> { IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default); } ``` `IAsyncEnumerator<T>.MoveNextAsync()` takes no token. So a consumer that holds only the sequence, perhaps one returned from another method, can cancel only by giving a token to `GetAsyncEnumerator`. That is what `WithCancellation` does: ```csharp await foreach (var entry in source.WithCancellation(token)) { } ``` `WithCancellation` (on `TaskAsyncEnumerableExtensions`) returns a `ConfiguredCancelableAsyncEnumerable<T>`, and `await foreach` over it calls `GetAsyncEnumerator(token)` on the source. ## Why the token can vanish When the source is an async iterator, a method marked `async` returning `IAsyncEnumerable<T>` with `yield return`, the compiler writes `GetAsyncEnumerator` for you. The body cannot name the token passed to it; it can only use its own parameters. So the compiler needs to be told which parameter should receive that token: ```csharp static async IAsyncEnumerable<LogEntry> TailAsync( [EnumeratorCancellation] CancellationToken token = default) ``` Without the attribute, the generated `GetAsyncEnumerator` has nowhere to put the token, and it is **silently discarded**. In the first example below, a `CancellationTokenSource` that fires after 250 ms changes nothing: the loop reads all ten entries and finishes normally. The compiler does flag the risk: - **warning CS8425**: "Async-iterator 'TailAsync(CancellationToken)' has one or more parameters of type 'CancellationToken' but none of them is decorated with the 'EnumeratorCancellation' attribute, so the cancellation token parameter from the generated 'IAsyncEnumerable<>.GetAsyncEnumerator' will be unconsumed"; - **warning CS8424** when the attribute sits where it has no effect, for example on a parameter of an ordinary `async Task` method; - **error CS8426** when two parameters carry it. Teams that treat warnings as errors turn CS8425 into a build break, which is the cheapest guard against this bug. ## What the attribute does at run time With `[EnumeratorCancellation]` in place, the generated code decides what the body's `token` parameter holds when enumeration starts: | Argument passed to the method | Token passed to `GetAsyncEnumerator` | Token the body sees | |---|---|---| | none (default) | none | `CancellationToken.None` | | `a` | none | `a` | | none | `b` (via `WithCancellation`) | `b` | | `a` | `b` | a combined token, cancelled when `a` **or** `b` is | The last row is worth remembering. An experiment on .NET 10 cancelled only the argument's source in one run and only the `WithCancellation` source in another; both stopped the loop, and the `OperationCanceledException` carried neither original token, which is what a linked token source produces. The C# 8 design notes describe the enumerator's token as superseding the argument, but the shipped compiler combines them, and the compiler is what runs. ## Cancellation is still cooperative The attribute only delivers the token. The iterator must use it: 1. pass `token` to every awaited operation that accepts one, such as `Task.Delay(100, token)` or an HTTP call; 2. call `token.ThrowIfCancellationRequested()` in loops that do CPU work between awaits; 3. let the resulting `OperationCanceledException`, often its subclass `TaskCanceledException`, propagate. The exception surfaces from `MoveNextAsync()`, so `await foreach` rethrows it inside the consumer, after awaiting `DisposeAsync()`, which runs the iterator's `finally` blocks. In the second example the loop stops after two entries with `TaskCanceledException`. ## Picking the right call site - **You call the iterator method yourself:** pass the token as an argument, `TailAsync(token)`. - **You receive an `IAsyncEnumerable<T>` from elsewhere:** use `source.WithCancellation(token)`. - **You write the iterator:** always mark its `CancellationToken` parameter with `[EnumeratorCancellation]`, so both styles work. ## Proving it in a test A cancellation bug here is silent, so it deserves a test of its own: - create a `CancellationTokenSource` with a short timeout, enumerate through `WithCancellation(cts.Token)`, and assert that the loop throws `OperationCanceledException` (or a subclass) before it reads every element; - repeat with the token passed as the method argument, so both entry points are covered; - use a source that would run far longer than the timeout, so a passing test cannot be a race the loop won by finishing first. The `CancellationTokenSource` mechanics themselves, linked sources, timeouts and callbacks, are covered with cancellation tokens; what is specific here is the single entry point and the attribute that wires it to the iterator.

  • Does a token passed to MoveNextAsync work instead?
    There is no such overload: `IAsyncEnumerator<T>.MoveNextAsync()` takes no parameters. The design deliberately puts the token on `GetAsyncEnumerator`, once per enumeration, so the only ways in are an argument to the iterator method or a token handed to `GetAsyncEnumerator`, which `WithCancellation` and `await foreach` do for you.
  • Can you combine WithCancellation with ConfigureAwait(false) on the same loop?
    Yes. Both return `ConfiguredCancelableAsyncEnumerable<T>`, which has its own `WithCancellation` and `ConfigureAwait` methods, so `source.WithCancellation(token).ConfigureAwait(false)` chains in either order and `await foreach` honours both settings.
  • Why does the exception after cancellation not carry the token the caller cancelled?
    When the iterator receives a token both as an argument and through `GetAsyncEnumerator`, it observes a combined token that is cancelled when either is. Operations inside the body were given that combined token, so the `OperationCanceledException` they throw reports it. Check `token.IsCancellationRequested` on your own source rather than comparing `ex.CancellationToken` to it.

  • WithCancellation cancels any IAsyncEnumerable<T>, even an iterator that never uses the token.
  • Any CancellationToken parameter on an async iterator receives the WithCancellation token.
  • Pass the token to MoveNextAsync on each iteration to cancel the loop.
  • When both are given, the WithCancellation token replaces the argument token.
  • [EnumeratorCancellation] stops the iterator by itself, even without checking the token.