In C#, why can a token passed with WithCancellation fail to reach an async iterator, and what does [EnumeratorCancellation] change?
answer
- GetAsyncEnumerator takes the token
- the compiler needs to know which parameter
- warning CS8425 on an undecorated token
- both tokens become one combined token
basics
~20 sWithCancellation passes its token to GetAsyncEnumerator, and an async iterator ignores that token unless one CancellationToken parameter carries [EnumeratorCancellation]. With the attribute, the body sees that token, combined with any token passed as an argument.
solid answer
~40 s`IAsyncEnumerable<T>` has one place to receive a token: `GetAsyncEnumerator(CancellationToken)`; `MoveNextAsync()` takes none. `WithCancellation(token)` and the `CancellationToken` parameter of the `AsyncEnumerable` operators feed that method. For an async iterator the compiler generates `GetAsyncEnumerator`, and it only knows where to put the token if one `CancellationToken` parameter is marked `[EnumeratorCancellation]` (from `System.Runtime.CompilerServices`). Without it the token is dropped, the loop runs to the end, and the compiler warns with CS8425. With it, a token passed to `GetAsyncEnumerator` reaches the body through that parameter; if the caller also passed a token as an argument, the body sees a combined token that is cancelled when either one is. Cancellation stays cooperative: the body must pass the token to its awaits or check it.
code
csharp · 14 linesusing var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(250));
int count = 0;
await foreach (int id in TailAsync().WithCancellation(cts.Token))
count++;
Console.WriteLine($"loop finished normally after {count} items");
static async IAsyncEnumerable<int> TailAsync(CancellationToken token = default)
{
for (int id = 1; id <= 10; id++)
{
await Task.Delay(100, token);
yield return id;
}
}go deeper
Recall that an async iterator should take a CancellationToken parameter marked [EnumeratorCancellation] and pass it to the awaits inside.
Explain the single entry point, GetAsyncEnumerator, and how WithCancellation reaches it, so you can say why an undecorated parameter never sees the token.
Diagnose the silent no-cancel bug from warning CS8425, know that both tokens are combined, and make the build fail on that warning in shared code.
Argue for a team convention that every public async iterator takes a decorated token and that CS8425 is an error, since the failure is otherwise invisible.
## Where a token can enter an async sequence Cancelling a loop over an `IAsyncEnumerable<T>` is harder than cancelling a single `Task`, because the interface gives a token exactly one entry point: ```csharp public interface IAsyncEnumerable<out T> { IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default); } ``` `IAsyncEnumerator<T>.MoveNextAsync()` takes no token. So a consumer that holds only the sequence, perhaps one returned from another method, can cancel only by giving a token to `GetAsyncEnumerator`. That is what `WithCancellation` does: ```csharp await foreach (var entry in source.WithCancellation(token)) { } ``` `WithCancellation` (on `TaskAsyncEnumerableExtensions`) returns a `ConfiguredCancelableAsyncEnumerable<T>`, and `await foreach` over it calls `GetAsyncEnumerator(token)` on the source. ## Why the token can vanish When the source is an async iterator, a method marked `async` returning `IAsyncEnumerable<T>` with `yield return`, the compiler writes `GetAsyncEnumerator` for you. The body cannot name the token passed to it; it can only use its own parameters. So the compiler needs to be told which parameter should receive that token: ```csharp static async IAsyncEnumerable<LogEntry> TailAsync( [EnumeratorCancellation] CancellationToken token = default) ``` Without the attribute, the generated `GetAsyncEnumerator` has nowhere to put the token, and it is **silently discarded**. In the first example below, a `CancellationTokenSource` that fires after 250 ms changes nothing: the loop reads all ten entries and finishes normally. The compiler does flag the risk: - **warning CS8425**: "Async-iterator 'TailAsync(CancellationToken)' has one or more parameters of type 'CancellationToken' but none of them is decorated with the 'EnumeratorCancellation' attribute, so the cancellation token parameter from the generated 'IAsyncEnumerable<>.GetAsyncEnumerator' will be unconsumed"; - **warning CS8424** when the attribute sits where it has no effect, for example on a parameter of an ordinary `async Task` method; - **error CS8426** when two parameters carry it. Teams that treat warnings as errors turn CS8425 into a build break, which is the cheapest guard against this bug. ## What the attribute does at run time With `[EnumeratorCancellation]` in place, the generated code decides what the body's `token` parameter holds when enumeration starts: | Argument passed to the method | Token passed to `GetAsyncEnumerator` | Token the body sees | |---|---|---| | none (default) | none | `CancellationToken.None` | | `a` | none | `a` | | none | `b` (via `WithCancellation`) | `b` | | `a` | `b` | a combined token, cancelled when `a` **or** `b` is | The last row is worth remembering. An experiment on .NET 10 cancelled only the argument's source in one run and only the `WithCancellation` source in another; both stopped the loop, and the `OperationCanceledException` carried neither original token, which is what a linked token source produces. The C# 8 design notes describe the enumerator's token as superseding the argument, but the shipped compiler combines them, and the compiler is what runs. ## Cancellation is still cooperative The attribute only delivers the token. The iterator must use it: 1. pass `token` to every awaited operation that accepts one, such as `Task.Delay(100, token)` or an HTTP call; 2. call `token.ThrowIfCancellationRequested()` in loops that do CPU work between awaits; 3. let the resulting `OperationCanceledException`, often its subclass `TaskCanceledException`, propagate. The exception surfaces from `MoveNextAsync()`, so `await foreach` rethrows it inside the consumer, after awaiting `DisposeAsync()`, which runs the iterator's `finally` blocks. In the second example the loop stops after two entries with `TaskCanceledException`. ## Picking the right call site - **You call the iterator method yourself:** pass the token as an argument, `TailAsync(token)`. - **You receive an `IAsyncEnumerable<T>` from elsewhere:** use `source.WithCancellation(token)`. - **You write the iterator:** always mark its `CancellationToken` parameter with `[EnumeratorCancellation]`, so both styles work. ## Proving it in a test A cancellation bug here is silent, so it deserves a test of its own: - create a `CancellationTokenSource` with a short timeout, enumerate through `WithCancellation(cts.Token)`, and assert that the loop throws `OperationCanceledException` (or a subclass) before it reads every element; - repeat with the token passed as the method argument, so both entry points are covered; - use a source that would run far longer than the timeout, so a passing test cannot be a race the loop won by finishing first. The `CancellationTokenSource` mechanics themselves, linked sources, timeouts and callbacks, are covered with cancellation tokens; what is specific here is the single entry point and the attribute that wires it to the iterator.
- Does a token passed to MoveNextAsync work instead?There is no such overload: `IAsyncEnumerator<T>.MoveNextAsync()` takes no parameters. The design deliberately puts the token on `GetAsyncEnumerator`, once per enumeration, so the only ways in are an argument to the iterator method or a token handed to `GetAsyncEnumerator`, which `WithCancellation` and `await foreach` do for you.
- Can you combine WithCancellation with ConfigureAwait(false) on the same loop?Yes. Both return `ConfiguredCancelableAsyncEnumerable<T>`, which has its own `WithCancellation` and `ConfigureAwait` methods, so `source.WithCancellation(token).ConfigureAwait(false)` chains in either order and `await foreach` honours both settings.
- Why does the exception after cancellation not carry the token the caller cancelled?When the iterator receives a token both as an argument and through `GetAsyncEnumerator`, it observes a combined token that is cancelled when either is. Operations inside the body were given that combined token, so the `OperationCanceledException` they throw reports it. Check `token.IsCancellationRequested` on your own source rather than comparing `ex.CancellationToken` to it.
- WithCancellation cancels any IAsyncEnumerable<T>, even an iterator that never uses the token.
- Any CancellationToken parameter on an async iterator receives the WithCancellation token.
- Pass the token to MoveNextAsync on each iteration to cancel the loop.
- When both are given, the WithCancellation token replaces the argument token.
- [EnumeratorCancellation] stops the iterator by itself, even without checking the token.