skip to content

Iteration Machinery

yield return turns a method into a lazy state machine, and foreach works through the enumerator pattern. Interviewers ask when iterator code actually runs.

on this pageshow

explore

questions

15

In C#, how do you write a Where-style extension method over `IEnumerable<T>` with `yield return`, and how does it stream inside a pipeline?

level: juniorimportance: must knowfreq 60%

answer

  1. static class, this parameter
  2. return type is an iterator interface
  3. foreach over source, yield matches
  4. one element per MoveNext
  5. Take stops the pulling

basics

~20 s

Write a static method in a non-generic static class, take this IEnumerable<T> source, return IEnumerable<T> and yield return each match from a foreach. Nothing runs at the call; each element is then read, tested and passed on before the next.

solid answer

~40 s

The operator is a static method in a non-generic static class whose first parameter is `this IEnumerable<T> source`, plus a `Func<T, bool>` predicate, and it returns `IEnumerable<T>`. The body is an iterator: `foreach (T item in source) if (!predicate(item)) yield return item;`. Calling it only builds an object; each `MoveNext` from the consumer resumes the loop until the next match, so a chain like `Sensor.Stream().WhereNot(...).Take(2)` reads, tests and hands on one reading at a time and stops reading when `Take` has its two items, even on an endless stream. The return type must be an iterator interface: declaring `List<T>` with `yield` fails with CS1624, and filling a `List<T>` and returning it instead reads the whole source before the first result.

code

csharp · 30 lines
csharp
var readings = Sensor.Stream();          // endless source
var alerts = readings.WhereNot(r => r.Celsius < 30).Take(2);
Console.WriteLine("pipeline built");
foreach (var r in alerts)
    Console.WriteLine($"alert #{r.Id}: {r.Celsius}");

record Reading(int Id, int Celsius);

static class Sensor
{
    public static IEnumerable<Reading> Stream()
    {
        int[] temps = [21, 34, 25, 31, 40];
        for (int id = 0; ; id++)
        {
            Console.WriteLine($"  read #{id}");
            yield return new Reading(id, temps[id % temps.Length]);
        }
    }
}

static class SequenceOps
{
    public static IEnumerable<T> WhereNot<T>(this IEnumerable<T> source, Func<T, bool> predicate)
    {
        foreach (T item in source)
            if (!predicate(item))
                yield return item;
    }
}

go deeper

for a junior

Recall the four parts: a non-generic static class, the this parameter on IEnumerable<T>, an IEnumerable<T> return type, and yield return inside a foreach over the source.

for a middle

Explain that the call only builds an object and that each MoveNext resumes the loop, then trace one element through a chain with Take to show the interleaving.

for a senior

Point out how a hidden ToList or a concrete return type turns the operator eager, and why the built-in operators validate arguments before the lazy part starts.

for a principal

Frame custom operators as a team asset: small, documented as streaming or buffering, tested on endless and single-use sources, so pipelines stay predictable in cost.

## The shape of a custom operator A **custom sequence operator** is an extension method that takes a sequence and returns a sequence, so it chains with the built-in ones. Four pieces make it a lazy, streaming operator: 1. A **non-generic static class** to hold it. Putting an extension method in a generic class fails with CS1106, "Extension method must be defined in a non-generic static class". 2. A first parameter marked `this IEnumerable<T> source`, which lets callers write `readings.WhereNot(...)`. 3. A return type of `IEnumerable<T>` (or `IEnumerator<T>`, or the non-generic forms). Apart from the async-stream interfaces used by async iterators, these are the only return types an iterator may declare. 4. A body that uses `yield return` inside a `foreach` over `source`. ```csharp static class SequenceOps { public static IEnumerable<T> WhereNot<T>(this IEnumerable<T> source, Func<T, bool> predicate) { foreach (T item in source) if (!predicate(item)) yield return item; } } ``` ## What the compiler makes of it Because the body contains `yield return`, the method is an **iterator**. The compiler rewrites it into a hidden class that implements `IEnumerable<T>` and `IEnumerator<T>`; calling `WhereNot` just creates that object and stores `source` and `predicate` in it. None of the loop runs yet. (How that state machine is laid out is covered in the lesson on yield-based methods.) When a consumer calls `MoveNext`, the loop starts or resumes where it last stopped, runs until the next `yield return`, publishes that item as `Current` and pauses again. When the loop ends, `MoveNext` returns `false`. ## How elements flow through a pipeline Chain the operator between an endless sensor stream and `Take(2)`: - The consumer's `foreach` asks `Take` for an element. - `Take` asks `WhereNot` for one; `WhereNot` asks the sensor iterator for readings, one at a time, until one passes the predicate. - That reading goes straight back up to the consumer before any further reading is taken from the sensor. - After the second match, `Take` checks its count before pulling again, sees it is done, and disposes the chain. The sensor is never asked for another reading. In the example on this page the console shows `pipeline built` first, then `read #0`, `read #1`, `alert #1`, `read #2`, `read #3`, `alert #3`, and nothing more. Work is **interleaved** across the stages: no stage holds a collection of results, and the total work is only what the consumer asked for. ## Streaming versus building a list | Body style | When the source is read | Endless source | Memory | |---|---|---|---| | `yield return` inside `foreach` | one element per consumer `MoveNext` | fine once something like `Take` stops pulling | the current element | | fill a `List<T>`, then `return list;` | all of it, during the call | the call never returns | the whole result | The list version is a legitimate method, but it is an eager one: it reads the full source at the call and can never be cut short by a later `Take`. ## Mistakes that break the operator - **Declaring a concrete return type.** `public static List<T> WhereNot<T>(...)` with `yield return` inside fails with CS1624: "The body of '...' cannot be an iterator block because 'List<T>' is not an iterator interface type". - **Mixing in `return value;`.** An iterator ends with `yield break`; `return someList;` inside one fails with CS1622, "Cannot return a value from an iterator". - **Materialising inside.** A `source.ToList()` at the top quietly turns a streaming operator into a buffering one. - **Expecting a null `source` to fail at the call.** In this form the `foreach` only runs at the first `MoveNext`, so a null `source` throws `NullReferenceException` then; the built-in operators check arguments eagerly with a non-iterator wrapper, which the lesson on local functions covers. - **Reading `source` twice** (for example `Any()` followed by `foreach`), which reruns or loses work on some sources; that hazard has its own question. ## Signatures that compose well - Make the operator generic over `T` and take `IEnumerable<T>`, so it works on arrays, lists, other operators' results and endless iterators alike. - Take behaviour as a delegate (`Func<T, bool>`, `Func<T, TKey>`) rather than hard-coding a rule, the way `Where` and `Select` do. - Return `IEnumerable<T>` even when the body could produce something richer, so the next operator in the chain can stay lazy. Written this way, a dozen small operators compose into one pipeline that touches each element once, in order, and only as far as the consumer goes.

  • Why does `pipeline built` print before the sensor reads anything?
    Calling `WhereNot` and `Take` only creates their enumerable objects and stores the arguments. The iterator body, and the sensor iterator it wraps, start running when the consumer's `foreach` calls `GetEnumerator` and the first `MoveNext`.
  • What happens if you keep the `yield return` body but declare the return type as `List<T>`?
    The build fails with CS1624: the body cannot be an iterator block because `List<T>` is not an iterator interface type. An iterator must return `IEnumerable<T>`, `IEnumerator<T>` or their non-generic forms; to hand back a list you would drop `yield` and build the list eagerly.
  • Where does a null `source` passed to this `WhereNot` get reported?
    Not at the call. The `foreach` over `source` sits inside the iterator body, so the `NullReferenceException` surfaces on the consumer's first `MoveNext`. Built-in operators throw `ArgumentNullException` at the call by splitting a plain public method from a private iterator.

  • yield return builds the full result list before the method returns
  • An extension method can live in a generic static class
  • The operator should return List<T> so callers get a real collection
  • Each operator in a chain runs over the whole source before the next starts
  • Calling the operator runs its body up to the first yield return
open as a page

In C#, what does a type need for foreach to compile over it, and what does the compiler turn the loop into?

level: juniorimportance: must knowfreq 62%

basics

~20 s

foreach needs a GetEnumerator() method whose return type has a public bool MoveNext() and a readable Current; no interface is required. The compiler lowers the loop to a while over MoveNext inside a try/finally that disposes a disposable enumerator.

open as a page

In a C# method that returns IEnumerable<T>, what do yield return and yield break do, and why is a plain return rejected?

level: juniorimportance: must knowfreq 70%

basics

~20 s

yield return hands the caller the next element and pauses the method until another is requested; yield break ends the sequence. Any yield makes the method an iterator, so a return statement, even a bare return;, is error CS1622.

open as a page

In C#, when does the body of an iterator method actually start running, and what happens if its result is enumerated twice?

level: middleimportance: must knowfreq 68%

basics

~20 s

Calling an iterator runs none of its body; it only returns an object. The body starts on the first MoveNext, pauses at each yield return, and a second foreach runs it again from the top, side effects included.

open as a page

In C#, why isn't .NET's `Chunk(3)` a sliding window over sensor readings, and how do you write a `WindowBy` operator that yields safe windows?

level: middleimportance: should knowfreq 30%

basics

~20 s

Chunk(3) splits the source into disjoint arrays of three, with a shorter last one; a sliding window overlaps and advances by one. WindowBy keeps the last size elements in a Queue<T> and yields a fresh array per window, so consumers can keep each one.

open as a page

In C#, what may a custom `Pairwise` operator over an endless `IEnumerable<Reading>` hold between elements, and what would make it hang before `Take(5)`?

level: middleimportance: should knowfreq 45%

basics

~20 s

A streaming Pairwise keeps only the previous element and a flag, and is safe on an endless source. It hangs if it, or a stage before Take, must read to the end first, as ToList, ToArray, Count, Max, Reverse or OrderBy do.

open as a page

When you implement IEnumerable<T> and IEnumerator<T> by hand in C#, which members must you write and what does each one promise?

level: middleimportance: should knowfreq 38%

basics

~20 s

IEnumerable<T> needs GetEnumerator() plus the non-generic one, each returning a fresh cursor. IEnumerator<T> needs MoveNext, Current, the non-generic Current, Reset and Dispose; Current is undefined before the first MoveNext and after the end, and foreach never calls Reset.

open as a page

In C#, why does List<T>.GetEnumerator return a public struct, and why does foreach over the same list allocate when it is typed as IEnumerable<T>?

level: middleimportance: should knowfreq 42%

basics

~20 s

List<T>.Enumerator is a struct so foreach over a List<T> keeps the enumerator in a local with no heap allocation. Through IEnumerable<T>, GetEnumerator must return IEnumerator<T>, so the struct is boxed once per loop and each call becomes an interface call.

open as a page

A C# iterator opens a log file and closes it in finally; when does that finally run if a foreach breaks after 20 errors?

level: middleimportance: should knowfreq 50%

basics

~20 s

At the break: foreach calls Dispose on the iterator's enumerator, and the generated Dispose runs the finally blocks around the paused yield return, closing the file. Code that calls MoveNext by hand without disposing leaves the file open.

open as a page

What does the C# compiler generate for an iterator method returning IEnumerable<T>, and what jobs do its state and hoisted fields do?

level: middleimportance: should knowfreq 40%

basics

~20 s

A private nested sealed class implementing IEnumerable<T>, IEnumerator<T> and IDisposable. Parameters and locals that live across a yield become fields, a state field records where the body paused, and MoveNext jumps back there; the method itself just creates the object.

open as a page

A C# custom `Pairwise` operator calls `source.GetEnumerator()` directly; why does the sensor stream's `finally` never close the port when the consumer stops at `Take(1)`?

level: seniorimportance: should knowfreq 30%

basics

~20 s

The operator holds the source enumerator in a plain local and never disposes it. When Take stops early, Dispose reaches the operator's iterator but not the source's, so the source's finally never runs. Declare it with using IEnumerator<T> e = source.GetEnumerator().

open as a page

A C# custom operator calls `source.Max()` and then loops over `source` to scale each reading; why does it print 104% on a live sensor feed, and how do you fix it?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Max() and the foreach each enumerate the source, so a live feed is read twice and the second read returns newer, higher readings than the peak. Read the source once: keep one enumerator, or copy it into an array and compute over the copy.

open as a page

With .NET's Dictionary<TKey,TValue>, why can a foreach remove entries without an exception while adding one throws InvalidOperationException?

level: seniorimportance: should knowfreq 35%

basics

~20 s

Enumerators snapshot the collection's version and MoveNext throws InvalidOperationException when it changed. Since .NET Core 3.0 Dictionary's Remove and Clear no longer bump the version, because removal frees a slot without moving entries; adding a key still does.

open as a page

A C# iterator wraps yield return int.Parse(row) in try/catch and fails with CS1626; why, and how do you restructure it to skip bad rows?

level: seniorimportance: should knowfreq 35%

basics

~20 s

C# forbids yield return inside a try block that has a catch (CS1626) and inside a catch block (CS1631). Move the try/catch into an ordinary helper that parses one row and reports success, then yield outside the try.

open as a page

In C#, why does a class that keeps a List<T>.Enumerator in a readonly field return the default value forever instead of advancing?

level: seniorimportance: nice to knowfreq 20%

basics

~10 s

List<T>.Enumerator is a mutable struct, and calling MoveNext on a readonly struct field runs on a defensive copy. The field never advances, so MoveNext keeps returning true and Current keeps returning default(T).

open as a page