skip to content

Custom Sequence Operators

Writing your own Where-style extension with yield gives a lazy, composable pipeline, even over infinite sequences. Interviewers ask how to write one that streams.

on this pageshow

explore

questions

5

In C#, how do you write a Where-style extension method over `IEnumerable<T>` with `yield return`, and how does it stream inside a pipeline?

level: juniorimportance: must knowfreq 60%

answer

  1. static class, this parameter
  2. return type is an iterator interface
  3. foreach over source, yield matches
  4. one element per MoveNext
  5. Take stops the pulling

basics

~20 s

Write a static method in a non-generic static class, take this IEnumerable<T> source, return IEnumerable<T> and yield return each match from a foreach. Nothing runs at the call; each element is then read, tested and passed on before the next.

solid answer

~40 s

The operator is a static method in a non-generic static class whose first parameter is `this IEnumerable<T> source`, plus a `Func<T, bool>` predicate, and it returns `IEnumerable<T>`. The body is an iterator: `foreach (T item in source) if (!predicate(item)) yield return item;`. Calling it only builds an object; each `MoveNext` from the consumer resumes the loop until the next match, so a chain like `Sensor.Stream().WhereNot(...).Take(2)` reads, tests and hands on one reading at a time and stops reading when `Take` has its two items, even on an endless stream. The return type must be an iterator interface: declaring `List<T>` with `yield` fails with CS1624, and filling a `List<T>` and returning it instead reads the whole source before the first result.

code

csharp · 30 lines
csharp
var readings = Sensor.Stream();          // endless source
var alerts = readings.WhereNot(r => r.Celsius < 30).Take(2);
Console.WriteLine("pipeline built");
foreach (var r in alerts)
    Console.WriteLine($"alert #{r.Id}: {r.Celsius}");

record Reading(int Id, int Celsius);

static class Sensor
{
    public static IEnumerable<Reading> Stream()
    {
        int[] temps = [21, 34, 25, 31, 40];
        for (int id = 0; ; id++)
        {
            Console.WriteLine($"  read #{id}");
            yield return new Reading(id, temps[id % temps.Length]);
        }
    }
}

static class SequenceOps
{
    public static IEnumerable<T> WhereNot<T>(this IEnumerable<T> source, Func<T, bool> predicate)
    {
        foreach (T item in source)
            if (!predicate(item))
                yield return item;
    }
}

go deeper

for a junior

Recall the four parts: a non-generic static class, the this parameter on IEnumerable<T>, an IEnumerable<T> return type, and yield return inside a foreach over the source.

for a middle

Explain that the call only builds an object and that each MoveNext resumes the loop, then trace one element through a chain with Take to show the interleaving.

for a senior

Point out how a hidden ToList or a concrete return type turns the operator eager, and why the built-in operators validate arguments before the lazy part starts.

for a principal

Frame custom operators as a team asset: small, documented as streaming or buffering, tested on endless and single-use sources, so pipelines stay predictable in cost.

## The shape of a custom operator A **custom sequence operator** is an extension method that takes a sequence and returns a sequence, so it chains with the built-in ones. Four pieces make it a lazy, streaming operator: 1. A **non-generic static class** to hold it. Putting an extension method in a generic class fails with CS1106, "Extension method must be defined in a non-generic static class". 2. A first parameter marked `this IEnumerable<T> source`, which lets callers write `readings.WhereNot(...)`. 3. A return type of `IEnumerable<T>` (or `IEnumerator<T>`, or the non-generic forms). Apart from the async-stream interfaces used by async iterators, these are the only return types an iterator may declare. 4. A body that uses `yield return` inside a `foreach` over `source`. ```csharp static class SequenceOps { public static IEnumerable<T> WhereNot<T>(this IEnumerable<T> source, Func<T, bool> predicate) { foreach (T item in source) if (!predicate(item)) yield return item; } } ``` ## What the compiler makes of it Because the body contains `yield return`, the method is an **iterator**. The compiler rewrites it into a hidden class that implements `IEnumerable<T>` and `IEnumerator<T>`; calling `WhereNot` just creates that object and stores `source` and `predicate` in it. None of the loop runs yet. (How that state machine is laid out is covered in the lesson on yield-based methods.) When a consumer calls `MoveNext`, the loop starts or resumes where it last stopped, runs until the next `yield return`, publishes that item as `Current` and pauses again. When the loop ends, `MoveNext` returns `false`. ## How elements flow through a pipeline Chain the operator between an endless sensor stream and `Take(2)`: - The consumer's `foreach` asks `Take` for an element. - `Take` asks `WhereNot` for one; `WhereNot` asks the sensor iterator for readings, one at a time, until one passes the predicate. - That reading goes straight back up to the consumer before any further reading is taken from the sensor. - After the second match, `Take` checks its count before pulling again, sees it is done, and disposes the chain. The sensor is never asked for another reading. In the example on this page the console shows `pipeline built` first, then `read #0`, `read #1`, `alert #1`, `read #2`, `read #3`, `alert #3`, and nothing more. Work is **interleaved** across the stages: no stage holds a collection of results, and the total work is only what the consumer asked for. ## Streaming versus building a list | Body style | When the source is read | Endless source | Memory | |---|---|---|---| | `yield return` inside `foreach` | one element per consumer `MoveNext` | fine once something like `Take` stops pulling | the current element | | fill a `List<T>`, then `return list;` | all of it, during the call | the call never returns | the whole result | The list version is a legitimate method, but it is an eager one: it reads the full source at the call and can never be cut short by a later `Take`. ## Mistakes that break the operator - **Declaring a concrete return type.** `public static List<T> WhereNot<T>(...)` with `yield return` inside fails with CS1624: "The body of '...' cannot be an iterator block because 'List<T>' is not an iterator interface type". - **Mixing in `return value;`.** An iterator ends with `yield break`; `return someList;` inside one fails with CS1622, "Cannot return a value from an iterator". - **Materialising inside.** A `source.ToList()` at the top quietly turns a streaming operator into a buffering one. - **Expecting a null `source` to fail at the call.** In this form the `foreach` only runs at the first `MoveNext`, so a null `source` throws `NullReferenceException` then; the built-in operators check arguments eagerly with a non-iterator wrapper, which the lesson on local functions covers. - **Reading `source` twice** (for example `Any()` followed by `foreach`), which reruns or loses work on some sources; that hazard has its own question. ## Signatures that compose well - Make the operator generic over `T` and take `IEnumerable<T>`, so it works on arrays, lists, other operators' results and endless iterators alike. - Take behaviour as a delegate (`Func<T, bool>`, `Func<T, TKey>`) rather than hard-coding a rule, the way `Where` and `Select` do. - Return `IEnumerable<T>` even when the body could produce something richer, so the next operator in the chain can stay lazy. Written this way, a dozen small operators compose into one pipeline that touches each element once, in order, and only as far as the consumer goes.

  • Why does `pipeline built` print before the sensor reads anything?
    Calling `WhereNot` and `Take` only creates their enumerable objects and stores the arguments. The iterator body, and the sensor iterator it wraps, start running when the consumer's `foreach` calls `GetEnumerator` and the first `MoveNext`.
  • What happens if you keep the `yield return` body but declare the return type as `List<T>`?
    The build fails with CS1624: the body cannot be an iterator block because `List<T>` is not an iterator interface type. An iterator must return `IEnumerable<T>`, `IEnumerator<T>` or their non-generic forms; to hand back a list you would drop `yield` and build the list eagerly.
  • Where does a null `source` passed to this `WhereNot` get reported?
    Not at the call. The `foreach` over `source` sits inside the iterator body, so the `NullReferenceException` surfaces on the consumer's first `MoveNext`. Built-in operators throw `ArgumentNullException` at the call by splitting a plain public method from a private iterator.

  • yield return builds the full result list before the method returns
  • An extension method can live in a generic static class
  • The operator should return List<T> so callers get a real collection
  • Each operator in a chain runs over the whole source before the next starts
  • Calling the operator runs its body up to the first yield return
open as a page

In C#, why isn't .NET's `Chunk(3)` a sliding window over sensor readings, and how do you write a `WindowBy` operator that yields safe windows?

level: middleimportance: should knowfreq 30%

basics

~20 s

Chunk(3) splits the source into disjoint arrays of three, with a shorter last one; a sliding window overlaps and advances by one. WindowBy keeps the last size elements in a Queue<T> and yields a fresh array per window, so consumers can keep each one.

open as a page

In C#, what may a custom `Pairwise` operator over an endless `IEnumerable<Reading>` hold between elements, and what would make it hang before `Take(5)`?

level: middleimportance: should knowfreq 45%

basics

~20 s

A streaming Pairwise keeps only the previous element and a flag, and is safe on an endless source. It hangs if it, or a stage before Take, must read to the end first, as ToList, ToArray, Count, Max, Reverse or OrderBy do.

open as a page

A C# custom `Pairwise` operator calls `source.GetEnumerator()` directly; why does the sensor stream's `finally` never close the port when the consumer stops at `Take(1)`?

level: seniorimportance: should knowfreq 30%

basics

~20 s

The operator holds the source enumerator in a plain local and never disposes it. When Take stops early, Dispose reaches the operator's iterator but not the source's, so the source's finally never runs. Declare it with using IEnumerator<T> e = source.GetEnumerator().

open as a page

A C# custom operator calls `source.Max()` and then loops over `source` to scale each reading; why does it print 104% on a live sensor feed, and how do you fix it?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Max() and the foreach each enumerate the source, so a live feed is read twice and the second read returns newer, higher readings than the peak. Read the source once: keep one enumerator, or copy it into an array and compute over the copy.

open as a page