In C#, why isn't .NET's `Chunk(3)` a sliding window over sensor readings, and how do you write a `WindowBy` operator that yields safe windows?
answer
- overlapping versus disjoint groups
- last chunk may be shorter
- a bounded Queue<T>
- a fresh array per window
- consumers that keep windows
basics
~20 sChunk(3) splits the source into disjoint arrays of three, with a shorter last one; a sliding window overlaps and advances by one. WindowBy keeps the last size elements in a Queue<T> and yields a fresh array per window, so consumers can keep each one.
solid answer
~40 s`Chunk(3)` is part of .NET's LINQ: it cuts `20, 22, 27, 25, 30` into `[20,22,27]` and `[25,30]`, disjoint groups with a shorter last one, each a newly allocated array. A sliding window advances by one element, giving `[20,22,27]`, `[22,27,25]`, `[27,25,30]`, and `System.Linq` has no operator for it. A streaming `WindowBy` keeps the last `size` readings in a `Queue<T>`, dequeuing the oldest when it is full, and yields `window.ToArray()` once it holds `size` items. That copy matters: yielding one shared buffer looks fine in a `foreach` that uses each window at once, but a consumer that keeps windows, such as `ToList()`, ends up with several references to one array holding only the last window. Memory stays bounded by `size`, so the operator works on an endless stream.
code
csharp · 34 linesint[] temps = [20, 22, 27, 25, 30];
Console.WriteLine(Show(temps.Chunk(3)));
Console.WriteLine(Show(temps.WindowBy(3)));
var kept = temps.WindowByShared(3).ToList(); // a consumer that keeps every window
Console.WriteLine(Show(kept));
static string Show(IEnumerable<int[]> groups) =>
string.Join(" ", groups.Select(g => $"[{string.Join(",", g)}]"));
static class SensorOps
{
public static IEnumerable<T[]> WindowBy<T>(this IEnumerable<T> source, int size)
{
var window = new Queue<T>(size);
foreach (T item in source)
{
if (window.Count == size) window.Dequeue();
window.Enqueue(item);
if (window.Count == size) yield return window.ToArray(); // a fresh array per window
}
}
public static IEnumerable<T[]> WindowByShared<T>(this IEnumerable<T> source, int size)
{
var buffer = new T[size];
int seen = 0;
foreach (T item in source)
{
if (seen < size) buffer[seen] = item;
else { Array.Copy(buffer, 1, buffer, 0, size - 1); buffer[size - 1] = item; }
if (++seen >= size) yield return buffer; // the same array every time
}
}
}go deeper
Recall that Chunk splits a sequence into disjoint arrays with a shorter last one, while a sliding window overlaps and moves by one element.
Write WindowBy with a Queue<T> that dequeues before enqueuing, and explain why each window is copied with ToArray before it is yielded.
Diagnose the all-identical windows a shared buffer produces under ToList, and weigh the per-window allocation against a documented reuse contract.
Set the rule for shared helpers: windows the consumer owns by default, reuse only behind an explicit contract, and a documented policy on partial windows.
## Chunk versus a sliding window `Enumerable.Chunk(int size)`, part of .NET's LINQ, splits a sequence into **disjoint** groups. Every chunk except the last has exactly `size` elements; the last one holds whatever remains. Reading the runtime source shows that every chunk is a newly allocated `TSource[]`: an array source is copied slice by slice, and any other source is read through an enumerator that `Chunk` holds in a `using` declaration. A **sliding window** of size 3 is a different shape: consecutive windows overlap and the window advances by one element, so each reading appears in up to three windows. `System.Linq` has no such operator, so analytics code that wants a moving average or "three readings in a row above the limit" has to write one. | | `Chunk(3)` | `WindowBy(3)` | |---|---|---| | Output for `20, 22, 27, 25, 30` | `[20,22,27] [25,30]` | `[20,22,27] [22,27,25] [27,25,30]` | | Overlap | none | `size - 1` elements | | Partial group at the end | yes, shorter last chunk | none in this design | | Groups for `n` readings | `n / 3`, rounded up | `n - 2`, none if `n < 3` | ## A streaming WindowBy ```csharp public static IEnumerable<T[]> WindowBy<T>(this IEnumerable<T> source, int size) { var window = new Queue<T>(size); foreach (T item in source) { if (window.Count == size) window.Dequeue(); window.Enqueue(item); if (window.Count == size) yield return window.ToArray(); } } ``` The design choices, one by one: - **`Queue<T>` as a ring buffer.** `Dequeue` drops the oldest reading and `Enqueue` adds the newest. Dequeuing *before* enqueuing keeps the count at `size` or below, so the queue never grows past the capacity it was created with. - **`ToArray()` per window.** `Queue<T>.ToArray()` copies from head to tail, so the array is ordered oldest to newest, and each window is an independent object the consumer owns. - **Streaming behaviour.** The operator holds at most `size` elements and yields as soon as the first window fills, so it is safe on an endless sensor stream cut by `Take`. - **Argument checks.** Validating `size` inside the iterator would throw only on the first `MoveNext`; making it throw at the call needs the non-iterator wrapper the lesson on local functions covers. ## The shared-buffer bug A tempting optimisation is to keep one `T[]`, shift it left on each reading, and `yield return buffer;`. The output looks right in a simple loop, because each window is printed before the next `MoveNext` changes the array. It breaks for any consumer that **keeps** windows: 1. `ToList()` collects three references to the same array. 2. By the time the list is read, the array holds the last window, so the example on this page prints `[27,25,30]` three times. 3. The same happens with deferred work: a window captured in a lambda, stored in a dictionary or handed to another thread changes under the consumer. Nothing throws, because arrays have no modification check. Yielding the `Queue<T>` itself has the same problem: the consumer gets the operator's live, mutable state. ## Cost and when to reuse a buffer The copy costs one array of `size` elements per window, roughly one allocation per reading once the window is full. For most code that is the right trade: windows are safe to keep, and the allocation is small and short-lived. A hot loop that cannot afford it can reuse a buffer, but then the operator's contract must say "each window is valid only until the next `MoveNext`", and its consumers must copy anything they keep. That contract is easy to break silently, so make it an explicit, documented exception rather than the default. Replacing the shift loop with a ring index over a fixed array saves copying inside the operator, but it does not change the ownership question: whatever object you yield, either the consumer owns it or the contract says it does not. ## Checklist for windowing helpers - Decide and document whether a trailing partial window is yielded. - Return windows the consumer owns, or document the reuse contract. - Keep memory bounded by the window size, so endless sources work. - Test with `ToList()`, not only with a `foreach` that prints.
- Why not yield the `Queue<T>` itself as an `IEnumerable<T>`?The consumer would receive the operator's live state. The next reading dequeues and enqueues on that same object, so any window the consumer kept, stored or captured in a lambda silently changes to the latest window. Copying with `ToArray()` gives each consumer a snapshot it owns.
- How much does `WindowBy(3)` allocate over a million readings?One three-element array per window, so 999,998 small arrays after the first two readings fill the queue, plus the queue itself once. The arrays are short-lived. Removing them means reusing one buffer, which only works under a documented valid-until-next-MoveNext contract.
- Should `WindowBy` yield a shorter window at the end, as `Chunk` yields a shorter chunk?It is a design decision to document, not a rule. A moving average usually wants only full windows, so this version yields none shorter than `size`, and a source with fewer than `size` readings produces no windows at all. A variant that flushes partial windows should say so in its name or summary.
- Chunk(3) yields overlapping groups that advance by one element
- Reusing one buffer array for every window is safe because windows arrive in order
- A sliding window must read the whole source before yielding its first window
- Chunk returns views over the source, so editing a chunk changes the source
- Queue<T>.ToArray returns the newest element first