skip to content

In Ruby, how do Enumerator#next, peek and rewind behave, and what happens when an enumerator runs out of values?

level: middleimportance: should knowfreq 35%

answer

  1. external iteration pulls one value
  2. peek does not advance
  3. StopIteration, an IndexError subclass
  4. StopIteration#result: the method's return value
  5. rewind restarts, side effects repeat

basics

~10 s

next returns the next value and advances; peek returns it without advancing; both raise StopIteration at the end, and keep raising until rewind resets the position. StopIteration#result holds the underlying method's return value.

solid answer

~40 s

`Enumerator#next` is **external iteration**: the caller pulls one value at a time instead of passing a block. `peek` returns the same value `next` would return without moving the position, which makes look-ahead merges easy. When the underlying method finishes, `next` and `peek` raise `StopIteration` (a subclass of `IndexError`), and they keep raising it on every later call; `StopIteration#result` carries the method's return value, such as the array that `each` returns. `rewind` resets the position so the next `next` starts the iteration over, re-running the underlying method and any side effects it has. Under the hood CRuby runs the iteration in a separate Fiber, which costs more than a block and hides fiber-local variables from the enumerated code.

code

ruby · 21 lines
ruby
e = [101, 99, 103].each

p e.next  # => 101
p e.peek  # => 99
p e.next  # => 99
p e.next  # => 103

begin
  e.next
rescue StopIteration => ex
  p ex.result          # => [101, 99, 103], what each returned
end

begin
  e.peek
rescue StopIteration
  puts "still exhausted"
end

e.rewind
p e.next  # => 101

go deeper

for a junior

Recall that next returns one value at a time, peek looks without advancing, and StopIteration marks the end.

for a middle

Explain that StopIteration repeats until rewind, what StopIteration#result carries, and why rewind re-runs the underlying method.

for a senior

Use peek-driven merges for multiple streams, and account for the Fiber cost and hidden fiber-locals when external iteration sits on a hot path.

for a principal

Choose internal iteration by default and reserve external enumerators for interleaving or pausing, where their overhead buys real design simplicity.

## Internal vs external iteration in Ruby Most Ruby iteration is **internal**: you pass a block to `each`, and the collection drives the loop. An `Enumerator` also supports **external** iteration, where your code decides when to take the next value: ```ruby e = [101, 99, 103].each e.next # => 101 e.peek # => 99 (position unchanged) e.next # => 99 e.next # => 103 e.next # raises StopIteration ``` The rdoc for `Enumerator` names the external-iteration methods: `next`, `next_values`, `peek` and `peek_values` (plus `Array#zip` with a non-Array argument, which uses `next` internally). They keep their own position and do not disturb internal iteration such as `each` on the same Enumerator. ## The methods | Method | Returns | Moves position? | At the end | |---|---|---|---| | `next` | the next yielded value | yes | raises `StopIteration` | | `peek` | the value `next` would return | no | raises `StopIteration` | | `next_values` | the yielded values as an Array | yes | raises `StopIteration` | | `peek_values` | the same, without moving | no | raises `StopIteration` | | `rewind` | the Enumerator | back to the start | always works | `next_values` exists because `next` cannot distinguish `yield` from `yield nil`, or `yield [1, 2]` from `yield 1, 2`; `next_values` returns `[]`, `[nil]`, `[[1, 2]]` and `[1, 2]` respectively. ## Running out When the underlying method returns, the Enumerator raises **`StopIteration`**: - `StopIteration` is a subclass of `IndexError` (defined in `enumerator.c`). - It is raised **again** on every later `next` or `peek`, until `rewind`. - `StopIteration#result` is the return value of the iterated method. For `[1, 2, 3].each` that is the array itself; for a custom method it is whatever the method returned. `Kernel#loop` rescues `StopIteration` and ends quietly, which is why `loop { v = e.next; ... }` is the usual consumer; how `loop` does that belongs to the control-flow material. ## rewind and side effects `rewind` resets the position. The next `next` starts the underlying method **from the beginning**, so any work it does is done again. For a tick feed that fetches pages over HTTP, rewinding means fetching page 1 again. If the receiver responds to `rewind` itself, `Enumerator#rewind` calls it too. ## Merging two tick streams with peek `peek` is what makes a streaming merge of two time-ordered feeds short and memory-flat: ```ruby def merge_by_time(a, b) Enumerator.new do |y| loop do pick = if (ta = (a.peek rescue nil)).nil? then b elsif (tb = (b.peek rescue nil)).nil? then a else ta.at <= tb.at ? a : b end y << pick.next end end end ``` When both feeds are exhausted, `b.next` raises `StopIteration`, `loop` ends, and the merged Enumerator finishes. ## Costs and caveats The rdoc is explicit that external iteration differs significantly from internal iteration because it runs on a **Fiber**: 1. It is slower than a block because every `next` switches Fibers. 2. A backtrace from inside the enumerated code shows only the Enumerator's stack. 3. Fiber-local variables (`Thread.current[:x]`) are **not** visible inside the enumerated code; Fiber storage (`Fiber[:x]`) is inherited. 4. `next`, `peek`, `rewind` and `feed` raise `FrozenError` on a frozen Enumerator, since they change its state. So prefer internal iteration for plain loops and reach for `next`/`peek` when you genuinely need to interleave several sources or pause between values. ## Summary - `next` pulls and advances; `peek` looks ahead. - The end is signalled by `StopIteration`, repeatedly, with the method's return value in `result`. - `rewind` restarts the underlying method, side effects included.

  • Why can code inside an enumerator see Thread.current[:request_id] under each but not under next?
    `next` runs the enumerated method in a separate Fiber, and fiber-local variables accessed through `Thread.current[]` are not inherited by it. Under `each` the code runs in the caller's Fiber. Fiber storage (`Fiber[:key]`) is inherited, so it works in both cases.
  • When would you use next_values instead of next?
    When the underlying method yields several values or might yield nothing or nil, and you must tell those cases apart. `next_values` always returns an Array of exactly what was yielded, so `yield` gives `[]` and `yield nil` gives `[nil]`.

A ticket dispenser at a deli counter: next tears off a ticket, peek reads the number showing without tearing it, and when the roll runs out every pull says empty until someone loads the roll again, which is rewind.

saying these in an interview costs you the question

  • next returns nil when the enumerator is exhausted
  • peek advances the position just like next
  • After StopIteration, the next call to next starts over automatically
  • rewind replays cached values without re-running the method
  • External iteration costs the same as passing a block