skip to content

In Rust, what happens when a custom feed iterator's next is called again after it returned None, and what do Iterator::fuse and FusedIterator change?

level: seniorimportance: nice to knowfreq 25%

answer

  1. None is not always final
  2. may or may not resume
  3. a wrapper that stops forever
  4. a marker that makes fuse free
  5. a false marker breaks fuse

basics

~10 s

Whatever the iterator's next does: Rust's protocol lets an iterator resume after None. fuse() wraps it so None is final; FusedIterator is a marker promising that already, which turns fuse into a no-op.

solid answer

~40 s

`Iterator::next` documents that after `None` an implementation may choose to resume, so a feed iterator that returns `None` when caught up and records from a later batch on the next call is legal. A `for` loop stops at the first `None`, but code that keeps the iterator — `by_ref()`, a second loop, a stored field — can see more items. `Iterator::fuse` wraps it in `Fuse<I>`, which returns `None` forever after the first `None`. `std::iter::FusedIterator`, stable since 1.26, is a marker trait with no methods promising that behaviour already; `Fuse` is specialised to a no-op for such types, so an honest impl makes `fuse()` free and a false one makes `fuse()` silently pass resumed items through. std's own advice is to call `.fuse()` rather than bound generic code on `FusedIterator`.

code

rust · 37 lines
rust
use std::collections::VecDeque;

/// Follows a paginated feed: `None` means "caught up for now",
/// and the next call may yield records from a later batch.
struct Feed {
    batches: VecDeque<Vec<u32>>,
    current: std::vec::IntoIter<u32>,
}

impl Iterator for Feed {
    type Item = u32;
    fn next(&mut self) -> Option<u32> {
        if let Some(r) = self.current.next() {
            return Some(r);
        }
        if let Some(batch) = self.batches.pop_front() {
            self.current = batch.into_iter(); // load it, but report "caught up"
        }
        None
    }
}

fn feed() -> Feed {
    Feed { batches: VecDeque::from([vec![3, 4]]), current: vec![1, 2].into_iter() }
}

fn main() {
    let mut f = feed();
    let first: Vec<u32> = f.by_ref().collect();
    let later: Vec<u32> = f.by_ref().collect();
    println!("{first:?} then {later:?}"); // [1, 2] then [3, 4]

    let mut fused = feed().fuse();
    let first: Vec<u32> = fused.by_ref().collect();
    let later: Vec<u32> = fused.by_ref().collect();
    println!("{first:?} then {later:?}"); // [1, 2] then []
}

go deeper

for a junior

Know that a for loop stops at the first None, and that fuse() makes None final for code that keeps calling next.

for a middle

Explain that the protocol allows resuming after None, what Fuse does, and that FusedIterator is a marker with no methods.

for a senior

Explain the fuse no-op for FusedIterator types and the bug a false impl causes, and say which std adapters propagate the marker.

for a principal

Decide whether a feed API should signal caught-up with None at all, or expose a method or item type that cannot be confused with the end.

## The protocol leaves None open The documentation of `Iterator::next` says it returns `None` when iteration is finished, and then adds that individual implementations **may choose to resume** iteration, so calling `next` again may or may not eventually return `Some` again. Nothing in the trait or the compiler forces `None` to be final. That freedom is useful for a **feed follower**: an iterator over a paginated feed that returns `None` when it has caught up and, on a later call, yields records from a batch that arrived since. It is also a trap for any code that assumes the first `None` ends the iterator forever. | Who consumes it | What it sees | |---|---| | `for r in feed` | stops at the first `None`; the loop consumed the iterator, so resumption is invisible | | `feed.by_ref().collect()` twice | the first call gets the current batch, the second the next batch | | a struct field polled later | sees resumed items | | `feed.fuse()` | `None` after the first `None`, forever | ## fuse(): making None final `Iterator::fuse(self)` returns the adapter `Fuse<I>`. For an ordinary iterator, `Fuse` keeps the inner iterator in an `Option` and drops it at the first `None`, so every later call returns `None` without touching the inner iterator. Use it when: - code you call may poll your iterator after it ended and must not see resumed items; - you hand an iterator from an unknown source to an algorithm that needs a final end; - you are writing an adapter whose own logic would break if the inner iterator resumed. ## FusedIterator: the promise that makes fuse free `std::iter::FusedIterator` is a **marker trait** — no methods, supertrait `Iterator` — stable since Rust 1.26. Implementing it promises that once `next` returns `None`, every further call returns `None` too. Its documentation says all iterators that behave this way should implement it, because it allows optimising `Iterator::fuse()`. std uses specialisation internally so that `Fuse<I>` for `I: FusedIterator` just forwards to the inner iterator, with no extra state check. That has a sharp edge, stated in the documentation of `fuse`: the wrapper is a no-op on `FusedIterator` types, so `fuse()` may behave incorrectly if the trait is implemented improperly. Add `impl FusedIterator for Feed {}` to the resuming feed, and `feed.fuse()` passes the second batch straight through. Because the trait is safe, this is a logic bug, not undefined behaviour. Two more rules from the same documentation: 1. **Do not bound generic code on `FusedIterator`** when you need a fused iterator; call `.fuse()` instead. If the iterator is already fused, the wrapper costs nothing. 2. **Implement it honestly on your own types.** An ID block or a tree walker that returns `None` forever once empty should carry the empty impl. ## Which std iterators are fused The marker propagates through adapters only when the inner iterator has it: - `vec::IntoIter`, slice iterators and integer ranges are fused. - `Map<I, F>`, `Rev<I>` and `Peekable<I>` are fused when `I` is. `Peekable` remembers a peeked `None` but documents that this does not by itself make it fused. - `iter::successors` is always fused: once its stored item is `None` it never calls the closure again. - `iter::from_fn` is never fused, because std cannot see what the closure does. - `Chain<A, B>` stops calling `A` after its first `None` but does not fuse `B`; the chain as a whole is fused only when both halves are. The distinction matters most when you **write an adapter**. An adapter that alternates between two inner iterators, or that keeps polling one after the other is exhausted, must either call `fuse()` on its inputs or track exhaustion itself; otherwise a resuming input makes the adapter yield items after it reported the end. ## Designing a resuming iterator - **Document it.** A resuming `next` surprises readers; say in the type's documentation that `None` means "nothing now". - **Consider a different signal.** Often a method such as `poll_batch() -> Option<Vec<Record>>`, or a `Result`/enum item that distinguishes "caught up" from "closed", communicates better than overloading `None`. For data that genuinely arrives over time in async code, a stream is the tool, covered with async streams. - **Never mark it fused.** A resuming iterator must not implement `FusedIterator`, or `fuse()` stops protecting its callers.

  • Is a wrong impl FusedIterator undefined behaviour?
    No. `FusedIterator` is a safe trait, so a false impl is a logic bug: `fuse()` becomes a no-op and resumed items leak through. Only unsafe traits carry promises that unsafe code may rely on; std's internal `TrustedFused` is the unsafe counterpart used for its own optimisations.
  • Should a generic function take I: FusedIterator when it needs a fused iterator?
    std's documentation advises against it: call `.fuse()` on the iterator instead. An already-fused iterator pays nothing for the wrapper, and the function also accepts the many iterators that never implemented the marker.

  • Calling next after None is undefined behaviour in Rust.
  • Every Iterator returns None forever after the first None.
  • FusedIterator adds a method that resets the iterator.
  • fuse() still guards an iterator that falsely implements FusedIterator.
  • Peekable is always fused because it caches a peeked None.