In Rust, what happens when a custom feed iterator's next is called again after it returned None, and what do Iterator::fuse and FusedIterator change?
answer
- None is not always final
- may or may not resume
- a wrapper that stops forever
- a marker that makes fuse free
- a false marker breaks fuse
basics
~10 sWhatever the iterator's next does: Rust's protocol lets an iterator resume after None. fuse() wraps it so None is final; FusedIterator is a marker promising that already, which turns fuse into a no-op.
solid answer
~40 s`Iterator::next` documents that after `None` an implementation may choose to resume, so a feed iterator that returns `None` when caught up and records from a later batch on the next call is legal. A `for` loop stops at the first `None`, but code that keeps the iterator — `by_ref()`, a second loop, a stored field — can see more items. `Iterator::fuse` wraps it in `Fuse<I>`, which returns `None` forever after the first `None`. `std::iter::FusedIterator`, stable since 1.26, is a marker trait with no methods promising that behaviour already; `Fuse` is specialised to a no-op for such types, so an honest impl makes `fuse()` free and a false one makes `fuse()` silently pass resumed items through. std's own advice is to call `.fuse()` rather than bound generic code on `FusedIterator`.
code
rust · 37 linesuse std::collections::VecDeque;
/// Follows a paginated feed: `None` means "caught up for now",
/// and the next call may yield records from a later batch.
struct Feed {
batches: VecDeque<Vec<u32>>,
current: std::vec::IntoIter<u32>,
}
impl Iterator for Feed {
type Item = u32;
fn next(&mut self) -> Option<u32> {
if let Some(r) = self.current.next() {
return Some(r);
}
if let Some(batch) = self.batches.pop_front() {
self.current = batch.into_iter(); // load it, but report "caught up"
}
None
}
}
fn feed() -> Feed {
Feed { batches: VecDeque::from([vec![3, 4]]), current: vec![1, 2].into_iter() }
}
fn main() {
let mut f = feed();
let first: Vec<u32> = f.by_ref().collect();
let later: Vec<u32> = f.by_ref().collect();
println!("{first:?} then {later:?}"); // [1, 2] then [3, 4]
let mut fused = feed().fuse();
let first: Vec<u32> = fused.by_ref().collect();
let later: Vec<u32> = fused.by_ref().collect();
println!("{first:?} then {later:?}"); // [1, 2] then []
}go deeper
Know that a for loop stops at the first None, and that fuse() makes None final for code that keeps calling next.
Explain that the protocol allows resuming after None, what Fuse does, and that FusedIterator is a marker with no methods.
Explain the fuse no-op for FusedIterator types and the bug a false impl causes, and say which std adapters propagate the marker.
Decide whether a feed API should signal caught-up with None at all, or expose a method or item type that cannot be confused with the end.
## The protocol leaves None open The documentation of `Iterator::next` says it returns `None` when iteration is finished, and then adds that individual implementations **may choose to resume** iteration, so calling `next` again may or may not eventually return `Some` again. Nothing in the trait or the compiler forces `None` to be final. That freedom is useful for a **feed follower**: an iterator over a paginated feed that returns `None` when it has caught up and, on a later call, yields records from a batch that arrived since. It is also a trap for any code that assumes the first `None` ends the iterator forever. | Who consumes it | What it sees | |---|---| | `for r in feed` | stops at the first `None`; the loop consumed the iterator, so resumption is invisible | | `feed.by_ref().collect()` twice | the first call gets the current batch, the second the next batch | | a struct field polled later | sees resumed items | | `feed.fuse()` | `None` after the first `None`, forever | ## fuse(): making None final `Iterator::fuse(self)` returns the adapter `Fuse<I>`. For an ordinary iterator, `Fuse` keeps the inner iterator in an `Option` and drops it at the first `None`, so every later call returns `None` without touching the inner iterator. Use it when: - code you call may poll your iterator after it ended and must not see resumed items; - you hand an iterator from an unknown source to an algorithm that needs a final end; - you are writing an adapter whose own logic would break if the inner iterator resumed. ## FusedIterator: the promise that makes fuse free `std::iter::FusedIterator` is a **marker trait** — no methods, supertrait `Iterator` — stable since Rust 1.26. Implementing it promises that once `next` returns `None`, every further call returns `None` too. Its documentation says all iterators that behave this way should implement it, because it allows optimising `Iterator::fuse()`. std uses specialisation internally so that `Fuse<I>` for `I: FusedIterator` just forwards to the inner iterator, with no extra state check. That has a sharp edge, stated in the documentation of `fuse`: the wrapper is a no-op on `FusedIterator` types, so `fuse()` may behave incorrectly if the trait is implemented improperly. Add `impl FusedIterator for Feed {}` to the resuming feed, and `feed.fuse()` passes the second batch straight through. Because the trait is safe, this is a logic bug, not undefined behaviour. Two more rules from the same documentation: 1. **Do not bound generic code on `FusedIterator`** when you need a fused iterator; call `.fuse()` instead. If the iterator is already fused, the wrapper costs nothing. 2. **Implement it honestly on your own types.** An ID block or a tree walker that returns `None` forever once empty should carry the empty impl. ## Which std iterators are fused The marker propagates through adapters only when the inner iterator has it: - `vec::IntoIter`, slice iterators and integer ranges are fused. - `Map<I, F>`, `Rev<I>` and `Peekable<I>` are fused when `I` is. `Peekable` remembers a peeked `None` but documents that this does not by itself make it fused. - `iter::successors` is always fused: once its stored item is `None` it never calls the closure again. - `iter::from_fn` is never fused, because std cannot see what the closure does. - `Chain<A, B>` stops calling `A` after its first `None` but does not fuse `B`; the chain as a whole is fused only when both halves are. The distinction matters most when you **write an adapter**. An adapter that alternates between two inner iterators, or that keeps polling one after the other is exhausted, must either call `fuse()` on its inputs or track exhaustion itself; otherwise a resuming input makes the adapter yield items after it reported the end. ## Designing a resuming iterator - **Document it.** A resuming `next` surprises readers; say in the type's documentation that `None` means "nothing now". - **Consider a different signal.** Often a method such as `poll_batch() -> Option<Vec<Record>>`, or a `Result`/enum item that distinguishes "caught up" from "closed", communicates better than overloading `None`. For data that genuinely arrives over time in async code, a stream is the tool, covered with async streams. - **Never mark it fused.** A resuming iterator must not implement `FusedIterator`, or `fuse()` stops protecting its callers.
- Is a wrong impl FusedIterator undefined behaviour?No. `FusedIterator` is a safe trait, so a false impl is a logic bug: `fuse()` becomes a no-op and resumed items leak through. Only unsafe traits carry promises that unsafe code may rely on; std's internal `TrustedFused` is the unsafe counterpart used for its own optimisations.
- Should a generic function take I: FusedIterator when it needs a fused iterator?std's documentation advises against it: call `.fuse()` on the iterator instead. An already-fused iterator pays nothing for the wrapper, and the function also accepts the many iterators that never implemented the marker.
- Calling next after None is undefined behaviour in Rust.
- Every Iterator returns None forever after the first None.
- FusedIterator adds a method that resets the iterator.
- fuse() still guards an iterator that falsely implements FusedIterator.
- Peekable is always fused because it caches a peeked None.