How does a stream pipeline like source.filter(...).map(...).forEach(...) actually traverse elements — does it build an intermediate filtered list, and in what order are the operations applied?
answer
- Fusion: one pass, no intermediate lists
- Per element: full chain before next element
- Within element = chain order; across elements = interleaved
- Failed filter -> mapper never called
- sorted/distinct/limit/skip = stateful barriers
basics
~20 sNo intermediate lists are built. Each element flows through the whole chain one at a time: an element is filtered, then if it passes, mapped, then handled by forEach, before the next element starts. The operations are fused into a single pass.
solid answer
~40 sStreams use operation fusion. Rather than running filter over the entire source to produce a filtered list, then map over that list, the pipeline processes each element through the full chain before moving to the next. For filter(...).map(...).forEach(...), element 1 is tested by the predicate; if it passes it is mapped and passed to forEach; only then does element 2 start. This is a depth-first, per-element pull traversal, not stage-by-stage. The benefit is that no intermediate collections are materialized between stages, which saves memory and allocations, and it makes short-circuiting possible: a downstream limit or findFirst can stop the upstream from pulling more elements. The order within one element is the chain order (filter before map), but across elements the engine interleaves stages rather than completing one stage for all elements first.
go deeper
Knows that no intermediate list is built and elements go through the chain one at a time.
Can describe the per-element fused traversal, distinguish within-element vs across-element order, and note that a failed filter skips the mapper.
Explains the Sink/push implementation, distinguishes stateless vs stateful ops (sorted/distinct/limit/skip buffering barriers), and ties fusion to short-circuit and memory behavior.
Reasons about traversal semantics for parallel splitting, ordering guarantees, the cost of stateful barriers in large/parallel pipelines, and when to restructure a pipeline to preserve laziness.
## The naive (wrong) mental model Many people imagine a pipeline `filter().map().forEach()` works like chained collection transforms: 1. Run `filter` over *all* elements → produce a filtered list. 2. Run `map` over *that whole list* → produce a mapped list. 3. Run `forEach` over the mapped list. That would allocate two throwaway lists and walk the data three times. **Streams do not work this way.** ## What actually happens: fusion + per-element traversal When the terminal operation starts, the stream engine processes the source **one element at a time**, pushing each element all the way down the chain before fetching the next: ``` for each element e from source: if predicate(e): // filter stage m = mapper(e) // map stage consumer(m) // forEach stage // then move to the next element ``` This is called **operation fusion**: the separate intermediate stages are *fused* so that a single traversal of the source drives them all. The key facts: - **No intermediate collection** is created between `filter` and `map`. There is no "filtered list". - The data is walked **once**, not once per stage. - An element that fails the filter never reaches `map` — the mapper is simply not called for it. ## Order of operations Two different "orders" matter and people confuse them: - **Within a single element**, stages run in *chain order*: filter, then map, then the consumer. So for any given element you can reason left-to-right. - **Across elements**, the engine *interleaves* stages: it finishes element 1 through the whole chain, then element 2, and so on. It does **not** finish the filter stage for all elements before starting map. (Some stateful ops like `sorted` are an exception — see below.) ## How it's implemented (push model with Sinks) Internally each stage is a `Sink` that wraps the next stage's `Sink`. The source spliterator pushes each element into the head Sink, which (for `filter`) conditionally forwards to the next Sink (`map`), which transforms and forwards to the terminal Sink (`forEach`'s consumer). This chaining of sinks is exactly the fusion mechanism, and it is what allows a downstream `cancellationRequested()` signal to short-circuit upstream traversal. ## Stateful operations break the per-element pipelining Most intermediate ops (`filter`, `map`, `peek`, `flatMap`) are **stateless** — they handle each element independently and pipeline cleanly. A few are **stateful**: `sorted`, `distinct`, `limit`, and `skip` may need to *buffer* or count elements. `sorted`, for instance, must consume the entire upstream into a buffer before it can emit a single element in order, creating a barrier in the otherwise single-pass flow. So "no intermediate buffering ever" is only true for fully stateless pipelines. ## Why this matters in practice - **Memory:** big sources don't blow up memory, because nothing between stages is materialized. - **Side effects:** because mappers/predicates run per consumed element interleaved, ordering of side effects (e.g. logging in `peek`) follows element-by-element traversal, not stage-by-stage. - **Short-circuiting:** a `limit(2)` after `map` means `map` runs only for as many elements as needed to produce 2 results, not for the whole source. ## One-line summary A stream is a single, fused, depth-first pull/push over the source: each element runs the full chain before the next, with no intermediate collections — except where a stateful op (sorted/distinct/limit/skip) imposes a buffering barrier.
- If you put a peek before and after a filter, what order do the prints appear in for a 3-element source?Interleaved per element: pre-peek(e1), [if it passes] post-peek(e1), pre-peek(e2), ... Each element is fully driven through the chain before the next, so you do NOT see all pre-peeks first.
- Which common intermediate operation forces the whole upstream to be consumed before it emits anything?sorted — it must buffer and order all upstream elements before it can produce the first element, creating a barrier in the single-pass flow. distinct and limit/skip are also stateful but only partially block.
saying these in an interview costs you the question
- Claiming filter builds a filtered list that map then iterates
- Saying the source is traversed once per stage
- Thinking ALL filtering finishes before any mapping starts
- Forgetting sorted/distinct must buffer and break the single-pass flow