skip to content

A filtering step is added to an assembled stream pipeline, yet every value still arrives — why?

level: juniorimportance: should knowfreq 48%

answer

  1. building, not doing
  2. the step hands something back
  3. the original source is unchanged
  4. each step wraps, never edits
  5. discarded return, discarded step

basics

~20 s

Each pipeline step returns a new source that wraps the old one; it never modifies the source it was applied to. Discarding that returned value throws the filtering step away and leaves the original, unfiltered chain in the variable.

solid answer

~40 s

Assembling a pipeline builds an immutable description, and a step is a function from a source to a source. Applying a step that keeps only matching values does not install a test inside the existing source: it returns a **new** source that wraps the old one and applies the test to whatever the old one produces. So a statement like `pipeline.keepWhere(isRecent)` whose result is dropped is a legal but useless expression — the variable still holds the unfiltered chain, and running it emits everything. The fix is to keep the result: reassign it, or chain the next step directly onto it. That same value-returning design is what makes one assembled chain safe to hand to two callers, since neither can add a step the other sees.

code

pseudocode · 10 lines
pseudocode
base = sourceOfEvents()

// the step is applied, its result is dropped
base.keepWhere(item -> item.isRecent)
run(base)                 // every event arrives, unfiltered

// the step is applied and the result is kept
recent = base.keepWhere(item -> item.isRecent)
run(recent)               // only recent events arrive
run(base)                 // base is still the full sequence

go deeper

for a junior

Remember that applying a step gives you something back. If the line that adds a step ends without using its result, the step never became part of anything you run.

for a middle

Explain the mechanic: a step is a function from source to source, so the chain grows by wrapping, and the previous chain stays valid and unchanged.

for a senior

Spot the pattern in review and in incident work, and say what the value-returning design protects: a chain stored in a field cannot gain steps that its holder never wrote.

for a principal

Frame it as an interface choice. A builder that mutates saves a keystroke and costs you shared, reusable descriptions; decide which property your platform's pipelines are meant to have.

## Two moments, one chain A pipeline has two moments that are easy to collapse into one. **Assembly** is when the chain is built: a source is taken and steps are applied to it, producing a description of work. **Execution** is when that description is run and values actually move through it. The chain you hold in a variable is a *value* produced at the first moment and consumed at the second — and like any value, what you do with it at assembly decides what exists to run later. At assembly, every step is a **function from a source to a source**. A step that keeps only matching values does not reach into the source and register a test on it. It returns a new source that holds a reference to the old one and knows how to apply the test to whatever the old one produces. The original source is untouched and still describes exactly what it described before. ## Why a discarded result loses the step When the result of applying a step is dropped, the step goes with it: - The variable still points at the source it pointed at before, which knows nothing about the new step. - The new source that *did* carry the step is unreferenced and is never run. - Running the variable therefore produces the unfiltered sequence, exactly as before the line was added. - No failure is reported: applying a step and ignoring the result is a well-formed expression, merely a pointless one. - The symptom is a pipeline that behaves as though one step were missing, because one step really is missing. This is why the bug survives review so often. The line that adds the step is present, spelled correctly, and reads as an instruction. It is not an instruction; it is an expression whose only product was thrown away. ## Value-returning versus mutating builders | Property | Step returns a new source | Step mutates the source | |---|---|---| | Ignoring the result | the step is lost | the step still applies | | Sharing one chain between two callers | safe; neither sees the other's steps | unsafe; each changes the other's | | Growing two variants from one base | apply different steps to the base | needs an explicit copy first | | Reasoning about a stored chain | meaning fixed when built | meaning depends on who touched it last | Both designs exist in ordinary programming, and collection libraries differ on exactly this axis: some sort in place and return nothing, others return a sorted copy and leave the original alone. Stream pipeline steps are uniformly of the second kind, and the reason is not style. A chain that could be mutated after it was handed out would have no stable meaning: a module that stored one could find extra steps in it that it never wrote. ## Diagnosing it 1. Find the statement that adds the step and ask whether its result is **used** — assigned back, returned, or fed straight into the next step. A statement whose whole text is `chain.someStep(...)` with nothing on either side is the bug. 2. Check what the variable being run actually points at. If the run is handed the same variable that existed before the step was added, the step is not in it. 3. Confirm by adding a second, obviously destructive step the same way. If it also has no effect, the pattern rather than the particular step is at fault. ## What the immutability buys Because a step yields a new chain rather than editing one, the assembled description behaves like any other value: - **Build once, use in several places.** A base chain can be stored in a field and specialised differently by each caller, with no copying step. - **Decorate by application.** Adding cross-cutting behaviour to a chain means applying more steps to it and returning the result, not reaching inside it. - **Cross module boundaries safely.** A chain handed across a boundary cannot be altered underneath its owner. - **Test the base independently.** The unspecialised chain is still runnable on its own, because nothing consumed or modified it. The cost is exactly the discipline this question probes: every step must be plumbed into the chain by keeping what it returns. Chaining calls in one expression makes the mistake hard to write, which is why the fluent form dominates — the value has nowhere to leak away to.

  • After adding a step, is the source it was applied to still usable?
    Yes, and unchanged. Because the step produced a separate chain, the base still describes exactly what it did before, so two different specialisations can be grown from one base without copying it first.
  • What does this design buy a team that shares one assembled chain across modules?
    A stable meaning. No module can add a step that another module's copy will see, so a chain stored in a field cannot acquire behaviour behind its owner's back, and a chain handed to two callers cannot be corrupted by either.
  • Why does writing the whole chain as one fluent expression make this bug hard to commit?
    Because each step's result is immediately consumed by the next step and the final result is assigned or returned. There is no standalone statement for the value to leak out of, which is the shape the bug requires.

Applying a step is like writing a revised copy of a recipe rather than crossing a line out of the original. If you throw the copy away, the kitchen still cooks from the recipe you kept.

saying these in an interview costs you the question

  • Thinks a step mutates the source it is applied to
  • Says the filtering step registers itself on the existing source
  • Blames the source for emitting too much instead of the dropped result
  • Diagnoses it as a subscription-timing problem rather than a lost step
  • Believes reassigning the variable would duplicate the earlier steps