What are Java's Stream Gatherers, what does Stream.gather add over map, flatMap, mapMulti and a Collector, and when should you write one?
answer
- extension point for intermediate operations
- what collect did for terminals
- state, end-of-input flush, early stop
- previewed in 22 and 23, final in 24
- check the Gatherers class first
basics
~20 sStream Gatherers are custom intermediate operations, plugged in with Stream.gather and final since Java 24. A Gatherer can keep state across elements, emit at end of input and stop early, so use one when no built-in operation fits.
solid answer
~50 sA Stream Gatherer is a custom intermediate operation: `Stream.gather(Gatherer)` is to intermediate operations what `Stream.collect(Collector)` is to terminal ones. It was previewed in Java 22 and 23 and became final in Java 24. The functions given to `map`, `flatMap` and `mapMulti` are specified as stateless, handle a single element per call and have no end-of-input hook or way to ask for no more input; a gatherer adds private state, a finisher that can emit after the last element, and an integrator that can return `false` to stop, even on an infinite stream. A `Collector` has state too, but it ends the pipeline and cannot signal that it is finished. Use a built-in operation when one fits, then `Gatherers` (`windowFixed`, `windowSliding`, `fold`, `scan`, `mapConcurrent`), and write your own when later operations need the output of a transform that needs state, a flush or an early stop.
code
java · 31 linesimport java.util.HashSet;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Gatherer;
import java.util.stream.Gatherers;
import java.util.stream.Stream;
public class GathererUse {
// Keeps the first element for each key: state that map, flatMap and mapMulti may not hold.
static <T, K> Gatherer<T, ?, T> distinctBy(Function<? super T, ? extends K> key) {
return Gatherer.ofSequential(
HashSet<K>::new,
Gatherer.Integrator.ofGreedy((seen, element, downstream) ->
!seen.add(key.apply(element)) || downstream.push(element)));
}
public static void main(String[] args) {
List<String> firstByLength = Stream.of("foo", "bar", "baz", "quux")
.gather(distinctBy(String::length))
.toList();
System.out.println(firstByLength); // [foo, quux]
// A built-in gatherer on an infinite sequential stream: limit ends it.
List<List<Integer>> windows = Stream.iterate(0, i -> i + 1)
.gather(Gatherers.windowFixed(3))
.limit(2)
.toList();
System.out.println(windows); // [[0, 1, 2], [3, 4, 5]]
}
}go deeper
Recall that a gatherer is a custom intermediate operation applied with Stream.gather, and that the Gatherers class already offers windows, fold, scan and concurrent mapping. That definition is enough to show you follow the language.
Name the three things map, flatMap and mapMulti lack by contract or design: state across elements, an end-of-input hook and a signal to stop. Contrast them with a Collector, which accumulates but ends the pipeline.
Show judgment about cost: gather exists only on Stream, at JDK 27 it drops the size information, and a gatherer without a combiner is evaluated sequentially in a parallel pipeline. Say when a named gatherer beats a plain loop and when it does not.
Weigh whether a team should keep a small shared library of named gatherers or leave such logic in loops. Readability for people new to the API, the Java 24 minimum and reviewing state handling are the tradeoffs.
## What a Stream Gatherer is A **stream pipeline** has a source, any number of **intermediate operations** that each return a new stream, and one **terminal operation** that starts the evaluation. The terminal side has long been extensible through `Stream.collect(Collector)`, while the intermediate side was a fixed list of methods on `Stream`. A **gatherer** closes that gap. `java.util.stream.Gatherer<T, A, R>` describes a custom intermediate operation that turns input elements of type `T` into output elements of type `R`, optionally using private state of type `A`. You apply one with `Stream.gather(Gatherer)`, which the API documents as a stateful intermediate operation and an extension point. The transform can be one-to-one, one-to-many, many-to-one or many-to-many. JEP 485 goes as far as saying that every existing intermediate operation on `Stream` can be implemented as a gatherer. The feature arrived in stages: a preview in Java 22 (JEP 461), a second preview in Java 23 (JEP 473), and final in Java 24 (JEP 485), with the API unchanged across those steps. Java 25, a long-term support release, and Java 27 carry identical `Gatherer` and `Gatherers` source files. ## What it adds over map, flatMap, mapMulti and a Collector | Capability | `map`, `flatMap`, `mapMulti` | `Collector` | `Gatherer` | |---|---|---|---| | Place in the pipeline | Intermediate | Terminal | Intermediate | | Output per input element | `map` one, the others zero or more | Nothing until the end, then one result | Any number | | State across elements | Function is specified as stateless | Mutable result container | Private state from an initializer | | Action at end of input | No hook | Finisher produces the result | Finisher can push more elements | | Signal that it wants no more input | None | None | Integrator returns `false` | | Output for later stream operations | Yes | Only by starting a new stream | Yes | Three of those rows carry the argument: - **Statelessness is a contract the compiler only partly enforces.** A lambda cannot reassign a captured local variable, which must be effectively final, but it can update a captured mutable object such as an `int[]` or a `Set`, or a field, and that compiles and may appear to work on a sequential stream. The function parameters of `map`, `flatMap` and `mapMulti` are specified as stateless, though, and the package documentation warns that stateful behavioural parameters can make pipeline results nondeterministic or incorrect. - **End of input is a hook those functions lack.** Each call handles a single element and none follows the last one, so a partial batch has no way out. A gatherer's **finisher** can push it. - **Stopping is a real gap.** JEP 485 shows fixed-size grouping written as a `Collector`: it needs a `limit` first, because a collector cannot signal to `collect` that it has finished while an infinite source keeps producing. A gatherer's **integrator** can return `false`, after which the input counts as ended. The JEP's other motivating case is distinctness by string length. The built-in `distinct` is stateful, but it compares whole elements with `equals`, so the workaround is to wrap each string in a class with a custom `equals`. A small gatherer holding a set of seen keys says the same thing directly. ## The built-in gatherers `java.util.stream.Gatherers` ships five, worth checking before writing your own: - `windowFixed(n)` groups elements into lists of size `n`, and the last list may be shorter. - `windowSliding(n)` emits overlapping lists, each shifted by one element. - `fold(initial, folder)` builds one aggregate and emits it at end of input, for reductions that are order-dependent or have no combiner. - `scan(initial, scanner)` emits each running value, which is a prefix scan. - `mapConcurrent(maxConcurrency, mapper)` runs the mapper concurrently on virtual threads up to a limit and keeps stream order. ## When to reach for one Work down this list and stop at the first match: 1. **A built-in `Stream` operation fits.** Use it. JEP 485 accepts that using a gatherer is not as succinct as using a built-in operation. 2. **You want one result at the end of the pipeline, and it can be combined.** Use a `Collector` or `reduce`, the terminal side's own tools. 3. **A gatherer in `Gatherers` fits.** Use it, composing several with `andThen` when needed. 4. **Later operations need the output of a transform that needs state, a flush at end of input or its own early stop.** Write a gatherer: `Gatherer.ofSequential` for an order-dependent transform, `Gatherer.of` when a combiner makes sense. A gatherer also replaces a common retreat from streams. JEP 485 shows a loop over `source.toList()` tracking the previous element, which `gather(Gatherers.windowSliding(2))` followed by `filter` expresses in a few lines. ## Costs and limits - **Only `Stream` has `gather`.** `IntStream`, `LongStream` and `DoubleStream` do not declare it, so primitive values have to become objects first, with `boxed()` or `mapToObj`. - **Size information is lost at JDK 27.** That is implementation rather than contract: the package-private `GathererOp` marks its output as not sized. On JDK 27, `count()` over `peek` and `gather` runs the pipeline, while `count()` over `peek` and `map` on a sized source skips it. - **Parallelism needs a combiner.** A gatherer without one can sit in a parallel pipeline but is evaluated sequentially, and in the JDK 27 source all five built-ins are in that group. Upstream and downstream operations can still run in parallel. - **Minimum version.** Java 21 and earlier do not have the API, and Java 22 and 23 need `--enable-preview`.
- Why not keep state in a captured mutable object inside map instead of writing a gatherer?A captured local must be effectively final, but a captured `int[]` or `Set` compiles and can appear to work on a sequential stream. `map`'s function is still specified as stateless, and the package documentation warns that stateful behavioural parameters can make results nondeterministic or incorrect; a parallel run would also share that object across threads. A gatherer's state comes from the initializer, and the library gives each partition its own or, without a combiner, evaluates the gatherer sequentially.
- How do you end an infinite stream with a gatherer, and do the built-in gatherers end one themselves?A custom gatherer's integrator returns `false`, after which the input counts as ended, so a `Stream.iterate` source stops. In the JDK 27 source the five built-ins use greedy integrators and never start a stop. `Gatherers.windowFixed(3)` on an infinite sequential stream needs a later short-circuiting operation such as `limit`; on a parallel one that `limit` does not help, and it runs out of memory. `Gatherers.fold` emits nothing there at all, because it pushes only at end of input.
- When is a Collector still a better choice than Gatherers.fold?When you want the result at the end of the pipeline and it can be combined: a `Collector` can accumulate in parallel through its combiner, and `Collectors` already offers many. `Gatherers.fold` is documented for reductions where no combiner can be implemented or which are intrinsically order-dependent. At JDK 27 it runs sequentially and emits its one result into the stream, so later operations can still act on it.
- A Gatherer is a terminal operation that replaces Collector, so the stream ends at gather.
- Stream Gatherers are still a preview feature and need --enable-preview on Java 27.
- A stateful lambda in map is an equally valid way to carry state across elements.
- gather is available on IntStream, LongStream and DoubleStream as well as on Stream.
- A gatherer without a combiner throws an exception as soon as its stream is parallel.
- Stream Gatherers (Stream.gather, Gatherer, Gatherers)
- Java 24pre-release 22
Assumes Java 27; Stream Gatherers were previews in 22 and 23 and are final since 24, and the Gatherer and Gatherers sources read the same at 25.
context
- JEP 485: Stream Gatherersopenjdk.org
- Stream (Java SE 27 & JDK 27)docs.oracle.com
- java.util.stream (Java SE 27 & JDK 27)docs.oracle.com