skip to content

What are Java's Stream Gatherers, what does Stream.gather add over map, flatMap, mapMulti and a Collector, and when should you write one?

level: middleimportance: should knowfreq 45%

answer

  1. extension point for intermediate operations
  2. what collect did for terminals
  3. state, end-of-input flush, early stop
  4. previewed in 22 and 23, final in 24
  5. check the Gatherers class first

basics

~20 s

Stream Gatherers are custom intermediate operations, plugged in with Stream.gather and final since Java 24. A Gatherer can keep state across elements, emit at end of input and stop early, so use one when no built-in operation fits.

solid answer

~50 s

A Stream Gatherer is a custom intermediate operation: `Stream.gather(Gatherer)` is to intermediate operations what `Stream.collect(Collector)` is to terminal ones. It was previewed in Java 22 and 23 and became final in Java 24. The functions given to `map`, `flatMap` and `mapMulti` are specified as stateless, handle a single element per call and have no end-of-input hook or way to ask for no more input; a gatherer adds private state, a finisher that can emit after the last element, and an integrator that can return `false` to stop, even on an infinite stream. A `Collector` has state too, but it ends the pipeline and cannot signal that it is finished. Use a built-in operation when one fits, then `Gatherers` (`windowFixed`, `windowSliding`, `fold`, `scan`, `mapConcurrent`), and write your own when later operations need the output of a transform that needs state, a flush or an early stop.

code

java · 31 lines
java
import java.util.HashSet;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Gatherer;
import java.util.stream.Gatherers;
import java.util.stream.Stream;

public class GathererUse {

    // Keeps the first element for each key: state that map, flatMap and mapMulti may not hold.
    static <T, K> Gatherer<T, ?, T> distinctBy(Function<? super T, ? extends K> key) {
        return Gatherer.ofSequential(
            HashSet<K>::new,
            Gatherer.Integrator.ofGreedy((seen, element, downstream) ->
                !seen.add(key.apply(element)) || downstream.push(element)));
    }

    public static void main(String[] args) {
        List<String> firstByLength = Stream.of("foo", "bar", "baz", "quux")
            .gather(distinctBy(String::length))
            .toList();
        System.out.println(firstByLength); // [foo, quux]

        // A built-in gatherer on an infinite sequential stream: limit ends it.
        List<List<Integer>> windows = Stream.iterate(0, i -> i + 1)
            .gather(Gatherers.windowFixed(3))
            .limit(2)
            .toList();
        System.out.println(windows); // [[0, 1, 2], [3, 4, 5]]
    }
}

go deeper

for a junior

Recall that a gatherer is a custom intermediate operation applied with Stream.gather, and that the Gatherers class already offers windows, fold, scan and concurrent mapping. That definition is enough to show you follow the language.

for a middle

Name the three things map, flatMap and mapMulti lack by contract or design: state across elements, an end-of-input hook and a signal to stop. Contrast them with a Collector, which accumulates but ends the pipeline.

for a senior

Show judgment about cost: gather exists only on Stream, at JDK 27 it drops the size information, and a gatherer without a combiner is evaluated sequentially in a parallel pipeline. Say when a named gatherer beats a plain loop and when it does not.

for a principal

Weigh whether a team should keep a small shared library of named gatherers or leave such logic in loops. Readability for people new to the API, the Java 24 minimum and reviewing state handling are the tradeoffs.

## What a Stream Gatherer is A **stream pipeline** has a source, any number of **intermediate operations** that each return a new stream, and one **terminal operation** that starts the evaluation. The terminal side has long been extensible through `Stream.collect(Collector)`, while the intermediate side was a fixed list of methods on `Stream`. A **gatherer** closes that gap. `java.util.stream.Gatherer<T, A, R>` describes a custom intermediate operation that turns input elements of type `T` into output elements of type `R`, optionally using private state of type `A`. You apply one with `Stream.gather(Gatherer)`, which the API documents as a stateful intermediate operation and an extension point. The transform can be one-to-one, one-to-many, many-to-one or many-to-many. JEP 485 goes as far as saying that every existing intermediate operation on `Stream` can be implemented as a gatherer. The feature arrived in stages: a preview in Java 22 (JEP 461), a second preview in Java 23 (JEP 473), and final in Java 24 (JEP 485), with the API unchanged across those steps. Java 25, a long-term support release, and Java 27 carry identical `Gatherer` and `Gatherers` source files. ## What it adds over map, flatMap, mapMulti and a Collector | Capability | `map`, `flatMap`, `mapMulti` | `Collector` | `Gatherer` | |---|---|---|---| | Place in the pipeline | Intermediate | Terminal | Intermediate | | Output per input element | `map` one, the others zero or more | Nothing until the end, then one result | Any number | | State across elements | Function is specified as stateless | Mutable result container | Private state from an initializer | | Action at end of input | No hook | Finisher produces the result | Finisher can push more elements | | Signal that it wants no more input | None | None | Integrator returns `false` | | Output for later stream operations | Yes | Only by starting a new stream | Yes | Three of those rows carry the argument: - **Statelessness is a contract the compiler only partly enforces.** A lambda cannot reassign a captured local variable, which must be effectively final, but it can update a captured mutable object such as an `int[]` or a `Set`, or a field, and that compiles and may appear to work on a sequential stream. The function parameters of `map`, `flatMap` and `mapMulti` are specified as stateless, though, and the package documentation warns that stateful behavioural parameters can make pipeline results nondeterministic or incorrect. - **End of input is a hook those functions lack.** Each call handles a single element and none follows the last one, so a partial batch has no way out. A gatherer's **finisher** can push it. - **Stopping is a real gap.** JEP 485 shows fixed-size grouping written as a `Collector`: it needs a `limit` first, because a collector cannot signal to `collect` that it has finished while an infinite source keeps producing. A gatherer's **integrator** can return `false`, after which the input counts as ended. The JEP's other motivating case is distinctness by string length. The built-in `distinct` is stateful, but it compares whole elements with `equals`, so the workaround is to wrap each string in a class with a custom `equals`. A small gatherer holding a set of seen keys says the same thing directly. ## The built-in gatherers `java.util.stream.Gatherers` ships five, worth checking before writing your own: - `windowFixed(n)` groups elements into lists of size `n`, and the last list may be shorter. - `windowSliding(n)` emits overlapping lists, each shifted by one element. - `fold(initial, folder)` builds one aggregate and emits it at end of input, for reductions that are order-dependent or have no combiner. - `scan(initial, scanner)` emits each running value, which is a prefix scan. - `mapConcurrent(maxConcurrency, mapper)` runs the mapper concurrently on virtual threads up to a limit and keeps stream order. ## When to reach for one Work down this list and stop at the first match: 1. **A built-in `Stream` operation fits.** Use it. JEP 485 accepts that using a gatherer is not as succinct as using a built-in operation. 2. **You want one result at the end of the pipeline, and it can be combined.** Use a `Collector` or `reduce`, the terminal side's own tools. 3. **A gatherer in `Gatherers` fits.** Use it, composing several with `andThen` when needed. 4. **Later operations need the output of a transform that needs state, a flush at end of input or its own early stop.** Write a gatherer: `Gatherer.ofSequential` for an order-dependent transform, `Gatherer.of` when a combiner makes sense. A gatherer also replaces a common retreat from streams. JEP 485 shows a loop over `source.toList()` tracking the previous element, which `gather(Gatherers.windowSliding(2))` followed by `filter` expresses in a few lines. ## Costs and limits - **Only `Stream` has `gather`.** `IntStream`, `LongStream` and `DoubleStream` do not declare it, so primitive values have to become objects first, with `boxed()` or `mapToObj`. - **Size information is lost at JDK 27.** That is implementation rather than contract: the package-private `GathererOp` marks its output as not sized. On JDK 27, `count()` over `peek` and `gather` runs the pipeline, while `count()` over `peek` and `map` on a sized source skips it. - **Parallelism needs a combiner.** A gatherer without one can sit in a parallel pipeline but is evaluated sequentially, and in the JDK 27 source all five built-ins are in that group. Upstream and downstream operations can still run in parallel. - **Minimum version.** Java 21 and earlier do not have the API, and Java 22 and 23 need `--enable-preview`.

  • Why not keep state in a captured mutable object inside map instead of writing a gatherer?
    A captured local must be effectively final, but a captured `int[]` or `Set` compiles and can appear to work on a sequential stream. `map`'s function is still specified as stateless, and the package documentation warns that stateful behavioural parameters can make results nondeterministic or incorrect; a parallel run would also share that object across threads. A gatherer's state comes from the initializer, and the library gives each partition its own or, without a combiner, evaluates the gatherer sequentially.
  • How do you end an infinite stream with a gatherer, and do the built-in gatherers end one themselves?
    A custom gatherer's integrator returns `false`, after which the input counts as ended, so a `Stream.iterate` source stops. In the JDK 27 source the five built-ins use greedy integrators and never start a stop. `Gatherers.windowFixed(3)` on an infinite sequential stream needs a later short-circuiting operation such as `limit`; on a parallel one that `limit` does not help, and it runs out of memory. `Gatherers.fold` emits nothing there at all, because it pushes only at end of input.
  • When is a Collector still a better choice than Gatherers.fold?
    When you want the result at the end of the pipeline and it can be combined: a `Collector` can accumulate in parallel through its combiner, and `Collectors` already offers many. `Gatherers.fold` is documented for reductions where no combiner can be implemented or which are intrinsically order-dependent. At JDK 27 it runs sequentially and emits its one result into the stream, so later operations can still act on it.

  • A Gatherer is a terminal operation that replaces Collector, so the stream ends at gather.
  • Stream Gatherers are still a preview feature and need --enable-preview on Java 27.
  • A stateful lambda in map is an equally valid way to carry state across elements.
  • gather is available on IntStream, LongStream and DoubleStream as well as on Stream.
  • A gatherer without a combiner throws an exception as soon as its stream is parallel.

Stream Gatherers (Stream.gather, Gatherer, Gatherers)
Java 24pre-release 22

Assumes Java 27; Stream Gatherers were previews in 22 and 23 and are final since 24, and the Gatherer and Gatherers sources read the same at 25.

context