What contract must the supplier, accumulator, and combiner of a collect() satisfy for a correct parallel result, and what goes wrong if it is violated?
answer
- combiner(c1,c2) == accumulate all into one container
- supplier must return a fresh empty container
- accumulator and combiner = same logical operation
- non-interfering: don't mutate the source
- bug only shows under parallel + large input
basics
~20 sThe combiner must produce the same result as accumulating everything into one container. The functions must not interfere with the source or share state badly. If they disagree, a parallel stream gives wrong or random results, even though the sequential version looks fine.
solid answer
~50 scollect() requires that the supplier, accumulator, and combiner form an associative reduction that is consistent with itself: merging two partially-accumulated containers via the combiner must yield the same logical result as accumulating all those elements into a single container sequentially. The accumulator and combiner must agree. The functions must also be non-interfering (they must not modify the stream source) and, except for a CONCURRENT collector's shared container, stateless with respect to outside data — each chunk works on its own container. If the combiner is inconsistent with the accumulator — say the accumulator deduplicates but the combiner just concatenates, or one applies a transform the other doesn't — a sequential stream (combiner never called) looks correct while a parallel stream produces wrong, order-dependent, or nondeterministic results. This class of bug is insidious because it only surfaces under parallelism and can vary with the split boundaries.
go deeper
Knows to use Collectors.toList()/toSet() and not write custom combiners; may not grasp the parallel contract.
Understands the combiner merges partial results and that accumulator and combiner must be consistent; relies on standard collectors.
States the associativity/consistency contract precisely, identifies why violations hide until parallel execution, and tests the combiner directly.
Audits custom Collectors for the full contract (fresh supplier, consistent accumulator/combiner, non-interference, statelessness), and sets team guidance on when parallel collection and custom collectors are acceptable.
## The contract `collect(supplier, accumulator, combiner)` — and equally a `Collector` — must satisfy a small algebra so it can be run **in parallel** correctly. Three requirements: ### 1. Identity / supplier freshness The **supplier** must return a **new, empty** container on every call. In parallel, it is called once per chunk; if it returned a shared or pre-populated container, chunks would corrupt each other or double-count. (`ArrayList::new` is fine; `() -> sharedList` is a bug.) ### 2. Accumulator/combiner consistency (the key one) Let `A(container, element)` be accumulation and `C(c1, c2)` be combining. The contract: for any partition of the elements, **combining the per-partition accumulations must equal accumulating all elements into one container**. Concretely, if you accumulate elements `[1,2]` into `c1` and `[3,4]` into `c2`, then `C(c1, c2)` must equal accumulating `[1,2,3,4]` into a single container. This is an **associativity + consistency** requirement: the accumulator and combiner must implement the *same* logical operation. The classic violation: an accumulator that does extra work the combiner forgets. Example — building a sorted, deduplicated list where the accumulator inserts-in-order-and-dedups but the combiner does a plain `addAll`. Sequentially the combiner is never called, so it looks perfect. In parallel, `addAll` concatenates two sorted-deduped halves into an unsorted, possibly-duplicated whole. Wrong answer, only in parallel. ### 3. Non-interference and statelessness The functions must be **non-interfering**: they must not modify the **stream source** during the operation (e.g. don't `remove` from the backing list inside the accumulator) — that risks `ConcurrentModificationException` or undefined results. They should also avoid relying on **mutable shared state** outside the per-chunk container; the only legitimate shared mutable state is a **CONCURRENT** collector's single thread-safe container, whose accumulator is explicitly safe for concurrent calls. ## Why violations hide The combiner is **only invoked for parallel streams**. So a buggy combiner is completely dormant in sequential execution and in unit tests that use small or sequential streams. It awakens only when: - the stream is `.parallel()`/`parallelStream()`, **and** - the source is large enough to actually split (small streams may not split), **and** - the spliterator chooses split points that exercise the combiner. That makes these bugs **nondeterministic and environment-dependent** — they may pass on a 2-core machine and fail on a 32-core one, or pass for 100 elements and fail for 100,000. ## Correct example ```java // Correct: accumulator and combiner implement the same 'append' semantics List<Integer> xs = stream.collect( ArrayList::new, List::add, // accumulate: append one List::addAll); // combine: append all — consistent with append ``` ## Incorrect example ```java // BUG: accumulator dedups, combiner does not -> parallel produces duplicates Set<Integer> uniques = stream.collect( HashSet::new, HashSet::add, // ok, Set dedups on add (a, b) -> { a.addAll(b); return; }); // ok here because Set still dedups on addAll // (Sets stay correct; the trap appears when the *container* doesn't enforce the invariant, // e.g. a plain List that the accumulator manually keeps sorted but addAll does not.) ``` ## How to stay safe - Prefer **standard Collectors** — their accumulator/combiner pairs are proven consistent. - If hand-writing, make the accumulator and combiner two views of the **same** operation; test the combiner explicitly (call it directly) rather than trusting a sequential run. - Never mutate the source; never share a non-thread-safe container unless the collector is CONCURRENT. - Test with a **parallel** stream over a **large** input to actually exercise the combiner.
- Why can a collect bug pass all sequential tests yet fail in production under load?The combiner only runs for parallel streams that actually split. Sequential tests never invoke it, and small inputs may not split, so an inconsistent combiner stays dormant until a large parallel run exercises it — making the failure nondeterministic and load-dependent.
- What does non-interference require of a collect's functions?They must not modify the underlying stream source while the terminal operation runs. Doing so risks ConcurrentModificationException or undefined results; the functions should operate only on their accumulation container.
saying these in an interview costs you the question
- Trusting a sequential test to validate the combiner — it is never called sequentially
- Letting the accumulator maintain an invariant (sort/dedup) the combiner ignores
- Returning a shared or pre-filled container from the supplier
- Mutating the stream source inside the accumulator