Why do event-sourced systems typically partition the event log into one stream per aggregate ID rather than writing every event for every entity into a single global stream?

## What an aggregate is, and why it owns a stream An aggregate, in the DDD sense that event sourcing borrows, is a **consistency boundary** — a cluster of entities and value objects (e.g., an `Order` plus its `OrderLines`) that must be kept invariant-consistent as a single transactional unit, and whose id (e.g., `order-91a3`) uniquely identifies that boundary. Event sourcing partitions the log by giving each aggregate instance its own stream — a strictly ordered, independently addressable sequence of events: order-91a3's events live only in stream `order-91a3`, account-482's events live only in stream `account-482`, and the two never share a stream. This is a deliberate structural choice, not an accident of implementation, and it follows directly from where business invariants actually live: - an invariant like 'an order can't ship before it's paid' only ever needs to reason about events within one order's history — it never needs to know what happened to a different order. - Because invariants are aggregate-scoped, **the ordering guarantee the system actually needs is aggregate-scoped too**: you need a total order of events within `order-91a3`, but you never need a total order across `order-91a3` and `order-77b1` — they're causally independent. ## Two mechanisms that depend on it This matters concretely for two mechanisms already central to event stores. 1. **First, optimistic concurrency control** operates at exactly the stream granularity: an expected-version check on `order-91a3` only contends with other writers to `order-91a3`. If every aggregate shared one global stream, then any two concurrent writes anywhere in the system — a payment on one order, a shipment on a completely unrelated order — would race for the same expected-version slot, meaning unrelated business operations would spuriously conflict and force retries on each other purely because they happened to land in the same tiny time window. Partitioning by aggregate ID makes concurrency conflicts only occur when they're semantically real (two commands actually touching the same entity). 2. **Second, read cost**: loading an aggregate to handle a command means replaying its stream from the last snapshot to rebuild in-memory state. If all events for all aggregates lived in one stream, loading `order-91a3` would mean scanning (or maintaining a secondary index over) potentially the entire system's event history to filter down to the events that actually belong to that order — partitioning makes this a read proportional to that one aggregate's own events, not the whole system's history. ## The trade-off on the query side The trade-off shows up on the query side. Business questions that are naturally cross-aggregate — 'show me every order placed in the last hour,' 'list all accounts over their credit limit' — can't be answered by reading one stream anymore, because no single stream holds events from multiple aggregates. Event stores solve this by exposing (or letting you build) a secondary, system-wide ordering in addition to per-stream ordering: - **EventStoreDB** provides a virtual `$all` stream that gives a single global, monotonically-increasing position across every stream in the store, letting a subscriber consume 'everything, in commit order' even though writers only ever address individual per-aggregate streams. - **Kafka's** analogous structure is a topic split into partitions (commonly keyed by aggregate/entity id so all of one entity's events land on the same partition and keep their relative order), plus consumers that can subscribe across all partitions if they need the full picture, accepting that events on different partitions have no defined relative order to each other. Either way, cross-aggregate views are built by a downstream projection that consumes the global feed and folds it into a purpose-built read model, rather than by querying the event store's per-aggregate streams directly. ## Failure modes — the boundary, not the mechanism Failure modes here usually come from getting the aggregate boundary wrong, not from the partitioning mechanism itself. - **If a team defines aggregates too coarsely** — e.g., one stream per warehouse rather than per SKU — every inventory adjustment to any item in that warehouse contends on the same stream, producing exactly the artificial optimistic-concurrency-conflict storm partitioning was supposed to prevent, because now unrelated business operations really are sharing a stream. - **Conversely, defining aggregates too finely** can scatter data that's always read and written together across many tiny streams, forcing every command handler to stitch together multiple stream reads and increasing the chance of accidentally violating an invariant that spans what should have been one aggregate. - **Another common failure is treating the global-position feed as a substitute for proper read models** — subscribing to it ad hoc from application code for a specific query instead of materializing a dedicated projection — which tends to produce slow, fragile queries that re-scan large swaths of history on every request. ## Where it shows up A concrete real-world shape: a banking event store gives every account its own stream (`account-482`), so a deposit to Alice's account and a withdrawal from Bob's account never contend on the same expected-version check and both replay independently in milliseconds, while a monthly regulatory report ('all transactions across all accounts in March') is served by a separate projection that has already consumed the global ordered feed and materialized it into a queryable report table.