How does SolrCloud's compositeId router decide which shard a document lands on?

## The hash space When a collection is created with the default `compositeId` router, Solr divides a 32-bit hash space into `numShards` contiguous ranges and writes them into `/collections/<name>/state.json`. A two-shard collection looks like `shard1: 80000000-ffffffff`, `shard2: 0-7fffffff`. Indexing a document means hashing its uniqueKey to a 32-bit value and delivering it to the shard whose range contains that value; the receiving node forwards it to that shard's leader. Querying without a routing hint means fanning out to one replica of every shard, because any shard could hold a match. This is why `numShards` is mandatory for a compositeId collection and why you cannot simply "add a shard" to one: the ranges must partition the space exactly. To add capacity you SPLITSHARD, which halves an existing shard's range into two sub-shards and retires the parent. ## The composite part The router is called *composite* because the id may be composed of a routing prefix and a document id, separated by `!`: ``` acme!inv-42 ``` Instead of hashing the whole string as one unit, Solr hashes the prefix and the remainder separately and builds the final 32-bit value from the **high bits of the prefix hash** and the **low bits of the document-id hash**. Every document whose prefix is `acme` therefore shares the same high bits and lands inside one narrow contiguous band of the hash space — which, unless that band happens to straddle a boundary, means one shard. The payoff is at query time. `_route_=acme!` tells the aggregator which shard owns that prefix, so the request goes to exactly one shard instead of all of them. On a 20-shard collection that removes 19 shard requests, 19 sets of top-N merging, and the tail latency of the slowest shard. The same parameter works for deletes and for atomic updates. ## Bit masking for uneven tenants Co-location is only a win while the co-located set is a reasonable size. Give one tenant 200 million documents in a cluster whose average shard holds 20 million, and that shard becomes a hot spot no amount of replication fixes — replicas share the query load but every one of them still stores the whole oversized shard. For that case compositeId supports a bit count: `acme/4!inv-42` says "take only 4 bits from the prefix hash". Fewer prefix bits mean the tenant's documents occupy a *wider* slice of the hash space and therefore spread over more shards, while queries with `_route_=acme/4!` still restrict the fan-out to just that slice rather than the whole collection. It is a dial between perfect co-location and even distribution — but it must be chosen up front, because changing the bit count changes where documents hash to and requires a reindex of that tenant. CompositeId also supports a second level (`app!user!docId`), which lets you route at either granularity; the high half of the hash is shared between the two prefixes. ## The implicit router The alternative is `router.name=implicit`: Solr does no hashing at all, and the document goes to the shard you name — either through the `_route_` parameter on the request or through a field on the document named by `router.field`. You supply the shard names with the `shards` parameter at creation, and you can add more later with CREATESHARD (which is only allowed for implicit collections). This suits time-based or region-based partitioning where you want explicit control over which slice a document lands in, at the price of owning balance yourself. ## Failure modes to name in an interview - **Routing prefix baked into the id.** The prefix is part of the uniqueKey. `acme!inv-42` and `inv-42` are different documents, and moving a document between tenants means delete-and-reindex. - **Forgetting `_route_` on reads.** Indexing with prefixes but querying without them gives you all the skew and none of the speed-up. - **`_route_` on a query that must see everything.** Restricting the fan-out means results from other shards are silently missing, not merely deprioritised — a correctness bug, not a performance one. - **Skew after growth.** A tenant that was small at design time becomes the hot shard later; monitor per-shard document counts and index size, not just cluster totals. - **Assuming SPLITSHARD fixes prefix skew.** Splitting a shard whose documents all share one prefix's high bits may put nearly everything into one sub-shard, because the prefix pins the high bits that the split boundary uses.