How would you decide whether to use custom routing for a multi-tenant Elasticsearch index with thousands of tenants?

## Frame the decision as a cost comparison Two costs are in tension, and the tenant size distribution decides which dominates. **Fan-out cost.** Without custom routing, hashing on `_id` spreads every tenant's documents across every shard. A search scoped to one tenant must still ask every shard, because any shard could hold a match. Each shard-level request consumes a slot in the node's search thread pool, and the coordinating node cannot answer until the slowest of them replies. Per-query cost is proportional to the shard count, independent of how small the tenant is. With thousands of small tenants querying constantly, the search tier saturates on coordination overhead rather than on real work. **Skew cost.** With custom routing, a tenant's documents all hash to one shard. Shard sizes now follow your tenant size distribution, which in every real multi-tenant product is long-tailed. The largest tenant defines the largest shard, and a shard is atomic: the cluster can move it between nodes but cannot split one tenant's data across two. No amount of rebalancing helps. ## Measure before choosing The inputs you should bring to this decision: - **Tenant size distribution** — median, p99 and maximum documents and bytes per tenant. A distribution where the largest tenant is 50× the median is a different problem from one where it is 5,000×. - **Query mix** — what fraction of searches are tenant-scoped versus cross-tenant? Routing only pays where searches carry the routing key. Global admin search, cross-tenant analytics and background jobs still fan out. - **Query rate per tenant** — skew in *traffic* is as damaging as skew in data. A small tenant with enormous query volume also creates a hot shard. - **Growth and churn** — tenants that grow 10× in a quarter turn a comfortable shard into a problem, and routing offers no incremental relief. - **Lifecycle requirements** — deleting a tenant from a routed shared index means a delete-by-query and the deleted documents linger until merges reclaim them; deleting a dedicated index is instantaneous. ## The operational contract routing imposes Once documents carry a custom routing value, that value is part of their identity for every future operation. A get, update or delete by id without it hashes the `_id`, lands on the wrong shard, and reports the document missing — silently. Two mechanisms make this safe: ``` "mappings": { "_routing": { "required": true } } ``` turns an omitted routing value into an error rather than a misrouted write. And a per-tenant alias can carry both a filter and the routing values, so application code targets `orders-tenant-7` and cannot forget either: ``` { "index": "orders", "alias": "orders-tenant-7", "filter": { "term": { "tenant": "tenant-7" } }, "routing": "tenant-7" } ``` The alias's `routing` sets both index-time and search-time routing; `index_routing` and `search_routing` set them separately when they must differ. ## The middle options This is rarely a binary choice. - **`index.routing_partition_size`.** Set to *k*, a routing value maps to *k* shards rather than one: the shard is chosen from the routing hash combined with the `_id` hash. A tenant's searches then touch *k* shards instead of all of them, while its data spreads over *k* shards instead of piling onto one. It is a static setting, must be smaller than the primary count, requires routing to be mandatory, and does not coexist with join fields. It is the right tool for mid-size tenants where one shard is too small and full fan-out is too expensive. - **Tiering by size.** Give the handful of whale tenants their own indices — full fan-out inside a dedicated index is cheap when that index has few shards, and lifecycle operations become index operations. Put the long tail in one routed shared index. Hide the split behind aliases so the application never knows which tier a tenant is in, and make promotion between tiers a reindex you can run on a schedule. - **No routing at all**, if searches are dominated by cross-tenant work or if the index is small enough that fan-out is a handful of shards. ## Second-order effects worth naming A tenant-scoped search on a routed index touches one shard, so its relevance scoring uses one consistent set of term statistics — an accidental improvement over the fan-out case. Conversely, a cross-tenant search over a routed index now faces a deliberately skewed document distribution, which is exactly the condition under which per-shard statistical skew distorts ranking. And the pre-filter that lets time-based searches skip shards does nothing for tenant fan-out, because every shard plausibly holds matching data. ## What interviewers listen for The answer that reads as principal-level starts by asking for the tenant size distribution and query mix instead of answering yes or no, names the hot-shard failure mode as unfixable by rebalancing rather than as a tuning problem, treats the routing value as a permanent contract with an enforcement mechanism, and proposes a tiered design with aliases as the seam — because the real decision is not "routing or not" but "how do we keep the option to change our mind per tenant".