A particular shard key value is receiving disproportionately more traffic than others - a 'hot' shard or hot key. What causes hotspots in a sharded system, and what techniques mitigate them?

## What a hotspot actually is A hotspot is what happens when the theoretical load-distribution property of your partitioning scheme fails to hold in practice: even though your hashing or ranging strategy assigns roughly equal shares of data or key space to each shard, the actual traffic - reads, writes, or both - concentrates heavily on one shard, turning it into a bottleneck while its siblings sit comparatively idle. It's a mismatch between how you partitioned the key space and how real-world access to that key space is distributed, and it's one of the most common ways a sharded system fails under real production load even after passing every load test that used synthetic, uniformly-distributed keys. ## Where hotspots come from Three causes show up repeatedly. 1. **The first is a skewed key popularity distribution**, often called the 'celebrity problem': in a social platform sharded by user ID, a celebrity account with millions of followers generates order-of-magnitude more reads and writes than a typical user, and no partitioning scheme distributes a single key's traffic across multiple shards by default - the celebrity's data lives on exactly one shard, so that shard absorbs the celebrity's entire load regardless of how evenly other users' keys are spread. 2. **The second is temporal/sequential clustering:** if the shard key is or correlates with a timestamp or auto-incrementing ID, all new writes land on whichever shard owns the current top of the range, making 'newest data' a permanent write hotspot even though total data volume across shards stays balanced over time. 3. **The third is workload skew unrelated to the key itself** - a marketing campaign, a viral post, or a retry storm from a misbehaving client can suddenly concentrate load on one or a few keys that were previously unremarkable. ## Read hotspots versus write hotspots The mitigation strategy depends heavily on whether the hotspot is a read hotspot or a write hotspot, because they have very different fixes. - **Read hotspots are the more tractable case.** Because reads don't have to be linearizable with every other read, you can absorb them with a cache in front of the shard (Redis, Memcached, or a CDN for read-mostly public data) so only cache misses ever reach the database, or by adding more read replicas specifically for that shard and routing reads to whichever replica is least loaded. - **Write hotspots are harder**, because every write ultimately has to land somewhere durable, and you can't cache your way out of write volume. The standard technique here is **key salting** (sharding key splitting): instead of writing all of a hot key's data under one literal key, append a random or rotating suffix (e.g. `celebrity123#0` through `celebrity123#9`) so the data, and critically the write traffic, spreads across N shards instead of one. Reads for that logical key then have to fan out across all N salted variants and merge results, trading some read complexity for write scalability; this pattern is explicitly recommended in DynamoDB's and Bigtable's own scaling guidance for exactly this reason. ## Dedicating capacity to a known-hot key Another mitigation is dedicating extra or isolated capacity to a known-hot key up front rather than reactively - if you can identify in advance that a specific tenant or entity will be disproportionately large (a known enterprise customer signing a big contract, a flagship product SKU), placing it on its own dedicated shard or shard group avoids it ever contending with the noisy neighbors sharing a generic hash bucket; this is a common pattern in multi-tenant SaaS, sometimes called **tenant isolation** or a 'silo' model for the largest tenants while smaller tenants share 'pooled' shards. ## The structural fix: a compound shard key A structural mitigation that prevents many hotspots before they occur is choosing a compound shard key that mixes a naturally meaningful component with a well-distributed one from the start - e.g. instead of sharding purely by `customer_id` (which lets one giant customer dominate a shard) or purely by a random hash (which loses any locality), some systems shard by a combination like `hash(customer_id) mod small_bucket_count` combined with the ID itself, deliberately spreading even a single very large customer's data across a bounded number of shards rather than one. ## How it looks in production In production, hotspots show up as one node in a monitoring dashboard with dramatically higher CPU, IOPS, or queue depth than its siblings while aggregate cluster metrics look fine - this asymmetry is exactly why per-shard, not just cluster-average, monitoring and alerting is essential; averaging across shards can completely hide a hotspot that's actively degrading service for the users unlucky enough to be routed to it. Bigtable's public documentation and DynamoDB's adaptive capacity feature, which automatically shifts extra throughput capacity toward a hot partition on the fly, both exist specifically because this failure mode is common enough to warrant first-class platform support rather than leaving every team to invent salting logic independently.