A single Redis hash has grown to several million fields and application code calls HGETALL on it on every request. What goes wrong, and how would you read, shrink and eventually restructure that key?

## Why a huge hash is an operational problem Redis executes commands one at a time. A command's cost is therefore not just that caller's latency — it is a stall for every other client on that shard. `HGETALL` is O(N) in the number of fields, so on a hash with millions of entries a single call can occupy the server for tens or hundreds of milliseconds. Every other request queues behind it, and the symptom shows up as unexplained latency spikes on unrelated keys. Second, the reply must be **materialised**. The whole field/value set is serialised into the client's output buffer before it drains over the network. That is a large, sudden allocation; with several clients doing it concurrently, memory rises sharply, and if a client is slow to read, the connection can hit `client-output-buffer-limit` and be killed — after the server has already paid the full CPU cost. Third, the network and the client pay again: the payload crosses the wire and is then deserialised into application objects, usually to have two fields read from it. ## Reading safely - **`HMGET key f1 f2 f3`** — the first fix. If the request needs three attributes, fetch three attributes. Same single round trip, bounded cost. - **`HSCAN key cursor [MATCH pat] [COUNT n]`** — cursor iteration for when you genuinely need everything (an export, a migration, a rebuild). Each call returns a chunk and the next cursor; iteration ends when the cursor comes back `0`. The guarantees are worth stating precisely: every field present from start to end of the iteration is returned **at least once**; fields added or removed during iteration may or may not appear; **duplicates are possible**, so the consumer must be idempotent; and there is **no ordering**. `COUNT` is a hint about work per call, not an exact page size — raise it to reduce round trips, but not so far that one call becomes the very stall you were avoiding. Note also that `MATCH` filters *after* fetching, so a restrictive pattern can return empty chunks while still doing full work. - **`HLEN`** (O(1)) and **`HSTRLEN`** answer "how big is it" without transferring anything. ## Finding these keys before they hurt `redis-cli --bigkeys` samples the keyspace and reports the largest key per type; `--memkeys` reports by memory. `MEMORY USAGE key` gives a per-key estimate. `SLOWLOG GET` will already be full of `HGETALL` entries if this is happening in production. A useful habit is to alert on `HLEN` for known collection keys, since collections that grow unboundedly are almost always a modelling mistake that is cheap to catch early. ## Shrinking and deleting Deleting a huge key is itself an O(N) operation: `DEL` frees every field synchronously and stalls the server just like `HGETALL` did. Use **`UNLINK`**, which unlinks the key from the keyspace immediately and reclaims the memory in a background thread — the caller sees O(1). The `lazyfree-lazy-user-del` configuration makes `DEL` behave like `UNLINK` by default, and related `lazyfree-*` settings do the same for eviction and expiry. If you only need to trim, delete in batches: iterate with `HSCAN` and issue `HDEL` for a few hundred fields at a time, pacing the loop so the server keeps serving traffic. ## Restructuring: the real fix A key is the unit of distribution. In Redis Cluster a key belongs to exactly one hash slot on one shard, and resharding moves whole keys, so a single multi-million-field hash: - cannot be spread across shards, no matter how many you add; - concentrates all reads and writes for that object on one node; - makes that node's memory and CPU the ceiling for the whole object; - makes slot migration painful, since the key moves as one indivisible blob. The standard remedy is **sharding the hash by field name**: choose N sub-keys and place a field in `obj:{id}:<hash(field) mod N>`. Reads compute the same function, so a single-field read still costs one command; only full scans need to visit N keys. Choose N so each sub-hash stays in the low thousands of fields. Note that a hash tag like `{id}` keeps all the sub-keys of one object in the same slot — convenient for multi-key operations, but it also means they stay on the same shard, so drop the shared tag if what you need is to spread the load across nodes. Other restructurings worth considering: - **Is it really one object?** A hash with millions of fields is usually an index in disguise — "all events for a tenant", "every user's last-seen". Those want per-entity keys, or a sorted set, or a stream. - **Does it need per-field lifetime?** If entries should age out, per-field TTL (`HEXPIRE`, Redis 7.4+) or window-scoped keys with a whole-key `EXPIRE` remove the growth problem at the source. - **Does the hot path need the whole object at all?** Frequently the answer is a small summary key alongside the large one. ## Encoding, briefly Small hashes use a compact flat representation and convert to a hash table once they exceed the configured entry-count or value-length thresholds. A multi-million-field hash is long past that boundary, so its per-field overhead is the hash-table cost — one more reason the memory footprint of one giant key is worse than the same data spread over many small ones. ## The answer that lands Name all three costs of `HGETALL` (server-wide stall, reply-buffer memory, client deserialisation), give the bounded alternatives (`HMGET`, `HSCAN` with its exact guarantees), remember that deletion is equally O(N) and needs `UNLINK`, and finish on the structural point: one key is one shard, so the durable fix is to stop having a single key that large.