skip to content

You are designing a 'vehicles near me' lookup on Redis Cluster for a fleet of two million vehicles that report their position every few seconds. How would you lay out the geospatial keys, and what specifically breaks if you keep every vehicle in a single key?

level: principalimportance: nice to knowfreq 20%

answer

  1. one key = one slot = one shard = one core
  2. partition by city or geohash prefix, hash tag {sf}
  3. fan out to 9 neighbour cells, merge client-side
  4. no per-member TTL → last-seen zset + ZREM sweeper
  5. always COUNT + ASC; reads can go to replicas

basics

~20 s

One key lives in one hash slot on one shard, so a single global index pins every position write and every query to one core, and dense-area queries scan huge candidate sets. Partition by region (geo:{city}) or geohash-prefix cell, fan out to neighbouring cells client-side, and sweep stale members with ZREM since geo members have no TTL.

solid answer

~60 s

**Why one key fails**: a key is one hash slot on one shard, so 2M vehicles × a write every few seconds is several hundred thousand `GEOADD`s per second all landing on one core, plus every `GEOSEARCH`. Redis executes commands one at a time, so writes and queries compete for the same thread; a wide radius in a dense city scans a large candidate set and adds latency for everyone. The shard cannot be split — resharding moves whole keys — so you cannot grow out of it. **Layout**: partition geographically. One key per city or per coarse geohash cell — `geo:{sf}`, or `geo:{9q8y}` for a fixed-precision cell — so load spreads across shards and each key stays small. Queries near a boundary fan out to the containing cell plus its neighbours (a handful of `GEOSEARCH` calls, merged and re-sorted client-side). **Write shaping**: only publish when a vehicle has moved beyond a threshold, and batch triples into one `GEOADD`. **Staleness**: geo members carry no TTL. Keep a parallel sorted set scored by last-seen and sweep with `ZREM`. **Reads**: `GEOSEARCH` is read-only, so replicas can serve queries.

code

text · 12 lines
text
# writes: batch a region's updates, and record last-seen in the same slot
GEOADD veh:{9q8y} -122.41 37.77 v:8123 -122.42 37.78 v:9004
ZADD   seen:{9q8y} 1755100000 v:8123 1755100001 v:9004

# read: query the containing cell plus neighbours, merge client-side
GEOSEARCH veh:{9q8y} FROMLONLAT -122.415 37.775 BYRADIUS 3 km ASC COUNT 20 WITHDIST
GEOSEARCH veh:{9q8z} FROMLONLAT -122.415 37.775 BYRADIUS 3 km ASC COUNT 20 WITHDIST

# sweeper: members not refreshed in 60s (geo members have no TTL)
ZRANGEBYSCORE seen:{9q8y} -inf 1755099940
ZREM veh:{9q8y} v:8123
ZREM seen:{9q8y} v:8123

go deeper

for a junior

Know that a Redis key lives on one shard, so one giant geo key cannot be scaled by adding nodes.

for a middle

Propose partitioning by city or geohash cell with hash tags, and describe the client-side fan-out and merge for boundary queries.

for a senior

Add write shaping (movement threshold, batching, short member names), the last-seen sorted set plus ZREM sweeper for staleness, and bounded queries with COUNT/ASC served from replicas.

for a principal

Own the tradeoff space: cell size versus fan-out, skew versus simplicity, non-atomic cell handover as an accepted inconsistency, and a clear line for when the query shape outgrows circles and boxes and belongs in a GIS behind Redis's candidate filter.

## Framing the problem Two million vehicles reporting every few seconds is on the order of 300k–700k writes per second, plus a read whenever a rider opens the app. The question is not "can Redis do geo" — it can — but how the key layout maps that load onto shards and onto the single command-execution thread of each shard. ## What a single global key costs you **It is one slot on one shard.** Redis Cluster distributes by key, and resharding relocates whole keys. A key holding all two million members lives entirely on one primary; adding shards does not relieve it. That is the fundamental ceiling: your capacity is one node's capacity, forever. **Writes and reads serialise against each other.** Every `GEOADD` and every `GEOSEARCH` for the whole product queues on that one shard's execution thread. Position updates are individually trivial, but at hundreds of thousands per second they saturate a core, and a query's latency now includes waiting behind them. **Query cost tracks density, not just key size.** `GEOSEARCH` scans the covering geohash cell plus its eight neighbours and then distance-filters the candidates. In a dense downtown a 5 km radius touches an enormous number of members; that work happens inside one command, so it delays every other client on that shard. A single unbounded query in the busiest city is enough to produce a visible latency spike. **Operations get worse as the key grows.** Deleting or migrating a multi-million-member sorted set is expensive; snapshots and replica sync carry the whole thing; and the memory of that one key is bounded by one node's RAM. ## The layout that works: geographic partitioning Split the index by area so both writes and queries spread across shards. **Option A — one key per operational region**: `veh:{sf}`, `veh:{nyc}`. Natural if the business already thinks in cities and dispatch never crosses them. Simple to reason about, but load is as skewed as your cities are — one megacity is still a hot shard. **Option B — one key per fixed-precision geohash cell**: `veh:{9q8y}` using, say, a 4–5 character geohash prefix so each cell is a few kilometres across. This spreads dense regions across many keys automatically and puts a natural ceiling on any one key's size. The cost is fan-out: a query centred near a cell edge, or with a radius larger than a cell, must query the containing cell plus its neighbours and merge the results. With either option, hash tags (`{sf}`) let you co-locate related keys deliberately — for example a region's geo index and its last-seen sorted set — so a Lua script or a `GEOSEARCHSTORE` involving both stays inside one slot. **Client-side fan-out** is the price of partitioning: issue N `GEOSEARCH` calls in parallel (pipelined per shard), concatenate, re-sort by distance and truncate to the requested count. Cap N by choosing a cell size comfortably larger than the typical search radius, so the common case touches 1–9 cells and not hundreds. ## Shaping the write path - **Suppress non-movement.** A parked vehicle re-reporting the same coordinate is pure load. Publish only when displacement exceeds a threshold or a heartbeat interval elapses. - **Batch.** `GEOADD` accepts many triples per call; aggregating a few hundred milliseconds of updates per region collapses round trips and per-command overhead dramatically. - **Keep member names short.** Member strings dominate sorted-set memory; use a numeric ID, not a UUID string or JSON. - **Handle region crossing.** A vehicle moving between cells must be `ZREM`ed from the old key and `GEOADD`ed to the new one. Because those keys may be in different slots, that is two commands and is not atomic — a brief double or missing presence is acceptable for a candidate index, but the writer must be idempotent and the reader must tolerate duplicates. ## Staleness: there is no per-member TTL A geo member is a sorted-set member; TTL exists only on the whole key. So a crashed client's vehicle stays in the index forever unless you remove it. The standard pattern is a parallel sorted set per region scored by last-seen epoch, updated in the same pipeline as `GEOADD`; a sweeper periodically runs `ZRANGEBYSCORE seen -inf <cutoff>` and issues `ZREM` on both keys in bounded batches. Readers should additionally treat a position older than a few seconds as unusable. An alternative for genuinely ephemeral data is a rolling key per time window (`veh:{sf}:<minute>`), with an `EXPIRE` on the whole key and queries covering the current and previous window. It trades duplicate members across windows for free expiry. ## Reads `GEOSEARCH` is a read-only command (unlike the deprecated `GEORADIUS`, whose optional `STORE` made it a write), so replicas can serve proximity queries. Always send `COUNT n` with `ASC` to bound both work and reply size; consider `COUNT n ANY` where "some nearby vehicles" is acceptable and "the nearest" is not required — it short-circuits instead of ranking the whole candidate set. ## When to leave Redis If the queries stop being "points inside a circle or box" — polygon geofences, route-time ranking, road snapping, multi-attribute filters like *nearby AND electric AND rated > 4.5* — Redis is the wrong arbiter. The durable pattern is Redis as a millisecond candidate generator that reduces millions of vehicles to tens, with the precise, expensive logic running on that shortlist.

  • A vehicle drives from one cell key into the next. How do you handle the handover?
    The writer must `ZREM` the member from the old key and `GEOADD` it to the new one. Those keys may sit in different hash slots, so it is two separate commands and cannot be made atomic by a transaction or script. For a candidate index that is acceptable: make the writer idempotent, have readers deduplicate by member ID after merging the fan-out results, and rely on the staleness sweeper to clear a leftover entry if the removal was lost.
  • How do you stop one very dense city from becoming a hot shard even after partitioning by city?
    Move to finer, fixed-precision geohash cells instead of city-sized keys, so a dense city becomes many keys spread across slots by the cluster's own hashing. Alternatively split that one city manually into sub-region keys. Either way the goal is that no key's member count and no shard's command rate is set by the largest metro.
  • How would you decide the cell size for the geohash-prefix layout?
    Work backwards from the typical search radius: a cell should be comfortably larger than the radius so a query touches the containing cell plus at most its eight neighbours. Then check the two ceilings — members per key (keep it small enough that a dense-area GEOSEARCH stays sub-millisecond) and write rate per shard. If those conflict, you shrink cells and accept more fan-out.

A single global geo key is one clerk holding the only ledger for the whole country: everyone queues at that desk. Regional ledgers let many clerks work at once, at the cost of asking two or three of them when you live near a county line.

saying these in an interview costs you the question

  • Assuming Redis Cluster will spread one large key across shards — resharding moves whole keys, never parts of one
  • Ignoring that geo members have no TTL and expecting stale vehicles to disappear on their own
  • Running unbounded GEOSEARCH without COUNT in dense areas
  • Cross-slot GEOSEARCHSTORE or multi-key scripts without a shared hash tag
  • Choosing tiny cells that force queries to fan out to dozens of keys, trading one bottleneck for network amplification

context