skip to content

How does Couchbase map a document key to a node using vBuckets?

level: middleimportance: must knowfreq 70%

answer

  1. Client knows where the key lives
  2. Fixed partitions between keys and nodes
  3. 1024 of them per bucket
  4. Hash the key, then read a map
  5. "Not my vBucket" triggers a map refresh

basics

~20 s

Couchbase hashes the document key into one of a bucket's 1024 vBuckets, then looks that vBucket up in the cluster map to find the node holding its active copy. The SDK does this locally, so requests go straight to the owning node.

solid answer

~50 s

Every Couchbase bucket is split into a fixed number of partitions called **vBuckets** — 1024 on production platforms. The SDK hashes the document key (CRC32) and takes it modulo the vBucket count to get a vBucket id. It then consults the **cluster map**, which the SDK fetches and keeps refreshed from the cluster, to learn which node currently holds the *active* copy of that vBucket and which nodes hold its replicas. The SDK opens a connection to that node and sends the operation directly — there is no routing proxy or coordinator hop, so a key-value read or write is a single network round trip. The vBucket count never changes; only the vBucket-to-node assignment changes when the topology changes. If the SDK's map is stale, the node replies "not my vBucket", the SDK refreshes the map and retries.

code

javascript · 4 lines
javascript
// How a Couchbase SDK locates a key: no router, no coordinator hop
const vbId = crc32(documentKey) % 1024;      // fixed partition count
const activeNode = clusterMap.active[vbId];  // cached, refreshed by the cluster
// operation is sent straight to activeNode

go deeper

for a junior

Be able to say that Couchbase splits a bucket into fixed partitions called vBuckets and the client hashes the key to find the right one, so requests go straight to the owning node.

for a middle

Explain both steps precisely: CRC32 of the key modulo 1024 gives the vBucket, the cached cluster map gives the active node, and a stale map produces a 'not my vBucket' retry rather than a wrong answer.

for a senior

Show you have operated this: connection fan-out from every app node to every data node, retry behaviour and timeouts during map churn, and why replica reads are a consistency decision rather than a load-balancing one.

for a principal

Own the sizing consequence — a fixed partition count bounds how far one bucket usefully spreads, and client-side topology awareness means SDK version and configuration-push health are part of your availability story.

## The problem vBuckets solve A distributed store has to answer one question on every operation: which node owns this key? Hashing the key straight onto nodes (`hash(key) % nodeCount`) is fatal, because adding or removing a node changes `nodeCount` and therefore remaps almost every key. Couchbase inserts a fixed layer of indirection between keys and nodes: the **vBucket** (virtual bucket), a hash partition of a bucket's key space. ## The two-step lookup Every Couchbase bucket is divided into a fixed number of vBuckets — **1024** on production platforms. The count is set when the bucket is created and does not change for the life of the bucket. Step one is deterministic and purely local: the SDK computes a CRC32 hash of the document key and takes it modulo the vBucket count, producing a vBucket id in `0..1023`. Because the count is constant, a given key always lands in the same vBucket, on every client, forever. Step two is a table lookup. The **cluster map** (also called the bucket configuration or vBucket map) is a table with one row per vBucket listing the node that holds its **active** copy and the nodes holding its **replica** copies. The SDK fetches this map when it opens the bucket and receives updates as the topology changes. With the vBucket id in hand, the SDK reads off the active node and sends the operation directly to it. ## No proxy hop This is the operationally important consequence. The client is topology-aware, so a key-value `get` or `upsert` is one round trip from the application to the node that actually holds the data. There is no router process in the path to size, monitor, or fail over. It also means every application node holds connections to every data node, which is worth knowing when you plan connection counts. ## Replicas ride the same map A bucket is configured with 0 to 3 replicas. Each vBucket therefore has one active copy and up to three replica copies, and the cluster map places them on distinct nodes (and, when server groups / rack awareness is configured, in distinct groups). Replicas are kept up to date by streaming mutations from the active copy over DCP, Couchbase's internal change protocol. Replica copies do not serve normal reads: an ordinary `get` always goes to the active. The SDKs expose explicit replica-read calls for the case where you would rather have a possibly-stale answer than an error, and using them is a deliberate consistency trade, not a load-balancing trick. ## What happens when the topology changes Adding, removing, or failing over a node does not change which vBucket a key belongs to — it changes which node owns that vBucket. During a rebalance, whole vBuckets are streamed to new owners and the map is updated; after a failover, replica vBuckets on surviving nodes are promoted to active and the map is updated. Either way the mapping from key to vBucket is untouched, so only the affected fraction of the key space moves rather than nearly all of it. Clients discover the change in two ways. The cluster pushes updated configurations to connected clients, and, as a backstop, a node that receives an operation for a vBucket it no longer owns replies with a **"not my vBucket"** error carrying the current configuration. The SDK refreshes its map and retries, so a brief window of staleness shows up as retried operations rather than errors — provided the operation is retried inside its timeout. ## Why the count is fixed A fixed, generously large partition count is what makes rebalancing incremental. With 1024 vBuckets, moving from four nodes to five means moving roughly a fifth of the vBuckets, each of which is a self-contained unit that can be streamed and handed over independently. The cost is that the count is also a cap on useful data-node parallelism for one bucket: you cannot spread a bucket over more nodes than it has vBuckets, and long before that the per-node vBucket count gets uneven enough to skew load. ## Interview traps Candidates often say the client asks a coordinator or a `mgmt` node where a key lives on every operation — it does not; the map is cached client-side. Others claim reads are load-balanced across replicas by default — they are not. And confusing the vBucket count (fixed per bucket) with the replica count (configurable per bucket, 0–3) is a reliable sign someone has read about Couchbase but not operated it.

  • Why does Couchbase hash keys onto a fixed partition count instead of directly onto nodes?
    Hashing straight onto nodes means the divisor changes whenever the node count changes, remapping nearly every key and forcing a near-total data movement. A fixed 1024-partition space decouples the two: the key-to-vBucket mapping is permanent, and only the vBucket-to-node assignment changes on a topology change, so a rebalance moves roughly the fraction of data that the ownership change implies.
  • Can a Couchbase client read from a replica vBucket, and what does that cost you?
    Yes — the SDKs expose explicit replica-read operations that query replica copies instead of, or alongside, the active. You do this when availability beats freshness, typically while a node is down and its actives have not been promoted yet. The cost is staleness: replicas are updated asynchronously over DCP, so a replica read can return a value older than the last acknowledged write, and it must never be used as a general load-balancing strategy.
  • What does the SDK do when it sends an operation to a node that no longer owns the vBucket?
    The node answers with a "not my vBucket" error that carries the current cluster configuration. The SDK installs the new map and retries the operation against the correct node, all inside the caller's timeout, so the application usually sees a slightly slower operation rather than an error. Persistent "not my vBucket" churn during a rebalance is normal; sustained failures after it finishes point at a client that is not receiving configuration updates.

saying these in an interview costs you the question

  • Says the client asks a router or coordinator per operation
  • Thinks reads are spread across replicas by default
  • Confuses the vBucket count with the replica count
  • Claims the vBucket count changes when nodes are added
  • Believes adding a node remaps every key in the bucket

context

resolving links…