A team's dedicated managed vector cluster averages five percent utilisation; when does a serverless vector index cost less, and what limits come with it?
answer
- paying for idle node-hours
- storage plus per-request units
- break-even on monthly query volume
- memory sets a size floor
- quotas, caps, fewer knobs
basics
~20 sA dedicated cluster bills provisioned nodes whether busy or idle; a serverless index bills storage plus each read and write. Serverless wins while monthly request charges stay below the cluster's fixed bill, but brings rate quotas, caps and fewer tuning knobs.
solid answer
~50 sFive percent utilisation means the team pays for capacity it does not use, and a dedicated cluster cannot always shrink to match, because its nodes must still hold the index in memory. A serverless index swaps that fixed bill for storage plus per-request units, so the comparison is arithmetic on real traffic: monthly queries and writes times their unit prices, plus storage, against the cluster's monthly cost. Low or spiky traffic usually favours serverless; high, steady query rates eventually cost more per request than owning nodes, and some services charge a query by how much data it touches, which lowers the break-even. The limits are the service's: request-rate and concurrency quotas that throttle bursts, caps on result count, metadata size and namespaces, and few or no index or hardware knobs, so latency tuning is no longer yours.
go deeper
Know that a dedicated cluster bills for nodes whether busy or idle, and a serverless index bills storage plus each read and write.
Explain why a dedicated cluster cannot always shrink with its traffic: the index must still fit in its nodes' memory. List the usual serverless caps and quotas.
Compute the break-even from real traffic and unit prices, and show how a per-query charge that grows with data size moves it. Name which serverless limit the design hits first.
Turn the arithmetic into a policy: which workloads default to serverless, the traffic and latency thresholds that move one to dedicated, and the spend alerts that catch a bill spike.
## The two billing models A **dedicated managed cluster** is a set of nodes the provider runs for you alone. You choose a node size and count, the provider bills per node-hour, and the bill is the same whether the nodes answer ten queries a second or none. Latency is more predictable because no other customer shares the nodes, and capacity changes are explicit resizes. A **serverless vector index** hides the nodes. The provider pools hardware across customers and bills for what you consume, typically: - **storage**, per gigabyte-month of vectors and metadata; - **reads**, per query or per read unit; - **writes**, per upsert or delete, or per write unit. Units differ between services. Some count one unit per request; **some charge a query by how much data it touches**, so the same query costs more as a namespace grows. Read the pricing model, not just the price. ## Why five percent utilisation is not an automatic switch Low utilisation says the cluster is oversized for its traffic. Before leaving the shape, check whether a smaller cluster would do: a dedicated cluster's floor is set by **memory**, because the nodes must hold the index, so a cluster sized for its corpus rather than its traffic cannot shrink to a twentieth. If the memory floor is what keeps it large, serverless is the shape that removes the idle cost; sizing the index to memory itself is a separate subject. ## Doing the arithmetic The figures below are a scenario's, invented to show the method; real prices and units vary by provider. | Item | Scenario figure | |---|---| | Dedicated cluster | $1,800 per month | | Serverless storage | $0.40 per GB-month, 20 GB stored | | Serverless reads | $10 per million queries, one unit per query | | Serverless writes | $4 per million upserts | | Traffic | 3 million queries and 1 million upserts per month | Serverless monthly cost: storage 20 x $0.40 = **$8**, reads 3 x $10 = **$30**, writes 1 x $4 = **$4**, total **$42**, against **$1,800** for the cluster. Break-even on queries, holding storage and writes fixed: ($1,800 - $8 - $4) / $10 per million = **178.8 million queries a month**. A 30-day month has 2,592,000 seconds, so that is about **69 queries per second on average**. Now suppose the service charges **five read units per query** because of the namespace's size. Each query costs five times as much, break-even falls to 178.8 / 5 = **35.8 million queries a month**, about **14 per second**. The billing model moved the answer by a factor of five without the traffic changing. ## The limits that come with serverless 1. **Rate and concurrency quotas.** Bursts above the quota are throttled, often with HTTP status 429, so clients need backoff and a burst plan. 2. **Caps.** Limits on result count per query, metadata size per record, vector dimensions and namespaces per index are common; check each against the design. 3. **Fewer knobs.** Index parameters, hardware type and replica counts are mostly the provider's, so latency tuning becomes a request to the provider. 4. **Variable latency.** Shared hardware and data that is not kept hot make latency less predictable; how a service pages cold data in is a separate subject. 5. **Spend tracks traffic.** A traffic spike or a bulk reload is a bill spike, so budgets need alerts, not just a fixed line item. Write-heavy workloads deserve their own line in the model. A nightly reload of the whole corpus bills every upsert, and on services that price writes by size, larger vectors and metadata cost more per record. A workload that is cheap to query can still be expensive to keep fresh. ## When each wins - **Serverless** fits low or spiky traffic, many small indexes or tenants, early products whose traffic is unknown, and teams that want no capacity work. - **Dedicated** fits high, steady query rates past the break-even, strict latency targets, workloads that need index tuning, and budgets that prefer a flat line. ## How to answer in an interview State both billing models, do the break-even on the candidate's own numbers aloud, and point out the two traps: the memory floor that keeps a dedicated cluster large, and per-unit billing that scales with data size. Then list the limits and say which of them the design would hit first.
- Why might the serverless bill jump even though query volume stayed flat?If the service charges a query by the data it touches, growing a namespace raises every query's cost with no change in traffic. Writes are another source: a bulk reload or backfill bills every upsert. Track cost per query and per write as metrics, and alert on spend, not only on the request count.
- What would make you keep the dedicated cluster despite low utilisation?A strict latency target that shared hardware cannot hold, sustained traffic close to the break-even, a need to tune index parameters, or bursts larger than the serverless quota allows. In that case right-size the cluster to its memory floor, and consider whether several small indexes could share it.
- Serverless is always cheaper because you only pay for what you use.
- Every serverless service charges a query the same whatever the index size.
- A cluster at five percent utilisation can shrink to five percent of its size.
- Serverless vector indexes have no rate limits, so bursts never throttle.
- Utilisation alone decides the choice; query and write volumes do not matter.