skip to content

A team building a product catalog on a key-value store creates four separate index tables alongside the base product table — one each to look up products by category, by brand, by warehouse, and by price tier. What operational cost does this design add to every single product write, and how does that cost scale as more index tables are added?

level: middleimportance: should knowfreq 50%

answer

  1. N+1 writes per mutation in the worst case
  2. delete+insert if indexed value changes, not just update
  3. cost scales linearly with number of index tables
  4. throttling on index table even when base table looks healthy
  5. reconsider design past ~3-4 hand-rolled indexes

basics

~20 s

Every time a product is created or changed, the app now has to write not just to the main table but to all four index tables too — so one logical update turns into up to five separate writes. Add more index tables and every write gets slower and more expensive, even if nothing about that particular field changed.

solid answer

~50 s

Each index table represents an additional write on every mutation that touches its indexed attribute (and a delete+insert pair, not just an update, if the attribute's value changes). With four index tables, a single product creation can mean five total writes (one base + four index), and an update to a field that appears in three of the four indexes still touches four tables. This is write amplification, and it scales roughly linearly with the number of index tables: N indexes means up to N+1 writes per mutation, each consuming its own throughput/capacity, latency, and failure surface. It also means N+1 places that can partially fail and drift out of sync, and N tables' worth of storage overhead. Past a handful of index tables, teams typically reconsider — batching the writes transactionally where possible, propagating asynchronously to bound the synchronous cost, or switching to a store/search index (e.g., Elasticsearch, or a relational read replica) purpose-built for multi-attribute querying instead of hand-rolling N single-attribute indexes.

go deeper

for a junior

Should grasp that more index tables mean more writes per update, in plain terms, without needing exact multipliers.

for a middle

Should be able to reason through which specific index tables get touched by a given field change, and name write amplification and storage as the concrete costs.

for a senior

Should identify throttling/latency on an index table (while the base table looks fine) as a diagnostic signature of this cost, and propose concrete mitigations (batching, async propagation, composite indexes).

for a principal

Should set a practical threshold for when to stop hand-rolling more index tables and move to a native secondary-index feature or dedicated search index, weighing operational cost against the alternative's own trade-offs.

## The shape of the cost Write amplification is the direct, mechanical cost of the index table pattern, and it's worth being precise about how it accumulates rather than treating it as a vague 'more writes are slower' hand-wave. In the scenario described, the base product table holds the authoritative row keyed by SKU or product ID. Each of the four additional tables — by category, by brand, by warehouse, by price tier — exists purely to answer a query the base table's key can't answer directly. None of these four tables are optional extras the store maintains for you; every single one is a hand-maintained structure the application is responsible for writing to, in addition to the base write, on every mutation. ## Counting the writes Concretely: - **Creating a brand-new product** means one write to the base table plus one write to each of the four index tables that has to include this new product — five writes total for one logical 'create a product' operation. - **Updating an existing product's price** (assuming price tier is a bucketed value, like 'under $25' or '$25–$100') means: one write to the base table, plus — if the price tier bucket changed — a delete of the old price-tier index entry and an insert of the new one (two operations), plus, if category/brand/warehouse didn't change, no writes to those three index tables at all. So the actual amplification factor per write is not a fixed constant; it depends on how many of the indexed attributes changed in that particular mutation: 1. a worst case of roughly 2N+1 operations (N deletes + N inserts + 1 base write) if every indexed attribute's value changed; 2. a best case of N+1 (N inserts, no deletes) on initial creation; 3. down to just 1 (the base write) if the mutation touched no indexed attribute at all. ## Three dimensions of scale This cost scales in three dimensions as more index tables are added. 1. **First, throughput/capacity cost scales roughly linearly with N**: on a provisioned-capacity system like DynamoDB, each of those N extra writes consumes its own write capacity units against its own table, so the effective write cost of maintaining a heavily-indexed entity can be several times the cost of the base write alone, even before accounting for the delete+insert doubling on attribute changes. 2. **Second, latency scales with N** if the writes are done sequentially and synchronously — a caller waiting for a product creation to fully complete is waiting on the slowest of five (or more) round trips, unless the store's transaction API lets some of them be batched into one round trip (with the scope caveats discussed for consistency strategies) or the writes are parallelized. 3. **Third, and often underweighted, failure surface scales with N**: every one of those N+1 writes is a place a partial failure can happen, and each partial failure is a potential drift between the base table and one specific index table, so operationally, N index tables mean N independent 'is this index correct?' questions to monitor and N independent repair paths to build, not one. ## How it surfaces in production The failure mode this produces in production is a slow accumulation of small inconsistencies plus a very real throttling risk: a hot product catalog under load that updates category, brand, warehouse, and price tier in bursts (a bulk re-pricing job, a seasonal recategorization) can see the index tables throttle or queue up well before the base table does, because the index tables are absorbing a multiple of the base table's write volume. Teams that hit this commonly report the symptom as 'writes are slow/throttled' on a table that, looked at in isolation, has moderate load — the real load is the sum across all the index tables plus the base table. ## The standard mitigations The standard mitigations, once a design like this starts to hurt, fall into a few buckets: - **batch** what can be batched into a single transactional write where the store's transaction scope allows it; - **move index maintenance off the synchronous write path** entirely via async propagation (accepting eventual consistency in exchange for bounding the caller-visible cost to just the base write); - **collapse indexes** that are always queried together into a single composite-key index table rather than maintaining them separately; or - past some number of indexed attributes (a rule of thumb many teams use is once you're maintaining more than three or four hand-rolled index tables for one entity), **reconsider the store choice altogether** — a relational database with native multi-column indexing, a managed secondary-index feature like a DynamoDB Global Secondary Index, or a dedicated search index like Elasticsearch/OpenSearch fed by a change stream, all of which push the index-maintenance cost into infrastructure that's built and optimized for exactly this problem rather than reinventing it per attribute in application code.

  • Does updating a product's warehouse location require touching the category index table too?
    No — only index tables for attributes that actually changed need to be touched. A warehouse-location change only requires updating the warehouse index table (delete the old entry, insert the new one) plus the base table; the category, brand, and price-tier index tables are untouched because their indexed value didn't change.
  • How would you reduce the write cost if categories and brands are almost always queried together (e.g., 'brand X products in category Y')?
    Collapse the two separate index tables into one composite index table keyed by, say, category+brand (or brand+category, depending on which is more selective), rather than maintaining them independently. This trades some query flexibility (you can no longer query brand alone via that table) for cutting the number of maintained indexes and their associated write cost.
  • At what point would you tell this team to stop adding index tables and reconsider the data store?
    There's no fixed universal number, but once you're maintaining more than a handful of single-attribute index tables for one entity — commonly cited around three or four — the maintenance and consistency burden usually outweighs the benefit, and it's worth evaluating a store with native secondary indexing or a purpose-built search index fed asynchronously instead.

It's like keeping a filing cabinet of paper documents plus four separate rolodexes (by category, by brand, by location, by price range) that each need a new card added or moved every time you file one document. File one document, touch up to five things by hand.

saying these in an interview costs you the question

  • Says the extra writes are 'basically free' or negligible without acknowledging capacity/throughput cost
  • Doesn't recognize that a delete+insert (not a plain update) is needed when the indexed value itself changes
  • Assumes every mutation touches every index table regardless of which fields actually changed
  • Has no mitigation beyond 'just provision more capacity'
  • Can't identify throttling on an index table as a symptom traceable back to write amplification

context