skip to content

How do you decide that a join or aggregation is expensive enough to justify permanently storing redundant data in a transactional schema?

level: seniorimportance: should knowfreq 40%

answer

  1. rank by total time, then read the plan
  2. covering index first — same win, no second truth
  3. strong case: aggregate over unbounded children
  4. read-to-write ratio must strongly favour reads
  5. sorting by a foreign column is a real justification

basics

~20 s

Measure first: confirm the query is hot and that the join or aggregate dominates its plan. Exhaust cheaper fixes — indexes, especially covering ones, query rewrites, caching. Denormalize when the cost grows with data you cannot bound, the read rate greatly exceeds the write rate, and you can name a sync mechanism and a drift check.

solid answer

~60 s

I want four things to be true before adding redundancy. **1. Evidence.** The query is genuinely hot — total time, not just per-call latency — and its plan shows the join or aggregation as the dominant cost, not a missing index or a bad predicate. **2. Cheaper fixes exhausted.** An index that supports the join, or a covering index that answers the query from the index alone, removes the same work without creating a second copy of the truth. Query rewrites and short-lived caches come next; a cache expires, which makes staleness self-correcting in a way a stored column is not. **3. Unbounded growth.** The strongest case is a cost that scales with something you do not control — counting child rows where one parent may have millions. A join to a small, indexed lookup table is cheap and stays cheap; duplicating it buys little. **4. Read-heavy ratio and an owner.** Reads must greatly outnumber writes, and I must be able to state what keeps the copy in sync, what staleness is acceptable, and what query detects drift. If any of the four is missing, I fix the index instead.

code

text · 8 lines
text
GroupAggregate  (rows=50)                actual time=812ms
  ->  Nested Loop Left Join              actual time=6ms..790ms
        ->  Index Scan on post           actual rows=50
        ->  Index Scan on comment        actual rows=1,910,442 total
              Index Cond: (post_id = post.id)

reading 1.9M child rows to produce 50 counts
-> no index makes this cheap; a stored counter makes it constant

go deeper

for a junior

Say you would measure first and try an index before duplicating data, and that redundancy has to be kept in sync.

for a middle

Show the reasoning from the plan: which operation dominates, why an index does or does not fix it, and how the read-to-write ratio drives the call.

for a senior

Give the decision procedure with thresholds and the cheaper alternatives, and commit to a sync mechanism, a staleness statement and a drift check.

for a principal

Treat it as adding a second source of truth with an owner and an exit condition; discuss recording the justifying measurement so the redundancy can later be removed.

## Start from evidence, not intuition The first question is whether the query matters. Rank by *total* time — calls multiplied by mean duration — rather than by worst single call. A 900 ms report run four times a day is not the problem; a 12 ms query run nine thousand times a minute may be. Then read the actual plan: it frequently shows the real cost is a sequential scan from a missing index, a filter applied after the join, or a predicate that defeats an index. Denormalizing on top of a fixable plan permanently bakes in a redundancy you never needed. ## Exhaust the cheaper corrections in order **Indexes.** An index on the join column turns a hash or merge join over a large table into a bounded lookup. A covering index — one that includes the columns the query returns — can answer it without touching the table at all, which is often the same win a duplicated column would give, with the engine maintaining consistency for you. Indexes cost write throughput and space, but never correctness. **Query shape.** Filtering earlier, avoiding a repeated correlated aggregate, or fetching a page of parents and then their children in one keyed query often removes more time than the schema change would. **Caching.** A result cache with a short expiry is redundancy with a built-in correction mechanism: it is wrong for at most its expiry window and then fixes itself. A denormalized column is wrong until someone notices. If the read tolerates seconds of staleness, a cache is strictly the safer instrument. ## What makes the case for redundancy strong **Cost that grows with unbounded data.** The clearest case is an aggregate over child rows whose count is uncontrolled: counting comments, summing ledger entries, computing a running balance. No index makes counting a million rows cheap, so the work is proportional to data you cannot bound, and a stored value converts it to constant time. Contrast a join to a lookup table of two hundred rows — that is a few microseconds after the first read, and duplicating its columns buys nothing while costing consistency. **A read-to-write ratio strongly favouring reads.** Redundancy moves work from read time to write time. If a value is read ten thousand times per write, that is an excellent trade. If it is read twice per write, you have made the system slower and less correct simultaneously. **A query on a path where latency is contractual.** A checkout step or a first-paint API call justifies more than an internal admin screen. **Sorting or filtering, not just projection.** Needing to order a large result by a value that lives in another table is much harder to fix with indexes than merely displaying that value — the engine cannot use an index on a column it must join to first. Sorting and pagination by a foreign attribute is one of the most legitimate reasons to copy a column. ## What makes the case weak The join is to a small table; the query is not actually hot; the plan is fixable; the duplicated value changes frequently, so you are paying constant synchronisation cost for it; the write path already contends on the parent row you would be updating; nobody can say who repairs drift. Also weak: "joins are slow" as a general belief. On indexed keys with reasonable row counts, relational engines join extremely well — the pathological cases are wide fan-outs, many-table joins that defeat the planner's estimates, and aggregation over unbounded children. ## Ship it with a contract If all four conditions hold, the change is not just a column. It is: the mechanism that maintains it (trigger, same-transaction write, or asynchronous job), the staleness the mechanism implies, the reconciliation query that detects drift, and the measurement that justified it recorded next to the schema. That last item is what lets a future engineer remove the redundancy when the workload changes — without it, every denormalization is permanent by default because nobody can prove it is still needed. ## Verify afterwards Measure the same query after the change, and also measure the write path you just made heavier. It is common to move a read from 40 ms to 2 ms while adding contention that costs more than it saved under peak concurrency. A denormalization that was never re-measured is a belief, not a result.

  • When is a covering index the better answer than duplicating a column?
    Whenever the query only needs to read a bounded set of columns and the cost is lookup or projection rather than aggregation over many rows. The index gives the same reduction in work while the engine keeps it consistent automatically, so there is no second source of truth and no reconciliation job. It costs write throughput and disk, never correctness.
  • How do you know later whether a denormalization is still earning its keep?
    Record the measurement that justified it alongside the schema, then re-measure periodically: is the query still hot, and does removing the stored value bring back the cost. Workloads shift, indexes improve and features die, so without that record the redundancy survives forever simply because nobody can prove it is unnecessary.

saying these in an interview costs you the question

  • Asserting that joins are inherently slow, rather than identifying which join in which plan is slow.
  • Denormalizing before checking the execution plan or trying an index.
  • Ignoring the read-to-write ratio, so the redundancy costs more on the write path than it saves on reads.
  • Duplicating columns from a small lookup table, where the join was already nearly free.
  • Never re-measuring after the change, or after the workload evolves.

context