skip to content

When the real record for an inferred dimension member arrives, why overwrite it instead of adding a Type 2 version?

level: middleimportance: should knowfreq 38%

answer

  1. did the world change, or did we?
  2. the placeholder was never true of anyone
  3. facts already point at that surrogate key
  4. a correction is not a change
  5. clear the flag, keep the key

basics

~20 s

Because the placeholder attributes were never true. A Type 2 version would record a fake history in which the customer really was named "Unknown". Overwrite in place, keeping the surrogate key so existing facts stay attached.

solid answer

~50 s

Type 2 versioning exists to record that **reality changed** at a point in time. A placeholder repair records that **we were ignorant**, not that the world changed. If you end-date the inferred row and insert a new version, you have asserted that the customer was genuinely called "Unknown" and genuinely in segment "Unknown" until the day your master feed happened to run — a history that never happened, and one that reports will faithfully show. So repair it as an in-place overwrite: update the descriptive attributes, clear the inferred flag, leave `effective_from` at its original low sentinel so the whole window is covered, and — critically — do not touch the surrogate key. Fact rows loaded during the gap already carry it, and they become correct the moment the attributes are fixed, with no restatement at all.

code

text · 13 lines
text
inferred row created 2024-05-01, natural key 88123
sk    customer_name  segment  is_inferred  effective_from  is_current
9041  Unknown        Unknown  true         1900-01-01      true

master record arrives 2024-05-04: "Acme Foods", segment Retail

WRONG - Type 2 split, inventing a history of being called "Unknown"
9041  Unknown        Unknown  false        1900-01-01      false
9612  Acme Foods     Retail   false        2024-05-04      true
   -> facts loaded 2-3 May stay attributed to "Unknown" forever

RIGHT - overwrite in place, same sk, facts stay attached
9041  Acme Foods     Retail   false        1900-01-01      true

go deeper

for a junior

Recall that placeholder attributes are a stand-in for missing knowledge, not real history, and that fixing them means updating the existing row rather than adding another one.

for a middle

Be able to state the correction-versus-change distinction cleanly and explain why keeping the same surrogate key means no fact rows need restating.

for a senior

Show that you would keep an audit marker on the repaired row and handle the mixed case where the arriving record carries both the true original values and a later genuine change.

for a principal

Set the rule for the whole platform: what counts as a correction versus a business change, who may issue in-place overwrites to a published dimension, and how those repairs are disclosed to consumers.

## Corrections versus changes The whole question turns on one distinction that dimensional modelling makes constantly and that weak candidates blur: - A **change** is the world becoming different. The customer moved from Berlin to Hamburg on 1 April. Both values were true, each during its own window. Type 2 exists precisely to keep both. - A **correction** is the warehouse becoming less wrong. The customer was always called "Acme Foods"; we simply did not know it yet, so we wrote "Unknown" as a stand-in. Only one value was ever true. An inferred member is a correction case by construction. Its placeholder attributes were never facts about the customer — they were an admission of ignorance, deliberately chosen to look like one. Versioning them promotes an admission of ignorance into recorded history. ## What the wrong answer produces Suppose the placeholder was created on 1 May and the master record arrives on 4 May. Split it Type 2 and the dimension now says: this customer was named "Unknown", in segment "Unknown", from the beginning of time until 3 May, and became "Acme Foods" in segment "Retail" on 4 May. Every consequence of that is wrong: - A segment-mix report for the first days of May shows an "Unknown" segment that has no business meaning. - Any "how many customers changed segment this month?" metric counts a change that did not happen. - Facts loaded on 2 and 3 May stay pointed at the old surrogate key and therefore stay attributed to "Unknown" forever, unless you *also* restate them — work you created for yourself out of nothing. ## What the right answer does Overwrite the attributes on the existing row: ```sql UPDATE dim_customer SET customer_name = 'Acme Foods', segment = 'Retail', is_inferred = FALSE WHERE customer_id = 88123 AND is_inferred = TRUE; ``` The surrogate key does not move. Facts already loaded against it are now attributed to the correct customer name and segment without a single fact row being touched. This is exactly why the inferred member was given the natural key in the first place — the natural key is the matching handle, and the stable surrogate key is the payoff. ## Keep the effective window intact If the dimension carries Type 2 history, the inferred row was created with `effective_from` at a low sentinel (`1900-01-01` or similar) so that facts with older event dates still fell inside its window. Leave that alone during the repair. Moving `effective_from` forward to the repair date would push any earlier fact outside every version's window, and the point-in-time key assignment would then find nothing. ## The one case where you do both Sometimes the arriving master record contains not only the true original attributes but also a genuine change that occurred after the placeholder was created. Then you perform two distinct operations in order: 1. **Repair** the inferred row to the values that were true during the window it already covers. 2. **Version** the later change normally, end-dating the repaired row at the change date and inserting a new version from it. The result is two rows that are distinguishable and both truthful, rather than one blended row that conflates "we learned something" with "something happened". ## Do not reissue the surrogate key A recurring bad instinct is to delete the placeholder and reload the dimension from the master, which mints a fresh surrogate key. Every fact loaded during the gap is then orphaned — pointing at a key that no longer exists, in a warehouse where nothing enforces referential integrity, so nothing complains. The join silently drops those facts. The surrogate key is meaningless by design; there is no reason for it to change just because the attributes did, and every reason for it not to. ## Leave an audit trail After a clean repair, the row is indistinguishable from one that was always correct — which is convenient for reporting and inconvenient for auditing. Keep the marker: repurpose the inferred flag into a resolved marker, or add a repair timestamp column. When somebody asks why last week's segment report changed between two runs, the answer is either in that column or it is nowhere. ## Contrast with a genuine Type 1 overwrite Mechanically this is a Type 1 overwrite, and it is worth saying so explicitly in an interview: the repair uses Type 1 semantics *inside* a Type 2 dimension, which is not a contradiction. Type 2 governs how business changes are recorded; corrections to data the warehouse never knew are a separate category and are overwritten. Being able to say that cleanly is the signal an interviewer is listening for.

  • What if the source record also contains a genuine change that happened after the placeholder was created?
    Then you do both, in order. First repair the inferred row to the values that were true during the window it already covers, then apply the later change as a normal Type 2 version end-dated from its own effective date. The correction and the change are separate events and should leave two distinguishable rows, not one blended row.
  • Should the surrogate key of an inferred member ever be reissued when the real record arrives?
    No. Facts loaded during the placeholder period already carry it, and reissuing orphans every one of them in a warehouse where nothing enforces referential integrity — the join simply drops them. The surrogate key is meaningless by design; it does not have to change because the attributes did.
  • How would an auditor tell a repaired placeholder from a value that was always correct?
    Only if you keep the trail. Retain the inferred flag as a resolved marker or add a repair timestamp alongside the row's load metadata. Without it, a repaired row looks exactly like one that was never wrong, and "why did last week's segment report change?" has no answer at all.

saying these in an interview costs you the question

  • End-date the placeholder and insert a new version
  • Delete the placeholder and reload the dimension fresh
  • Assign a new surrogate key to the repaired member
  • Treat every attribute write as a Type 2 change
  • Move effective_from forward to the repair date

context