skip to content

In an SCD Type 2 load, why does closing the old row and inserting the new one take two steps?

level: middleimportance: should knowfreq 50%

answer

  1. how many rows change for one source change?
  2. an upsert touches a matched row once
  3. order of the two writes is not free
  4. what if only one of them commits?

basics

~20 s

One changed source row implies two writes to the dimension: an update that end-dates the current version and an insert of a new open version. An upsert acts on a matched target row once, so the load runs both writes in one transaction or feeds the merge a doubled source.

solid answer

~50 s

A Type 2 change is two row operations against the same table for one incoming key — end-date and unflag the version that was current, then insert a fresh open version carrying the new attributes. An upsert statement matches a source row to a target row and does one thing to it; it cannot both update that row and insert a second one from the same match. So the standard load is two statements inside one transaction: update the matched current rows to closed, then insert the new versions from the same staging set. The alternative is to duplicate the source — one "close" row that matches the existing version and one "insert" row that deliberately does not — so a single merge covers both arms. Either way the transaction boundary is the point: a partial failure leaves the member with two open rows or none.

code

sql · 16 lines
sql
-- inside one transaction
-- 1) close the versions whose tracked attributes changed
update dim_customer
set effective_to = :batch_ts,
    is_current   = 0
where is_current = 1
  and customer_id in (select customer_id from stg_changed);

-- 2) insert the replacement open version
insert into dim_customer
    (customer_id, customer_name, city, tier, hashdiff,
     effective_from, effective_to, is_current)
select
     customer_id, customer_name, city, tier, hashdiff,
     :batch_ts, timestamp '9999-12-31 00:00:00', 1
from stg_changed;

go deeper

for a junior

Recall that one changed source row produces two dimension writes — end-date the old version, insert a new one — and that a plain insert-or-update on the business key would erase the history instead.

for a middle

Be ready to write both statements, explain why the close comes first, and name the three source populations the load must handle: new key, changed key, unchanged key.

for a senior

Expect to be pushed on atomicity and on the ambiguous fourth case: what the load does when a key disappears from the snapshot, and how you keep a partial failure from leaving two open rows.

for a principal

Judge which pattern the team should standardize on — two auditable statements versus one atomic doubled-source merge — and weigh readability and reviewability against the transaction guarantees your platform actually gives you.

## The shape of a Type 2 change When a tracked attribute changes for a dimension member, the dimension must end up with two rows for that member's business key: the previous version, now closed (its end date set to the change point and its current flag cleared), and a new version, open, carrying the new attribute values and the new surrogate key. That is two distinct row operations — an UPDATE and an INSERT — driven by one incoming source row. This is why a Type 2 load never reduces to a plain upsert. An upsert answers the question "does a row with this key already exist?" and then either updates it or inserts it. A Type 2 change needs *both* branches for the same key at the same time. ## The two-statement pattern The common, readable implementation stages the changed set first, then runs two writes over it: ```sql -- staged: one row per key whose tracked attributes changed create table stg_changed as select s.* from stg_customer_hashed s join dim_customer d on d.customer_id = s.customer_id and d.is_current = 1 where d.hashdiff <> s.hashdiff; ``` Then close, then insert. Order matters. If you insert first and close second, a close predicate written as "the current row for this key" now matches *two* rows and closes the version you just created. Writing the close first, and scoping it to rows that are currently open, avoids the ambiguity. If you must insert first, scope the close by surrogate key or by effective-from rather than by the current flag. Both statements must sit inside one transaction. Halfway through, the member has either two open rows (insert succeeded, close did not) or zero (close succeeded, insert did not). Every downstream query that filters on the current row then returns duplicates or drops the member entirely — and it does so silently, because nothing errors. ## The doubled-source pattern The single-statement variant makes the source carry both operations. For each changed key you emit two staging rows: one whose join key matches the existing open version (so the merge's matched arm closes it) and one whose join key deliberately matches nothing (so the not-matched arm inserts it). Typically the join is on the business key and the "insert" row nulls that key out, or the union carries an explicit action column. This is compact and runs as one atomic statement, which is its main attraction. The cost is readability: the trick is invisible to the next reader, and a mistake in how the two halves are keyed produces the same two-open-rows defect with no obvious cause. Many teams prefer the two-statement form precisely because a reviewer can see what each write does. ## New members and unchanged members A full load has three populations, and the pattern must handle all three: - **New key, no row in the dimension** — insert one open version, no close. This is the only case a plain upsert handles by itself. - **Existing key, tracked attributes changed** — close plus insert, as above. - **Existing key, nothing changed** — write nothing. Skipping this case is what makes the load rerun-safe; a load that inserts unconditionally piles up identical versions. A fourth population — a key that has disappeared from the source snapshot — is a policy decision rather than a mechanical one. A source snapshot that omits a member does not necessarily mean deletion (it may mean a filtered extract), so most loads either leave the version open, or close it and insert a version carrying a deleted flag, so that history still shows when the member stopped appearing. ## What interviewers are testing The question looks like statement trivia but is really about whether you have run a load in anger. The tells of experience are: naming the transaction boundary without being prompted; noticing that statement order interacts with the current-flag predicate; and separating the three populations explicitly rather than hoping one clever statement covers them. The tell of inexperience is describing the load as "upsert into the dimension", which produces a Type 1 overwrite and quietly destroys the history the design exists to keep.

  • What goes wrong if the insert runs before the close?
    A close predicate written as "the open row for this key" now matches two rows — the old version and the one just inserted — so the load closes its own new version and leaves the member with no current row. Either close first, or scope the close by surrogate key or effective-from rather than by the current flag.
  • How should the load treat a key that vanished from the source snapshot?
    Decide deliberately, because absence is ambiguous: a filtered or partial extract looks identical to a deletion. Common policies are to leave the version open, or to close it and insert a version flagged deleted so history records when the member stopped appearing. Never silently hard-delete the dimension rows — facts still point at those surrogate keys.
  • Why is a plain upsert on the business key wrong for a Type 2 dimension?
    It updates the existing row in place, which is a Type 1 overwrite: the prior attribute values are gone and no version boundary is recorded. Any fact already joined to that surrogate key silently starts reporting the new attributes for old events, which is exactly the restatement Type 2 exists to prevent.

saying these in an interview costs you the question

  • Describing the load as a plain upsert on the business key
  • Running the close and the insert outside one transaction
  • Inserting first, then closing by the current flag
  • Assuming one merge statement can update and insert one key
  • Hard-deleting dimension rows for keys missing from the source

context