In a Data Vault, why are hub and link keys hashed business keys?

## The problem hash keys solve Data Vault is designed to be loaded by many independent pipelines, fed by many source systems, on different schedules. The obstacle to that is key resolution. If a hub assigns its warehouse key from a sequence, then nothing that references the hub can be loaded until the hub row exists and its key is known. Every link and satellite load must first join back to the hub to translate a business key into a warehouse key. The pipeline becomes a strict order — hubs, then links, then satellites — with a lookup at every step and a single point of contention at the sequence. A hash key breaks the dependency by making the key a **deterministic function of the business key**: ``` customer_hk = hash( upper(trim(customer_id)) ) ``` Now any process can compute the parent key for any row without consulting the warehouse at all. The satellite load and the link load do not wait for the hub load, do not join to it, and do not care whether it has run yet. Loads run in parallel and out of order, and any of them can be restarted independently. That property — not storage efficiency or index behaviour — is why Data Vault reaches for hashing. ## Determinism across systems and time The same property has a second payoff. Because the key is a pure function of the business key, it is identical in every environment and on every platform. A key computed in a staging job on one system matches the key computed by a different job, in a different tool, next year. That makes it possible to distribute the vault across platforms, to rebuild a table from source without breaking references from other tables, and to compare environments row by row. A sequence-assigned key has none of these properties: rebuild the hub and every downstream reference is wrong. ## Composite keys and the standardization contract The hash is only as good as the string fed into it, and this is where implementations actually fail. Every pipeline must apply an identical standardization rule before hashing: ``` hub key: hash( upper(trim(customer_id)) ) link key: hash( upper(trim(order_id)) || '||' || upper(trim(customer_id)) ) hashdiff: hash( coalesce(name,'') || '||' || coalesce(tier,'') ) ``` The rules that must be written down and enforced everywhere are: trimming and casing, the exact delimiter used between the parts of a composite or link key, the ordering of those parts, and the token substituted for a null or empty business key. If one pipeline trims and another does not, `C-42` and `C-42 ` hash differently, the vault silently holds two hubs for one customer, and history splits across them. The failure is silent because nothing is violated — both hubs are perfectly valid rows. Detecting it later means comparing business keys, not hash keys. The delimiter also has to be a character that cannot occur inside a key value, or two different key combinations can concatenate to the same string and legitimately collide. ## Hashdiff — the same trick for change detection Satellites apply the idea to attributes rather than keys. A `hashdiff` column holds a hash over the concatenated descriptive columns, so deciding whether an incoming record differs from the stored one is a single comparison rather than a column-by-column check across forty nullable columns. It is faster and, more importantly, it cannot be broken by someone adding a column and forgetting to extend the comparison. It carries the same standardization hazard: null and empty string must be represented consistently, or unchanged rows will look changed and the satellite will fill with spurious versions. ## What hashing costs **Width.** A hash key is far wider than an integer — sixteen or thirty-two bytes rather than four or eight. Every link and satellite carries copies of it, so storage grows and joins move more bytes. On a large vault this is a real, measurable cost rather than a theoretical one. **Opacity.** Hash values are unreadable, so debugging by eye is painful and every investigation starts by joining back to a hub to recover the business key. **Collisions.** Two different business keys hashing to the same value would silently merge two entities. With a wide digest and realistic row counts the probability is negligible, but it is not zero, and shops that care choose a wider digest rather than the shortest one available. A collision here is a data-integrity failure, not a performance problem, which is why the choice of digest width is worth a deliberate decision rather than a default. **Discipline debt.** The standardization contract must survive staff turnover and new pipelines written by teams who never read the original convention. That is a governance cost, and it is the one that actually bites. ## When the trade-off flips If the whole vault is loaded by a single orchestrated pipeline into one platform, with modest volumes and no cross-platform ambition, the parallel-loading benefit is small and the width cost is real; a sequence-assigned key with lookup joins is defensible. The larger and more distributed the loading landscape, the more decisively hashing wins.