skip to content

To protect an email column that analysts use as a join key, when would you tokenize it, encrypt it, or replace it with a keyed hash?

level: middleimportance: must knowfreq 55%

answer

  1. joins need the same output every time
  2. reversible or not
  3. where the secret lives
  4. plain hash is brute-forceable
  5. result is pseudonymous, not anonymous

basics

~20 s

Use a keyed hash when analysts only need a stable join key and nobody needs the email back. Tokenize when an authorised process must recover it. Encrypt when the value itself must be recoverable by key holders. None of the three makes it anonymous.

solid answer

~50 s

All three replace the email with a surrogate, and the choice turns on **joins, reversibility and where the secret lives**. A **keyed hash** (an HMAC with a secret key) gives the same output for the same email, so joins still work, and without the key nobody can recompute it — a **plain** hash is not enough, because anyone can hash a list of known emails and match. **Tokenization** swaps the value for a random token and stores the mapping in a protected vault; deterministic tokens keep joins, and an authorised service can **detokenize** when the real value is needed, such as for sending a message. **Encryption** is reversible by anyone with the key; randomised encryption breaks joins, and deterministic encryption preserves equality but reveals which rows share a value. In every case the result is **pseudonymous**: whoever holds the key or vault can link it back, so it is still personal data.

code

pseudocode · 7 lines
pseudocode
# join key for analytics: stable, not reversible without the key
key = secrets_store.get("analytics-email-hmac-key")   # never stored in the warehouse
normalised = lower(trim(email))
email_key = hex(HMAC_SHA256(key, normalised))

# plain hash - DO NOT USE: anyone can hash a list of emails and match
weak_key = hex(SHA256(normalised))

go deeper

for a junior

Know that a plain hash of an email can be reversed by hashing guesses, and that a keyed hash or token avoids that.

for a middle

Explain how keyed hashing, tokenization and encryption differ on joins, reversibility and key location, and choose one for a scenario.

for a senior

Plan key custody, rotation and detokenization rights, and explain why the output is still pseudonymous personal data.

for a principal

Set the organisation's standard for join keys across platforms, including who holds keys and how rotation is handled without breaking history.

## The requirement Analysts join customer activity across tables on the email address, but they do not need to *see* email addresses. The goal is a column that **still joins** while exposing as little as possible — and the right tool depends on who, if anyone, must get the real value back. ## The three techniques | Technique | How it works | Joins still work? | Reversible? | Where the secret lives | |---|---|---|---|---| | **Keyed hash** | a hash computed with a secret key (for example HMAC-SHA-256) | yes — same input, same output | no, except by guessing inputs *with* the key | the key, held outside the analytics platform | | **Tokenization** | replace the value with a random token, store value↔token in a vault | yes, if tokens are deterministic per value | yes, by the vault for authorised callers | the vault and its mapping | | **Encryption** | encrypt the value with a key | only with deterministic encryption | yes, by any key holder | the encryption key | | **Plain hash** | a public hash function with no key | yes | effectively yes, by brute force over likely values | nowhere — that is the problem | ## Why a plain hash fails A hash function is public and deterministic. Emails, phone numbers and IDs come from a space an attacker can enumerate, so hashing a candidate list and matching recovers the originals. NIST IR 8053 describes a public taxi dataset whose medallion numbers were published as unsalted MD5 hashes; because there were few possible medallion numbers, they were all recovered by hashing every candidate. NIST SP 800-188 therefore lists **hashing with a keyed hash** — for example an HMAC with a 256-bit randomly generated key — as a pseudonymisation technique, and notes that hashing without a key generally confers no security. ## Choosing 1. **Analytics only needs a stable key, and nobody downstream needs the email** → keyed hash. Keep the key in a secrets store the analytics platform cannot read. 2. **An operational process must recover the email** (sending a notice, answering an access request) → tokenization, with detokenization allowed only to that service and logged. 3. **The value must travel protected and be read back by specific key holders** → encryption, accepting that deterministic encryption leaks equality and frequency, and randomised encryption breaks joins. ## What none of them does - **They do not anonymise.** Anyone holding the key, the vault or the encryption key can link records back to people; the output is **pseudonymous data**, and privacy law generally still treats it as personal data. - **They do not protect quasi-identifiers.** A keyed-hash email next to date of birth and postcode can still single people out. - **Key custody is the control.** If the key leaks, every protected value is exposed at once; rotation changes every surrogate and breaks historical joins, so plan it. ## Why interviewers ask it It tests whether a candidate understands **join-preserving pseudonymisation** and its limits. Strong answers reject the plain hash, tie the choice to reversibility and key custody, and say clearly that the result is still personal data.

  • Why normalise the email before hashing it?
    Because the same person's email may arrive as "[email protected]" and "[email protected] ". Without trimming and lower-casing first, the two produce different surrogates and the join silently misses matches.
  • What happens to historical joins when you rotate the hashing key?
    Every surrogate changes, so old and new data stop joining. Teams either re-hash history with the new key in one migration, or keep the old key available for a transition period and map old surrogates to new ones.
  • Does replacing emails with tokens reduce who can see personal data?
    Yes for everyone without detokenization rights, which is the point. The vault and the services allowed to detokenize become the sensitive core, so their access must be narrow and logged.

A token is like a coat-check ticket. It is useless to a stranger, but the cloakroom that issued it, the vault, can always hand the coat back.

saying these in an interview costs you the question

  • Using an unkeyed hash of the email and calling it anonymised
  • Storing the hashing key in the same warehouse as the hashed data
  • Choosing randomised encryption for a column that must be joined
  • Assuming tokenised data is no longer personal data
  • Hashing without normalising the value first