Why tokenize client names before an LLM call instead of just deleting them?
answer
- the model still has to refer to someone
- one-way versus reversible
- same person, same surrogate, one scope
- the map never leaves your boundary
- restore into the screen, not the trace
basics
~20 sTokenization swaps each identifier for a stable surrogate such as CLIENT_0447, so the model can still track who did what and the application can restore the real name afterwards. Deleting the name destroys the references the answer depends on.
solid answer
~50 sBoth are pre-send redaction, but they buy different things. **Masking** is one-way: the identifier becomes `[REDACTED]` and is gone. That is right when the task never needs identity — classification, topic tagging, policy checks. Its cost is referential collapse: if three clients in a meeting note all become `[REDACTED]`, the model cannot say which one asked for the bond ladder, and the summary comes back useless or subtly wrong. **Tokenization** (pseudonymisation) replaces each distinct entity with a stable surrogate — the same person is `CLIENT_0447` everywhere in the request — so coreference, comparisons and structure survive. The map from surrogate to real value lives in a vault inside your trust boundary and never leaves it; the application detokenizes only when rendering the answer to an authorised human. The critical discipline is that detokenization happens at render time only. Traces, prompt caches, eval snapshots and feedback exports must keep the surrogate, or you have simply re-created the plaintext in three more stores.
code
python · 24 linesimport re
class NameVault:
def __init__(self):
self._to_token = {}
self._to_name = {}
def tokenize(self, text, names):
for name in names:
token = self._to_token.setdefault(name, f"CLIENT_{len(self._to_token):04d}")
self._to_name[token] = name
text = text.replace(name, token)
return text
def detokenize(self, text):
return re.sub(r"CLIENT_\d{4}",
lambda m: self._to_name.get(m.group(0), m.group(0)),
text)
vault = NameVault()
prompt = vault.tokenize("Ana Rey asked about Ana Rey's bond ladder.", ["Ana Rey"])
print(prompt) # CLIENT_0000 asked about CLIENT_0000's bond ladder.
# send `prompt` to the model; log `prompt` as-is; detokenize only for the user
print(vault.detokenize(prompt)) # Ana Rey asked about Ana Rey's bond ladder.go deeper
Know that text sent to a hosted model has left your systems, and that redacting before the call is the control. Be able to say plainly what a placeholder loses that a consistent code name keeps.
Explain masking versus tokenization and when each fits, why surrogates must be stable within a request, where the mapping vault lives, and why detokenization belongs only in the path back to the user.
Show you have operated this: how you measure detector recall on real traffic, what happens on low-confidence detections, and how you keep restored names out of traces, caches and eval sets.
Own the tradeoff between surrogate scope and utility: broader stability buys cross-document reasoning and simultaneously builds a durable linking key. Decide where that line sits per data class, and who is accountable for the vault.
## Why there is a boundary at all When an application sends text to a hosted model, that text leaves your infrastructure and is processed by a third party. Everything sensitive in it — names, account numbers, addresses, portfolio values, case references — has been disclosed at the moment of the call, regardless of what the provider promises afterwards. The engineering control is therefore to transform the text *before* the call so the sensitive parts never cross, and to restore them, if the task needs them, only on the way back out to an authorised person. There are two shapes of that transform, and interview answers turn on the difference between them. ## Masking: one-way, information-destroying Masking replaces the identifier with a placeholder that carries no information: "Client [REDACTED] asked about [REDACTED]." It is irreversible by construction, which is exactly what you want when the downstream task genuinely does not need identity — routing a message to a queue, scoring sentiment, deciding whether a note contains a complaint. Its cost is **referential collapse**. A private-bank meeting note that mentions three clients, two funds and a date becomes a text in which nothing can be told apart. Ask a model to summarise it and you get either an unusable answer or, worse, a confident one that merges the three clients into one. The failure is silent: nothing errors, the summary just quietly attributes the wrong position to the wrong person. ## Tokenization: reversible, structure-preserving Tokenization — pseudonymisation, in privacy vocabulary — replaces each distinct entity with a surrogate that is **stable within a scope**: every occurrence of one client in one request becomes `CLIENT_0447`, every account number becomes `ACCT_0012`. The model can now reason about relations ("CLIENT_0447 wants to reduce exposure that CLIENT_0448 wants to increase") without ever seeing a real name. A good surrogate scheme has three properties: - **Format hints.** A surrogate that looks like the type it replaced (`ACCT_0012`, `2026-XX-XX`) helps the model keep using it as that type. A raw UUID in the middle of a sentence often gets ignored or mangled. - **Stability at the right scope.** Consistency within the request is mandatory. Consistency *across* requests is a decision: it enables useful cross-document reasoning, but it also makes the surrogate itself a linking key — anyone with access to your traces can now correlate everything CLIENT_0447 ever did. Deterministic surrogates derived from the value (a keyed hash) are convenient for exactly that reason and risky for exactly that reason. - **A vault you own.** The surrogate-to-value map is the sensitive artefact now. It stays inside your boundary, with its own access control and retention, and it is never included in a prompt, a tool result or a trace. ## Detokenization is the part teams get wrong Restoring the real names into the *rendered* answer for an authorised user is the point of the whole design. Restoring them anywhere else undoes it. If your pipeline detokenizes before writing the trace, before caching the response, or before appending the example to an eval set, you have created plaintext copies in three systems that were never in scope for the original data-handling review — and those systems typically have longer retention and broader read access than the source database. The correct shape is: tokenize → call → store the tokenized request and response everywhere → detokenize only in the response path to the end user. ## Detection is the weakest link All of this assumes you found the identifiers. Structured fields are easy. Free text is not. Detectors are evaluated on precision and recall, and the two fail differently: a false positive mangles a legitimate word and degrades the answer, while a **false negative silently ships real data to a third party**. For this control, recall is the metric that matters, and it must be measured on a labelled corpus that looks like your traffic — including the languages, name forms and document layouts you actually receive. Detectors also miss what is not a named entity. "The widow who sold her stake in the family manufacturer last March" contains no name and identifies exactly one person. That is a **quasi-identifier**, and it is why removing names is pseudonymisation, not anonymisation: the data is still personal data, still in scope for your obligations, and re-identifiable by anyone with a little context. ## What to do when detection is uncertain Decide the policy in advance rather than at the call site: block the request, degrade to a masked (non-reversible) version, or route it to a lower-exposure path such as a regional or self-hosted model. Any of those is defensible; what is not defensible is a detector whose low-confidence output is silently treated as clean. ## What interviewers listen for That you distinguish reversible from irreversible redaction and can say when each is right; that you know the map is now the crown jewel; that you place detokenization at render time only; and that you do not claim the redacted text is anonymous.
- If the same surrogate is reused for one client across every request, what have you gained and what have you created?You gain cross-document reasoning: the model can connect this month's note to last month's. You create a persistent linking key. Anyone with read access to traces, evals or the cache can now assemble a full profile of that person under the surrogate, and if any single record ever leaks the mapping, the whole history de-anonymises at once. Scope the surrogate to the narrowest window the task actually needs.
- Your detector has 99% precision and 92% recall on free-text notes. Is that good enough to ship?Precision is the comfortable number and recall is the one that matters: 8% of identifiers are being shipped in plaintext. At any real volume that is a steady leak, not an edge case. Either raise recall on the classes you care about, add a second complementary detector, or make low-confidence documents fail closed — block, mask entirely, or route to a lower-exposure path — rather than passing them through.
- Does redaction let you say the text is anonymous and therefore out of scope?No. Removing direct identifiers gives you pseudonymised data, not anonymised data — you kept a map that reverses it, and quasi-identifiers such as an unusual role, a date and a location can single out a person with no name present. Treat redacted prompts as still-personal data with reduced exposure, and keep the same retention and access controls on them.
It is the difference between blacking out every name in a case file and swapping them for consistent code names. The blacked-out file is safe and unreadable; the code-named one can still be reasoned about, provided the code book stays locked in your own office.
saying these in an interview costs you the question
- Calls redacted text anonymous, so no handling rules apply
- Detokenizes before writing the trace, recreating plaintext downstream
- Uses one global surrogate map, making it a cross-customer linking key
- Assumes high precision on a detector implies high recall
- Thinks removing names removes re-identification risk