Take a piece of sensitive data that enters at the edge of a distributed system and ends up encrypted in storage. Explain how you would reason about where it exists in cleartext along the way, and what design moves reduce the number of components that ever see the plaintext.

## The question behind the question Teams routinely say a value is "encrypted end to end" when what they mean is that each hop was encrypted and the database column is encrypted. Between those, at every hop, the value existed as plaintext in a process, and often in that process's buffers, queue payloads, retry stores and diagnostics. The design discipline is to make that set explicit and then make it small. The unit of reasoning is **plaintext locality**: for a given sensitive value, the set of components that can observe it in the clear, and for how long. Exposure grows with the size of that set because each member contributes its own memory, its own logs, its own crash artefacts, its own operators and its own dependencies. ## Step 1 — draw the path and mark the cleartext points Walk the value from where it is created to where it comes to rest, and mark every place it is readable: - **Connection termination points.** Wherever a transport-encrypted connection ends and is re-established, the payload exists in the clear in that component's memory. A gateway, a proxy or a service mesh sidecar that inspects or routes traffic sees it. - **Parsing and object mapping.** The parsed request object holds the value in whatever containers the framework chose, with lifetimes it controls. - **Asynchronous hand-offs.** A message placed on a queue is *durable* plaintext: it persists, it is replicated, it goes to a dead-letter store on failure, and it is readable by anyone with queue-inspection access. This is one of the most commonly missed cleartext locations because the code looks like a function call. - **Retry and outbox stores.** A retry buffer or transactional outbox holding the request payload keeps plaintext on disk with a lifetime governed by failure handling. - **Caches**, whose eviction policies were chosen for hit rate. - **Diagnostics.** Every component on the path can serialise the value into a log, an error report or a trace attribute — so each hop multiplies not just memory exposure but log exposure. - **Backups and replicas** of any of the durable items above. Counting these honestly is the deliverable; the count is usually two to five times what people guess. ## Step 2 — the moves that shrink the set **Substitute a reference for the value, as early as possible.** The highest-leverage move: at the first component capable of doing so, exchange the sensitive value for an opaque reference and let everything downstream carry the reference. A single custodian component holds the mapping and the plaintext; every other service handles a value that is worthless if disclosed. This is structural separation applied at architecture scale — downstream services do not *fail* to leak the secret, they have nothing to leak. The costs are honest ones: the custodian becomes a high-value target and an availability dependency, and any function that genuinely needs the real value must call it. **Encrypt at the field level for the intended consumer.** Where a reference will not work, encrypt the field so that only the consumer that must read it holds the key. Intermediate hops then transport ciphertext. This preserves the routing topology while removing every intermediary from the plaintext set — but only if the key is genuinely not available to them, which is the part that gets fudged when everyone shares one key. **Move the operation to the data, not the data to the operation.** If a component needs a decision derived from the value rather than the value itself — is this card valid, does this password match, is this person over eighteen — send the question to the custodian and get the answer back. The plaintext never leaves. **Keep plaintext off durable intermediates.** If a value must cross an asynchronous boundary, send a reference and let the consumer resolve it, so the queue, its replicas and its dead-letter store hold nothing sensitive. Where that is impossible, encrypt the payload and treat the queue's retention as a security parameter. **Collapse pass-through hops.** A service whose only role is forwarding still contributes memory, logs, crash artefacts and operators to the exposure set. Removing it is a security improvement as well as a latency one. **Minimise at the edge.** The cheapest reduction is not to collect the field, to collect a coarser version of it, or to discard it immediately after the single operation that needed it. Every downstream question about protecting it disappears. **Bound retention everywhere it does land.** Cleartext in a log, a queue or a cache is defined by its retention policy, and those policies are frequently defaults nobody chose. ## Step 3 — state and enforce the invariant A principal-level answer ends with an invariant a team can hold itself to, in this shape: > Plaintext for this value exists only in the edge component that receives it, in the custodian, and in the consumer that acts on it. Every other component receives a reference. This is enforced by the schema at the service boundary and by the fact that no other service holds the key or the resolution permission. Then name what would violate it and how you would notice: a new service added to the path that resolves the reference "for convenience", a debugging change that logs the resolved value, a queue payload that starts carrying the real field for performance, a cache introduced in front of the custodian. Enforcement lives in the contract (the boundary schema does not have a field for the plaintext), in the permission model (only the custodian may resolve), and in review of any change that adds a resolution call — with scanning of logs and queues as a backstop rather than a plan. ## The trade-offs to name out loud Centralising plaintext into one custodian concentrates risk and creates an availability dependency and a latency cost; that is a deliberate trade of *many weak boundaries for one strong one*, and it is only correct if the custodian is genuinely hardened, small, and monitored. Field-level encryption complicates search, indexing, analytics and support workflows. Reference substitution breaks naive joins and reporting. A principal answer states these costs and the conditions under which each is worth paying, rather than presenting the pattern as free. ## Answering the question Name plaintext locality as the unit of analysis, enumerate the cleartext points including the ones people miss (queues, retry stores, dead-letter topics, diagnostics at every hop), give the shrinking moves in order — do not collect it, substitute a reference early, encrypt for the intended consumer, move the operation to the data, collapse pass-through hops — and finish with a stated invariant, its enforcement mechanism, and the costs you accepted.