Keyed hashing costs throughput on a hot path — how do you set fleet-wide policy against hash flooding?

## The decision, not the mechanism Every reasonable engineer already knows the menu: unpredictable keyed hashing, caps on key count, cost-bounded buckets. The principal-level question is which of them you make mandatory for a hundred-odd services maintained by a few dozen teams, given a latency budget, a migration cost, and a finite amount of attention. ## Step 1: measure, because the premise is usually softer than it feels "Keyed hashing is expensive" is an intuition formed from cryptographic hashing of large payloads. Table keys are short — tens of bytes — and modern keyed functions designed for short inputs run at a few cycles per byte. Benchmark on your real key distribution and on the real request path, and express the answer as a share of end-to-end request cost rather than as a microbenchmark ratio. Frequently the honest number is a fraction of a percent of a request that also parses text, touches a socket, and waits on a database. That number, not a debate, should decide the policy. Where the cost is genuinely material is a narrow class: extremely hot in-memory maps in tight loops, keyed by short internal identifiers, called millions of times per second. Those exist. They are also usually the maps that never see untrusted text — which is why the exemption criterion should be *provenance of the keys*, not *hotness*. ## Step 2: place the default where forgetting is safe A policy that requires every team to remember something will be violated by the newest service on its busiest week. So the default belongs in the layers everyone already uses without thinking: the request parsing layer, the document decoder, the shared container defaults. A team should have to take deliberate action to become vulnerable, not deliberate action to become safe. This is also the answer to the audit-cost problem. Auditing "which maps see untrusted keys" is not a one-time exercise; it is a question re-opened by every refactor that widens what flows into a cache key. A secure default converts a recurring audit into a one-time platform decision. ## Step 3: keep an orthogonal layer that does not depend on the hash Cap the number of fields a request may produce, cap body size, and give requests a CPU budget that a runaway handler cannot exceed. These bound the quadratic term directly and, crucially, they fail *independently* of the hash decision. If a hash key leaks, is accidentally shared fleet-wide, or is derived from something predictable, the caps still hold. Different failure modes are the whole justification for depth here, and they are why the answer to "which one do we do" should usually be "the cheap one and the strong one". ## Step 4: exemptions with evidence, and an expiry A workable exemption process: a service may opt a specific map out of unpredictable hashing if it documents (a) a measurement showing material end-to-end cost, and (b) an argument that only self-minted identifiers reach that map. Exemptions carry a review date, because the second half rots — the map that only held internal identifiers acquires a client-supplied cache dimension two quarters later. ## Step 5: sequence the migration around what depends on stable hashes The change that surprises people is not throughput; it is that per-process keys make hash values non-reproducible. Anything that assumed stability breaks: iteration order varies run to run, and, more seriously, hash values written to storage, used to select a shard, or exchanged between processes are no longer meaningful. The correct structure is two functions with two jobs — a fixed, documented function for persistence and partitioning, and an unpredictable per-process one for in-memory tables. Doing the separation first, and the key randomization second, is the difference between a quiet rollout and a data-placement incident. ## Step 6: prefer boring mechanisms your team can keep A bespoke cost-bounded container written in-house is a legitimate engineering answer and a poor organizational one if a single engineer understands it. Weigh the ongoing cost of the clever structure against the cost of the attack it prevents and against the simpler layers available. The judgment interviewers are listening for is not "which is fastest" but "which set of defenses will still be correct after two years of staff turnover". ## How to say it in an interview Name the tradeoff explicitly: an availability risk with an asymmetric cost curve, mitigated by measures whose price is small but non-zero and spread across every team. Decide by measurement, default to safe in shared code, keep one hash-independent layer, allow narrow evidence-backed exemptions, and sequence the rollout behind the separation of persisted from in-memory hashing.