Your OPA decision logs are rate-limited and an auditor wants proof one request was evaluated last Tuesday — what can you show?

## The question behind the question An auditor is not asking about your logging pipeline. They are asking whether a control was in force on a given day, and they have been pointed at your decision log as the evidence. The interview is testing whether you understand what that stream can and cannot support — and the correct senior answer starts by narrowing the claim rather than by promising more than the data holds. ## Three ways an entry disappears **Rate limiting.** The plugin can be configured with `max_decisions_per_second`. Above that rate, decisions are discarded. This is not a queue — the events are gone. It is a perfectly sensible protection for a hot admission or authorization path, and it means your log is a sample whose selection criterion is "whatever arrived when the engine was not already busy logging". That is the worst possible sampling rule for evidence: it correlates with traffic bursts, which is exactly when interesting things happen. **A drop policy.** Sampling is often implemented deliberately, as a Rego rule at the drop decision path that suppresses events you have decided not to keep. Done thoughtlessly — drop nine in ten — it produces a stream that cannot answer for any specific decision. Done well it is the right tool, because it is *deterministic and explainable*: you can state which decisions are always kept. **Buffering.** Events for an HTTP service sink are held in memory, batched, and uploaded. The buffer is bounded; when it fills, events are discarded. And it is memory: if the OPA process restarts — a rollout, an OOM kill, a node drain — anything not yet uploaded is lost. OPA does not persist the buffer and cannot replay it. Watch OPA's own logs for warnings that events were dropped; that is your only signal that the stream has holes. ## Why absence proves nothing All three failure modes share a property that is easy to state and easy to forget: **the sink contains no marker where an event should have been**. A decision that was rate-limited, dropped by policy, evicted from a full buffer, or lost in a restart looks identical in the log to a decision that never happened. So the argument "there is no deny for that request, therefore it was allowed" and the argument "there is no entry, therefore the request never reached the policy" are both unsound. Making that claim to an auditor and being wrong is worse than saying up front that the stream is sampled. The converse still holds, and it is worth stating explicitly, because it is what you *can* offer: an entry that is present is real. It names a decision id, an input, a result, a timestamp and a bundle revision. Those entries support statements about what the policy did on the requests they cover, and the bundle revisions in them support a statement about which policy version was in force across the window. ## What you say on the day Separate what the data supports from what you are being asked to assert. 1. Show the entries you have for that window, and say plainly that the stream is sampled and by what mechanism. 2. Show which bundle revisions appear across that window, and the policy source at those revisions — that supports "this rule was loaded and evaluating" independently of whether the one request of interest survived sampling. 3. Do not reconstruct the missing entry from another source and present it as a decision log record. If it did not come from the engine, it is not what the engine decided. ## What you change afterwards The design mistake was treating log volume as a uniform problem with a uniform lever. Replace it with a policy-shaped one: - **Sample in the drop policy, not with a rate limit.** Keep every deny. Keep every decision on the policy paths a control depends on. Shed the routine high-volume allows on paths nobody will ever ask about. The result is a stream where you can state precisely what is guaranteed present. - **Size the buffer for the burst, not the average**, since eviction hits exactly when the system is busiest. - **Alarm on discards.** A silent hole is the whole problem; OPA reporting that it discarded events is the one chance you have to know. - **Decide, in advance and in writing, what the stream is allowed to be used for.** If someone will need per-decision assurance, sampling the paths that carry it is a decision to say no to. A candidate who answers this well does two things: refuses to overclaim, and converts a volume knob into a policy that expresses which decisions matter.