An incident agent's log-search tool returns 40 MB of JSON. Where do you reduce it?

## Why 40 MB is not a formatting problem A raw six-hour log dump is roughly ten million tokens; the window it must fit in is a fraction of that. But even a result that technically fits is expensive in a way that a single-shot call is not. Tool results persist: whatever lands in the conversation is re-sent on every subsequent turn until something removes it. A 60,000-token result in an agent loop that runs thirty more turns is charged thirty more times, adds latency to every one of them, and dilutes the model's attention across a mass of low-signal text — the failure mode usually called context rot, where recall of the material that matters drops as irrelevant volume around it grows. So the real question is never "how do I fit this?" but "what is the smallest set of facts that lets the agent make its next decision?" For an incident agent, that is almost never the events. It is the distinct failure signatures, their counts, when each first and last appeared, and which service emitted them. ## Reduction point 1: at write time, inside the tool The cheapest reduction is the one that happens before the result is ever serialized. The wrapper runs the query, aggregates in code, and returns something like twelve fingerprints with counts and timestamps — a few hundred tokens carrying nearly all the decision-relevant signal. This is exact, deterministic, free of model tokens, and testable like any other code. Its limit is that you must know the reduction in advance. A fixed aggregation is perfect when the question is always "which errors spiked", and wrong when the agent needs a field you did not think to keep. Design against that by making the aggregation parameterized — group-by, filter, limit as tool arguments — so the model can ask for a different projection instead of asking for everything. Beware of doing this reduction with a model call. Summarizing at write time with an LLM costs tokens and latency on every invocation and can drop the one line that mattered. Reserve it for genuinely unstructured payloads where code cannot do the job. ## Reduction point 2: inside a code-execution sandbox The stronger version gives the agent a code environment and lets it write the reduction itself. The agent emits a short program that calls the log query, and the 40 MB materializes inside the sandbox process, not in the transcript. The program groups by fingerprint, counts, sorts, and prints the twelve lines it decided were interesting. Only that printed output crosses back into context. This keeps the flexibility of raw access — the agent can filter on any field, join two sources, compute a rate — while paying context only for the code and its output. Reported reductions of one to two orders of magnitude on large tool payloads are routine, because the ratio of raw data to decision-relevant facts in observability and analytics work is enormous. It also composes: intermediate results stay as sandbox variables or files across steps rather than being re-narrated each turn. The costs are real. You need a sandbox with a security boundary, a runtime and a lifecycle; the agent's code can itself be buggy and needs an error path back to the model; and there is a latency floor per execution. For an agent whose whole job is data-heavy investigation, it usually pays anyway. ## Reduction point 3: editing history after the fact The last lever operates on results already in the window: drop or replace tool results older than some age or beyond some count, keeping the assistant's reasoning and the extracted conclusions. Some providers expose this automatically, clearing older tool results once the window pressure crosses a threshold; you can equally implement it yourself when you own the message list. Treat it as damage control rather than design. It cannot refund the tokens already spent on the turns where the result was resident, and it is lossy in a way you may not notice: if the agent had not yet extracted the fingerprint from a cleared result, that information is simply gone and the agent may re-run the same expensive query. The discipline that makes it safe is to have the agent write its conclusion somewhere durable — a scratchpad file, a structured note — before the source result is eligible for clearing. ## Return handles, not payloads Cutting across all three: the tool should hand back a reference alongside the summary — the query id, the object-store path, the row count — so the raw data remains reachable through a follow-up call without being resident. This is the just-in-time posture: the context holds pointers and the agent dereferences on demand. It is what lets you be aggressive about reduction without ever making the underlying data unreachable. ## Choosing Aggregate at write time when the useful projection is predictable. Give the agent a sandbox when it is not, and the payload is large and structured. Clear history when a long run has accumulated results whose value has already been extracted. Whatever you choose, cap the result: a hard byte or token limit per tool result with explicit truncation wording is the backstop that keeps one unlucky query from destroying a run.