Context Engineering

Preloading spends the attention budget on material the model mostly will not use, and quality degrades as windows fill. Just-in-time loading carries lightweight identifiers — paths, IDs, queries — and pulls full content through a tool only when a step actually needs it.

Everything the model can use in a turn is text in its context window, and that window is rebuilt from scratch for each new session. A fact survives only if the application writes it to a store outside the window and puts it back in later.

Put the instruction after the document, or at both ends. Language models attend most reliably to the beginning and the end of a prompt, so an instruction stated once before a long body is the easiest part to miss.

A context window is shared by the prompt and everything the model generates. Fill it with input and the answer has nowhere to go: responses get truncated mid-sentence or the request is rejected. Reserve output space first.

Four things only the conversation knows: the goal with any standing constraints, the decisions already made and why, the artifacts still open mid-edit, and the approaches already ruled out. Anything readable from a file or a query should be a pointer, not a copy.

Progressive disclosure gives the model identifiers and thin metadata first — 300 table names with row counts — and full DDL only for the two or three tables a query actually joins. Selection happens over cheap descriptors; expensive content is expanded on request.

Split the file by how often its content is actually needed. Keep only always-true conventions in the always-loaded file, move task-specific procedures into on-demand skill files that load when the task matches, and delete anything stale outright rather than leaving it in place.

This is the lost-in-the-middle effect: retrieval accuracy traces a U-shaped curve against position. Models use information best when it appears first or last in the prompt and measurably worst when it is buried in the middle, even though every document is inside the window.

Grep when the query is an exact string — a symbol, an import, an error code — because lexical search returns every match and reads the current files. Embedding search earns its cost only when the wording is unknown.

Advertised length is an acceptance limit, not a quality guarantee. Retrieval accuracy and instruction-following degrade as the window fills — the effect commonly called context rot — so the usable ceiling is typically a fraction of the maximum, and it must be measured.

When the window is full of bulky tool output rather than long reasoning. Dropping old tool results is mechanical, costs no model call, and leaves the reasoning thread intact — compaction is for when the conversation itself, not its payloads, has grown long.

Cache reuse requires the request prefix to be identical to a previous one. Compaction rewrites the history near the front, so everything after the edit is new and the next turn is billed and computed as an uncached prefill.

Trigger on a high-water mark measured against the usable ceiling, not the advertised window, with headroom for the largest plausible next tool result. Then compact deep, to a low-water target, so the session cannot re-trigger within a few turns.

Keep the prompt append-only. Put stable material — system instructions, tool definitions, the catalog — at the front, never reorder or edit it, and append everything fetched mid-session at the end. Anything inserted above a cache boundary invalidates every cached token after it.

Count round trips, not dollars. Preloading costs one hop and a permanently degraded window; on-demand fetching costs a hop per hop and a cleaner window. Decide on hit rate, how serial the hops are, and the task's latency budget — then measure both.

Compare the candidate fact against what is already stored and choose one outcome: add it as new, update the existing entry in place, delete an entry it invalidates, or do nothing when it is already known. Blind appending leaves both versions alive.

Two levers exist: restate the template at the end of the prompt, or reorder so the instruction follows the document instead of preceding it. Restating costs a duplicate instruction per call; reordering is free but changes the prompt shape. Shrinking the document reduces the exposure behind both.

Yes. Position effects are mild when a prompt uses a small fraction of the window and grow sharply as utilization rises. As of mid-2026, reported behaviour is that past roughly half the window the U-shape flattens into a distance-based bias favouring the end, so early material loses its protection.

Chunk size times top-k is the token bill. Pick the smallest k at which end-to-end answer accuracy stops improving — not the k that maximises retrieval recall — and deduplicate near-identical passages before they crowd out the one that matters.

Late chunking encodes the entire document first so every token attends to the rest of it, then pools token embeddings per chunk. Each chunk vector therefore carries document context that independent chunk-by-chunk embedding throws away.

Hold one graded task set fixed and re-run it at several fill levels — for example 20%, 50% and 80% of the window — padding with realistic in-domain material. Plot score against utilization, then budget below where the curve breaks.

Needle-in-a-haystack plants one distinctive sentence and asks for it back, which is close to a keyword lookup. A perfect score says nothing about tracking many facts, reconciling contradictions, or reasoning across a long document — the things real workloads need.

Persist by exception rather than by default, consolidate asynchronously between sessions to merge duplicates and promote repeated patterns, and expire facts by class instead of one global lifetime. Growth is a policy problem: without one, every store eventually costs more than it returns.

Agentic chunking pays when documents lack reliable structural markers, the corpus is small and slow-changing, and retrieval failures trace to boundaries cutting through procedures. Otherwise its per-document model call, nondeterminism and re-embedding cost outweigh cheaper structural splitting.

Almost always because the listing does not discriminate. Entries like doc_final_v3.pdf give the model nothing to choose on, so it expands everything — paying a listing round trip on top of loading the whole corpus, which is worse than preloading.