How would you summarize a 3-hour video into chapters when it exceeds the context window?

## Why hierarchy rather than a bigger window Even with a million-token context, three hours of footage at any useful frame rate does not fit, and the fraction that does fit is expensive on every call. Hierarchical processing trades one impossible request for many affordable ones, and the structure it imposes turns out to be useful in its own right, because chapters are themselves a hierarchy. ## The map pass Split the footage into windows. Window length is a real parameter: too short and each summary lacks the context to know what matters, too long and you are back at the original problem. Several minutes is a common landing zone for continuous footage, and for structured content the natural units, a half of a match, a scene, an agenda item, beat the clock. Overlap the windows. An event that begins at 41:58 and ends at 42:06 is mangled by a hard cut at 42:00, and the reduce pass has no way to know it was mangled. Thirty seconds of overlap on each side costs a few per cent of the budget and removes an entire failure class, at the price of duplicate observations that the reduce pass must merge. What each window emits matters more than most teams expect. If the map pass returns a paragraph of prose, the reduce pass receives three hours compressed into fluent text that has already discarded the specifics. A structured emission survives much better: timestamped observations, the entities involved, a salience or confidence score, and a flag for anything that appears unresolved at the window boundary. The reduce pass can then rank and group rather than re-write. ## The reduce pass The reduce pass sees only text. This is the defining constraint and the source of the main failure mode: it cannot verify anything, cannot recover detail the map pass dropped, and will happily produce a confident chapter list built on a map-pass hallucination. Error compounds one way only. It is also where cross-window structure is created. Arcs that develop slowly, a mood shifting across an hour, a score changing across a match, a machine degrading across a shift, exist nowhere in any single window summary and must be assembled here. That only works if the map pass emitted comparable, structured signals rather than independent prose. When the summaries themselves do not fit, reduce in more than one level: summarize groups of windows, then summarize those. Each additional level is another lossy compression, so stop at the shallowest hierarchy that fits. ## The refine pass For most real products chapters are an index, not the deliverable. A highlights reel needs the actual moments, so a third pass takes the intervals the reduce pass chose and re-analyses only those at a much higher frame rate, where precise boundaries and fine detail matter and the token cost is affordable because the intervals are short. This coarse-then-fine structure is what makes the whole design economic: cheap sampling everywhere, expensive sampling only where the answer lives. ## Cost, latency and evaluation Cost is roughly the whole recording sampled once at the coarse rate, plus a small reduce, plus the refined intervals. That is dramatically less than a single dense pass and is the number to quote when someone asks why not just use a longer context. Latency is not the sum of the windows, because the map pass is embarrassingly parallel; wall-clock time is one window plus the reduce, subject to rate limits, which is usually the binding constraint in practice. Evaluation is the part teams skip. Chapter quality has no single metric, so pick two: agreement with human chapter boundaries within a tolerance, and a check that every claim in a chapter summary is grounded in a real timestamp. The second catches compounded map-pass errors, which are otherwise invisible because the output reads beautifully. ## The honest uncertainty There is no settled best architecture for long-video understanding as of 2026. Hierarchical map-reduce is the reliable, boring, well-understood option. Research alternatives, memory-style architectures that carry state forward across windows and token-compression schemes that squeeze many frames into few tokens, are actively moving and can beat it on benchmarks, but they are harder to reason about when they fail. A principal-level answer says which one it is choosing and why, names the failure it is most worried about, and describes what would change its mind.