Why do video models get event ordering and duration wrong, and how do you mitigate it?

## The information that sampling destroys A video is continuous; a sample is a set of snapshots. Everything that lived strictly between two snapshots is gone, and a great deal of what humans call temporal understanding lives exactly there. Direction of travel, the moment of contact, which of two things moved first, whether a hand pushed or caught, whether a person entered or left: at one frame per second, all of these are inferences drawn from two static endpoints rather than observations. The model is very good at those inferences, which is the trap. It fills the gap with the most probable continuation given its training, and it reports the result in the same confident register as something it actually saw. ## Three failures worth naming separately **Ordering.** Given a frame with a pallet upright and a later frame with a pallet on the floor and a forklift nearby, the likely-cause prior says the forklift knocked it over. If the truth is that the load shifted and fell, and the forklift arrived afterwards, nothing in the sampled frames contradicts the wrong story. Ordering errors cluster exactly where a plausible causal narrative competes with the actual sequence. **Duration.** Frames have no inherent clock. If they arrive as an image list, the model has only their count and order, so it estimates duration from frame count. Under adaptive sampling this inverts reality: a three-second incident sampled densely might contribute thirty frames while a quiet hour contributes twelve, and the model will describe the incident as the long part of the recording. **Events between samples.** The simplest failure and the hardest to detect. If the thing happened entirely between two frames, the model is not wrong about what it saw; it answers about footage that never included the event, and it has no way to signal the absence. ## Mitigations, in the order you should reach for them **Give it the transition.** Where ordering matters, resample that interval at a much higher rate and re-ask. Judgments about causality need frames close enough together that the intermediate states are visible, which usually means several frames per second, not one. This is the only mitigation that adds real information; the rest manage the model's use of what it has. **Make time explicit.** State each frame's offset in the prompt, or overlay it into the pixels. Once the model can read the clock it stops equating frame count with elapsed time, and its duration estimates become anchored rather than invented. Burned-in overlays survive prompt manipulation that text labels may not. **Force per-frame grounding.** Ask for observations tied to specific frames or timestamps before any conclusion: what is visible at 00:14:02, what is visible at 00:14:03, and only then what happened. This separates observation from inference and makes the inference auditable. When the model cannot point to a frame, the claim is a guess, and that is now visible to you rather than buried in fluent prose. **Ask for uncertainty explicitly.** Instruct that if the sequence cannot be determined from the frames provided, it should say so and name what additional interval would settle it. Well-tuned models comply with this, and it converts a silent failure into a request for more footage. **Verify with a second pass.** For high-stakes ordering, re-ask on the resampled interval, or ask the reverse question, and treat disagreement as a signal to escalate to a human rather than to average the two answers. ## What no mitigation fixes Prompting does not recover information that was never sampled. If the design cannot afford dense frames anywhere, then fine-grained ordering is out of scope for that system, and the honest engineering answer is to say so and to route those questions to a human or to a purpose-built detector rather than to accept a confident guess. Similarly, precise measurement, how many seconds elapsed, how fast something moved, is a poor fit for this class of model even under good sampling; if the number matters, compute it from the frame indices and the known frame rate, and use the model only to identify which frames bound the event. ## Judgment an interviewer is testing The weak answer is that the model is bad at temporal reasoning. The strong answer explains which specific information sampling removed, distinguishes ordering from duration from missed events, proposes mitigations that each address a named cause, and admits the class of question that should not be asked of sampled frames at all.