When does uniform frame sampling fail, and how would you sample video adaptively instead?

## The assumption uniform sampling makes Sampling every N seconds implicitly assumes that any interval is as informative as any other. That is roughly true for a densely edited film and badly false for the footage most systems actually process: security cameras, dashcams, machine monitoring, operating-room recordings, long meetings. In those, information arrives in bursts separated by long stretches where nothing changes. The consequence is a budget spent in the wrong place. Over an eight-hour shift, a uniform sample produces hundreds of near-identical frames of an empty aisle, each fully billed and each contributing nothing, while the ten seconds that the whole review exists for get the same two or three frames as any other ten seconds. ## The second failure: events smaller than the interval A sampling interval sets a hard floor on what is observable. If frames arrive every five seconds, an event that begins and ends inside three seconds may be represented by zero frames, or by one ambiguous frame at its very edge. There is no prompt that recovers information that was never sent. This is why lowering the rate to fit a budget is not a soft quality tradeoff: past a threshold set by the shortest event you care about, detection does not degrade gracefully, it goes to zero. ## The shape of an adaptive sampler Adaptive sampling separates the cheap job of deciding what is interesting from the expensive job of understanding it. **A cheap signal.** Frame differencing, a motion-energy or scene-change score, an encoder's own keyframe and motion-vector data, or a small purpose-built detector that already runs on the stream. All of these are orders of magnitude cheaper per frame than a model call, so you can afford to compute them on every decoded frame. **A policy over that signal.** Three parameters carry most of the value. A floor rate, so quiet periods are still sampled thinly and you never go completely blind. A burst rate, applied while the signal exceeds a threshold, dense enough to resolve the event. And pre-roll plus post-roll: when a trigger fires, include some seconds before and after it, because causality lives in the approach and the aftermath, not in the instant of the impact. **A budget cap.** Without one, a windy day of moving shadows triggers continuously and you have rebuilt uniform sampling at the burst rate. Cap frames per window and degrade by raising the trigger threshold when the cap binds. ## Keyframe selection as the other flavour A related technique picks a fixed number of frames spread by content rather than by clock: cluster frames by visual similarity and keep one representative per cluster, or keep the frames where the scene changes most. This is the right tool when you must hit an exact frame budget, for example to fit a context window, and it produces a storyboard rather than a timeline. Its weakness is the mirror image of motion gating: it under-samples long, slow, important activity because that activity is visually homogeneous. ## What adaptive sampling silently loses Every gate has a blind spot, and naming yours is what separates a senior answer from a clever one. A motion gate misses events defined by absence or stillness: a person who stops moving, a machine that should be running and is not, an item that was there an hour ago and is gone now. It also misses slow change, such as a gradual spill, because the per-frame difference never exceeds the threshold even though the hour-scale difference is enormous. It also breaks timing. If the model receives ten frames from one dense burst and two from an hour of quiet, and the frames arrive without explicit timestamps, it will tend to read them as an evenly spaced sequence and badly misjudge durations. Adaptive sampling therefore makes explicit per-frame timing more important, not less. Finally, the gate becomes a component you must evaluate. Its recall against a labelled set of known incidents is a number you should be able to quote, because the model can only be as good as the frames the gate let through, and a gate failure is invisible downstream: the model answers confidently about footage it never saw. ## Judgment to demonstrate Good answers tie the burst rate to the event duration, name the pre-roll and post-roll, admit the stillness blind spot, and treat the sampler as an evaluated component with its own recall metric rather than as a preprocessing detail.