How does temporal anti-aliasing accumulate samples across frames, and why do thin railings leave ghost trails during a fast camera pan?
answer
- supersampling spread over time
- sub-pixel jitter each frame
- reproject history with motion vectors
- clamp history to the neighbourhood
basics
~20 sTAA jitters the projection by a sub-pixel offset each frame, reprojects the accumulated history with motion vectors and blends in a little of the new frame. Ghosts appear when history no longer matches the surface: disocclusion, wrong motion vectors, weak rejection.
solid answer
~50 sTAA spreads supersampling over time. Each frame the projection is offset by a different **sub-pixel jitter** from a low-discrepancy sequence, so successive frames sample different points inside each pixel. A resolve pass uses **motion vectors** to find where each pixel's surface was last frame, fetches the accumulated **history** there and blends: `out = history * (1 - a) + current * a`, with `a` around 0.1, an exponential average dominated by the last twenty or so frames. A ghost is history that no longer belongs to the pixel: background just uncovered, an object that wrote no motion vectors, or a railing thinner than a pixel whose pixels carry the sky's motion vector while the camera also translates. The standard defences are **neighbourhood clamping**, which limits history to the colour range of the current 3x3 neighbourhood, dilated motion vectors and depth-based disocclusion tests. Clamping can in turn erase sub-pixel railings and make them flicker.
code
pseudocode · 19 lines// Per output pixel p. velocity = currentUV - previousUV, with jitter removed.
current = currentColor[p]
velocity = velocityOfNearestDepth(p) // nearest surface in the 3x3 neighbourhood
prevUV = uv(p) - velocity
if prevUV is outside [0, 1] x [0, 1]:
result = current // no history off screen: restart here
else:
history = sampleCatmullRom(historyColor, prevUV)
lo = +infinity
hi = -infinity
for each offset o in the 3x3 neighbourhood:
c = toYCoCg(currentColor[p + o])
lo = min(lo, c)
hi = max(hi, c)
history = fromYCoCg(clamp(toYCoCg(history), lo, hi))
a = 0.1 // weight of the new frame
result = history * (1 - a) + current * a
nextHistoryColor[p] = result
output[p] = resultgo deeper
Recall that TAA reuses previous frames: each frame samples a slightly different point in every pixel and the results are averaged over time.
Explain jitter, motion-vector reprojection, the exponential blend and neighbourhood clamping, and name the causes of ghosting: disocclusion, missing motion vectors and thin geometry.
Diagnose from symptoms: trails behind thin railings, flicker after clamping, softness in motion and ghosts after cuts, and know which fix each needs and what it costs.
Treat TAA as a contract the whole content pipeline must honour, with motion vectors and reactive masks for every moving thing, and weigh its softness against players' clarity expectations.
## Supersampling spread over time A single frame takes one sample per pixel, so edges stair-step and anything thinner than a pixel, such as a distant railing or a power line, appears and disappears as it moves. Supersampling would fix that by taking many samples per pixel in one frame, at many times the cost. **Temporal anti-aliasing (TAA)** takes the samples over successive frames instead: one new sample per pixel per frame, at a different position inside the pixel each time, accumulated into a running **history** image. When the camera and scene are still, a few dozen frames approximate a heavily supersampled image for the price of one extra full-screen pass. ## One TAA frame, step by step 1. **Jitter.** The projection is offset by a sub-pixel amount taken from a low-discrepancy sequence (a Halton sequence of 8 or 16 positions is common), so each frame samples a different point inside every pixel. 2. **Render** the frame normally with that jittered projection. 3. **Motion vectors.** Each pixel records how far its surface moved on screen since the last frame, computed from the **unjittered** current and previous positions so the jitter never reads as motion. Static geometry can derive it from depth and the two camera transforms; moving, skinned and vertex-animated objects must write their own. 4. **Reproject.** The resolve pass subtracts the motion vector from the pixel's position to find where that surface was last frame and samples the history there, usually with a sharp filter such as Catmull-Rom to limit blur. 5. **Rectify.** The fetched history is checked against the current frame and corrected, as described below. 6. **Blend** with an exponential moving average, `out = history * (1 - a) + current * a`, with `a` around 0.1. 7. **Store** the result as next frame's history and pass it on to post-processing. With `a = 0.1`, whatever the history held before a change keeps weight `0.9^n` after n frames: about 35% after 10 frames and below 10% only after 22. That slow decay is why an uncorrected ghost lingers. ## Why ghosts appear A ghost is history that no longer describes what the pixel shows, blended in for many frames. The usual sources: - **Disocclusion**: background just uncovered by a moving object has no valid history, and reprojection lands on the object that covered it. - **Missing or wrong motion vectors**: an object that writes none is reprojected as if only the camera moved, so its old colour is fetched where it used to be. Particles, scrolling materials and animated textures commonly lack them. - **Thin geometry**: a railing narrower than a pixel covers few samples, so its pixels often carry the background's motion vector. A pure rotation moves near and far surfaces alike on screen, but a game camera that pans while following the player also translates, so near railings and the distant sky slide at different speeds, and the history fetch misses the railing and leaves a trail. - **Lighting changes and camera cuts**, after which every pixel's history is suddenly wrong. ## Keeping history honest - **Neighbourhood clamping**: take the minimum and maximum of the current frame's 3x3 neighbourhood and clamp the history colour into that box, so history that cannot belong to this area is pulled back. Working in a luma-chroma space such as **YCoCg**, and clipping toward the box centre instead of clamping per channel, reduces colour shifts. - **Variance clipping**: build the box from the neighbourhood's mean plus or minus a multiple of its standard deviation, which is tighter than minimum and maximum. - **Depth-based disocclusion**: compare reprojected depth with last frame's depth and drop history where they disagree. - **Dilated motion vectors**: use the motion vector of the nearest surface in the 3x3 neighbourhood, so thin foreground edges carry their own motion. - **Adaptive blend weight**: raise `a` when motion is large or the clamp moved the history far. - **Reactive masks**: let particles and animated materials mark pixels where history should count less. ## The railings and power lines during a fast pan | Symptom | Likely cause | Fix | |---|---|---| | Smeared trail behind a railing | Railing pixels use the sky's motion vector | Dilate motion vectors to the nearest depth | | Power lines crawl and flicker | The line is missing from some jittered frames and clamping erases its history | Keep wires at least a pixel wide and fade their opacity instead of thinning them | | Image goes soft while moving | History resampled with a soft filter every frame | Sharper history filter, mild sharpening after the resolve | | Ghost lingers after a cut | History never reset | Discard history on cuts | ## The costs TAA brings - **Softness**: accumulation and resampling blur fine texture detail, which is why most TAA ends with a sharpening pass. - **Flicker versus ghosting**: a tight clamp suppresses ghosts but lets sub-pixel detail flicker; a loose one keeps detail but ghosts. - **A content contract**: everything that moves must write motion vectors, including skinned characters, vertex-animated foliage and scrolling materials. - **Convergence**: newly revealed areas look aliased for a few frames until their history builds up. The same machinery, history plus motion vectors plus rejection, is what temporal upscalers build on.
- Why are TAA motion vectors computed from unjittered positions?If the jitter offset were included, a static scene would report a small, changing motion every frame. The resolve would fetch history from a slightly wrong place each time, so the image would wobble and blur instead of converging. The jitter belongs to the sample positions, not to the scene's movement, so it is removed from the vectors and applied only to the projection.
- Why do TAA resolves clamp history in YCoCg rather than in RGB?Clamping each RGB channel independently can push a colour to a hue none of the neighbours had. YCoCg separates luminance from two chroma axes, so the neighbourhood box fits how colours cluster more tightly, and clipping toward its centre changes brightness and tint more faithfully. It is a refinement that reduces ghost tinting, not a requirement.
- How should a TAA renderer handle transparent surfaces and particles?They usually write no depth or motion vectors, so TAA reprojects the opaque surface behind them and smears them. Renderers draw them before the resolve with a reactive mask that lowers history weight there, or composite some after the resolve; either way they trade a little aliasing on those pixels for fewer trails.
TAA is like a long-exposure photograph of a busy street taken in many short shots: the buildings sharpen as shots stack up, but anyone who moved leaves a smear unless you know where they went in each shot and line them up.
- TAA just blurs the current frame with the previous one.
- TAA's jitter is random noise added to hide banding.
- Motion vectors should include the jitter offset so history lines up.
- Neighbourhood clamping removes ghosting with no side effects.
- Particles without motion vectors are handled automatically by TAA.
- Blending 10% of each new frame means stale history is gone after 10 frames.