skip to content

After a renderer swaps its own TAA for a temporal upscaler at a lower internal resolution, textures blur and particles smear; which inputs did it get wrong?

level: seniorimportance: should knowfreq 38%

answer

  1. TAA with a larger history
  2. mip bias for the output
  3. motion for everything that moves
  4. reactive mask, reset, HUD after

basics

~20 s

A temporal upscaler is TAA that reconstructs a larger output, so it needs correct jitter, depth, motion vectors and exposure. Blurry textures mean no negative mip bias for the lower internal resolution; smearing particles mean missing motion vectors or reactive mask.

solid answer

~50 s

A temporal upscaler (DLSS, FSR and XeSS are the rival implementations) renders a **jittered** frame at a lower internal resolution and accumulates it, guided by **motion vectors** and **depth**, into a history at output resolution. Blurry textures point at the **mip bias**: textures choose mips from the internal grid's screen derivatives, so at half resolution per axis materials need a bias of about `log2(0.5) = -1` to sample the detail the output can show. Smearing particles lack **motion vectors** and depth, so they are reprojected with the surface behind them; they need their own motion, or a **reactive mask** that tells the upscaler to trust history less there. The rest of the checklist: a jitter sequence long enough for the scale factor, jitter removed from motion vectors and passed exactly as the projection used it, exposure supplied, history reset on cuts, and HUD and film grain applied after the upscale.

go deeper

for a junior

Know that a temporal upscaler renders fewer pixels and rebuilds a full-resolution image from several jittered frames using motion vectors and depth.

for a middle

Explain why it needs jittered colour, depth, unjittered motion vectors and exposure, and why its history lives at output resolution rather than internal resolution.

for a senior

Map each symptom to its input: mip bias for blur, motion vectors or reactive masks for smears, jitter mismatch for swimming, resets for ghosts after cuts, the HUD after the upscale.

for a principal

Make the renderer's inputs a contract enforced in content and code review, so adding or swapping upscaler implementations costs integration work, not visual regressions.

## What a temporal upscaler is A **temporal upscaler** is temporal anti-aliasing whose history lives at a higher resolution than the frame it renders. Each frame is rendered at a lower **internal resolution** with a sub-pixel **jitter**; the upscaler reprojects its output-resolution history with **motion vectors**, decides per pixel how much of that history is still valid, and merges the new samples in at their exact jittered positions. Over several frames each output pixel collects samples from many slightly different positions, so it can resolve detail finer than one frame's grid. DLSS, FSR and XeSS are the rival implementations; they differ in how they weigh the history, some with a trained neural network and others with hand-written heuristics, but they ask the renderer for essentially the same inputs. At two thirds of the output resolution per axis the renderer shades about 44% of the pixels; at half per axis, 25%. ## The inputs it trusts The upscaler cannot know what the renderer did not tell it, and most artefacts trace back to one of these inputs: | Input | What goes wrong when it is wrong | |---|---| | Jittered colour at internal resolution | Samples land in the wrong place, so the image blurs or shimmers | | The jitter offset itself | A value or sign that differs from the projection's makes the image swim | | Depth | Disocclusion goes undetected, so ghosts appear | | Motion vectors without jitter, for everything that moves | Objects without them smear and trail | | Exposure, or a pre-exposure value | Ghosting or flicker when brightness changes | | A reactive or transparency mask | Particles, glass and animated textures smear | | A reset flag on camera cuts | The old shot ghosts over the new one | ## Symptom to cause in this renderer - **Distant textures look blurry** next to native rendering. Texture sampling chose mips for the internal resolution, because screen-space derivatives are larger on a smaller grid. Apply a negative **mip bias** of `log2(internal width / output width)`: about -0.58 at two thirds, -1 at half. The upscaler can reconstruct only detail the samples contain. Too much extra bias brings shimmer back. - **Particles and sparks smear.** They are blended and usually write no depth or motion, so the upscaler reprojects them with the surface behind them. Give them motion vectors where possible, and mark them in a **reactive mask** so history counts less there. - **The image swims slightly when still.** The jitter passed to the upscaler does not match the projection's offset, or the motion vectors include the jitter. - **Scrolling or animated materials ghost**, such as flowing water or a screen with moving content: their texture moves while the geometry does not, so their motion vectors are zero; mark them reactive too. - **A ghost of the last shot after a cut**: the history was never reset. ## Where it sits in the frame 1. Render opaque and transparent geometry at internal resolution with jitter. 2. Run the **upscaler** where TAA used to run, on linear HDR colour, producing output resolution. 3. Run post effects. Bloom, motion blur and depth of field can run on either side, trading cost against quality, but film grain, sharpening and similar noise-like effects belong after it, or the upscaler treats their noise as signal and smears it. 4. Tone map and grade. 5. Composite the **HUD** at output resolution, unjittered and outside the history. ## Things the renderer must still do - Keep **motion vectors** complete for all content: skinned meshes, vertex-animated foliage, cloth and spawned debris. - Rescale every **screen-space effect** that assumed native resolution, such as kernel radii measured in pixels. - Use a jitter sequence with **more positions for larger scale factors**, so every output pixel is covered by enough distinct samples. - Pass the **exposure** the tone mapper will use, so history weighting sees the same brightness as the final image. ## Why the result depends on the renderer A temporal upscaler reconstructs from evidence. With correct jitter, depth, motion and masks it produces an image close to native TAA, sometimes sharper, for a fraction of the shading cost. With a missing or wrong input, no quality mode fixes it, because the evidence is wrong rather than the reconstruction. Changing implementation will not help either: each of them reads the same inputs.

  • Why not keep the native mip bias and add a stronger sharpening pass after the upscaler?
    Sharpening only raises contrast in what was sampled. If textures were sampled from mips chosen for the internal grid, the fine detail never reached the samples, so there is nothing to accumulate and sharpening mostly exaggerates noise. The negative bias lets the jittered samples across frames pick up the finer mip the output resolution can display.
  • What does a reactive mask tell a temporal upscaler, and what does overusing it cost?
    It marks pixels whose current colour cannot be predicted from history, such as blended particles, glass and animated textures, so the upscaler weights the new frame more heavily there. That trades some aliasing and noise for fewer smears. Marking too much gives back the aliasing and instability the upscaler was meant to remove.
  • How does dynamic resolution combine with a temporal upscaler?
    The internal resolution changes from frame to frame within set bounds to hold the frame time, while the output and its history stay at a fixed size. The renderer passes the current internal size, jitter and matching mip bias every frame. History stays valid because it lives at output resolution, so the change shows up as softness, not as a reset.

  • A temporal upscaler is a spatial filter that needs only the final colour image.
  • Blurry textures after upscaling are fixed with a stronger sharpening pass.
  • Particles need no motion vectors because the upscaler detects them itself.
  • The HUD should be rendered before the upscaler so it is anti-aliased too.
  • Motion vectors should include the jitter so the upscaler knows the offset.
  • Moving to another upscaler implementation fixes smearing from missing motion vectors.