skip to content

In a phone photo-editing app, a live filter preview drops frames while a slider drags over a 48-megapixel photo; how should the preview be designed?

level: middleimportance: must knowfreq 54%

answer

  1. the screen shows only millions of pixels
  2. a screen-sized proxy, made once
  3. latest value wins, older ones dropped
  4. the main thread swaps, never renders
  5. full resolution after the finger lifts

basics

~20 s

Preview from a screen-sized proxy, not the 48-megapixel original, rendered off the main thread with only the newest slider value; the main thread just swaps in finished images, and the full-resolution render runs after release.

solid answer

~50 s

One full-resolution pass touches about 48 million pixels, far beyond a 120 Hz frame's few milliseconds, and the slider can produce a value every frame. Keep a **proxy**: a downscaled copy sized to the area showing the photo, a few megapixels. On each slider change the main thread only records the value as *pending*, overwriting any older pending one. At most one preview render runs at a time on a worker or the GPU; when it finishes, the main thread swaps the displayed image and, if a newer value arrived meanwhile, starts the next render. Stale values are dropped, never queued. When the finger lifts, the full-resolution render runs in the background in tiles, is cancelled if the slider moves again, and replaces the preview only if its value is still current. Disk writes, decodes and parsing wait until the gesture ends.

code

pseudocode · 24 lines
pseudocode
on slider_changed(value):            # main thread, up to once per frame
    cancel(full_render_job)
    pending_value = value             # overwrite: only the newest value matters
    if not preview_in_flight:
        start_preview_render()

start_preview_render():
    preview_in_flight = true
    value = pending_value
    pending_value = none
    run_on_worker:
        image = apply_filter(proxy, value)       # screen-sized, never the original
        post_to_main: show_preview(image)

show_preview(image):                  # main thread: a swap, not a render
    display(image)
    preview_in_flight = false
    if pending_value != none:
        start_preview_render()

on slider_released(value):
    full_render_job = run_in_background:
        result = render_original_in_tiles(value)
        post_to_main: if current_value() == value: display(result)

go deeper

for a junior

Remember the three moves: a smaller proxy image for the preview, the heavy work off the main thread, and only the newest slider value rendered.

for a middle

Explain the gesture loop step by step: record the pending value, keep one render in flight, swap on the main thread, then start the next render only if a newer value is waiting.

for a senior

Show the full lifecycle: proxy size and zoom, tiled full-resolution render after release, cancelling on a new drag, and the parameter scaling that keeps proxy and final render consistent.

for a principal

Weigh preview fidelity against frame time and memory across the device range, and decide which filters may run live at all on the slowest supported phones.

## Why the original cannot be previewed live A 48-megapixel photo has about **48 million pixels**; at four bytes a pixel that is about 192 MB of pixel data to read and write on every filter pass. A slider drag can produce a new value on every frame, and at 120 Hz a frame leaves the main thread a few milliseconds. How long one full-resolution pass takes depends on the filter and the phone — assume, for this design, tens to hundreds of milliseconds on a mid-range device, and measure it. Either way it is an order of magnitude past the deadline: - run it on the main thread and every frame of the drag is dropped; - run every value on a background thread and the queue grows faster than it drains, so the preview trails the finger by seconds. ## The proxy: render what the screen can show The area that displays the photo has a few million physical pixels: a full-width screen of 2400 x 1080, for example, is about 2.6 megapixels. A **proxy** is a downscaled copy of the photo at about that size, made once when the photo opens (producing it at the view's size is the image pipeline's decode step). Filtering a 2.5-megapixel proxy instead of the 48-megapixel original is roughly twenty times less work, and the result looks the same on screen because the extra pixels were never going to be displayed. When the user zooms in, the proxy becomes the visible region of the original, again at screen size. ## The gesture loop 1. The main thread receives a slider value and stores it as the **pending value**, overwriting any earlier pending one. This costs microseconds. 2. If no preview render is in flight, it starts one, on a worker thread or the GPU, with the pending value. 3. The worker applies the filter to the proxy and posts the finished image back. 4. The main thread swaps the displayed image — an assignment, not a render — and, if a newer value arrived meanwhile, starts the next render. This is **coalescing**, or "latest wins": at most one render is in flight, the preview lags the finger by at most one proxy render, and no queue ever grows. Intermediate values the user dragged past are never rendered, and nobody misses them. The design also degrades gracefully: on a slower phone the proxy render takes longer, so the preview updates less often, but the slider thumb itself still moves on every frame because the main thread never waits for a render. ## When the finger lifts - Start the **full-resolution render** in the background, in tiles of a few megabytes each, so it never needs a second full-size buffer at once. - If the slider moves again, cancel it; its value is already stale. - When it finishes, check that its value is still the current one, then swap it in on the main thread. - Export and save always use the full render, never the proxy. ## Keep everything else off the gesture During the drag, the main thread should do nothing but take input and swap images. Writing the undo history to disk, decoding thumbnails for a filter strip, reading preset files and logging analytics either run on other threads or wait until the gesture ends. One platform's guidance is explicit that even work scheduled *asynchronously* on the main thread can land in the middle of a continuous interaction and cost a frame. ## Trade-offs | Approach | Main-thread cost per frame | Preview lag | What goes wrong | |---|---|---|---| | Full resolution on the main thread | the whole render | none, once it finishes | every frame of the drag is dropped | | Full resolution on a worker, every value queued | a swap | grows with the drag | stale previews, memory spikes | | Proxy on a worker, latest value wins | a swap | about one proxy render | proxy may differ slightly from the final | The last row has a real cost: filters that depend on pixel distances — sharpening, blur radius, grain — look different at proxy scale unless their parameters are scaled to it. The usual answer is to scale those parameters and let the full-resolution render, once it lands, show the exact result.

  • How large should the proxy be, and what changes when the user zooms in?
    About the pixel size of the area that displays the photo, so no rendered pixel is wasted; a few megapixels on a phone. When the user zooms, render the visible region of the original at that same screen size instead of enlarging the proxy, so detail appears without ever filtering the whole 48-megapixel image during a gesture.
  • The filter is cheap enough to run on the GPU at full resolution on your test phone. Do you still need the proxy?
    Measure on the slowest supported phone at its highest refresh rate before dropping it. The GPU's work has its own per-frame deadline, a full-resolution texture is another 192 MB of memory on phones that share RAM between CPU and GPU, and the margin vanishes with the next, heavier filter. The proxy keeps the gesture cheap everywhere.

  • Running the filter asynchronously on the main thread keeps the slider smooth
  • Every slider value must be rendered, in order, or the preview is wrong
  • A live preview during a drag must be rendered from the 48-megapixel original
  • A background render can update the image view directly from its own thread
  • Cancelling a stale render is unnecessary because its result is simply overwritten later