skip to content

GPU Submission and Synchronization

How recorded GPU work reaches the queue: command buffers, pre-built pipeline state, barriers, fences, frames in flight, frame graphs. Interviews probe it to test explicit-API rendering knowledge.

on this pageshow

explore

questions

6

How do GPU command buffers let several CPU threads record one frame's draws, and how is the recorded work ordered at submission?

level: middleimportance: must knowfreq 50%

answer

  1. record now, execute later
  2. no shared hidden context
  3. one command memory pool per thread
  4. the submitting thread sets the order

basics

~20 s

A command buffer is a list of GPU commands the CPU records for later execution. Each thread records its own buffers from its own command memory, and one submission hands them to a queue in the order the frame needs.

solid answer

~40 s

A **command buffer** holds encoded GPU commands: pipeline and resource binds, draws, dispatches, copies and barriers. Recording only writes into it; nothing runs until it is submitted to a **queue**. Driver-managed APIs routed every call through one implicit context with hidden state, so recording was effectively single-threaded. Explicit APIs make each command buffer self-contained, so each worker takes a slice of a pass (say 2,500 of a 10,000-draw scene) and records into its own buffer from its own command memory pool, with no locks. One thread then submits the buffers in the order the frame needs, slices in range order. The queue starts work in that order, but consecutive work can overlap unless a barrier orders it.

go deeper

for a junior

Recall that a command buffer is recorded on the CPU and executed later, only after it is submitted to a GPU queue.

for a middle

Explain why a shared driver context forced single-threaded recording, how per-thread pools and buffers remove it, and why submission order, not finishing order, fixes the frame.

for a senior

Show the recording split you would choose for a 10,000-draw pass, the per-buffer overhead that limits it, and where cross-queue signals are needed when copy or compute queues join.

for a principal

Weigh parallel recording against its complexity: when a frame is GPU-bound it buys little, and the threading model then shapes how every pass in the renderer is written.

## What a command buffer is A **command buffer** (also called a command list) is a block of memory into which the CPU encodes GPU work: binding a pipeline state, binding resources, setting viewports, issuing draws and compute dispatches, copying data and inserting barriers. **Recording** only writes these encoded commands. The GPU runs nothing until the buffer is **submitted** to a **queue**, the stream of work that one of the GPU's hardware engines consumes. That split, record now and execute later, is the core of every explicit graphics API. ## Why driver-managed APIs recorded on one thread In older driver-managed APIs the application called into one implicit **context**. Each call changed hidden state the driver tracked: the bound shaders, the blend mode, the textures, and the hazards between resources. The driver turned those calls into its own internal command buffer, validating state and inserting synchronization as it went. Because that hidden state was shared, two threads could not issue calls at once without racing, so engines funnelled every rendering call through one render thread. A scene of ten thousand draws cost one core ten thousand trips through the driver. Explicit APIs remove the shared context: - each command buffer carries its own state, so nothing leaks from one buffer into another; - validation and shader compilation move to object creation (pre-built pipeline state), so recording one draw is cheap; - the application, not the driver, inserts synchronization, so the driver does not need to see the whole frame to record any part of it. ## Recording one frame on several threads 1. The render thread builds the frame's sorted draw list for each pass. 2. It splits each large pass into contiguous ranges, for example four ranges of 2,500 draws out of the scenario's 10,000. 3. Each worker records its range into its **own command buffer**, allocated from its **own command memory pool**. Where an API exposes such pools, one is typically not safe to allocate from on two threads at once, so one pool per thread, per frame in flight, removes the lock. 4. When every worker has finished, one thread submits the buffers in range order, then the next pass's buffers. Which CPU threads run those workers, and how the engine's job system schedules them, is an engine concern. What matters at the queue is that four buffers recorded in parallel and submitted in range order give the GPU the same sequence as one buffer recorded serially. Where an API scopes render-target setup to a single command buffer, a pass split across threads needs that API's way of continuing one pass across several buffers, such as reusable sub-lists that a primary buffer executes. ## Submission order and queues **Submission order is the frame's order.** The order in which workers *finish* recording is irrelevant; the submitting thread decides. A queue **starts** commands in submission order, but GPUs overlap work aggressively: the draws of one pass run concurrently, and consecutive passes can overlap unless a **barrier** makes one wait for the other. Submission order defines what "earlier" means for those barriers; it does not by itself make each command finish before the next begins. Many GPUs expose more than one kind of queue: | Queue | Accepts | Typical use | |---|---|---| | Graphics | draws, dispatches, copies | the main frame | | Compute | dispatches, copies | compute work overlapped with rendering (async compute) | | Copy | copies only | streaming uploads on a dedicated copy engine | Work on different queues runs independently, so ordering across queues needs explicit GPU-side signals. The number and kind of queues differ between GPUs, and a renderer must work when only one graphics queue exists. ## Common mistakes - Treating recording as execution, and expecting a draw's result to exist when the record call returns. - Sharing one command memory pool between threads behind a lock, which serializes recording again. - Submitting buffers in the order threads happen to finish, which shuffles draws from frame to frame. - Splitting so finely that each thread records a handful of draws: per-buffer and per-submission overhead then outweighs the parallelism, so a few buffers per pass is the usual granularity. - Expecting parallel recording to make the GPU faster. It removes a CPU bottleneck and helps only when the frame is limited by recording on the CPU. Parallel recording is therefore a CPU-side win with a GPU-side rule attached: the buffers may be built in any order, on any thread, but they must reach the queue in the order the frame was designed to run.

  • Where an API exposes command memory pools, why does each recording thread get its own?
    A pool hands out the memory a command buffer grows into, and allocating from one pool on two threads at once is typically not safe without a lock. One pool per thread, and per frame in flight, removes the lock, and a whole pool can be reset in one call once the GPU has finished the frame that used it, which is far cheaper than freeing buffers one by one.
  • If four threads record the opaque pass, how do you keep the draws in a sensible order?
    Sort the pass's draw list once, split it into four contiguous ranges, record each range into its own command buffer and submit the four buffers in range order. The GPU then sees exactly the sequence single-threaded recording would have produced; only the CPU work ran in parallel.
  • Why would a renderer submit uploads on a copy queue instead of the graphics queue?
    Where the GPU has a dedicated copy engine, copies submitted to a copy queue run alongside rendering instead of taking time on the graphics queue. The price is cross-queue synchronization: the graphics queue must wait on the copy queue's signal before it reads the uploaded data.

  • Recording a command buffer makes the GPU execute those draws immediately.
  • Recording on several threads needs a lock around every draw call.
  • The order in which threads finish recording decides the order the GPU runs them.
  • Commands in one submission each finish before the next one starts, with no overlap.
  • Recording on eight threads makes the GPU render eight times faster.
open as a page

How do fences let the CPU record two or three frames ahead of the GPU, and what breaks when a per-frame buffer is reused early?

level: middleimportance: must knowfreq 52%

basics

~20 s

The GPU signals a fence when a frame's work completes. Each frame in flight has its own buffers; before reusing a set, the CPU waits on the fence of the frame that last used it, or overwrites data the GPU still reads.

open as a page

Why do per-draw state changes and resource binding cost CPU time in a renderer, and what does pre-built pipeline state remove?

level: middleimportance: should knowfreq 45%

basics

~20 s

Driver-managed APIs validated and translated each state change at draw time, sometimes recompiling shaders. A pre-built pipeline state object bakes shaders and fixed-function state into one validated object at load, so a draw just switches to it.

open as a page

A post-processing pass reads stale or half-written data from the colour target the previous pass rendered, but only on some GPUs; what barrier is missing, and how do you place barriers without stalling the GPU?

level: seniorimportance: should knowfreq 40%

basics

~20 s

A barrier from colour-target write to shader read is missing; it orders the passes, makes the writes visible to texture reads and transitions the target's layout. Batch transitions per pass boundary and narrow their stages so the GPU drains once.

open as a page

An open-world game hitches whenever streamed meshes and textures are copied to the GPU; how would you redesign the upload path, its heap types and its residency handling?

level: seniorimportance: should knowfreq 34%

basics

~20 s

Sub-allocate from large pre-made heaps, stage data through a mapped upload ring, copy on a separate queue into device-local memory, and gate use and reuse with fences. Cap bytes per frame and evict least-recently-used data under the budget.

open as a page

When should a renderer adopt a frame graph that orders passes, derives barriers and aliases transient targets, and what does the move cost a rendering team?

level: principalimportance: should knowfreq 24%

basics

~20 s

Adopt a frame graph when many changing passes, several platforms or tight GPU memory make hand-placed barriers and allocations error-prone. It costs rewriting every pass, new debugging tools and aliasing bugs, so a small, stable renderer near ship can skip it.

open as a page