skip to content

How do GPU command buffers let several CPU threads record one frame's draws, and how is the recorded work ordered at submission?

level: middleimportance: must knowfreq 50%

answer

  1. record now, execute later
  2. no shared hidden context
  3. one command memory pool per thread
  4. the submitting thread sets the order

basics

~20 s

A command buffer is a list of GPU commands the CPU records for later execution. Each thread records its own buffers from its own command memory, and one submission hands them to a queue in the order the frame needs.

solid answer

~40 s

A **command buffer** holds encoded GPU commands: pipeline and resource binds, draws, dispatches, copies and barriers. Recording only writes into it; nothing runs until it is submitted to a **queue**. Driver-managed APIs routed every call through one implicit context with hidden state, so recording was effectively single-threaded. Explicit APIs make each command buffer self-contained, so each worker takes a slice of a pass (say 2,500 of a 10,000-draw scene) and records into its own buffer from its own command memory pool, with no locks. One thread then submits the buffers in the order the frame needs, slices in range order. The queue starts work in that order, but consecutive work can overlap unless a barrier orders it.

go deeper

for a junior

Recall that a command buffer is recorded on the CPU and executed later, only after it is submitted to a GPU queue.

for a middle

Explain why a shared driver context forced single-threaded recording, how per-thread pools and buffers remove it, and why submission order, not finishing order, fixes the frame.

for a senior

Show the recording split you would choose for a 10,000-draw pass, the per-buffer overhead that limits it, and where cross-queue signals are needed when copy or compute queues join.

for a principal

Weigh parallel recording against its complexity: when a frame is GPU-bound it buys little, and the threading model then shapes how every pass in the renderer is written.

## What a command buffer is A **command buffer** (also called a command list) is a block of memory into which the CPU encodes GPU work: binding a pipeline state, binding resources, setting viewports, issuing draws and compute dispatches, copying data and inserting barriers. **Recording** only writes these encoded commands. The GPU runs nothing until the buffer is **submitted** to a **queue**, the stream of work that one of the GPU's hardware engines consumes. That split, record now and execute later, is the core of every explicit graphics API. ## Why driver-managed APIs recorded on one thread In older driver-managed APIs the application called into one implicit **context**. Each call changed hidden state the driver tracked: the bound shaders, the blend mode, the textures, and the hazards between resources. The driver turned those calls into its own internal command buffer, validating state and inserting synchronization as it went. Because that hidden state was shared, two threads could not issue calls at once without racing, so engines funnelled every rendering call through one render thread. A scene of ten thousand draws cost one core ten thousand trips through the driver. Explicit APIs remove the shared context: - each command buffer carries its own state, so nothing leaks from one buffer into another; - validation and shader compilation move to object creation (pre-built pipeline state), so recording one draw is cheap; - the application, not the driver, inserts synchronization, so the driver does not need to see the whole frame to record any part of it. ## Recording one frame on several threads 1. The render thread builds the frame's sorted draw list for each pass. 2. It splits each large pass into contiguous ranges, for example four ranges of 2,500 draws out of the scenario's 10,000. 3. Each worker records its range into its **own command buffer**, allocated from its **own command memory pool**. Where an API exposes such pools, one is typically not safe to allocate from on two threads at once, so one pool per thread, per frame in flight, removes the lock. 4. When every worker has finished, one thread submits the buffers in range order, then the next pass's buffers. Which CPU threads run those workers, and how the engine's job system schedules them, is an engine concern. What matters at the queue is that four buffers recorded in parallel and submitted in range order give the GPU the same sequence as one buffer recorded serially. Where an API scopes render-target setup to a single command buffer, a pass split across threads needs that API's way of continuing one pass across several buffers, such as reusable sub-lists that a primary buffer executes. ## Submission order and queues **Submission order is the frame's order.** The order in which workers *finish* recording is irrelevant; the submitting thread decides. A queue **starts** commands in submission order, but GPUs overlap work aggressively: the draws of one pass run concurrently, and consecutive passes can overlap unless a **barrier** makes one wait for the other. Submission order defines what "earlier" means for those barriers; it does not by itself make each command finish before the next begins. Many GPUs expose more than one kind of queue: | Queue | Accepts | Typical use | |---|---|---| | Graphics | draws, dispatches, copies | the main frame | | Compute | dispatches, copies | compute work overlapped with rendering (async compute) | | Copy | copies only | streaming uploads on a dedicated copy engine | Work on different queues runs independently, so ordering across queues needs explicit GPU-side signals. The number and kind of queues differ between GPUs, and a renderer must work when only one graphics queue exists. ## Common mistakes - Treating recording as execution, and expecting a draw's result to exist when the record call returns. - Sharing one command memory pool between threads behind a lock, which serializes recording again. - Submitting buffers in the order threads happen to finish, which shuffles draws from frame to frame. - Splitting so finely that each thread records a handful of draws: per-buffer and per-submission overhead then outweighs the parallelism, so a few buffers per pass is the usual granularity. - Expecting parallel recording to make the GPU faster. It removes a CPU bottleneck and helps only when the frame is limited by recording on the CPU. Parallel recording is therefore a CPU-side win with a GPU-side rule attached: the buffers may be built in any order, on any thread, but they must reach the queue in the order the frame was designed to run.

  • Where an API exposes command memory pools, why does each recording thread get its own?
    A pool hands out the memory a command buffer grows into, and allocating from one pool on two threads at once is typically not safe without a lock. One pool per thread, and per frame in flight, removes the lock, and a whole pool can be reset in one call once the GPU has finished the frame that used it, which is far cheaper than freeing buffers one by one.
  • If four threads record the opaque pass, how do you keep the draws in a sensible order?
    Sort the pass's draw list once, split it into four contiguous ranges, record each range into its own command buffer and submit the four buffers in range order. The GPU then sees exactly the sequence single-threaded recording would have produced; only the CPU work ran in parallel.
  • Why would a renderer submit uploads on a copy queue instead of the graphics queue?
    Where the GPU has a dedicated copy engine, copies submitted to a copy queue run alongside rendering instead of taking time on the graphics queue. The price is cross-queue synchronization: the graphics queue must wait on the copy queue's signal before it reads the uploaded data.

  • Recording a command buffer makes the GPU execute those draws immediately.
  • Recording on several threads needs a lock around every draw call.
  • The order in which threads finish recording decides the order the GPU runs them.
  • Commands in one submission each finish before the next one starts, with no overlap.
  • Recording on eight threads makes the GPU render eight times faster.