How can register pressure lower occupancy and slow a texture-heavy fragment program after an edit that barely changes its instruction count?
answer
- a fixed register file per processor
- registers per lane times lanes
- fewer resident groups, less hiding
- allocation follows the heaviest path
- live ranges, unrolling, spills
basics
~20 sEach GPU processor has a fixed register file shared by its resident SIMD groups. A program needing more registers per lane fits fewer groups, so fewer are ready to run while others wait on texture reads, and memory latency stops being hidden.
solid answer
~50 sA GPU hides memory latency by keeping many **SIMD groups** resident on each processor and switching to a ready one while another waits on a texture or buffer read. All resident groups share a fixed **register file**, so how many fit, the **occupancy**, is roughly the file size divided by registers per lane times lanes per group. An edit that keeps a few more values alive across a texture read can push the allocation past a granularity step and cut the resident groups, so stalls appear although the instruction count barely moved. Confirm it with the compiler's register count and a GPU capture's occupancy and stall figures, then shorten live ranges, recompute cheap values instead of keeping them, use 16-bit types where the GPU supports them, cap unrolling, or move a heavy, rarely taken path out. Higher occupancy is not always faster: an arithmetic-bound program may not need it.
go deeper
Recall that GPUs keep many groups of lanes in flight to hide slow memory reads, and that registers are a shared, limited resource. More registers per lane means fewer groups in flight.
Explain occupancy as register file divided by each group's need, with allocation steps. Show why a texture-heavy program suffers first when occupancy drops.
Diagnose the regression from compiler register counts and capture counters, then reduce live ranges, unrolling or the heaviest path. Know that more occupancy is not automatically faster.
Set register budgets for the hottest programs and decide when an uber-shader's worst-path allocation justifies splitting it into variants, weighed against the variant count the team can afford.
## Latency hiding by switching groups A texture or buffer read that misses the caches takes hundreds of cycles. A GPU processor does not wait for it: it keeps several **SIMD groups** resident (typically 32 or 64 lanes each, depending on the GPU) and, each cycle, issues an instruction from any group that is ready. While one group waits on its read, others compute. The scheme works only if enough groups are resident to cover the wait. The share of the processor's group slots that are actually filled is called **occupancy**. ## Registers decide how many groups fit Each lane's live values sit in **registers**, and every resident group needs its full allocation for its whole lifetime. The register file is fixed, so: `resident groups = min(hardware group limit, floor(register file / (registers per lane x lanes per group)))` Other limits apply as well, such as on-chip shared memory for compute workgroups, and the lowest one wins. Allocation is rounded up to a granularity, so occupancy falls in steps rather than smoothly. As an illustration, take a processor whose register file holds 16 groups at 32 registers per lane (real sizes and steps vary by GPU): | Registers per lane | Resident groups | |---|---| | 32 | 16 | | 48 | 10 | | 64 | 8 | | 96 | 5 | | 128 | 4 | An edit that moves a program from 64 to 65 registers per lane, with a granularity of 8, is allocated 72 and drops from 8 resident groups to 7 (`512 / 72 = 7.1`). For a texture-heavy program that relied on those groups to cover its reads, the processor now idles more often. ## What raises register pressure - **Long live ranges.** A value computed early and used late occupies a register the whole time, especially across texture reads. - **Fetches hoisted early.** Compilers start reads as early as possible to overlap their latency, and every in-flight result needs a register. - **Full loop unrolling.** It removes loop overhead but multiplies the values live at once. - **Dynamically indexed local arrays.** They may be kept in registers in full or pushed to memory. - **A heavy branch.** Registers are allocated for the program's heaviest path, so a rarely taken expensive branch raises the cost of every invocation. - **Wide types.** 32-bit values where 16-bit precision would do use twice the space on GPUs that pack 16-bit values. When a program needs more registers than the hardware allows per lane, the compiler **spills** values to memory, which turns register reads into memory traffic and is usually far worse than lower occupancy. ## Diagnosing the regression 1. Compare the compiler's statistics before and after the edit: registers per lane, any spills, and the instruction count. 2. Check whether the change crossed an allocation step; a one-register change can matter more than fifty instructions. 3. Take a GPU capture and compare occupancy and the stall reasons it reports; a rise in time spent waiting on memory confirms that latency is no longer hidden. 4. Try the smallest reversal that brings the register count back under the step and measure again. ## Fixes and their trade-offs - **Shorten live ranges**: compute values just before use, and reorder code so fewer values are held across reads. - **Recompute instead of keeping**: a cheap expression recalculated later can be cheaper than a register held for its whole life. - **Use 16-bit types** where precision allows and the GPU packs them. - **Limit unrolling** for loops that issue reads. - **Move heavy, rare paths out**, into a separate program variant, so common materials are not allocated for them. Higher occupancy is not automatically faster. An arithmetic-bound program with few long waits keeps the units busy with a handful of groups, and some programs deliberately hold more data in registers to avoid fetching it again. Too many resident groups can also thrash the caches they share. The goal is enough occupancy to hide this program's latency, measured, not the maximum. ## Compute programs too A compute workgroup that reserves a large block of shared on-chip memory limits occupancy the same way: fewer workgroups fit on the processor. Workgroup size matters as well, because a processor holds a limited number of groups and lanes. Tuning a compute pass means checking all three limits and finding which one binds.
- Why can full loop unrolling make a program slower?Unrolling removes loop overhead and lets the compiler issue many texture reads early, but every fetched value must sit in a register until used. The extra live values can push registers per lane over an allocation step, cutting resident groups, or overflow into spills to memory. The saved instructions are then lost to stalls.
- When is low occupancy acceptable?When the program is arithmetic-bound with few long memory waits, a handful of groups keeps the units busy and more would add nothing. Some programs deliberately use many registers to keep data on-chip and reuse it, which beats fetching it repeatedly. Measure the stall reasons before chasing occupancy.
- What limits occupancy in a compute program besides registers?Shared on-chip memory per workgroup: a workgroup that reserves a large block leaves room for fewer workgroups on the processor. The workgroup size matters too, since a processor holds a limited number of groups and lanes. Whichever of these limits is lowest sets occupancy.
- Instruction count alone predicts how fast a fragment program runs.
- Registers used only inside a rarely taken branch cost other pixels nothing.
- Maximum occupancy is always the fastest configuration.
- Each extra register adds a cycle to every instruction.
- Register spills are harmless because they land in a cache.