What is the difference between an instrumenting profiler and a sampling profiler, and when does a game programmer reach for each?
answer
- marked zones versus periodic stacks
- exact timeline versus statistics
- what runs unmarked
- a blocked thread leaves no samples
basics
~20 sAn instrumenting profiler records exact begin and end times of code regions you marked, giving a per-frame timeline; a sampling profiler periodically records call stacks, showing statistically where CPU time goes in all code, but not exact per-frame timings.
solid answer
~50 sAn **instrumenting** profiler times zones the programmer placed in the code: each zone records a name, a thread and two timestamps, and the viewer lays every thread out on a frame-by-frame timeline. It shows exactly which frame and which system ran long, but only for code someone marked, and markers in tight loops inflate what they measure. A **sampling** profiler needs no code changes: it interrupts threads at a regular interval and counts which call stacks it finds, so it sees unmarked code, libraries and the driver, but its result is a statistical share over many frames, which averages a rare spike away. By default it only sees threads that are running, so a thread blocked on a lock or a file read shows nothing without context-switch tracing. In practice you instrument to find the frame, thread and zone, then sample to find the function inside it.
code
pseudocode · 11 linesfunction zone_begin(name):
return { name: name, thread: current_thread_id(), start: monotonic_now_ns() }
function zone_end(zone):
zone.end = monotonic_now_ns()
append zone to the current thread's event buffer // sent to the viewer later, off the hot path
// usage around one system update
zone = zone_begin("ai_update")
update_ai(world, dt)
zone_end(zone)go deeper
Recall the two models: instrumentation times regions you marked, sampling periodically records call stacks from everything that runs.
Explain the trade-offs: exact per-frame timelines and marker overhead versus statistical coverage of unmarked code, and why a sampler misses blocked threads.
Show the workflow: find the frame, thread and zone on an instrumented timeline, sample within it, and separate waiting from working before optimising anything.
Decide which zones stay compiled into every development build, what overhead the team accepts for always-on instrumentation, and which profiling modes each platform must support.
## Two ways to see where time goes A CPU profiler answers "where did the time go?", but there are two fundamentally different ways to collect the answer, and they answer different questions. Knowing which one produced a given view is most of the skill of reading it. ## Instrumentation: timing what you marked An **instrumenting profiler** relies on **zones** (also called scopes or markers) that the programmer places in the code. Entering a zone reads a high-resolution clock; leaving it reads the clock again and records the name, the thread and both timestamps. A viewer then draws every zone of every thread on a shared timeline, frame by frame. - **Exact and ordered:** you can see that one particular frame spent 7.9 ms in the AI update on the main thread, and what ran before and after it. - **Per-frame:** a single slow frame stands out on the timeline; nothing is averaged away. - **Cross-thread:** zones from worker threads, a render thread and the main thread line up on one clock, so dependencies and waits become visible. - **Blind outside the marks:** time in code nobody marked shows up only as a gap, or inside a large parent zone. - **Cheap but not free:** a zone costs a couple of clock reads and a small write. That is negligible around a 2 ms system update and very visible around a function called a hundred thousand times per frame, where the markers inflate the measurement. Engines therefore keep coarse zones (systems, passes, jobs) compiled into development builds permanently, and add finer zones temporarily while investigating. ## Sampling: statistics over everything A **sampling profiler** needs no code changes. It interrupts each thread at a regular interval, records the call stack it finds, and afterwards counts how often each function appears. A function present in 30 % of the samples used roughly 30 % of the sampled CPU time. - **Sees everything that runs:** unmarked code, third-party libraries, the graphics driver's CPU work and the operating system. - **Function-level detail** without editing anything, which makes it the quick way to find which function inside a hot zone is the problem. - **Statistical:** at an illustrative rate of 1,000 samples per second, a 16.7 ms frame gets about 17 samples per thread, so a 1 ms function in one frame is caught once or not at all. Sampling shines over many frames and dilutes a rare spike into the aggregate. - **On-CPU by default:** a thread blocked on a lock, a file read or the GPU is not running, so it produces no stacks. Seeing waits needs **context-switch tracing**, which records when threads stop and resume. ## Side by side | | Instrumentation | Sampling | |---|---|---| | Code changes | Zones must be added | None | | Timing | Exact durations per zone | Statistical share of time | | Per-frame view | Yes, on a timeline | Aggregated over the capture | | Coverage | Only marked code | Everything running on a CPU | | Overhead | Grows with the number of zones hit | Set mostly by the sample rate | | Best for | Which frame, which system, which thread | Which function inside a hot region | ## A workflow that uses both 1. Watch the frame-time graph with instrumentation on and find the frames, or the steady cost, that break the budget. 2. Read the timeline: which thread is on the critical path, which zone grew, whether a thread is working or waiting. 3. Narrow down with finer zones, or sample that scenario to see which functions dominate the hot zone. 4. Change one thing, then measure again with the same scenario and the same build configuration. Game teams name a familiar set of tools in this space, among them Tracy, PIX, RenderDoc, Nsight, Superluminal and VTune. They differ in whether they instrument, sample, time the GPU or replay a captured frame, and several combine modes, so the useful question about any view is which of these models it is built on. ## Traps - **Profiling a debug build.** Unoptimised code changes relative costs, because loops that inline away in an optimised build dominate a debug profile; profile an optimised build that keeps debug symbols. - **Reading a sampling profile as per-frame timing.** Its percentages are averages across the whole capture. - **Instrumenting tight inner loops** and then optimising the overhead the markers added. - **Ignoring wait time:** a frame can be slow while its threads are mostly idle, and neither an on-CPU sampler nor zones that only wrap work will show what they were waiting for.
- Why can a sampling profile of a capture look healthy when one frame in it stalled for 8 ms?Two reasons. The stall is one frame among thousands, so its few samples vanish into the aggregate percentages. And if the thread was blocked, waiting on a file, a lock or the GPU, it was not running, so an on-CPU sampler recorded nothing for it. Find the frame with an instrumented timeline and turn on context-switch tracing to see the wait.
- Why profile an optimised build rather than a debug build?Optimisation changes relative costs: small functions inline away, loops vectorise and debug checks disappear, so the hotspots of a debug build are often not the hotspots players hit. Profile an optimised build that keeps debug symbols, so the profiler can still name functions.
- A sampling profiler gives exact per-frame durations for each function.
- Instrumentation sees every function without any code changes.
- Low sampled CPU time in a frame proves the frame was fast.
- Profiling a debug build is fine because relative costs stay the same.
- Zones inside tight inner loops have no measurable overhead.