skip to content

How do sys.settrace and sys.setprofile differ in the events they deliver?

level: middleimportance: should knowfreq 30%

answer

  1. One counts lines, one counts calls
  2. Only one of them sees C functions
  3. Only one has a per-frame second level
  4. Line events versus call and return events
  5. Coverage needs one, a call profiler the other

basics

~20 s

A trace function set by sys.settrace fires per executed line as well as on calls, returns and exceptions, and uses per-frame local callbacks. A profile function set by sys.setprofile fires only on call and return, plus calls into C functions, and its return value is ignored.

solid answer

~40 s

`sys.settrace` gives **statement granularity**: a global callback on frame entry whose return value receives `'line'`, `'return'` and `'exception'` events for that frame. `sys.setprofile` gives **call granularity**: one callback per thread, fired on `'call'` and `'return'`, plus `'c_call'`, `'c_return'` and `'c_exception'` for calls into C functions that a trace function never sees. A profile function has no local level - its return value is ignored - and it is never told about lines. Cost follows directly: tracing is proportional to lines executed, profiling to calls made, which is why line coverage needs `settrace` while the standard library's pure-Python profiler is built on `setprofile`. Both are per-thread, both are read back with `sys.gettrace` and `sys.getprofile`, and both occupy a single slot that the last installer wins.

code

python · 20 lines
python
import sys

events = []

def profiler(frame, event, arg):
    events.append(event)      # return value is ignored

def helper(n):
    return n * 2

def main():
    total = 0
    for i in range(3):
        total += helper(i)
    return total

sys.setprofile(profiler)
main()
sys.setprofile(None)
print(events)                 # calls and returns only, never 'line'

go deeper

for a junior

Remember the one-line distinction: the trace hook can report every executed line, the profile hook reports function calls and returns. Knowing which one a coverage tool needs is the expected depth here.

for a middle

Give the full event sets, including the C-call events only a profile function receives, and explain that a profile function has no per-frame second level and its return value is ignored. Tie each to its natural consumer.

for a senior

Reason about cost in production terms: tracing bills per executed line and profiling per call, so a per-line hook can consume a tight latency budget on its own. Know that both are per-thread and both vanish silently if the callback raises.

for a principal

Set the policy on what instrumentation is permitted where - test-time only versus a sampled slice of production - and decide whether measurement should ride these Python-level callbacks at all rather than a lower-overhead mechanism.

## Two hooks, two granularities CPython exposes two closely related callback hooks, and the interview question is almost always about which one costs what and sees what. **`sys.settrace(func)`** installs a *trace* function. It is called on frame entry with `event == 'call'`, and the callable it returns receives that frame's `'line'`, `'return'` and `'exception'` events. Statement granularity is the point: you learn that line 47 executed. **`sys.setprofile(func)`** installs a *profile* function. It is called with the same `(frame, event, arg)` signature, but the event set is different and the two-level structure is absent: * `'call'` - a Python function is about to be entered; * `'return'` - a Python function is about to return; * `'c_call'` - a C function is about to be called, with `arg` being the C function object; * `'c_return'` and `'c_exception'` - that C function returned or raised. There is **no `'line'` event** for a profile function, and its return value is discarded: one callback services the whole thread rather than spawning per-frame callbacks. ## The C-call asymmetry That `'c_call'` family is the part candidates forget. A trace function is blind to calls into C: from its point of view, a line that spends its whole time inside a C routine is just one `'line'` event. A profile function sees the boundary crossing, which is exactly what a call-counting profiler needs in order to attribute time to a C routine rather than to its Python caller. ## Cost model Both hooks deliver events by making a real Python-level call from the evaluation loop, so the honest way to think about overhead is *how many events does my program generate*: * a trace function bills you **per executed line**, so a tight numeric loop is the worst case and several-times-slower is routine; * a profile function bills you **per call**, so straight-line code with few calls is nearly free while a call-heavy program can still be slowed noticeably. Neither is a production instrument. If a request handler has, say, a 40 ms 92nd-percentile budget, a per-line hook can eat the whole budget by itself. Instrumented code also cannot take the specialized fast paths the adaptive interpreter would otherwise use, which compounds the direct callback cost. ## Reading and clearing `sys.gettrace()` and `sys.getprofile()` return whatever is installed for the calling thread, or `None`. Both are cleared by passing `None`. Both are single-slot per thread, so a debugger and a coverage tool genuinely conflict, and so do two profilers. Both are per-thread state: installing on the main thread does nothing for a worker thread, which is why `threading.settrace` and `threading.setprofile` exist for threads started through the `threading` module. ## Which one do you actually reach for * **Line coverage** needs to know which statements executed, so it needs `'line'` events - a trace function. * **A debugger** needs to stop between statements, so it needs a trace function too; `bdb.Bdb` is exactly that, wrapped in a class. * **A deterministic call profiler** needs entry and exit timestamps per function, including C functions, so a profile function is the right hook, and the standard library's pure-Python profiler is built on `sys.setprofile`. Its C counterpart uses the equivalent hook at the C level, which is why it is markedly cheaper: the callback never becomes a Python-level call. ## Errors, in both If either callback raises, the interpreter unsets that hook, as if `None` had been installed. That is the same silent failure mode in both APIs: instrumentation stops mid-run and only a `sys.gettrace()` or `sys.getprofile()` check afterwards reveals it. ## One more practical difference Because a profile function has no local level, you cannot cheaply exclude a subtree of code from profiling the way a trace function's global handler can return `None` for uninteresting frames. Filtering a profile hook means testing the frame on every call event, inside the callback, and eating the callback cost regardless. That asymmetry - a trace function can decline work, a profile function cannot - is worth saying out loud in an interview, because it is the practical reason a well-written tracer can be cheaper than a naive profiler on call-heavy code. ## The hook does not trace itself Both hooks suppress themselves while the callback runs: work done inside your trace or profile function does not generate further events, so a callback that calls other Python functions cannot recurse into the interpreter's dispatch. That is a safety property, not a performance one - the time your callback spends still lands on the traced program's wall clock, so an expensive callback distorts exactly the measurement it exists to take. Keep the body to appending a tuple to a list and do the analysis after the run; formatting, string building or I/O inside the callback is how a profile becomes a work of fiction. ## Choosing in practice Ask what the smallest event that answers your question is. "Did this statement run?" needs line events, so it is a trace function and you accept the cost. "Which functions dominate, and how often are they called?" needs call granularity, so it is a profile function - and if the cost still hurts, the honest next step is to move measurement out of the Python-level callback layer entirely rather than to make the callback cleverer.

  • Which of the two hooks tells you that a C function was called, and why does that matter?
    Only a profile function installed with `sys.setprofile`; it receives `'c_call'`, `'c_return'` and `'c_exception'` events with the C function object as the third argument. A trace function never sees the boundary, so time spent inside a C routine looks like time spent on the Python line that invoked it. A deterministic call profiler needs those events to attribute cost correctly.
  • Can a profile function exclude a subtree of code the way a trace function can?
    Not as cheaply. A trace function's global callback returns `None` at the `'call'` event and the frame then runs completely uninstrumented. A profile function has no local level and its return value is ignored, so it is invoked on every call and return regardless; any filtering happens inside the callback, after you have already paid for the call.

saying these in an interview costs you the question

  • Thinks sys.setprofile also delivers line events
  • Expects a profile function's return value to mean something
  • Believes a trace function reports calls into C functions
  • Says profiling and tracing hooks can both be active per thread with no conflict
  • Assumes either hook applies process-wide across threads

context

resolving links…