skip to content

Error Reports

13 roadmaps12 questions

Turning a failure into something a human or a tracker can read: formatting and capturing tracebacks, installing the last-chance handlers, and the extra detail modern releases put in the output.

part ofPythonoverview, primer and where to startread it →
on this pageshow

explore

questions

12

In a Python traceback, what do the `~~~^^^` marks under the source line point at?

level: juniorimportance: must knowfreq 58%

answer

  1. A line number was not enough
  2. Something changed in Python 3.11
  3. The compiler records more than lines
  4. Per-instruction start and end columns
  5. PEP 657 fine-grained error locations

basics

~10 s

They mark the exact sub-expression that failed. Since Python 3.11 (PEP 657) every bytecode instruction carries start and end column offsets, so the traceback underlines the failing operation instead of blaming the whole line.

solid answer

~40 s

Since **Python 3.11** (PEP 657) the compiler stores a start line, end line, start column and end column for every bytecode instruction, and the default exception display uses them to draw an anchor under the source line. The `~` characters span the expression that was being evaluated; the `^` characters sit under the operation that actually raised - the operator in a binary expression, the argument list of a call, the `[...]` of a subscript. Before 3.11 you got a line number only, so `a["x"] / a["y"]` left you guessing. The anchor line is omitted when it would cover the whole expression, since an all-caret line adds nothing. The position data is static in the code object, costs nothing per execution, and can be dropped with `PYTHONNODEBUGRANGES=1` or `-X no_debug_ranges`.

code

python · 7 lines
python
import dis

def rate(sched):
    return sched["legs"] / sched["hours"]

for instr in dis.get_instructions(rate):
    print(instr.opname, instr.positions)

go deeper

for a junior

Be able to read an anchor out loud: tildes span the expression being evaluated, carets sit under the operation that failed. Knowing this turns a three-candidate line into one.

for a middle

Explain where the data comes from - per-instruction start and end columns stored in the code object since Python 3.11 - and that it is decoded only when a traceback is rendered, not while the code runs.

for a senior

Know the opt-out and its trade: PYTHONNODEBUGRANGES=1 or -X no_debug_ranges shrinks .pyc files and code-object memory at the price of every future traceback being vaguer. Also know that logging only an exception's message discards the anchor entirely.

for a principal

Own the decision about what your production images keep. Position data is cheap insurance against long incidents; argue the memory saving on measured numbers, and make sure whatever you choose is set in one place rather than per developer.

Through Python 3.10 a traceback frame told you a file, a line number and the text of that line. On a line such as `return sched["legs"] / sched["hours"]` that leaves three plausible culprits - two subscripts and a division - and no way to choose between them. PEP 657, shipped in **Python 3.11**, closed the gap. The compiler now records for every bytecode instruction not just a line number but a **start line, end line, start column and end column**, and the default exception display uses those columns to draw an anchor beneath the printed source: ``` File "differ.py", line 2, in rate return sched["legs"] / sched["hours"] ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~ ``` ### Reading the anchor The two symbols have two jobs. The `~` characters span **the whole expression that was being evaluated**. The `^` characters sit under **the specific operation that raised**. In the example the division is the failing operation, and the two dictionary lookups are its operands, so the operator gets the caret and the operands get tildes. For a call, the callee is tilded and the parenthesised argument list is caretted - that is why `rate({...})` in the calling frame shows `~~~~^^^^^^^^`. For a subscript that raises `KeyError` the carets land on the brackets and the key. For a bare name that is not defined, the carets cover just the name. There is one rule that surprises people: **if the anchor would cover the entire displayed expression, CPython omits the anchor line altogether.** A row of carets under everything communicates nothing, so a frame whose whole line failed shows source with no marks under it. That is normal output, not a missing feature. ### Where the data lives The column offsets are stored in the code object, in a compressed table alongside the line-number table. Nothing decodes that table while your program runs - it is consulted only when something asks for a position: a traceback being displayed, a debugger, a coverage or profiling tool. So the feature has no per-instruction execution cost. Its real cost is space: larger `.pyc` files and more memory held by the code objects of a large codebase. Any tool can read the same data. `dis.get_instructions()` yields instruction records whose `positions` field carries exactly the four numbers the traceback uses, which is a useful way to convince yourself the anchor is derived data rather than heuristics over source text. ### Turning it off, and why you usually should not Because the cost is space, CPython offers an opt-out: start the interpreter with `-X no_debug_ranges`, or set `PYTHONNODEBUGRANGES=1` in the environment. Both keep the source line in the traceback and drop the anchor line under it. The switch also affects code compiled while it is in effect, so a `.pyc` written under it stays position-free until it is regenerated. The trade is bad in most deployments: you are saving a small amount of memory by giving up the diagnostic that most shortens an incident. Reach for it only in genuinely memory-constrained images, and know that you have made every future traceback vaguer. ### It is a display-time enrichment This is the part that bites in real systems. The anchor is computed while the exception is being *rendered*; it is not part of the exception's message. `str(exception)` for a `ZeroDivisionError` is still `"division by zero"` with no hint of which division. A service that catches an exception and logs only its message therefore throws away the column information the interpreter went to some trouble to record. If you want the precision, the rendered traceback is what has to reach your log store. ### Why it matters in practice The anchors pay off exactly where line numbers stop helping: chained lookups (`legs[i]["dep"]`), one-line comprehensions, arithmetic over several dictionary reads, and calls nested inside calls. A schedule differ that computes `legs[i]["dep"] - legs[i - 1]["arr"]` will, on bad input, tell you whether the missing key was the departure or the arrival - which used to require adding a print statement and reproducing the failure. Since **Python 3.13** the interpreter also colours the traceback when it believes it is writing to a terminal, and the anchor is one of the things it highlights, so on a developer machine the failing operation is picked out twice: by position and by colour.

  • Why does a traceback sometimes print the source line with no caret line under it?
    Two ordinary reasons. Either the anchor would have covered the entire expression, in which case CPython suppresses it because a row of carets under everything says nothing; or the interpreter was started with `-X no_debug_ranges` or `PYTHONNODEBUGRANGES=1`, which strips the column data. A `.pyc` compiled while that switch was set also lacks positions until it is regenerated.
  • Does storing all those column offsets slow the program down?
    No. The positions are static data compressed into the code object next to the line-number table, and nothing decodes them while your code runs. They are read only when a traceback is rendered or a tool asks for them. The cost is space - bigger `.pyc` files and more memory held by code objects - which is precisely why the opt-out exists.
  • If I catch an exception and log `str(exception)`, do I still get the anchor?
    No. The anchor is produced while the exception is being displayed, from the code object of the frame; it is not part of the exception's message. Logging only the message keeps the text `"division by zero"` and discards both the frame and the column information. Log the rendered traceback if you want the precision.

A line number is a street address; the caret anchor is the flat number. Same building, far less door-knocking.

saying these in an interview costs you the question

  • Says the carets point at the line where the exception was caught
  • Thinks a third-party formatter, not CPython, draws the anchors
  • Claims tracebacks have always underlined the failing sub-expression
  • Believes the anchor text appears inside `str(exception)`
  • Says the column data costs time on every instruction executed

context

resolving links…

open as a page

What is sys.excepthook, and when does CPython call it?

level: juniorimportance: must knowfreq 44%

basics

~20 s

sys.excepthook is the callback CPython runs when an exception escapes the main thread's top level. The default prints a traceback to sys.stderr and the process exits non-zero. Replace it to log or report the crash.

open as a page

What does traceback.format_exc() return, and how does traceback.print_exc() differ?

level: juniorimportance: must knowfreq 68%

basics

~20 s

traceback.format_exc() returns the full report for the exception currently being handled as one string. traceback.print_exc() renders the same text but writes it to sys.stderr and returns None. Both read the current exception, so call them inside an except block.

open as a page

Why does `str(exc)` on a caught `NameError` omit the "Did you mean" hint the console prints?

level: middleimportance: should knowfreq 34%

basics

~20 s

The suggestion is computed when the exception is displayed, not when it is raised. The message string stays "name 'x' is not defined"; the traceback machinery adds the closest matching name it can find in the failing frame.

open as a page

Why doesn't sys.excepthook run when a threading.Thread's target raises?

level: middleimportance: should knowfreq 38%

basics

~20 s

A thread's exception never reaches the main thread's stack, so sys.excepthook is never involved. The threading bootstrap catches it and calls threading.excepthook instead, which prints 'Exception in thread ...' and lets the process carry on with an unchanged exit status.

open as a page

What does traceback.extract_tb() return, and when do you want it over rendered text?

level: middleimportance: should knowfreq 36%

basics

~20 s

traceback.extract_tb() turns a traceback object into a StackSummary: a list of FrameSummary items carrying filename, lineno, name and the source line. You want it when the failure must become structured fields rather than one blob of text.

open as a page

What does traceback.format_exception() return when passed an exception object?

level: middleimportance: should knowfreq 44%

basics

~20 s

It returns a list of strings that concatenate to exactly the report the interpreter would print. Since Python 3.10 you pass just the exception instance; by default it also renders the exceptions reachable through cause and context.

open as a page

Capturing frame locals in error reports made a flight-schedule differ leak memory - what is going wrong?

level: seniorimportance: should knowfreq 36%

basics

~20 s

The reports keep exception objects alive. An exception holds its traceback, the traceback holds every frame, and each frame holds its locals, so a count-bounded buffer pins whole schedules. The captured locals also leak secrets.

open as a page

In asyncio, what reports an exception from a task nobody ever awaited?

level: seniorimportance: should knowfreq 34%

basics

~20 s

The event loop's exception handler. A failing asyncio.Task stores its exception; if nothing retrieves it, the loop calls its handler with 'Task exception was never retrieved' when the task object is collected. Install your own with the loop's set_exception_handler method.

open as a page

Why capture a failure as a traceback.TracebackException instead of keeping the exception?

level: seniorimportance: should knowfreq 32%

basics

~10 s

traceback.TracebackException.from_exception() pre-renders everything a report needs — frames, source lines and the chained exceptions — and keeps no live frames. Holding the exception instead pins every frame's locals in memory until you release it.

open as a page

How should a Python service's error output differ between local development, CI and production?

level: principalimportance: should knowfreq 28%

basics

~20 s

Vary three things deliberately: colour, position anchors and captured content. Keep colour and carets for developers, force colour off where logs are stored, keep anchors everywhere unless memory is desperate, and never capture frame locals in deployment.

open as a page

Which hook reports an exception raised inside a __del__ method?

level: seniorimportance: nice to knowfreq 20%

basics

~20 s

sys.unraisablehook, added in Python 3.8. A finalizer runs at an arbitrary moment with no meaningful caller, so its exception cannot propagate; CPython prints 'Exception ignored in: ...' and keeps going. Replacing the hook lets you log those instead of losing them.

open as a page