A telemetry collector arms signal.alarm per sensor read; why do descriptors leak and unrelated code raise TimeoutError?

This is the failure mode that makes people distrust `signal.alarm`, and it is worth taking apart carefully, because both halves are consequences of the same two facts: there is **one** wall-clock timer for the whole process, and the exception it produces lands **wherever the main thread happens to be**. ### Symptom one: the stale alarm The collector arms a five-second deadline around each sensor read. Nearly all reads finish in tens of milliseconds. If the code disarms only in the `except TimeoutError` branch — a very common shape — then the success path leaves 4.96 seconds pending on a process-wide timer. Some later stretch of main-thread work is interrupted instead: the next sensor's setup, a flush, an unrelated code path in the same process. The traceback points at a frame that never mentions a deadline, which is why this is usually misdiagnosed as a flaky sensor. It is worst under test. A 27-minute regression suite runs thousands of guarded regions in one process, and a stale alarm from test 40 fails test 41 at a random point. The failure does not reproduce in isolation, which is the signature of this bug rather than a reason to dismiss it. The cure is mechanical: disarm in `finally`, not in `except`. ### Symptom two: the leaked descriptor CPython runs the Python handler between bytecode instructions, so `TimeoutError` can be raised at *any* boundary in the guarded region. Consider: ```python sock = socket.create_connection(addr) # returns, then the alarm fires here with sock: # never reached payload = sock.recv(4096) ``` The connection exists and its descriptor is open, but the exception unwinds before `with` takes responsibility for it. The same hole exists in the hand-rolled form — if the alarm fires between `f = open(path)` returning and the name being bound, the `finally` that calls `f.close()` raises `NameError` instead, hiding the original timeout. Repeat that a few thousand times over a long run and the process hits its descriptor limit and starts failing with an OS error that looks nothing like a timeout. The narrower you make the guarded region, the smaller the window. Guard the *blocking call*, not the whole acquire-use-release sequence, and let a `with` own every resource inside it. Where the acquisition itself is the thing that can hang, accept the window and add a defensive sweep: track the objects you created and close them in the handler-free part of the unwind. ### Symptom three, usually also present: the hijacked handler `signal.signal` returns the handler it replaced. Code that ignores that return value permanently installs its own `SIGALRM` handler for the process. Anything else in the interpreter that used `SIGALRM` — a library, a debug tool, an outer deadline — now delivers into your handler. Save the previous handler and restore it, remembering that it is often `signal.SIG_DFL`. ### The shape that fixes all three ```python import signal from contextlib import contextmanager @contextmanager def deadline(seconds): def on_alarm(signum, frame): raise TimeoutError(f"exceeded {seconds}s") previous = signal.signal(signal.SIGALRM, on_alarm) signal.setitimer(signal.ITIMER_REAL, seconds) try: yield finally: signal.setitimer(signal.ITIMER_REAL, 0) signal.signal(signal.SIGALRM, previous) ``` `signal.setitimer` rather than `signal.alarm` because a per-read budget is rarely a whole number of seconds. The `finally` disarms unconditionally and restores the previous disposition. `signal.getitimer(signal.ITIMER_REAL)` at a region boundary returns `(0.0, 0.0)` when the manager is behaving, which makes the invariant assertable in tests. ### How to confirm the diagnosis before you change anything Assert `signal.getitimer(signal.ITIMER_REAL) == (0.0, 0.0)` on entry to each guarded region; a non-zero delay means someone upstream left a timer armed. Log the descriptor count periodically and correlate its slope with timeout events. And read the tracebacks for the *frames*, not the exception type: `TimeoutError` raised inside a function that takes no timeout argument is the tell. ### The deeper judgement The out-of-band deadline is a last resort. If the sensor transport exposes a timeout of its own, use it — a call that fails cleanly at its own boundary leaks nothing, needs no signal machinery, and works off the main thread. Keep `signal.alarm` or `signal.setitimer` as a coarse outer backstop with real slack above the transport's own timeout, so it fires only when the inner mechanism has genuinely failed. And when the blocking work is compiled code that never returns to the eval loop, no signal will save you: only running it in a child process you can terminate gives a hard bound.