How does Python's development mode help you find a file-descriptor leak in a service that dies at a 1,200-request-per-minute peak?

## The shape of the problem A service — say a flight-schedule differ that compares published and actual departure times — runs fine for hours and then fails at its 1,200-request-per-minute peak with `OSError: [Errno 24] Too many open files`. The descriptor count climbs with traffic and never comes back down. Raising the limit buys hours and fixes nothing. ## CPython is already telling you File objects, sockets and `subprocess.Popen` objects all emit a `ResourceWarning` from their **finalizer** when they are collected while still open. In a normal run you never see it, because the default warning filters ignore `ResourceWarning` entirely. That is the first move: run the service under `python -X dev`, which applies the default filter as part of **development mode**, and the unclosed objects start reporting themselves as they are collected. ## Why the bare warning is often not enough The warning is emitted at *collection* time and can only describe the object being collected — `unclosed file <_io.TextIOWrapper ...>` — which tells you what leaked but not who opened it. In a codebase with one helper that opens everything, that is nearly useless. The fix is to **combine two switches**: development mode does **not** enable `tracemalloc`, so add `-X tracemalloc=25` (or `PYTHONTRACEMALLOC=25`). With memory tracing active, CPython attaches the traceback of the *allocation* site to the `ResourceWarning`, and you get the frames that ran when the descriptor was opened. The number is the **frame depth**: deeper is more informative and more expensive, and 10–25 is usually plenty to cross the helper boundary and reach real application code. ## Why collection time matters - In CPython, an object whose refcount drops to zero is finalized immediately, so a leaked file inside a function that returns normally warns almost at once. - But an object caught in a **reference cycle** — the common case in a service, where a handler holds a client that holds a connection that holds a callback back to the handler — is only reclaimed when the cyclic collector runs, so the warning arrives late, in a batch, on an unrelated request's stack. Do not read the timing of the warning as the timing of the leak; read the allocation traceback. If the warnings only appear when you force a collection, you have learned something important: the objects are in cycles, and a `with` block would have closed them deterministically regardless. ## Making the leak reproducible Do this under a load profile that matches the peak, **in staging, not in the production process**. Development mode costs real performance — the allocator debug hooks touch every allocation and free — and `tracemalloc` adds per-allocation bookkeeping on top, so throughput at 1,200 requests per minute will not match production and should not be expected to. You are hunting a functional bug, not measuring latency. In a narrow reproduction — one request replayed in a loop — `-W error::ResourceWarning` is even better than displaying: the first leaked descriptor raises, and you get a full traceback from the point of collection alongside the allocation traceback. ## Two more things development mode gives you here - It makes `io.IOBase` destructors *log* exceptions raised by `close()` instead of swallowing them, which is how you discover that a file was being closed but the close itself was failing. - And it enables `faulthandler`, so if descriptor exhaustion ends in a hard crash somewhere in a C extension rather than a clean `OSError`, you get a Python traceback out of it. ## What it will not find Development mode surfaces resource, deprecation, import and allocator problems. It does not check your logic. If the same service also computes a slightly wrong delay in minutes because of a floating-point rounding drift, no amount of `-X dev` will say a word about it — that is a test's job. Being clear about the boundary is part of a good answer: this is an **instrumentation switch with a specific catchment, not a correctness checker**. ## Then fix ownership, not symptoms The durable fix is **deterministic closing**: - a `with` block around each acquisition, - `contextlib.ExitStack` where the number of resources is dynamic, - an explicit `close()` in a service shutdown path for long-lived clients, - and never relying on the garbage collector to release an operating-system resource. Keep the `ResourceWarning` visible in CI afterwards so the next one is caught by a test rather than by a peak.