A threading.Semaphore caps concurrent calls in a fraud-scoring service, but permits leak until throughput reaches zero. What causes that, and how do you prevent it?

### What a semaphore is here `threading.Semaphore(value)` wraps an integer and a condition. `acquire()` waits until the counter is positive, decrements it, and returns `True`; `release(n=1)` increments it and wakes waiters. That is the whole contract, and two properties of it drive every real bug: * **No ownership.** Unlike `threading.Lock` — and unlike `threading.RLock`, which raises when a non-owner releases — a semaphore does not record who took a permit. Any thread may release, and a thread may release a permit it never acquired. Nothing raises. * **No self-healing.** The counter is pure state. A permit that is never released is not reclaimed when the thread dies, when the request finishes, or when the garbage collector runs. It is simply gone. Used as a concurrency limiter — "at most eight outbound calls in flight at once" — those two properties mean the limiter's effective capacity is a monotonically non-increasing function of how many code paths forget to release. ### The failure, in order A scoring request fans out across a dependency graph of seventeen services. Each outbound call takes a permit. One dependency starts returning a malformed payload on a rare input, the parse raises, and the worker's `except Exception:` logs nothing useful and returns a default score. The request looks *successful* from the outside — that is what makes a swallowed exception so expensive here — but the `release()` on the line after the parse never ran. Now: capacity 8 becomes 7, then 6. Queueing delay in front of `acquire()` grows superlinearly as capacity shrinks against constant arrival rate, so the first symptom is a latency graph bending upward with no corresponding change in downstream latency. Eventually the counter reaches zero, every worker blocks in `acquire()`, and the service stops. There is no exception, no crash, no restart — just threads parked in a wait. A process dump shows every worker thread with the same stack ending in `acquire`, which is the fingerprint of a leaked-permit stall as opposed to a slow dependency. ### The fixes, strongest first **1. Make release structural, never a statement.** `threading.Semaphore` implements the context-manager protocol: `with sem:` acquires on entry and releases in `__exit__`, and `__exit__` runs whether the body returns, raises, or breaks. If you cannot use `with` — because you need a timeout — write the explicit form: ```python if not sem.acquire(timeout=2.0): raise TimeoutError("scorer at capacity") try: call_dependency() finally: sem.release() ``` The `finally` is the entire point. A `release()` sitting at the end of the happy path is a bug waiting for its first exception. **2. Use BoundedSemaphore for anything that models capacity.** `threading.BoundedSemaphore(value)` remembers its initial value and raises `ValueError` if `release()` would push the counter above it. It cannot detect a *missing* release — nothing can, since a permit legitimately stays out for as long as the work takes — but it catches the mirror-image defect: a double release, or a release on a path that never acquired, which silently *raises* the concurrency limit and defeats the protection you thought you had. Since a bounded semaphore fails at the defective call rather than at some later victim, it should be the default for a limiter. **3. Never acquire without a bound.** `acquire(timeout=...)` returns `False` instead of raising when it gives up, and `acquire(blocking=False)` returns immediately. Either turns "hang forever" into a decision you can make — shed the request, return a degraded score, increment a saturation counter. A limiter that can only queue is a limiter that converts a capacity problem into an availability problem. **4. Stop swallowing exceptions.** The permit leak is a symptom; the broad `except` that returns a default score while discarding the traceback is the disease. Catch narrowly, log with the exception attached, and let anything you did not anticipate propagate — then the `finally` still runs and the alert fires on the real cause. **5. Instrument it.** A semaphore exposes no supported public counter to read, so track saturation yourself: count timed-out acquisitions and the time spent waiting, and alert when either trends up. Both move well before the counter reaches zero. A last design note: because a semaphore has no owner, it is not a substitute for a lock and is not reentrant. A thread that takes two permits from a limiter of size one to guard a recursive call will deadlock, and no exception will tell you so.