In FastAPI, where do a plain def yield dependency's setup and cleanup run, and why does its cleanup get its own thread limiter?
answer
- def vs async def generator
- worker threads, not the event loop
- two separate thread calls
- exit code skips the shared limiter
basics
~20 sFastAPI runs a def yield dependency's setup in the thread pool under anyio's shared limiter, and its exit code in a worker thread under a fresh one-slot limiter, so cleanup never queues behind the shared one.
solid answer
~50 sFor a plain `def` generator, FastAPI wraps it with `contextlib.contextmanager` and then with its own `fastapi.concurrency.contextmanager_in_threadpool`. Setup, the code up to the `yield`, runs through `run_in_threadpool`, which waits for a token from anyio's default limiter shared with `def` endpoints and `def` dependencies. The exit code runs through `anyio.to_thread.run_sync` with a new `CapacityLimiter(1)` created for that call, so it does not queue behind that shared limiter. The source explains why: an exit that waits for a free thread can deadlock when the resource has its own internal pool, because the threads holding tokens may be waiting for the very connection that exit would return. Setup and cleanup are two separate thread calls, so nothing guarantees the same thread. An `async def` generator runs both halves on the event loop instead, where any blocking call stalls every request.
code
python · 24 linesimport tempfile
import threading
from typing import Annotated
from fastapi import Depends, FastAPI
app = FastAPI()
def get_spool(): # plain def: FastAPI runs setup and teardown in worker threads
print("setup on", threading.current_thread().name)
spool = tempfile.SpooledTemporaryFile(max_size=1024)
try:
yield spool
finally:
spool.close() # blocking cleanup stays off the event loop
print("teardown on", threading.current_thread().name)
@app.post("/spool")
async def write_spool(spool: Annotated[tempfile.SpooledTemporaryFile, Depends(get_spool)]):
print("endpoint on", threading.current_thread().name)
spool.write(b"chunk")
return {"bytes": spool.tell()}go deeper
Know that FastAPI accepts both def and async def yield dependencies, and that a def one runs its code in worker threads rather than on the event loop.
Explain contextmanager_in_threadpool: setup through run_in_threadpool under the shared limiter, exit through a fresh CapacityLimiter(1), and why the two halves may run on different threads.
Connect the private exit limiter to the deadlock it prevents when a pooled resource is exhausted, and catch blocking cleanup hidden in an async def generator.
Set a team rule for which yield dependencies must be def because a half blocks, and make thread affinity a review question for any resource opened in one.
## Two kinds of yield dependency A **yield dependency** is a generator function passed to `fastapi.Depends`: code before the `yield` is setup, the yielded value is injected, code after it is **exit code**. FastAPI accepts both kinds of generator: - an `async def` generator, which FastAPI wraps with `contextlib.asynccontextmanager` and runs **on the event loop**; - a plain `def` generator, which FastAPI wraps with `contextlib.contextmanager` and then hands to `fastapi.concurrency.contextmanager_in_threadpool`, so both halves run **in worker threads**. The choice follows the same rule as for endpoints: blocking code belongs in `def`, non-blocking awaitable code in `async def`. A yield dependency has two places where blocking work tends to hide, opening and closing, and the second one is easy to forget. ## How the thread-pool wrapper runs the two halves `contextmanager_in_threadpool` is a short async context manager in `fastapi/concurrency.py`. For one request it does this: 1. **Enter**: it awaits `run_in_threadpool(cm.__enter__)`. Starlette's `run_in_threadpool` calls `anyio.to_thread.run_sync` with no explicit limiter, so the call waits for a token from anyio's **default thread limiter**, the same one `def` endpoints and `def` dependencies use (40 tokens by default). 2. **Hand over**: the yielded value goes back to the event loop and is injected into the endpoint. 3. **Exit**: when FastAPI's exit stack closes, the wrapper calls `anyio.to_thread.run_sync(cm.__exit__, ..., limiter=exit_limiter)`, where `exit_limiter` is a **new `CapacityLimiter(1)`** made for that one call. 4. **Exceptions**: if the endpoint raised, the exception is passed to `__exit__`, which throws it into your generator at the `yield`; if the generator does not handle it, it propagates on to FastAPI. | Half | Runs on | Limiter | |---|---|---| | Setup (up to `yield`) | a worker thread | anyio's shared default limiter | | Exit code (after `yield`) | a worker thread | a fresh `CapacityLimiter(1)` per call | | Both halves of an `async def` generator | the event loop | none | ## Why the exit gets its own limiter The comment in the source states the reason: an exit that has to wait for a free thread "can create race conditions/deadlocks if the context manager itself has its own internal pool (e.g. a database connection pool)". Picture the failure without it: - all default-limiter tokens are held by threads blocked waiting for a pooled connection; - the requests that hold the connections have finished their endpoints and want to run exit code, which would return those connections; - if that exit code needed a default-limiter token, it would wait for a thread that is itself waiting for a connection. Because each exit gets a private one-slot limiter, it never waits for a token from the shared limiter. Since the limiter is new for every call, the `1` places no cap across requests; the source calls the value arbitrary. ## Consequences you can observe - **Different threads.** Setup and exit code are two separate `run_sync` calls, so they may run on different worker threads. A resource that must be closed on the thread that opened it, or a `threading.local` value set in setup, is not safe in a `def` yield dependency. - **No loop stalls from cleanup.** A blocking `close()` in a `def` generator's `finally` runs off the loop, so it never freezes other requests. - **Async generators get no such help.** A blocking call after the `yield` of an `async def` dependency runs on the event loop and holds up every request on that worker while it runs. - **Setup still competes for the shared limiter.** Under heavy load, a `def` yield dependency's setup waits for a token just like a `def` endpoint. How to size that limiter is covered in the material on offloading blocking work. ## Choosing def or async def Write the dependency as `async def` when both opening and closing are awaitable, for example an async outbound client with an `aclose()` method. Write it as plain `def` when either half blocks, such as a synchronous client, file work like removing a large temporary directory, or a library with no async interface. Mixing is fine: an `async def` endpoint can depend on a `def` yield dependency, and FastAPI moves only the dependency's halves to threads. The reverse also holds: a `def` endpoint can use an `async def` yield dependency, whose halves still run on the loop while the endpoint body runs in a worker thread.
- Can a def yield dependency rely on thread-local state set during setup still being there at cleanup?No. FastAPI runs setup and exit code as two separate calls into the thread pool, so they may land on different worker threads. Anything bound to the opening thread, such as a `threading.local` value or a handle that must be closed on the thread that created it, needs a different design, for example keeping the state on the yielded object itself.
- What happens if an async def yield dependency calls a blocking close() in its finally block?The blocking call runs on the event loop, because FastAPI runs both halves of an `async def` generator there. Every other request on that worker waits until `close()` returns. Either make the dependency a plain `def` generator so FastAPI moves its exit code to a thread, or offload that one call to a thread.
- A def yield dependency runs entirely on the event loop thread.
- Setup and cleanup of a def yield dependency always share one thread.
- Cleanup of a def yield dependency waits for a token from the shared default limiter.
- Blocking calls in an async def yield dependency are offloaded to threads automatically.