skip to content

How does FastAPI decide whether a path operation function or a dependency runs on the event loop or in its thread pool?

level: middleimportance: should knowfreq 40%

answer

  1. is it a coroutine function
  2. callable instances and __call__
  3. partial and wraps are looked through
  4. run_in_threadpool to anyio.to_thread

basics

~20 s

FastAPI 0.143 awaits a callable on the event loop when it is an async def function or an object whose call is async def, after looking through functools.partial and @functools.wraps; anything else goes through run_in_threadpool to an anyio worker thread.

solid answer

~40 s

FastAPI classifies every endpoint and dependency it calls. If the callable is an `async def` function, or an object whose `__call__` is `async def`, it is awaited directly on the event loop. Everything else is called through `run_in_threadpool`, which wraps `anyio.to_thread.run_sync` and runs the function on a worker thread under anyio's default limiter. A class used as a dependency counts as synchronous, so its `__init__` runs on a worker thread. In 0.143 the check looks through `functools.partial` objects and `@functools.wraps` decorators to the original function. For a `def` endpoint with a response model, validating the returned value also runs on a worker thread. Helpers you call yourself are never scheduled; they run wherever their caller runs.

code

python · 31 lines
python
import functools
import threading

from fastapi import FastAPI
from fastapi.testclient import TestClient

app = FastAPI()


def timed(func):
    @functools.wraps(func)
    async def wrapper(*args, **kwargs):  # an async wrapper around a def body
        return func(*args, **kwargs)

    return wrapper


@app.get("/plain")
def plain():
    return {"thread": threading.current_thread().name}


@app.get("/wrapped")
@timed
def wrapped():
    return {"thread": threading.current_thread().name}


with TestClient(app) as client:
    print(client.get("/plain").json())  # {'thread': 'AnyIO worker thread'}
    print(client.get("/wrapped").json())  # the event loop's thread, not a worker

go deeper

for a junior

Remember that FastAPI awaits async def functions on the event loop and runs plain def functions on worker threads, for endpoints and dependencies alike.

for a middle

Walk through the classification: callable instances by their call, classes as synchronous, partial and wraps looked through, and the response-model validation that also runs on a thread for def endpoints.

for a senior

Diagnose a decorator that silently moved a blocking def body onto the event loop, or a sync wrapper whose coroutine was never awaited, and say how to fix the wrapper.

for a principal

Decide which shared decorators a team may apply to endpoints and require both sync and async variants with functools.wraps, so no wrapper changes where code runs.

## Two paths for every callable FastAPI calls two kinds of your functions for you: **path operation functions** (the functions under `@app.get` and the other method decorators) and **dependencies** (the callables passed to `Depends`). For each one it makes a binary choice: - **awaited on the event loop**: the coroutine runs on the single thread that also serves every other request on that worker process; - **sent to the thread pool**: FastAPI calls `run_in_threadpool` (Starlette's helper, re-exported by `fastapi.concurrency`), which is a thin wrapper over `anyio.to_thread.run_sync`. The function runs on an **anyio worker thread**, and the event loop awaits its result. Every such call borrows one token from anyio's **default thread limiter** (40 tokens per event loop) for as long as it runs. ## How a callable is classified FastAPI 0.143 asks one question, "is this a coroutine function?", in several forms: | Callable you pass | Where FastAPI runs it | |---|---| | `async def` function | event loop | | plain `def` function | worker thread | | instance whose `__call__` is `async def` | event loop | | instance whose `__call__` is plain `def` | worker thread | | a class, as in `Depends(Pagination)` | worker thread (its `__init__` runs there) | | `functools.partial` of an `async def` function | event loop (the partial is unwrapped) | | `def` wrapper made with `@functools.wraps` around an `async def` function | event loop (the wrapper is unwrapped through `__wrapped__`) | Two points in that table catch people out: 1. A **class used as a dependency** is treated as synchronous even when it has `async` methods, because what FastAPI calls is the class itself, and constructing it is synchronous. 2. The unwrapping of `functools.partial` and of decorated functions arrived in 0.123.5, and combining both was fixed in 0.123.6. Older releases did not look through them, so material written against those releases can describe a wrapped callable being classified by its wrapper alone. ## What runs on a worker thread for a def endpoint For one request to a `def` endpoint, parsing and validating the request's parameters and body happen on the event loop, and FastAPI makes separate thread-pool calls for: 1. each `def` dependency, one call each, in dependency order; 2. the endpoint function itself; 3. validating the returned value against the declared response model (only for `def` endpoints; an `async def` endpoint validates on the loop). Each call borrows a limiter token and a worker thread for its duration and gives them back, so one request may run its pieces on **different threads**. Values you need later should travel through return values, not thread-local storage. A `def` **yield dependency** is entered through `run_in_threadpool`, but FastAPI runs its exit code with `anyio.to_thread.run_sync` and a fresh one-token limiter, so teardown never queues behind the shared pool; the source comment says waiting there can cause deadlocks when the resource has its own pool, such as a database connection pool. What the teardown does and when it runs is the yield-dependency material. ## Decorators: where the classification goes wrong Because FastAPI inspects the **outermost** callable first, a decorator can move code between the loop and the pool: - An **`async def` wrapper around a `def` endpoint** makes FastAPI see a coroutine function. The endpoint body is then called synchronously inside that coroutine, **on the event loop**, so a blocking body now stalls every request. The code example shows this. - A **plain `def` wrapper around an `async def` function** is fine in 0.143 only if it uses `@functools.wraps`, because FastAPI follows `__wrapped__` to the coroutine function and awaits what the wrapper returns. Without `wraps`, FastAPI sees a sync function, runs it on a worker thread, gets back a coroutine object it never awaits, and the request fails with a 500. A timing, caching or retry decorator therefore has to match the function it wraps: an `async def` wrapper for `async def` functions and a `def` wrapper for `def` functions, both with `functools.wraps`. ## What FastAPI does not touch - **Helpers you call yourself** run where their caller runs: a `def` helper called from `async def` code runs on the loop. - FastAPI **never inspects a function body** for blocking calls. The keyword, after unwrapping, is the whole decision.

  • Does a FastAPI def dependency run on the same worker thread as its def endpoint?
    Not reliably. Each `def` dependency and the `def` endpoint are separate `run_in_threadpool` calls, each borrowing a limiter token and an idle worker thread in turn, so one request can use several threads. Pass values through the dependency's return value, never through thread-local storage.
  • Why does a FastAPI def yield dependency's teardown not wait for a free thread-pool token?
    FastAPI enters it through `run_in_threadpool` but runs its exit with `anyio.to_thread.run_sync` and a new `CapacityLimiter(1)` per call. The source comment explains that an exit waiting on a busy shared pool can cause deadlocks when the resource keeps its own pool, such as database connections.

  • FastAPI scans the function body for blocking calls before choosing a thread
  • A class dependency with async methods is constructed on the event loop
  • An async def decorator around a def endpoint keeps its body on a worker thread
  • Only path operations are offloaded; def dependencies always run on the loop
  • Helpers declared with def are moved to the thread pool when called from async code

Assumes FastAPI 0.143; since 0.123.5 FastAPI looks through functools.partial and @functools.wraps before choosing the event loop or the thread pool.