A FastAPI async def endpoint calls a synchronous SDK, and latency rises on every endpoint served by that worker; why, and how do you fix it?
answer
- one loop per worker process
- FastAPI never offloads async def
- the health check slows too
- def, run_in_threadpool, or an async client
basics
~20 sFastAPI awaits an async def endpoint on the worker's single event loop, so a blocking SDK call there freezes every request on that worker. Declare the endpoint def, wrap the call in run_in_threadpool, or switch to an async client.
solid answer
~40 sFastAPI awaits an `async def` endpoint directly on the event loop and never moves its body to a thread. A synchronous SDK call inside it holds the loop's only thread until it returns, so every request on that worker process, including unrelated `async def` endpoints and the health check, waits behind it; latency rises everywhere while CPU stays low. There are three fixes: declare the endpoint plain `def`, so FastAPI runs it through `run_in_threadpool`; keep `async def` and write `await run_in_threadpool(sdk.call, ...)` or `await anyio.to_thread.run_sync(...)` around the blocking call; or switch to a client you can `await`. The first two share anyio's default limiter of 40 threads per event loop. Prove the fix with concurrent requests: four 0.5 s blocking calls take about 2 s inside `async def` and about 0.5 s offloaded.
code
python · 32 linesimport time
from concurrent.futures import ThreadPoolExecutor
from fastapi import FastAPI
from fastapi.concurrency import run_in_threadpool
from fastapi.testclient import TestClient
app = FastAPI()
def sdk_call() -> dict:
time.sleep(0.5) # a synchronous SDK or database driver call
return {"status": "ok"}
@app.get("/stalls")
async def stalls():
return sdk_call() # blocks the event loop for 0.5 s
@app.get("/offloaded")
async def offloaded():
return await run_in_threadpool(sdk_call) # runs on a worker thread
with TestClient(app) as client, ThreadPoolExecutor(max_workers=4) as senders:
for path in ("/stalls", "/offloaded"):
start = time.perf_counter()
list(senders.map(client.get, [path] * 4)) # four requests at once
print(path, round(time.perf_counter() - start, 1))
# /stalls 2.0
# /offloaded 0.5go deeper
Know that a blocking call inside async def runs on the event loop and holds up other requests, and that a def endpoint avoids this.
Explain why FastAPI cannot offload async def code on its own, and show run_in_threadpool or anyio.to_thread.run_sync around the call.
Read the symptoms as one diagnosis: unrelated endpoints slow, low CPU, failing health checks. Pick def, offloading or an async client, and prove it with a concurrency test.
Weigh replacing a synchronous SDK with an async client against offloading it permanently, counting the thread-limiter budget and who owns the test that guards async paths.
## Why one endpoint slows all of them A FastAPI app served by Uvicorn runs **one event loop per worker process**. The event loop is a single thread that interleaves many requests: whenever a coroutine reaches an `await` on something not ready yet, the loop switches to another request. FastAPI's scheduling rule is simple: a **plain `def`** path operation or dependency is sent to an anyio worker thread through `run_in_threadpool`, while an **`async def`** one is awaited **directly on the event loop**. FastAPI never inspects the body of an `async def` function, and it never moves part of it to a thread. So when an `async def` endpoint calls a synchronous SDK, the call runs on the event loop's thread. Until the SDK returns: - no other coroutine on that loop can run, because nothing reaches an `await`; - requests to **every other endpoint** on that worker queue up, including cheap `async def` ones and the liveness or readiness probe; - timeouts you wrote with `await` cannot fire until the call returns, since the loop cannot run them. ## Recognising it in production The pattern has a recognisable signature: 1. latency rises on **unrelated endpoints**, not only the one that calls the SDK; 2. CPU usage stays low, because the process is waiting on the network, not computing; 3. throughput per worker collapses to roughly one slow call at a time; 4. health checks time out under load, and the platform may restart a worker that is only waiting. asyncio's debug mode logs every callback or task step that holds the loop longer than its `slow_callback_duration`, 100 ms by default, naming the task that held it. A sampling profiler showing the event loop thread inside the SDK's socket read confirms it. ## Three fixes | Fix | What changes | Cost | |---|---|---| | Declare the endpoint `def` | FastAPI runs the whole function through `run_in_threadpool` | uses a worker thread per call; no `await` possible inside | | Keep `async def`, offload the call | `await run_in_threadpool(sdk.charge, order)` or `await anyio.to_thread.run_sync(sdk.charge, order)` | uses a worker thread per call; the rest of the function stays async | | Use an awaitable client | `await client.charge(order)` | no thread at all; a library change and its own tests | The first two both draw on anyio's **default thread limiter**, 40 tokens per event loop, shared with every `def` endpoint and `def` dependency in the app. Under a burst of more than 40 slow calls the extra ones wait for a token, but they wait **without** blocking the loop, so other endpoints stay responsive. If you choose the third fix, check the whole call chain: one leftover synchronous helper inside the async client path brings the problem back. Note the argument shapes: `run_in_threadpool(func, *args, **kwargs)` forwards keyword arguments, while `anyio.to_thread.run_sync(func, *args)` takes positional arguments only, so keyword arguments go through `functools.partial`. ## Things that do not fix it - **Adding `await asyncio.sleep(0)`** before or after the call gives the loop a turn at that point, but the SDK call itself still holds the loop for its full duration. - **Moving the call into a `def` helper** changes nothing: a helper you call yourself runs on the caller's thread, which here is the event loop. - **Running more worker processes** spreads requests over more loops, so fewer requests share each stall, but every loop still freezes on every call. Worker counts belong to deployment, not to this fix. - **`BackgroundTasks`** run after the response is sent, so they cannot produce the value the endpoint returns, and an `async def` task with a blocking call blocks the loop just the same. ## Proving it with a test You can show the stall in process, with no server: send four requests at once with `fastapi.testclient.TestClient` from a few sender threads, and time them. The code example does exactly that: - `/stalls` calls a 0.5 s blocking function inside `async def`; four concurrent requests take about **2.0 s**, because they run one after another on the loop; - `/offloaded` wraps the same call in `await run_in_threadpool(...)`; four requests take about **0.5 s**, because four worker threads wait in parallel. A test like this, with an assertion on the elapsed time and a generous margin, is a cheap guard against someone reintroducing a blocking call into an `async def` path.
- Is offloading the SDK call with run_in_threadpool unlimited in FastAPI?No. `run_in_threadpool` calls `anyio.to_thread.run_sync`, which borrows a token from anyio's default limiter, 40 per event loop, shared with every `def` endpoint and `def` dependency. Beyond that, calls wait for a token, but they wait on the loop without blocking it, so other endpoints keep answering.
- What happens to an offloaded FastAPI call if the task awaiting it is cancelled?A running thread cannot be interrupted. `anyio.to_thread.run_sync` defaults to `abandon_on_cancel=False`, which shields the awaiting task until the call returns, so the thread and its limiter token stay busy until the SDK call finishes. With `abandon_on_cancel=True` the task moves on while the thread still runs to completion.
- Why does the FastAPI health-check endpoint time out when it never calls the SDK?It runs on the same event loop as the blocking endpoint. While the SDK call holds the loop's thread, the health check's coroutine cannot start or finish, so the probe's timeout expires even though its own code is trivial.
- FastAPI detects blocking calls inside async def and moves them to a thread
- Only the endpoint that calls the SDK gets slower
- Adding await asyncio.sleep(0) after the call unblocks the event loop
- Calling the SDK from a def helper keeps it off the event loop
- A blocked event loop always shows up as high CPU usage