How do you hand work to a running asyncio event loop from another OS thread?

middleshould knowfreq 55%

answer

Loop internals assume one thread
Only two documented crossing points
One queues a callback, one schedules a coroutine
The coroutine one returns a blocking future
Capture the loop object inside the loop

basics

~10 s

Use the two thread-safe entry points: asyncio.run_coroutine_threadsafe(coro, loop) to schedule a coroutine and get a concurrent.futures.Future back, or loop.call_soon_threadsafe(callback, *args) to queue a plain callback. Every other asyncio API is single-thread only.

solid answer

~40 s

Almost nothing in asyncio is thread-safe: tasks, futures and asyncio.Queue all assume they are touched only from the loop's own thread. Exactly two APIs cross the boundary. loop.call_soon_threadsafe(cb, *args) queues a callback and wakes the loop through its self-pipe, returning a handle you can cancel but no result. asyncio.run_coroutine_threadsafe(coro, loop) schedules a coroutine and returns a concurrent.futures.Future, so the foreign thread can call .result(timeout) and block for the value. Capture the loop object inside the loop thread with asyncio.get_running_loop() and pass it to the thread - a foreign thread has no running loop of its own. Never call .result() from the loop thread itself: that blocks the very thread that must run the coroutine, and it deadlocks.

Almost nothing in asyncio is thread-safe: tasks, futures and `asyncio.Queue` all assume they are touched only from the loop's own thread. Exactly two APIs cross the boundary. `loop.call_soon_threadsafe(cb, *args)` queues a callback and wakes the loop through its self-pipe, returning a handle you can cancel but no result. `asyncio.run_coroutine_threadsafe(coro, loop)` schedules a coroutine and returns a `concurrent.futures.Future`, so the foreign thread can call `.result(timeout)` and block for the value. Capture the loop object inside the loop thread with `asyncio.get_running_loop()` and pass it to the thread - a foreign thread has no running loop of its own. Never call `.result()` from the loop thread itself: that blocks the very thread that must run the coroutine, and it deadlocks.

code

python · 18 lines

python

import asyncio
import threading


def producer(loop, q):
    for i in range(3):
        loop.call_soon_threadsafe(q.put_nowait, i)


async def main():
    q = asyncio.Queue()
    loop = asyncio.get_running_loop()
    threading.Thread(target=producer, args=(loop, q)).start()
    for _ in range(3):
        print(await q.get())


asyncio.run(main())

go deeper

for a junior

Know that asyncio objects belong to one thread and that there is a dedicated, thread-safe way to hand work to a running loop from elsewhere. Recognising the two function names is enough at this level.

for a middle

Explain the split: a callback scheduler that returns a cancellable handle versus a coroutine scheduler that returns a blocking future, why the loop object must be captured inside the loop, and why an asyncio.Queue cannot be fed directly from a thread.

for a senior

Demonstrate the failure modes you have actually hit: self-deadlock from waiting on the loop's own thread, workers stuck on futures a stopped loop will never complete, and a shutdown order that joins producers before closing the loop.

for a principal

Argue about where the boundary belongs at all. Traffic constantly crossing between threads and a loop is a design smell; decide which subsystems are async, which are blocking, and how many crossings the architecture tolerates.

Why a special API is needed at all

An event loop keeps a plain list of ready callbacks, a heap of timers, and a selector registration table. None of that is protected by a lock, because in normal operation only one thread ever touches it. So a background thread that calls loop.create_task(...), asyncio.Queue.put_nowait(...) or future.set_result(...) is mutating loop state concurrently with the loop: items get lost, a waiting consumer is never woken, and a selector already parked in select() does not notice the new work until some unrelated event happens to wake it. The failure is intermittent and looks like a hang.

CPython therefore documents exactly two thread-safe operations on a loop.

call_soon_threadsafe

loop.call_soon_threadsafe(callback, *args) appends the callback under the loop's internal lock and then writes a byte to the loop's self-pipe so a blocked selector wakes immediately. It returns a handle with .cancel(), and that is all - there is no return value and no way to await it, because the caller is not a coroutine. It is the right primitive for fire-and-forget signalling: pushing an item a worker produced, setting an asyncio.Event, telling the loop to stop.

The canonical use is feeding an asyncio.Queue from a thread. asyncio.Queue.put_nowait must run on the loop thread, so the thread does not call it directly - it schedules it: loop.call_soon_threadsafe(q.put_nowait, item). If the coupling matters more than the queue's asyncio-ness, the alternative is a plain queue.Queue drained with await asyncio.to_thread(q.get).

run_coroutine_threadsafe

asyncio.run_coroutine_threadsafe(coro, loop) is the request/response direction. It schedules coro on loop and immediately returns a concurrent.futures.Future - deliberately the blocking-world future, not an asyncio.Future, because the caller lives in the blocking world. The calling thread can ignore it, poll it, add a done callback, or block on .result(timeout) and .cancel() it.

Three rules keep it safe:

Get the loop from the loop. Call asyncio.get_running_loop() inside a coroutine and hand the object to the thread when you start it. A foreign thread has no running loop, and reaching for a loop-getter there will not find yours.
Always pass a timeout to .result(). If the loop has stopped, or was never running, the future simply never completes and the thread waits forever.
Never call it on the loop's own thread. .result() blocks that thread, which is the only thread that can run the coroutine - an instant self-deadlock. If you are already inside the loop, you are inside async code: just await.

Converting the other direction

If a coroutine needs to wait on a concurrent.futures.Future produced by a thread or process pool, asyncio.wrap_future(fut) turns it into an awaitable, and that is what the loop's run_in_executor() does internally. So the full picture is symmetric: asyncio.to_thread() and run_in_executor() go loop-to-thread, run_coroutine_threadsafe() and call_soon_threadsafe() go thread-to-loop, and wrap_future() adapts a pool result back into the async world.

Shutdown is where this bites

A background thread holding the loop object outlives the loop unless you arrange otherwise. Once the loop closes, every subsequent run_coroutine_threadsafe raises or hangs depending on timing, and call_soon_threadsafe raises RuntimeError on a closed loop. Production code makes the ownership explicit: the thread is told to stop, joined, and only then is the loop closed - and the thread's own submissions always carry timeouts so a stopped loop surfaces as an error rather than a stuck worker.

What this is not

This is not a way to run two loops cooperatively, and it is not a general concurrency model. Each loop still belongs to one thread; these calls are a mailbox between worlds. If you find most of your traffic crossing the boundary, the real answer is usually to move the boundary - make the producer async, or run the whole subsystem in threads and bridge once at the edge.

## Why a special API is needed at all An event loop keeps a plain list of ready callbacks, a heap of timers, and a selector registration table. None of that is protected by a lock, because in normal operation only one thread ever touches it. So a background thread that calls `loop.create_task(...)`, `asyncio.Queue.put_nowait(...)` or `future.set_result(...)` is mutating loop state concurrently with the loop: items get lost, a waiting consumer is never woken, and a selector already parked in `select()` does not notice the new work until some unrelated event happens to wake it. The failure is intermittent and looks like a hang. CPython therefore documents exactly two thread-safe operations on a loop. ## call_soon_threadsafe `loop.call_soon_threadsafe(callback, *args)` appends the callback under the loop's internal lock and then writes a byte to the loop's self-pipe so a blocked selector wakes immediately. It returns a handle with `.cancel()`, and that is all - there is no return value and no way to await it, because the caller is not a coroutine. It is the right primitive for fire-and-forget signalling: pushing an item a worker produced, setting an `asyncio.Event`, telling the loop to stop. The canonical use is feeding an `asyncio.Queue` from a thread. `asyncio.Queue.put_nowait` must run *on the loop thread*, so the thread does not call it directly - it schedules it: `loop.call_soon_threadsafe(q.put_nowait, item)`. If the coupling matters more than the queue's asyncio-ness, the alternative is a plain `queue.Queue` drained with `await asyncio.to_thread(q.get)`. ## run_coroutine_threadsafe `asyncio.run_coroutine_threadsafe(coro, loop)` is the request/response direction. It schedules `coro` on `loop` and immediately returns a `concurrent.futures.Future` - deliberately the blocking-world future, not an `asyncio.Future`, because the caller lives in the blocking world. The calling thread can ignore it, poll it, add a `done` callback, or block on `.result(timeout)` and `.cancel()` it. Three rules keep it safe: * **Get the loop from the loop.** Call `asyncio.get_running_loop()` inside a coroutine and hand the object to the thread when you start it. A foreign thread has no running loop, and reaching for a loop-getter there will not find yours. * **Always pass a timeout to `.result()`.** If the loop has stopped, or was never running, the future simply never completes and the thread waits forever. * **Never call it on the loop's own thread.** `.result()` blocks that thread, which is the only thread that can run the coroutine - an instant self-deadlock. If you are already inside the loop, you are inside async code: just `await`. ## Converting the other direction If a coroutine needs to wait on a `concurrent.futures.Future` produced by a thread or process pool, `asyncio.wrap_future(fut)` turns it into an awaitable, and that is what the loop's `run_in_executor()` does internally. So the full picture is symmetric: `asyncio.to_thread()` and `run_in_executor()` go loop-to-thread, `run_coroutine_threadsafe()` and `call_soon_threadsafe()` go thread-to-loop, and `wrap_future()` adapts a pool result back into the async world. ## Shutdown is where this bites A background thread holding the loop object outlives the loop unless you arrange otherwise. Once the loop closes, every subsequent `run_coroutine_threadsafe` raises or hangs depending on timing, and `call_soon_threadsafe` raises `RuntimeError` on a closed loop. Production code makes the ownership explicit: the thread is told to stop, joined, and only then is the loop closed - and the thread's own submissions always carry timeouts so a stopped loop surfaces as an error rather than a stuck worker. ## What this is not This is not a way to run two loops cooperatively, and it is not a general concurrency model. Each loop still belongs to one thread; these calls are a mailbox between worlds. If you find most of your traffic crossing the boundary, the real answer is usually to move the boundary - make the producer async, or run the whole subsystem in threads and bridge once at the edge.

What goes wrong if a worker thread calls loop.create_task() directly?
It mutates the loop's ready queue with no lock and without waking a parked selector, so the task may start late, start on the wrong side of a shutdown, or corrupt scheduling state. The symptom is an intermittent hang or a task that never runs, which is far harder to debug than the exception you would prefer. Schedule with `run_coroutine_threadsafe` instead.
Why does asyncio.run_coroutine_threadsafe() return a concurrent.futures.Future rather than an asyncio.Future?
Because the caller is a blocking thread, not a coroutine. An `asyncio.Future` is only usable from the loop's thread and can only be awaited; a `concurrent.futures.Future` has the blocking-world API the caller needs - `result(timeout)`, `exception()`, `cancel()`, `add_done_callback()`. The reverse adapter is `asyncio.wrap_future()`, which turns a pool future into something a coroutine can await.
How do you cleanly stop a loop that a background thread is still submitting work to?
Signal the thread first, join it, and only then stop and close the loop - typically by calling `loop.call_soon_threadsafe(loop.stop)` from the shutdown path once no producer remains. Every submission from the thread should carry a timeout, so if the ordering slips the worker raises instead of blocking forever on a future the stopped loop will never complete.

saying these in an interview costs you the question

Calling loop.create_task() or Queue.put_nowait() from a foreign thread
Assuming asyncio objects are thread-safe like queue.Queue
Blocking on run_coroutine_threadsafe().result() inside the loop thread
Calling .result() with no timeout on a possibly stopped loop
Expecting call_soon_threadsafe() to return the callback's value
Creating a second loop in the worker thread to run the coroutine

How do you hand work to a running asyncio event loop from another OS thread?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

Why a special API is needed at all

call_soon_threadsafe

run_coroutine_threadsafe

Converting the other direction

Shutdown is where this bites

What this is not

▾common follow-ups3 questions

▾red flags6 flags

§ context

▾related questions7 questions

▾related topics3 topics

Why a special API is needed at all

call_soon_threadsafe

run_coroutine_threadsafe

Converting the other direction

Shutdown is where this bites

What this is not

answer

go deeper

context