skip to content

Why doesn't a contextvars.ContextVar set before ThreadPoolExecutor.submit reach the worker, and what did 3.14 add?

level: seniorimportance: should knowfreq 30%

answer

  1. Two different mechanisms, often confused
  2. The queue carries the callable, not the context
  3. Fires at thread start, not per task
  4. A lazily created worker captures one submitter
  5. Snapshot at submit, run inside the copy

basics

~10 s

concurrent.futures.ThreadPoolExecutor never copies the submitting thread's context onto a work item, so the worker reads defaults. Python 3.14 added -X thread_inherit_context and PYTHON_THREAD_INHERIT_CONTEXT, but they only apply when a threading.Thread is started.

solid answer

~40 s

Context lives in a context object, and nothing on the executor path copies it: `submit` queues the callable and a long-lived worker runs it, so a `ContextVar` set by the submitter reads its default inside the task. Python 3.14 added `-X thread_inherit_context` (and `PYTHON_THREAD_INHERIT_CONTEXT`) so a thread started with `threading.Thread` copies the context of the caller of `start` — off by default on the standard build, on for free-threaded builds. That flag does **not** fix a pool: workers are started lazily, so a worker captures the context of whichever submitter happened to create it and keeps it for every later task, which is worse than a uniform default. The explicit fix is per submission: `ctx = contextvars.copy_context()` then `pool.submit(ctx.run, fn, *args)`. `asyncio.to_thread` already does this; `loop.run_in_executor` does not.

code

python · 13 lines
python
import contextvars
from concurrent.futures import ThreadPoolExecutor

number_format = contextvars.ContextVar("number_format", default="C")
number_format.set("de_DE")

def render():
    return number_format.get()

with ThreadPoolExecutor(max_workers=1) as pool:
    print("plain submit:", pool.submit(render).result())
    ctx = contextvars.copy_context()
    print("via Context.run:", pool.submit(ctx.run, render).result())

go deeper

for a junior

Recall the plain fact that a thread you hand work to does not automatically see values set in the thread that handed it over, and that something has to carry them across explicitly.

for a middle

Be able to explain the two mechanisms separately: what a context is, that a thread historically started empty, and that copy_context plus Context.run is how you move a snapshot across a boundary.

for a senior

An interviewer expects the failure analysis: why the 3.14 flag is a thread-start feature rather than a per-task one, why a lazily created pool worker captures one submitter, and how the wrong output looks plausible instead of crashing.

for a principal

Own the propagation policy across services: whether ambient context crosses thread boundaries at all, whether an interpreter-level flag may be load-bearing when it is invisible in source, and how you keep the rule uniform across every hand-off point.

Two separate mechanisms are involved, and mixing them up is what makes this bug so persistent. ## Mechanism one: a new thread starts with an empty context `contextvars` values are stored in a *context*, and until Python 3.14 a thread you started never received a copy of anyone's context — it began with an empty one, so every `ContextVar` read fell back to its default (or raised if there is no default). Nothing about `threading` propagated context. Python 3.14 made that configurable. The `-X thread_inherit_context` command-line option and the matching `PYTHON_THREAD_INHERIT_CONTEXT` environment variable, when enabled, cause a thread created with `threading.Thread` to start with a *copy* of the context of whichever thread called that thread's `start` method. The default depends on the build: the standard interpreter leaves it off, and free-threaded builds turn it on, since that is the configuration the option was designed for. ## Mechanism two: the executor never copies anything `concurrent.futures.ThreadPoolExecutor.submit` does not touch contexts at all. It wraps your callable and its arguments in a work item, puts it on a queue, and a worker eventually runs it. There is no snapshot of the submitting thread's context anywhere in that path, in 3.14 or before. So on a default 3.14 interpreter, the worker runs with an empty context and your `ContextVar` reads its default. ## Why turning the flag on is not the fix Here is the part that separates a rehearsed answer from a real one. The flag applies at `Thread.start`, and a pool's worker threads are started lazily by whichever `submit` call first needed a new worker. With the flag on, that worker therefore captures **the context of the submitter that happened to create it** — and keeps it for the rest of the pool's life. Every later task on that worker, submitted by anyone, runs with that first submitter's values. Consider a sensor-telemetry collector that keeps each device's locale-dependent number format in a `ContextVar` and renders readings on a shared pool. With the flag off, every reading renders with the default format — wrong, but uniformly and visibly wrong. With the flag on, readings render with the format belonging to whichever device happened to warm that worker up, which is plausible, per-worker, and effectively random. It is exactly the shape of defect that ships through three release trains before someone trusts the complaint. So: the flag is useful for plain `threading.Thread` hand-offs, and it is not a substitute for explicit propagation across a pool. ## The explicit route, which is the answer Snapshot the context at submit time and run the callable inside it: ```python ctx = contextvars.copy_context() future = pool.submit(ctx.run, render, reading) ``` `contextvars.copy_context()` takes a shallow copy of the *calling* thread's context — the correct one, captured at the correct moment — and `Context.run` executes the callable with that context active. Two properties make this the right primitive: the copy is per submission, so nothing persists on the worker for the next task; and because it is a copy, a `ContextVar.set` inside the task does not propagate back to the submitter. If you need a value back, return it from the callable. The stdlib already does this for you in one place: `asyncio.to_thread` copies the current context and runs the function inside it, so context survives that hand-off. `loop.run_in_executor` does not, which is a genuinely useful pair of facts to have straight — two adjacent APIs for "run this on a thread" with opposite context behaviour. ## Operating it * Decide once, centrally. Wrap submission in a helper that always copies the context, rather than leaving each call site to remember. * Make the failure loud. If the value is required, read it without a default so a missing context raises at the boundary instead of silently rendering with a fallback. * Treat the interpreter flag as a deployment-level setting, not a code fix. It changes behaviour for the whole process, it is invisible in the source, and a second entry point launched without it behaves differently. If you rely on it, pin it in the process launcher and say so in the code that depends on it. * Do not reach for `threading.local` here. It is keyed by the worker thread, so it is subject to the same first-submitter capture problem in a more permanent form, plus the staleness that thread reuse brings. The one-sentence version for an interview: **thread hand-offs do not carry context unless something copies it — the 3.14 switch covers `Thread.start`, the executor covers nothing, and `copy_context` plus `Context.run` is the explicit, per-task mechanism that actually holds.**

  • Why can enabling -X thread_inherit_context make a pooled workload behave worse rather than better?
    Because it applies at thread start, and a pool starts workers lazily on demand. A worker therefore inherits the context of whichever submitter triggered its creation and keeps it for the pool's lifetime, so later tasks from other submitters silently run with a stranger's values. A uniform default is at least consistently wrong and easy to spot.
  • If a task calls ContextVar.set inside a copied context, does the submitting thread see the change?
    No. `contextvars.copy_context()` returns a copy, and `Context.run` activates that copy for the callable, so any set inside affects only the copy. That one-way isolation is the property you want for a pool — nothing leaks back to the submitter and nothing persists on the worker. Return the value from the callable if the caller needs it.
  • Which stdlib API for running a function on a thread already propagates context, and which does not?
    `asyncio.to_thread` copies the current context and runs the function inside it, so a ContextVar set before the await is visible in the function. `loop.run_in_executor` does not copy anything, so the same function reads defaults there. Two adjacent APIs, opposite behaviour — worth checking before assuming either way.

saying these in an interview costs you the question

  • Says ThreadPoolExecutor.submit copies the caller's context
  • Claims -X thread_inherit_context is on by default everywhere
  • Thinks the 3.14 flag propagates context per submitted task
  • Reaches for threading.local as the propagation fix
  • Expects a set inside a copied context to reach the submitter
  • Assumes run_in_executor behaves like asyncio.to_thread

context

resolving links…