skip to content

Why can't sibling asyncio tasks see each other's ContextVar values, and what do you do instead?

level: seniorimportance: should knowfreq 34%

answer

  1. Ambient state has a direction
  2. The snapshot has a timestamp
  3. Two children, two separate copies
  4. Moving one line changes what is inherited
  5. Up or sideways needs an explicit channel

basics

~20 s

Each task runs in its own copy of the context taken when it was created, so writes are private: they never reach a sibling, and a parent's write after creation never reaches an already-running child. Share by passing values explicitly or through an object all the tasks hold.

solid answer

~50 s

Context propagation in asyncio is one-way and one-shot. `asyncio.create_task` and `asyncio.TaskGroup.create_task` snapshot the current context at the call, so each sibling gets its own copy: a `ContextVar.set` in one is invisible to the others and to the parent, and a `set` the parent performs **after** creating a child never reaches that child. Code that appears to work usually depends on an accidental ordering — the value happened to be set before the tasks were created — and breaks the moment a refactor moves the creation earlier or wraps it in a task group. The fixes, in order of preference: set the value before creating any task so all copies inherit it; pass it as an argument to the coroutine; hand a prepared `contextvars.Context` to `create_task` with the `context` keyword; or, when tasks genuinely must communicate, use an explicit channel such as an `asyncio.Queue` or a shared object rather than ambient state.

code

python · 15 lines
python
import asyncio, contextvars

section = contextvars.ContextVar("section", default="unset")

async def render():
    await asyncio.sleep(0)
    return section.get()

async def main():
    async with asyncio.TaskGroup() as tg:
        task = tg.create_task(render())
        section.set("headcount")   # too late: the snapshot is already taken
    print(task.result())           # unset

asyncio.run(main())

go deeper

for a junior

Remember the direction: values flow from creator to task at creation time only. If two pieces of concurrent work must exchange data, pass it as an argument or return it rather than relying on ambient state.

for a middle

Explain why each sibling has its own copy and why a set after create_task cannot reach the child. Know the three fixes: set before creating, pass a parameter, or hand over a prepared context.

for a senior

Diagnose it from symptoms: default values inside tasks, a refactor that moved a line, a wrapper capturing the context in its own frame. Be ready to inspect a task's context and to write the concurrent regression test that pins the ordering.

for a principal

Own the policy on ambient state: what is allowed to travel invisibly, where it must be re-established because context does not cross thread, process or interpreter boundaries, and how to keep an escape-hatch shared object from becoming the team's accidental global.

### The scenario A nightly report generator fans out one task per section — headcount for an 11-person team, burn-rate, incident counts — inside an `asyncio.TaskGroup`. A `contextvars.ContextVar` carries the run id so every log line can be attributed. It works for months. Then someone reorders the group body so the run id is assigned just after the tasks are created rather than just before, and every section starts logging the default value. Nothing raises; the logs simply become useless. That is the failure mode this leaf exists for: **an ordering assumption**. The code never said *the run id must be set before any task is created*, but that is what it depended on. ### The rule that explains it Each task runs in a **copy** of the context that was active when it was created. `asyncio.create_task` takes the snapshot during the call; `asyncio.TaskGroup.create_task` does the same for each child at the moment that child is created. Three consequences follow, and a senior answer states all three: 1. **A child cannot write to its parent.** Its `ContextVar.set` lands in its own copy and is discarded when it finishes. 2. **A child cannot write to a sibling.** Siblings hold separate copies made from the same source, not a shared mapping. 3. **A parent cannot write to an already-created child.** The snapshot was taken at creation; later writes in the parent go into the parent's own mapping only. Point 3 is the one that turns into an incident, because it makes correctness depend on statement order across a refactor rather than on anything the type system or a test naturally checks. ### Diagnosing it The symptom is always the same shape: a ContextVar reads its default, or a stale value, inside a task, while the parent can prove it set something. Work through it in this order: * **Find the creation point** of the task and ask what was set *before* it. Not before the `await`, before the `create_task`. * **Check for a task group.** Children are created where `create_task` is called in the group body, not where the group is awaited on exit — moving a line inside `async with` changes the snapshot. * **Look for a wrapper** that creates the task on your behalf, such as a scheduling helper or a background-job launcher: the snapshot is taken in *its* frame, which may be nowhere near yours. * **Inspect the task's context directly.** Since Python 3.12 `asyncio.Task.get_context()` returns the context a task is running in, and a `contextvars.Context` is a mapping, so you can dump exactly what the child inherited instead of inferring it. ### Fixing it, in order of preference 1. **Set before you create.** The smallest fix and usually the right one: establish ambient values at the top of the unit of work, then fan out. Add a test that creates two tasks concurrently and asserts each sees the right value, so the ordering is pinned by something other than habit. 2. **Pass it as a parameter.** If only the coroutine needs it, ambient state was never required. Explicit arguments cannot be broken by a reordering. 3. **Hand over a prepared context.** Build one with `contextvars.copy_context()`, populate it via `Context.run` with `ContextVar.set`, and pass it as the `context` keyword to `asyncio.create_task` — available since Python 3.11, and accepted by `asyncio.TaskGroup.create_task` too. The task's ambient state then does not depend on what the creating frame happened to hold. 4. **Use a real channel for genuine communication.** If sections must actually tell each other something, that is not ambient state at all: use an `asyncio.Queue`, a shared mutable object created before the fan-out, or return values collected from the tasks. Reaching for a ContextVar here is a design smell, because the mechanism is explicitly built to prevent exactly that visibility. ### The design point worth making ContextVars are for values that flow **down** with the work — correlation ids, tenants, deadlines — and their isolation is a feature, not a limitation: it is what makes them survive arbitrary interleaving. The moment you want a value to flow up or sideways, you have left what ambient state is for, and the honest fix is an explicit channel.

  • How would you write a regression test that pins this ordering?
    Create two tasks concurrently, each expecting a different ambient value, and assert each result carries its own — a single-task test cannot catch it. Set the value before the fan-out in the code under test, and add a case where a task is created before the value is assigned to prove it reads the default rather than silently inheriting. Since Python 3.12 you can also assert on `asyncio.Task.get_context()` directly.
  • A helper function creates the background task for you. Where is the context captured?
    In the helper's frame, at the moment it calls `asyncio.create_task` — not in your calling frame and not when the coroutine first runs. If the helper does any work before creating the task, anything it sets is included; anything you set after calling it is not. When that indirection matters, take the capture out of the helper's hands by building a `contextvars.Context` yourself and passing it through as the `context` keyword.
  • Sibling tasks both hold a dict from the same ContextVar. Can they see each other's changes to it?
    Yes, and this is the sharp edge. Copying a context copies the bindings, not the objects, so if a dict was in the ContextVar before the tasks were created, every sibling holds the same dict and mutations are visible to all of them. You have shared mutable state again, with none of the isolation the ContextVar appeared to promise — so it needs the same discipline as any other shared object.
  • When is wanting sibling visibility a sign the design is wrong?
    Almost always. ContextVars carry values that flow down with the work — correlation ids, tenants, deadlines — and their per-task isolation is what makes them survive interleaving. If tasks need to exchange information, that is communication, not ambient state: use an `asyncio.Queue`, a shared object created before the fan-out, or the tasks' return values, all of which make the data flow visible in the code.

saying these in an interview costs you the question

  • Expects a parent's later set to reach a running child
  • Thinks TaskGroup children share one context
  • Uses a ContextVar as a channel between tasks
  • Assumes the snapshot is taken when the coroutine first runs
  • Calls the isolation a limitation rather than the point
  • Believes a shared dict inside a ContextVar is isolated too

context

resolving links…