In Celery, when task_routes, a task's own queue option and apply_async(queue=) disagree, which one decides where the task is sent?

middleshould knowfreq 30%

answer

closest to the call wins
call, then task, then routers
exact names before glob patterns
first router returning something

basics

~20 s

The apply_async(queue=) argument wins, then a queue option set on the task itself, then the first router in task_routes that returns a route. Within a task_routes dict an exact task name beats a glob pattern.

solid answer

~40 s

Celery resolves the destination when the message is sent, in a fixed order: routing arguments passed to apply_async first, then routing options declared on the task (@app.task(queue='gpu')), then the routers in task_routes. Mechanically, the router's result is merged with the message options and any non-None option overrides it. The routers are consulted in order and the first one that returns a value is used; inside a dict router an exact task name is checked before glob or regex patterns. One trap: app.send_task('video.tasks.transcode') by name never sees the task's decorator options, so only task_routes applies there. That is why I keep routing in task_routes rather than in decorators.

Celery resolves the destination when the message is sent, in a fixed order: routing arguments passed to `apply_async` first, then routing options declared on the task (`@app.task(queue='gpu')`), then the routers in `task_routes`. Mechanically, the router's result is merged with the message options and any non-`None` option overrides it. The routers are consulted in order and the first one that returns a value is used; inside a dict router an exact task name is checked before glob or regex patterns. One trap: `app.send_task('video.tasks.transcode')` by name never sees the task's decorator options, so only `task_routes` applies there. That is why I keep routing in `task_routes` rather than in decorators.

code

python · 20 lines

python

app.conf.task_routes = {
    'video.tasks.*': {'queue': 'transcode'},
    'video.tasks.notify_upload_done': {'queue': 'notifications'},
}


@app.task(name='video.tasks.transcode', queue='gpu')
def transcode(video_id):
    ...


@app.task(name='video.tasks.notify_upload_done')
def notify_upload_done(video_id):
    ...


notify_upload_done.delay(1)                         # exact name wins -> 'notifications'
transcode.delay(1)                                  # task option beats the glob -> 'gpu'
transcode.apply_async(args=(1,), queue='transcode_bulk')  # call wins -> 'transcode_bulk'
app.send_task('video.tasks.transcode', args=(1,))   # no task object -> 'transcode'

go deeper

for a junior

Know that a queue can be named in three places and that the value passed to apply_async wins over the others.

for a middle

Explain the order (call, task options, task_routes), first-match router semantics, and that exact names are matched before glob or regex patterns.

for a senior

Point out that send_task by name skips decorator options, so services enqueueing by name can route differently; keep routing in task_routes to avoid that drift.

for a principal

Argue for one routing authority in configuration, reviewed like any other contract, so a queue topology change is a single edit rather than a hunt through call sites.

Where the destination gets decided

When a Celery task is sent, the sending process (the producer) picks a queue before anything reaches the broker. The router is the code that makes that choice. Three sources can name a queue, and they can disagree:

Call-time options — transcode.apply_async(args=(42,), queue='transcode_bulk').
Task options — @app.task(queue='gpu'), stored as an attribute of the task class.
Routers — the task_routes setting: a dict, a list of (pattern, route) pairs, a router function, or a dotted path to one.

The Celery routing guide states the order: call arguments, then attributes defined on the task, then the routers in task_routes.

How the merge actually works

Reading celery/app/routes.py makes the order concrete:

Task.apply_async builds the message options from the task's own options (queue, routing_key, exchange, priority and more) and lets anything passed to the call override them.
Router.route() asks each router in task_routes, in order, and stops at the first one that returns a value.
The found route is merged with the message options using a left-precedence merge: a message option replaces the route's value unless that option is None.
If nothing named a queue, task_default_queue (by default celery) is used.

So a route from task_routes only fills gaps the caller and the task left open.

Inside a dict router

A dict in task_routes becomes a MapRoute:

An exact task name is looked up first, whatever its position in the dict.
Keys containing * are compiled as glob patterns; compiled re objects are used as regexes. Patterns are tried in the order they were given.
A route value may be a dict ({'queue': 'transcode', 'routing_key': 'transcode'}) or a bare string, which is read as a queue name.

When pattern order matters, the docs show the list-of-pairs form wrapped in a one-element tuple, task_routes = ([('video.tasks.*', {'queue': 'transcode'}), ...],), because a bare list is read as a list of separate routers.

Worked example: the video platform

call	task option	matching route	where it goes
`notify_upload_done.delay(...)`	none	exact name → `notifications`	`notifications`
`transcode.delay(...)`	`queue='gpu'`	glob `video.tasks.*` → `transcode`	`gpu`
`transcode.apply_async(..., queue='transcode_bulk')`	`queue='gpu'`	glob → `transcode`	`transcode_bulk`
`app.send_task('video.tasks.transcode')`	not consulted	glob → `transcode`	`transcode`

The last row surprises people. send_task publishes by name and has no task object, so decorator options are invisible to it; only task_routes and the call's own arguments count. A separate service that enqueues transcodes by name would land them in a different queue from the web app that imports the task.

Router functions

A router can be a function with the signature (name, args, kwargs, options, task=None, **kw). Returning None passes the decision to the next router; returning a dict or a queue name ends the search. That is how you route on arguments — for example, sending transcodes of files over 4 GB to a transcode_large queue while ordinary ones stay on transcode. List such a function before the dict in task_routes = (route_by_size, {...}), because the first router that returns something ends the search and the dict would otherwise answer first for every transcode.

A router can also be given as a dotted string, 'video.routers.route_by_size', which Celery imports lazily. Remember that routers run in the sending process — the web server, or another task — so they should be cheap and should avoid a database query on every send.

Debugging and practical guidance

When a task lands in an unexpected queue, walk the order from the top:

Search the call sites for apply_async(..., queue= and for send_task( with a queue= argument.
Check the task decorator for queue=, routing_key= or exchange= options.
Read task_routes in order, remembering first-match and exact-before-pattern.
Confirm the registered task name the route keys must match; a renamed module silently breaks an exact-name route.

Once the cause is found, the lasting fixes are usually the same:

Keep routing in task_routes, one place in configuration, so every producer agrees.
Treat apply_async(queue=...) as an exception for a specific call, not a routing strategy.
Avoid decorator queue= when other services call the task with send_task.
A router that raises an exception fails the send, so keep router functions trivial.

## Where the destination gets decided When a Celery task is sent, the sending process (the **producer**) picks a queue before anything reaches the broker. The **router** is the code that makes that choice. Three sources can name a queue, and they can disagree: - **Call-time options** — `transcode.apply_async(args=(42,), queue='transcode_bulk')`. - **Task options** — `@app.task(queue='gpu')`, stored as an attribute of the task class. - **Routers** — the `task_routes` setting: a dict, a list of `(pattern, route)` pairs, a router function, or a dotted path to one. The Celery routing guide states the order: call arguments, then attributes defined on the task, then the routers in `task_routes`. ## How the merge actually works Reading `celery/app/routes.py` makes the order concrete: 1. `Task.apply_async` builds the message options from the task's own options (`queue`, `routing_key`, `exchange`, `priority` and more) and lets anything passed to the call override them. 2. `Router.route()` asks each router in `task_routes`, in order, and stops at the **first one that returns a value**. 3. The found route is merged with the message options using a left-precedence merge: a message option replaces the route's value **unless that option is `None`**. 4. If nothing named a queue, `task_default_queue` (by default `celery`) is used. So a route from `task_routes` only fills gaps the caller and the task left open. ## Inside a dict router A dict in `task_routes` becomes a `MapRoute`: - An **exact** task name is looked up first, whatever its position in the dict. - Keys containing `*` are compiled as **glob** patterns; compiled `re` objects are used as **regexes**. Patterns are tried in the order they were given. - A route value may be a dict (`{'queue': 'transcode', 'routing_key': 'transcode'}`) or a bare string, which is read as a queue name. When pattern order matters, the docs show the list-of-pairs form wrapped in a one-element tuple, `task_routes = ([('video.tasks.*', {'queue': 'transcode'}), ...],)`, because a bare list is read as a list of separate routers. ## Worked example: the video platform | call | task option | matching route | where it goes | |---|---|---|---| | `notify_upload_done.delay(...)` | none | exact name → `notifications` | `notifications` | | `transcode.delay(...)` | `queue='gpu'` | glob `video.tasks.*` → `transcode` | `gpu` | | `transcode.apply_async(..., queue='transcode_bulk')` | `queue='gpu'` | glob → `transcode` | `transcode_bulk` | | `app.send_task('video.tasks.transcode')` | not consulted | glob → `transcode` | `transcode` | The last row surprises people. `send_task` publishes by **name** and has no task object, so decorator options are invisible to it; only `task_routes` and the call's own arguments count. A separate service that enqueues transcodes by name would land them in a different queue from the web app that imports the task. ## Router functions A router can be a function with the signature `(name, args, kwargs, options, task=None, **kw)`. Returning `None` passes the decision to the next router; returning a dict or a queue name ends the search. That is how you route on arguments — for example, sending transcodes of files over 4 GB to a `transcode_large` queue while ordinary ones stay on `transcode`. List such a function **before** the dict in `task_routes = (route_by_size, {...})`, because the first router that returns something ends the search and the dict would otherwise answer first for every transcode. A router can also be given as a dotted string, `'video.routers.route_by_size'`, which Celery imports lazily. Remember that routers run in the **sending** process — the web server, or another task — so they should be cheap and should avoid a database query on every send. ## Debugging and practical guidance When a task lands in an unexpected queue, walk the order from the top: - Search the call sites for `apply_async(..., queue=` and for `send_task(` with a `queue=` argument. - Check the task decorator for `queue=`, `routing_key=` or `exchange=` options. - Read `task_routes` in order, remembering first-match and exact-before-pattern. - Confirm the registered task name the route keys must match; a renamed module silently breaks an exact-name route. Once the cause is found, the lasting fixes are usually the same: - Keep routing in **`task_routes`**, one place in configuration, so every producer agrees. - Treat `apply_async(queue=...)` as an exception for a specific call, not a routing strategy. - Avoid decorator `queue=` when other services call the task with `send_task`. - A router that raises an exception fails the send, so keep router functions trivial.

How would you route transcodes of very large files to a different queue from ordinary ones?
Write a router function with the signature `(name, args, kwargs, options, task=None, **kw)` that inspects the arguments and returns `{'queue': 'transcode_large'}` for big files and `None` otherwise, then list it before the dict in `task_routes`. Returning `None` hands the decision to the next router.
Why does the list-of-pairs form of task_routes need to be wrapped in a tuple?
`task_routes` itself may be a list of routers. A bare list of `(pattern, route)` pairs would be read as several routers, one per pair. Wrapping it as `([...],)` makes the whole list one ordered `MapRoute`, so patterns are tried in the order written.

saying these in an interview costs you the question

task_routes always overrides whatever the caller passes
Dict order decides whether an exact name or a glob matches first
app.send_task by name honours the task decorator's queue option
Every router in task_routes is applied and their results combined

In Celery, when task_routes, a task's own queue option and apply_async(queue=) disagree, which one decides where the task is sent?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Where the destination gets decided

How the merge actually works

Inside a dict router

Worked example: the video platform

Router functions

Debugging and practical guidance

▾common follow-ups2 questions

▾red flags4 flags

Where the destination gets decided

How the merge actually works

Inside a dict router

Worked example: the video platform

Router functions

Debugging and practical guidance

answer

go deeper