skip to content

In Celery, why might apply_async(priority=9) fail to move password-reset emails ahead of queued transcodes, and how do RabbitMQ and Redis priorities differ?

level: seniorimportance: should knowfreq 30%

answer

  1. priority needs queue support
  2. one broker counts up, one down
  3. a queue argument must exist
  4. buckets in the Redis transport
  5. prefetched messages keep their order

basics

~20 s

On RabbitMQ, priority is ignored unless the queue was declared with x-max-priority, and higher numbers go first. On Redis, kombu emulates it with separate lists where 0 is served first, so 9 is the lowest priority.

solid answer

~40 s

Celery just attaches the number; the transport decides what it means. On RabbitMQ the queue must be declared with `x-max-priority` — via `queue_arguments={'x-max-priority': 10}` on a kombu `Queue`, or `task_queue_max_priority` for every queue Celery declares — otherwise priority is ignored; with it, higher numbers are delivered first. On Redis, kombu splits each queue into lists by `priority_steps` (default `[0, 3, 6, 9]`), clamps values to 0-9 and serves lower numbers first. A message sent without a priority counts as 0, the top bucket, so `priority=9` pushes a reset *behind* everything else. Either way, messages a worker has already prefetched are not reordered. For resets versus transcodes I'd use separate queues and keep priority for ordering inside a lane.

code

python · 12 lines
python
from kombu import Exchange, Queue

# RabbitMQ: the queue must allow priorities; higher number goes first
app.conf.task_queues = (
    Queue('transcode', Exchange('transcode'), routing_key='transcode',
          queue_arguments={'x-max-priority': 10}),
)
transcode.apply_async(args=(42,), priority=9)   # ahead of priority 0

# Redis: kombu emulates priority; 0 is served first
transcode.apply_async(args=(43,), priority=0)   # top bucket
transcode.apply_async(args=(44,), priority=9)   # served last

go deeper

for a junior

Remember that Celery's priority argument only works if the broker supports it, and that separate queues are the simpler way to keep fast tasks fast.

for a middle

Explain x-max-priority on RabbitMQ versus kombu's emulated lists on Redis, including which direction wins on each and where task_queue_max_priority fits.

for a senior

Name why configured priorities still misbehave: prefetch, no pre-emption, unprioritised Redis messages in the top bucket, and the cost of recreating a live RabbitMQ queue.

for a principal

Decide where priority is worth its broker coupling: isolate latency classes with queues, reserve priority for ordering within a lane, and document the direction for whoever changes brokers.

## What `priority` does in Celery `apply_async(priority=...)` (and the `priority` key in a `task_routes` entry, or `task_default_priority` for every task) attaches a number to the message. Celery itself does not reorder anything: the **transport** — the kombu driver for RabbitMQ, Redis or SQS — decides what the number means. That is why the same call behaves differently depending on the broker, and why "we set priority and nothing changed" is a common complaint. ## RabbitMQ: native, but opt-in per queue RabbitMQ supports priorities natively, but only on queues declared with the `x-max-priority` argument: - Per queue: `Queue('notifications', Exchange('notifications'), routing_key='notifications', queue_arguments={'x-max-priority': 10})`, or the kombu `Queue(..., max_priority=10)` shortcut. - For every queue Celery declares: `task_queue_max_priority = 10` (default `None`, meaning no priority support). - **Higher numbers are delivered first**, up to the queue's maximum. A queue already declared without the argument keeps its original arguments; RabbitMQ refuses to redeclare an existing queue with different ones. Adding priority to a live queue therefore means draining and recreating it, or moving to a new queue name. Two related settings matter on RabbitMQ. `task_default_priority` (default `None`) gives every task a priority unless the call or route sets one. `task_inherit_parent_priority` (default `False`) makes tasks called from inside a prioritised task, and later steps of a chain, carry the parent's priority — useful when a premium user's transcode fans out into follow-up work that should stay ahead of the batch backlog. ## Redis: emulated by kombu, and inverted Redis has no message priority, so kombu's Redis transport emulates it: 1. Each queue becomes several lists, one per entry in the `priority_steps` transport option — default `[0, 3, 6, 9]`, so `transcode` is really `transcode` plus three suffixed lists. 2. A message's priority is clamped to 0-9 and placed in the nearest step at or below it: 1 and 2 share the 0 list, 4 and 5 share the 3 list. 3. Workers poll the lists in step order, so **the lowest number is served first**. 4. A message with no priority is treated as **0**, the top bucket. The consequence is counter-intuitive: on Redis, `priority=9` on a password reset makes it the *last* thing served. To favour resets you demote the transcodes (`priority=6` or more), or set `task_default_priority` to a middle value such as 5 and send resets with `priority=0`. More granularity comes from `broker_transport_options = {'priority_steps': list(range(10))}` at the cost of more lists to poll. ## Comparison, including Amazon SQS kombu's SQS transport has no message-priority support at all, so on SQS urgent work always goes to a separate queue. | | RabbitMQ | Redis | SQS | |---|---|---|---| | mechanism | native, per queue | emulated lists per step | none | | must enable | `x-max-priority` / `task_queue_max_priority` | nothing (steps on by default) | — | | which number wins | highest | lowest (0) | — | | unprioritised messages | behind prioritised ones | in the top bucket | — | ## Why priorities still disappoint - **Prefetching.** A worker reserves several messages ahead of time. Those reserved messages are not reordered when a more urgent one arrives, so the Celery routing guide recommends lowering `worker_prefetch_multiplier`, ideally to 1, when priority matters. - **Running tasks are not pre-empted.** If every process is 15 minutes into a transcode, a top-priority reset still waits for one to finish. - **Shared-lane starvation remains.** Priority only reorders messages *within* one queue. - **Direction is easy to get wrong.** A config written for RabbitMQ and moved to Redis keeps working, but the tasks marked most urgent are now served last, and nothing logs a warning. ## What to do on the video platform Priority is a tool for **ordering**, not for **isolation**. The reset problem is an isolation problem: short tasks sharing capacity with very long ones. So: - Put transcodes and resets in **separate queues** with separate workers; that isolation does not depend on the broker. - Use priority **inside** a lane, for example so premium users' transcodes jump the batch backlog, and state the transport's direction in a comment beside the config. - If the broker may change, avoid encoding business meaning in raw priority numbers; the direction flips between RabbitMQ and Redis.

  • Why can a high-priority Celery task still wait behind lower-priority ones even with priorities configured correctly?
    Workers prefetch messages; the ones already reserved are not reordered when a more urgent message arrives, and running tasks are never pre-empted. The Celery docs suggest a `worker_prefetch_multiplier` of 1 when ordering matters, at some throughput cost for short tasks.
  • On Redis with default settings, do priority=1 and priority=2 behave differently from 0?
    No. With the default `priority_steps` of `[0, 3, 6, 9]`, a value maps to the highest step not above it, so 0, 1 and 2 all share the top list. Set `priority_steps` to `list(range(10))` in `broker_transport_options` for ten distinct levels.

saying these in an interview costs you the question

  • A higher priority number is always served first on every Celery broker
  • Setting priority on apply_async is enough on RabbitMQ
  • Redis supports message priorities natively
  • A top-priority task will interrupt a long task already running
  • Priorities remove the need for a separate queue for slow tasks