skip to content

In Celery, how do you list what each worker is running right now, and what do inspect active, reserved and scheduled each show?

level: juniorimportance: should knowfreq 35%

answer

  1. a broadcast question to every worker
  2. the worker's memory, not the queue
  3. running vs prefetched vs ETA-held
  4. a one-second reply window

basics

~20 s

celery -A proj inspect active asks every worker, over the broker, which tasks it is executing. inspect reserved lists tasks it prefetched but has not started, inspect scheduled the ETA or countdown tasks it holds; messages still queued appear in none.

solid answer

~40 s

`celery -A proj inspect <command>` broadcasts a remote-control request through the broker and prints each worker's reply. `active` lists the tasks a worker is executing now, with `id`, `name`, `args`, `time_start`, `worker_pid` and whether the message is `acknowledged`. `reserved` lists tasks it has already pulled from the broker but not started, and `scheduled` lists ETA or countdown tasks held in its timer. None of them shows messages still sitting in the queue, only what workers have received. Workers get 1 second to answer by default (`--timeout`), `-d` targets named nodes, and `celery -A proj status` just pings who is online. Remote control works on the RabbitMQ and Redis transports, not on SQS. For a hung PDF export, `inspect active` gives you the task id and the child pid to act on.

code

bash · 5 lines
bash
celery -A proj status
celery -A proj inspect active
celery -A proj inspect reserved -d celery@worker-pdf-1
celery -A proj inspect scheduled --timeout 5
celery -A proj inspect active --json

go deeper

for a junior

Recall the three lists: active is running now, reserved is prefetched but not started, scheduled is ETA or countdown tasks a worker holds. Know that celery -A proj status pings workers.

for a middle

Explain that inspect is a broadcast answered from each worker's memory, so queued messages never appear, and that the one-second timeout makes a slow worker look absent.

for a senior

Use inspect active to find a stuck task's id, node and worker_pid before acting, and know where remote control does not exist, such as SQS or workers with remote control disabled.

for a principal

Decide what monitoring relies on remote control and what must come from the broker and logs, especially if the platform might move to a transport without remote control.

## What `celery inspect` is Celery workers listen on a **remote-control** channel: a broadcast on the broker that every worker consumes alongside its task queues. `celery -A proj inspect <command>` publishes one request on that channel, waits for replies, and prints them per worker node (for example `celery@worker-pdf-1`). Nothing is read from the broker's task queues or from the result backend: every answer comes from a worker's **own memory** of what it has received. The CLI has two families built on the same channel: - **`inspect`** commands are read-only questions: `active`, `reserved`, `scheduled`, `stats`, `registered`, `revoked`, `active_queues`, `query_task`. - **`control`** commands change something: `revoke`, `terminate`, `time_limit`, `enable_events`, `shutdown`, `pool_grow`. `celery -A proj status` is the lightest check of all: it pings every worker and lists the nodes that answered. ## The three task lists | Command | What it lists | Where those tasks are | |---|---|---| | `inspect active` | Tasks executing right now | Running in a pool process or thread | | `inspect reserved` | Tasks received but not started | Prefetched into the worker, waiting for a free slot | | `inspect scheduled` | Tasks with an ETA or countdown | Held in the worker's timer until their time comes | Each entry carries the task `id`, `name`, `args` and `kwargs`, the worker `hostname`, `time_start` (a Unix timestamp set once the task starts), whether the message is `acknowledged`, the `delivery_info` it arrived with, and the `worker_pid` of the pool process running it. `scheduled` wraps the same request data together with its `eta` and a priority. ## What none of them can show A message still waiting in the broker's queue has not reached any worker, so **it appears in no list**. If the PDF-export queue holds 3,000 messages and each worker has prefetched a handful, `inspect reserved` shows only that handful. Queue depth is a question for the broker, not for `inspect`. Two more blind spots matter in practice: - **Periodic tasks are not "scheduled" here.** `inspect scheduled` means ETA and countdown messages a worker already holds, not the beat schedule. - **Silence is not death.** Workers get **1 second** by default (`--timeout`) to reply. A worker on a slow network, or one whose main process is itself blocked (the `solo` pool running a stuck task, for instance), simply drops out of the output. Retry with a longer timeout before concluding anything. ## Finding the hung PDF export Exports have stalled. With the default prefork pool the worker's main process keeps answering remote control even while a child process is stuck, so: 1. Run `celery -A proj status` to see which nodes are up. 2. Run `celery -A proj inspect active` and look for `exports.render_pdf` entries whose `time_start` is far older than a normal export takes. 3. Note the task `id`, the node name and the `worker_pid`. The id is what `revoke` and `terminate` act on; the pid tells you which child process to look at with ordinary process tools. 4. Run `celery -A proj inspect reserved -d celery@worker-pdf-1` to see how many exports on that node are stuck behind it, waiting for a free slot. Add `--json` to any of these to feed the replies to a script, or call the same thing from Python with `app.control.inspect(timeout=2.0).active()`, which returns a dict keyed by node name, or `None` when nobody replied. ## Other inspect commands worth knowing - `inspect stats`: the worker's pool and process details, broker connection, resource usage and running totals per task name. - `inspect registered`: the task names this worker can run, the fastest check when a task is never picked up. - `inspect active_queues`: the queues each worker consumes from. - `inspect query_task <id>`: whether a given id is active or reserved on a worker, with its details. - `inspect revoked`: the ids this worker currently treats as revoked. ## Transport limits Remote control needs a broadcast the transport can deliver. The CLI documents `inspect` for the **RabbitMQ and Redis** transports, and Celery's SQS documentation states that SQS does not support worker remote control commands, so on SQS an `inspect` call gets no replies. Setting `worker_enable_remote_control=False` switches the channel off on any transport. On those setups you are left with worker logs and the broker's own queue metrics.

  • A worker is clearly processing tasks but is missing from celery inspect output; what do you check?
    Missing from `inspect` output means no reply arrived within the timeout, 1 second by default, not that the worker is dead. Retry with `--timeout 5` and with `-d` naming the node. If it still stays silent, check whether its main process can answer at all: a `solo` pool running a blocked task cannot, and `worker_enable_remote_control=False` or an SQS broker means no remote control exists.
  • How do you find out how many PDF exports are still waiting, since inspect cannot see the queue?
    Ask the broker, not the workers. `inspect reserved` only counts what workers have prefetched; the backlog is the queue's message count, read from the broker's own tooling or metrics. Add the reserved counts from every worker to it to get the full picture of work not yet started.

saying these in an interview costs you the question

  • inspect active lists every task waiting in the broker queue
  • inspect reserved shows the tasks that are executing right now
  • inspect scheduled lists the periodic entries beat will send
  • No reply within the timeout proves the worker process is dead
  • celery inspect works the same on the Amazon SQS transport