Why does a Celery task's `AsyncResult.state` still read `PENDING` while a worker is running it, and which states can the task report?

middleshould knowfreq 36%

answer

states live in the result backend
unknown id looks like waiting
STARTED is opt-in
RETRY is not a ready state

basics

~20 s

Celery writes STARTED only when task_track_started, or the task's track_started, is enabled, and it is off by default; PENDING just means the backend has no record of the id. Built-in states: PENDING, STARTED, RETRY, SUCCESS, FAILURE, REVOKED.

solid answer

~40 s

A task's state is whatever the result backend holds for its id, and PENDING is what Celery returns when it holds nothing: the task may be queued, running untracked, never sent, mistyped, expired, or run with ignore_result=True. STARTED is stored only when task_track_started or the task's track_started=True is set; both default to False, so a running label task reads PENDING until it finishes. After that the backend holds SUCCESS with the return value, FAILURE with the exception and traceback, RETRY with the exception that caused a retry, or REVOKED. SUCCESS, FAILURE and REVOKED are the ready states; RETRY is not, so ready() stays False while another attempt is pending. Custom states such as PROGRESS come from self.update_state().

A task's state is whatever the result backend holds for its id, and `PENDING` is what Celery returns when it holds nothing: the task may be queued, running untracked, never sent, mistyped, expired, or run with `ignore_result=True`. `STARTED` is stored only when `task_track_started` or the task's `track_started=True` is set; both default to `False`, so a running label task reads `PENDING` until it finishes. After that the backend holds `SUCCESS` with the return value, `FAILURE` with the exception and traceback, `RETRY` with the exception that caused a retry, or `REVOKED`. `SUCCESS`, `FAILURE` and `REVOKED` are the ready states; `RETRY` is not, so `ready()` stays `False` while another attempt is pending. Custom states such as `PROGRESS` come from `self.update_state()`.

code

python · 18 lines

python

from celery import Celery

from shipping.client import LabelClient

app = Celery("shop", broker="amqp://guest@localhost//", backend="redis://localhost/1")
app.conf.task_track_started = True   # store STARTED; default is False


@app.task(bind=True)
def create_shipping_label(self, shipment_id):
    self.update_state(state="REQUESTING_LABEL", meta={"shipment": shipment_id})
    return LabelClient().create(shipment_id)


# caller side
res = create_shipping_label.delay(42)
res.state    # 'PENDING' while queued, then 'STARTED', 'REQUESTING_LABEL', 'SUCCESS'
res.ready()  # False until SUCCESS, FAILURE or REVOKED

go deeper

for a junior

Recall the built-in states and that PENDING is what an unknown id returns. Know that STARTED needs task_track_started.

for a middle

Explain why a running task reads PENDING, why RETRY is not a ready state, and how update_state writes a custom progress state.

for a senior

Show the operational reading: PENDING is ambiguous, Reject and Ignore store nothing, and pollers need timeouts rather than trusting the state forever.

for a principal

Weigh what state reporting is worth: per-task backend writes for STARTED and progress against the users or systems that actually read them.

Where task states come from

A Celery task's state lives in the result backend — a store such as Redis or a database that workers write to and callers read from. Calling create_shipping_label.delay(42) returns an AsyncResult that holds little more than the task id; each time you read result.state, Celery asks the backend for that id's latest record. With no result backend configured there is nothing to read, and states are not available at all.

The built-in states are defined in celery.states:

State	Stored when	Meaning
`PENDING`	not stored by the worker; returned for unknown ids	Waiting, or Celery has no record of the id
`STARTED`	when a worker begins the task, only if tracking is on	Running; the metadata holds the worker's `pid` and `hostname`
`RETRY`	when a retry is scheduled	The result holds the exception that caused it
`SUCCESS`	when the task returns	The result holds the return value
`FAILURE`	when the task raises, or its retries run out	The result holds the exception and traceback
`REVOKED`	when the task is revoked	It will not run, or was terminated

Why a running task still says `PENDING`

task_track_started defaults to False, and so does the per-task option track_started. Celery's reasoning is that most callers only need to know whether a task is waiting, finished or being retried. So by default the worker writes nothing when it starts create_shipping_label; the backend still has no record, and state returns PENDING for the whole run — even if the carrier call takes a minute. Enabling tracking makes the worker store STARTED first, unless the task ignores its results:

python

app.conf.task_track_started = True

PENDING covers every case where the backend has no record, not only a queued task:

the id was never sent, or it has a typo;
the task runs with ignore_result=True, so nothing is stored;
the stored result has expired and been removed;
the task ended by raising Reject or Ignore, which store no state, so the id keeps whatever it had before.

So PENDING means "Celery has no record of this id", not "it is definitely queued".

Ready states and retries

SUCCESS, FAILURE and REVOKED form celery.states.READY_STATES: the task will not change again, and result.ready() returns True. RETRY is deliberately not ready. A retried task runs again under the same id, so ready() stays False, and a later attempt overwrites the record with its own outcome. A page showing a label's progress should treat RETRY as "still in flight", show result.info — the exception — as the reason, and wait for SUCCESS or FAILURE.

The order of events for a label task that times out once and then succeeds, with tracking on, is:

PENDING while the message waits in the queue;
STARTED when a worker picks it up;
RETRY when the carrier times out and self.retry() schedules another attempt;
STARTED again when a worker runs the retry;
SUCCESS with the label's return value.

With tracking off, steps 2 and 4 disappear and the caller sees PENDING, RETRY, SUCCESS.

Custom states for progress

Inside a bound task, self.update_state(state="REQUESTING_LABEL", meta={...}) writes any state name you choose; the meta dict becomes result.info. Custom states are ordinary strings: Celery does not count them as ready, so ready() stays False until a built-in terminal state replaces them. Like every state, they need a result backend, and each call is one more write to it.

States that live in events, not the backend

celery.states also defines RECEIVED, REJECTED and IGNORED, and the worker does not write them to the result backend. RECEIVED and REJECTED come from the event stream — the task-received and task-rejected events that a monitor such as Flower reads when the worker sends events. Flower can therefore show a task as rejected while AsyncResult.state for the same id still says PENDING. IGNORED is used inside the worker only.

Reading states well

Turn on task_track_started where people watch long tasks; leave it off for high-volume short tasks, where the extra write per task buys little.
Never treat PENDING as proof that a task is queued.
Pass exc= when retrying, so each RETRY record says why.
Give pollers a timeout: a task that ended in Reject or Ignore never reaches a ready state.

## Where task states come from A Celery task's state lives in the **result backend** — a store such as Redis or a database that workers write to and callers read from. Calling `create_shipping_label.delay(42)` returns an `AsyncResult` that holds little more than the task id; each time you read `result.state`, Celery asks the backend for that id's latest record. With no result backend configured there is nothing to read, and states are not available at all. The built-in states are defined in `celery.states`: | State | Stored when | Meaning | |---|---|---| | `PENDING` | not stored by the worker; returned for unknown ids | Waiting, or Celery has no record of the id | | `STARTED` | when a worker begins the task, **only if tracking is on** | Running; the metadata holds the worker's `pid` and `hostname` | | `RETRY` | when a retry is scheduled | The result holds the exception that caused it | | `SUCCESS` | when the task returns | The result holds the return value | | `FAILURE` | when the task raises, or its retries run out | The result holds the exception and traceback | | `REVOKED` | when the task is revoked | It will not run, or was terminated | ## Why a running task still says `PENDING` `task_track_started` defaults to `False`, and so does the per-task option `track_started`. Celery's reasoning is that most callers only need to know whether a task is waiting, finished or being retried. So by default the worker writes nothing when it starts `create_shipping_label`; the backend still has no record, and `state` returns `PENDING` for the whole run — even if the carrier call takes a minute. Enabling tracking makes the worker store `STARTED` first, unless the task ignores its results: ```python app.conf.task_track_started = True ``` `PENDING` covers every case where the backend has no record, not only a queued task: - the id was never sent, or it has a typo; - the task runs with `ignore_result=True`, so nothing is stored; - the stored result has expired and been removed; - the task ended by raising `Reject` or `Ignore`, which store no state, so the id keeps whatever it had before. So `PENDING` means "Celery has no record of this id", not "it is definitely queued". ## Ready states and retries `SUCCESS`, `FAILURE` and `REVOKED` form `celery.states.READY_STATES`: the task will not change again, and `result.ready()` returns `True`. `RETRY` is deliberately not ready. A retried task runs again under the **same id**, so `ready()` stays `False`, and a later attempt overwrites the record with its own outcome. A page showing a label's progress should treat `RETRY` as "still in flight", show `result.info` — the exception — as the reason, and wait for `SUCCESS` or `FAILURE`. The order of events for a label task that times out once and then succeeds, with tracking on, is: 1. `PENDING` while the message waits in the queue; 2. `STARTED` when a worker picks it up; 3. `RETRY` when the carrier times out and `self.retry()` schedules another attempt; 4. `STARTED` again when a worker runs the retry; 5. `SUCCESS` with the label's return value. With tracking off, steps 2 and 4 disappear and the caller sees `PENDING`, `RETRY`, `SUCCESS`. ## Custom states for progress Inside a bound task, `self.update_state(state="REQUESTING_LABEL", meta={...})` writes any state name you choose; the `meta` dict becomes `result.info`. Custom states are ordinary strings: Celery does not count them as ready, so `ready()` stays `False` until a built-in terminal state replaces them. Like every state, they need a result backend, and each call is one more write to it. ## States that live in events, not the backend `celery.states` also defines `RECEIVED`, `REJECTED` and `IGNORED`, and the worker does not write them to the result backend. `RECEIVED` and `REJECTED` come from the **event stream** — the `task-received` and `task-rejected` events that a monitor such as Flower reads when the worker sends events. Flower can therefore show a task as rejected while `AsyncResult.state` for the same id still says `PENDING`. `IGNORED` is used inside the worker only. ## Reading states well - Turn on `task_track_started` where people watch long tasks; leave it off for high-volume short tasks, where the extra write per task buys little. - Never treat `PENDING` as proof that a task is queued. - Pass `exc=` when retrying, so each `RETRY` record says why. - Give pollers a timeout: a task that ended in `Reject` or `Ignore` never reaches a ready state.

Why is `RETRY` not one of Celery's ready states?
A retried task is not finished: `self.retry()` has published another attempt under the same task id, and that attempt will overwrite the record with `STARTED`, `RETRY`, `SUCCESS` or `FAILURE`. Counting `RETRY` as ready would let a caller stop waiting while work is still queued, so `ready()` stays `False` and the record holds the exception that caused the retry.
What does enabling `task_track_started` cost in Celery?
One extra write to the result backend for every task execution, storing `STARTED` with the worker's `pid` and `hostname`. For a few long label tasks that is nothing; for millions of short tasks it adds backend load and latency for a state nobody reads. It is also skipped for tasks that ignore their results, since there is nowhere to show it.

saying these in an interview costs you the question

PENDING means the task is definitely sitting in the queue waiting for a worker.
Celery stores STARTED for every task as soon as a worker picks it up.
A task in the RETRY state is finished, so result.ready() returns True.
A task that raised Reject is stored with the REJECTED state in the result backend.
Task states are available even without a result backend configured.

Why does a Celery task's `AsyncResult.state` still read `PENDING` while a worker is running it, and which states can the task report?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Where task states come from

Why a running task still says PENDING

Ready states and retries

Custom states for progress

States that live in events, not the backend

Reading states well

▾common follow-ups2 questions

▾red flags5 flags

Where task states come from

Why a running task still says PENDING

Ready states and retries

Custom states for progress

States that live in events, not the backend

Reading states well

answer

go deeper

Why a running task still says `PENDING`

Why a running task still says `PENDING`