skip to content

In Celery, what does AsyncResult.get() do to the caller waiting on it, and what do its timeout and propagate arguments and forget() control?

level: middleimportance: should knowfreq 45%

answer

  1. the caller blocks, the worker does not care
  2. timeout=None means no limit
  3. failures re-raise in the caller
  4. release what the backend holds

basics

~20 s

Celery's AsyncResult.get() blocks the caller until the task finishes, forever by default. timeout raises TimeoutError without stopping the task; propagate re-raises the task's exception in the caller; forget() deletes the stored result from the backend.

solid answer

~40 s

`AsyncResult.get()` turns asynchronous work back into a blocking call: the caller waits until the task reaches `SUCCESS`, `FAILURE` or `REVOKED`. `timeout` defaults to `None`, so without it a payouts request handler can hang for as long as the queue is backed up; with `timeout=5` the caller gets `celery.exceptions.TimeoutError`, while the task itself keeps running on the worker. `propagate=True` (the default) re-raises the task's exception in the caller; `propagate=False` returns the exception instance instead. `forget()` deletes the stored result (and its parents') from the backend, and Celery's docstring asks callers to eventually call `get()` or `forget()` on every `AsyncResult` so backend resources are released. For a status endpoint, poll `ready()` or `state` instead of blocking.

code

python · 15 lines
python
from celery.exceptions import TimeoutError

from payouts.tasks import send_payout  # an @app.task

result = send_payout.delay(42)

try:
    receipt = result.get(timeout=5)  # blocks at most 5 seconds
except TimeoutError:
    receipt = None  # the task is still queued or running on a worker
else:
    result.forget()  # drop the stored result now that it has been read

# failures as values instead of exceptions
outcome = result.get(timeout=5, propagate=False)

go deeper

for a junior

Recall that get() waits for the result and blocks, and that it needs a result backend to work at all.

for a middle

Explain timeout, TimeoutError and the fact that the task keeps running, plus what propagate and forget() each change.

for a senior

Keep get() out of request paths: return an id, poll ready() or state, bound any unavoidable wait, and release results with forget() or ignore_result.

for a principal

Treat synchronous waiting on queued work as a design smell, and set rules for when callers may block and how long results are retained.

## What get() is for Calling `send_payout.delay(42)` returns an `AsyncResult` at once: a task id plus a handle to the configured **result backend**. `get()` is the method that waits for the outcome. It needs a result backend; with none configured it raises `NotImplementedError`. Its signature starts `get(timeout=None, propagate=True, interval=0.5, ...)`. Each argument changes what the **caller** experiences; none of them changes what the **worker** does. ## Blocking and timeout `get()` returns only when the task is in a ready state (`SUCCESS`, `FAILURE` or `REVOKED`). - With the default `timeout=None` it waits indefinitely. If the payouts queue is backed up for twenty minutes, a web request that calls `get()` holds its thread or process for twenty minutes. - With `timeout=5`, the caller gets `celery.exceptions.TimeoutError` after five seconds. The docstring is explicit that this is a client-side limit: **the task is not terminated**, and it will still pay out when a worker reaches it. - A caller-side timeout is not a task time limit; stopping a task that runs too long is a worker setting. `interval` (0.5 seconds) is the polling period for backends that poll, such as a database. The Redis and RPC backends are notified instead of polling, so `interval` has no effect on them. ## propagate When the task failed, the stored result is the exception. | Call | Task succeeded | Task raised `ValueError` | |---|---|---| | `get()` | returns the return value | re-raises `ValueError` in the caller | | `get(propagate=False)` | returns the return value | returns the `ValueError` instance | `REVOKED` also propagates. With the default settings, errors from parent tasks in a chain are re-raised too. ## forget() and releasing results A stored result occupies the backend until it expires under `result_expires` (one day by default) or is removed. `forget()` removes the result for this task and its parents immediately. The `get()` docstring carries a warning worth quoting in an interview: backends use resources to store and transmit results, so you must eventually call `get()` or `forget()` on every `AsyncResult` returned by a task call. Practical uses: 1. A caller that read a large return value calls `forget()` so the payload does not sit in Redis for a day. 2. A caller that no longer needs a result it will never read calls `forget()` to release it early. 3. Tasks whose results nobody ever reads are better declared with `ignore_result=True`, so nothing is stored in the first place; `get()` on such a result returns `None` immediately. Not every backend implements `forget()`; one that does not raises `NotImplementedError`. ## A worked request flow A payouts API that never blocks on Celery looks like this: 1. `POST /payouts` creates the `Payout` row, sends `send_payout.delay(payout.id)`, stores the returned task id on the row, and answers 202 with the payout id. 2. The worker pays out and marks the row as paid or failed. 3. `GET /payouts/<id>` reads the row; if more detail is wanted, it reads `AsyncResult(task_id).state` without calling `get()`. 4. The stored result expires under `result_expires`, or the task sets `ignore_result=True` because the row already records the outcome. No step waits on the queue, so a backlog slows payouts but never exhausts web workers. ## Why not to block in a web request A payouts API that calls `get()` inside the request ties request capacity to queue depth, which defeats the point of queuing the work. Better patterns: - Return the task id (or better, the payout id) with an HTTP 202, and let the client poll a status endpoint. - In that endpoint, read `AsyncResult(task_id).ready()`, `.successful()` or `.state` without blocking. - Remember that an unknown or already-expired task id reads as `PENDING`, so a status page that trusts `PENDING` forever can mislead users. Calling `get()` from inside another task is a separate hazard that Celery blocks by default (`disable_sync_subtasks=True` raises `RuntimeError`); it belongs to workflow design rather than to reading results. The interview-ready summary: `get()` blocks the caller, `timeout` bounds the wait without cancelling the work, `propagate` decides between raising and returning the error, and `forget()` or `result_expires` decides how long the backend keeps the result.

  • Why does AsyncResult(task_id).state say PENDING for a task id that never existed?
    Celery keeps no registry of issued task ids. PENDING is defined as unknown: the backend has no record for that id, whether because the task is still queued, the result expired under `result_expires`, it was forgotten, results are ignored, or the id is simply wrong. A status endpoint should treat long-lived PENDING as unknown, not as queued.
  • Does get(timeout=...) interact with the task's own time limits?
    No. The `timeout` argument is enforced in the caller and only stops the caller waiting. Limits that stop a task running too long are configured on the task or worker and enforced there. A caller can time out after five seconds while the task runs for another hour and completes normally.

saying these in an interview costs you the question

  • get(timeout=5) cancels the task on the worker when the five seconds run out.
  • get() returns PENDING straight away if the task has not finished.
  • forget() revokes the task so no worker will run it.
  • Calling get() in a request handler is fine because Celery runs it asynchronously.
  • propagate=False makes a failed task return None from get().