skip to content

Defining & Calling Tasks

2 roadmaps6 questions

Declaring tasks with @app.task or @shared_task, bound tasks, and calling them with delay() or apply_async() options. Interviewers probe what is serialised and when the call returns.

on this pageshow

explore

questions

6

In Celery, how does calling a task with delay() differ from apply_async(), and what does either call return to the caller?

level: juniorimportance: must knowfreq 65%

answer

  1. one is a shortcut for the other
  2. execution options live on one
  3. returns before any worker runs
  4. an AsyncResult carrying the task id

basics

~20 s

delay(*args, **kwargs) is a shortcut for apply_async(args, kwargs) with no execution options. Both serialize the arguments, publish one message to the broker and return an AsyncResult with the task id at once; neither waits for a worker.

solid answer

~40 s

`render_invoice_pdf.delay(order_id)` is exactly `render_invoice_pdf.apply_async((order_id,), {})`. I reach for `apply_async` when I need execution options: `countdown` or `eta` to start later, `expires` to drop a stale message, `queue`, `priority`, `serializer`, `ignore_result`, a custom `task_id`, or `link` callbacks. Either call checks the arguments against the task's signature, serializes them, publishes one message and returns an `AsyncResult` straight away, so the checkout request is not held up by PDF rendering. Calling the task like a plain function, `render_invoice_pdf(order_id)`, runs the body inline in the current process and sends nothing. The classic slip is `apply_async(order_id)`: `args` must be a tuple or list, so it is `apply_async((order_id,))`.

code

python · 14 lines
python
from celery import Celery

app = Celery('shop', broker='redis://localhost:6379/0')

@app.task
def render_invoice_pdf(order_id):
    ...

# In the checkout view, after the order is saved:
result = render_invoice_pdf.delay(42)                 # same as apply_async((42,))
later = render_invoice_pdf.apply_async((42,), countdown=30, expires=3600)
print(result.id)                                      # a UUID string; the task may not have started

render_invoice_pdf(42)                                # runs inline, sends no message

go deeper

for a junior

Recall that delay() is apply_async() without options, that both publish a message and return an AsyncResult at once, and that args go in a tuple.

for a middle

Explain the call-site steps: signature check, serialization, publish with retry, AsyncResult handle, and why a direct function call runs inline.

for a senior

Show how call-site failures surface in a web request: EncodeError, OperationalError after publish retries, and why blocking on get() in a view is a design smell.

for a principal

Frame the calling API as a contract: which options a team standardises on, such as expires for time-sensitive work, and how call failures are handled when the broker is down.

## What a Celery call actually does A **Celery task** is a Python function registered with a Celery application, for example with `@app.task`. The function's body runs in a separate **worker** process, usually on another machine. The web process that wants the work done is the **producer**: it does not run the body, it sends a **message** through a **broker** (RabbitMQ, Redis or Amazon SQS) that says "run the task named X with these arguments". A worker consumes that message and executes the body. In an online shop, checkout should answer the customer in milliseconds, while rendering the invoice PDF and sending the confirmation email can take seconds. So the view calls: - `send_order_confirmation.delay(order.id)` - `render_invoice_pdf.delay(order.id)` and returns the response. Both calls come back after publishing, long before any worker touches the job. ## delay() versus apply_async() `delay()` is defined in `celery/app/task.py` as one line: `return self.apply_async(args, kwargs)`. It is a convenience that mirrors a normal function call. `apply_async()` is the full calling API. | | `delay(*args, **kwargs)` | `apply_async(args, kwargs, **options)` | |---|---|---| | Task arguments | star arguments, like a function call | a tuple or list, plus a dict | | Execution options | none | `countdown`, `eta`, `expires`, `queue`, `priority`, `serializer`, `ignore_result`, `task_id`, `link`, `link_error`, `headers`, and more | | Returns | `AsyncResult` | `AsyncResult` | Because `delay()` forwards every keyword to the task itself, `render_invoice_pdf.delay(42, countdown=60)` does **not** schedule anything: it tries to pass `countdown=60` as a task argument. With argument checking on (the default), that raises `TypeError` in the caller. ## What comes back: AsyncResult Both methods return a `celery.result.AsyncResult`, a lightweight handle holding the **task id** (a UUID4 unless you pass `task_id`). It is not the task's return value. With a result backend configured, the handle can later report the state or fetch the value; how results are stored and read is its own topic. Two consequences matter in a request handler: 1. The call is non-blocking; the handler can store or return `result.id` and move on. 2. Blocking on the result inside the web request (`result.get()`) throws away the benefit and ties the request to worker latency. With `ignore_result=True` (per task or per call) the handle is still returned, but it is marked as ignored: its `get()` returns `None` immediately, and nothing is stored for it. ## Calling the function directly, apply() and eager mode - `render_invoice_pdf(42)` calls `Task.__call__`: the body runs **inline** in the current process and no message is sent. - `render_invoice_pdf.apply((42,))` also runs locally and returns an `EagerResult`, which is mostly useful in tests. - `task_always_eager=True` makes `apply_async()` execute locally instead of publishing; it is a test convenience, never a production setting. ## Failure modes at the call site The call does real work before it returns, and each step can raise in the producer: - **Signature check.** Celery apps default to `strict_typing=True`, so a wrong number of arguments raises `TypeError` in the caller, not later on the worker. `@app.task(typing=False)` turns the check off. - **Serialization.** An argument the serializer cannot encode raises `kombu.exceptions.EncodeError` from the call. - **Broker unreachable.** Publishing is retried by default (`task_publish_retry=True`, with a policy of up to 3 retries); after that the call raises `kombu.exceptions.OperationalError`, so a checkout view should decide what to do when the queue is down. ## Choosing in practice Use `delay()` when you only pass arguments, which is most calls. Switch to `apply_async()` the moment you need an option; the invoice task might run with `apply_async((order.id,), expires=3600)` so a backlog does not render invoices hours late. Keep argument lists small and serializable (identifiers, not objects), and never block a request on the result. ## Common mistakes - **Passing a bare value as `args`.** `apply_async(42)` is not a call with one argument; `args` must be a sequence, so write `apply_async((42,))` or `apply_async([42])`. - **Hiding options inside `delay()`.** Any keyword given to `delay()` is a task argument. Scheduling, routing and expiry need `apply_async()`. - **Reading the AsyncResult as success.** A returned handle only proves the message was published; the task can still fail, be revoked, or never be picked up. - **Testing only with direct calls.** `render_invoice_pdf(42)` skips serialization and the broker, so a test that calls the function directly can pass while `delay()` would raise `EncodeError` in production. - **Enqueueing the same work twice.** Two `delay()` calls produce two messages with two different ids; Celery does not merge them, so a retried checkout request can render two invoices unless the task checks what already exists.

  • What happens at the delay() call if the Celery broker is down?
    The producer retries the publish because `task_publish_retry` defaults to `True`, following `task_publish_retry_policy` (up to 3 retries with short intervals). If the broker is still unreachable, the call raises `kombu.exceptions.OperationalError` in the caller. A checkout view has to decide whether to fail the request, fall back, or record the job for later.
  • What does passing ignore_result=True to apply_async change about the returned AsyncResult?
    You still get an `AsyncResult` with the task id, but it is marked as ignored and the worker stores no result for that execution. Its `get()` returns `None` immediately instead of waiting. It suits fire-and-forget work like the confirmation email, where nobody reads the return value.
  • Why can a wrong argument count fail in the web process rather than on the worker?
    Celery apps default to `strict_typing=True`, so `apply_async` checks the arguments against the task function's signature before publishing and raises `TypeError` in the caller. Setting `typing=False` on the task skips the check; the bad call is then published and fails later on the worker.

saying these in an interview costs you the question

  • delay() waits for the worker and returns the task's return value.
  • apply_async takes the arguments directly, as in apply_async(order_id).
  • Passing countdown=60 to delay() schedules the task a minute later.
  • Calling render_invoice_pdf(order_id) as a function still sends it to a worker.
  • The AsyncResult means the task has already finished successfully.
open as a page

What does bind=True change about a Celery task, and what can a bound task read from self.request while it runs?

level: middleimportance: should knowfreq 40%

basics

~20 s

With bind=True, Celery passes the task object itself as the first argument, self. Through self.request a running task reads its execution context: task id, retry count, args and kwargs, worker hostname, delivery info, eta and expires.

open as a page

In Celery 5.6, which task argument types survive the default JSON serializer, and what does switching a task to pickle cost?

level: middleimportance: should knowfreq 45%

basics

~20 s

Celery's default JSON carries strings, numbers, booleans, None, lists and dicts, plus datetime, Decimal, UUID and bytes, which kombu tags and restores; anything else raises EncodeError at the call. Pickle carries more but lets broker writers run code on workers.

open as a page

Why would a Celery worker log 'Received unregistered task of type' for an invoice task, and how does Celery name tasks by default?

level: middleimportance: should knowfreq 35%

basics

~20 s

A Celery message carries only the task's name, looked up in the worker's own registry. Names default to module path plus function name, so a worker that never imported the module, or imported it under another path, rejects it as unregistered.

open as a page

In Celery, where does a task sent with apply_async(countdown=...) wait before it runs, and why is a countdown of hours risky?

level: seniorimportance: should knowfreq 35%

basics

~20 s

apply_async turns countdown into an eta and publishes at once. On Redis or a classic RabbitMQ queue a worker takes the message immediately and holds it unacknowledged in memory until due, so hours-long waits pile up and can run twice.

open as a page

A Celery invoice task declared with rate_limit='10/m' starts about 40 times a minute across four workers; why, and what does rate_limit actually enforce?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Celery's rate_limit is a token bucket kept separately by each worker instance for each task type, not a cluster-wide limit. Four workers at '10/m' allow about 40 starts a minute; tasks over the limit wait inside the worker rather than failing.

open as a page