In Celery, what is a task signature, and how do .s() and .si() differ when the signature runs inside a chain?
answer
- a call packed as data
- task name, args, kwargs, options
- later arguments go in front
- immutable skips the parent result
basics
~20 sA Celery signature is one task call packed as data: task name, args, kwargs and execution options. In a chain, a .s() signature gets the previous step's result prepended to its arguments; an immutable .si() signature ignores it.
solid answer
~40 sA signature (`celery.Signature`, a `dict` subclass) wraps one task invocation: the task name, its args and kwargs, and execution options such as `countdown`. Because it is plain data it can be serialized, passed to another task, or composed with `chain`, `group` and `chord`. `task.s(*args)` builds a **partial** signature: arguments supplied later, including a parent task's return value in a chain or a `link` callback, are **prepended** to the stored ones, and kwargs are merged. `task.si(*args)` builds an **immutable** signature, the same as `signature(..., immutable=True)`: it runs with exactly the arguments it was given and the parent's result is dropped; only execution options can still be changed. In a photo pipeline, `make_thumbnail.s(640)` receives the photo id from the step before, while `notify_owner.si(user_id)` just runs.
code
python · 22 linesfrom celery import Celery
app = Celery('photos', broker='redis://localhost:6379/0')
@app.task
def save_original(upload_id):
return 42 # the new photo_id
@app.task
def make_thumbnail(photo_id, size):
return f'thumbs/{photo_id}_{size}.jpg'
@app.task
def notify_owner(user_id):
print(f'photo ready for user {user_id}')
workflow = (
save_original.s('up-7')
| make_thumbnail.s(640) # runs make_thumbnail(42, 640)
| notify_owner.si(1001) # runs notify_owner(1001), result dropped
)
workflow.delay()go deeper
Recall that a signature is a task call stored as data, that .s() is a partial and .si() is immutable, and that .delay() sends it.
Explain the merge rule: later args are prepended, kwargs and options merged with new values winning, and why that dictates parameter order in a chain.
Show you design task parameters for composition: parent result first, bound arguments after, .si() for steps that need no input, and .set() for options.
Discuss signatures as a serializable workflow description: what it buys in composability and what it costs when task parameter lists change under running workflows.
## What a signature is A **signature** is Celery's name for *one task call that has not been sent yet*. It is an instance of `celery.Signature`, which subclasses Python's `dict`, and it stores four things: - the **task name** (for example `photos.make_thumbnail`), - the positional **args** and the keyword **kwargs**, - the **execution options** (`countdown`, `queue`, `link` and so on), - an **immutable** flag. Because a signature is data rather than a running call, it can travel inside a message, be stored, be handed to another function, or be combined with other signatures. Every canvas primitive (`chain`, `group`, `chord`, `chunks`) is built from signatures, and every callback passed as `link` or `link_error` is one. A signature does nothing until you send it with `.delay()` or `.apply_async()`. Calling it directly, `sig()`, is different again: `Signature.__call__` runs the task function **in the current process**, with no message and no worker. ## Ways to build one | Form | What it produces | |---|---| | `make_thumbnail.signature((42, 640), countdown=10)` | Signature with args and an execution option | | `make_thumbnail.s(42, 640)` | Shortcut for `.signature((42, 640), {})`; mutable | | `make_thumbnail.si(42, 640)` | Shortcut for `.signature(..., immutable=True)` | | `signature('photos.make_thumbnail', args=(42, 640))` | Same, by task name, without importing the task | `.s()` and `.si()` accept only args and kwargs. To add execution options to one, chain `.set()`: `make_thumbnail.s(42, 640).set(countdown=10)`. ## Partial arguments: prepended, not appended A signature built with `.s()` may be **incomplete**, a *partial*. When it is finally applied, Celery merges what it receives with what it already holds: 1. Extra **positional args** are **prepended** to the stored args. `make_thumbnail.s(640).delay(42)` runs `make_thumbnail(42, 640)`. 2. Extra **kwargs** are merged, and the new values win. 3. Extra **options** are merged, and the new values win. This rule is what makes a chain work. When a step finishes, Celery applies the next signature with the finished step's return value as a partial argument, so it lands in the **first** position. The function signature has to be written for that: the parent's result is the first parameter, the arguments you bound with `.s()` come after it. ## Immutability: .si() Sometimes the next step does not want the parent's result at all. Sending the owner a notification after thumbnails are published needs only the user id. An **immutable** signature, built with `.si()`, ignores partial args and kwargs: it runs with exactly what it was built with. Only execution options can still be set on it. Without `.si()`, the parent's return value would be prepended and the call would fail with a `TypeError` for an unexpected argument, or silently bind the wrong value to the first parameter. ## The photo upload pipeline In the photo-sharing upload flow, signatures describe each step before anything runs: - `save_original.s(upload_id)` stores the upload and returns a `photo_id`; - `make_thumbnail.s(640)` receives that `photo_id` first, then its bound size; - `notify_owner.si(user_id)` runs after the others without taking any result. Composed as `save_original.s(upload_id) | make_thumbnail.s(640) | notify_owner.si(user_id)`, the `|` operator builds a `chain` from the three signatures, and each step is a separate task message, possibly on a different worker. Because signatures are data, the steps still to run travel inside the task message itself: when a step finishes, the worker takes the next signature and sends it with the result prepended. That also means signatures obey the task serializer. With the default JSON serializer, every argument you bind with `.s()` or `.si()` must be JSON-serializable, one more reason to bind ids such as `photo_id` rather than objects. ## Common traps - **Order of arguments.** Writing `def make_thumbnail(size, photo_id)` for a signature built as `make_thumbnail.s(640)` inside a chain swaps the values, because the parent's result is prepended. - **Forgetting `.si()`** on a step that takes no input: it receives an unexpected extra argument. - **Calling `sig()`** instead of `sig.delay()`: the work runs synchronously in the caller. - **Passing options to `.s()`**: keyword arguments to `.s()` are task kwargs, not execution options; use `.set()` or `.signature(args, **options)`. The short version for an interview: a signature is a serializable task call; `.s()` is a partial that takes the parent's result in front of its own arguments; `.si()` is frozen and ignores it.
- What happens if you call a Celery signature directly, as sig(), instead of sig.delay()?`Signature.__call__` runs the task function in the current process, merging any partial args first. No message is sent, no worker is involved, and the caller blocks until it returns. `sig.delay()` and `sig.apply_async()` are the calls that publish a task message. Canvas objects differ here: calling a `chain`, `group` or `chord` directly does send its tasks asynchronously.
- How do you give a signature built with .s() an execution option such as countdown?Keyword arguments to `.s()` become task kwargs, so options go elsewhere: chain `.set(countdown=10)` onto the signature, or build it with `task.signature((42, 640), countdown=10)`. Options passed later to `apply_async()` are merged in and win over the stored ones. Immutable signatures still accept execution options; only their args and kwargs are frozen.
A signature is a pre-filled form with some blanks. The previous desk in a chain writes its answer into the first blank before passing the form on; a stamped 'no changes' form (.si()) is processed exactly as written.
saying these in an interview costs you the question
- A signature starts the task as soon as it is created.
- Arguments supplied later are appended after the signature's own arguments.
- Calling sig() sends a task message exactly like sig.delay().
- An immutable .si() signature cannot take execution options either.
- Keyword arguments to .s() such as countdown=10 become execution options.