In Celery 5.6, which task argument types survive the default JSON serializer, and what does switching a task to pickle cost?

middleshould knowfreq 45%

answer

the broker only carries bytes
task_serializer and accept_content
kombu tags a few extra types
unpickling can run code

basics

~20 s

Celery's default JSON carries strings, numbers, booleans, None, lists and dicts, plus datetime, Decimal, UUID and bytes, which kombu tags and restores; anything else raises EncodeError at the call. Pickle carries more but lets broker writers run code on workers.

solid answer

~40 s

Every argument is encoded into the message by kombu. The defaults are task_serializer='json', result_serializer='json' and accept_content=['json']. Celery 5.6 sits on kombu 5.6, whose JSON encoder also wraps datetime, date, time, Decimal, UUID and bytes in a tagged envelope and restores them, so an invoice total as Decimal arrives as Decimal. Tuples arrive as lists and non-string dict keys become strings; a set or an arbitrary object raises kombu.exceptions.EncodeError from delay(). Pickle carries most Python objects, but unpickling a crafted message executes code, it ties producer and worker to the same class definitions, and workers reject it unless accept_content lists it. Since 5.5, @app.task(pydantic=True) validates plain dicts into Pydantic models on the worker, which usually beats pickle.

Every argument is encoded into the message by kombu. The defaults are `task_serializer='json'`, `result_serializer='json'` and `accept_content=['json']`. Celery 5.6 sits on kombu 5.6, whose JSON encoder also wraps `datetime`, `date`, `time`, `Decimal`, `UUID` and `bytes` in a tagged envelope and restores them, so an invoice total as `Decimal` arrives as `Decimal`. Tuples arrive as lists and non-string dict keys become strings; a `set` or an arbitrary object raises `kombu.exceptions.EncodeError` from `delay()`. Pickle carries most Python objects, but unpickling a crafted message executes code, it ties producer and worker to the same class definitions, and workers reject it unless `accept_content` lists it. Since 5.5, `@app.task(pydantic=True)` validates plain dicts into Pydantic models on the worker, which usually beats pickle.

code

python · 16 lines

python

from decimal import Decimal

from celery import Celery

app = Celery('shop', broker='redis://localhost:6379/0')

@app.task
def render_invoice_pdf(order_id, total, skus):
    ...

# Decimal is tagged by kombu's JSON encoder and arrives as Decimal('19.90')
render_invoice_pdf.delay(42, Decimal('19.90'), ['A1', 'B2'])

# A set has no JSON encoding: raises kombu.exceptions.EncodeError here,
# in the caller, and nothing is published
render_invoice_pdf.delay(42, Decimal('19.90'), {'A1', 'B2'})

go deeper

for a junior

Recall that JSON is the default serializer, that arguments must be JSON-friendly, and that pickle is off because it can execute code.

for a middle

Explain which Python types kombu's JSON restores, where EncodeError is raised, and how the serializer choice and accept_content interact.

for a senior

Show the silent failure of a pickle task hitting JSON-only workers, the code-execution risk of pickle on a shared broker, and pydantic=True as the safer typed alternative.

for a principal

Treat the message format as a cross-service contract: stable plain payloads, no class coupling between deploys, and a deliberate decision before widening accept_content.

What gets serialized

A Celery call does not hand the worker Python objects. The producer encodes the task's args and kwargs into bytes, publishes them through the broker, and the worker decodes them in another process, often on another machine running another deploy of the code. The encoding is chosen by a serializer registered in kombu, Celery's messaging library, and every message carries a content_type header naming it.

The relevant settings and their defaults in Celery 5.6:

task_serializer = 'json', used when publishing task messages
result_serializer = 'json', used when storing return values
accept_content = ['json'], the content types a worker will decode

The serializer for one call is picked in this order: the serializer execution option on apply_async, then the task's serializer attribute, then task_serializer.

What JSON carries in Celery 5.6

Plain JSON only knows strings, numbers, booleans, null, arrays and objects. Kombu's JSON encoder (kombu/utils/json.py in kombu 5.6) adds a tagged envelope, {"__type__": ..., "__value__": ...}, for a handful of common Python types and turns them back on decode.

Argument sent	Arrives on the worker as
`str`, `int`, `float`, `bool`, `None`	the same
`list`, `dict` with string keys	the same
`tuple`	a `list`
`dict` with integer keys	a `dict` with string keys
`datetime`, `date`, `time`	the same type, via ISO format
`Decimal`	a `Decimal` with the same digits
`UUID`	a `UUID`
`bytes`	`bytes` (UTF-8 or base64 inside the envelope)
`set`, a dataclass, a custom class	nothing: the call raises `EncodeError`

Older prose, including the serializer section of Celery's own calling guide, still says JSON lacks dates and decimals; the kombu source Celery 5.6 ships with handles both. An object that defines a __json__() method is encoded with whatever that method returns, which is a one-way conversion.

Where it fails

In the producer. An argument the encoder cannot handle raises kombu.exceptions.EncodeError from delay() or apply_async(), in the checkout request, before anything is published.
On the worker. A message whose content type is not in the worker's accept_content is refused as disallowed content: it is logged as an error and rejected without being requeued, so the task never runs.

The second failure is the dangerous one, because the producer saw a successful publish. It appears when one task is switched to serializer='pickle' in the producer's code while the workers still run with the default accept_content.

Pickle and why it is not the default

Pickle encodes almost any Python object, and it does keep tuples, sets and non-string keys intact. The costs:

Code execution. Unpickling a crafted payload can call arbitrary functions. Anyone who can publish to the broker, or read and replay its messages, can run code on every worker. Celery's security guide warns about this directly.
Coupling. The worker needs the same classes, importable at the same paths, to rebuild the objects. A renamed class in a deploy breaks messages already queued.
Python only. No other language can produce or consume the tasks.
Configuration on both sides. Workers must add 'pickle' to accept_content, which widens what every worker will decode.

Pydantic models since 5.5

Celery 5.5 added @app.task(pydantic=True). Arguments annotated with a pydantic.BaseModel subclass are validated from the incoming dict into a model instance on the worker, and a returned model is dumped back to a dict. The call side is unchanged: the caller still sends JSON-friendly data such as payload.model_dump(mode='json'). It gives typed task bodies without pickle's risks.

Practical rules for the invoice task

Send identifiers and plain values: the order id, a Decimal total if needed, a list of SKUs rather than a set.
Keep accept_content at JSON unless a real requirement forces otherwise.
If a task needs structured input, use a dict validated by a Pydantic model rather than pickling an object.
Test the call path, not just the body: EncodeError only appears when the arguments are actually encoded.
Remember results travel the same way: with a result backend configured, a return value such as a set or a custom object cannot be encoded under result_serializer when the worker stores it, even though the body itself ran.

## What gets serialized A Celery call does not hand the worker Python objects. The producer encodes the task's `args` and `kwargs` into bytes, publishes them through the broker, and the worker decodes them in another process, often on another machine running another deploy of the code. The encoding is chosen by a **serializer** registered in **kombu**, Celery's messaging library, and every message carries a `content_type` header naming it. The relevant settings and their defaults in Celery 5.6: - `task_serializer = 'json'`, used when publishing task messages - `result_serializer = 'json'`, used when storing return values - `accept_content = ['json']`, the content types a worker will decode The serializer for one call is picked in this order: the `serializer` execution option on `apply_async`, then the task's `serializer` attribute, then `task_serializer`. ## What JSON carries in Celery 5.6 Plain JSON only knows strings, numbers, booleans, null, arrays and objects. Kombu's JSON encoder (`kombu/utils/json.py` in kombu 5.6) adds a tagged envelope, `{"__type__": ..., "__value__": ...}`, for a handful of common Python types and turns them back on decode. | Argument sent | Arrives on the worker as | |---|---| | `str`, `int`, `float`, `bool`, `None` | the same | | `list`, `dict` with string keys | the same | | `tuple` | a `list` | | `dict` with integer keys | a `dict` with string keys | | `datetime`, `date`, `time` | the same type, via ISO format | | `Decimal` | a `Decimal` with the same digits | | `UUID` | a `UUID` | | `bytes` | `bytes` (UTF-8 or base64 inside the envelope) | | `set`, a dataclass, a custom class | nothing: the call raises `EncodeError` | Older prose, including the serializer section of Celery's own calling guide, still says JSON lacks dates and decimals; the kombu source Celery 5.6 ships with handles both. An object that defines a `__json__()` method is encoded with whatever that method returns, which is a one-way conversion. ## Where it fails 1. **In the producer.** An argument the encoder cannot handle raises `kombu.exceptions.EncodeError` from `delay()` or `apply_async()`, in the checkout request, before anything is published. 2. **On the worker.** A message whose content type is not in the worker's `accept_content` is refused as disallowed content: it is logged as an error and rejected without being requeued, so the task never runs. The second failure is the dangerous one, because the producer saw a successful publish. It appears when one task is switched to `serializer='pickle'` in the producer's code while the workers still run with the default `accept_content`. ## Pickle and why it is not the default Pickle encodes almost any Python object, and it does keep tuples, sets and non-string keys intact. The costs: - **Code execution.** Unpickling a crafted payload can call arbitrary functions. Anyone who can publish to the broker, or read and replay its messages, can run code on every worker. Celery's security guide warns about this directly. - **Coupling.** The worker needs the same classes, importable at the same paths, to rebuild the objects. A renamed class in a deploy breaks messages already queued. - **Python only.** No other language can produce or consume the tasks. - **Configuration on both sides.** Workers must add `'pickle'` to `accept_content`, which widens what every worker will decode. ## Pydantic models since 5.5 Celery 5.5 added `@app.task(pydantic=True)`. Arguments annotated with a `pydantic.BaseModel` subclass are validated from the incoming dict into a model instance on the worker, and a returned model is dumped back to a dict. The call side is unchanged: the caller still sends JSON-friendly data such as `payload.model_dump(mode='json')`. It gives typed task bodies without pickle's risks. ## Practical rules for the invoice task - Send identifiers and plain values: the order id, a `Decimal` total if needed, a list of SKUs rather than a set. - Keep `accept_content` at JSON unless a real requirement forces otherwise. - If a task needs structured input, use a dict validated by a Pydantic model rather than pickling an object. - Test the call path, not just the body: `EncodeError` only appears when the arguments are actually encoded. - Remember results travel the same way: with a result backend configured, a return value such as a set or a custom object cannot be encoded under `result_serializer` when the worker stores it, even though the body itself ran.

Why is a pickle message that the producer published successfully never executed by default Celery workers?
Workers only decode content types listed in `accept_content`, which defaults to `['json']`. A pickle message is refused as disallowed content, logged as an error and rejected without requeue. The producer saw a normal publish, so the loss is silent unless someone reads worker logs. Either keep JSON or add `pickle` to `accept_content` deliberately.
Does pydantic=True let the caller pass a Pydantic model object to delay()?
No. The Pydantic integration only works on the task side: it validates incoming dicts into models and dumps returned models. The caller still has to send JSON-serializable data, typically `model.model_dump(mode='json')`; passing the model object itself fails to encode under the JSON serializer.

saying these in an interview costs you the question

Celery's JSON serializer turns a Decimal argument into a float.
A task can send a set as an argument and receive a set on the worker.
Pickle is safe as long as the broker needs a password.
Setting serializer='pickle' on the producer is enough for workers to accept it.
Serialization errors only show up later in the worker log.

In Celery 5.6, which task argument types survive the default JSON serializer, and what does switching a task to pickle cost?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

What gets serialized

What JSON carries in Celery 5.6

Where it fails

Pickle and why it is not the default

Pydantic models since 5.5

Practical rules for the invoice task

▾common follow-ups2 questions

▾red flags5 flags

What gets serialized

What JSON carries in Celery 5.6

Where it fails

Pickle and why it is not the default

Pydantic models since 5.5

Practical rules for the invoice task

answer

go deeper