Why does a `tuple` subclass with a custom `__new__` fail to unpickle, and what fixes it?
answer
- Loading does not call `__init__`
- The allocator needs its arguments back
- Immutable values are fixed at construction
- The inherited hook matches the base signature
- One method whose return matches `__new__`
basics
~20 sLoading rebuilds the instance by calling cls.new(cls, *args), where args come from getnewargs. A tuple subclass inherits one that returns the whole tuple, so a two-parameter new receives a single argument and raises TypeError. Define getnewargs to match new.
solid answer
~40 sFor protocol 2 and later, `object.__reduce_ex__` builds a default reduction that reconstructs the instance through `cls.__new__(cls, *args)` and never calls `__init__`. Those arguments come from `__getnewargs__` — or `__getnewargs_ex__`, which returns a positional tuple and a keyword dict — and default to nothing at all. That is fine for a mutable class, whose blank instance can have its attributes filled in afterwards, but an immutable type carries its value in construction, so the arguments must be reproduced. A `tuple` subclass inherits `tuple.__getnewargs__`, which returns a one-item tuple holding the whole tuple, so a `__new__` declared as `(cls, x, y)` gets a single argument and the load fails with a `TypeError` about a missing parameter. The fix is to define `__getnewargs__` returning exactly what your own `__new__` expects.
code
python · 19 linesimport pickle
class Bad(tuple):
def __new__(cls, x, y):
return super().__new__(cls, (x, y))
try:
pickle.loads(pickle.dumps(Bad(3, 4)))
except TypeError as exc:
print("without:", exc) # Bad.__new__() missing 1 required positional argument: 'y'
class Point(tuple):
def __new__(cls, x, y):
return super().__new__(cls, (x, y))
def __getnewargs__(self):
return (self[0], self[1])
print("with:", pickle.loads(pickle.dumps(Point(3, 4))))go deeper
Know that rebuilding a pickled object does not run __init__, so a class whose constructor takes required arguments needs a way to hand those arguments back.
Explain the default reconstruction through cls.__new__(cls, *args), where those arguments come from, and why an inherited hook describes the base class's signature rather than yours.
Show that you round-trip every persisted type in tests, because this mismatch is silent at write time and only fails in the process that later reads the data.
Decide whether value types that are persisted should rely on language hooks at all, or be defined by an explicit schema with construction rules that do not shift when a constructor signature changes.
### Where the arguments come from When a class defines no `__reduce__` of its own, `object.__reduce_ex__(protocol)` composes a default reduction. Under protocol 2 and later the callable it produces reconstructs the object by calling `cls.__new__(cls, *args)` — the class's own allocator, with `__init__` deliberately skipped, because rerunning an initialiser would re-do work and could have side effects. The `args` in that call are not invented: they come from `__getnewargs__` if the class has one, from `__getnewargs_ex__` if keyword arguments are needed, and otherwise from nothing at all, giving `cls.__new__(cls)`. For an ordinary mutable class this is invisible. `__new__` accepts no extra parameters, a blank instance appears, and the attribute state recorded in the reduction is applied afterwards. The object is whole and nobody thinks about the hook. ### Why immutable types are different An immutable object cannot be filled in after the fact. Its value is fixed by `__new__`, so a reduction that constructs a blank instance and then assigns attributes cannot reproduce it. This is exactly what `__getnewargs__` exists for: it lets the class say "to build me again, call `__new__` with these arguments". The built-in immutable types implement it — `"abc".__getnewargs__()` returns `('abc',)`, and `(3, 4).__getnewargs__()` returns `((3, 4),)`, one argument holding the whole tuple, which is exactly what `tuple.__new__` wants. ### The trap in a subclass Now subclass `tuple` with your own two-parameter constructor: ```python class Bad(tuple): def __new__(cls, x, y): return super().__new__(cls, (x, y)) ``` Dumping works. Loading fails, with `TypeError: Bad.__new__() missing 1 required positional argument: 'y'`. The reason is that `Bad` inherited `tuple.__getnewargs__`, which reports one argument — the whole tuple — while `Bad.__new__` demands two. The inherited hook describes the base class's constructor signature, not yours, and nothing warns you at dump time; the mismatch only surfaces in whatever program later reads the data. The fix is one method: ```python class Point(tuple): def __new__(cls, x, y): return super().__new__(cls, (x, y)) def __getnewargs__(self): return (self[0], self[1]) ``` The rule is simply that `__getnewargs__` must return a tuple that can be splatted into your own `__new__`. If your constructor needs keyword-only parameters, define `__getnewargs_ex__` instead, returning a two-item tuple of a positional argument tuple and a keyword dictionary; a class should define one or the other, not both. ### How this relates to writing `__reduce__` These hooks belong to the *default* reduction. The moment a class defines `__reduce__`, `object.__reduce_ex__` delegates to it wholesale and `__getnewargs__` is never consulted — your tuple is the recipe, start to finish. That is worth stating explicitly in an interview, because candidates often add both and then debug a hook that is not running. Choose one: `__getnewargs__` when the default machinery is nearly right and only the construction arguments are missing, a full `__reduce__` when you want to control the callable as well. ### Related sharp edges The same reasoning applies to any class whose `__new__` has required parameters, immutable or not — a class hierarchy that pushes required arguments into `__new__` will fail to round-trip until the newargs hook matches. Frozen value objects assembled by libraries usually generate the hook for you; hand-written ones frequently do not. And because the failure appears at load time in a different process or a different day's run, it is the sort of defect that reaches production data before anyone sees it, so a round-trip test of every persisted type is cheap insurance. ### Versions The newargs hooks only participate in protocol 2 and later; protocols 0 and 1 use an older reconstruction path. That is a non-issue in practice, since `pickle.DEFAULT_PROTOCOL` is 5 on Python 3.14 and was 4 from 3.8 through 3.13. `__getnewargs_ex__`, the keyword-capable form, is likewise a protocol 2-and-later mechanism. ### Reading the failure The diagnostic habit worth showing is to ask the object what it will produce *before* trusting a round trip. Calling `__getnewargs__` on an instance and mentally splatting the result into your own `__new__` answers the question in one line, and a round trip through `pickle.dumps` and `pickle.loads` in a unit test answers it for real. The error text is unusually helpful here: it names the class and the missing parameter, which points straight at a constructor signature rather than at the serialisation machinery, and that is why the cause is so often mislabelled as a pickle bug when it is really an arity mismatch the class itself declared. ### A note on what travels Only the newargs values are consumed by the allocator; anything else the object carries is handled by the state part of the default reduction and applied afterwards. So a subclass of an immutable built-in that also keeps extra attributes needs both halves to be right: the arguments must rebuild the underlying value, and the extra attributes ride along separately. Keeping the newargs tuple minimal — exactly the parameters `__new__` declares, nothing more — is what keeps the two halves from overlapping and writing the same data twice.
- When do you need `__getnewargs_ex__` rather than `__getnewargs__`?When the constructor takes keyword arguments, including keyword-only parameters. `__getnewargs_ex__` returns a two-item tuple of a positional argument tuple and a keyword dictionary, which is then splatted into `cls.__new__`. A class defines one form or the other; defining both is a source of confusion because only one is consulted.
- If a class defines `__reduce__`, is `__getnewargs__` still used?No. `object.__reduce_ex__` delegates entirely to a user-defined `__reduce__`, so the default reconstruction path — and with it the newargs hook — is bypassed. Your tuple names the callable and its arguments outright. Adding both and then wondering why the newargs hook never runs is a common debugging detour.
- Why do mutable classes almost never need this hook?Their default reduction allocates a blank instance with no constructor arguments and restores attribute state afterwards, which is enough to reproduce the object. The hook is needed when a value cannot be assigned after allocation — immutable types — or when `__new__` declares required parameters that the blank-allocation path cannot satisfy.
saying these in an interview costs you the question
- Thinks unpickling calls `__init__` on the rebuilt instance
- Assumes subclassing an immutable built-in needs no pickle work
- Returns the wrong number of items from `__getnewargs__`
- Confuses construction arguments with restored attribute state
- Defines both `__reduce__` and `__getnewargs__` and expects both to run
- Expects the mismatch to be reported when dumping rather than loading