Why does a custom exception fail to unpickle with a TypeError about missing arguments?

**How an exception pickles.** `BaseException` defines `__reduce__`, and every exception inherits it unless it overrides it. It returns a two- or three-item tuple: the class, `self.args`, and -- only when the instance dictionary is non-empty -- that dictionary as state. Unpickling then does two things in order: it *calls* the class with the stored args as positional arguments, and it updates the new object's `__dict__` with the stored state. Both halves matter, and the first is where custom exceptions break. **Where `args` comes from.** `args` is whatever was passed to `BaseException.__init__` -- in practice, whatever you handed to `super().__init__(...)`. The overwhelmingly common custom-exception idiom builds a nice message and passes only that: ```python class ScrapeFailed(Exception): def __init__(self, service, status): super().__init__(f"{service} returned {status}") self.service = service self.status = status ``` That object is perfect in-process: `str(exc)` reads well, and `exc.service` is there. But its `args` is a one-tuple containing the formatted string, while `__init__` demands two positional parameters. Reconstructing it means calling `ScrapeFailed("metrics-7 returned 503")`, which raises `TypeError: ScrapeFailed.__init__() missing 1 required positional argument: 'status'`. The exception cannot cross a process boundary at all, and the error the parent finally sees is a `TypeError` about a constructor -- not the scrape failure you were trying to report. **The asymmetry that confuses people.** The attributes are not the problem. `self.service` and `self.status` live in the instance `__dict__`, which `__reduce__` does carry as state, so they would be restored faithfully -- if the object could be constructed in the first place. This is why the bug reads as nonsense at first: the data is all in the pickle, and the failure is in the call that must happen before the data is applied. **Three fixes, in order of preference.** 1. *Make `args` mirror the signature.* Pass every constructor parameter through: `super().__init__(service, status)`. Then `args == (service, status)`, the reconstruct call matches, and you get the message back by defining `__str__` instead of pre-formatting. This is the smallest change and needs no pickle knowledge from the next reader. 2. *Override `__reduce__`.* Return `(type(self), (self.service, self.status))` explicitly. Useful when you cannot change what goes into `args` -- for instance when a message must stay in `args[0]` for compatibility with existing handlers -- and it is the standard tool for exceptions with genuinely awkward constructors. 3. *Give every parameter a default.* `def __init__(self, service=None, status=None)` makes the mismatched call succeed and the state dict then repairs the attributes. It works, but it silently permits half-built exceptions and it hides the real intent, so treat it as a last resort. **The second gate: is the payload itself picklable?** Even with a correct signature, every value in `args` and in the instance dict has to serialize. Attaching a live handle to an exception -- a lock, an open socket, a database connection, a file object -- raises `TypeError: cannot pickle '_thread.lock' object` in the *child*, at send time, so once again the real failure disappears and is replaced by a serialization error. Exceptions that cross processes should carry plain data: strings, numbers, tuples, dicts of those. If you want the offending object's identity, carry its repr or its identifier, not the object. **How to catch this before production.** It is trivially testable without any concurrency at all: `pickle.loads(pickle.dumps(exc))` in a unit test, asserting the type, the message and each attribute survive. Any exception class intended to be raised inside a worker deserves exactly that one-line round-trip test. Debugging it later, from a parent-side `TypeError` naming a constructor you did not think was being called, is far more expensive than writing it. **One trap inside the second fix.** If you override `__reduce__` and return a two-item tuple -- the class and the arguments -- you have replaced the inherited behaviour entirely, including the part that carried the instance dictionary. Anything that is not re-created by the constructor call is then lost: an attribute set on the exception after it was raised, a retry counter incremented by a handler, a note attached on the way up. Return a three-item tuple, adding `self.__dict__` as state, whenever the instance may carry more than the constructor puts there. It is a small detail with a nasty signature: the exception arrives looking complete, and only the field added late is missing.