Why does `copy.deepcopy()` raise TypeError on an object holding a `threading.Lock`, and how do you share it instead?

## The failure, and why pickling is involved at all A log-ingest pipeline builds one configured stage object and clones it per worker with `copy.deepcopy()`. The stage carries plain data — a name, a dict of counters sitting at an 83% cache-hit rate, a running mean of parse latency — and one `threading.Lock` guarding the counters. The clone raises: ``` TypeError: cannot pickle '_thread.lock' object ``` Nothing was pickled. The message comes from `deepcopy`'s own fallback path: for each object with no `__deepcopy__` and no entry in its internal dispatch table, it asks `copyreg.dispatch_table`, then calls `__reduce_ex__(4)` and rebuilds from the result. That is the pickle protocol used purely in memory. A lock is a thin wrapper over an OS primitive with no meaningful reduce, so it refuses, and the refusal surfaces as a `TypeError` from a copy the caller never connected to serialization. Recognizing this as "deep copy reached something unreducible" rather than "someone pickled my object" is most of the diagnosis. Note what does *not* fail: `copy.copy()` on the same stage succeeds, because a shallow copy rebinds the existing attribute values into a new instance without recursing into any of them. The lock is not copied; it is referenced. That contrast is a useful probe when you meet this error in the wild. ## Why duplicating the lock would be worse than the error It is tempting to read the `TypeError` as an annoyance to route around — swap in a fresh `threading.Lock()` for the clone and move on. Sometimes that is exactly right, and sometimes it silently destroys the invariant the lock existed to protect. In the pipeline above, all worker clones update one shared rolling mean. If every clone gets its own lock, each one acquires a mutex nobody else respects, the read-modify-write on the shared accumulator interleaves freely, updates are lost, and the reported mean drifts away from the true value by a small, irreproducible floating-point margin that grows with load. That is the worst class of bug this leaf produces: no exception, no crash, just numbers that are slightly wrong and a discrepancy nobody can reproduce on a single worker. The rule to state in an interview: **a lock is an identity, not a value.** Copying its bits means nothing; what matters is whether the copies are meant to exclude each other. Answer that question first, then write the hook. ## Writing the hook ```python def __deepcopy__(self, memo): cls = type(self) new = cls.__new__(cls) memo[id(self)] = new new.name = self.name new.totals = copy.deepcopy(self.totals, memo) new.lock = self.lock # shared on purpose return new ``` Four things are load-bearing. `cls = type(self)` keeps subclass copies correct. `cls.__new__(cls)` skips `__init__`, which would allocate a *new* lock and defeat the point. `memo[id(self)] = new` comes before any recursion so cycles terminate. And `copy.deepcopy(self.totals, memo)` passes the memo down so shared subgraphs stay shared. The same shape covers the other handles that trigger this error: an open file object, a socket, a database connection, a compiled callback registry, a thread or a running task. For each, choose one of three treatments — share the reference, create a fresh one, or set the attribute to `None` and reconnect lazily on first use. Sharing is right when the copies collaborate on one resource; a fresh instance is right when the copy is genuinely independent; `None` plus lazy rebuild is right when the resource is expensive and the copy may never need it. ## Keeping the hook honest The cost of a hand-written `__deepcopy__` is that it is a second constructor. Any attribute added to `__init__` later and not added to the hook simply vanishes from copies, usually as an `AttributeError` far from the change. Two habits contain that. Copy the bulk generically and then override only the exceptions — start from `new.__dict__.update(copy.deepcopy(...))` over a filtered dict, or explicitly list the shared names in a class-level tuple the hook reads. And test the hook: assert that the copy's data attributes are not the originals and that the shared handle *is* the original, so a future edit that duplicates the lock fails a test instead of drifting a metric. Finally, ask whether the clone is needed at all. A stage built by a factory function taking the shared lock as a parameter has no copy semantics to get wrong, and in a pipeline that constructs workers once at startup, that is usually the smaller design.