In a search-index rebuilder, a module-level `object()` sentinel fails its `is` test inside worker processes - why?

seniorshould knowfreq 30%

answer

Two processes, two object graphs
Serialization rebuilds, it does not share
Deep copy breaks it without any workers
Round-trip the sentinel and assert `is`
`__reduce__` returning a name, or an enum member

basics

~20 s

Identity does not survive serialization. Pickling the sentinel to a worker, or deep-copying it, rebuilds a fresh instance, so is compares two different objects. Fix it with a __reduce__ that returns the name, or an enum member.

solid answer

~50 s

A sentinel's only guarantee is that it is that object, and object identity is local to one interpreter's heap. When a task descriptor holding _MISSING is pickled to a worker process, pickle reconstructs a brand-new empty instance, so field is _MISSING is False for every field in the child - and because object compares by identity, == fails too. copy.deepcopy, a module imported twice under two names, and importlib.reload break it the same way. Three fixes, in order of preference: normalize the sentinel away before anything is serialized; give the sentinel type a __reduce__ returning the string name of its module-level binding, so pickle resolves it back to the same global and copy treats it as atomic; or make the marker an enum.Enum member, which pickles by name and returns the canonical member. Verify with one assertion rather than reasoning about it.

A sentinel's only guarantee is that it is *that* object, and object identity is local to one interpreter's heap. When a task descriptor holding `_MISSING` is pickled to a worker process, `pickle` reconstructs a brand-new empty instance, so `field is _MISSING` is False for every field in the child - and because `object` compares by identity, `==` fails too. `copy.deepcopy`, a module imported twice under two names, and `importlib.reload` break it the same way. Three fixes, in order of preference: normalize the sentinel away before anything is serialized; give the sentinel type a `__reduce__` returning the string name of its module-level binding, so pickle resolves it back to the same global and `copy` treats it as atomic; or make the marker an `enum.Enum` member, which pickles by name and returns the canonical member. Verify with one assertion rather than reasoning about it.

code

python · 14 lines

python

import copy
import pickle
from enum import Enum

class _SentinelEnum(Enum):
    MISSING = "MISSING"

MISSING = _SentinelEnum.MISSING
plain = object()

print(pickle.loads(pickle.dumps(plain)) is plain)      # False
print(copy.deepcopy(plain) is plain)                    # False
print(pickle.loads(pickle.dumps(MISSING)) is MISSING)   # True
print(copy.deepcopy(MISSING) is MISSING)                # True

go deeper

for a junior

Take away the headline: is compares object identity, and identity is meaningful only inside one running interpreter. Anything that is copied or serialized comes back as a different object even when it looks identical.

for a middle

Explain the pickle protocol's split - classes and functions by name reference, ordinary instances rebuilt by value - and name the in-process causes too: copy.deepcopy, a module imported twice under two names, and importlib.reload.

for a senior

Show the diagnosis path: one round-trip assertion rather than speculation, then id and module-name logging on both sides. Argue for normalizing the sentinel away at the boundary before reaching for __reduce__, and reject the tempting __eq__ fix out loud.

for a principal

Own the boundary rule: identity-based markers are an in-process idiom, and anything crossing a process, cache or wire boundary needs an explicit representation. Tie it to the 3.14 start-method default, which moves more state through serialization and surfaces this class of assumption across a fleet.

The symptom is specific and confusing: a rebuilder walks a 17-service dependency graph, builds one task descriptor per service where _MISSING means "inherit this setting from the parent service", and fans the descriptors out to a worker pool. In the parent every is _MISSING check behaves. In the children not one of them matches, so every service is treated as having an explicit override, several workers decide they own the shared index manifest, and the run ends in a race on state that was never supposed to be contended.

The mechanism. pickle serializes values, not object graphs shared across processes. Classes and functions are pickled by qualified name - a reference the other side resolves by import - but ordinary instances are pickled by value through __reduce_ex__, which records how to build an equivalent object and then does exactly that on load. A bare object() has no state at all, so the reconstruction is a fresh, empty instance at a different address. object inherits identity-based __eq__, so the copy is neither is nor == the original. The sentinel has not been corrupted; there are simply now two of them, and the check that made the design work was identity.

The same reduction protocol powers copy.copy and copy.deepcopy, so a sentinel that passes through a deep-copied configuration dictionary breaks in-process, without any concurrency involved at all. Two more in-process causes are worth knowing because they are hard to see: a module imported under two names - once from inside its package and once as a top-level module by a script run directly - is executed twice and produces two distinct sentinels; and importlib.reload rebinds the module's globals, so objects captured before the reload no longer match the new binding.

Where the version pin matters. Since Python 3.14 the default multiprocessing start method on Unix other than macOS is forkserver; macOS and Windows continue to use spawn, and fork must now be requested explicitly. This did not create the identity problem - identity never crossed a process boundary under any start method - but it exposes more of it. Under fork the child was a memory copy of the parent, so module-level state the parent had built after import was simply there; under forkserver and spawn the child re-imports your modules and receives everything else through pickle. Code that quietly depended on inherited process state now has to survive serialization, and latent assumptions like this one surface on upgrade.

Fix one: do not send sentinels across the boundary. The strongest answer is to normalize at the edge. A sentinel is a marker about the call, not a value in the domain, so resolve it before serializing: substitute the inherited value, or omit the key entirely and let the child's own is _MISSING default apply. Then nothing that crosses the boundary depends on identity, and no serialization format has to cooperate. This also happens to be the only fix that works when the boundary is JSON or YAML, which have no concept of identity at all.

Fix two: make the sentinel pickle-stable. pickle has a documented shortcut: if __reduce__ returns a string, that string is treated as the name of a global in the object's module, looked up on load, and the existing object is returned.

python

class _MissingType:
    def __repr__(self):
        return "MISSING"
    def __reduce__(self):
        return "MISSING"

MISSING = _MissingType()

Now pickle.loads(pickle.dumps(MISSING)) is MISSING is True, and because copy treats a string reduction as atomic, copy.deepcopy(MISSING) is MISSING is True as well. The string must match the module-level name the object is bound to, which is a coupling worth a comment.

Fix three: use an enum member. An enum.Enum member reduces to its class and value, and calling an enum class with an existing value returns the canonical member from the class's lookup table rather than constructing anything, so identity survives both pickling and deep-copying for free. It also gives a readable repr, a natural home for more than one marker, and a type a checker can reason about.

Diagnosis. Do not reason about this from first principles under pressure - assert it. assert pickle.loads(pickle.dumps(MISSING)) is MISSING and the same for copy.deepcopy take one line each and fail loudly on exactly the property the design assumes. If they pass and the bug persists, log id(MISSING) together with type(MISSING).__module__ and __name__ on both sides: two different module names for the same file is the double-import case, and a differing id within one process points at the copy path.

The wrong fix to name and reject. Adding an __eq__ to the sentinel type so that reconstructed copies compare equal turns a unique marker into a class-membership test - now any instance of that type, including one a caller constructs, is "the sentinel". That trades a loud failure for a quiet one.

The symptom is specific and confusing: a rebuilder walks a 17-service dependency graph, builds one task descriptor per service where `_MISSING` means "inherit this setting from the parent service", and fans the descriptors out to a worker pool. In the parent every `is _MISSING` check behaves. In the children not one of them matches, so every service is treated as having an explicit override, several workers decide they own the shared index manifest, and the run ends in a race on state that was never supposed to be contended. **The mechanism.** `pickle` serializes *values*, not object graphs shared across processes. Classes and functions are pickled by qualified name - a reference the other side resolves by import - but ordinary instances are pickled by value through `__reduce_ex__`, which records how to build an equivalent object and then does exactly that on load. A bare `object()` has no state at all, so the reconstruction is a fresh, empty instance at a different address. `object` inherits identity-based `__eq__`, so the copy is neither `is` nor `==` the original. The sentinel has not been corrupted; there are simply now two of them, and the check that made the design work was identity. The same reduction protocol powers `copy.copy` and `copy.deepcopy`, so a sentinel that passes through a deep-copied configuration dictionary breaks in-process, without any concurrency involved at all. Two more in-process causes are worth knowing because they are hard to see: a module imported under two names - once from inside its package and once as a top-level module by a script run directly - is executed twice and produces two distinct sentinels; and `importlib.reload` rebinds the module's globals, so objects captured before the reload no longer match the new binding. **Where the version pin matters.** Since Python 3.14 the default `multiprocessing` start method on Unix other than macOS is `forkserver`; macOS and Windows continue to use `spawn`, and `fork` must now be requested explicitly. This did not create the identity problem - identity never crossed a process boundary under any start method - but it exposes more of it. Under `fork` the child was a memory copy of the parent, so module-level state the parent had built after import was simply there; under `forkserver` and `spawn` the child re-imports your modules and receives everything else through pickle. Code that quietly depended on inherited process state now has to survive serialization, and latent assumptions like this one surface on upgrade. **Fix one: do not send sentinels across the boundary.** The strongest answer is to normalize at the edge. A sentinel is a marker about the *call*, not a value in the domain, so resolve it before serializing: substitute the inherited value, or omit the key entirely and let the child's own `is _MISSING` default apply. Then nothing that crosses the boundary depends on identity, and no serialization format has to cooperate. This also happens to be the only fix that works when the boundary is JSON or YAML, which have no concept of identity at all. **Fix two: make the sentinel pickle-stable.** `pickle` has a documented shortcut: if `__reduce__` returns a string, that string is treated as the name of a global in the object's module, looked up on load, and the *existing* object is returned. ```python class _MissingType: def __repr__(self): return "MISSING" def __reduce__(self): return "MISSING" MISSING = _MissingType() ``` Now `pickle.loads(pickle.dumps(MISSING)) is MISSING` is True, and because `copy` treats a string reduction as atomic, `copy.deepcopy(MISSING) is MISSING` is True as well. The string must match the module-level name the object is bound to, which is a coupling worth a comment. **Fix three: use an enum member.** An `enum.Enum` member reduces to its class and value, and calling an enum class with an existing value returns the canonical member from the class's lookup table rather than constructing anything, so identity survives both pickling and deep-copying for free. It also gives a readable `repr`, a natural home for more than one marker, and a type a checker can reason about. **Diagnosis.** Do not reason about this from first principles under pressure - assert it. `assert pickle.loads(pickle.dumps(MISSING)) is MISSING` and the same for `copy.deepcopy` take one line each and fail loudly on exactly the property the design assumes. If they pass and the bug persists, log `id(MISSING)` together with `type(MISSING).__module__` and `__name__` on both sides: two different module names for the same file is the double-import case, and a differing `id` within one process points at the copy path. **The wrong fix to name and reject.** Adding an `__eq__` to the sentinel type so that reconstructed copies compare equal turns a unique marker into a class-membership test - now any instance of that type, including one a caller constructs, is "the sentinel". That trades a loud failure for a quiet one.

How would you prove the diagnosis in one line before changing any code?
`assert pickle.loads(pickle.dumps(MISSING)) is MISSING`, and the same assertion with `copy.deepcopy`. Either failure confirms that identity cannot survive the boundary the value is crossing, which is the whole hypothesis. It is a faster and more reliable answer than reading the worker code, and it belongs in the test suite afterwards so the property cannot regress.
Why does an `enum.Enum` member survive a pickle round trip when a bare `object()` does not?
An enum member reduces to its class and value rather than to instructions for building a new instance. On load, pickle calls the enum class with that value, and calling an enum class with an existing value returns the canonical member from its lookup table instead of constructing anything. `copy.deepcopy` goes through the same protocol, so identity holds there too.
Did the 3.14 change to the multiprocessing start method cause this bug?
No - object identity never crossed a process boundary under any start method. But since 3.14 the Unix default other than macOS is `forkserver`, so children re-import modules and receive their inputs through pickle instead of inheriting the parent's memory as `fork` did. More state now crosses a serialization boundary, so assumptions that were latent under `fork` start failing on upgrade.

Identity is like a handwritten signature: photocopying the page reproduces the marks but not the signing. What arrives in the worker is a photocopy that looks right and is not the same document.

saying these in an interview costs you the question

Claims `is` works across processes because the module is the same
Adds `__eq__` to the sentinel so copies compare equal
Blames the GIL or a thread race for the identity mismatch
Assumes `copy.deepcopy` preserves every singleton
Thinks pickling an instance stores it by name reference
Fixes it by forcing the fork start method and moving on

In a search-index rebuilder, a module-level `object()` sentinel fails its `is` test inside worker processes - why?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

▾common follow-ups3 questions

▾analogy~10 s

▾red flags6 flags

§ context

▾related questions4 questions

▾related topics4 topics

answer

go deeper

context