skip to content

A video-metadata extractor module defines its own open(); why do that module's other functions call it instead of the built-in?

level: seniorimportance: should knowfreq 42%

answer

  1. Two of the four layers, in order
  2. One layer is consulted before the other
  3. The blast radius stops at the file
  4. The lookup happens when the call runs
  5. One module lets you reach the original

basics

~20 s

Module globals are searched before built-ins, so once a module binds the name open at top level, every bare open(...) in that module resolves to it. The shadow is confined to that module's namespace; other modules still get the built-in.

solid answer

~50 s

The G layer precedes the B layer, so a module-level definition wins over the `builtins` entry for that whole module — including functions written before the shadowing definition, because the lookup happens at call time, not at definition time. In an extractor where the wrapper also registers each probed file, a helper that meant the built-in ends up registering the file a second time: the same metadata gets recorded twice per clip, with no error anywhere. The shadow is scoped to the one module; a caller in another module calling `open` still falls through to built-ins, which is why the bug reproduces only on some code paths. Reach past a shadow deliberately with `import builtins` and `builtins.open(...)`. The real fix is to rename — `open_probe` — and to let a linter's builtin-shadowing rule catch the next one.

code

python · 14 lines
python
import builtins

def open(path):                 # shadows the built-in for this module
    return f"probe:{path}"

def extract(path):
    return open(path)           # module globals win over builtins

def read_header(path):
    with builtins.open(path, "rb") as fh:   # reach past the shadow
        return fh.read(8)

print(extract("clip.mp4"))                  # probe:clip.mp4
print("open" in vars(builtins), open is builtins.open)   # True False

go deeper

for a junior

Recall that naming a variable after a built-in such as list, id or open hides that built-in for the rest of the scope, and that the fix is simply to pick a different name.

for a middle

Explain the ordering that causes it — module globals are searched before built-ins — that the lookup happens at call time so earlier functions are affected too, and how builtins lets you reach the original.

for a senior

Demonstrate the diagnosis: a silent behavioural change confined to one module, reproduced on some call paths only, traced by comparing the bound name against the built-in, and prevented by a lint rule and by banning wildcard imports.

for a principal

Own the codebase-wide stance: which lint rules are mandatory, whether wrappers may reuse built-in names at all, and why mutating the shared built-in namespace is a process-wide hazard rather than a convenience.

## The mechanism in one line A bare name is searched Local, Enclosing, Global, Built-in. A module-level `def open(...)` or `open = ...` puts `open` in the module's global namespace, and the search reaches G before B, so the module's own binding wins for every bare `open(...)` written anywhere in that file. ## Why the ordering surprises people Three properties make this sharper than it looks. **It is retroactive within the module.** Functions defined *above* the shadowing definition are affected too. A function body compiles to an instruction that looks the name up in the module globals when it runs; it does not capture whatever `open` meant at `def` time. So adding a wrapper at the bottom of a file changes the behaviour of code at the top. **It is silent.** Both the built-in and the shadow are callable and the call arity often matches for the common case, so nothing raises. The failure surfaces as behaviour, not as an exception — in a metadata extractor whose wrapper also records each file it opens, a helper that intended the built-in now goes through the wrapper and the record is written twice per clip. Every count downstream is doubled, and there is no traceback pointing at the cause. **It is per-module.** Each module has its own global namespace, and built-in lookup falls through to the shared `builtins` module only on a miss. So the same call written in a different module behaves correctly. That asymmetry is what makes the bug reproduce on one code path and not another, and it is the observation that usually cracks the diagnosis. ## Diagnosing it When a call behaves unlike the built-in you think you are calling, check what the name is actually bound to in that module: compare it against the attribute of the same name on the `builtins` module, or print the function object and look at where it was defined. A wildcard import is a common way for a shadow to arrive without anyone typing it in the file — the imported module's public names land straight in your global namespace, so a helper called `open`, `filter`, `id`, `type`, `input`, `hash` or `format` in the other module becomes yours. Static tooling catches this class of defect cheaply: linters ship a rule specifically for a variable, argument or attribute shadowing a Python builtin, and turning it on is a one-line configuration change with an immediate payoff. ## Reaching past a shadow When the shadow is intentional — a wrapper you actually want most call sites to use — code that needs the real thing goes through the module explicitly: ```python import builtins def read_header(path): with builtins.open(path, "rb") as fh: return fh.read(8) ``` That is explicit, greppable and immune to further shadowing, because it is an attribute access rather than a bare-name lookup. ## The blast radius of the other direction Assigning *into* the `builtins` module — rebinding an attribute there — is a different and far more dangerous act: the built-in namespace is shared, so every module in the process, including third-party code and the standard library, sees the change. It is occasionally used to inject a helper into every module in a small script or teaching environment, and it should be treated as off-limits in application code. If a helper needs to be available everywhere, that is what a module and an import are for. ## What good practice looks like Rename rather than shadow: `open_probe`, `filter_valid`, `item_id`, `content_type`. The cost is a few characters and the benefit is that the name means exactly one thing in the file. Where a wrapper genuinely should replace a built-in for a whole module, make the intent loud — a distinct name plus an explicit `builtins.` call at the sites that need the original is clearer than a same-name shadow. Avoid wildcard imports in application code so shadows cannot arrive invisibly. And enable the linter rule so the next occurrence is caught in review rather than in a doubled count nobody notices for a month. ## Interview shape Expect to be handed a snippet or a symptom and asked to explain the behaviour. Lead with the lookup order (G before B), say that the shadow is module-scoped and that the lookup happens at call time so earlier code is affected too, name `builtins` as the way to reach past it, and finish on prevention: rename, no wildcard imports, linter rule. Mentioning that mutating `builtins` itself is process-wide, and therefore a different order of risk, is the detail that reads as production experience.

  • Does a module-level open() in one module affect calls to open() in other modules?
    No. Each module has its own global namespace, and only a miss there falls through to the shared built-in namespace. So the shadow is confined to the file that defines it — which is exactly why the symptom appears on some code paths and not others, and why the diagnosis often starts with noticing that asymmetry.
  • How would you find every built-in that a codebase shadows?
    Turn on the linter rule for builtin shadowing, which flags shadowing variables, arguments and attributes across the tree in one pass. As a quick ad hoc check, walk the module's global names and test each against the `builtins` module's attributes. Then remove wildcard imports, since they are how shadows arrive without appearing in the file.
  • Is rebinding an attribute on the builtins module ever an acceptable way to share a helper?
    Practically never in application code. The built-in namespace is shared by every module in the process, including the standard library and third-party packages, so the change is invisible at every call site and can collide with names those libraries rely on. A normal module plus an explicit import gives the same reach with none of the ambiguity.

saying these in an interview costs you the question

  • Says built-ins always win over module-level names
  • Thinks the shadow leaks into other modules
  • Believes functions defined earlier keep the built-in
  • Expects an error or warning at import time
  • Suggests patching the builtins module as the fix
  • Cannot name a way to reach the original built-in

context