sys.path and sys.modules

## The import statement does not know what "standard library" means `import random` is not a request for a specific, blessed module. It is a request to resolve the *name* `random` through the import machinery, and for anything that is not compiled into the interpreter binary that resolution walks `sys.path` — an ordinary Python list of directory strings — and takes **the first match**. Nothing in that walk gives the standard library priority. The standard library is simply a set of directories that happen to sit somewhere in that list, and by default they sit *after* your own code. ## Where your file gets its head start When you run `python app.py`, CPython prepends the directory containing `app.py` to `sys.path`. Since Python 3.11 that entry is an absolute path (earlier versions inserted it as written, which made it change meaning if the process chdir'd). For an interactive session or `python -c`, the leading entry is `''`, which means "the current working directory, resolved at each import". Either way, entry zero is *your* directory. Everything the standard library ships — `random`, `json`, `types`, `select`, `logging`, `email`, `code`, `test` — is reachable only from later entries, so a same-named file next to your script outranks it. The one class of exception is modules compiled into the interpreter executable itself. Those names are listed in `sys.builtin_module_names` and are resolved before the path search happens at all, which is why a local `sys.py` is harmless while a local `random.py` is not. ## Why it is so confusing in practice The shadowing import does not fail. There is no warning, no collision error, no hint that anything unusual happened; the name binds and execution continues. The symptom appears later and somewhere else: * `AttributeError: module 'random' has no attribute 'randint'` — your file simply does not define what the caller expected. * A traceback that points *inside* library code. Absolute imports in third-party or standard-library modules go through the same `sys.path`, so a module you never touched can end up importing your file. * An import-time side effect firing at a bizarre moment, because your file's top-level code runs the first time anyone imports that name. * A test suite that passes under one runner and fails under another, because the two runners put different directories at the front of the path. The worst version is a file whose name matches a module the standard library imports internally during startup or during the first use of some feature — the failure then looks like the interpreter itself is broken. ## Diagnosing it in ten seconds Ask the module where it came from: ```pycon >>> import random >>> random.__file__ '/home/me/project/random.py' ``` A `__file__` under your project instead of under the interpreter's `lib` directory settles it. Two supporting checks: `'random' in sys.stdlib_module_names` tells you the name belongs to the standard library on this version (that attribute exists from 3.10), and `python -v` prints where each module was loaded from as it loads. Printing `sys.path[:3]` shows you which directory is winning. ## Fixing and preventing it The fix is to rename the file — and to *restart the process*. Renaming does not help a running interpreter, because the wrong module object is already sitting in `sys.modules` and every subsequent `import random` will keep returning it. Prevention is structural rather than vigilance: * **Put application code in a package.** If your modules live in `myapp/`, the top-level names on the path are `myapp` and nothing else, so the collision surface shrinks to one name you chose. * **Do not develop from a directory full of loose top-level modules**, especially one that is also the current working directory of your tooling. * **Use `-P`, or `PYTHONSAFEPATH=1`** (both 3.11 and later) when you want CPython to stop prepending the script directory or the current directory entirely; imports then resolve only from installed locations. * **Check the name before you create the file.** `python -c "import sys; print('email' in sys.stdlib_module_names)"` costs nothing. The same mechanism bites one level up. A *directory* on the path named like an installed distribution's importable package shadows that installed package just as completely, and because installed distributions and importable packages often have different names, the message you get can name something you never installed. The rule that resolves every one of these cases is the same: the import machinery searches `sys.path` in order and stops at the first hit, so whoever is earlier on the path wins.