Implicit Namespace Packages

### The three outcomes of an import scan An `import ingest` at top level asks the import system to find the name `ingest`. It consults `sys.modules` first, then walks the finders over each entry of `sys.path` **in order**. For each entry, three things can happen: 1. The entry contains `ingest/__init__.py` — a **regular package**. The scan stops here; the `__init__.py` body is executed as the package's module body. 2. The entry contains `ingest.py` (or a compiled extension of that name) — a plain **module**. The scan stops here too. 3. The entry contains a directory `ingest` with no `__init__.py` — the path is **recorded as a portion** and the scan *continues* to the next entry. Only if the scan reaches the end of `sys.path` having recorded at least one portion and never hit case 1 or 2 does the import system create an **implicit namespace package**. That rule ordering is the whole design: a real package or module anywhere on the path beats every bare directory, so namespace packages never shadow ordinary code. ### What the resulting object looks like The namespace package is an ordinary module object with a few tells: - `__file__` is `None` — there is no source file behind it. Code that does `os.path.dirname(pkg.__file__)` to locate data files raises `TypeError` here; use `importlib.resources.files()` instead. - `__spec__.origin` is `None`, and `repr(pkg)` says `(namespace)`. - `__path__` is not a plain list. It is a dynamic path object that **recomputes itself from `sys.path`**, so a portion added to the search path after the import becomes importable without a reload. - No module body ran. There is no `__all__`, no re-exported submodules, no version constant, no logging configuration — every submodule must be imported explicitly by its own name. ### Why PEP 420 exists Before Python 3.3, sharing a single top-level name across separately installed projects required cooperating boilerplate: an `__init__.py` in every distribution calling `pkgutil.extend_path`, or a setuptools-specific declaration. Every distribution had to ship the same shim, and if one of them shipped a plain `__init__.py` by mistake, it silently hid the others. PEP 420 moved the merge into the import system itself: leave the shared directory empty of `__init__.py` in *all* distributions and the interpreter assembles the merged `__path__` for you. `pkgutil.extend_path` still exists for legacy layouts, but new code does not need it. ### The accident case Most engineers meet namespace packages by accident rather than by design. Running a test runner via `python -m` or a script from the repository root puts the source directory on `sys.path`, so a package directory whose `__init__.py` was never created — or was deleted in a rename, or excluded by a `.gitignore` rule matching `__init__.py` — still imports perfectly from the checkout. The failure surfaces somewhere else entirely: a build backend's automatic package discovery enumerates directories that contain `__init__.py` and quietly leaves the others out of the wheel, and `unittest`'s test discovery dropped its namespace-package support in Python 3.11, so a `tests/` directory without `__init__.py` is skipped and the run exits with status 5, "NO TESTS RAN". Both symptoms look unrelated to the missing file. ### An empty directory counts too The rule needs no `.py` files at all: an empty directory on the search path imports cleanly as a namespace package, which is how a typo in a directory name, a stale build output folder, or an unpacked archive can shadow nothing yet still satisfy an import that should have failed loudly. The import succeeds, the package has no attributes, and the first `AttributeError` arrives somewhere far from the cause. It is also why `import somepkg` succeeding is not evidence that `somepkg` is installed — only that some directory of that name sits on the path. ### Telling the two apart At runtime, `pkg.__file__ is None` is the one-line check; `importlib.util.find_spec("pkg")` answers the same question *before* importing, since a namespace spec has `origin` of `None` and a populated `submodule_search_locations`. In a repository, the check is structural: any directory that contains `.py` files but no `__init__.py` is either a deliberate namespace portion or a bug, and it is worth a CI lint that forces you to say which. ### When it is the right answer Deliberate uses are narrow but real: a company-wide top-level name whose subpackages ship on independent release cadences, a plugin namespace third parties can drop into, or a directory of scripts you never intended to be a package at all. Everywhere else, an explicit — even empty — `__init__.py` is the safer default: it pins the package to one directory, gives you a place to hang `__all__` and version metadata, and makes the packaging and test tooling see what you see.