skip to content

How do you split one Python package namespace across two separately installed distributions?

level: seniorimportance: nice to knowfreq 18%

answer

  1. One name, several wheels
  2. Nobody may ship the init file
  3. Disjoint submodules, first portion wins
  4. Discovery must be told about the directories
  5. Install them together in CI and assert

basics

~10 s

Give every distribution the same top-level directory with no init.py in it, let each own disjoint submodules, and declare those directories to the build backend so they ship. The import system merges the portions.

solid answer

~50 s

Use a PEP 420 namespace: every distribution installs the same top-level directory, and **none** of them may contain an `__init__.py` in it — one regular package terminates the import scan and hides every other portion. Each distribution owns a disjoint set of submodules, since duplicate names resolve silently to whichever portion is earlier in `sys.path`. Configure the build so those directories actually ship: automatic package discovery keys on `__init__.py`, so you need namespace-aware discovery or an explicit package list. Do not mix PEP 420 with the older `pkgutil.extend_path` shim in the same namespace. Then verify it: install every distribution together in CI and assert that `list(pkg.__path__)` has the expected number of portions and that a submodule from each imports. Reach for this only when independent release cadence is the actual requirement; one distribution with optional extras is simpler.

code

python · 16 lines
python
import pathlib
import sys
import tempfile

root = pathlib.Path(tempfile.mkdtemp())
for dist, submodule in (("wheel_a", "parser"), ("wheel_b", "shipper")):
    portion = root / dist / "ingest"
    portion.mkdir(parents=True)
    (portion / f"{submodule}.py").write_text(f"OWNER = {dist!r}\n")
    sys.path.append(str(root / dist))

import ingest
from ingest import parser, shipper

print(len(list(ingest.__path__)), parser.OWNER, shipper.OWNER)
print(ingest.__spec__.origin, ingest.__file__)

go deeper

for a junior

Know that one top-level name can come from several installed projects and that the shared directory has no init.py. You will not usually be the person designing such a layout.

for a middle

Explain the mechanics you would follow: no init.py anywhere in the shared root, disjoint submodule names, and packaging configuration that actually includes the namespace directories in the wheel.

for a senior

Show the failure modes you would guard against — one distribution's init.py collapsing the namespace, silent submodule shadowing by path order, and an install layout that merges differently in production than in a checkout — plus the CI test that catches each.

for a principal

Own the decision and the governance: who holds the top-level name, how submodule ownership is allocated, whether independent release cadence justifies the split at all, and what gate stops any distribution from breaking the others.

### What the pattern buys you A shared namespace lets several separately released projects live under one top-level name — `ingest.parser` from one wheel, `ingest.shipper` from another — so consumers see a single coherent import surface while each part ships on its own schedule. That last clause is the only good reason to do it. If everything releases together on the same three-week train anyway, one distribution with optional dependency groups gives you the same import surface with none of the failure modes below. ### The rules every distribution must follow **No `__init__.py` in the shared directory — in any distribution.** This is rule zero and it is absolute. The import scan terminates at the first directory containing `__init__.py`, so a single distribution that adds one converts the namespace into an ordinary package containing only its own submodules; every other distribution's modules become unimportable. That makes adding the file a breaking change to code the author has never seen, which is why it belongs in a lint rather than in a convention document. **Own disjoint submodule names.** Portions are searched in `sys.path` order and the first hit wins, silently. Two distributions shipping `ingest/parser.py` produce no error, no warning — just one of them losing, depending on install order and path layout. Assign each distribution a documented prefix or subpackage and enforce it at review time. **Do not put anything but subpackages in the shared root.** There is no module body to hold a version constant, an `__all__`, logging setup or a convenience re-export, and there never can be. If consumers need `ingest.__version__`, the namespace pattern is the wrong shape — or that name has to live one level down in a subpackage that one distribution owns. **Make the build ship the directories.** Automatic package discovery looks for `__init__.py`; without it your namespace directories are quietly omitted from the wheel and consumers get an empty or partial namespace. Either enable the backend's namespace-aware discovery with a pattern that matches your layout, or list the packages explicitly — explicit is better here, because the failure mode of getting it wrong is silent absence rather than a build error. **Do not mix mechanisms.** Before PEP 420 the same effect required an `__init__.py` in every distribution calling `pkgutil.extend_path`, or a setuptools-specific declaration. Those still work in isolation, but mixing them with PEP 420 portions in one namespace reintroduces exactly the shadowing problem rule zero exists to prevent: the shim distribution's `__init__.py` terminates the scan. Pick one mechanism for the whole namespace, and for anything new that mechanism is PEP 420. ### Operational consequences **Installation location matters.** The portions merge only if each lands on a directory that is on `sys.path` at import time. Two virtual environments, a stray user-site install, or an editable install pointing at a checkout can all produce a namespace that merges differently in production than in development. `list(pkg.__path__)` in the target environment is the ground truth. **Uninstalling is imperfect.** Removing one distribution can leave an empty shared directory behind. An empty directory is still a portion, which is harmless for imports but confusing when you are diagnosing why a submodule is missing. **Version skew is now yours to manage.** Independent release cadence is the feature, so consumers can install combinations you never tested. If the parts share internal interfaces, publish version constraints between them; if you find yourself doing that heavily, the split was not real and the distributions want to be one. **Import cost rises slightly**, because a namespace scan cannot short-circuit and must visit every remaining `sys.path` entry. ### Verifying it works The test is an integration test, not a unit test, and it belongs in CI: install every distribution into one clean environment, then assert that `len(list(pkg.__path__))` equals the number of portions you expect and that one submodule from each distribution imports. Add the negative case if the namespace is public — build a wheel that wrongly contains an `__init__.py` in the shared root, install it, and confirm your check fails. That is a five-line test that prevents the one mistake capable of breaking every other distribution at once. ### When to say no Shared namespaces are governance, not just packaging: someone must own the top-level name, adjudicate submodule ownership, and hold the line on rule zero across teams that release independently. If no one owns it, the namespace decays into shadowed modules and mystery imports. A single distribution, or distinct top-level names with a thin facade, is the right default; a shared namespace earns its keep when independent release cadence and a unified import surface are both genuinely required.

  • What single mistake by one distribution breaks the whole namespace?
    Shipping an `__init__.py` in the shared top-level directory. The import scan terminates at the first directory containing one, so `__path__` collapses to that distribution's directory and every other distribution's submodules raise `ModuleNotFoundError`. Nothing warns; the break appears in downstream code the author never touched, which is why the rule belongs in CI.
  • How would you prove in CI that the namespace really merged?
    Install every distribution into one clean environment, then assert `len(list(pkg.__path__))` equals the expected portion count and import one submodule from each distribution. Run it after a real install rather than from the checkout, because the source tree merges differently from site-packages.
  • When would you choose one distribution with optional extras instead?
    Whenever independent release cadence is not a hard requirement. A single distribution gives the same import surface, keeps a place for version constants and re-exports, avoids silent submodule shadowing, and removes the cross-distribution version skew you otherwise have to constrain. Split only when the parts genuinely ship on different schedules or to different audiences.

It is a shared shelf in a communal library: everyone may add folders, nobody may nail a cover sheet to the shelf itself, and two people filing under the same label means one of them is never read.

saying these in an interview costs you the question

  • Adds an __init__.py to the shared directory for tidiness
  • Puts a version constant or re-exports in the shared root
  • Assumes the build ships namespace directories automatically
  • Mixes pkgutil.extend_path shims with PEP 420 portions
  • Lets two distributions ship the same submodule name
  • Tests the namespace only from the source checkout

context