skip to content

A package's __init__.py imports all its submodules eagerly — what does that cost you?

level: seniorimportance: should knowfreq 35%

answer

  1. It runs before anything else in the package
  2. Every consumer pays it, not just direct ones
  3. Bodies define things, they do not do things
  4. Measure before you trim
  5. -X importtime prints the bill

basics

~20 s

Everything init.py imports is paid for by every process that touches any part of the package, before any of its own code runs. That means slower start-up, more memory, import-time side effects, and hidden coupling between submodules.

solid answer

~40 s

`__init__.py` is the package body, so it executes on the first import of the package or of *any* submodule beneath it, before that submodule's own body runs. A body that imports every submodule therefore turns "import one thing" into "import all things": start-up latency and resident memory for a short-lived job that needed one module, and any import-time side effect — reading config, opening a connection, registering handlers — executed whether or not the caller wanted it. It also couples submodules to each other through the parent body, which is where circular-import failures come from, and it silently makes `pkg.anything` resolve, so consumers start depending on an ordering guarantee nobody wrote down. Measure the real bill with `-X importtime`, keep the body cheap and side-effect free, and let callers import what they use.

code

python · 14 lines
python
import pathlib
import sys
import tempfile

root = pathlib.Path(tempfile.mkdtemp())
pkg = root / "recon"
pkg.mkdir()
(pkg / "__init__.py").write_text("print('recon/__init__.py body running')\n")
(pkg / "ledger.py").write_text("print('recon/ledger.py body running')\n")
sys.path.insert(0, str(root))

import recon.ledger   # parent body first, then the child
import recon.ledger   # prints nothing: both are already in sys.modules
print(recon.ledger is sys.modules["recon.ledger"])

go deeper

for a junior

Know the basic fact: the package's init.py runs whenever anything inside the package is imported, so whatever it imports gets imported too, even for code that needed only one submodule.

for a middle

Explain the ordering — parent body before child body, once per process — and name the concrete costs: start-up time, resident memory, side effects executed at import, and circular-import risk.

for a senior

Show the diagnosis: measure with -X importtime, separate latency from side effects, and know that trimming a parent body is a contract change for every caller that relied on the names it exposed.

for a principal

Set the policy across the codebase: whether package bodies may import submodules, whether import-time side effects are ever allowed, and how you keep a growing team from accreting convenience re-exports that become an undocumented import-order dependency.

## The mechanic underneath the cost Importing `pkg.ledger` imports `pkg` first: the parent package body must run before the child body. So `__init__.py` is not an opt-in file that only direct users of the package pay for — it is on the path of every import that goes anywhere near the package. Anything it imports is transitively imported for all of them, once per process, at whatever moment the first such import happens. ## Four distinct costs **Latency and memory.** For a long-running service, a fat package body is a one-off cost at boot and rarely worth arguing about. For a short-lived process it dominates. Take a payment reconciliation job invoked once per settlement batch: a run that does a few seconds of real work but spends a large fraction of its wall time executing import bodies it never calls into is paying the fat body on every invocation, multiplied by the batch count. Every module executed also stays resident in `sys.modules` for the life of the process. **Import-time side effects.** Code at module level in an eagerly imported submodule runs during import. Opening a database connection, reading environment configuration, spawning a thread, or contacting a service at import time makes the package hostile to test collection, to `--help`, to any tool that merely imports your code, and to worker processes that re-import the entry module under a spawning start method. The rule that survives contact with production is: a module body defines things; it does not do things. **Circular-import fragility.** A parent body that imports its children while a child imports something from the parent produces a partially initialized module and an `ImportError` that names the wrong file. The more the parent body pulls in, the larger the graph in which that can happen, and the more likely a routine refactor trips it. **Hidden coupling and the ordering assumption.** This is the one that hurts an eleven-person team the most. Once the parent body imports everything, `pkg.anything` resolves from any module in the process, and people write it. Nothing in the code says whether `pkg.reports` is part of the package's offering or an accident of the parent body. The day someone trims one line from `__init__.py` for start-up time, unrelated modules break with `AttributeError`, and the change that caused it is nowhere near the failure. The convenience is real and so is the coupling; the team has to decide which it is buying. ## How to decide what stays Measure first. Run the entry point with `-X importtime` and read the cumulative column: it prints each imported module and its self and cumulative cost, and it names the parent body's real bill instead of your guess about it. `python -X importtime -c "import pkg"` against a package you own is usually enough to find the one heavy transitive import that dominates. Then apply a simple standard. Keep in `__init__.py` only what is cheap and pure: a small set of names callers genuinely use constantly, a version constant, nothing that touches the network, filesystem or clock. Move heavy submodule imports out of the body and let callers import them by name — explicit imports also document what a module really depends on. Where a name must stay importable from the package but its module is expensive, Python has a supported hook for deferring the import until first attribute access, which keeps the public spelling without the up-front cost. Decide the policy once, per codebase, rather than per package: whether `__init__.py` files are allowed to import submodules at all, and whether import-time side effects are ever permitted. A team that has not decided ends up with both conventions in the same repository, which is worse than either. ## What the interviewer is listening for That you know the parent body runs for everyone, not only for direct importers. That you separate the *latency* argument (measurable, sometimes irrelevant) from the *side-effect* and *coupling* arguments (which are correctness and maintenance issues and do not go away on a fast machine). And that you measure with `-X importtime` before trimming, rather than deleting imports until something breaks.

  • How do you tell whether a fat package body is actually hurting a given workload?
    Compare its cost to the process lifetime. Run the entry point under `-X importtime` and read the cumulative column for the package: a few hundred milliseconds is noise in a service that boots once and serves for days, and is the whole story in a job that runs for two seconds and is invoked thousands of times a day. Optimise where the ratio is bad, not where the number is big.
  • You want to trim the parent body but other modules already rely on the names it exposes. What now?
    Treat it as a contract change, not a cleanup. Find the real usages, decide which names the package will keep offering, and make the callers of the rest import their module directly. For a name that must stay importable from the package but is expensive, defer it to first attribute access rather than importing it in the body.
  • Why are import-time side effects worse than import-time slowness?
    Slowness is measurable and bounded; a side effect changes what importing means. Opening a connection or reading config at import time breaks test collection, breaks tooling that merely imports the module, and re-runs in every worker process that re-imports the entry module. Slowness costs seconds; side effects cost correctness.

saying these in an interview costs you the question

  • Thinks only direct importers of the package pay the cost
  • Treats module-level connection setup as normal
  • Trims imports by guessing instead of measuring
  • Ignores that removing a re-export breaks callers
  • Says start-up cost never matters for any workload

context