Bytecode Cache Invalidation

## Two phases, one of them cacheable Importing a Python source module always does two things: compile the text into a **code object**, then execute that code object in a fresh module namespace to produce the module's attributes. Only the first phase can be cached. Execution has to happen once per process, which is why a `.pyc` never speeds up a running program at all: it only removes parse-and-compile work from the first import. ## Where the file lands Since PEP 3147 (Python 3.2) the cache lives in a directory named `__pycache__` next to the source, and the filename carries the interpreter's **cache tag**. So `catalog/loader.py` caches to `catalog/__pycache__/loader.cpython-314.pyc` on CPython 3.14. `sys.implementation.cache_tag` holds that tag, `importlib.util.cache_from_source()` computes the cache path for a source path, and `importlib.util.source_from_cache()` inverts it. Tagging matters because one checkout is routinely imported by more than one interpreter; before PEP 3147 a 3.x and a 3.y process would overwrite each other's cache on every import. The whole cache tree can be relocated: `sys.pycache_prefix` (settable with `-X pycache_prefix=DIR` or the `PYTHONPYCACHEPREFIX` environment variable) mirrors the directory structure under `DIR` instead of writing into the source tree. That is what you reach for when the source tree must stay pristine or is mounted read-only. ## What the header holds A `.pyc` starts with a fixed 16-byte header followed by the **marshalled** code object (the `marshal` format is a CPython implementation detail, versioned with the interpreter, and is not an interchange format you should hand-craft). * Bytes 0-3: the magic number, available as `importlib.util.MAGIC_NUMBER`. It changes whenever the bytecode format or the compiler's output changes, which is often between feature releases. A `.pyc` written by another version therefore fails on the very first check. * Bytes 4-7: a flags word, added by PEP 552 in 3.7. Bit 0 says whether the file is hash-based; bit 1, on a hash-based file, says whether the hash must be verified. * Bytes 8-15: in the default timestamp mode, the source's modification time truncated to 32 bits (one-second resolution) followed by the source's size truncated to 32 bits. In hash-based mode, an 8-byte hash of the source bytes instead. ## How staleness is decided On import, the loader stats the source file and reads the header. It rejects the cache if the magic number differs. In timestamp mode it then compares the stored mtime with the source's current mtime and the stored size with the source's current size — both are **equality** checks. That is worth saying out loud in an interview, because the common mental model is *the source is newer than the cache*. It is not: any change in either direction, and any change of size at the same timestamp, invalidates. The flip side is that a source whose mtime and size are both restored to exactly what they were is considered fresh even if its contents changed, which is the root of most stale-bytecode stories. On a mismatch the module is recompiled and the cache is rewritten atomically (write to a temporary file, then rename), so two processes importing at once cannot see a half-written file. ## When nothing is written Writing the cache is best-effort and never fatal: * `sys.dont_write_bytecode` is `True` when the interpreter was started with `-B` or with `PYTHONDONTWRITEBYTECODE` set, and no `.pyc` is written at all. * If the directory cannot be created or written, the write is silently skipped and the module is recompiled on every process start. There is no error, just slower startup. * Code compiled from a string with `compile()` or run through `exec()` is never cached, and neither is `__main__`: the script you name on the command line is compiled fresh every run, only the modules it imports are cached. * You can also populate the cache ahead of time with `py_compile.compile()` for one file or the `compileall` module for a tree, which is how a build step warms the cache before the process ever serves traffic. ## Sourceless imports One footgun worth knowing: the legacy pre-3.2 layout — a `.pyc` sitting directly beside where the source used to be, not inside `__pycache__` — is still importable, via `importlib.machinery.SourcelessFileLoader`, but only when no `.py` file exists there. That mechanism is behind the classic "I deleted the module and it still imports". A `.pyc` inside `__pycache__` is never used without its source. Finally, deleting `__pycache__` is always safe. The worst it can cost is one recompile.