Why does os.replace raise OSError with errno EXDEV when the temp file is in /tmp?

## What a rename actually does Renaming does not touch file contents. It removes a name from one directory and adds it to another, both within the same filesystem, leaving the underlying file exactly where it was. That is why it is fast regardless of file size and why it can be atomic at all. It is also why it cannot cross a boundary: a directory entry refers to a file by an identifier that is only meaningful inside its own filesystem, so there is nowhere for the destination directory to point. The kernel does not silently upgrade the operation to a copy; it returns the `EXDEV` error, and Python raises `OSError` with `errno` set to `errno.EXDEV`. ## Why /tmp trips it so reliably On most modern Unix systems `/tmp` is not part of the root filesystem. It is commonly a memory-backed filesystem, or its own partition, or - inside a container - a mount separate from the volume where application data lives. Meanwhile the target of an atomic write is usually under a data directory on a different mount, often a network or block volume attached specifically to hold state. The two are different filesystems, so the rename cannot connect them. The symptom is characteristic and worth recognising in an interview: the helper works on a developer machine, where everything is one big filesystem, and fails immediately in production or in a container, where the data directory is a separate mount. A route-optimisation job publishing a 17-service dependency graph hits it the first time the graph file moves onto a dedicated volume, with a stack trace that names `os.replace` and an error message about a cross-device link. ## The wrong fix The tempting repair is to catch the error and fall back to copying the file over the destination, then deleting the temp file - which is what the standard library's higher-level move helper does. That is a legitimate way to *move* a file, and a completely illegitimate way to *replace* one atomically: copying over the destination truncates it and rewrites it in place, reintroducing exactly the window where a reader sees a partial file. If your fallback path is a copy, your atomic write is atomic only on machines where it was never needed. ## The right fix Derive the temporary file's directory from the target rather than from the environment: ```python import os import tempfile directory = os.path.dirname(os.path.abspath(target)) fd, tmp = tempfile.mkstemp(dir=directory) ``` The target's entry has to be created in that parent directory, so a temporary file created there is guaranteed to be on the same filesystem. This is stronger than comparing device numbers with `os.stat`, and it needs no probing or configuration. Two practical notes: the parent directory must already exist, because `mkstemp` will not create it; and if the target path is a symbolic link, resolving it first decides whether you replace the link or the file it points at - both are defensible, but the choice should be deliberate. ## os.rename versus os.replace Both are renames and both raise `EXDEV` across filesystems. They differ on an existing destination: `os.replace` overwrites it on every platform, while `os.rename` overwrites on Unix but raises `FileExistsError` on Windows, where renaming onto an existing name is not permitted. That is why the atomic-write recipe always specifies `os.replace` - a helper written with `os.rename` and tested only on Linux breaks the first time it runs on Windows, and the failure looks nothing like a filesystem problem. ## Windows and other edges On Windows the equivalent boundary is the drive letter: replacing a file on `D:` with a temporary file on `C:` fails for the same structural reason. Windows adds a failure mode Unix does not have - if another process holds the destination open without permitting deletion, the replace fails with a permission error rather than succeeding - so services that update files other processes read often wrap the swap in a short retry. And on Unix, a target that is itself a bind-mounted single file cannot be replaced by a rename at all, because the name and the mount are pinned together; that is a deployment shape to avoid for files you intend to update.