When do you use tempfile.NamedTemporaryFile, TemporaryDirectory or mkstemp?

## The problem `tempfile` solves Every non-trivial program eventually needs scratch space on disk: a file to stream a large download into, a directory for a tool that only knows how to read from a directory, a staging file that becomes the real output later. Hand-rolling that as `open("/tmp/job.csv", "w")` is wrong for three separate reasons. The system temp directory is shared and world-writable, so another user's process can interfere with a name you chose. The name is fixed, so two runs of your own program collide. And nothing removes the file, so the directory fills up over months. `tempfile` addresses all three, and it offers three shapes depending on how much control you need. ## `TemporaryDirectory` — a whole scratch tree `tempfile.TemporaryDirectory()` creates a fresh directory with permissions `0o700` and a random name. Used as a context manager, entering it returns the directory path as a `str`; leaving it removes the entire tree with `shutil.rmtree`. It also exposes a `cleanup()` method for non-`with` use, and it registers a finalizer so the tree still goes away if the object is simply garbage-collected. Two keyword arguments matter in production. Since Python 3.10, `ignore_cleanup_errors=True` swallows failures during that removal — valuable on Windows, where a file another process still has open cannot be deleted and would otherwise raise from the `__exit__`. Since Python 3.12, `delete=False` keeps the tree on disk while still giving you the context-manager shape, which is how you leave evidence behind for a post-mortem without restructuring the code. This is the default choice whenever a step produces more than one artefact, because cleanup becomes one operation over one root instead of bookkeeping per file. ## `NamedTemporaryFile` — one file with a usable name `tempfile.NamedTemporaryFile()` returns an already-open file object, default mode `'w+b'`, whose `.name` attribute is a genuine path on the filesystem. When it is closed — explicitly or by leaving its `with` block — it deletes itself. The classic wrinkle is Windows: an open file cannot be opened a second time by name there, so the natural pattern "write it, close nothing, hand `.name` to another component" fails on that platform. Python 3.12 added the `delete_on_close` parameter for exactly this. With `delete=True, delete_on_close=False`, `close()` no longer deletes; removal is deferred to the context-manager exit, so you can close, reopen by name, read it back, and still get automatic cleanup. Two relatives round out the family. `tempfile.TemporaryFile()` has no usable name at all and is the cheapest option when only your own process will read the data. `tempfile.SpooledTemporaryFile()` keeps content in memory until it exceeds a `max_size` and only then rolls over to disk. ## `mkstemp` — the low-level primitive `tempfile.mkstemp()` returns a two-tuple `(fd, path)`. The `fd` is an operating-system file descriptor opened with the exclusive-create flags and mode `0o600`; the `path` is absolute. Nothing is registered for cleanup. You wrap the descriptor with `os.fdopen` (or close it with `os.close`) and you call `os.unlink` yourself, normally in a `finally`. Reach for it when the file must outlive the object that made it, or when you need the descriptor rather than a Python file object — for example a file you hand to a child process, or one whose deletion is conditional on success. `tempfile.mkdtemp()` is the same idea for directories: it returns a path and cleans up nothing. ## Choosing, in one pass - Several files, or a directory some tool writes into → `TemporaryDirectory`. - One file, automatic cleanup, name occasionally needed → `NamedTemporaryFile`. - A file whose lifetime you control explicitly → `mkstemp`. - `tempfile.mktemp()` → never; it returns a name without creating anything and is unsafe. ## Where the files land `tempfile.gettempdir()` resolves the base directory from the `TMPDIR`, `TEMP` and `TMP` environment variables, then platform defaults, and caches the result. Every helper above accepts a `dir=` argument to override it per call. Overriding is worth doing when the default temp filesystem is a small RAM-backed mount that a multi-gigabyte scratch file would exhaust, or when you intend to rename the file into its final directory afterwards and want to stay on one filesystem. `shutil.disk_usage(path)` returns a named tuple of `total`, `used` and `free` bytes and is the cheap way to check before writing something large. ## Cleanup is a discipline, not a guarantee Context managers cover normal exits and exceptions. They do not cover a hard kill or a power loss, so a long-lived service that creates temp files should still expect stragglers and sweep its own scratch directory on startup — which is much easier when everything it writes lives under one directory it created itself.