skip to content

Files, Paths, and I/O

Reading and writing files, where encoding, resource leaks and path handling quietly go wrong: open() and its modes, pathlib, temp files, stream buffering, and writes that survive a crash.

part ofPythonoverview, primer and where to startread it →
on this pageshow

questions

20

What do open()'s 'w', 'a' and 'x' modes each do to an existing file?

level: juniorimportance: must knowfreq 70%

answer

  1. One letter decides create-or-destroy
  2. Two of them will make the file
  3. One of them empties it immediately
  4. One refuses when the file exists
  5. '+' adds a direction, never truncation

basics

~10 s

'w' truncates the file to zero bytes, creating it if absent. 'a' leaves the contents alone and writes at the end. 'x' refuses to touch an existing file and raises FileExistsError.

solid answer

~40 s

The mode string passed to `open()` picks exactly one creation behaviour. `'r'` (the default) requires the file to exist and raises `FileNotFoundError` otherwise. `'w'` creates the file or **truncates it to zero bytes at open time**, before you have written anything. `'a'` creates the file if needed and forces every write to the current end, so a `seek()` does not move the write position back. `'x'` is exclusive creation: it succeeds only if the file did not exist, raising `FileExistsError` if it did, and the check-and-create happens in one operation in the OS rather than as a separate existence test. A `'b'` makes the stream binary (`bytes` in and out) and a `'+'` adds the missing direction, so `'r+'` reads and writes without truncating while `'w+'` still truncates.

code

python · 19 lines
python
import os

with open("notes.txt", "w", encoding="utf-8") as f:
    f.write("first\n")

with open("notes.txt", "a", encoding="utf-8") as f:
    f.write("second\n")

with open("notes.txt", encoding="utf-8") as f:
    print(f.read())

try:
    open("notes.txt", "x", encoding="utf-8")
except FileExistsError as exc:
    print("x refused:", exc.filename)

with open("notes.txt", "w", encoding="utf-8"):
    pass
print("size after w:", os.path.getsize("notes.txt"))

go deeper

for a junior

Memorise what each letter does to a file that already exists: 'r' reads it, 'w' empties it, 'a' adds to the end, 'x' refuses. Know that 'r' is the default and that 'b' switches you from str to bytes.

for a middle

Explain the mechanics: truncation happens at open time, append is enforced per write by the OS flag, and '+' adds a direction without changing creation behaviour. Be able to say why 'w+' is not the safe read-write mode people assume.

for a senior

Show judgement about data loss. Justify 'x' over check-then-open for anything another process might create, and explain why replacing a file people are reading means writing a new name rather than truncating the original.

for a principal

Own the convention across a codebase: which paths may ever be opened 'w', where appends are the contract for multiple writers, and how the team stops an incidental 'w' from being the reason a night's export is empty.

`open()`'s second argument is not an enum but a short string of independent flags. Exactly one character chooses the *creation and access* behaviour — `r`, `w`, `a` or `x`; an optional `b` or `t` chooses binary or text (text is the default); and an optional `+` adds the direction the base letter lacks. `open(path)` therefore means `open(path, "rt")`. ## The four base modes **`'r'` — read an existing file.** The file must exist; otherwise you get `FileNotFoundError`. Nothing is created and nothing is modified. This is the default, which is why a bare `open(path)` never destroys data. **`'w'` — create or truncate.** If the file does not exist it is created. If it does exist, it is emptied *at the moment of the open call*, not when you first write. That timing is the part candidates miss: if your program opens a file for writing and then raises before writing anything, you have replaced a good file with an empty one. Anywhere the old contents still matter, write to a new name and swap it into place rather than truncating the original. **`'a'` — append.** The file is created if missing, and existing bytes are never removed. The important detail is that appending is a property of every write, not just of the initial position: the stream is opened with the OS-level append flag, so each write goes to the current end of file even if you called `seek()` first. Two processes appending short records to the same file therefore do not overwrite each other's bytes the way two `'w'`-mode writers would. In `'a+'` you can read as well, and there `seek()` does move the *read* position while writes still land at the end. **`'x'` — exclusive creation.** Succeeds only if the file did not already exist, and raises `FileExistsError` if it did. It is worth preferring over the pattern of testing for existence first and then opening with `'w'`: between your test and your open, another process can create the file, and you silently clobber it. `'x'` pushes the test and the create into a single OS operation, so the loser of the race gets an exception instead of destroying the winner's file. Lock files, run-once markers and "do not overwrite an export that already exists" checks are its natural home. ## The modifiers **`'b'` versus text.** Without `b` you get a text stream: you read and write `str`, the bytes are decoded and encoded through a codec, and line endings are translated. With `b` you get raw `bytes`, no codec and no line-ending translation. A binary-mode open also rejects the `encoding` and `newline` arguments with `ValueError`, which is a useful reminder that those two only exist for text. **`'+'`.** `'r+'` opens an existing file for reading and writing and leaves the contents intact, positioned at the start; `'w+'` also reads and writes but still truncates first; `'a+'` reads and appends; `'x+'` creates exclusively and allows reading. The `+` never changes whether the file is truncated or created — that is entirely the base letter's job. This is where the common mix-up lives: people reach for `'w+'` when they meant "open it for read and write" and are surprised the file is empty. ## Choosing in practice Ask two questions. First, *must the file already exist?* If yes, `'r'` or `'r+'`. If it must **not** exist, `'x'`. If you do not care, `'w'` or `'a'`. Second, *do I want the old contents?* If yes, `'a'` or `'r+'`; if no, `'w'`. A few consequences follow directly. `'w'` is the mode that loses data, so treat every `'w'` on a path a user cares about as a decision, not a default. `'a'` is the mode for logs and for anything appended by more than one writer. `'x'` is the mode that turns a race into an exception. And because the truncation in `'w'` happens at open time, holding the file open for a long stretch of work while other readers watch it is a design choice you should make deliberately rather than by accident. Finally, none of these letters say anything about *when* your bytes reach the disk — that is the stream's buffering, a separate concern from the mode. The mode only decides what exists when the call returns and where writes are allowed to land.

  • What is the difference between opening a file with 'r+' and with 'w+'?
    Both give a readable and writable stream. `'r+'` requires the file to exist, keeps its contents and starts at offset zero, so you can overwrite parts of it in place. `'w+'` creates the file if needed and truncates it to zero bytes first, so there is never anything to read until you have written it. The `+` only adds a direction; the base letter still decides truncation.
  • Why is opening with 'x' better than checking whether the path exists and then opening with 'w'?
    The check-then-open pattern has a window between the two steps in which another process can create the file, and your `'w'` then truncates it. `'x'` asks the operating system to create the file only if it does not exist, as a single indivisible operation, so the loser of that race gets `FileExistsError` rather than silently destroying the winner's data.
  • What happens if you pass encoding= to a file opened in binary mode?
    `open()` raises `ValueError`. Binary mode hands you raw `bytes` with no codec and no line-ending translation, so `encoding` and `newline` are meaningless there and are rejected rather than ignored. If you need decoded text you either open in text mode or decode the bytes yourself.

'w' is handing the clerk a fresh blank form, 'a' is adding a line to the bottom of the existing form, and 'x' is being told there is already a form on file and you may not start a second one.

saying these in an interview costs you the question

  • Says 'w' appends when the file already exists
  • Thinks 'w' only truncates once you write
  • Believes seek() moves the write position in 'a' mode
  • Thinks 'r' creates the file when it is missing
  • Treats 'w+' as a non-destructive read-write mode
  • Thinks the mode letters control flushing to disk

context

open as a page

How does pathlib.Path use the / operator to join paths, and what do .name, .stem and .parent return?

level: juniorimportance: must knowfreq 72%

basics

~10 s

pathlib.Path overloads the division operator, so Path("exports") / "schedule.json" builds a new path object with the right separator. On that result .name is "schedule.json", .stem is "schedule", .suffix is ".json" and .parent is Path("exports").

open as a page

When do you use tempfile.NamedTemporaryFile, TemporaryDirectory or mkstemp?

level: juniorimportance: must knowfreq 58%

basics

~10 s

tempfile.TemporaryDirectory gives a whole scratch directory removed when its with-block ends. tempfile.NamedTemporaryFile gives one auto-deleted open file. tempfile.mkstemp gives a raw file descriptor plus a path you must close and unlink yourself.

open as a page

How does the write-to-temp-then-os.replace pattern make a file update atomic?

level: middleimportance: must knowfreq 45%

basics

~20 s

The new contents go into a temporary file in the target's own directory; os.replace(tmp, target) then swaps the directory entry in one step, so a reader sees either the whole old file or the whole new one.

open as a page

Which encoding does open() use in text mode when encoding= is omitted?

level: middleimportance: must knowfreq 60%

basics

~20 s

On Python 3.14 a text-mode open() with no encoding= uses the machine's locale encoding, not UTF-8. The same script can read a file on one machine and raise UnicodeDecodeError on another, so pass encoding= explicitly.

open as a page

Why can open(path, 'w') expose an empty or half-written file to readers?

level: juniorimportance: should knowfreq 30%

basics

~20 s

Mode 'w' truncates the file the instant it is opened, before a single byte is written, and later writes reach it piecemeal as buffers flush. A reader that opens the path meanwhile sees an empty or half-written file.

open as a page

What does the flush=True argument to Python's print() do?

level: juniorimportance: should knowfreq 45%

basics

~20 s

flush=True makes print() push its text out of Python's own userspace buffer to the operating system immediately, instead of leaving it there until the buffer fills. Without it, output can sit unseen for a long time.

open as a page

Why does os.replace raise OSError with errno EXDEV when the temp file is in /tmp?

level: middleimportance: should knowfreq 28%

basics

~20 s

A rename only rewrites a directory entry inside one filesystem; it cannot move data between filesystems. When /tmp is a separate mount from the target, os.replace fails with EXDEV. Create the temporary file in the target's own directory instead.

open as a page

In Python's open(), what do buffering=0, buffering=1 and -1 mean?

level: middleimportance: should knowfreq 35%

basics

~20 s

0 means unbuffered and is legal only in binary mode; 1 means line buffering and is meaningful only in text mode; -1 is the default block buffering. Any other positive number is an explicit buffer size in bytes.

open as a page

How do you make Python's sys.stdout line-buffered from inside a running program?

level: middleimportance: should knowfreq 40%

basics

~10 s

Call sys.stdout.reconfigure(line_buffering=True). It flushes the text layer and then switches it to flush on every newline, so piped output stops stalling, without touching the launch command or any print() call.

open as a page

What does open()'s newline= argument control in text mode?

level: middleimportance: should knowfreq 42%

basics

~20 s

It controls line-ending translation. The default newline=None turns any of \r\n, \r or \n into \n when reading and turns \n into the platform's line ending when writing; newline='' disables translation while still splitting lines.

open as a page

What is the difference between Path.glob and Path.rglob in pathlib, and what do they return?

level: middleimportance: should knowfreq 55%

basics

~10 s

Both return a lazy generator of Path objects, not a list. Path.glob(pattern) matches the pattern against entries under that directory; Path.rglob(pattern) is shorthand for glob("**/" + pattern), so it searches every subdirectory recursively.

open as a page

What do the parents and exist_ok arguments to pathlib's Path.mkdir() do?

level: middleimportance: should knowfreq 50%

basics

~20 s

parents=True creates every missing intermediate directory instead of raising FileNotFoundError. exist_ok=True suppresses FileExistsError when the target directory already exists. Both default to False, so a bare mkdir() demands that the parent exists and the target does not.

open as a page

What is the difference between shutil.copy, shutil.copy2 and shutil.move?

level: middleimportance: should knowfreq 50%

basics

~10 s

shutil.copy duplicates contents plus permission bits. shutil.copy2 additionally preserves metadata such as modification and access times. shutil.move relocates a path, renaming it when possible and otherwise copying with copy2 and deleting the source.

open as a page

After os.replace returns, why is os.fsync on the containing directory still needed?

level: seniorimportance: should knowfreq 30%

basics

~20 s

os.replace makes the swap visible to other processes at once, but the new directory entry may still live only in the kernel's cache. A power loss can revert the name, so durability needs os.fsync on the file and on its directory.

open as a page

A Python ad-auction bidder is SIGKILLed after a 45-second cold start and its log file ends mid-line — why, and what do you change?

level: seniorimportance: should knowfreq 40%

basics

~10 s

The tail of the output was still in the process's userspace buffer. SIGKILL cannot be caught, so no flush, with-block exit or atexit handler runs, and the buffer dies with the process.

open as a page

An importer's helper returns a lazy line generator built inside `with open(...)`; why does the caller get ValueError?

level: seniorimportance: should knowfreq 35%

basics

~20 s

The with block ends when the helper returns, so the file is closed before the caller consumes the lazy generator, and the first read raises ValueError. Fix it by materialising inside the block or yielding from a generator function.

open as a page

In pathlib, how does Path.resolve() differ from Path.absolute(), and how do you check a path stays under a base directory?

level: seniorimportance: should knowfreq 44%

basics

~10 s

Path.absolute() only prepends the current working directory; it leaves symlinks and '..' in place. Path.resolve() also follows symlinks and eliminates '..'. Containment means resolving both the base and the candidate, then calling candidate.is_relative_to(base).

open as a page

What can go wrong when shutil.rmtree deletes a directory tree?

level: seniorimportance: should knowfreq 34%

basics

~20 s

It can stop partway and leave a half-deleted tree, refuse a path that is itself a symlink, and fail on read-only or still-open files. Handle failures with the onexc callback or ignore_errors, and never point it at unvalidated input.

open as a page

Why is a temp file path built from tempfile.gettempdir() and a fixed name unsafe?

level: seniorimportance: should knowfreq 42%

basics

~20 s

The system temp directory is shared and world-writable, so a guessable name is a race: another process can create that path first, often as a symlink, and your write lands somewhere else. It also makes concurrent runs of your own job collide.

open as a page