Where does io.StringIO stop being a faithful stand-in for a file opened by open()?

### What the two in-memory streams actually are `io.StringIO` is a text stream whose backing store is a `str` in memory; `io.BytesIO` is a binary stream backed by `bytes`. Both implement the same file-object protocol as the objects `open()` returns — `read`, `write`, `seek`, `tell`, iteration by line, the context-manager protocol — which is why they slot into code that was written to take a file object. `getvalue()` returns everything written without disturbing the stream position, which is what makes them so convenient as test sinks. The mistake is treating that protocol overlap as full equivalence. `open()` returns, by default, an `io.TextIOWrapper` sitting on a buffered binary reader sitting on a real operating-system file. Each of those layers does something an `io.StringIO` does not. ### Gap one: no encoding step This is the expensive one. A text file is bytes on disk; `open()` decodes them with the `encoding` argument, or with the locale encoding when you omit it. `io.StringIO` starts from `str`, so there is no decode at all. A test that seeds `io.StringIO("café")` proves the parser handles that text — it proves nothing about whether the file will decode. The production failures that follow are the familiar ones: a `UnicodeDecodeError` on a latin-1 byte the test never saw, or mojibake from a locale-dependent default encoding differing between your laptop and a container. When encoding is part of what you are testing, keep the decode in the test: ```python raw = io.BytesIO("café\n".encode("utf-8")) stream = io.TextIOWrapper(raw, encoding="utf-8", newline="") ``` Now the bytes are real, the encoding is explicit, and feeding it malformed bytes produces the same `UnicodeDecodeError` the production path would. The related habit: always pass `encoding=` to `open()` in application code. `-X warn_default_encoding` (added in 3.10, also `PYTHONWARNDEFAULTENCODING`) emits an `EncodingWarning` at every call site that omitted it, which is the fastest way to find them; on 3.14 the omitted default is still the locale encoding, reported by `locale.getencoding()`. ### Gap two: newline handling differs by default `open()` defaults to `newline=None`, universal-newline mode: `\r\n` and `\r` in the file become `\n` in the string you read, and `\n` you write becomes the platform separator. `io.StringIO` defaults to `newline="\n"`, which performs no translation on the initial value or on writes. ```pycon >>> io.StringIO("a\r\nb").read() 'a\r\nb' >>> io.StringIO("a\r\nb", newline=None).read() 'a\nb' ``` A line-splitting routine tested only against `io.StringIO` with CRLF input therefore sees carriage returns that the real file would already have stripped, or the reverse. Pass `newline=None` when you want to mimic the default `open()` behaviour, and `newline=""` when the code does its own newline handling. ### Gap three: there is no file `io.StringIO` has no descriptor. `fileno()` exists on the object but raises `io.UnsupportedOperation`, and so does anything else that requires a real file underneath: passing the stream to a child process as its stdin, memory-mapping it, taking a lock on it, calling `os.stat` on it. It also has no name and no path, so code that does `stream.name`, or that reopens the file by path, or that renames it after closing, cannot be tested this way at all. That last case is the tell: if the code under test needs a *path*, an in-memory stream is the wrong tool and a `tempfile.TemporaryDirectory` is the right one. A smaller difference in the same family: `io.StringIO` never actually flushes anywhere, so a test can never catch a bug where the code forgets to flush or close before another reader opens the file. ### Choosing between them The decision is short. Use `io.StringIO` when the unit genuinely takes a text stream and you are testing what it does with the characters — a formatter, a report renderer, a line parser. Use `io.BytesIO`, or `io.TextIOWrapper` over one, when bytes, encodings or binary formats are part of the behaviour. Use a real temporary file or directory when the code deals in paths, descriptors, permissions, renames or another process. Getting this wrong in the safe direction — a temp file where a `StringIO` would do — costs a few milliseconds; getting it wrong in the other direction ships an encoding bug. The design point underneath: a function that takes an already-open stream is more testable than one that takes a filename and opens it itself, because the caller owns the encoding decision and the test can supply any stream it likes. Splitting "open the file" from "process the stream" gives you both — a thin, boring path-handling wrapper, and a pure stream function you can test in memory without giving up the encoding coverage, because the wrapper's own test uses a real file.