How does CPython choose sys.stdout's text encoding, and how do you force it to UTF-8?

## Where the encoding comes from Writing `str` to a stream requires encoding it to bytes, and something must pick the codec. `sys.stdout` is an `io.TextIOWrapper` created during interpreter startup, and the encoding it is given is derived from the process's locale — the same value `locale.getencoding()` reports. This is decided once. Nothing re-reads the locale later, so a program cannot change it by calling into the locale module halfway through. The **error handler** matters just as much as the codec, and the two standard output streams are given different ones on purpose: * `sys.stdout` uses a strict handler in the ordinary case, so a character the codec cannot represent raises `UnicodeEncodeError` rather than silently mangling data. * `sys.stderr` uses `backslashreplace`. Unrepresentable characters become escape sequences instead of an exception — deliberately, so that a traceback containing an exotic character can always be printed. A diagnostic that cannot be displayed is worse than a lossy one. You can read both at runtime: `sys.stdout.encoding` and `sys.stdout.errors`. ## Why this shows up as an environment bug The classic report is "it works on my machine and crashes on the host". The developer's shell has a UTF-8 locale; the host runs the process under a minimal or legacy environment whose locale encoding is an 8-bit codec. The first non-ASCII character — a name, a currency symbol, a check mark in a status line — raises `UnicodeEncodeError` on a `print()` that has never failed anywhere else. The code is identical; the interpreter's startup configuration is not. One mitigation is already built in: **UTF-8 mode is enabled automatically when the locale is `C` or `POSIX`**, so the most common bare-environment case does end up with UTF-8. It is the genuinely non-UTF-8 locale, not the empty one, that still bites. ## The three overrides **`PYTHONIOENCODING=encoding[:errors]`** — the narrowest. It changes only the standard streams, and it takes an optional error handler after a colon: `PYTHONIOENCODING=utf-8:backslashreplace`. Use it when the process must emit UTF-8 regardless of where it runs and you do not want to touch anything else about text handling. **UTF-8 mode**, enabled with `-X utf8` on the command line or `PYTHONUTF8=1` in the environment — the broadest. It makes the interpreter ignore the locale for text encoding purposes and use UTF-8 as the locale-encoding default throughout, with `surrogateescape` on stdin and stdout so undecodable input round-trips rather than raising. `sys.flags.utf8_mode` reports whether it is on. On CPython 3.14 it is **opt-in** except for the automatic C/POSIX case above — do not write code that assumes it is on. **`sys.stdout.reconfigure(encoding="utf-8", errors="backslashreplace")`** — from inside the program, available on `io.TextIOWrapper` since Python 3.7. It flushes and re-creates the text layer over the same buffered stream. The catch is ordering: it only governs writes that happen after it runs, so it belongs as close to the top of the entrypoint as you can put it, and it cannot help output produced by an import that ran earlier. As a rule, prefer configuring the environment for a service you deploy, and reach for `reconfigure` when you are shipping a program whose environment you do not control and cannot instruct. ## Two things not to confuse it with `sys.getdefaultencoding()` always returns `'utf-8'` on Python 3 and is about the implicit `str`/`bytes` conversions inside the interpreter — it says nothing about your streams. And the encoding of `sys.stdout` is independent of its **buffering**: `PYTHONIOENCODING` does not make output appear sooner, and `PYTHONUNBUFFERED` does not make an unrepresentable character printable. They are two separate startup decisions about the same object. ## Verifying instead of assuming Print `sys.stdout.encoding`, `sys.stdout.errors` and `sys.flags.utf8_mode` from inside the deployed environment. Those three values explain essentially every mystery in this area, and they take one line to obtain. If the answer is a legacy codec, decide deliberately: force UTF-8 and keep the data intact, or keep the locale's codec and choose a lossy error handler so the process cannot be brought down by a character.