skip to content

Why can't a password held in a Python str be erased from memory, and what can be?

level: seniorimportance: nice to knowfreq 20%

answer

  1. One property of str makes this impossible
  2. Nothing edits those characters in place
  3. Freed memory is not zeroed either
  4. You never had only one copy
  5. Mutable buffer, slice-assigned zeros, never resized

basics

~20 s

A str is immutable, so no operation overwrites its characters in place, and the interpreter usually holds several copies of them. A bytearray is mutable, so its bytes can be overwritten with zeros before the last reference is dropped.

solid answer

~50 s

Immutability is the whole problem: `s[0] = 'x'` raises `TypeError`, and every method that looks like editing a `str` returns a new object, so the original characters sit in the heap untouched until the last reference goes away. Freeing them does not zero them either, and by then copies typically exist anyway: a literal lives in the code object's constants for the module's lifetime, `sys.intern` can share short strings, and decoding bytes creates yet another object. A `bytearray` is mutable and fixed in place if you never resize it, so `buf[:] = bytes(len(buf))` in a `finally` really does overwrite the bytes; `memoryview` lets you pass slices around without copying, and `hmac.compare_digest` accepts bytes-like input so you never need to decode. Treat scrubbing as defence in depth, not a guarantee: swap, core dumps and container snapshots are outside your reach. The dominant control is a short lifetime and few copies.

code

python · 9 lines
python
s = "sk-live-8f3c"
b = s.encode()          # a second copy of those characters now exists
try:
    s[0] = "x"
except TypeError as exc:
    print("str is immutable:", exc)

s = ""                  # rebinds the name; the old characters are untouched
print(len(b))

go deeper

for a junior

Recall that a str cannot be changed in place: slicing, replace and strip all return new objects. That single fact is why a password stored as text cannot be overwritten once it exists.

for a middle

Explain the mechanics: rebinding moves a name, deallocation does not zero memory, and literals, interning and decoding create copies you hold no reference to. Know that a bytearray can be slice-assigned zeros without reallocating.

for a senior

Demonstrate the working pattern and its limits: bytes from the file or socket, memoryview slices, hmac.compare_digest, a wipe in finally, no resizing - and an explicit statement that swap and core dumps sit outside it.

for a principal

Rank the controls for the organisation: lifetime and copy count first, process hardening such as disabled core dumps second, in-memory scrubbing last, and be clear about which threat model justifies paying for the bytes-only plumbing at all.

## Why immutability defeats erasure A Python `str` is immutable by language definition. There is no assignment form that edits one, `s[0] = 'x'` raises `TypeError`, and every method that reads like an edit - `strip()`, `replace()`, `lower()`, slicing - builds and returns a **new** object while leaving the original bytes in the heap exactly as they were. So the usual C habit of overwriting a password buffer the moment you are done with it has no Python equivalent for `str`. The next instinct is to drop the reference: `del password`, or rebind `password = ""`. Neither erases anything. Rebinding just points the name somewhere else. `del` removes one reference, and if it was the last one the object is deallocated, which returns its memory to the allocator's free list **without zeroing it**. The characters remain readable in that region until something else happens to allocate over them. For a short-lived script that hardly matters. For a process that runs for weeks and might be captured in a heap dump, a core file or a container checkpoint, it is exactly the window you were trying to close. ## The copies you did not make Even if you could scrub the object you hold, you would probably be scrubbing one copy of several. A string literal in your source is compiled into the enclosing code object's constants table, so it lives as long as the module does. That alone makes a hardcoded credential unfixable at runtime, which is a good independent reason not to have one. Short identifier-like strings can be interned and shared, and `sys.intern` makes that explicit; an interned object is deliberately kept alive. Reading a secret as bytes and calling `.decode()` produces a second copy while the first still exists. Passing it through a parser, a `.strip()`, an f-string, a dict key, or an HTTP client that builds a header line copies it again. By the time the value reaches the code that uses it, the process may hold half a dozen instances of those characters in unrelated places, and you have references to none of them. ## What a bytearray buys `bytearray` is the mutable counterpart of `bytes`. Its contents can be assigned in place, so a real overwrite is possible: ```python buf[:] = bytes(len(buf)) # writes zeros over the existing storage ``` Slice assignment of the same length does not reallocate, so the zeros land on the storage that actually held the secret. Two disciplines make that meaningful. First, never grow the buffer: `append` or `extend` may reallocate and copy, leaving the previous, unzeroed allocation behind, so allocate the full length once. Second, never decode it - the moment you call `.decode()` you have minted an immutable copy and lost the property you were protecting. Work on the bytes: `memoryview(buf)` hands out slices that share the storage rather than copying it, and `hmac.compare_digest` accepts any bytes-like object, so a comparison never forces a `str`. The practical shape is to keep the secret in bytes from the moment it enters the process - read a mounted file with a bytes read rather than a text read, take socket bytes as they are - and to wrap the whole use in `try` / `finally` so the wipe happens even on an exception path. ## The honest limits This is defence in depth, not a guarantee, and saying so is part of a good answer. The operating system may have paged that memory to swap, where a copy persists beyond your control unless swap is disabled or encrypted. A crash can write a core dump containing the whole heap, which is why disabling core dumps for a process that handles credentials is a more effective control than any amount of scrubbing. A virtual machine snapshot or a container checkpoint captures memory wholesale. And any interactive entry point hands you a `str` to begin with, so the earliest bytes-only boundary you can build is usually the file or socket read, not the prompt. A related non-fix worth naming: reaching for `ctypes.memset` to overwrite a `str` object's internal buffer. It is possible, and it is a bad idea. It mutates an object the language guarantees is immutable, it can corrupt an interned or shared object that other code is still using, and it still does nothing about the copies you do not have references to. ## What actually reduces exposure Rank the controls honestly. Shorten the lifetime: fetch the credential as late as possible, use it, and drop it, rather than parking it in a module-level constant for the life of the process. Reduce the number of copies: pass a single object rather than re-deriving it, avoid string formatting with it, and keep it out of any structure that gets serialized or printed. Prefer bytes end to end so the wipe is at least available. Then, and only then, scrub. A candidate who leads with the scrub and never mentions lifetime has the priorities backwards.

  • If you never decode the credential, how do you compare it with an expected value?
    `hmac.compare_digest` accepts any bytes-like object, including `bytearray` and `memoryview`, and compares in constant time, so there is no need to build a `str` and no early-exit timing signal. Avoid `==` on secrets for the same reason. If a library insists on text, that call is your bytes-only boundary and you should treat everything past it as uncontrollable.
  • Can growing a bytearray undo the wipe you are relying on?
    Yes. `append` and `extend` can trigger a reallocation that copies the contents to a new block and leaves the old, unzeroed block on the free list, so the secret survives in memory you no longer have a handle on. Allocate the buffer at its final length once and only assign into slices; if you must resize, wipe before the operation that grows it.
  • Given swap and core dumps, is zeroing worth doing at all?
    It is worth doing, but it is not the main control. Zeroing narrows the window in which a heap dump or a debugger attached to a long-lived process finds the plaintext. What actually reduces exposure is a short lifetime, few copies, disabling core dumps for the process, and keeping the value out of anything serialized or printed. Lead with those and treat the wipe as the last layer.

A str is ink already printed on a page you cannot reach; a bytearray is a whiteboard you still hold the eraser for, though the photocopies made along the way are gone either way.

saying these in an interview costs you the question

  • Says del or reassignment erases the characters
  • Believes garbage collection zeroes the memory it frees
  • Assumes only one copy of the secret ever exists
  • Calls ctypes.memset on a str a safe wipe
  • Thinks bytearray.append keeps the same underlying storage
  • Treats scrubbing as a substitute for a short lifetime

context