Why read a binary file with readinto and a preallocated bytearray instead of read?

## Two shapes of the same read Binary file objects expose two ways to pull bytes out: * `read(n)` — returns a new `bytes` object of at most `n` bytes. The allocation size is decided by the call. * `readinto(b)` — copies at most `len(b)` bytes into the writable buffer `b` you supply, and returns the number of bytes actually copied. Functionally they are equivalent; operationally they differ in who allocates. Streaming a large file with `read(65536)` in a loop produces one fresh `bytes` object per iteration; the allocator and the reference-counting machinery handle every one of them, and if you then slice or concatenate the chunk you copy again. `readinto` moves the allocation out of the loop: you make one `bytearray` up front and refill it. ## What readinto accepts, and what it returns The argument must be a *writable* object supporting the buffer protocol — `bytearray`, a writable `memoryview`, an `array.array`. Passing an immutable `bytes` raises `TypeError: readinto() argument must be read-write bytes-like object, not bytes`, which is the buffer protocol's read-only flag being enforced at the boundary. The return value is the byte count, and the two edge cases matter: * A short read is legal. On a buffered reader a full buffer is typical, but on a pipe, a socket-backed file or a raw unbuffered file you can get fewer bytes than you asked for with more data still to come. Code that assumes `readinto` fills the buffer will silently process stale trailing bytes from the previous iteration. * `0` means end of file. That is the loop's terminating condition; there is no falsy empty-`bytes` sentinel as with `read`. `readinto1` exists on buffered readers and issues at most one call to the underlying raw stream, which matters when you want latency rather than a full buffer. ## Tracking the fill position with a memoryview The natural pattern is to read a record of known size into a fixed buffer, looping until it is full. The trap is expressing "the unfilled tail" as `buf[filled:]` — that slices the `bytearray`, which *copies* the tail into a new object, and `readinto` then fills the copy that you promptly throw away. The whole point of the exercise disappears, silently and without an error. The fix is to take one `memoryview` of the buffer and slice the view instead: `view[filled:]` is a window onto the tail of the same memory, so `readinto` writes into the real buffer. ```python view = memoryview(buf) filled = 0 while filled < len(buf): n = src.readinto(view[filled:]) if not n: raise EOFError("short record") filled += n ``` Release the view (`memoryview.release()` or a `with` block) when the loop ends, or the `bytearray` stays pinned and cannot be resized. ## Reusing the buffer safely Because there is only one buffer, everything derived from an iteration must be consumed or copied before the next `readinto` overwrites it. Parse the record and keep the parsed values; if you keep a view of the raw bytes instead, the next read mutates what you kept. Anything that outlives the iteration needs `memoryview.tobytes()` or an equivalent copy. This is the discipline that makes buffer reuse safe, and it is where reused-buffer code usually goes wrong. ## Where else the pattern appears `socket.socket.recv_into` is the same contract for sockets, and it is where reuse matters most, because a receive loop runs continuously. Hash objects' `update`, `struct.unpack_from` and a binary file's `write` all accept a buffer, so a whole read-parse-hash-forward path can run without materializing a single intermediate `bytes` object. ## When not to bother For a small configuration file, `read()` in one shot is clearer and the allocation is irrelevant. `readinto` earns its extra state — a buffer, a fill counter, a short-read branch — in loops that run millions of times, in receive paths where allocation churn shows up as latency variance, and where the destination memory is fixed by something outside your control. Reach for it after a profile, not before.