Why does appending to a bytearray raise BufferError while a memoryview of it is alive?
answer
- Someone else is holding the memory
- Growing may move the block
- The exporter counts outstanding exports
- In-place writes are allowed, resizes are not
- release() or a with block ends the export
basics
~20 sA memoryview holds an exported buffer: a raw pointer into the bytearray's storage. Resizing could reallocate that storage and leave the view dangling, so the bytearray refuses while any export is outstanding and raises BufferError. Release the view first.
solid answer
~50 sWhen you build a `memoryview` over a `bytearray`, the buffer protocol hands out a pointer into the bytearray's internal storage and increments its export count. Resizing — `append`, `extend`, `+=`, deleting a slice — may reallocate that storage, which would leave the exported pointer dangling, so `bytearray` refuses any resize while exports exist and raises `BufferError: Existing exports of data: object cannot be re-sized`. Writing *through* the view is fine; only length changes are blocked. The fix is to end the export explicitly: call `memoryview.release()`, or use the view in a `with` block, since `memoryview` is its own context manager. Relying on garbage collection is unreliable — a view captured in a traceback, a closure or a cache keeps the export alive indefinitely, and the resize keeps failing at an unrelated place in the code.
code
python · 9 linesbuf = bytearray(b"abc")
view = memoryview(buf)
try:
buf.append(100)
except BufferError as exc:
print("BufferError:", exc)
view.release()
buf.append(100)
print(buf)go deeper
Recall that a memoryview points into a bytearray's memory, so the bytearray cannot change length while the view exists. Know that memoryview.release() or a with block ends the view.
Explain the export count and why only resizes are blocked while in-place writes are not, and name the operations that can reallocate. Be able to show the with memoryview(buf) as view: form and why it is preferred over hoping for collection.
Demonstrate the diagnosis: BufferError names the class of bug, so trace which views escaped into caches, queues, closures or tracebacks. Show that you would copy with tobytes() at boundaries and size the buffer once so growth is never needed.
Own the aliasing policy. Decide which layers may hand out views and which must hand out copies, make buffer lifetimes explicit rather than incidental, and weigh the throughput that zero-copy buys against the review cost of code where two names share one block of memory.
## The export count Objects that export buffers keep a count of outstanding exports. `memoryview(buf)` increments it; the view's release decrements it. While the count is non-zero, the exporter has promised that the memory it described stays at the same address with the same length, because the consumer may be a C extension holding nothing but a raw pointer. A `bytearray` stores its bytes in a single heap allocation. Growing it can `realloc` that block to a new address, and shrinking it can hand memory back. Either would break the promise, so `bytearray` checks the export count before any operation that changes its length and raises `BufferError: Existing exports of data: object cannot be re-sized`. Mutating in place is untouched: `buf[0] = 65` works with views open, and so does writing through a writable view, because neither moves the block. ## What actually trips it Everything that changes the length: `append`, `extend`, `+=`, `insert`, `pop`, `remove`, `clear`, `del buf[:n]`, and assigning a slice of a different size. Everything that does not change the length is fine. The surprise in real code is that the failing statement is usually far from the statement that created the view. ## A concrete failure: a geocoding batch with cached views Consider a geocoding batch that reads fixed-width result records into one reusable `bytearray` and keeps a small cache mapping an address key to the packed coordinate bytes. Two bugs live in that design, and both come from the same misunderstanding of what a view is. First, the cache stores `memoryview` slices of the shared buffer. Those slices are windows, not values, so the next record overwrites the memory they point at. A lookup that hits the cache then reads whatever the most recent record wrote: a stale cached value that looks plausible, points at the wrong city, and never raises. The cache must store `memoryview.tobytes()` — a real copy — or parsed values, not views. Second, when a record turns out to be longer than the buffer, the natural response is to grow the buffer with `extend`, and that raises `BufferError` because the cached views are still exporting it. The traceback points at the growth, but the cause is the cache several thousand records earlier. In a batch whose full run takes 27 minutes, this is the kind of defect that only appears near the end, on the one oversized record, long after the run looked healthy. ## Releasing deliberately `memoryview.release()` drops the export immediately. After it, any use of the view raises `ValueError: operation forbidden on released memoryview object` — which is a feature: a released view fails loudly instead of silently pointing at memory that has since moved. `memoryview` also implements the context-manager protocol, so the idiomatic form scopes the export: ```python buf = bytearray(b"abc") with memoryview(buf) as view: process(view) buf.extend(b"def") ``` Slices of a view are separate views with their own exports, so releasing the parent does not release the children; release each one you kept, or scope them all inside the same block. Do not lean on refcounting to clean up: a view referenced by a traceback held in a logging record, by a closure, or by a cache stays alive as long as that reference does. ## Diagnosing it in a running system `BufferError` is narrow enough that its presence identifies the class of bug immediately: someone is holding a buffer export of an object that something else wants to resize. The work is finding the holder. Read the code path for every `memoryview` constructed over the object, look for views escaping into caches, queues, deferred callbacks and log records, and check whether any `except` block captured a view in a traceback frame. The structural fix is almost always the same: make the boundary explicit, so that views live only inside the function that created them and anything crossing that boundary is copied with `tobytes()`. ## Design guidance Treat a writable exporter and its views as a single unit with a defined lifetime. Decide the buffer's size once, up front, so it never needs to grow; keep views inside a `with` block; and copy at any API boundary. That converts a whole family of aliasing and pinning bugs into an allocation you can see and measure.
- Does writing through a writable memoryview also raise BufferError?No. Only length changes are blocked. Assigning through the view or through the `bytearray` writes into the existing block, which does not move it, so the exported pointer stays valid. `append`, `extend`, `insert`, `pop`, `clear`, `del buf[:n]` and differently-sized slice assignment are the operations that can reallocate, and those are the ones that raise.
- Why not just let the view be garbage collected instead of calling release()?Because you do not control when the last reference dies. A view captured in a traceback held by a log record, in a closure, in a cache or in an exception object stays alive indefinitely, and the resize then fails at some unrelated later point. `memoryview.release()` or a `with` block ends the export at a place you chose, and a released view raises `ValueError` if anyone touches it afterwards.
- What happens to slices of a view when the parent view is released?Nothing — each slice is an independent view with its own export, so releasing the parent leaves the children exporting the buffer and the resize still fails. Either scope every derived view inside the same `with` block, or release each one you kept. This is the usual reason a `BufferError` survives an apparently correct cleanup.
A view is a bookmark wedged into a book someone is still reading. You may correct a word on a page, but you may not send the book back to be rebound in a larger binding until every bookmark is out.
saying these in an interview costs you the question
- Thinks any mutation of the bytearray is blocked
- Assumes the view is copied so resizing is safe
- Relies on garbage collection to end the export
- Caches memoryview slices of a reused buffer as values
- Believes releasing a parent view releases its slices
- Treats BufferError as a bug in the standard library