skip to content

A Rust debug build aborts with a stack overflow at `Box::new([0u8; 16 * 1024 * 1024])`; why, and how do you heap-allocate that buffer safely?

level: seniorimportance: should knowfreq 35%

answer

  1. arguments are evaluated first
  2. no guaranteed placement
  3. the optimiser may hide it
  4. vec! then try_into
  5. Box::new_zeroed since 1.92

basics

~20 s

Box::new takes its argument by value, so the 16 MiB array is built on the stack and then copied to the heap, and an unoptimised build overflows the stack. Allocate on the heap directly with vec![0u8; N].try_into() or Box::new_zeroed().

solid answer

~40 s

`Box::new(x)` is an ordinary function: `[0u8; N]` is evaluated first as a stack temporary and then moved into the new heap block, and Rust guarantees no in-place construction. 16 MiB exceeds a typical 8 MiB main-thread stack and the 2 MiB std gives spawned threads, so the debug build aborts; an optimised build may elide the copy, but that is not a guarantee. Build the buffer on the heap instead: `vec![0u8; N]` allocates and zeroes it there, and `try_into()` turns it into a `Box<[u8; N]>` in place. Since Rust 1.92, `Box::new_zeroed()` plus an `unsafe` `assume_init` also works for types where zero bytes are valid. After that, moving the `Box` moves only the 8-byte pointer.

code

rust · 6 lines
rust
const N: usize = 16 * 1024 * 1024;

fn main() {
    let frame: Box<[u8; N]> = Box::new([0u8; N]);
    println!("first byte {}", frame[0]);
}

go deeper

for a junior

Know that a Box holds its value on the heap and that moving a Box moves only the pointer, not the data it owns.

for a middle

Explain why Box::new can still touch the stack: its argument is evaluated first as a temporary, and Rust guarantees no in-place construction.

for a senior

Diagnose the debug-only abort, refuse to call the release build proof, and fix it with vec! plus try_into or Box::new_zeroed, stating when assume_init is sound.

for a principal

Turn it into review guidance: large buffers are built with heap-first constructors, large enum variants are boxed, and stack-size bumps are not accepted as the fix.

## What `Box::new` actually does `Box::new(x: T)` is an ordinary function that takes its argument **by value**. Rust evaluates arguments before the call, so in ```rust let frame: Box<[u8; N]> = Box::new([0u8; N]); // N = 16 MiB ``` the array expression `[0u8; N]` is built first as a temporary in the current stack frame, and only then does `Box::new` allocate a heap block and move the bytes into it. Rust has **no guaranteed placement**: nothing in the language promises that the value is constructed directly in the heap. A 16 MiB temporary is larger than a typical main-thread stack (often 8 MiB, set by the operating system) and far larger than the 2 MiB that std requests by default for threads it spawns. In an unoptimised build the program aborts with: ```plaintext thread 'main' (...) has overflowed its stack fatal runtime error: stack overflow, aborting ``` ## Why the release build "works" With `-C opt-level=3` the same program ran fine on rustc 1.99.1: the optimiser noticed the temporary was only ever copied into the heap and wrote the zeroes there directly. That is an **optimisation, not a guarantee**. A different element type, a non-constant initialiser, an extra function boundary or a compiler upgrade can bring the stack copy back, so a debug-only crash is a real bug, not noise. ## Allocating directly on the heap Pick a constructor that never materialises the whole value on the stack: 1. **`vec![0u8; N]` then `try_into()`**. The `vec!` macro allocates the buffer on the heap and fills it there, requesting exactly `N` elements. `TryFrom<Vec<T>>` for `Box<[T; N]>` (stable since Rust 1.66) then converts in place when the capacity equals `N`, and hands the vector back in `Err` if the length is wrong. This is entirely safe code. 2. **`Box::new_zeroed()`** (stable since Rust 1.92) returns a `Box<MaybeUninit<T>>` whose memory is filled with zero bytes; `assume_init` is `unsafe` and is sound only because all-zero bytes are a valid `[u8; N]`. For a type where all-zero bytes are invalid, such as a struct holding a reference or a `Box`, it is undefined behaviour. 3. **`Box::new_uninit()`** (stable since Rust 1.82) gives uninitialised heap memory, but `Box::write(boxed, value)` (stable since Rust 1.87) and `MaybeUninit::write` both take the value by value, so writing a whole 16 MiB array through either has the same problem: `Box::write(Box::new_uninit(), [0u8; N])` overflowed exactly like `Box::new` in a debug build. Fill it in place instead, element by element or through a raw pointer, which is the unsafe material taught elsewhere. | Construction | Builds the value on the stack first? | Safe code? | |---|---|---| | `Box::new([0u8; N])` | yes, unless optimised away | yes | | `Box::from([0u8; N])` | yes, takes `T` by value | yes | | `vec![0u8; N].try_into()` | no | yes | | `Box::new_zeroed()` + `assume_init` | no | needs `unsafe` | ## Moving the `Box` is cheap Once the buffer is on the heap, the `Box<[u8; N]>` handle is one pointer: 8 bytes on a 64-bit target. Passing it to a function, returning it, storing it in a struct or pushing it into a `Vec` **moves the pointer, not the 16 MiB**. In the example below the heap address printed in `main` and inside `consume` is the same, and `size_of::<Frame>()` is 8. That is the second reason to box large values: - a large struct passed through several layers of calls is copied at each by-value move unless the optimiser elides it, while a boxed one moves 8 bytes; - an enum is as large as its largest variant, so one 4 KiB variant makes every value of the enum about 4 KiB; boxing that variant shrinks the enum to pointer size (clippy's `large_enum_variant` lint flags this); - the price is one heap allocation and one pointer indirection on every access. ## How to diagnose it - The abort names the thread and says it overflowed its stack; a debugger backtrace points at the frame containing the large temporary. - Check the sizes involved with `std::mem::size_of` rather than guessing. - Raising the stack size (a bigger spawned-thread stack, or the OS limit for the main thread) only hides the copy; fix the construction instead.

  • Why is `unsafe { Box::<[u8; N]>::new_zeroed().assume_init() }` sound, but not for every `T`?
    `new_zeroed` returns a `Box<MaybeUninit<T>>` filled with zero bytes, and `assume_init` asserts those bytes are a valid `T`. All-zero is a valid `u8`, so the array is fine. For a type holding a reference, a `Box` or a `NonNull`, all-zero means a null pointer, which is an invalid value, so the same call is undefined behaviour.
  • When does boxing a large value help even if nothing overflows?
    When the value moves a lot or sits in an enum. Each by-value move of a large struct may copy it, while a boxed one moves 8 bytes. An enum is as large as its largest variant, so one 4 KiB variant makes every value 4 KiB; boxing it shrinks the enum, which clippy's `large_enum_variant` lint suggests. The cost is an allocation and an indirection.

  • Box::new allocates first and builds the value straight into the heap block.
  • Release builds always elide the copy, so a debug-only crash can be ignored.
  • Passing a Box to a function copies the boxed value onto the callee's stack.
  • Box::new_zeroed is safe for any T because zero is a valid default.
  • Writing an array into Box::new_uninit() avoids the stack because the memory starts uninitialised.

Assumes Rust 1.99; Box::new_zeroed has been stable since 1.92, Box::write since 1.87 and Box::new_uninit since 1.82, while the vec! plus try_into route works on any recent toolchain.