skip to content

A Rust service cuts display names to a 32-byte limit with `name.truncate(32)`; why does it panic on some names, and how do you cut safely?

level: seniorimportance: should knowfreq 40%

answer

  1. String must stay valid UTF-8
  2. byte 32 may be mid-character
  3. is_char_boundary is asserted
  4. floor_char_boundary rounds down (1.91)

basics

~10 s

String::truncate takes a byte length and panics when it falls inside a multi-byte character, because a String must stay valid UTF-8. Cut at name.floor_char_boundary(32), stable since Rust 1.91, or step down with is_char_boundary.

solid answer

~40 s

`String::truncate(new_len)` measures in bytes and asserts `self.is_char_boundary(new_len)`; the panic message is "assertion failed: self.is_char_boundary(new_len)". A name whose 32nd byte lands inside an accented letter or an emoji would otherwise leave half a character, which would break the guarantee that a `String` is always valid UTF-8, so std panics instead. `&name[..32]` fails the same way, reporting the offending character and its byte range. The fix is to round the limit down to a boundary first: `let cut = name.floor_char_boundary(32); name.truncate(cut);`. `floor_char_boundary` was stabilised in Rust 1.91 and returns `len()` when the index is past the end; on an older MSRV, loop `while !name.is_char_boundary(cut) { cut -= 1; }`, which stops by byte 0 at the latest. Even then a cut can split a grapheme cluster such as an emoji sequence.

code

rust · 14 lines
rust
fn truncate_to_bytes(s: &mut String, max_bytes: usize) {
    let cut = s.floor_char_boundary(max_bytes); // stable since 1.91
    s.truncate(cut);
}

fn main() {
    let mut name = String::from("Zoë Šimić");
    println!("len = {}, boundary at 3? {}", name.len(), name.is_char_boundary(3));
    truncate_to_bytes(&mut name, 3);
    println!("{name:?} ({} bytes)", name.len());

    let mut raw = String::from("Zoë Šimić");
    raw.truncate(3); // panics: byte 3 is inside the e with diaeresis
}

go deeper

for a junior

Recall that truncate and range slicing take byte positions and panic if the position falls inside a multi-byte character.

for a middle

Explain char boundaries, why a String may never hold half a character, and how floor_char_boundary or an is_char_boundary loop picks a valid cut.

for a senior

Treat any byte-index cut on user input as a crash path in review, test with accented and emoji names, and know floor_char_boundary arrived in 1.91 when the crate's MSRV is older.

for a principal

Decide whether limits are bytes or graphemes per field and say so in the schema; byte-safe cuts are cheap, grapheme-safe cuts need a dependency, and mixing them confuses users.

## Why the panic happens A `String` guarantees its bytes are valid **UTF-8** at all times; every `str` method relies on that. **`String::truncate(new_len)`** shortens the string to `new_len` **bytes**, and its documentation says it panics if `new_len` does not lie on a `char` boundary. The implementation asserts it, so the failure reads: ```plaintext assertion failed: self.is_char_boundary(new_len) ``` A **char boundary** is a byte index where a UTF-8 sequence starts, or the end of the string. In "Zoë Šimić" the `ë` occupies bytes 2 and 3, so index 3 is not a boundary. Truncating there would keep the first byte of `ë` alone, which is not valid UTF-8, and std refuses rather than produce a broken `String`. ASCII-only test data never hits this, which is why it reaches production: the first user with a long accented name or an emoji crashes the request. Slicing has the same rule. `&name[..3]` panics with a message that names the character: "end byte index 3 is not a char boundary; it is inside 'ë' (bytes 2..4 of string)". The non-panicking slice `name.get(..3)` returns `None` instead, which tells you the cut is bad but does not pick a better one. ## The fix on current Rust Rust 1.91 stabilised two rounding methods on `str`: - **`floor_char_boundary(index)`** returns the largest boundary that is `<= index`, and `len()` when `index >= len()`. - **`ceil_char_boundary(index)`** returns the smallest boundary that is `>= index`. A byte limit needs the floor, because the ceiling can overshoot the limit. Their documentation names this exact use: truncating a string so it stays valid UTF-8 without exceeding a given number of bytes. ```rust fn truncate_to_bytes(s: &mut String, max_bytes: usize) { let cut = s.floor_char_boundary(max_bytes); s.truncate(cut); } ``` For a borrowed view, `&s[..s.floor_char_boundary(max_bytes)]` gives the same result without changing `s`. ## The fix for an older MSRV A crate that supports compilers before 1.91 can write the loop itself: 1. Start at `cut = max_bytes.min(s.len())`. 2. While `!s.is_char_boundary(cut)`, decrement `cut`. 3. Truncate or slice at `cut`. The loop runs at most three times, because a UTF-8 sequence is at most four bytes, and it always terminates because index 0 is a boundary. `is_char_boundary` itself never panics: it returns `false` for an index past the end, which is why the `min` comes first. ## What a safe cut still gets wrong | Problem | Cause | What to do | |---|---|---| | an emoji loses its second half | the cut fell between scalar values of a joined sequence | cut on grapheme clusters, using a segmentation crate | | an accent is dropped from its letter | combining mark split from its base | same: grapheme-aware cutting | | capacity stays large | `truncate` does not shrink the allocation | call `shrink_to_fit` only if the memory matters | | no visible sign of the cut | text silently shortened | append an ellipsis only after the cut, inside the budget | The `floor_char_boundary` docs warn about the first row themselves: rounding happens at the character level and can still split a grapheme, turning a scientist emoji into a plain person. ## Where this shows up - **Byte-limited storage and wire fields**, such as a database column, a log line cap or a header length. - **Untrusted input in a server**, where a panic in a request handler is a denial-of-service path that any user can trigger with one emoji. - **Code ported from ASCII assumptions**, where `truncate(n)` or `&s[..n]` was written with a character count in mind. One more edge: if `new_len` is greater than or equal to the current length, `truncate` does nothing, so rounding is only needed when the string is actually too long. ## Testing the cut A byte-limit function earns a small table-driven test, because ASCII-only fixtures never exercise it: - a pure ASCII string exactly at the limit and one byte over it; - a two-byte letter such as `ë` straddling the limit, which must be dropped whole; - a four-byte emoji ending one byte past the limit; - a limit of 0, which must yield an empty string rather than panic; - a limit larger than the string, which must leave it unchanged. Assert two properties for every case: `result.len() <= limit`, and `original.starts_with(&result)`. Both hold for a correct floor-based cut. To catch a cut that drops more than it must, also assert that whenever `result` is shorter than the original, adding the next character's `len_utf8()` would exceed the limit.

  • Why not use `ceil_char_boundary` for a byte limit?
    It rounds up to the next boundary, so the result can exceed the limit by up to three bytes and overflow the column or field you were protecting. `floor_char_boundary` rounds down and always fits; `ceil_char_boundary` suits cases such as extending a selection to include a whole character.
  • Can you avoid the panic by slicing with `name.get(..32)` instead?
    It avoids the panic, returning `None` when byte 32 is inside a character, but it does not find a valid cut for you. You still need `floor_char_boundary` or an `is_char_boundary` loop to choose the largest boundary that fits.
  • Does `truncate` free the memory it cuts off?
    No. The docs say it has no effect on the allocated capacity, so the buffer keeps its old size. Call `shrink_to_fit` afterwards only when the freed memory matters, since it may reallocate.

Cutting a String to a byte budget is like cutting film to a running time: you may only cut between frames. truncate refuses to cut through a frame, and floor_char_boundary winds back to the previous frame edge so the cut stays inside the time limit.

  • String::truncate counts characters, so truncate(32) keeps 32 letters.
  • truncate silently rounds down to the previous char boundary.
  • is_char_boundary panics when the index is past the end of the string.
  • floor_char_boundary never splits an emoji sequence or a combining accent.
  • ceil_char_boundary is the safe choice for a byte limit.

Assumes Rust 1.99; str::floor_char_boundary and ceil_char_boundary are stable since Rust 1.91, so an older MSRV needs the is_char_boundary loop.