skip to content

Why does converting between string and []byte in Go allocate and copy, and when does the compiler skip it?

level: middleimportance: must knowfreq 58%

answer

  1. The result must not be writable through the source
  2. One side is mutable, the other is not
  3. Both directions are O(n) plus an allocation
  4. A few read-only expressions are special-cased
  5. Map index and comparison against a literal

basics

~20 s

Strings are immutable, so []byte(s) and string(b) each allocate a new array and copy the bytes, keeping the two independent. The compiler elides that copy only in read-only patterns such as m[string(b)] and string(b) == "literal".

solid answer

~40 s

A string must never change once it exists. If `[]byte(s)` handed back a slice sharing the string's array, writing to that slice would silently rewrite the string, so the conversion allocates a fresh array and copies. `string(b)` copies for the mirror reason: the caller still owns `b` and may write to it afterwards. Both are O(n) and show up in profiles as allocations in hot loops. The compiler recognises a few cases where the converted value provably cannot outlive the expression or be written, and skips the allocation: a map index `m[string(b)]`, comparisons like `string(b) == "GET"` or a `switch` on `string(b)`, and ranging over `string(b)`. The language also lets you avoid the conversion entirely — `append(b, s...)` and `copy(b, s)` both accept a string source directly.

code

go · 8 lines
go
counts := map[string]int{}

// allocates: a real string value is created and kept
key := string(line)
counts[key]++

// no allocation: the bytes are hashed and compared in place
counts[string(line)]++

go deeper

for a junior

Be ready to say that converting between string and []byte copies the bytes, and that it is not a free reinterpretation. Knowing it costs an allocation is enough at this level.

for a middle

Explain the immutability argument in both directions, and name the map-index and comparison elisions plus why assigning to a variable defeats them.

for a senior

Show how you find these in a real service — allocations per operation in a benchmark with -benchmem, then a decision about which representation the data should live in end to end rather than a cheaper conversion.

for a principal

Own the guidance about where the string/[]byte boundary sits in an API. Converting at every layer is a design smell; deciding once, at the edge, is the call worth writing down.

## The invariant that forces the copy Go guarantees that a `string` value never changes. Every optimisation that rests on that guarantee — sharing bytes between substrings, keeping literals in read-only memory, using strings as map keys whose hash is stable — depends on there being no write path to a string's bytes. A conversion is exactly the place where a write path would appear: - `[]byte(s)` produces a **writable** slice. If it shared `s`'s array, `b[0] = 'X'` would mutate `s` and every other string sharing those bytes. - `string(b)` produces an **unwritable** string from an array the caller still owns and can write to a nanosecond later. So both directions allocate a new array and copy the bytes across. The cost is O(n) in the length, plus a heap allocation and the garbage it eventually creates. ## Where this bites The classic shape is a decode loop that converts on every iteration: reading a line as `[]byte`, converting it to `string` to compare or to use as a map key, and doing that a million times. A benchmark with `-benchmem` shows it immediately as a high `allocs/op` with no obvious allocation in the source — the allocation is the conversion. ## The elisions the compiler performs The compiler can skip the copy when it can prove the converted value is only read and does not escape the expression. The well-known cases: - **Map index:** `m[string(b)]` hashes and compares the bytes directly against the map's keys without materialising a string. This is the single most valuable one — it turns a lookup-by-bytes into an allocation-free operation. - **Comparison:** `string(b) == "POST"`, `string(b) < other`, and a `switch string(b) { case ... }` compare the bytes in place. - **Ranging:** `for i, r := range string(b)` decodes runes straight out of `b`. These are compiler optimisations, not language guarantees — treat them as "the standard idioms are free" rather than as a rule you can extrapolate to a new expression. Assigning the conversion to a variable first (`s := string(b); m[s]`) defeats all of them, because now a real string value exists with a lifetime the compiler cannot bound. Separately, escape analysis can put a small non-escaping conversion result on the stack rather than the heap. That removes the allocation but **not** the copy. ## Avoiding the conversion instead of optimising it Go builds in several string/`[]byte` bridges that never need a conversion: - `append(b, s...)` appends a string's bytes to a byte slice. - `copy(b, s)` copies from a string into a byte slice — a documented special case of `copy`. - `io.WriteString(w, s)` writes a string to an `io.Writer` and uses the writer's own `WriteString` method when it has one, avoiding `[]byte(s)`. - `strings.Builder` accumulates into a byte buffer and hands back a string with no final copy. - The `bytes` package mirrors most of `strings`, so code that already holds `[]byte` can stay on that side of the fence — `bytes.HasPrefix`, `bytes.Equal`, `bytes.Split` and friends. The first design question in a hot path is therefore "which representation should this data live in end to end?", not "how do I make the conversion cheaper". ## The unsafe escape hatch, and why it is a separate decision `unsafe.String` and `unsafe.Slice` can build a string over existing bytes with no copy at all. They also hand you the obligation to guarantee nobody ever writes those bytes again, which the compiler and the race detector cannot check for you. That is a policy decision for a codebase, not a routine optimisation. ## What a good answer sounds like Name the immutability invariant first, then the two directions and why each needs its own copy, then the map-key and comparison elisions as the ones worth memorising, and finish with `append`/`copy`/`strings.Builder` as the ways to sidestep the question entirely.

  • Why does `s := string(b); m[s]` allocate when `m[string(b)]` does not?
    The elision needs the converted value to be confined to the expression, where the compiler knows it is only hashed and compared. Once you assign it to a variable, a real string exists with an unbounded lifetime — it may be stored, returned or captured — so the bytes must be copied into memory the string can own.
  • Does escape analysis remove the cost of a conversion?
    It can remove the heap allocation for a small result that does not escape, moving the bytes to the stack. The copy still happens, and the GC pressure disappears rather than the CPU cost. Confirm with `go build -gcflags=-m` and a benchmark run with `-benchmem`.
  • You are comparing an incoming `[]byte` header value against a fixed word. What is the cheapest correct form?
    Either `string(b) == "POST"`, which the compiler compares in place, or `bytes.Equal(b, []byte("POST"))` with the literal hoisted to a package-level variable. Both avoid allocating; the first is usually the more readable.

saying these in an interview costs you the question

  • Says the conversion just reinterprets the same bytes
  • Claims string(b) is free because strings are immutable
  • Assigns string(b) to a variable and expects the map elision
  • Thinks escape analysis removes the copy, not just the allocation
  • Reaches for unsafe before trying append, copy or the bytes package