Reviewing a C++ image decoder that computes int pixels = width * height from header fields, loops for (int i = 0; i <= pixels; ++i), and sets alpha only for one-channel input, which undefined behaviours do you flag?
answer
- header fields are attacker input
- int times int overflows
- less-or-equal reads one extra
- a variable set on one branch
- size_t, pre-checks, initializers
basics
~20 sSigned overflow in width * height and again in pixels * 4, an off-by-one i <= pixels that reads and writes past both buffers, and an uninitialized alpha read. Validate the header, size in std::size_t with pre-checks, initialize alpha.
solid answer
~50 sTreat the header as untrusted. `int pixels = width * height` is **signed overflow** for a large or hostile header, and `pixels * 4` overflows again; a negative result converted to `std::size_t` becomes a huge allocation request. `i <= pixels` runs one iteration too many, so `src[i]` and `out[4 * i]` go **out of bounds** on every call. `int alpha;` is assigned only when `channels == 1`, yet a header claiming three or more channels reaches the read of `alpha`, an **uninitialized read**. Fixes: reject non-positive dimensions and unknown channel counts, compute sizes in `std::size_t` after checking `width <= SIZE_MAX / 4 / height`, loop with `<`, and initialize `alpha` where it is declared. None of these lines draws a warning from Apple clang 21 at `-Wall -Wextra`; an AddressSanitizer run flags the out-of-bounds read.
code
cpp · 31 lines#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <limits>
#include <optional>
#include <vector>
struct Header { int width; int height; int channels; };
std::optional<std::vector<std::uint8_t>> to_rgba(const Header& h,
const std::uint8_t* src, std::size_t src_len) {
if (h.width <= 0 || h.height <= 0 || (h.channels != 1 && h.channels != 2))
return std::nullopt;
const std::size_t w = h.width, ht = h.height, ch = h.channels;
if (w > std::numeric_limits<std::size_t>::max() / 4 / ht) return std::nullopt;
const std::size_t pixels = w * ht;
if (src_len / ch < pixels) return std::nullopt;
std::vector<std::uint8_t> out(pixels * 4);
for (std::size_t i = 0; i < pixels; ++i) {
const std::uint8_t g = src[i * ch];
out[4 * i] = out[4 * i + 1] = out[4 * i + 2] = g;
out[4 * i + 3] = ch == 2 ? src[i * 2 + 1] : std::uint8_t{255};
}
return out;
}
int main() {
const std::uint8_t grey[4] = {1, 2, 3, 4};
auto img = to_rgba({2, 2, 1}, grey, sizeof grey);
std::printf("%zu bytes, last alpha %d\n", img->size(), img->back());
}go deeper
Spot the obvious ones: the loop's <= runs one element past the end, and a variable set on only one branch must be initialized first.
Name each line's rule: signed overflow in int arithmetic, out-of-bounds indexing through operator[] and a raw pointer, and an indeterminate read of alpha.
Treat the header as hostile input, compute sizes in std::size_t with division-based pre-checks, pass the buffer length, and explain why tests on small grey images never failed.
Turn the findings into team rules for parsers of untrusted input: validated headers, span-based buffers, sanitizer and fuzzing runs before release.
## The function under review A decoder for a simple grey or grey-plus-alpha format expands each pixel to RGBA. The header comes from the file, so every field is attacker-controlled: ```cpp struct Header { int width; int height; int channels; }; std::vector<std::uint8_t> to_rgba(const Header& h, const std::uint8_t* src) { int pixels = h.width * h.height; // A std::vector<std::uint8_t> out(pixels * 4); // B int alpha; // C if (h.channels == 1) alpha = 255; for (int i = 0; i <= pixels; ++i) { // D std::uint8_t g = src[i * h.channels]; out[4 * i + 0] = out[4 * i + 1] = out[4 * i + 2] = g; out[4 * i + 3] = h.channels == 2 ? src[i * 2 + 1] : static_cast<std::uint8_t>(alpha); // E } return out; } ``` It compiles with no warning under Apple clang 21 with `-std=c++17 -Wall -Wextra`, at `-O0` and at `-O2`. A review checklist catches what the compiler does not. ## The checklist, line by line 1. **Line A, signed overflow.** `int * int` is computed in `int`. A header of 50000 by 50000 gives 2.5 billion, above `INT_MAX`, so the multiplication is undefined behaviour, not a wrapped value. 2. **Line B, overflow again, then conversion.** `pixels * 4` overflows for any `pixels` above about 536 million. If the result were negative, conversion to the `std::vector` size parameter, a `std::size_t`, would produce a huge value and the constructor would throw `std::length_error` or `std::bad_alloc`; but because the multiply is already undefined, even that outcome is not guaranteed. 3. **Line D, off by one.** `i <= pixels` runs `pixels + 1` iterations. The last one reads `src[pixels * channels]` and writes `out[4 * pixels]` through `out[4 * pixels + 3]`, past the end of both buffers, on **every** call, including well-formed images. `std::vector::operator[]` does not check. 4. **Lines C and E, uninitialized read.** `alpha` is assigned only when `channels == 1`. For any channel count other than 1 or 2 the conditional's third operand reads `alpha` while it holds an indeterminate value: undefined behaviour up to C++23, erroneous behaviour in C++26. 5. **Missing validation.** Zero or negative `width`, `height` or `channels` make every index computation meaningless, and nothing checks that `src` holds `pixels * channels` bytes. ## Why testing missed it | Line | Typical symptom | Why tests pass | |---|---|---| | A, B | huge or failed allocation | test images are small | | D | silent heap or stack overwrite | the overrun lands in mapped memory | | C, E | random alpha | test files are all grey or grey-plus-alpha | A run with AddressSanitizer on a 2 by 2 grey image reports `stack-buffer-overflow` at the `src[i * h.channels]` line; the sanitizer flags themselves are covered with the compiler toolchain. ## The repaired function - Reject `width <= 0`, `height <= 0` and channel counts other than 1 or 2 before any arithmetic. - Convert to `std::size_t` first, and check `w > SIZE_MAX / 4 / h` before computing `w * h * 4`, so no multiplication can wrap even with a 32-bit `std::size_t`. - Pass the source length in and check it against `pixels * channels`, or take a `std::span<const std::uint8_t>`. - Loop with `i < pixels` over `std::size_t`. - Initialize `alpha` at its declaration, or compute it per pixel from the channel count. The corrected version appears in the code example below. It returns `std::optional` so that a malformed header becomes an ordinary "no image" result rather than an exception or a crash, and it takes the source length explicitly, because a raw pointer carries no size. ## Presenting the review An interviewer listens for structure as much as for findings: 1. **Classify the input first**: say which values come from the file and therefore cannot be trusted. 2. **Walk the arithmetic** before the loops: overflow decides how big every buffer is. 3. **Check every index** against the size of the buffer it touches, including the source buffer. 4. **Check every variable** for a path on which it is read before it is written. 5. **Propose the fix and the test** that would have caught each finding: a hostile-header unit test, a fuzzing target, and a sanitizer build. In an interview, naming the line, the rule it breaks, and the concrete fix for each is what separates a reviewer from someone who says "this looks unsafe".
- Why check w > SIZE_MAX / 4 / ht instead of computing w * ht * 4 and testing the result?Unsigned arithmetic wraps silently, so testing the product afterwards cannot tell a wrapped result from a real one, and doing the multiply in `int` would already be undefined. Dividing the limit first keeps every intermediate value in range, which matters most where `std::size_t` is 32 bits.
- Would replacing out[...] with out.at(...) have been enough to fix line D?Only half of it. `at()` would throw `std::out_of_range` on the write past `out`, turning a silent overwrite into an exception, but the read `src[i * h.channels]` goes through a raw pointer and is still undefined. The real fix is the loop condition `i < pixels` plus a checked source length.
- Image dimensions from the file header can be trusted once the magic bytes match.
- int pixels = width * height simply wraps to a negative number when it overflows.
- The loop bound <= is harmless because the extra element is only read.
- If -Wall -Wextra prints nothing, the function has no undefined behaviour.
- Converting the product to std::size_t after multiplying makes the overflow safe.