skip to content

A telemetry decoder using binary.Read returns plausible but wrong numbers. How do you tell a byte-order bug from a struct-layout bug?

level: seniorimportance: should knowfreq 38%

answer

  1. the decode never fails, only the numbers are wrong
  2. look at the raw bytes first
  3. one failure hits every field, the other hits a suffix
  4. compare the declared width to the spec's frame length
  5. a C compiler pads where encoding/binary does not

basics

~20 s

Hexdump the raw frame and compare it field by field with the specification. Byte-swapped values across every field point to the wrong binary.ByteOrder. A correct first field followed by garbage points to a width or padding mismatch in the Go struct.

solid answer

~50 s

Capture the raw bytes before decoding and dump them in hex, then decode the first field by hand. The two failures have different fingerprints. A byte-order mistake swaps every multi-byte field at once and in a characteristic way: a temperature of 25, `0x0019`, comes back as `0x1900` — 6400. Switching the `binary.ByteOrder` fixes every field simultaneously. A layout mistake is positional: the first field is right and everything after it is shifted, because the Go struct's widths or padding disagree with the device's frame. `binary.Size(&frame)` against the frame length in the spec settles that in one line — if the numbers differ, the struct is the wrong shape. The usual cause when porting from C is alignment padding the sending compiler inserted; `encoding/binary` packs fields with no implicit padding, so you declare it explicitly as blank `_ [n]byte` fields, which `binary.Read` skips and `binary.Write` zero-fills.

code

text · 6 lines
text
01 00 00 19 0c 80
|  |  |     |
|  |  |     +-- Millivolts = 0x0c80 = 3200
|  |  +-------- TempC      = 0x0019 = 25 (big-endian)
|  +----------- one padding byte
+-------------- Version    = 1

go deeper

for a junior

Be ready to say that a binary decode cannot detect a wrong layout on its own, and that the first move is to print the raw bytes in hex and check the first field by hand against the specification.

for a middle

Explain the two fingerprints: a byte-order mistake swaps every multi-byte field at once and is fixed by one change, while a width or padding mistake is correct up to a point and shifted after it.

for a senior

Show the whole diagnosis: capture the bytes, compare binary.Size against the documented frame width and the bytes actually arriving, declare the device's padding as blank fields, and leave behind a golden-bytes test and a range check so it cannot recur.

for a principal

Own the standard that any binary integration ships with a golden-frame test, an asserted frame width and validation at the decode boundary, because a format with no self-description turns an integration bug into corrupted data nobody notices for weeks.

## Why this failure is silent A binary frame carries no self-description. There is no field name, no type tag, no length that has to agree with anything. If your struct declaration and the device's frame layout disagree, the decode still "succeeds": `binary.Read` consumes exactly `binary.Size` bytes and fills every field. You get numbers. They are simply not the numbers that were sent. That is why a telemetry collector can run for weeks reporting a temperature of 6400 degrees before anyone treats it as a bug rather than a broken sensor. So the diagnosis has to start below the decoder. ## Step one: look at the bytes Capture the frame before it reaches `binary.Read` — tee the reader, or log `%x` of the buffer — and print a hexdump alongside the layout from the specification. Then decode the first two or three fields with a pencil. This single step separates the two failure modes, because they leave different marks. ``` 01 00 00 19 0c 80 | | | | | | | +-- Millivolts = 0x0c80 = 3200 | | +-------- TempC = 0x0019 = 25 | +----------- one padding byte +-------------- Version = 1 ``` ## Fingerprint one: the wrong byte order Read that same frame with the wrong `binary.ByteOrder` and `TempC` becomes `0x1900` = 6400, `Millivolts` becomes `0x800c` = 32780. The signature is: - **Every** multi-byte field is wrong, all at once. - Single-byte fields (`Version`) are still correct — byte order does not affect them. - The wrong values are byte-swapped versions of the right ones: small numbers become suspiciously large, and values near 256 or 65536 give it away instantly. - Flipping `binary.BigEndian` to `binary.LittleEndian` fixes all of them together. That last property is the confirmation. If changing the order fixes some fields and breaks others, you have two bugs, or a frame that mixes orders — which real formats occasionally do, and which is an argument for decoding field by field with the accessors instead of one `binary.Read` over the whole struct. ## Fingerprint two: the wrong layout A layout mismatch is positional. The fields before the point of disagreement are right; from there on, everything is shifted and reads as garbage that does not respond to a byte-order change. The causes, in rough order of frequency when porting a struct from another language: 1. **Alignment padding.** A C compiler routinely inserts padding so that a `uint16` starts on an even offset and a `uint32` on a multiple of four. `encoding/binary` inserts **none** — it packs the fields back to back. So a C struct of `{uint8; uint16; uint16}` is six bytes on the wire and the naive Go translation is five. 2. **A field width changed.** `int` where the format says 32 bits; `int32` where a firmware revision widened the field to 64. 3. **A field was added or reordered** on one side only. `binary.Size` is the fastest test: compare `binary.Size(reading{})` with the frame length the specification states, and with the number of bytes actually arriving per frame. Three numbers, and any disagreement localises the bug immediately. ## Declaring padding The fix for padding is to declare it. A struct field named `_` is treated as padding: `binary.Read` consumes its width and discards the bytes, and `binary.Write` emits zeros for it. So `_ [1]byte` between `Version` and `TempC` makes the Go struct describe the device's frame exactly, and `binary.Size` snaps to 6. This is far better than the alternatives people reach for — reading into a `[]byte` and slicing by hand-computed offsets, or worse, adding a dummy exported field. A blank field documents *why* the gap exists and cannot be read by accident. ## Making it stay fixed Once the bug is understood, three cheap guards keep it from recurring: - **A golden-bytes test.** Encode a known frame and compare against a literal hexdump taken from the specification, then decode that literal and assert the field values. This test fails the moment someone widens a field. - **A size assertion.** `binary.Size(reading{})` against the documented frame width, in the same test. - **A sanity range on decoded values.** A temperature of 6400 is not a valid reading; rejecting it at the boundary turns a silent data-quality problem into a loud decode error. ## Two error signals worth wiring up `binary.Read` returns `io.EOF` only if it read nothing at all, and `io.ErrUnexpectedEOF` if it got part of a frame. A loop that treats both as a clean end of stream will drop a truncated final frame silently — and a stream that consistently ends mid-frame is itself evidence that your struct is wider than the device's frame. Distinguishing those two errors turns a layout bug into a message rather than a shrug. ## The general lesson Binary decoding has no verification built in, so you supply it: dump the bytes, assert the size, pin the layout with golden bytes, and range-check the values. Every one of those is a few lines, and together they convert an invisible failure into an immediate one.

  • The device's C struct has padding. Why can you not just let Go's own struct alignment match it?
    Because `encoding/binary` ignores in-memory alignment entirely — it writes and reads fields packed back to back regardless of how the compiler lays the struct out in memory. The wire layout is fixed by the field types alone. So the padding has to be part of the declaration, as blank `_ [n]byte` fields, which Read skips and Write zero-fills.
  • How would you pin this layout so the bug cannot come back?
    A test with golden bytes: a literal `[]byte` copied from the specification's example frame, decoded and asserted field by field, plus the reverse encode compared back to the same literal. Add an assertion that `binary.Size` equals the documented frame width. Widening a field then fails the test at build time instead of on a device in the field.
  • What does it tell you if binary.Read keeps returning io.ErrUnexpectedEOF at the end of every stream?
    That the struct is wider than the device's frame, so the last frame always comes up short — a layout bug wearing an I/O error's clothes. io.ErrUnexpectedEOF means some bytes arrived but not enough; io.EOF alone means none did. Comparing `binary.Size` with the bytes actually received per frame confirms it in one line.
  • A frame arrives with a value that is physically impossible. Where should that be caught?
    At the decode boundary, immediately after `binary.Read`, as a range check on the decoded field. A binary format carries no validation of its own, so an out-of-range value is your earliest evidence that the layout or byte order is wrong. Catching it there produces a loud decode error instead of a silent data-quality problem discovered weeks later downstream.

Two ways to misread a form: filling it in right to left, or starting one box too far along. The first garbles every entry the same way; the second is correct until the slip and nonsense after it.

saying these in an interview costs you the question

  • Assumes a wrong layout makes binary.Read return an error
  • Adjusts field values with a fudge factor instead of finding the layout bug
  • Expects encoding/binary to insert alignment padding to match a C struct
  • Never looks at the raw bytes before changing the struct
  • Treats io.ErrUnexpectedEOF as an ordinary end of stream