skip to content

Why does a Go UDP agent reading into a 512-byte buffer silently lose part of a 1400-byte datagram?

level: middleimportance: should knowfreq 40%

answer

  1. one read, one whole message
  2. the kernel keeps no remainder
  3. a full buffer is a suspicious buffer
  4. size it from the maximum, not the average
  5. about 64 KiB is the IPv4 ceiling

basics

~20 s

A datagram read is all or nothing: ReadFrom copies at most len(buf) bytes and the kernel discards the rest of that message. You get n of 512, a nil error, and the other 888 bytes are gone.

solid answer

~50 s

UDP delivers whole messages, so a read either takes the datagram or truncates it — the remainder is not queued for the next call the way a stream's unread bytes are. On Unix-like systems `ReadFrom` returns `n == len(buf)` and `err == nil`, which is why the loss is silent: the agent parses a half line and reports a parse error, or worse, accepts a plausible prefix. Size the buffer to the largest message the protocol accepts, not the average one — the ceiling for an IPv4 UDP payload is 65507 bytes, and a message meant to cross Ethernet without fragmenting has to fit in roughly 1472. The portable way to notice truncation is to allocate one byte more than your maximum and treat a read that fills the buffer as an oversized message to drop and count.

code

go · 14 lines
go
const maxMsg = 1472 // largest metric line the agent accepts

buf := make([]byte, maxMsg+1) // one spare byte to detect oversize
for {
	n, addr, err := pc.ReadFrom(buf)
	if err != nil {
		return err
	}
	if n > maxMsg {
		log.Printf("oversized datagram from %s, dropped", addr)
		continue
	}
	handleMetric(buf[:n])
}

go deeper

for a junior

Remember that a datagram read takes one whole message and throws away whatever does not fit, so the buffer must be big enough before the first read, not after.

for a middle

Explain why n equal to len(buf) with a nil error is the signature of truncation, and how you size a buffer from the protocol's maximum message rather than its typical one.

for a senior

Diagnose it from the outside: wire datagram counts match read counts while byte counts do not, and parse errors cluster on the largest messages. Then bound the protocol so one message fits one packet.

for a principal

Own the message-size contract you publish to every team that sends to the agent, including what happens to anything larger and how that limit is enforced and observed.

## Datagram reads are all-or-nothing On a stream connection, a `Read` that returns 512 of the 1400 bytes in flight means "here is what I have, ask again for the rest". The remaining bytes stay in the kernel's receive queue and the next `Read` continues where you left off. A datagram socket has no such continuation. The kernel's receive queue holds **messages**, not bytes. When `ReadFrom` dequeues a message it copies up to `len(p)` bytes into your slice and then **drops the whole message**, including the part that did not fit. There is no second read that recovers the tail. This is the semantics of `recvfrom(2)` and Go passes it straight through. So with a 512-byte buffer and a 1400-byte datagram, on Linux and the BSDs you get: - `n == 512` - `addr` = the real sender - `err == nil` Nothing in the return values says "there was more". That silence is the whole trap. Windows is the exception worth knowing: an oversized read there can fail with a message-size error rather than truncating quietly, so the same code behaves differently across platforms. ## How the failure looks in production An agent whose messages are newline-delimited metric lines will chop the last line in half. The parser rejects it and the agent logs a parse error, so the symptom presents as "malformed input from a client" rather than "my buffer is too small" — and it only appears for the largest messages, which are usually the interesting ones. If the format is length-prefixed or binary, a truncated read can even decode into something plausible and wrong. Counting datagrams on the wire and comparing against what the agent accepted separates this from real client bugs: the wire count matches the read count exactly, but the byte counts do not. ## Sizing the buffer Three numbers matter: - **65507** — the largest possible IPv4 UDP payload (65535 minus a 20-byte IP header and an 8-byte UDP header). Allocate that if you cannot bound the message size at all; a single 64 KiB buffer per read loop is cheap. - **~1472** — a payload that fits one 1500-byte Ethernet frame with IPv4 and UDP headers. Messages up to this size cross the network as a single packet. - **Your protocol's maximum** — the number you should actually design to. A datagram protocol works best when one message fits one packet, so most agents declare a limit (say 1472 or 8192) and reject anything larger. A datagram bigger than the path MTU is not illegal: IP fragments it and the receiving kernel reassembles it, so your `ReadFrom` still sees one whole message. But every fragment must arrive for the message to survive, so loss probability climbs sharply with size. "One message, one packet" is the reason to bound it. ## Detecting truncation portably Allocate one byte more than the largest message you accept, and treat a read that fills the buffer as oversized: ```go const maxMsg = 1472 buf := make([]byte, maxMsg+1) n, addr, err := pc.ReadFrom(buf) if n > maxMsg { /* oversized: drop and count */ } ``` Because a legitimate message can never exceed `maxMsg`, `n == maxMsg+1` can only mean the sender sent more and the rest was cut. `(*net.UDPConn).ReadMsgUDP` also returns a `flags` value the operating system fills in, which on some platforms marks a truncated message — but the spare-byte trick works everywhere and needs no build tags. ## Buffer lifetime One buffer, allocated once outside the loop, is the right shape: allocating a fresh 64 KiB slice per datagram turns a metrics agent into a garbage generator. The rule that comes with reuse is that `buf[:n]` is only valid until the next `ReadFrom`. If the payload is handed to another goroutine, a channel, or anything that outlives the iteration, copy it first: ```go msg := make([]byte, n) copy(msg, buf[:n]) ``` Skipping that copy produces a corruption bug that only shows up under load, when reads come fast enough to overwrite the buffer before the consumer looks at it. ## The rule to remember Size the read buffer from the protocol's maximum message, never from its typical one; assume a full buffer means a truncated message; and design the message format so one message fits comfortably in one packet.

  • How can the read loop tell a complete datagram from a truncated one?
    Allocate one byte more than the largest message you accept and reject any read where `n` exceeds the maximum — a valid message can never fill that buffer, so a full one proves the sender sent more. `(*net.UDPConn).ReadMsgUDP` also returns operating-system flags that can mark truncation, but the spare-byte check is portable and needs no platform-specific code.
  • Does the same truncation happen on a Unix stream socket?
    No. The `"unix"` network is a byte stream, so a short read just means more bytes are waiting and the next read continues. The `"unixgram"` network behaves like UDP: message boundaries are preserved, and a read into a buffer smaller than the message truncates it and discards the rest.
  • Is reusing one buffer across every read safe?
    Yes inside the loop, and it is what you want — a fresh 64 KiB allocation per datagram is pure garbage. But `buf[:n]` is only valid until the next `ReadFrom` overwrites the array. Anything that outlives the iteration, such as a payload sent to a worker goroutine over a channel, must get its own copy first.

saying these in an interview costs you the question

  • Says the rest of the datagram arrives on the next read
  • Sizes the read buffer from the average message length
  • Expects an error whenever a datagram is truncated
  • Loops on ReadFrom to reassemble one large message
  • Hands the reused buffer to another goroutine without copying