skip to content

TCP, UDP, and DNS

The raw socket layer beneath HTTP: TCP servers and clients, UDP datagrams, and two very different name lookups. Asked when you claim protocol work; dns.lookup on the libuv threadpool is a favourite.

on this pageshow

explore

questions

6

Why does a Node.js net.Socket 'data' handler sometimes receive half a device message or two messages at once, and how should you frame them?

level: middleimportance: must knowfreq 55%

answer

  1. a byte stream has no boundaries
  2. chunks follow reads, not writes
  3. keep the leftover tail
  4. delimiter or length prefix
  5. setEncoding guards split characters

basics

~20 s

A net.Socket delivers a TCP byte stream, and each 'data' chunk is whatever one read returned, not one write. Frame messages yourself: buffer bytes, split on a delimiter or length prefix, and keep the incomplete tail.

solid answer

~50 s

TCP carries an ordered stream of bytes with no message boundaries, and Node emits `'data'` with whatever the last read from the kernel returned, so one device write can arrive in two chunks and two writes can arrive in one. The fix is a framer per connection: append each chunk to a buffer, extract every complete message, and keep the remainder for the next chunk. For a line protocol, call `socket.setEncoding('utf8')` so a multi-byte character split across chunks still decodes correctly, then split on `'\n'`, or let `readline.createInterface({ input: socket })` emit `'line'` events. For a binary protocol, prefix each message with its length, read it with an accessor such as `buf.readUInt32BE(0)`, and wait until the whole body has arrived. Cap the buffered tail so a client that never sends a delimiter cannot exhaust memory.

code

javascript · 31 lines
javascript
import net from 'node:net';

const MAX_LINE = 4096;

net.createServer((socket) => {
  socket.setEncoding('utf8');
  let pending = '';

  socket.on('data', (chunk) => {
    pending += chunk;
    let newline;
    while ((newline = pending.indexOf('\n')) !== -1) {
      handleReading(pending.slice(0, newline));
      pending = pending.slice(newline + 1);
    }
    if (pending.length > MAX_LINE) {
      socket.destroy(new Error('line too long'));
    }
  });

  socket.on('end', () => {
    if (pending.length > 0) console.warn(`dropped partial line: ${pending.length} chars`);
  });
  socket.on('error', (err) => {
    console.error(`device socket error: ${err.message}`);
  });
}).listen(7070);

function handleReading(line) {
  console.log(`reading: ${line}`);
}

go deeper

for a junior

Recall that a TCP socket is a byte stream: 'data' chunks follow reads, not writes, so one message can be split or merged with the next.

for a middle

Walk through a framer: per-connection buffer, extract complete messages, keep the tail, and explain why setEncoding('utf8') matters for characters split across chunks.

for a senior

Show the hardening: a cap on the buffered tail, validating length headers before buffering, and what an unterminated message at 'end' should mean for the protocol.

for a principal

Argue for one shared, tested framing helper for every raw-socket service, since hand-rolled framers repeat the same unbounded-buffer and split-character bugs.

## Why chunks do not match messages A `net.Socket` from `node:net` is a **Duplex stream over a TCP connection**, and TCP delivers an ordered stream of bytes, not a sequence of messages. When bytes arrive, libuv reads whatever the kernel has buffered for that socket and Node emits one `'data'` event with that `Buffer`. The size of each chunk depends on how the bytes happened to cross the network and how much had accumulated before the read, not on how the sender called `write()`. So an IoT device that sends `temp=21\n` and then `temp=22\n` can produce, on the server: - one `'data'` event holding both lines; - two events, one line each; - `temp=2` in one event and `1\ntemp=22\n` in the next. All three are correct behaviour. Code that treats one chunk as one message works on a quiet local network and fails under load, over slow links, or as soon as messages grow past a single packet. Turning off Nagle's algorithm with `socket.setNoDelay()` changes when the sender transmits, not what the receiver's reads return, so it does not remove the need for framing. ## Two framing schemes | Scheme | How a message ends | Good for | Watch out for | |---|---|---|---| | **Delimiter** | A byte sequence such as `\n` | Text protocols, simple device feeds | The delimiter must never appear inside a message; cap the line length | | **Length prefix** | A fixed-size header carries the body length | Binary payloads, arbitrary bytes | Validate the length before buffering; agree on byte order | Either way, the receiver keeps a per-connection buffer of bytes it has received but not yet consumed. ## A line framer for an IoT server For each connection: 1. Call `socket.setEncoding('utf8')` once, so `'data'` hands you strings and a character split across two chunks is decoded correctly. 2. On each `'data'`, append the chunk to a `pending` string. 3. While `pending` contains a `\n`, cut off everything before it as one message and keep the rest. 4. If `pending` grows past a protocol maximum with no newline, destroy the socket with an error. 5. On `'end'`, decide what an unterminated tail means for your protocol; usually it is a truncated message to log and drop. `node:readline` can do steps 2 and 3 for you: `readline.createInterface({ input: socket })` emits a `'line'` event per line, and the interface can be consumed with `for await...of`. It still leaves the length cap to you. ## Strings, Buffers and split characters Calling `chunk.toString()` on every `Buffer` looks harmless and is a classic bug. UTF-8 encodes many characters in two to four bytes, and a chunk boundary can fall in the middle of one; decoding each half separately produces replacement characters. `readable.setEncoding()` (which `socket.setEncoding()` delegates to) keeps the incomplete bytes and joins them with the next chunk before decoding. For binary framing, stay with `Buffer` throughout: accumulate with `Buffer.concat()`, slice complete messages with `subarray()`, and decode only a complete body. How those `Buffer` methods copy or share memory is covered with binary data handling; for framing it is enough that `subarray()` shares memory and is cheap. ## Hardening the framer - **Cap the tail.** Without a maximum, a client that streams bytes and never sends a delimiter, or announces a huge length, makes the server buffer without limit. - **Validate length headers first.** Reject an announced length above the protocol maximum before accepting any of the body. - **One framer per socket.** A module-level buffer shared across connections mixes clients' bytes together. - **Handle the last message on `'end'`.** A peer can end the stream right after a message with no trailing delimiter, depending on the protocol's rules. - **Do not parse in `'close'`.** By then the socket is gone; process complete messages as they appear. - **Respect slow handlers.** If handling a message is slow, `socket.pause()` and `socket.resume()` stop reading while you catch up; the stream mechanics behind that belong to the streams material on backpressure. The same reasoning applies to the client side: a reply from the server can also arrive in pieces, so a client that waits for one `'data'` event and treats it as the whole response has the same bug.

  • How would you frame a binary protocol with a 4-byte length prefix in Node?
    Keep a `Buffer` of pending bytes and append each chunk with `Buffer.concat()`. While at least 4 bytes are buffered, read the length with `readUInt32BE(0)`; if `4 + length` bytes are present, take the body with `subarray(4, 4 + length)`, handle it and drop those bytes, otherwise wait for more data. Reject a length above the protocol maximum before buffering the body.
  • Does calling socket.setNoDelay() stop messages from arriving merged?
    No. `setNoDelay()` disables Nagle's algorithm so small writes leave sooner, which helps latency, but the receiver still reads a byte stream and still sees merged or split chunks. Framing is required either way. Note that a raw `net` socket keeps Nagle enabled unless you call `setNoDelay()` or pass `noDelay: true`, whereas `node:http` servers default `noDelay` to `true`.

  • Each socket.write() on the sender arrives as exactly one 'data' event.
  • Calling socket.setNoDelay(true) makes message boundaries reliable.
  • Calling chunk.toString() on every Buffer is safe for UTF-8 text.
  • Small messages always arrive whole, so only large ones need buffering.
  • One shared module-level buffer is fine for every connection's partial data.
open as a page

In Node.js, why can slow name resolution stall unrelated fs and crypto calls, and how do dns.lookup() and dns.resolve4() differ?

level: seniorimportance: must knowfreq 55%

basics

~20 s

dns.lookup() runs the OS resolver getaddrinfo(3) on libuv's threadpool, shared with fs, crypto and zlib, so slow lookups queue that work. dns.resolve4() sends DNS queries over the network asynchronously, never uses the threadpool, and ignores /etc/hosts.

open as a page

In Node.js, when a net.createServer TCP server accepts a connection, what do the Socket's 'data', 'end', 'error' and 'close' events each tell you?

level: juniorimportance: should knowfreq 40%

basics

~20 s

Each connection arrives as a net.Socket: 'data' delivers a chunk of received bytes, 'end' means the peer stopped sending, 'error' reports a failure and is followed by 'close', and 'close' fires when the socket is fully shut.

open as a page

In Node.js, what does tls.connect() verify by default, and why can it fail with ERR_TLS_CERT_ALTNAME_INVALID when https.get() to the same host succeeds?

level: middleimportance: should knowfreq 30%

basics

~20 s

tls.connect() refuses the connection unless the certificate chains to a trusted CA and matches the host name. Unlike https, it sends no SNI without servername, so a server hosting many names may present its default certificate, failing the name check.

open as a page

On a Node.js TCP server that calls socket.setTimeout(60000) on every net.Socket, why do connections from powered-off devices stay open for hours, and what fixes it?

level: seniorimportance: should knowfreq 40%

basics

~20 s

socket.setTimeout() only emits 'timeout' after idle time and never closes the socket, so the handler must call socket.destroy(). A powered-off device sends nothing at all, so also enable socket.setKeepAlive() to let the kernel detect the dead peer.

open as a page

In Node.js, what does a node:dgram socket.send() callback tell a UDP metrics sender, and why connect the socket to the collector once?

level: middleimportance: nice to knowfreq 20%

basics

~20 s

A dgram send() callback only says the datagram reached the operating system, or why it failed; nothing confirms delivery. Calling socket.connect() once resolves the collector's name once, whereas an unconnected send() to a host name runs dns.lookup() on every call.

open as a page