Approaching a link's capacity needs long code blocks - how do you weigh that against a latency budget?

principalnice to knowfreq 27%

answer

the guarantee is asymptotic in n
block length is delay, not just memory
rate margin buys a shorter block
error exponent grows as rate drops
feedback cuts latency, not capacity

basics

~20 s

The reliability guarantee is asymptotic in block length, so rates near capacity need long blocks a decoder must receive in full before it emits anything. Buying latency back means operating further below capacity and paying in throughput instead.

solid answer

~40 s

Block length is latency: the decoder generally waits for the whole block, so n channel uses of delay sit in front of every delivered payload, plus the buffering on both ends. The error exponent is the lever - at a given rate below capacity, error probability falls roughly exponentially in n, and the further below capacity you sit, the faster it falls. That gives a clean trade: give up some rate and you reach the same error target with a much shorter block, and therefore a much lower delay. So the decision is driven by the traffic. Bulk transfer over a long round trip wants long blocks and rates close to capacity; interactive or deadline-bound traffic wants margin and short blocks, even though it wastes part of the ceiling.

Block length is latency: the decoder generally waits for the whole block, so `n` channel uses of delay sit in front of every delivered payload, plus the buffering on both ends. The error exponent is the lever - at a given rate below capacity, error probability falls roughly exponentially in `n`, and the further below capacity you sit, the faster it falls. That gives a clean trade: give up some rate and you reach the same error target with a much shorter block, and therefore a much lower delay. So the decision is driven by the traffic. Bulk transfer over a long round trip wants long blocks and rates close to capacity; interactive or deadline-bound traffic wants margin and short blocks, even though it wastes part of the ceiling.

go deeper

for a junior

Take away the basic tension: stronger, more reliable coding generally means waiting for more of the transmission to arrive before anything can be handed on.

for a middle

Explain that the reliability promise holds as block length grows, so approaching the ceiling means long blocks, buffering on both ends and a heavier decoder.

for a senior

Show the trade in both directions and tie the retransmission policy to the round-trip time, rather than defaulting to the strongest code available.

for a principal

Own the operating point: split traffic classes, keep adaptation headroom, treat unused rate as the budget that pays for the deadline, and state plainly that feedback improves latency rather than the ceiling.

Why the theorem hands you a latency bill

The achievability half of the noisy-channel coding theorem is an asymptotic statement: for a rate below capacity, the error probability goes to zero as the block length grows. Nothing in it says the block length is small. The engineering consequence is direct:

The decoder usually cannot produce output until it has the entire block, so n channel uses of transmission sit in front of every delivery.
Both ends buffer a block, which is memory on a remote station and memory at the receiver.
Decoders for codes that sit close to the ceiling are the expensive ones, adding processing delay on top of the transmission delay.

So "run close to capacity" and "deliver quickly" are in direct tension, and the tension is structural rather than a defect of some particular code.

The lever: rate margin buys block length

Below capacity, the block-error probability falls roughly exponentially in the block length, at an exponent that grows as the rate drops further below C. That is the knob:

Operating point	Block length for a given error target	Delay	Payload throughput
Just under `C`	Long	High	Near the ceiling
Comfortably under `C`	Much shorter	Low	Visibly below the ceiling
Far under `C`	Short	Lowest	Wasteful

Read the table as a currency conversion: rate you choose not to use is bought back as latency. A design that steps its rate from 0.5 to 0.4 on a link of capacity 0.531 is not simply being timid; it is purchasing a shorter block at the same reliability.

Deciding it from the traffic

Characterise the deadline. Bulk archival transfer has no per-block deadline and should take the long block. A control loop or an interactive stream has one, and the block must fit inside it with room for the rest of the path.
Measure the round trip. Where feedback is cheap and fast, short blocks plus retransmission of the failures is often better than a long block that never fails: you pay the retransmission only on the bad blocks. Where the round trip is long, a retransmission costs more than the strongest code, and the design should aim to get it right first time.
Split the traffic. Different classes rarely want the same answer. A link can carry deadline-bound traffic at a conservative rate with short blocks and bulk traffic at an aggressive rate with long ones.
Leave adaptation headroom. If conditions vary, the operating point has to be able to move, which means not starting at the edge.

Feedback: what it does and does not do

A tempting proposal is to add a feedback channel and claim it raises what the link can carry. For a memoryless channel it does not: feedback does not increase the capacity. What it genuinely buys is operational, and that is worth a lot:

retransmission of only the blocks that actually failed, instead of protecting every block against the worst case;
rate adaptation, so the operating point tracks a channel whose noise moves;
shorter blocks and simpler decoders for the same delivered reliability, because the residual failures are handled by a retry rather than by the code.

Stating that distinction cleanly - capacity unchanged, latency and complexity improved - is usually what an interviewer is listening for.

The judgement to demonstrate

There is no single right operating point, which is what makes this a lead's call rather than an arithmetic exercise. What a good answer contains:

an explicit statement that block length is delay, not just memory;
the trade named in both directions - margin below capacity buys a shorter block, and a longer block buys rate;
traffic-class awareness, so the deadline-bound and the bulk paths are not forced into one compromise;
honesty about the estimate: the capacity figure came from a noise model, and a design pinned to the edge of it fails the first time the model is optimistic;
a retransmission policy chosen against the round trip rather than by default.

The failure mode to avoid is the one that sounds most technically ambitious: chase the ceiling, choose the longest block the memory allows, and discover that the achieved throughput is fine while the end-to-end delay has quietly broken the thing the link was built for. Capacity is a ceiling on one axis only, and the design lives in more than one dimension.

## Why the theorem hands you a latency bill The achievability half of the noisy-channel coding theorem is an asymptotic statement: for a rate below capacity, the error probability goes to zero **as the block length grows**. Nothing in it says the block length is small. The engineering consequence is direct: - The decoder usually cannot produce output until it has the entire block, so `n` channel uses of transmission sit in front of every delivery. - Both ends buffer a block, which is memory on a remote station and memory at the receiver. - Decoders for codes that sit close to the ceiling are the expensive ones, adding processing delay on top of the transmission delay. So "run close to capacity" and "deliver quickly" are in direct tension, and the tension is structural rather than a defect of some particular code. ## The lever: rate margin buys block length Below capacity, the block-error probability falls roughly exponentially in the block length, at an exponent that **grows as the rate drops further below `C`**. That is the knob: | Operating point | Block length for a given error target | Delay | Payload throughput | |---|---|---|---| | Just under `C` | Long | High | Near the ceiling | | Comfortably under `C` | Much shorter | Low | Visibly below the ceiling | | Far under `C` | Short | Lowest | Wasteful | Read the table as a currency conversion: **rate you choose not to use is bought back as latency**. A design that steps its rate from 0.5 to 0.4 on a link of capacity 0.531 is not simply being timid; it is purchasing a shorter block at the same reliability. ## Deciding it from the traffic 1. **Characterise the deadline.** Bulk archival transfer has no per-block deadline and should take the long block. A control loop or an interactive stream has one, and the block must fit inside it with room for the rest of the path. 2. **Measure the round trip.** Where feedback is cheap and fast, short blocks plus retransmission of the failures is often better than a long block that never fails: you pay the retransmission only on the bad blocks. Where the round trip is long, a retransmission costs more than the strongest code, and the design should aim to get it right first time. 3. **Split the traffic.** Different classes rarely want the same answer. A link can carry deadline-bound traffic at a conservative rate with short blocks and bulk traffic at an aggressive rate with long ones. 4. **Leave adaptation headroom.** If conditions vary, the operating point has to be able to move, which means not starting at the edge. ## Feedback: what it does and does not do A tempting proposal is to add a feedback channel and claim it raises what the link can carry. For a memoryless channel it does not: feedback does not increase the capacity. What it genuinely buys is **operational**, and that is worth a lot: - retransmission of only the blocks that actually failed, instead of protecting every block against the worst case; - rate adaptation, so the operating point tracks a channel whose noise moves; - shorter blocks and simpler decoders for the same delivered reliability, because the residual failures are handled by a retry rather than by the code. Stating that distinction cleanly - capacity unchanged, latency and complexity improved - is usually what an interviewer is listening for. ## The judgement to demonstrate There is no single right operating point, which is what makes this a lead's call rather than an arithmetic exercise. What a good answer contains: - an explicit statement that block length is delay, not just memory; - the trade named in both directions - margin below capacity buys a shorter block, and a longer block buys rate; - traffic-class awareness, so the deadline-bound and the bulk paths are not forced into one compromise; - honesty about the estimate: the capacity figure came from a noise model, and a design pinned to the edge of it fails the first time the model is optimistic; - a retransmission policy chosen against the round trip rather than by default. The failure mode to avoid is the one that sounds most technically ambitious: chase the ceiling, choose the longest block the memory allows, and discover that the achieved throughput is fine while the end-to-end delay has quietly broken the thing the link was built for. Capacity is a ceiling on one axis only, and the design lives in more than one dimension.

Does adding a feedback channel raise what a noisy link can carry?
Not the capacity of a memoryless channel - feedback does not increase it. What it buys is operational: retransmit only the blocks that actually failed, adapt the rate as conditions move, and hit the same delivered reliability with shorter blocks and simpler decoders. That is a latency and complexity win, not a ceiling win.
When is a short block plus retransmission better than a long, strong code?
When the round trip is short relative to the deadline, so paying a retry on the occasional failed block costs less than protecting every block against the worst case. On a long round trip the retry is the expensive path and the design should favour getting it right in one shot.
How do you justify running well below the ceiling to a stakeholder?
Frame the unused rate as what pays for the latency target and the margin against a channel worse than modelled. The ceiling was computed from a noise estimate and is asymptotic in block length, so an operating point pinned to it delivers neither the deadline nor the reliability it appears to promise.

saying these in an interview costs you the question

Treats block length as buffer size rather than delay
Claims a feedback channel raises the link's capacity
Chases the ceiling regardless of the deadline
Uses one operating point for bulk and interactive traffic
Assumes a longer block always improves the design

Approaching a link's capacity needs long code blocks - how do you weigh that against a latency budget?

§ answer

basics

solid answer

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Why the theorem hands you a latency bill

The lever: rate margin buys block length

Deciding it from the traffic

Feedback: what it does and does not do

The judgement to demonstrate

▾common follow-ups3 questions

▾red flags5 flags

Why the theorem hands you a latency bill

The lever: rate margin buys block length

Deciding it from the traffic

Feedback: what it does and does not do

The judgement to demonstrate

answer

go deeper