skip to content

A noisy link has a channel capacity of 0.53 bits per channel use - what does that number promise and what does it forbid?

level: middleimportance: must knowfreq 60%

answer

  1. a ceiling, not a target
  2. the theorem points in two directions
  3. long blocks buy the reliability
  4. below C: error falls towards zero
  5. above C: redundancy only lowers the rate

basics

~20 s

Channel capacity is a reliable-rate ceiling. Any code carrying fewer than 0.53 data bits per channel use can be made as close to error-free as you like by using long enough blocks; no code above that rate can, however much redundancy it adds.

solid answer

~40 s

Capacity is a rate, in payload bits per channel use, and the noisy-channel coding theorem makes two claims about it in opposite directions. The achievability half says that for any rate strictly below `C` there **exists** a code whose decoding-error probability falls towards zero as the block length grows - so reliability does not require slowing down towards zero rate. The converse says that at any rate above `C` the error probability is bounded away from zero no matter how clever the code is; piling on parity only helps by pushing the rate back under `C`. Practically: 0.53 tells you the ceiling on payload throughput per symbol sent, it tells you nothing about which code reaches it, and the price of getting near it is long blocks, decoding work and delay.

go deeper

for a junior

Remember the shape of the claim: capacity is a ceiling on useful bits per symbol sent, and it is set by how noisy the link is, not by how good your code is.

for a middle

Be able to state both halves in the right direction - below the ceiling a good code exists, above it nothing works - and explain that extra parity helps only by lowering the rate.

for a senior

Show you know the guarantee is asymptotic: name the block length, delay and decoder cost that approaching the ceiling buys, and say why real links run with margin under it.

for a principal

Frame it as a budget: the ceiling is fixed by physics and noise, so the argument is about how much of it to spend on payload versus on margin for a channel that is worse than modelled.

## What the number is a rate of A **channel use** is one transmitted symbol - one opportunity to put something on the wire, the air or the light path. **Capacity** `C` is the largest number of *payload* bits that can be delivered, reliably, per channel use. "0.53 bits per channel use" therefore means: over a long run, at most 53 payload bits per 100 symbols transmitted, and that ceiling holds no matter what encoding, modulation trick or decoder is put on either end. Two things capacity is **not**: - It is not a measured throughput of a particular product or link build. It is computed from the channel's noise statistics, and it bounds every possible design over that channel. - It is not a promise about any specific code. It is an existence statement. A rate under capacity says a good code exists; it does not say the one you wrote is good. ## The two halves of the noisy-channel coding theorem The theorem is a pair of claims that point in opposite directions, and an interviewer is usually checking that you know both. 1. **Achievability (below `C`).** For every rate `R < C` and every error target `e > 0`, there is a block length `n` and a code of that rate whose probability of decoding a block wrongly is below `e`. As `n` grows the error probability keeps falling, typically exponentially in `n`. 2. **Converse (above `C`).** For every rate `R > C`, the block-error probability is bounded away from zero; in the strong form it tends to **one** as the block length grows. Longer blocks make things worse, not better. The historically surprising part is the first half. The intuitive engineering answer to a noisy link - repeat each bit many times and take a majority vote - does drive errors down, but its rate goes to zero as it does so. Capacity says you can hold the rate **fixed and positive** and still send the error probability to zero. What you spend instead is block length. ## What you pay to approach it - **Delay.** The decoder generally cannot emit anything until it has the whole block, so block length is latency measured in channel uses. - **Buffering.** Both ends hold a block's worth of symbols. - **Decoder work.** Codes that sit close to capacity are the expensive ones to decode. - **Margin.** `C` is computed from an estimated noise level. If the real channel is worse than the estimate, a rate you believed was under the ceiling is above it. ## Reading a rate against the ceiling | Rate of the code | What the theorem says | What it looks like in practice | |---|---|---| | Well below `C` | Reliable codes exist, with short blocks | Easy margin, payload throughput left unused | | Just below `C` | Reliable codes exist, but only for long blocks | Best throughput, worst latency and decoder cost | | At `C` | No guarantee of vanishing error | An asymptotic boundary, not an operating point | | Above `C` | Error stays bounded away from zero | No redundancy rescues it; only lowering the rate does | ## Where engineers get it wrong The most common mistake in review is treating extra redundancy as a direct cure for errors. Redundancy helps for exactly one reason: it lowers the **rate**. A code sending twice as many symbols per payload symbol has halved its rate, and if that halving takes the rate from above the ceiling to below it, errors become fixable. If the rate is already under the ceiling and the link is still failing, the real cause is elsewhere - the noise is worse than modelled, the noise is bursty in a way the code was not designed for, or the decoder is not good enough - and adding checks will not touch any of those. The second mistake is reading capacity as a claim about the symbols rather than the payload. Nothing stops you running a link **above** capacity; you simply do not get vanishing error when you do. A link carrying uncorrected bit errors at a steady rate is a perfectly ordinary thing. Capacity is the boundary of the *reliable* regime, and the honest statement of the converse is "above `C`, not at vanishing error" rather than "above `C`, nothing arrives". The third is forgetting that the guarantee is asymptotic. "Rate below capacity" is a statement about what happens as blocks get long. At the block lengths a latency budget actually permits, the achievable error rate at a given rate is strictly worse than the asymptotic promise, which is why real designs sit with deliberate margin under the number rather than on it.

  • Does a rate below capacity mean any code at that rate will be reliable?
    No. The achievability half is an existence claim: some code of sufficient block length reaches any error target. Most codes at that rate are poor, and the guarantee is asymptotic in block length, so a short-block code operating just under the ceiling can still fail badly. You need a good code, a good decoder, and enough block length.
  • What exactly happens to error probability at a rate above capacity?
    It is bounded away from zero, and in the strong form of the converse it approaches one as the block length grows. Adding parity symbols changes nothing directly; it helps only by lowering the code's rate back under the ceiling. The only other lever is changing the channel itself, which changes the capacity.
  • Why did the theorem surprise engineers when it was published?
    The intuitive way to fight noise is repetition plus majority voting, which drives errors down only by driving the rate towards zero. The theorem showed that reliability and a fixed positive rate are compatible: the resource you spend to get arbitrarily small error is block length, not throughput.

saying these in an interview costs you the question

  • Treats capacity as the link's measured throughput rating
  • Believes adding redundancy fixes a rate above capacity
  • Says vanishing error requires driving the rate towards zero
  • Assumes capacity is reachable with short blocks and no delay
  • Thinks any code below capacity is automatically reliable