skip to content

What changes about a measured information content when it is expressed in nats instead of bits?

level: middleimportance: nice to knowfreq 22%

answer

  1. the unit is a choice, not physics
  2. the log base fixes the unit
  3. a constant factor, nothing more
  4. natural logarithm gives nats
  5. one nat is about 1.44 bits

basics

~20 s

Only the unit changes. The base of the logarithm sets the unit - base 2 gives bits, the natural logarithm gives nats - and the two differ by the constant factor ln 2, so every ordering, ratio and comparison is unchanged.

solid answer

~50 s

The measure is `-log_b p`, and the base `b` is nothing but the choice of unit: base 2 gives **bits**, base `e` gives **nats**, base 10 gives **bans**. Converting is a single multiplication, since `log_e p = log2 p * ln 2`. So 1 nat is about **1.4427 bits** and 1 bit is about **0.6931 nats** - a nat is the *larger* unit. A 10-bit outcome is about 6.93 nats; nothing about which outcome is more surprising, or by what ratio, moves. Bits are natural wherever you are counting fair yes/no answers or reasoning about storage and transmission. Nats show up in analytic work, because the derivative of the natural logarithm is clean and no `ln 2` factors clutter the algebra. Because the difference is a positive constant factor, no conclusion can depend on the choice.

go deeper

for a junior

Remember that the base of the logarithm only names the unit: base 2 gives bits, the natural logarithm gives nats, and the quantity being measured is identical either way.

for a middle

Explain the conversion and its direction: multiply bits by ln 2 to get nats, so one nat is about 1.4427 bits and a 10-bit outcome is about 6.93 nats.

for a senior

Show why nothing downstream depends on it: a positive constant factor preserves ordering, ratios, the zero at certainty and additivity, so no conclusion can hinge on the unit.

for a principal

Set the reporting convention: derive in whichever unit keeps the algebra clean, but publish in bits with the unit named, so figures stay comparable across teams and cannot be silently misread.

## The base of the logarithm is the unit Surprisal of a single outcome is `-log_b p`. Everything about the measure - that it decreases with probability, is zero at certainty, and adds over independent outcomes - holds for **any** base `b > 1`. The base does exactly one job: it names the unit. | logarithm base | unit | one unit equals | the anchor that defines it | |---|---|---|---| | 2 | bit (also called a shannon) | 1 bit | one fair yes/no outcome, p = 1/2 | | e | nat | about 1.4427 bits | the outcome with p = 1/e | | 10 | ban (also called a hartley) | about 3.3219 bits | one outcome in ten, p = 1/10 | The recurring surprise is the direction: because one nat is about 1.44 bits, **a nat is a bigger unit than a bit**, so the same quantity has a *smaller* number attached when measured in nats. ## Converting is one multiplication Change of base is `log_b x = log_c x / log_c b`, so converting between information units is always a constant factor - never an addition, never anything that depends on `p`: - bits to nats: multiply by `ln 2`, about 0.6931. - nats to bits: multiply by `1 / ln 2`, about 1.4427. - bits to bans: divide by `log2 10`, about 3.3219. Worked: a 10-bit outcome is `10 * 0.6931 = 6.93` nats, or `10 / 3.3219 = 3.01` bans. A one-in-1024 outcome is 10 bits, 6.93 nats and 3.01 bans - three names for one quantity. ## Why the choice cannot change a conclusion Because the conversion is multiplication by a fixed positive constant, everything a comparison could rest on survives it: - **Ordering survives.** If one outcome is more surprising than another in bits, it is more surprising in nats. - **Ratios survive.** An outcome that is twice as many bits as another is twice as many nats. - **The zero survives.** Certainty is zero in every unit, since `log_b 1 = 0` for any base. - **Additivity survives.** Independent outcomes add in every unit; only the size of the unit differs. So a result stated in nats and the same result stated in bits are the same result. What is *not* interchangeable is a bare number with no unit attached: "this outcome carries 6.93" is meaningless until you say of what. ## When each unit is convenient The units are interchangeable, but they are not equally convenient, and the reason is practical rather than mathematical: 1. **Bits, when you are counting halvings or sizing things.** One fair yes/no answer is one bit by definition, which makes bit figures directly comparable to counts of binary choices and to storage and transmission budgets. Whiteboard arithmetic is also easier: powers of two are exact. 2. **Nats, when calculus is involved.** The derivative of `ln x` is `1/x` with no constant attached, so analytic derivations and continuous optimisation come out cleaner in nats; expressing the same work in bits sprinkles factors of `ln 2` through every line for no benefit. 3. **Bans, historically and rarely.** Base 10 is convenient when the natural granularity of the problem is one in ten, but it is uncommon in engineering settings. A practical convention follows from this: do the analysis in whichever unit keeps the algebra clean, then **convert to bits before reporting**, because bits are the unit an engineering audience can interpret against yes/no choices and payload sizes. ## The mistakes worth naming - **Treating the units as different quantities.** They are the same measurement in different units, exactly like metres and feet. - **Getting the direction of the conversion backwards.** Multiplying bits by 1.4427 instead of 0.6931 inflates every figure by about 108 percent. - **Converting by addition.** The relationship is a scale factor, not an offset. - **Assuming a bit figure must be a whole number.** It need not be in any unit; an outcome at `p = 1/100` is 6.64 bits. - **Reporting a bare number.** Without the unit named, a recipient cannot tell 10 bits from 10 nats, which differ by more than 44 percent. ## Why this is asked at all It is a small question with a large diagnostic value. A candidate who knows the base is only a unit has understood that the measure was designed around additivity and continuity, with the constant left free. A candidate who thinks nats measure something different, or that switching base could reorder two outcomes, has memorised a formula without its structure.

  • An outcome carries 10 bits of surprisal. How many nats is that?
    About 6.93 nats, since bits convert to nats by multiplying by `ln 2`, roughly 0.6931. The number gets smaller because the nat is the larger unit - about 1.4427 bits each.
  • Why does analytic work often prefer nats?
    Because the derivative of the natural logarithm is `1/x` with no constant attached, so derivations and continuous optimisation stay free of `ln 2` factors. It is purely a convenience of the algebra; every result converts back to bits by one multiplication.

saying these in an interview costs you the question

  • Says nats and bits measure two different quantities.
  • Claims switching base can reorder which outcome is more surprising.
  • Thinks a nat is a smaller unit than a bit.
  • Converts between the units by adding rather than scaling.
  • Believes an information figure must be a whole number of bits.
  • Reports a bare number without naming the unit it is in.