skip to content

Floating-Point Robustness

Why 0.1 + 0.2 != 0.3, how to compare floats safely, why distant open-world geometry jitters and how origin rebasing fixes it. Interviewers probe it because float bugs like desyncs resist reproduction.

on this pageshow

explore

questions

5

Why does 0.1 + 0.2 not equal 0.3 in IEEE 754 double precision, and how should game code compare floats instead of using ==?

level: juniorimportance: must knowfreq 55%

answer

  1. binary fractions, not decimal ones
  2. sign, exponent and fraction bits
  3. one tenth repeats forever in base 2
  4. every operation rounds to nearest
  5. thresholds and tolerances, not equality

basics

~20 s

IEEE 754 stores binary fractions, so 0.1, 0.2 and 0.3 are each rounded to the nearest representable double and the sum lands one step above the stored 0.3. Compare computed floats with a threshold or tolerance, never ==.

solid answer

~50 s

An IEEE 754 number is a sign bit, a biased exponent and a fraction: a double has 1, 11 and 52 bits, a single-precision float 1, 8 and 23. Only fractions whose denominator is a power of two are exact, and one tenth in binary is `0.000110011...` repeating, so `0.1` and `0.2` are stored slightly above their true values and `0.3` slightly below. Each operation rounds to the nearest double, so the sum becomes `0.30000000000000004`, one step above the stored `0.3`. Nothing is broken; two routes to the same number simply land on neighbouring values. In game code I avoid equality on computed floats: I use a threshold such as `timer >= duration`, or a tolerance such as `abs(a - b) <= eps` chosen for the size of the values, and I keep exact things like ticks and counts in integers.

go deeper

for a junior

Recall the three fields of an IEEE 754 number, that only binary fractions are stored exactly, and that computed floats are compared with a threshold or a tolerance rather than ==.

for a middle

Explain why one tenth repeats in binary, that every operation rounds to the nearest representable value, and why a tolerance has to match the magnitude of the values compared.

for a senior

Point to where equality bugs hide in shipped code, such as timers stepped with +=, waypoint arrival checks and float fields in state hashes, and replace them with thresholds or integer counters.

for a principal

Set a team convention for which quantities are integers by rule (ticks, counts, grid cells) and which comparisons must go through one shared helper with documented tolerances.

## How an IEEE 754 number is stored IEEE 754 is the standard almost every CPU and GPU uses for binary floating point. A **normal** value is stored as three fields and decoded as `(-1)^sign * 1.fraction * 2^(exponent - bias)`: - the **sign** bit, 0 for positive and 1 for negative; - the **exponent**, stored with a bias (127 in single precision, 1023 in double) so it can encode negative powers of two; - the **fraction** (also called the mantissa or significand), the bits after an implicit leading 1. | format | sign | exponent | fraction | precision | gap after 1.0 | |---|---|---|---|---|---| | single, 32-bit | 1 bit | 8 bits | 23 bits | 24 bits, about 7 decimal digits | 2^-23, about 1.19e-7 | | double, 64-bit | 1 bit | 11 bits | 52 bits | 53 bits, about 15-16 decimal digits | 2^-52, about 2.22e-16 | Because the fraction is binary, a value is exact only when it is an integer divided by a power of two and fits in the available bits. `0.5`, `0.25` and `0.375` are exact. `0.1` is not, for the same reason one third has no finite decimal expansion. ## Why 0.1 + 0.2 becomes 0.30000000000000004 1. One tenth in binary is `0.0001100110011...`, with `0011` repeating forever. The stored double is the nearest 53-bit value, `0.1000000000000000055511...`, slightly **above** one tenth. 2. `0.2` has the same bit pattern one exponent higher, so it is stored at `0.2000000000000000111...`, also slightly above. 3. The literal `0.3` is rounded on its own and lands slightly **below**, at `0.2999999999999999888...`. 4. The exact sum of the two stored values falls exactly halfway between the stored `0.3` and the next double up. The default rounding mode, **round to nearest, ties to even**, picks the upper neighbour, `0.3000000000000000444...`. So `0.1 + 0.2 == 0.3` is false although the two sides agree to sixteen significant digits. The addition itself is not faulty: IEEE 754 requires addition, subtraction, multiplication, division and square root to be **correctly rounded**, meaning each returns the representable value nearest the exact result. The error comes from storing decimal fractions in binary, plus one rounding. In single precision the same expression happens to round onto the same float as `0.3` and compares equal. That is where the rounding happened to fall, not a rule, which is the real lesson: whether two computed floats are bit-equal depends on details nobody should have to track. The same expression can also give different bits when a compiler evaluates it in a different order or keeps an intermediate in a wider register, so code that relies on equality can pass every test on one build and fail on another. ## What this means in game code Game state is full of computed floats, and equality on them produces bugs that come and go with frame rate, build settings or data: - **Timers.** A timer advanced by a variable frame time jumps over its target (`2.98` then `3.01`), so `timer == 3.0` never fires. Even thirty steps of exactly `0.1` sum to `3.0000000000000013` in double. Use `timer >= duration`. - **Arrival checks.** A unit walking toward a waypoint rarely lands on it exactly. Test `distanceSquared <= arriveRadius * arriveRadius` instead of comparing positions. - **Tolerances when you do need "equal".** `abs(a - b) <= eps` works when you know the scale of the values; for values that range widely, the tolerance must grow with their magnitude, because the gap between neighbouring floats grows too. - **Integers for exact quantities.** Tick counters, inventory counts, grid coordinates and currency in the smallest unit are integers, and integer arithmetic is exact within its range. - **Exact cases still exist.** A float that was assigned and copied, never computed, keeps its bits: setting `speed = 0.0` and later testing `speed == 0.0` is safe. ## The one rule behind all of it Each operation returns the representable value nearest the exact result. The gap between neighbouring representable values, one **unit in the last place** (ULP), is fixed relative to the magnitude: in single precision it is about `1.2e-7` near `1.0` and about `0.002` near `20,000`. A tolerance that is generous for small values can therefore be smaller than one ULP for large ones, which is why "how close is close enough" always depends on the size of the numbers being compared.

  • Is 0.5 + 0.25 == 0.75 safe to rely on in floating point?
    Yes. All three values are integers divided by powers of two and fit easily in the fraction bits, so they are stored exactly and the correctly rounded sum is exact too. Exactness depends on the values being binary fractions within the available precision, not on floating point being fuzzy in general.
  • Does switching every float to double make equality comparisons safe?
    No. A double carries 53 significant bits instead of 24, so rounding errors shrink by a factor of about 2^29, roughly half a billion, but 0.1 is still not representable and 0.1 + 0.2 still differs from 0.3. Doubles make errors smaller; they do not make computed values equal.
  • Why can a timer checked with timer == 3.0 never fire?
    With a variable frame time the accumulated value steps over 3.0, for example from 2.98 to 3.01, and never equals it. Even fixed steps of 0.1 accumulate rounding: thirty of them give 3.0000000000000013 in double. Test timer >= 3.0, then subtract the duration or reset, so any overshoot is handled.

Write one third as a three-digit decimal and you get 0.333; add three of them and you get 0.999, not 1, although no step was done wrongly. Binary has the same problem with one tenth: it repeats forever and has to be cut off.

  • Floating-point addition is unreliable and gives random results on some CPUs.
  • 0.1 is stored exactly and only the addition introduces the error.
  • Switching to double precision makes == on computed values safe.
  • No fraction can ever be stored exactly in floating point.
  • One fixed epsilon like 0.00001 works for every comparison in a game.
open as a page

When comparing two floats in game code, how do absolute epsilon, relative epsilon and ULP-distance comparisons differ, and where does each fail?

level: middleimportance: should knowfreq 42%

basics

~20 s

An absolute epsilon fails for large values, where neighbouring floats are farther apart than it; a relative epsilon and a ULP distance fail near zero. Robust code combines an absolute floor with a relative tolerance.

open as a page

In an open-world game, why does the player's character model visibly shake 20 kilometres from the world origin, and how does origin rebasing fix it?

level: seniorimportance: should knowfreq 46%

basics

~20 s

Single-precision positions near 20,000 m snap to a grid about 2 mm wide, so vertices, bones and contacts jump between grid points. Origin rebasing periodically shifts the whole world so the player stays near zero, where the grid is fine.

open as a page

Which floating-point sources of non-determinism would you suspect when a lockstep strategy game's two clients drift apart after ten minutes despite identical inputs?

level: seniorimportance: should knowfreq 36%

basics

~20 s

Suspect anything that changes a last bit: a different operation order, fused multiply-add on one build, fast-math compiler flags, differing sin and cos implementations, and a changed rounding or denormal mode. Per-tick state hashes locate the first divergent tick.

open as a page

Should a cross-platform deterministic lockstep game simulate in fixed-point arithmetic or in tightly controlled IEEE floats, and what decides it?

level: principalimportance: should knowfreq 30%

basics

~20 s

Fixed point makes cross-platform determinism a property of integer arithmetic, at the cost of range limits, overflow care and hand-written math; controlled floats keep speed and familiar tooling but need strict flags, own transcendentals and cross-platform tests. The platform spread usually decides.

open as a page