skip to content

Why does `hmac.compare_digest` exist when `==` already compares two bytes objects?

level: juniorimportance: must knowfreq 50%

answer

  1. How long did the check take?
  2. Ordinary equality stops early
  3. One byte at a time
  4. Same work whatever the input
  5. hmac.compare_digest, re-exported by secrets

basics

~20 s

== on two bytes objects stops at the first differing byte, so the time it takes reveals how much of a secret an attacker guessed right. hmac.compare_digest always does the same work; secrets.compare_digest is the same function.

solid answer

~40 s

CPython compares two `bytes` objects for equality by checking their lengths and then running a byte-by-byte comparison that returns the moment two bytes differ. The result is correct, but the time it took is a measurement of how long the common prefix was. `hmac.compare_digest(a, b)` returns the same boolean while accumulating a difference over the whole buffer instead of returning early, so the timing does not depend on *where* the two values diverge. `secrets.compare_digest` is literally the same function object, re-exported from `hmac`. Reach for it whenever the value on one side is a secret an attacker can submit guesses against and see the result of: message authentication tags, signatures, session identifiers, API keys, password-reset tokens. It is not a faster equality and it is not needed for values nobody is trying to guess.

code

python · 9 lines
python
import hmac
import secrets

stored = secrets.token_bytes(32)
presented = bytes(stored)

print(stored == presented)
print(hmac.compare_digest(stored, presented))
print(secrets.compare_digest is hmac.compare_digest)

go deeper

for a junior

Be ready to name hmac.compare_digest on sight and say why plain == is wrong for a token or signature: it stops at the first differing byte, so its runtime leaks the length of the correct prefix.

for a middle

Explain the mechanics rather than the slogan — length check plus early-exit byte comparison on one side, a fixed-length accumulate-and-test on the other — and note that secrets.compare_digest is the very same function re-exported.

for a senior

Show that you know where == hides: membership tests against a collection of tokens, prefix checks, and comparison-based lookups. Look records up by a non-secret identifier, then run exactly one constant-time compare, and pair it with rate limiting.

for a principal

Own the policy: a review rule that secret comparison goes through one audited helper, an argument for why 'the network hides it' is not a defence you build on, and a judgement about which checks in the codebase are genuinely attacker-iterable.

### What ordinary equality actually does `a == b` for two `bytes` objects in CPython first compares their lengths — different lengths mean not equal, immediately — and then compares the buffers byte by byte, returning as soon as it finds a difference. That is exactly the right implementation for a general-purpose equality operator: it is correct, and it is fast because it does the least work it can get away with. Doing the least work possible is the problem. If the value on one side is a secret and the value on the other side was supplied by whoever is calling you, the *duration* of the comparison is a signal that leaks out of the process. A comparison that stops at the first differing byte takes measurably longer when the first byte happens to be right than when it is wrong. An attacker who can submit many guesses and measure the responses can recover the secret one byte at a time instead of guessing the whole thing at once — turning an impossible search into a linear one. ### What compare_digest does instead `hmac.compare_digest(a, b)` returns the same boolean that `a == b` would, but it is implemented so that the work it performs does not depend on the contents of the arguments. Instead of returning at the first mismatch it walks a fixed number of positions, accumulating the bit-differences it finds, and reports at the end whether the accumulator is still zero. The accelerated implementation is written in C with volatile declarations precisely so the compiler cannot 'optimise' the loop back into an early exit. Nothing about the operation is fast — it is *uniform*, which is the property being bought. `secrets.compare_digest` is not a second implementation; the `secrets` module imports the name straight from `hmac`, so the two names are the same object. Use whichever reads better in the module you are writing. ### Where to use it The rule of thumb: use it whenever one side is a secret, the other side is attacker-controlled, and the attacker can observe the outcome or the latency of the check repeatedly. In practice that means authentication tags on incoming webhooks, request signatures, bearer tokens and API keys, session and CSRF identifiers, password-reset and email-confirmation tokens, and one-time codes. It is not a general-purpose replacement for `==`. Comparing two values an attacker already knows leaks nothing. Comparing configuration constants leaks nothing. Sprinkling it everywhere costs readability and buys nothing. ### The Python-specific traps The subtle part is that `==` hides inside other constructs. `token in valid_tokens` on a `list` runs `==` against each element and short-circuits. On a `set` or as a `dict` key it hashes first, which is better, but a hash collision still falls back to `==`, and the lookup structure itself can leak. `presented.startswith(secret_prefix)` short-circuits. `sorted()` and any comparison-based container operation on secret values short-circuits. If a secret is being matched against a collection, the safe shape is to look the record up by a **non-secret** identifier — a key id, a tenant id, a username — and then run exactly one `hmac.compare_digest` against the single stored value for that record. Two more things it does not do. It does not make the check safe on its own: without rate limiting, an attacker with unlimited guesses does not need a timing side channel. And it does not hide the *length* of the values, only their contents — the loop still runs a number of times that depends on the size of what it was given, which is why fixed-length digests are the natural thing to feed it. ### Reading the timing argument honestly Over a noisy network a single measurement tells an attacker nothing; the differences involved are sub-microsecond and the jitter is milliseconds. The reason this is still non-negotiable in review is that noise is removable by averaging over enough samples, co-located attackers exist, and the fix costs one function call. A candidate who argues 'the network hides it' has understood the physics and missed the engineering: you do not build a system whose safety rests on an attacker not bothering to collect more samples. ### Why not hand-roll it A pure-Python loop cannot credibly deliver this property. `all(x == y for x, y in zip(a, b))` short-circuits by construction; a version that avoids short-circuiting is still at the mercy of the interpreter, which caches small integers, specialises bytecode as it runs, and allocates as it goes — none of which is uniform in time. The constant-time claim lives in a C loop written to defeat the compiler's optimiser, and that is why the answer is always the library function rather than a clever comprehension in your own module.

  • Is `token in valid_tokens` safe if `valid_tokens` is a set rather than a list?
    Better, but not a substitute. A `list` runs `==` against each element and short-circuits. A `set` hashes the value first, so the common path avoids a byte-wise compare — but a hash collision still falls back to `==`, and membership timing can vary with the structure. The safe shape is to look the record up by a non-secret identifier and then run exactly one `hmac.compare_digest` against that record's stored value.
  • Does `hmac.compare_digest` make a token check safe by itself?
    No. It closes one side channel. An attacker who can submit unlimited guesses does not need timing at all, so the check still needs rate limiting and lockout, a single generic failure response, and enough entropy in the token that guessing is hopeless. Constant-time comparison is a layer, not the design.
  • When is using `==` on a secret genuinely fine?
    When the attacker cannot iterate against it: values compared entirely inside the process against inputs they do not control, or comparisons whose outcome and latency they never observe. The honest answer in review is still that `hmac.compare_digest` costs one call, so the burden of proof sits with the person arguing for `==`.

A hotel clerk who stops reading your keycard number the moment a digit is wrong tells you, by how fast they look up, how many digits you got right. A clerk who always reads all sixteen tells you nothing.

saying these in an interview costs you the question

  • Claims compare_digest is a faster equality check
  • Thinks == on bytes already compares every byte
  • Believes network jitter makes timing leaks unexploitable
  • Uses == for MACs but compare_digest for passwords, or the reverse
  • Assumes secrets.compare_digest is stronger than the hmac one
  • Checks a secret with `in` against a list of valid tokens

context