Why does a dedicated harness like JMH exist instead of hand-rolling System.nanoTime() around a loop? Summarize the pitfalls it addresses.

## The seductive wrong way The obvious way to benchmark a Java method is: ```java long start = System.nanoTime(); for (int i = 0; i < N; i++) { doWork(); } long nsPerOp = (System.nanoTime() - start) / N; ``` This is *almost always wrong*, and wrong in ways that are invisible — it compiles, runs, and prints a confident number. The number is just not measuring what you think. ## The catalog of lies it tells Each is covered elsewhere; here's why a single harness is needed to fight *all of them at once*: 1. **No warmup.** Java runs **interpreted** first, then JIT-compiles hot code (tiered: interpreter → C1 → C2). Timing the first iterations measures startup, class loading, and compilation-in-progress — often 10-100x too slow and noisy. 2. **Dead-code elimination (DCE).** The optimizing JIT may **delete** work whose result is never used, because timing isn't observable behavior it must preserve. You time an empty loop. 3. **Constant folding / loop-invariant hoisting.** Fixed inputs let the compiler compute the answer once and reuse it, so per-iteration cost vanishes. 4. **GC and safepoint pauses.** Stop-the-world collection and global safepoints (for GC, deoptimization, lock revocation, thread dumps) inject pauses that a plain mean smears or hides. 5. **Timer issues.** `System.currentTimeMillis()` is a **wall clock** (can jump backward via NTP) with **coarse** (often ~milliseconds) resolution — useless for nanosecond work. `System.nanoTime()` is **monotonic** and high-resolution but is *not* a wall-clock time, has its own call overhead, and reading it inside a tight loop perturbs the measurement. 6. **Single-JVM state leakage.** Running variants back-to-back in one JVM lets profiles, inlining decisions, and heap state from variant A contaminate variant B, so order changes results. 7. **Single-shot vs steady-state confusion.** Measuring one execution vs the warmed-up steady state answers different questions; mixing them misleads. 8. **Coordinated omission** (for latency): a closed-loop loop under-reports the tail. ## How JMH addresses each **JMH** = the **Java Microbenchmark Harness**, part of OpenJDK, written by the people who build the JIT: - **Warmup:** explicit `@Warmup` iterations whose results are discarded; separate `@Measurement` iterations. - **DCE/folding:** `Blackhole.consume(...)` (or returning the value) keeps results observable; `@State` fields make inputs runtime-variable so they can't be folded. - **GC/variance:** runs multiple **forks** — each a *fresh JVM* — so prior state can't leak and per-fork variance is visible; `-prof gc` attributes allocation rate. - **Timer correctness:** the harness handles timing/aggregation; you express *what* to measure (`@Benchmark`), not raw `nanoTime()`. - **Modes:** `Throughput`, `AverageTime`, `SampleTime` (percentiles), `SingleShotTime` (cold/startup) — you pick steady-state vs single-shot deliberately. - **Statistics:** reports mean, error/confidence, and percentiles instead of one fragile number. - **Profilers:** `-prof perfasm`, `-prof gc`, etc., to see *why* a number is what it is. ## The deeper point Microbenchmarking Java is *adversarial*: you are trying to measure exactly the costs the runtime is engineered to make disappear (interpretation overhead, redundant work, pauses). Doing it right requires intimate, version-tracking knowledge of the JIT and GC. JMH encapsulates that knowledge and is maintained as the runtime evolves. Re-deriving it in a hand-rolled loop means re-discovering — usually too late — every pitfall above. So the professional answer to "why JMH?" is: because correct Java microbenchmarking is a solved, *hard* problem, and the harness is the solution. ## When even JMH isn't the whole answer JMH gives a *correct* number for the *micro* you wrote — but a faster micro doesn't guarantee a faster application (memory effects, inlining differences at the call site, real input distributions, contention). Always validate micro wins against end-to-end measurements.