Walk through setting up and running a minimal JMH benchmark project: dependency/annotation-processor setup, the generated harness, and how a benchmark gets executed.

Two deps: jmh-core + jmh-generator-annprocess (annotation processor)
@Benchmark + processor generates harness + Main at compile time
Build a self-contained benchmarks.jar, run with java -jar (regex filter, -f/-wi/-i)
Or drive via OptionsBuilder + Runner.run() (CI-friendly)
Per fork: setup -> warmup (discarded) -> measurement (recorded) -> teardown -> aggregate

basics

~20 s

Add the JMH core library plus its annotation processor as dependencies. Write a method annotated with @Benchmark. At build time the annotation processor generates harness code and a runnable Main. Then you run the benchmarks by executing that generated jar (or via a Runner in code), and JMH does warmup, measurement, and forking automatically.

solid answer

~60 s

A minimal JMH project needs two dependencies: jmh-core (the runtime) and jmh-generator-annprocess (the annotation processor that generates the harness at compile time). You annotate methods with @Benchmark and add lifecycle/config annotations (@State, @BenchmarkMode, @Warmup, @Measurement, @Fork) as needed. During compilation the processor reads those annotations and emits generated benchmark classes plus a Main with the standard JMH entry point — the recommended setup is a Maven/Gradle build producing a self-contained 'benchmarks.jar' (uber-jar) you launch with java -jar. JMH executes by, for each benchmark and each fork, starting a fresh JVM, running warmup iterations to let the JIT compile and stabilize, then measurement iterations whose results are recorded, and finally aggregating statistics across iterations and forks. You can instead drive it from code with an OptionsBuilder + Runner.run(...), which is convenient for filtering benchmarks by regex and setting options programmatically, e.g. in CI. Either way you control it via annotations and/or runner Options; the cardinal rule is never to run JMH benchmarks alongside other code or under an IDE's normal run without the generated harness, because the harness is what makes the measurement valid.

code

java · 25 lines

java

import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.options.*;
import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(2)                     // 2 fresh JVMs per benchmark
public class MinimalBench {

    @Benchmark
    public int add() {
        return 2 + 3;        // returned -> consumed (no dead-code elimination)
    }

    // Optional: launch programmatically instead of `java -jar benchmarks.jar`
    public static void main(String[] args) throws Exception {
        Options opt = new OptionsBuilder()
                .include(MinimalBench.class.getSimpleName())
                .build();
        new Runner(opt).run();
    }
}

for a junior

Can add the two dependencies, write an @Benchmark method, and run the generated jar to get a result.

for a middle

Understands the annotation processor generates the harness at compile time, can configure warmup/measurement/forks, and knows both the jar and the Runner launch paths.

for a senior

Sizes warmup/measurement/forks deliberately, integrates JMH into CI via the Runner with regex includes, and explains why running outside the generated harness invalidates results.

for a principal

Establishes benchmarking as a repeatable, reviewable practice — pins configurations, controls environment/noise, and ensures results feed real decisions rather than being run ad hoc and trusted blindly.

The pieces you need

A JMH benchmark is not just a method you call — it depends on generated code. Two dependencies:

jmh-core — the runtime library (annotations, the Runner, Blackhole, statistics).
jmh-generator-annprocess — the annotation processor. An annotation processor is a compiler plugin that runs during javac, reads your annotations, and generates extra source files. JMH uses it to turn each @Benchmark method into a correct, optimization-resistant harness class.

In Maven you typically also use the JMH archetype or the shade plugin to build a self-contained benchmarks.jar (an "uber-jar" bundling everything). Gradle has equivalent plugins.

Writing the benchmark

You annotate a method with @Benchmark and decorate the class/method with configuration annotations:

@State for inputs (see the state topic),
@BenchmarkMode / @OutputTimeUnit for what/how to report,
@Warmup(iterations=…, time=…) and @Measurement(iterations=…, time=…) to size the warmup and measured phases,
@Fork(n) for the number of fresh-JVM trials.

Return a value (or use a Blackhole) so the JIT can't dead-code-eliminate the work.

What the processor generates

At build time the processor emits, for each @Benchmark:

a generated harness class that wraps your method in the proper measurement loop (timing, warmup/measurement separation, Blackhole consumption, state injection),
a org.openjdk.jmh.Main entry point and a metadata list of all benchmarks.

You never write this code; it's the whole reason JMH's results are valid.

Two ways to run

1. The generated jar (recommended for real results). Build the uber-jar, then:

java -jar target/benchmarks.jar           # run all
java -jar target/benchmarks.jar MyBench    # filter by regex
java -jar target/benchmarks.jar -f 3 -wi 5 -i 5   # 3 forks, 5 warmup, 5 measure

Running from a clean java -jar (not inside the IDE's app process) is what guarantees a clean environment and correct forking.

2. Programmatically with a Runner. In code:

java

Options opt = new OptionsBuilder()
    .include(MyBench.class.getSimpleName())
    .forks(3).warmupIterations(5).measurementIterations(5)
    .build();
new Runner(opt).run();

This is handy in CI or when you want to set options in one place. Note: even when launched programmatically, JMH still forks fresh JVMs for the trials (unless @Fork(0)).

The execution lifecycle (what actually happens at run time)

For each benchmark, for each fork (fresh JVM):

JMH instantiates the @State objects and runs @Setup(Level.Trial).
Warmup iterations run the harness loop, discarding results, until the JIT has compiled and performance has stabilized.
Measurement iterations run the loop and record the score per iteration.
@TearDown runs; the fork exits. After all forks, JMH aggregates the per-iteration, per-fork scores into a final score with error/confidence and prints it (in the mode and time unit you chose).

Common setup mistakes

Forgetting the annotation processor dependency → no harness generated → @Benchmark does nothing.
Running benchmarks as a normal IDE 'main' against your application classpath instead of the generated jar → invalid measurements.
Sizing warmup too short so the JIT hasn't stabilized → measuring the interpreter.
Not consuming results → dead-code elimination.

## The pieces you need A JMH benchmark is not just a method you call — it depends on **generated code**. Two dependencies: 1. **`jmh-core`** — the runtime library (annotations, the `Runner`, `Blackhole`, statistics). 2. **`jmh-generator-annprocess`** — the **annotation processor**. An annotation processor is a compiler plugin that runs *during `javac`*, reads your annotations, and generates extra source files. JMH uses it to turn each `@Benchmark` method into a correct, optimization-resistant harness class. In Maven you typically also use the **JMH archetype** or the shade plugin to build a **self-contained `benchmarks.jar`** (an "uber-jar" bundling everything). Gradle has equivalent plugins. ## Writing the benchmark You annotate a method with **`@Benchmark`** and decorate the class/method with configuration annotations: - `@State` for inputs (see the state topic), - `@BenchmarkMode` / `@OutputTimeUnit` for what/how to report, - `@Warmup(iterations=…, time=…)` and `@Measurement(iterations=…, time=…)` to size the warmup and measured phases, - `@Fork(n)` for the number of fresh-JVM trials. Return a value (or use a `Blackhole`) so the JIT can't dead-code-eliminate the work. ## What the processor generates At build time the processor emits, for each `@Benchmark`: - a **generated harness class** that wraps your method in the proper measurement loop (timing, warmup/measurement separation, Blackhole consumption, state injection), - a **`org.openjdk.jmh.Main`** entry point and a metadata list of all benchmarks. You never write this code; it's the whole reason JMH's results are valid. ## Two ways to run **1. The generated jar (recommended for real results).** Build the uber-jar, then: ``` java -jar target/benchmarks.jar # run all java -jar target/benchmarks.jar MyBench # filter by regex java -jar target/benchmarks.jar -f 3 -wi 5 -i 5 # 3 forks, 5 warmup, 5 measure ``` Running from a clean `java -jar` (not inside the IDE's app process) is what guarantees a clean environment and correct forking. **2. Programmatically with a Runner.** In code: ```java Options opt = new OptionsBuilder() .include(MyBench.class.getSimpleName()) .forks(3).warmupIterations(5).measurementIterations(5) .build(); new Runner(opt).run(); ``` This is handy in CI or when you want to set options in one place. Note: even when launched programmatically, JMH still **forks fresh JVMs** for the trials (unless `@Fork(0)`). ## The execution lifecycle (what actually happens at run time) For each benchmark, for each **fork** (fresh JVM): 1. JMH instantiates the `@State` objects and runs `@Setup(Level.Trial)`. 2. **Warmup iterations** run the harness loop, **discarding** results, until the JIT has compiled and performance has stabilized. 3. **Measurement iterations** run the loop and **record** the score per iteration. 4. `@TearDown` runs; the fork exits. After all forks, JMH **aggregates** the per-iteration, per-fork scores into a final score with error/confidence and prints it (in the mode and time unit you chose). ## Common setup mistakes - Forgetting the annotation processor dependency → no harness generated → `@Benchmark` does nothing. - Running benchmarks as a normal IDE 'main' against your application classpath instead of the generated jar → invalid measurements. - Sizing warmup too short so the JIT hasn't stabilized → measuring the interpreter. - Not consuming results → dead-code elimination.

What goes wrong if you include jmh-core but forget jmh-generator-annprocess?
The annotation processor never runs, so no harness/Main is generated from your @Benchmark methods — the benchmarks effectively don't exist to JMH and won't run.
Name two ways to launch JMH benchmarks and when you'd pick each.
Build the uber benchmarks.jar and run java -jar (best for clean, trustworthy results and ad-hoc filtering); or drive programmatically with OptionsBuilder + Runner.run() (handy in CI or to set options in code). Both still fork fresh JVMs.

saying these in an interview costs you the question

Omitting the annotation-processor dependency, so no harness is generated
Running benchmarks from a normal IDE main on the app classpath instead of the generated jar
Sizing warmup too short and measuring the interpreter instead of JIT-compiled code
Assuming you must hand-write the measurement loop (the processor generates it)

Walk through setting up and running a minimal JMH benchmark project: dependency/annotation-processor setup, the generated harness, and how a benchmark gets executed.

answer

basics

solid answer

code

go deeper

The pieces you need

Writing the benchmark

What the processor generates

Two ways to run

The execution lifecycle (what actually happens at run time)

Common setup mistakes

context

The pieces you need

Writing the benchmark

What the processor generates

Two ways to run

The execution lifecycle (what actually happens at run time)

Common setup mistakes

Walk through setting up and running a minimal JMH benchmark project: dependency/annotation-processor setup, the generated harness, and how a benchmark gets executed.

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~2 min

The pieces you need

Writing the benchmark

What the processor generates

Two ways to run

The execution lifecycle (what actually happens at run time)

Common setup mistakes

▾common follow-ups2 questions

▾red flags4 flags

§ context

▾related questions5 questions

▾related topics2 topics

The pieces you need

Writing the benchmark

What the processor generates

Two ways to run

The execution lifecycle (what actually happens at run time)

Common setup mistakes

answer

go deeper

context