Why can a JMeter dashboard's 99th percentile miss a slow warm-up that its Max column still shows?
answer
- Not every column covers the same samples
- Older samples stop counting
- A per-row sliding window with a default size
- A jmeter.reportgenerator property, prefix included
basics
~20 sThe dashboard's percentile columns run over a sliding window of the most recent samples, sized by jmeter.reportgenerator.statistic_window (default 20,000 per row). Min, Max and Average are not windowed, so they still see the whole run.
solid answer
~40 sEach of the dashboard's Median and percentile figures is a `PercentileAggregator`, which wraps a commons-math3 `DescriptiveStatistics` created with a **window** of `jmeter.reportgenerator.statistic_window` — 20,000 by default. Once a row has taken that many samples, every further value rolls the oldest one out, so the percentile columns describe only the tail end of a long run. `Min` and `Max` in the same row are plain fields updated with `Math.min`/`Math.max`, and `Average` is a `MeanAggregator` incrementing a commons-math3 `Mean` over every sample, so all three still cover the whole run. That is how one row shows a Max from minute two and a 99th percentile that has never heard of it. The window is per row, and the Aggregate Report listener has no window at all.
go deeper
Recall that JMeter's dashboard percentiles are computed over a limited number of recent samples per row, and that the Min, Max and Average in the same row are not limited that way.
Explain the mechanism: a sliding window sized by jmeter.reportgenerator.statistic_window, defaulting to 20,000, that rolls the oldest value out once the row exceeds it.
Diagnose it. Check the row's sample count against the window, compare Max with the top percentile, look at where in time the slow samples were, and regenerate with a larger window to confirm.
Own the sizing rule for the team: what the largest per-label sample count is, how much report-JVM memory four windowed aggregators per label cost there, and whether whole-run percentiles are worth it.
## Which columns are windowed and which are not One row of the dashboard's Statistics table is one `StatisticsSummaryData`. Look at what it actually holds: | Column | Backing object | Covers | |---|---|---| | Median, and the three percentile columns | `PercentileAggregator` over a windowed `DescriptiveStatistics` | the most recent `statistic_window` samples | | Average | `MeanAggregator` wrapping commons-math3 `Mean` | every sample | | Min, Max | plain `long` fields updated with `Math.min` / `Math.max` | every sample | | # Samples, Error % | running counters | every sample | So a single row can honestly report `Max 12,000 ms` and a 99th percentile of `210 ms`, not because the tail was rare but because the samples that produced the Max were rolled out of the percentile window before the report was written. ## The property `PercentileAggregator` reads its window once: ``` jmeter.reportgenerator.statistic_window default 20000 ``` Unlike the estimator key, this one **does** carry the `jmeter.reportgenerator.` prefix — it is assembled from `REPORT_GENERATOR_KEY_PREFIX` plus `statistic_window`. The shipped example line lives in `bin/user.properties`, commented out. The manual's note is short and worth quoting to yourself before you raise it: higher values give better accuracy but need more memory. The default was in fact *reduced* to 20,000 for exactly that reason. ## How the window drops data A commons-math3 `DescriptiveStatistics` constructed with a window keeps values in an array. While fewer than `window` values have arrived it appends; from then on each `addValue` rolls the array forward, discarding the oldest. There is no weighting and no sampling — the discarded values are simply gone by the time `getPercentile` runs. Worked shape, one label, 25,000 samples, the first 5,000 of them slow at 2,000 ms and the remaining 20,000 at 100 ms: - dashboard 90th percentile with the default window: **100 ms** — the 5,000 slow samples have been rolled out; - Aggregate Report listener over the same file: **2,000 ms** — it keeps every value; - dashboard 90th percentile with `jmeter.reportgenerator.statistic_window=30000`: back to 2,000 ms. That is a warm-up, a cold cache, a JIT-cold first minute — anything front-loaded in the run is exactly what the window eats first. ## Diagnosing it on a real report 1. Read `# Samples` for the row. Below the window, the window is irrelevant and you are looking at an estimator difference instead. 2. Compare `Max` with the highest percentile column. A large, unexplained gap on a row with far more than 20,000 samples is the signature. 3. Check where the slow samples were in time. The over-time graphs still show them; the summary percentiles may not. 4. Re-generate the report with the window raised past the row's sample count and see whether the percentile moves. ## Sizing it The window is **per aggregator, per row**: four windowed aggregators for each sample name, plus the overall row. At the default, each holds up to 20,000 doubles — on the order of 160 KB — so a plan with a hundred labels is already carrying tens of megabytes in the report-generating JVM, and raising the window scales that linearly. Set it above the largest per-label sample count you expect if you want whole-run percentiles, and budget the memory in the JVM that generates the report, which need not be the injector. ## What it does not touch - The Aggregate Report and Aggregate Graph listeners. `StatCalculator` keeps a map of every distinct elapsed time with its count and has no window, which is why the manual warns that listener needs memory proportional to the number of distinct values. - The Backend Listener, which has its own window properties rather than this one. - The estimator. `statistic_window` decides *which samples* are in scope; `backend_metrics_percentile_estimator` decides *how* a percentile is read off them. Two independent reasons for two reports of one run to disagree.
- How would you get whole-run percentiles out of the dashboard?Set `jmeter.reportgenerator.statistic_window` above the largest sample count any single label will reach, then regenerate the report. Budget the memory: four windowed aggregators per row, each holding up to that many doubles, in the JVM that generates the report.
- Does the same window apply to the percentile graphs, not just the Statistics table?Yes. The percentile series in the percentiles-over-time graph are built by the same `PercentileAggregator`, so they inherit both the window and the estimator. Only the Min, Max, Average and count series escape the window.
- The row has 8,000 samples and the numbers still disagree with the listener. What now?Below the window size nothing has been discarded, so the window is not the cause. Look at the estimator instead: the dashboard's default `LEGACY` interpolates between neighbouring samples while the listener returns an observed elapsed time.
A dashcam that only keeps the last twenty minutes of footage. The odometer and the top-speed readout still cover the whole drive, so the trip looks fast overall while the tape has no memory of the slow crawl you started in.
saying these in an interview costs you the question
- Assumes every dashboard column covers the same samples
- Reads a low percentile next to a high Max as a contradiction
- Thinks the window samples evenly across the run
- Believes the Aggregate Report listener is windowed too
- Raises the window without budgeting report-JVM memory