skip to content

In an Allure results directory, what is `categories.json`, and what does one `Category` entry in it hold?

level: juniorimportance: must knowfreq 60%

answer

  1. a file, not an annotation
  2. lives with the results, not the report
  3. one array entry per bucket
  4. labels plus four match conditions

basics

~20 s

categories.json is a rule file placed in the Allure results directory. Each entry names one bucket and lists the conditions a result must meet to land in it: a message pattern, a trace pattern, matched statuses and a flaky flag.

solid answer

~40 s

`categories.json` is a JSON array Allure reads from the *results directory* when it generates a report — the same directory the adaptors write `-result.json` files into. Each element is a `Category`: `name`, plus `description` or `descriptionHtml` for the human-facing text, plus up to four match conditions — `messageRegex` against the failure message, `traceRegex` against the stack trace, `matchedStatuses` against the result's status, and `flaky` against the result's own flaky flag. Every condition you supply must hold; a condition you omit places no constraint at all. It is data, not code: no annotation, no listener, nothing on the classpath. In Allure 2 that is the whole `Category` shape, and Allure 3 reads a `categories.json` from the results directory too.

code

json · 15 lines
json
[
  {
    "name": "Infrastructure problems",
    "description": "The environment could not be reached",
    "matchedStatuses": ["broken", "unknown"],
    "messageRegex": ".*Connection refused.*"
  },
  {
    "name": "Runtime errors in checkout",
    "matchedStatuses": ["failed"],
    "messageRegex": ".*RuntimeException.*",
    "traceRegex": ".*checkout.*",
    "flaky": false
  }
]

go deeper

for a junior

Be ready to say what the file is and where it goes: a JSON array of rules in the results directory, read when the report is generated, each entry naming a bucket and the conditions for entering it.

for a middle

Explain the split between labels and conditions, and that the conditions combine with AND while a condition you leave out places no restriction on which results the rule accepts.

for a senior

Show you have operated it: the CI job copies the file into the results directory, one file serves the whole directory, and you can regenerate a kept results directory to test a rule change without re-running the suite.

for a principal

Own the question of whether failure buckets are defined per suite alongside its results or centrally in the report job, and who is accountable for the rule list once several teams write into one directory.

## What the file actually is `categories.json` is **data, not code**. It is a single JSON array that lives in the *results directory* — the directory an Allure adaptor writes its `-result.json` files into, whose location comes from the `allure.results.directory` property and defaults to `allure-results`. Nothing in your test code refers to it: there is no annotation to add, no listener to register, nothing to put on the classpath. When the report is generated the file is read, its rules are applied to the results that were collected, and the report gains a view that groups failures into named buckets. Because it is data and not code, it has two properties worth knowing. First, **changing a rule needs no re-run of the suite** — keep a results directory, edit the file, generate again, and you get a differently grouped report over identical results. Second, a rule is never "deployed": it travels with the run whose results it will be applied to. ## What one `Category` holds In Allure 2 a `Category` has seven fields, and they split cleanly into labels and conditions. | field | role | what it does | |---|---|---| | `name` | label | the bucket's title in the report | | `description` | label | plain text shown with the bucket | | `descriptionHtml` | label | the same text, as markup | | `messageRegex` | condition | tested against the failure's message | | `traceRegex` | condition | tested against the stack trace | | `matchedStatuses` | condition | the statuses the rule will accept | | `flaky` | condition | tested against the result's own flaky flag | Only `name` really constitutes the bucket. `description` and `descriptionHtml` exist for whoever opens the report; the remaining four decide membership. ## How a result reaches a bucket - **Every condition present must hold.** A rule carrying both `messageRegex` and `matchedStatuses` matches only results that satisfy both — the conditions combine with AND, not OR. - **A condition you omit constrains nothing.** A rule with only `messageRegex` will happily consider a result of any status, flaky or not. - **`matchedStatuses` is a list of lowercase status strings.** The report-side vocabulary is `failed`, `broken`, `passed`, `skipped` and `unknown`. - **`flaky` is read, never written.** A rule can require that a result already carries the flag; it cannot put the flag on anything. - **The first rule that matches wins**, so the order of the array is part of the configuration rather than a cosmetic detail. - **A failure matching no rule is not lost**: failed and broken results fall into built-in defaults, which Allure 2 renders as *Product defects* and *Test defects*. ## What it deliberately does not do This is where most of the confusion sits. A rule in `categories.json` does **not**: - change a result's recorded status — a `failed` result stays `failed` whichever bucket it lands in; - mark anything flaky, muted or known; - re-run, retry or quarantine the matched test; - change the report's pass and fail counts; - open, close or link a tracker item. It is a grouping laid over results that already exist, and the same results directory yields the same numbers with the file present or absent. ## Operating the file 1. **Keep it in version control** next to the suite it describes, and review changes to it the way you review code. 2. **Copy it into the results directory** in the job that runs before generation. A rule list that exists only on one engineer's machine buckets nothing in CI. 3. **Remember it is one file per results directory.** If several suites write into a single directory before you generate, they share one rule list — merge the arrays deliberately rather than letting the last copy step decide. 4. **Give every rule a `description` that says what to do** when a test lands in that bucket. A bucket named after a symptom with no instruction is a label, not a triage aid. 5. **Verify by regenerating**, not by re-reading the patterns. Nothing warns you about a rule that never matches. ## Why interviewers ask it It separates people who have only read an Allure report from people who have configured one. The give-away answers are "it is a plugin you install" and "it changes the test's status" — both wrong, and wrong in a way that shows the candidate has never put the file in a results directory and watched a bucket appear.

  • Three suites write their results into one directory before you generate. Whose `categories.json` applies?
    One results directory, one rule list. Whatever `categories.json` ends up in that directory is what the report uses, so consolidating suites means merging the arrays on purpose rather than letting the last copy step win. If the teams genuinely need different buckets, keep separate results directories and generate separate reports.
  • Can a rule in `categories.json` change whether the build is red?
    No. The rules group results that already carry their status, so the pass and fail totals are identical with the file present or absent. Anything deciding a build verdict has to read the results themselves. The category view only tells a human which failures look alike.

saying these in an interview costs you the question

  • Thinks a category rule changes a test's recorded status
  • Puts categories.json in the generated report directory
  • Calls it a plugin or an annotation rather than a data file
  • Believes a matching rule re-runs or quarantines the test