skip to content

In an Allure report the suite tree is built from the `parentSuite`, `suite` and `subSuite` labels. Why do some tests in one run sit three levels deep and others at the top, with no error anywhere?

level: seniorimportance: should knowfreq 52%

answer

  1. depth is per result, not per report
  2. an empty level is dropped, not shown
  3. no placeholder and no warning
  4. several values put one test in several branches

basics

~20 s

Grouping skips any of the three label names a result has no value for, instead of inserting a placeholder level. A test missing parentSuite moves up one level; a test carrying none of the three attaches straight to the tree root.

solid answer

~40 s

The tree is derived by walking the three label names in order and turning each one's values into a layer. A name the result has no value for yields an empty layer, and empty layers are **dropped** rather than rendered as `unknown` -- so depth is a property of each result, not of the report. A result with `suite` and `subSuite` but no `parentSuite` therefore appears one level shallower, as a sibling of other tests' `parentSuite` groups, and a result with none of the three hangs off the root. The same walk drives the `epic`/`feature`/`story` grouping. The mirror image is just as surprising: a result carrying two `feature` labels is placed under **both** branches, so a tree can hold more leaves than the run had tests. Nothing logs any of it.

code

json · 13 lines
json
[
  { "name": "adds an item to the cart",
    "labels": [
      { "name": "parentSuite", "value": "Cart" },
      { "name": "suite", "value": "Basket" },
      { "name": "subSuite", "value": "Quantity" }
    ] },
  { "name": "clears the cart",
    "labels": [
      { "name": "suite", "value": "Basket" },
      { "name": "subSuite", "value": "Quantity" }
    ] }
]

go deeper

for a junior

Know that the suite tree comes from three labels -- parentSuite, suite and subSuite -- and that a test appears only as deep as the labels it actually carries.

for a middle

Explain the walk: each name becomes a layer, a name with no value yields nothing, and empty layers are discarded rather than filled with a placeholder.

for a senior

Diagnose from the results rather than the report -- count how many results carry each of the three names before touching anything, and expect duplicate leaves wherever a name has several values.

for a principal

Decide whether suites are responsible for always supplying the full label set or the report is responsible for defaulting it, and say what a consolidated report can honestly promise under either choice.

## How the tree is actually built The suite tree is not stored anywhere. It is derived, per report, by walking a fixed sequence of label names -- `parentSuite`, then `suite`, then `subSuite` -- and turning each into a **layer** of the tree. For one result the walk goes like this: 1. Collect every value the result carries for the first name. Deduplicate them. 2. If that collection is **empty, discard the layer entirely.** Do not create a placeholder. 3. Repeat for the second and third names. 4. Descend through whatever layers survived, creating group nodes as needed, and attach the result as a leaf at the bottom. Step two is the whole answer. The layer for a label the result does not carry is not rendered as `unknown`, `none` or an empty string -- it simply **is not there**, and the walk continues one level shallower. ## What that produces Depth is therefore a property of each individual result, not of the report: | labels on the result | where it lands | |---|---| | `parentSuite`, `suite`, `subSuite` | three levels deep | | `suite`, `subSuite` only | two levels deep, beside other results' `parentSuite` groups | | `parentSuite` only | one level deep | | none of the three | attached directly to the root of the tree | A run mixing suites that set all three names with suites that set one lands them all in the same tree at different depths, and a group name that was meant to be a `suite` ends up as a sibling of names that were meant to be `parentSuite`s. The tree looks wrong in a way that is hard to describe, because nothing is missing -- every test is present -- and no error was raised. The same walk builds the behaviour grouping from `epic`, `feature` and `story`, so the same symptom appears there for the same reason. ## The mirror-image surprise: more leaves than tests The other half of the mechanism runs in the opposite direction. A layer is built from **all** the values the result carries for that name, and the walk expands over every one of them. So a result carrying two `feature` labels is placed as a leaf under **both** branches. That is deliberate: it is how a test that genuinely exercises two features gets found from either one. But it means the arithmetic people instinctively apply to a tree is wrong: - The number of leaves in the tree can exceed the number of tests in the run. - A result appearing in several branches contributes to each branch's totals. - Two branches whose numbers add up to more than the run size are not double-counting a bug; they are reporting shared membership. Read alongside the flattening, the pair is neat and worth stating that way in an interview: **a name with no values costs you a level, a name with several values costs you uniqueness.** ## Diagnosing it The fix is never in the report, because the report is faithfully rendering the labels it was handed. Work backwards from the results directory: 1. **Count coverage per label name.** For each of the three names, how many result files carry it at all? A name present on some suites and absent on others is the flattening, confirmed. 2. **Look at who sets each name.** Suite labels usually come from the adaptor, derived from the test's own class and package structure; the ones people add by hand are the ones that go missing. A suite using a different runner or a hand-rolled writer is the usual odd one out. 3. **Count values per name per result.** More than one value for a name is where duplicate leaves come from, and it is often unintentional -- an annotation on the class plus another on the method. 4. **Only then decide what to change.** Either make every suite supply the full set, or accept the varying depth and stop navigating by that tree. ## Filling the gap at report time In **Allure 3**, the `defaultLabels` configuration key supplies a value for a label name when the result carries no label of that name. It is applied as results are read, before anything is grouped, so a suite that never set `parentSuite` can be given one centrally instead of being edited. Note the semantics: it fills in only where the name is **entirely absent**; it does not merge with, or override, values the result already has. **Allure 2 has no equivalent.** There the fix genuinely has to happen where the results are written, which usually means the adaptor configuration or the test source. ## The trap to avoid The tempting conclusion, on seeing a flat tree, is that the labels were lost somewhere between the suite and the report -- that something dropped them. Almost always they were never written. The report has no memory of a label that did not arrive and no way to signal one, so "silently" is not sloppiness on Allure's part; it is the only thing an absence can do.

  • One test appears under two branches of the behaviour grouping. Is that a bug?
    Not on its own. A result carrying two `feature` labels is deliberately placed under both, because the grouping expands every value of a name into its own branch. It becomes a problem the moment you read counts off the tree, since that one test is a leaf in two places and contributes to both branches' totals.
  • Can you repair the depth at report time rather than editing every suite?
    In Allure 3, yes: the `defaultLabels` config key supplies a value for a label name when the result carries none of that name, so the missing level is filled in before grouping happens. Allure 2 has no equivalent, so there the fix has to happen where the results are written.

It behaves like a file path assembled from three optional segments. Leave the middle one out and you do not get an empty folder, you get a shorter path -- and the file simply lands higher up the tree, beside folders rather than inside one.

saying these in an interview costs you the question

  • Expects a placeholder group for the missing level
  • Blames the report generator for a flat tree
  • Assumes leaf count must equal tests run
  • Thinks a missing label makes the result disappear