One Allure 2 report is generated from several results directories, each carrying its own executor.json and environment.properties. How does Allure combine them?
answer
- two blocks, two different merge rules
- one is elected, the other is unioned
- election is on a single numeric field
- a key in dispute keeps every value
- missing buildOrder sorts first, so it loses
basics
~20 sDifferently. Every executor block is kept, but where one is needed Allure elects the latest by the largest buildOrder, with missing values sorting first. Environment entries are grouped by key and their values unioned, so a disputed key shows several values.
solid answer
~50 sThe two blocks merge on opposite principles. Every `executor.json` found is read and listed, but consumers that need exactly one - the report's fallback title, the three provenance fields on a trend point - ask for the *latest* executor, which Allure 2 defines as the maximum `buildOrder` with missing values sorted first. So a block without a `buildOrder` can never win against one that has it, and if none has one the choice is effectively arbitrary. The environment block instead groups entries by key across all directories and collects the values into a set, producing one row per key holding a list of values. Ten shards agreeing collapse to a single value; two disagreeing produce a row with two values, with no error and no winner. That multi-valued row is the cheapest signal you have that your shards did not run against the same thing.
go deeper
Know that a report can be built from several results directories at once, and that each of them may carry its own executor and environment files.
Explain the two merge rules: executors are elected by the largest buildOrder with missing values first, while environment values for one key are collected into a set.
Read a multi-valued environment row as evidence that the shards disagreed about the run, and decide case by case whether that is an honest description of a matrix or a real configuration drift.
Decide what a consolidated report is allowed to claim: which keys must be identical across shards, who writes the single executor block, and what a disagreement should do to the run.
Consolidating several results directories into one report is normal: a suite sharded across machines, or a run whose Java, browser and API suites each wrote their own directory. Both provenance blocks survive that, but they survive under **two different merge rules**, and the difference is not arbitrary - it follows from what each block is for. ## Executors: every block is kept, one is elected Every `executor.json` that is found is read into its own `ExecutorInfo` and all of them are listed, so a report consolidated from three directories can show three executor rows. But several consumers need exactly *one* executor. The report's fallback title needs one name. A trend point has room for one `buildOrder`, one `reportUrl` and one `reportName`. For those, Allure 2 elects the **latest** executor, defined precisely: - the maximum `buildOrder`, - compared with **missing values sorting first**. Two consequences follow, and both bite in practice: 1. A block with no `buildOrder` can never win against a block that has one. If the shard you care about omits the number, its `reportUrl` is not the one that gets stamped onto the trend. 2. If no block has a `buildOrder`, they are all equally "first" and the election is decided by nothing meaningful. You will get an executor; you have no say in which. Note what "latest" does **not** mean. It is not the newest file on disk, not the last directory listed on the command line, and not the block with the newest-looking `buildName`. It is a numeric comparison on one field. ## Environment: values are unioned, not resolved The environment block merges on the opposite principle. Entries from every results directory are grouped by their key, and the values for one key are collected into a **set**. The result is one row per key holding a **list** of values. That means: - Ten shards writing the same `browser=chrome` produce one row with one value, because the set collapses the duplicates. - Two shards writing `browser=chrome` and `browser=firefox` produce one row with **two** values. No error, no winner, no warning. There is no precedence rule here at all, because there is nothing sensible to prefer. Which browser is the "real" one when half the run used each? ## The multi-valued row is a signal, read it as one This is the part worth internalising. A row in the environment block with more than one value is telling you that the directories you consolidated **did not agree** about the run. Sometimes that is exactly right and you should leave it: a cross-browser matrix genuinely ran under several browsers, and one row listing all of them describes the run honestly. Sometimes it is a defect: an app version that differs between shards means half the run tested one build and half tested another, and every number in the consolidated report is now an average across two things. Because it costs nothing, an environment key that *should* be identical across shards - the build under test, the target's version - is the cheapest drift detector available. It shows up as a second value in a row rather than as an incident three weeks later. ## Why the two rules differ The asymmetry is not an accident of implementation. The two blocks answer different kinds of question, and the merge rule follows from the kind: - An executor block **identifies** something. A build has one number, one name and one address, so when a consumer needs a build it needs exactly one, and an election is unavoidable. Allure makes the rule explicit and numeric rather than guessing. - An environment block **describes** conditions. A run can honestly have happened under two browsers or in two regions, so collapsing that to one value would be a lie the report tells confidently. Keeping both is the only truthful merge. ## Practical shape - One CI build usually means **one** executor block. If a single build produced all of the shards, several `executor.json` files describing it are not more information, they are an election you did not intend to hold. - If you do write more than one, give every block a `buildOrder`, so the choice is yours rather than the comparator's. - Write the environment keys that must agree across shards into every shard, precisely so that disagreement becomes visible. - Do not expect the report to reconcile anything. Neither block validates, neither errors on conflict, and neither logs the losing value.
- You shard one suite across ten machines and every shard writes the same `environment.properties`. What does the report show?One row per key with a single value, because the values for a key are collected into a set before they are listed, so ten identical values collapse to one. The moment a single shard disagrees, that row grows a second value - which is exactly the cheap drift signal worth keeping, since it appears in the report rather than in an incident weeks later.
- Should every shard write its own `executor.json`?Only when they genuinely are different executors, and usually they are not: one CI build produced all of the shards. Several blocks describing the same build are not extra information, they are an election you did not intend to hold. If you do write more than one, give each a distinct `buildOrder` so the outcome is your decision rather than a comparator's.
saying these in an interview costs you the question
- Expects a conflicting environment key to fail the generation
- Thinks the first results directory read wins
- Assumes latest executor means the most recently written file
- Believes only one executor.json is allowed per report
- Reads a two-valued environment row as a rendering bug