skip to content

In ReportPortal, when auto-analysis puts a defect group on a failure it also sets `autoAnalyzed`. What does that boolean record, where does it live, and what does it deliberately not say?

level: middleimportance: must knowfreq 58%

answer

  1. who set the label, not how sure
  2. a boolean on the issue
  3. not stored on the test item
  4. the column is auto_analyzed
  5. lets a re-run target machine labels only

basics

~20 s

autoAnalyzed records provenance — analysis set this defect group, not a person. It is a boolean on the issue rather than on the test item, stored in the auto_analyzed column, and it carries no confidence at all.

solid answer

~40 s

`autoAnalyzed` is a provenance bit, not a score. It lives on the **issue** attached to a test item, not on the test item itself — `Issue` on the wire, `IssueEntity` in storage, column `auto_analyzed` — alongside `ignoreAnalyzer`. When analysis applies a proposal it rebuilds the issue with the resolved defect group and this flag set true, carrying the existing `ignoreAnalyzer` through unchanged. What it does not record is how good the match was: a marginal reuse and an obvious one both come out as `true`. Its real payoff is that it splits the project's labels into two populations, which is what lets a re-analysis run target the machine-set ones with `AnalyzeItemsMode` `AUTO_ANALYZED` and leave hand-made labels alone.

go deeper

for a junior

Know that a failure's defect group can be set either by a person or by analysis, and that ReportPortal keeps a flag saying which of the two it was.

for a middle

Explain that the flag sits on the issue rather than the test item, that it is written in the same operation that applies the group, and that it carries no score.

for a senior

Show you would audit the split between machine-set and hand-set labels, and that you know a manual update can re-assert the flag because it is taken from the request body.

for a principal

Argue for provenance as a governance requirement: without it you cannot roll back a bad batch of machine labels, and you cannot report on how much triage the team actually did.

## Where the flag lives, and why that matters A ReportPortal test item that failed carries an **issue**, and it is the issue — not the item — that carries the triage state. Two booleans sit there beside the defect group: - **`autoAnalyzed`** — this group was set by analysis rather than by a person. - **`ignoreAnalyzer`** — this failure is to be kept out of analysis. Both are on `Issue` as it crosses the API and on `IssueEntity` in storage, backed by the columns `auto_analyzed` and `ignore_analyzer`. Candidates routinely look for them on the test item and cannot find them; the item has a status, the issue has the verdict and the provenance. That placement is deliberate. A passing item has no issue at all, so there is nothing to say about who labelled it. The moment a failure acquires a triage state, it acquires a record of where that state came from. ## What gets written when analysis wins When the analyzer returns a match for an item, the server does not patch a field in place. It rebuilds the issue: it resolves the returned locator to the project's issue type, sets the group, sets `autoAnalyzed` to true, and carries the existing `ignoreAnalyzer` value across unchanged. It then recomputes the project's defect counters, because a failure has just moved from one group into another. Two consequences fall straight out of that: 1. **The flag is set as part of applying the label, in the same write.** You never see a machine-set group without the flag. 2. **It is a rebuild, not a merge.** The values that survive are the ones explicitly carried across. ## Provenance is not confidence This is the distinction interviewers are usually fishing for. `autoAnalyzed` answers *who*, never *how sure*: | question | answered by `autoAnalyzed`? | |---|---| | Did a machine set this group? | yes | | How closely did the failure match? | no | | Which past failure was it matched against? | no | | Under what settings was the match made? | no | | Should I trust this label? | no | The scores, the settings and the matched candidate live on the suggestion record, not here. One bit cannot carry them, and reading the flag as a quality signal is how teams end up trusting machine labels more than they have earned. ## What keeping provenance buys you Because the two populations are separable, a re-analysis run can be aimed at one of them. `AnalyzeItemsMode` names which items an analyze run should include — `TO_INVESTIGATE`, `AUTO_ANALYZED`, `MANUALLY_ANALYZED` and `IGNORE_IMMEDIATE`, of which a request may ask for the first three. In practice: - **`TO_INVESTIGATE`** takes the undecided failures and tries to label them. Nothing already decided is touched. - **`AUTO_ANALYZED`** takes exactly the machine-labelled ones. Those items are pulled out of the analyzer's index and their issues reset before the run, so a bad batch of machine labels can be redone from scratch. - **`MANUALLY_ANALYZED`** does the equivalent for the hand-set ones — which is why nobody runs it casually. Without the provenance bit none of that is expressible. You would be able to say "re-analyse this launch" and nothing finer, and every re-run would sweep away work people did by hand. ## The trap: the flag is not cleared for you When a person edits an issue through the update endpoint, the value written for `autoAnalyzed` comes from the **request body**. It is not recomputed and not automatically cleared. A client that reads an issue, changes the group, and sends the whole object back will happily re-assert `autoAnalyzed: true` on a label a human just chose by hand. The practical effects are worth stating plainly: - Your two populations can silently blur, so the `AUTO_ANALYZED` re-run stops meaning what you think it means. - A dashboard split by provenance drifts without anything logging an error. - The fix is on the writing side: whoever edits an issue is responsible for sending the flag that is now true. The item does separately record which user made the change, and the activity trail keeps the before and after, so the truth is recoverable — but not from this boolean alone. ## How to answer it well Say what it is (provenance), where it is (on the issue, not the item), what it is not (a confidence), and what it enables (targeted re-analysis by mode). If you have operated ReportPortal, add the update-endpoint trap — it is the kind of detail that only shows up once you have watched a well-meaning integration flip the flag on a whole project.

  • How would you find which failures in a launch were labelled by analysis rather than by a person?
    Filter on the issue's `autoAnalyzed` value — it is a queryable field, not a derived one. The same split is what an analyze run uses when you ask it for `AnalyzeItemsMode` `AUTO_ANALYZED`, which collects exactly the machine-labelled items. If the two populations look implausible, suspect a client that echoes the flag back on manual edits.
  • Does `autoAnalyzed` tell you which analyzer produced the label, or when?
    No. It is one bit: a machine did it. Anything finer — which analyzer instance, against which past failure, under which settings, at what score — has to come from the activity record or the suggestion record. Treating the boolean as an audit trail on its own leaves you unable to explain any individual label.

saying these in an interview costs you the question

  • Reading autoAnalyzed as a confidence score
  • Looking for the flag on the test item, not the issue
  • Assuming a manual edit clears the flag automatically
  • Believing the analyzer created a new defect group
  • Treating a machine-set label as a finished decision