skip to content

A recurring failure was tied to a tracker item, the ticket has since been closed, and the failure is back on this run. What does ReportPortal's stored link, and an Allure 3 `resolutions` rule, each still do to that failure?

level: seniorimportance: should knowfreq 45%

answer

  1. nothing polls the tracker
  2. no status field, no expiry field
  3. the rule re-matches, it does not re-check
  4. muted and accepted hide it from the gate
  5. a stale link copies itself forward

basics

~20 s

Both still treat the failure as the known one. ReportPortal's stored link has no status field, and an Allure 3 resolutions rule re-matches on text or id alone. Neither consults the tracker, so closing the ticket changes nothing.

solid answer

~40 s

Neither artefact knows the ticket closed. ReportPortal's `ExternalSystemIssue` has six fields - `ticketId`, `submitDate`, `btsUrl`, `btsProject`, `url`, `pluginName` - and not one is a state, so the returning failure is labelled and linked exactly as before. An Allure 3 `resolutions` rule is re-evaluated against this run's own results: if the message still matches `messageRegexp` or the identity is still in `testCaseId`, it fires again. What that costs you depends on the category. A `resolution` of `issue` annotates but still counts the failure; `muted` and `accepted` mark it as ignored and it is filtered out before the quality gate's rules ever see it. So a stale `muted` rule keeps a genuinely new regression out of the build's verdict while the report explains, confidently, why it was fine to ignore.

go deeper

for a junior

Know that neither a stored link nor a report rule contacts the tracker. Closing a ticket changes nothing in the results tool, so a labelled failure stays labelled until a person changes it.

for a middle

Explain the mechanics: the stored link carries six fields and no state, and a rule is re-evaluated against this run's failed and broken results with no expiry and no tracker lookup anywhere in the path.

for a senior

Demonstrate the production consequence - a stale muted or accepted rule keeps a new regression out of the verdict, and messageRegexp searching rather than matching whole strings means the rule widens over time.

for a principal

Own the review loop the products do not provide: a scheduled outbound sweep over the ticket keys in use, and a convention that any suppression without an owner and a review date fails review.

## Two artefacts, one blind spot Both mechanisms in question - ReportPortal's stored ticket link and an Allure 3 `resolutions` rule - answer "does this failure already have a ticket?" without ever asking the tracker. Neither is wired to the tracker's lifecycle. Closing the ticket is an event in a system that has no channel back into either artefact, so the honest answer to "what changed when the ticket closed?" is **nothing at all**. ## ReportPortal: six fields, none of them a state A stored link is an `ExternalSystemIssue` with exactly six fields - `ticketId`, `submitDate`, `btsUrl`, `btsProject`, `url` and `pluginName`. There is no status, no resolution, no close date. The server can fetch an item's current summary and state through the configured plugin when somebody opens the screen, but that is a live read whose answer is never written back into the stored entry. So on the run where the failure returns, the failure is labelled exactly as it was: same defect type, same link, same `submitDate` from months ago. There is a second-order effect that makes this worse rather than merely stale. When ReportPortal reuses an earlier verdict for a newly arrived failure - the mechanism that spares a human from re-triaging the same red every run - it copies the ticket set from that earlier decision onto the new failure's issue wholesale. One stale link therefore does not sit still; it reproduces onto each new occurrence on its own, and each new occurrence then looks freshly, confidently linked. ## Allure 3: the rule re-matches, it does not re-check An Allure 3 `resolutions` rule is a matcher plus a category. On every run the rule is evaluated against that run's own failed and broken results, and if the message still matches `messageRegexp`, or the identity is still in `testCaseId`, the rule fires again. Nothing in that evaluation consults a tracker, and there is no expiry field on a rule. What the rule then does depends on its category, and this is the distinction that decides whether the returning failure stays visible: | resolution | effect on the returning failure | |---|---| | `issue` | attaches the tracker link; the failure still counts as a failure | | `muted` | the failure is treated as ignored and dropped from the gate's view | | `accepted` | the failure is treated as ignored and dropped from the gate's view | That table is the practical answer. A failure resolved as `issue` is annotated but not hidden - it still lands in the failure counts a quality gate evaluates. A failure resolved as `muted` or `accepted` is filtered out before those rules ever see it. So a stale `muted` rule from a ticket closed months ago will hold a genuinely new regression out of the build's verdict, and the run comes back green while the report shows the test red with a comment explaining why it was fine to ignore. ## Why nobody notices Three properties conspire: - **The suppression is silent by design.** Its whole purpose was to stop a known failure being noisy, so it produces no output when it works. - **The matchers are broader than intended.** `messageRegexp` is a search rather than a whole-string match, so a rule written for one failure absorbs any later failure whose message merely contains the phrase. A new bug with a similar message inherits the old ticket's suppression. - **A rule that stops matching is equally silent.** When the failure is genuinely fixed, the rule simply matches nothing. Nothing warns that a rule has gone idle, so a config accumulates dead rules that are indistinguishable from live ones by inspection. ## What actually helps Neither product will tell you; the check has to be one you build and run on a schedule of your own: 1. **Enumerate outward, not inward.** Collect the distinct `ticketId` values currently attached across the project, ask the tracker for their states in one query, and report the ones already closed. `btsUrl`, `btsProject` and `pluginName` on each link are what let you route that query correctly. 2. **Diff the written baseline.** In Allure 3, `known-issues.json` records which failures were sitting under which issue id on a given run. Diffing it between runs shows a ticket quietly acquiring failures it was never about. 3. **Make suppressions expire, not persist.** Both `muted` and `accepted` require a `comment`; put a date and an owner in it, and treat an undated suppression as a defect at review time. A rule with a review date is the only version of this that degrades safely, because the artefact itself will never tell you it has gone stale.

  • Does a failure resolved as `issue` in Allure 3 still count against the build?
    Yes. Only `muted` and `accepted` mark a failure as ignored, and those are the ones filtered out before quality gate rules are evaluated. An `issue` resolution attaches the tracker link and leaves the failure in the counts, which is why it is the safe choice when you want visibility rather than silence.
  • How does one stale link in ReportPortal end up on failures nobody linked?
    When the server reuses an earlier verdict for a newly arrived failure, it copies that earlier issue's whole ticket set onto the new failure. So the link reproduces onto each new occurrence without anyone acting, and every occurrence then looks freshly and confidently linked.
  • Why does a rule that has stopped matching not show up as a problem?
    Because it produces no output either way. Only failed and broken results are handed to the rules, so once the failure is genuinely fixed the rule simply matches nothing. Nothing reports an idle rule, which makes a dead rule indistinguishable from a live one by reading the config.

A stored ticket link is a sticky note on a machine reading "known fault, ticket raised". Closing the ticket does not make the note fall off - somebody has to walk over and take it down.

saying these in an interview costs you the question

  • Assuming the tool re-checks the ticket's state
  • Believing an issue resolution hides the failure from the gate
  • Thinking a stale rule announces itself when it stops matching
  • Treating a linked failure as one already being worked on
  • Expecting the tracker to push a close event back into results