skip to content

A Java test method covered by an Allure report is renamed and moved into another class. What happens to that test's trend in the next generated report, and why is there no error telling you?

level: seniorimportance: should knowfreq 45%

answer

  1. identity follows the code, not the title
  2. the lookup misses in silence
  3. a rename looks like a new test
  4. the old series is orphaned, not deleted
  5. diff fullName, not the digests

basics

~20 s

Renaming or moving the method changes testCaseId, so historyId changes too and the result no longer matches its stored series. The report shows a new test with one point. Nothing errors: an unmatched key looks like a new test.

solid answer

~50 s

Adapters derive `testCaseId` from where the test sits in the code - the JUnit Platform adapter from the JUnit Platform identifier with invocation segments removed, the TestNG adapter from class plus method name - so a rename or a move produces a different value, and `historyId`, derived from it, moves with it. The stored series is looked up by that key, so the lookup misses: the test appears brand new with a one-point trend, and the old series sits on disk unreferenced. No error is possible, because a renamed test and a genuinely new test arrive in exactly the same shape - a key with nothing stored behind it. Confirm it by diffing `historyId`, `testCaseId` and `fullName` across two runs rather than reading the digests. The durable fix is to assign `historyId` explicitly for the tests whose trend anyone reads.

code

bash · 4 lines
bash
jq -r '[.testCaseId, .historyId, .fullName] | @tsv' \
  allure-results/*-result.json | sort > ids-current.tsv

diff ids-previous.tsv ids-current.tsv

go deeper

for a junior

Know that a test's identity follows its class and method rather than its title, so refactoring can move it while the test itself is unchanged.

for a middle

Explain the chain: the adapter derives case identity from code coordinates, the series key derives from that, and a changed key finds no stored series to extend.

for a senior

Be ready to diagnose this in a live report - keep the per-run identity list as an artefact, diff it, and separate a rename from a volatile parameter before proposing a fix.

for a principal

Decide when a trend is worth protecting from refactoring at all, and set the policy for which tests get an assigned key and who maintains it.

## Why the trend restarts An Allure adapter derives `testCaseId` from where the test sits in the code: the JUnit Platform adapter from the identifier the JUnit Platform assigns each test, with the parameterised-invocation segments removed, the TestNG adapter from the class name plus the method name. Rename the method, move it to another class, or move the class to another package, and that value is different. `historyId` is derived from `testCaseId`, so it is different too. The report matches this run's result against the stored series by that key. A key that has changed does not match anything, so: - the renamed test appears as a test the report has never seen, with a trend one point long; - the series stored under the old key is still on disk, but nothing will ever look it up again; - everything the report derives from a previous series -- the trend chart, any comparison against the last outcome -- has nothing to work from for that test. ## Why nothing errors This is the part interviewers are really probing. **A renamed test and a genuinely new test are byte-for-byte indistinguishable to the report.** Both arrive as a result carrying a key with no stored series behind it. There is no rename event, no forwarding pointer, and no signal in the file that says "this used to be called something else". A report generator that treated an unmatched key as an error would fail on every legitimately new test, so it cannot. The damage is therefore silent by construction, and it is silent in the direction that hurts: nothing turns red, the report still generates, and the loss shows up weeks later as "why does this test have no history". ## How to confirm it Do not try to read the digests. Compare the *inputs* between the last known-good run and the current one: 1. Extract `historyId`, `testCaseId` and `fullName` from every `*-result.json` in the results directory, and keep that list as a build artefact. 2. Diff this run's list against the previous run's. A rename shows as one line disappearing and one appearing, with a recognisable `fullName` on the new side. 3. Sanity-check the count of one-point series. A step change in that count between two builds, with no corresponding batch of new tests, is the signature of a refactor that moved identity. `fullName` is the useful column here precisely because it is human-readable, where `historyId` is not. ## What to do about it There is no way to make a report re-link the two series after the fact from the results alone, so the choice is made before or around the rename: - **Accept the reset.** Often the right call. If the trend is short, or the rename is part of a rewrite that changes what the test asserts anyway, the old series was not evidence about the new test. - **Pin the key.** Assign `historyId` explicitly rather than letting it be derived, anchoring it to something that survives refactoring -- an external case identifier, or a stable identifier stored on the test itself. The writer only derives the key when it is missing, so an assigned value is honoured. This is the durable fix, and it is worth doing for the small set of tests whose trend anyone actually reads. - **Batch the churn.** If a large package move is coming, do it in one change, and note in the release record that the trends restart on that build. A documented reset is far cheaper than a mystery. ## The wider point | identity derived from | survives a rename? | cost | |---|---|---| | code coordinates (class, method, signature) | no | zero setup; free by default | | an explicitly assigned key | yes | someone must assign and maintain it | | the display name | no, and it breaks more easily | worst of both | Deriving identity from code coordinates is the sensible default -- it needs no bookkeeping and it is correct as long as the code does not move. The price is that refactoring, an activity nobody wants to discourage, quietly resets history. Knowing which of your tests have a trend worth protecting, and pinning only those, is the judgement being tested.

  • Can you re-link the two series after the rename has already shipped?
    Not from the results alone - nothing in the written file records that the test used to be called something else, so there is no forwarding pointer to follow. The realistic options are to accept the reset, or to assign that test an explicit `historyId` from now on so the next refactor cannot do it again.
  • Which tests are actually worth pinning an explicit key to?
    The ones whose trend someone reads: long-lived acceptance or contract tests, and anything a release decision leans on. Pinning has a real cost, because the key must then be assigned and maintained, so applying it to an entire suite usually buys less than it costs.

saying these in an interview costs you the question

  • Expects the report to warn that an identity disappeared
  • Thinks the stored series is matched by display name
  • Believes the old and new series can be merged afterwards
  • Assumes editing a test body also resets its trend
  • Blames the history copy step for a rename-caused reset