skip to content

In ReportPortal, a proposed defect group comes back as a `SuggestInfo` record carrying `matchScore`, `esScore`, `resultPosition`, `esPosition`, `usedLogLines`, `minShouldMatch` and `userChoice`. What is that record for?

level: seniorimportance: nice to knowfreq 30%

answer

  1. the proposal carries its own receipt
  2. two rankers, two scores kept apart
  3. where the candidate placed, not how likely
  4. the settings it actually ran under
  5. and what the reviewer chose

basics

~20 s

A SuggestInfo record is the receipt for one proposal: two ranking scores and the positions they gave the candidate, the settings the match ran under, how long it took, which model produced it, and what the person eventually chose.

solid answer

~40 s

A proposal is never just a defect group — it arrives as a `SuggestInfo` carrying the evidence and the conditions behind it. Two rankers score the same candidate, so the record keeps `matchScore` and `esScore` together with `resultPosition` and `esPosition`, the places each ranking gave it. It also keeps the settings actually in force (`minShouldMatch`, `usedLogLines`), the cost (`processedTime`), the model's own identification (`modelInfo`, `modelFeatureNames`, `modelFeatureValues`) and `clusterId`. `userChoice` closes the loop: the surface returns each proposal wrapped as a `SuggestedItem` alongside the past test item and its error logs so a person can see the evidence, and `handleSuggestChoice(...)` sends the decision back. Accepted or rejected, that choice is the only thing that teaches anything.

code

json · 14 lines
json
{
  "matchScore": 82.0,
  "resultPosition": 0,
  "esScore": 41.5,
  "esPosition": 2,
  "usedLogLines": 5,
  "minShouldMatch": 95,
  "processedTime": 1.42,
  "userChoice": 0,
  "clusterId": 4212,
  "modelInfo": "...",
  "modelFeatureNames": "...",
  "modelFeatureValues": "..."
}

go deeper

for a junior

Know that a machine proposal arrives with more than a defect group attached, including scores and the past failure the proposal was drawn from.

for a middle

Explain why the settings in force are echoed back with the proposal, so a label can later be read against the configuration that produced it.

for a senior

Argue that the accept rate, not the number of labels applied, is the metric that says whether the suggestion surface is any good.

for a principal

Decide what evidence you require before a proposal may be applied at all, and make sure the workflow captures declines rather than letting people route around the surface.

## A proposal that shows its working The interesting design decision in ReportPortal's suggestion surface is that a proposal is not a bare answer. `searchSuggests(...)` returns `SuggestInfo` records, and each one carries the reasoning conditions around the proposal rather than only the proposed defect group. The service then builds a `SuggestedItem` around each: the **past test item** the suggestion came from, that item's error-level logs, and the `SuggestInfo` itself. A reviewer is shown the earlier failure and its evidence, not just a verdict to rubber-stamp. That framing is the whole point of the leaf. The machine is not classifying a failure from first principles; it is saying *this looks like that one, which you already judged*. Showing "that one" is what makes the proposal reviewable. ## What the record keeps, and why | field | what it holds | |---|---| | `matchScore`, `esScore` | two scores for the same candidate, from two rankers | | `resultPosition`, `esPosition` | where the candidate placed in each of those rankings | | `minShouldMatch`, `usedLogLines` | the settings actually in force for this match | | `processedTime` | what the answer cost to produce | | `modelInfo`, `modelFeatureNames`, `modelFeatureValues` | which model answered and what it was given | | `clusterId` | the failure grouping the suggestion is tied to | | `userChoice` | what the person did with the proposal | Three things fall out of that list: 1. **Two scores, deliberately.** Keeping a search score and a model score side by side, each with its own position, lets you see whether the second stage reordered the first stage's candidates and whether that reordering helped. One blended number would hide it. 2. **The settings travel with the answer.** Because `minShouldMatch` and `usedLogLines` are echoed back, a proposal from six weeks ago can be read against the configuration that produced it rather than against today's. 3. **A position is not a probability.** `resultPosition` says where a candidate ranked among the others, which is a very different claim from how likely it is to be right. A rank of first in a field of bad candidates is still a bad candidate. ## The feedback channel `userChoice` is the field that makes the surface a loop rather than a one-way suggestion. When a reviewer accepts or declines a proposal, `handleSuggestChoice(...)` sends that decision back to the machine-learning side. Nothing else in the flow carries that signal. The practical consequences are worth stating: - **Rejections are as valuable as acceptances.** A proposal quietly ignored teaches nothing; a proposal explicitly declined does. A workflow where people bypass the suggestion surface and edit the group directly starves the loop. - **The accept rate is the honest quality metric.** How many failures got labelled tells you about throughput; what share of proposals a person kept tells you whether the proposals were any good. - **The loop is also the trap.** Accepted labels become evidence for the next similar failure, so an accepted mistake propagates. Provenance and per-issue exclusion exist because of exactly this. ## What the record is not Be careful about over-reading it, because this is where candidates lose the plot: - It is **not an explanation of the model's reasoning**. `modelFeatureNames` and `modelFeatureValues` identify what the model was given; interpreting a model's behaviour from feature values is a different discipline entirely and is not what this record is for. - It is **not a calibrated confidence**. The scores are comparable within a ranking, not across projects, models or time. - It is **not the audit trail for the label**. If a proposal is applied, the provenance flag on the issue is what records that a machine set it. The suggestion record explains the proposal, not the state of the item afterwards. ## Where it comes up in an interview This is differentiator material rather than a screening question, and the way to use it is to make one sharp point: the tool treats a suggestion as a claim that must show its evidence and record its outcome. If you can say why two scores and two positions are kept rather than one blended number, and why a declined proposal matters as much as an accepted one, you have shown you understand the surface as a review workflow rather than an autocomplete.

  • Why keep a search score and a model score separately instead of one blended number?
    Because they answer different questions and disagree usefully. Keeping both, each with the position it gave the candidate, shows whether the second stage reordered the first stage's shortlist and whether that reordering improved anything. A single blended figure hides the disagreement, which is exactly the signal you would want when proposals start going wrong.
  • What happens to `userChoice` after a reviewer decides?
    It is sent back through `handleSuggestChoice(...)` to the machine-learning side. That is the only channel by which accepting or declining a proposal teaches anything, so a team that bypasses the suggestion surface and edits the defect group directly gets suggestions that never improve, however much triage they do.

saying these in an interview costs you the question

  • Reading matchScore as a probability of correctness
  • Treating feature values as an explanation of the choice
  • Comparing scores across projects or across models
  • Ignoring declined proposals as wasted signal
  • Assuming a proposal is applied without a choice