When you hand on a change-impact set, what does the confidence you attach to it come from?

seniornice to knowfreq 26%

answer

An assertion versus evidence somebody can weigh
How much came from links, not reading
Freshness of the links in this area
State a band with its reason attached
It exists for whoever receives the set

basics

~20 s

Confidence comes from observable properties of the walk: how recently the links in that area were maintained, how much of the set came from links rather than your own reading, and whether earlier sets caught what actually broke.

solid answer

~50 s

A confidence statement turns an impact set from an assertion into evidence somebody can weigh. Derive it from four observable things: how recently the traceability links in this area were touched, what share of the set came from explicit links rather than from your own reading, how much arrived only through shared components, and whether past escapes here fell inside or outside the sets you produced before. Express it as a band with its reason — "medium: criteria links are current, but two of the four contributing teams keep none" — never as a bare percentage, which looks derived, cannot be reproduced, and invites arithmetic downstream. Its purpose is for the receiver: it tells whoever acts on the set how much margin to add, it names a specific repair, and it makes the analysis falsifiable, because a stated band can be checked against what later escaped.

A confidence statement turns an impact set from an assertion into evidence somebody can weigh. Derive it from four observable things: **how recently the traceability links in this area were touched**, **what share of the set came from explicit links rather than from your own reading**, **how much arrived only through shared components**, and **whether past escapes here fell inside or outside the sets you produced before**. Express it as a band with its reason — *"medium: criteria links are current, but two of the four contributing teams keep none"* — never as a bare percentage, which looks derived, cannot be reproduced, and invites arithmetic downstream. Its purpose is for the receiver: it tells whoever acts on the set how much margin to add, it names a specific repair, and it makes the analysis falsifiable, because a stated band can be checked against what later escaped.

code

pseudocode · 14 lines

pseudocode

linkedItems    = count(items where source == "link")
inspectedItems = count(items where source == "inspection")
hubItems       = count(items where source == "hub")
staleHere      = lastLinkUpdate(area) older than two releases
missedBefore   = escapes(area) that fell outside previous impact sets

band = judge(
    derivedShare = linkedItems / (linkedItems + inspectedItems),
    hubShare     = hubItems / totalItems,
    freshness    = staleHere,
    trackRecord  = missedBefore
)

report(impactSet, confidenceBand = band, reason = sentenceThatSetTheBand)

go deeper

for a junior

Know that an impact set is a judgement with a quality attached rather than a fact. If the links behind it are old or partial, whoever receives it has to be told so before they act on it.

for a middle

Be able to name what the confidence is derived from: how recently the links in that area were maintained, the share of items that came from a link rather than from reading, and how much of the set arrived only through shared components.

for a senior

Show that you state a band with its reason and then check the band afterwards against what actually escaped. An interviewer wants the confidence used to set margin downstream, not deployed as a disclaimer that protects the author.

for a principal

Own what confidence reporting does to behaviour. Bands that are always medium inform nobody; bands that get the author blamed push everyone to state high. Decide what happens when a low band arrives and make that response routine, or people quietly stop stating one.

From assertion to evidence

An impact set handed over bare is an assertion: these are the things the change might have disturbed. The receiver has no way to weigh it. Was it built from a record maintained every week, or from links last touched before a reorganisation? Did most items arrive through explicit connections, or did the analyst reconstruct half of them by reading the requirement? A stated confidence answers that, and it is the whole difference between evidence somebody can act on and an opinion they have to take on trust.

The confidence is not a report of how sure you feel. It is a summary of observable properties of the walk you just did.

What the confidence is derived from

Input	What it tells you	Where it comes from
Freshness of links here	Whether the record still follows the product	Last update against recent release history
Derived share	How much of the set the record itself supports	Items from links over all items
Hub share	How much arrived only through common ground	Items reached solely via shared components
Contributor coverage	Whether every team feeding this area keeps links	Whose work appears in the record at all
Past escapes	Whether earlier sets contained what actually broke	Escapes inside versus outside previous sets

The last row is what makes the whole practice honest. If the sets you produced for the last four changes did not contain the thing that later broke, your confidence for this area is low however tidy the record looks. That is a measurement rather than an impression, and it is the only one available.

Say a band with a reason, not a number

Express the result as a band — low, medium, high — together with the sentence that produced it:

"High: criteria and case links here were touched this release, and every item came from a link."
"Medium: links are current, but two of the four contributing teams keep none, so their work cannot appear."
"Low: nothing here has been linked since the ownership change, and I reconstructed most of the set by reading."

A percentage is worse than it looks. It appears derived when it is not — nobody can reproduce eighty-two percent — and precise-looking figures get combined with other figures downstream as though they had been measured. A band cannot be arithmetic-ed and can be argued with, which is exactly what you want: somebody who knows the area may read your reason and tell you it is wrong.

Three further disciplines keep bands meaningful:

Attach the band to the set, not to the covering note. The set is what gets forwarded; the note is what gets dropped.
Never let a low band turn into a wider set. Widening to feel safe hides the gap instead of reporting it. State the band and name the blind spot.
Make the band falsifiable. Write down what would have to happen for it to have been wrong, so that it can be checked afterwards.

What it is for once the set is handed on

The confidence exists for the receiver, not for the author, and it has three concrete uses downstream.

Setting margin. Whoever decides depth and schedule can add deliberate slack in proportion to the band — a wider look at the area, a second pair of eyes, a slower rollout — rather than guessing how far to trust an unlabelled list.

Directing repair. A low band with a stated reason is a work item: this area carries no links from two of its contributors. That is far more actionable than a general complaint that traceability is poor, because it names the specific hole and the specific people.

Making the analysis reviewable. A confidence you stated can be checked against what actually escaped. Over a few releases that turns change-impact analysis from a ritual into a practice with a track record, and it is what lets a high band mean something the next time you state one.

There is one failure mode worth naming in advance. If a low band is received as the author's fault, everybody reports high and the signal dies inside two releases. The response to a low band has to be routine and blameless — normally a small, funded repair to the record plus extra margin this time round — or nobody states one honestly again. That is a decision about how the team behaves rather than about the analysis, but it decides whether the analysis is worth doing at all.

## From assertion to evidence An impact set handed over bare is an assertion: these are the things the change might have disturbed. The receiver has no way to weigh it. Was it built from a record maintained every week, or from links last touched before a reorganisation? Did most items arrive through explicit connections, or did the analyst reconstruct half of them by reading the requirement? A stated **confidence** answers that, and it is the whole difference between evidence somebody can act on and an opinion they have to take on trust. The confidence is not a report of how sure you feel. It is a summary of observable properties of the walk you just did. ## What the confidence is derived from | Input | What it tells you | Where it comes from | | --- | --- | --- | | Freshness of links here | Whether the record still follows the product | Last update against recent release history | | Derived share | How much of the set the record itself supports | Items from links over all items | | Hub share | How much arrived only through common ground | Items reached solely via shared components | | Contributor coverage | Whether every team feeding this area keeps links | Whose work appears in the record at all | | Past escapes | Whether earlier sets contained what actually broke | Escapes inside versus outside previous sets | The last row is what makes the whole practice honest. If the sets you produced for the last four changes did not contain the thing that later broke, your confidence for this area is low however tidy the record looks. That is a measurement rather than an impression, and it is the only one available. ## Say a band with a reason, not a number Express the result as a **band** — low, medium, high — together with the sentence that produced it: - *"High: criteria and case links here were touched this release, and every item came from a link."* - *"Medium: links are current, but two of the four contributing teams keep none, so their work cannot appear."* - *"Low: nothing here has been linked since the ownership change, and I reconstructed most of the set by reading."* A percentage is worse than it looks. It appears derived when it is not — nobody can reproduce eighty-two percent — and precise-looking figures get combined with other figures downstream as though they had been measured. A band cannot be arithmetic-ed and can be argued with, which is exactly what you want: somebody who knows the area may read your reason and tell you it is wrong. Three further disciplines keep bands meaningful: 1. **Attach the band to the set, not to the covering note.** The set is what gets forwarded; the note is what gets dropped. 2. **Never let a low band turn into a wider set.** Widening to feel safe hides the gap instead of reporting it. State the band and name the blind spot. 3. **Make the band falsifiable.** Write down what would have to happen for it to have been wrong, so that it can be checked afterwards. ## What it is for once the set is handed on The confidence exists for the receiver, not for the author, and it has three concrete uses downstream. **Setting margin.** Whoever decides depth and schedule can add deliberate slack in proportion to the band — a wider look at the area, a second pair of eyes, a slower rollout — rather than guessing how far to trust an unlabelled list. **Directing repair.** A low band with a stated reason is a work item: *this area carries no links from two of its contributors.* That is far more actionable than a general complaint that traceability is poor, because it names the specific hole and the specific people. **Making the analysis reviewable.** A confidence you stated can be checked against what actually escaped. Over a few releases that turns change-impact analysis from a ritual into a practice with a track record, and it is what lets a high band mean something the next time you state one. There is one failure mode worth naming in advance. If a low band is received as the author's fault, everybody reports high and the signal dies inside two releases. The response to a low band has to be routine and blameless — normally a small, funded repair to the record plus extra margin this time round — or nobody states one honestly again. That is a decision about how the team behaves rather than about the analysis, but it decides whether the analysis is worth doing at all.

Why is a percentage a worse way to state this confidence than a band with a reason?
A percentage looks derived and is not: nobody can reproduce eighty-two percent, and the receiver will combine it with other numbers as though it had been measured. A band plus the sentence that set it carries the same claim, cannot be arithmetic-ed, and can be argued with by somebody who knows the area — which is the point of stating it.
What would make you lower the confidence on an impact set that looks complete?
A large share of items added by your own reading rather than by a link, an area whose links have not been touched since a reorganisation, contributing teams that keep no links at all, or a previous escape in that area that fell outside the set you produced last time. The last one outweighs the others, because it is a measurement.

Like a forecast that says medium confidence because one weather station's readings are missing: the reason, not the number, is what tells you whether to carry a coat.

saying these in an interview costs you the question

States a bare percentage with no derivation behind it
Treats every impact set as equally trustworthy
Bases confidence on how long the analysis took
Never checks a stated band against what later escaped
Answers a low band by widening the set rather than naming the gap

When you hand on a change-impact set, what does the confidence you attach to it come from?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

From assertion to evidence

What the confidence is derived from

Say a band with a reason, not a number

What it is for once the set is handed on

▾common follow-ups2 questions

▾analogy~10 s

▾red flags5 flags

From assertion to evidence

What the confidence is derived from

Say a band with a reason, not a number

What it is for once the set is handed on

answer

go deeper