An assessment reads "likely, with low confidence" — is that a contradiction?
answer
- two statements, not one
- one about the world, one about evidence
- an ordered likelihood ladder
- unlikely plus low confidence misleads
- may and could sit nowhere on the ladder
basics
~20 sNo. Likelihood is how probable the analyst thinks the claim is; confidence is how good the sourcing and reasoning behind that estimate are. Likely with low confidence means the estimate leans yes but little new information would move it.
solid answer
~50 sThey are two separate statements and both are needed. **Likelihood** places the claim on an ordered ladder, from almost no chance through unlikely, roughly even chance, likely, very likely to almost certain, with `likely` mapping to roughly 55 to 80 percent. **Confidence** describes the basis for that estimate: how many independent sources, how much first-hand observation, how much weight the assumptions are carrying, whether the reasoning has been tested. So "likely, with low confidence" is coherent and common: my estimate leans yes, and one new piece of evidence could move it a long way. The error to avoid is translating low confidence into low probability, which readers do constantly, and the other error is hedging with words like may, could or possible, which sit nowhere on the ladder and let each reader supply their own number.
go deeper
Be able to say that likelihood and confidence answer different questions, one about the claim and one about the evidence behind it, and that low confidence does not mean probably false.
Explain what drives confidence, source independence, first-hand observation, assumption load and testing, and place terms on the likelihood ladder consistently rather than reaching for may or possible.
Demonstrate the judgement in writing: pick the ladder term you can defend, attach the confidence with a one-clause reason, and flag the unlikely-with-low-confidence case so nobody reads it as reassurance.
Own the house style: one ladder used by every analyst, a rule against confidence expressed as a number, and a mechanism that catches assessments published without a stated basis before a decision is made on them.
## Two axes that get collapsed into one Estimative language exists because natural English is hopeless at conveying uncertainty. "There is a serious possibility of a breach" has been shown, in study after study, to mean anything from ten to eighty percent depending on the reader. The discipline's answer is to fix two separate dimensions and to state both. **Likelihood** — how probable is the thing? The intelligence-community analytic standards (ICD 203) publish an ordered ladder of terms with approximate probability bands attached, running: almost no chance, very unlikely, unlikely, roughly even chance, likely (equivalently, probable), very likely, almost certain. "Roughly even chance" sits near fifty percent; "likely" occupies roughly the 55 to 80 percent range. The point of the ladder is not the numbers but that everyone in the organisation reads the same term the same way. **Confidence** — how good is the basis for that estimate? Confidence is a judgement about the evidence and the reasoning, not about the world. It typically rests on: - the **number and independence** of sources, and how much of it is first-hand observation rather than repetition; - the **reliability and credibility grades** of what you hold; - how much load the **assumptions** are bearing, and whether any of them are untested; - whether the reasoning has been **challenged** by someone who tried to break it; - whether the situation is **stable** enough that the evidence you have still applies. The standards are explicit that confidence must not be expressed as a degree of likelihood. "High confidence" does not mean "probably true", and "low confidence" does not mean "probably false". ## Reading the four corners - **Likely, high confidence** — several independent first-hand sources point the same way. Act on it. - **Likely, low confidence** — one uncorroborated report plus a plausible inference. Your best estimate leans yes; expect it to move, and say what would move it. - **Unlikely, high confidence** — you have looked properly, in the right places, and found nothing consistent with the claim. This is a strong statement and it is the one executives most need. - **Unlikely, low confidence** — you think not, but you barely looked or could not look. This is the corner that gets silently reported as "we are fine", and it is the most dangerous sentence in the discipline. That last pair is exactly why the two axes cannot be merged: they read identically to a hurried reader and mean completely different things about what work has been done. ## The words that do no work May, might, could, possible, potentially, cannot be ruled out. Each is technically true of almost anything and carries no position at all. They feel safe to write because they cannot be wrong, and that is precisely the objection: a judgement that cannot be wrong cannot inform a decision. If a claim is genuinely at the bottom of the ladder, the honest phrasing is "we assess this is very unlikely", with low confidence attached if the basis is thin. Reserve "we do not know" for the case where you truly have no basis at all, and pair it with what you are doing to get one. ## Two more rules that keep it usable **Do not mix registers.** Either use the ladder terms consistently or state numeric ranges, but do not let one paragraph say "likely" and the next say "about 70 percent" as though they were different findings. And never attach a percentage to confidence, which is not a probability at all. **Say what would change it.** A likelihood and a confidence together still leave the reader stuck if the assessment cannot be moved by evidence. Naming the specific observation that would raise or overturn the judgement turns it from an opinion into something falsifiable, and it doubles as your collection plan. ## Why interviewers ask this Because the conflation is near-universal and it silently corrupts decisions. A candidate who says "low confidence means we think it probably did not happen" has just told an executive to stand down on the basis of a statement about the analyst's own sourcing.
- Which of the four likelihood-confidence combinations most often misleads a reader?Unlikely with low confidence. It reads to a hurried executive as reassurance, when it actually says the team either could not look or barely looked. It must be written so the gap is unmissable, for example: we assess this is unlikely, at low confidence, because the sign-in logs covering the claimed period fell outside retention. That sentence names the missing evidence rather than hiding it.
- Why do the analytic standards discourage words like may and could?Because they express no position. They are true of nearly any claim, so the reader supplies their own probability and two readers act differently on the same sentence. They also protect the writer from ever being wrong, which is the opposite of what an estimate is for. If the honest answer is the bottom of the ladder, write very unlikely and attach the confidence.
- Should confidence ever be given as a percentage?No. Confidence is a judgement about the quality of sourcing and reasoning, not a probability, so a number gives it false precision and invites readers to multiply it against the likelihood. State it as low, moderate or high and, in one clause, say why: single uncorroborated source, untested assumption, or several independent first-hand observations.
A weather forecast gives you a chance of rain and, separately, how much data the model had. Seventy percent from one broken sensor and seventy percent from a full network are the same forecast on very different footing.
saying these in an interview costs you the question
- Reads low confidence as low probability
- Gives confidence as a percentage
- Hedges with may, could or cannot be ruled out
- Mixes ladder terms and percentages in one product
- Reports unlikely and low confidence as all clear
- Thinks high confidence means the judgement is proven