skip to content

What may a case honestly assert when the correlation trail it checks covers only the hops that emit records?

level: seniorimportance: nice to knowfreq 22%

answer

  1. Absence of a record is not absence
  2. A green trail covers only what records
  3. Name the segment the case actually proves
  4. Check an unrecorded step by its effect
  5. Partly kept records cannot carry a verdict

basics

~20 s

Only that the recorded hops were reached. Anything unrecorded is unknown, not proven absent and not proven correct. Scope the claim to the covered segment, verify uncovered steps by their effects, and say plainly which parts stay unverified.

solid answer

~50 s

A trail-based verdict can only see where records exist, so its pass covers the instrumented segment and nothing else. Two honesty rules follow. First, **absence of a record is not absence of work**: a missing hop means either the step did not run or the step records nothing, and a case that cannot tell them apart must not report one as the other. Second, **a green trail is not end-to-end coverage**: the case proves the recorded steps were reached under its own correlation value, not that the whole flow behaved. State the limit where a reader will see it: scope the assertion message and case name to the covered segment, verify an unrecorded step through its own effect where one is observable, and leave it explicitly unverified where it is not. Where records are only partially kept, the trail is a diagnostic aid, not evidence a verdict may rest on.

code

pseudocode · 12 lines
pseudocode
covered   = { "accepted", "priced", "settled" }   # these steps record
uncovered = { "archive_write" }                   # this one records nothing

seen = record_store.hops_for(correlation)
if not (seen contains all of covered):
    fail("trail incomplete, accepted..settled: " + (covered - seen))

# archive_write emits no marker: observe its effect directly
assert archive.contains(order_id)

# notification hand-off has neither marker nor observable effect here
# -> declared unverified by this case, not implied by the pass

go deeper

for a junior

Remember the one-line version: a step that writes no record is invisible to a check built on records, so its absence from the evidence tells you nothing about whether it ran.

for a middle

Be able to explain both misreadings - absence taken as proof of a skip, and presence taken as proof of full coverage - and how a case avoids each without weakening its assertions into uselessness.

for a senior

Show that you state the boundary before being pushed to it: which segment the case proves, which step is checked by its effect instead, and which part is left explicitly unverified.

for a principal

Own the instrumentation tradeoff: which flows deserve records complete enough to carry a verdict, what that costs to add and maintain, and which blind spots the organisation is accepting knowingly.

## The shape of the limit A correlation trail is evidence built out of records the system chose to write. That makes it a genuinely powerful test oracle and a strictly partial one: it can only speak about steps that emit something. Every claim a case makes from it inherits that boundary, and the failure mode is not that the technique breaks loudly - it is that a case quietly proves less than its name suggests while staying green. Two readings of the evidence are wrong in opposite directions: - **Over-reading absence.** "No record for the archive step, therefore the archive step did not run." This raises defects against working code whenever a step simply records nothing, and it burns the credibility of every other finding the case reports. - **Over-reading presence.** "Three hops recorded, so the flow is verified end to end." The pass covers three steps. It says nothing about the four that record nothing, and nothing about whether the three computed correctly. | Observation | Honest reading | Tempting misreading | |---|---|---| | Hop recorded under the case's value | that step was reached | the whole flow is correct | | Hop not recorded | the step did not run, or it records nothing | the step was definitely skipped | | All expected hops recorded | the covered segment ran | there are no gaps in the flow | | Records kept for only some flows | nothing conclusive about absence | a missing hop is a defect | ## Making the limit visible instead of implicit The repair is never to weaken the assertion until it says nothing; it is to say exactly what it does say, where the reader will meet it: 1. **Scope the case name and the failure text to the covered segment.** A case named for the whole flow, which in fact verifies acceptance through settlement, will be believed to cover more than it does - by the person who reads the green result six months from now, and by whoever decides the flow is well tested. 2. **Assert only over hops that genuinely record.** Adding an expected hop that emits nothing produces a case that fails permanently for a reason that is not a defect. 3. **Verify an unrecorded step through its own effect.** If the archive step leaves a stored artefact, query it directly. Direct observation of the effect is stronger evidence than any inference from the trail. 4. **Declare what stays unverified.** Where a step has neither a record nor an observable effect, write that down as a known gap rather than letting a green result imply coverage. 5. **Add a marker deliberately when a step matters enough.** If a hop needs to carry a verdict and records nothing today, the honest fix is a change to the product's instrumentation, agreed with whoever owns it - not a cleverer inference in the case. ## The inference trap The most seductive shortcut is transitive: hop B leaves no record, but hop C does, and C cannot run unless B ran - so C's record proves B. Sometimes that is sound, and when it is, the case should say so in one line, naming the guarantee it depends on. Very often it is not sound: - C has a fallback path and runs with a default when B is skipped. - C is reachable from a second trigger that does not pass through B. - The dependency is true of today's code, not of the design, and nothing fails when it stops being true. An inference that is not written down is an assumption nobody can review. When the guarantee is real, name it; when it is merely how the code currently reads, treat B as unverified. ## Partial records defeat the verdict entirely A trail is usable as a verdict only when the records for the case's own correlation value are kept completely. Where recording is thinned for cost, a missing hop has two explanations - the step did not run, or its record was simply not retained - and the case cannot distinguish them. That is not a weaker verdict; it is no verdict at all in the failing direction. The workable answers are to ensure the run's own flows are recorded in full, or to demote the trail to a diagnostic aid that helps explain a failure some other assertion produced. ## What a good answer sounds like An interviewer is listening for a candidate who states the boundary before being pushed to it: "this case proves the four recorded steps were reached for the flow it triggered, the archive step is checked directly through the stored artefact, and the notification hand-off is not verified here at all." That sentence is worth more than a larger set of assertions with an unexamined claim of coverage behind them, because every future reader of a green result inherits whatever the case implied.

  • A step leaves no record, but the next step cannot run without it. May the case treat the later record as proof?
    Only when that dependency is a guarantee of the design and the case names it in one line. Where the later step has a fallback, can be reached by another trigger, or merely happens to depend on the earlier one in today's code, the inference is false and the case silently proves less than it claims.
  • Why can a trail whose records are only partially kept not carry a pass or fail verdict?
    Because a missing hop then has two explanations - the step did not run, or its record was not retained - and the case cannot tell them apart. Absence stops being informative, so the failing direction of the assertion is meaningless. Either the run's own flows are recorded in full, or the trail is demoted to a diagnostic aid behind some other assertion.

saying these in an interview costs you the question

  • Reads a missing record as proof the step never ran
  • Reports a green trail as end-to-end coverage
  • Infers an unrecorded step from a later one
  • Uses partly kept records as pass or fail evidence
  • Leaves the uncovered segment unmentioned in the case