skip to content

A payroll run's behaviour changed with no edit to its call path, so what would you add to make declarative wiring discoverable during an incident?

level: seniorimportance: should knowfreq 42%

answer

  1. nothing in the path was edited
  2. wiring you cannot search for
  3. effective, not intended
  4. a report from the machinery itself
  5. tests pin what comments cannot

basics

~20 s

Publish the effective wiring rather than the intended wiring: a build or startup report naming which declarations actually got which behaviour, behaviour that emits its own trace events, tests that fail when a marker stops taking effect, and a loud complaint about markers nothing acts on.

solid answer

~50 s

During an incident you need the **effective** wiring, not the intended wiring, and a convention document is only the latter. Four things buy it. First, have the machinery that applies behaviour **emit a report** — per declaration, which markers were present, which were acted on, and with what settings — so a responder reads one artifact instead of every file. Second, make the behaviour **speak at run time**: a retry that emits its own event turns "the payroll run took three times as long" into a visible second attempt. Third, **pin the behaviour with tests** so its disappearance is a build failure rather than a production surprise. Fourth, **fail loudly on a marker nothing consumes**, since that is the one that reads like a promise and keeps none. Together they convert invisible wiring into evidence you can point at.

code

json · 11 lines
json
{
  "declaration": "payroll.runPayroll",
  "markersPresent": ["transactional", "retried", "owner"],
  "markersApplied": [
    { "marker": "transactional", "effect": "boundary opened around the call" },
    { "marker": "retried", "effect": "up to 3 attempts on failure", "order": 2 }
  ],
  "markersIgnored": [
    { "marker": "owner", "reason": "nothing acts on this marker" }
  ]
}

go deeper

for a junior

Recall that behaviour can change without any edit to the code you are reading, because it was wired by metadata on a declaration. When that happens, look for a report of what is actually applied.

for a middle

Explain the difference between intended wiring, such as a convention document, and effective wiring produced by the machinery itself, and say why only the latter helps once something is already broken.

for a senior

Demonstrate the build-out: a per-declaration report at build or startup, behaviour that emits its own timeline events, tests that fail when the effect disappears, and a loud complaint about markers nothing consumes.

for a principal

Decide how much of this a codebase must carry before declarative wiring is allowed to grow, who owns the report, and whether a diff of the effective wiring becomes a required review artifact for every release.

## The incident shape The distinctive incident in a marker-driven system is one where **nothing in the call path was edited**. The payroll method's body is unchanged, its callers are unchanged, and its behaviour is different. The cause is somewhere else: a marker added or removed on the declaration, a change to what the machinery does for markers of that kind, a change in what it matches, or a change in ordering when several behaviours apply to the same declaration. A responder holding only the source of the call path cannot distinguish these, because none of them is visible from there. So the goal is not "documentation". It is **evidence**: an artifact, produced by the same system that applies the behaviour, that says what is actually in force right now. ## Intended wiring versus effective wiring This is the distinction to lead with in an interview. | Artifact | What it tells you | Why it fails in an incident | |---|---|---| | A convention document | What the team agreed to do | Written once; drifts from reality silently | | A search for marker names | Where markers are written | Cannot say which ones anything acted on | | Reading the declarations | What is attached | Same problem, at much greater cost | | A report from the machinery | What behaviour is actually applied | This is the one that answers the question | The first three describe intent. Only the last describes effect, and in an incident effect is the only thing worth reading. ## Four things to build 1. **An effective-wiring report.** Whatever applies behaviour already knows, for every declaration, which markers it saw and what it did about them. Have it emit that as a machine-readable artifact at build or startup: declaration, markers present, markers acted on with their settings, markers ignored with the reason. This is cheap to produce because the information already exists at the moment of application, and expensive to reconstruct afterwards. 2. **Behaviour that announces itself.** A boundary that opens and a second attempt that starts should each emit something into the timeline — a log line, a trace span — from the machinery rather than from the business code. Without this, a responder infers a retry from a doubled duration; with it, the retry is a fact on the screen. The rule of thumb: *if a behaviour was invisible at the call site, it must be visible in the timeline.* 3. **Tests that pin the behaviour.** Assert the effect, not the presence of the marker: run the payroll operation against a failing dependency and assert that it was attempted more than once. Such a test fails the moment the behaviour stops taking effect — whichever of the causes above did it — which converts a silent production change into a build failure. A comment beside the marker offers none of this protection. 4. **Loud failure on an unconsumed marker.** A marker that nothing acts on is the worst object in the system: it reads to every future maintainer as a promise about behaviour and keeps none. Make the report list it, and where the risk justifies it, make its presence fail the build. ## Making the change visible in review Most of these incidents are introduced by a diff that looked small. Two habits close that gap: - Treat **markers as behaviour** in review. A deleted marker line gets the scrutiny of a deleted block of code, and the review description says what behaviour was removed. - **Diff the effective-wiring report** between builds. A change in which declarations receive which behaviour then appears as a reviewable artifact, even when the source diff shows only a marker line. This is what catches the widest-blast-radius change of all: one where the machinery changed and no marked file was touched. ## Operating with it during the incident With those artifacts in place the procedure is short and does not depend on anyone's memory of the convention: 1. Read the current effective-wiring report for the affected declaration; compare it with the report from the last known-good build. 2. Read the timeline for events emitted by the behaviour itself — did a second attempt happen, did a boundary open and roll back? 3. If the report and the timeline disagree, the fault is in the machinery rather than in the markers, and that is a different investigation with a different owner. ## The trade you are accepting All four items cost something: report generation, extra events in the timeline, tests that are slower than asserting a marker is present, and a build that can now fail for a reason unrelated to compilation. They are worth it in proportion to how much behaviour the codebase wires declaratively. A service with two markers does not need a report; a service where boundaries, retries, authorisation and metrics are all attached to declarations cannot be operated safely without one.

  • Why prefer a report emitted by the machinery over one assembled by a separate scan?
    Because only the machinery knows what it actually did. A separate scan reproduces the matching rules and will disagree with reality the first time those rules change, which is exactly the incident you are trying to diagnose. A report produced at the moment of application cannot drift from the thing it describes.
  • What do you do with a marker the report lists as ignored?
    Remove it, or make its ignored status impossible to miss. It costs nothing to run and everything to read: a maintainer sees a promise of behaviour that is not in force. Where the behaviour matters — a boundary, an authorisation check — make its presence without a consumer fail the build instead of appearing in a list.
  • How do you assert a declarative behaviour in a test without asserting the marker is present?
    Exercise the operation so that the behaviour must show itself: make the dependency fail once and assert the work was attempted again, or assert that a partial failure left nothing committed. Asserting the marker's presence only proves the text is there, which is the thing that was never in doubt.

saying these in an interview costs you the question

  • Says searching for marker names shows what actually applied.
  • Trusts a convention document as evidence of current wiring.
  • Claims a comment beside a marker protects it from removal.
  • Assumes a timeline shows retries with nothing emitting them.
  • Thinks reading every declaration is a workable incident procedure.
  • Asserts a marker is present and calls that a behaviour test.