skip to content

How can a parser field rename silently stop a SIEM rule from matching, with no error?

level: middleimportance: should knowfreq 50%

answer

  1. unknown field, not an error
  2. empty result is a legal answer
  3. the search still reports success
  4. parser ships without the content team
  5. predicate can never be true

basics

~20 s

Detection backends treat an unknown field as absent, so a predicate on it is simply false, not invalid. The search still runs, still reports success, and returns an empty result set - a legal answer, not a fault.

solid answer

~50 s

Search languages used for detection are deliberately schema-permissive: rules are written across heterogeneous sources where a missing field is normal, so referencing one that does not exist yields no match instead of an exception. When an in-house parser release renames `dns.query` to `dns.question_name`, every rule filtering on the old name goes permanently false. The job still runs on schedule, the platform reports success, the dashboards are green, and the only visible change is that alerts stop - which nobody monitors. Renames are not the only silent mute: a reshaped value (case folding, a trailing dot, a truncated long name), a moved timestamp field, or a routing change to another index all leave the rule running and matching nothing. The structural cause is that the parsing pipeline ships independently of the detection content and nothing validates the fields the content depends on.

code

json · 11 lines
json
{ "_parser_release": "r-2026.03.01",
  "event_kind": "dns_query",
  "client_ip": "10.14.7.32",
  "dns.query": "a3f9c2b1e7...d4.tunnel.example.net",
  "dns.qtype": "TXT" }
...
{ "_parser_release": "r-2026.03.14",
  "event_kind": "dns_query",
  "client_ip": "10.14.7.32",
  "dns.question_name": "a3f9c2b1e7...d4.tunnel.example.net",
  "dns.question_type": "TXT" }

go deeper

for a junior

Know that a search referencing a field that no longer exists usually returns nothing rather than failing, and that no alerts is therefore an expected symptom of a broken rule as well as of a calm network.

for a middle

Explain the mechanics precisely: unknown field means absent, absent means the predicate is false, and an empty result set is a successful search. Then list mutes beyond renaming, especially value reshaping and moved timestamps.

for a senior

Show where the control belongs. A contract test in the parser pipeline, driven by the field dependencies of deployed content, plus per-rule input-volume monitoring - not a promise that someone will read release notes.

for a principal

Be ready to negotiate the interface between a platform team that owns parsing and a content team that owns detections: who may change a field, what notice is owed, and which side's build breaks when the contract does.

## Why nothing errors A detection rule is compiled into a query and run by a search backend. Those backends are built for logs, not for a fixed schema: the same query is expected to run across sources where some fields are present and some are not, and where new fields appear all the time. So the near-universal semantics are that an unknown field is *absent*, and a comparison against an absent field is *false* - not an error. `dns.query` endswith one thing, `dns.query` matches a regular expression, `dns.query` has more than N distinct values per zone: after the field disappears, every one of those is false for every event, forever. Everything downstream then behaves correctly. The scheduled search executes on time. It scans millions of events. It returns zero rows. Zero rows is what a search returns when nothing matched, and that is exactly what the platform reports - success. No alert is created, so no analyst notices. The rule's own health page, if there is one, usually shows scheduling and completion status, and both are green. The failure is invisible at every layer that anyone actually looks at. ## The shapes silent mutes come in Renaming is the clean case; the family is larger, and they are worth being able to list: - **Field removed or renamed.** The predicate can never be true. This is the case above. - **Value reshaped while the field survives.** The most dangerous variety, because field-presence checks pass. Examples on resolver logs: the parser starts writing the queried name in lower case where the rule matched mixed case; it keeps or strips the trailing dot of a fully qualified name; it truncates long names at a fixed width, which destroys precisely the long-label and entropy signal that a tunnelling rule keys on. A DNS label may be up to 63 octets and a full name up to 255, so tunnelling traffic sits at the long end of the distribution - exactly what a truncating parser flattens. - **Type changed.** A string becomes a structured object, or a count becomes a string; comparisons stop behaving and often stop matching. - **Time moved.** The field the search windows on is renamed, or its parsed time zone changes, so events land outside every scheduled window even though they are indexed. - **Routing changed.** The data goes to a new index, data set or log-source label, and the rule's scope no longer covers it. Note that only the first one is caught by asking `does this field exist`. A real check has to look at population *and* shape. ## The organisational cause, and the check that fixes it In a shop where an in-house parsing pipeline is released on its own cadence, the parser team's contract with the SOC is implicit: they believe they are producing logs, and they are, while the SOC believes it is consuming a stable set of fields, and it is not. A release note saying *the query field is now `question_name` for consistency* is a perfectly reasonable engineering change, and asking humans to read every release note and cross-check it against hundreds of deployed rules is not a control - it is a hope. The durable fix is a contract test that lives in the parser's own pipeline, not in the SOC's calendar: 1. Derive, automatically, the list of fields the deployed detection content depends on - from the rules themselves, not from a hand-maintained document. 2. On every parser change, run a sample of real events through the new parser and assert that each of those fields is present, non-null and of the expected type and shape for the sources that should carry it. 3. Fail the parser build when the contract breaks. Renames then become a coordinated change with the content owners instead of a five-month blind spot. On the detection side, the complementary control is per-rule input monitoring: track the event count matching only the rule's log source and prefilter clauses, and alert when it falls off a cliff. That catches the reshaped-value and routing cases as well as the rename, and it is the signal that would have spoken on the fourteenth. ## Before and after, in one line of log The pair of records below is the whole failure. Same resolver, same client, same query - two consecutive parser releases. The rule was written against the first shape and never touched. Nothing in the platform reported anything wrong, and the only observable effect was the thing nobody was monitoring: alerts went to zero.

  • Name two silent mutes that are not a rename.
    A value-shape change - case folding, keeping or stripping the trailing dot of a fully qualified name, or truncating a long query name so the length and entropy signal disappears - and a timestamp change, where the field the search windows on moves or shifts time zone and events fall outside every scheduled window. In both, the field is still there and the rule still runs.
  • What single check in the parser's release pipeline would have caught this?
    A contract test over a sample of real events asserting that the fields the deployed detection content depends on are still present, populated and the expected shape after the change - with the dependency list generated from the rules themselves rather than maintained by hand. It fails the parser build instead of costing the SOC five months.
  • Would a stricter search language have helped?
    Partly. A backend that raises an error on an unknown field turns a silent zero into a loud failure, which is the behaviour you want here. Most detection backends deliberately do not, because rules span heterogeneous sources where absent fields are normal. So you buy that strictness yourself, with per-rule input-volume monitoring and a parser-side contract test.

saying these in an interview costs you the question

  • Assumes a renamed field makes the query error out
  • Says a green scheduled job proves the rule still matches
  • Believes only missing fields, never reshaped values, mute a rule
  • Relies on humans reading every parser release note
  • Treats the detection author as the only owner of the break

context