Portable Rule Formats

### What the format is Sigma exists to solve one problem: a detection idea is portable, but every query language that could express it is not. One team writes `mshta.exe` with an `http` URL on its command line as SPL, another as KQL, a third as a Lucene query, and none of them can share the result. Sigma is a YAML description of the detection itself, from which each of those queries can be generated. The document has a small, fixed shape. `title` and an optional `id` name it. `status` says whether it is experimental or stable. `logsource` declares the kind of data it needs - a `category` (for example `process_creation`, `network_connection`, `dns_query`), a `product` (`windows`, `linux`, `aws`), and sometimes a `service` (`security`, `sysmon`). `detection` contains one or more named search identifiers - each a map of field names to values, with modifiers such as `contains`, `startswith`, `endswith` or `re` - and a `condition` that combines them with `and`, `or`, `not` and `all of`. Around that sit `falsepositives` (free-text notes on what benign work will trip it), `level` (the author's severity), and `tags` (commonly ATT&CK technique identifiers such as `attack.t1218.005`). ### Why it is not a query language A query language has an engine that executes it. Sigma has none. It is compiled, and the compilation has two halves that people routinely conflate. The **backend** produces the dialect: it knows how to write a wildcard, a string comparison, a list membership and a boolean combination in SPL or KQL. The **processing pipeline** produces the environment binding: it resolves `product: windows` + `category: process_creation` to the actual table or index where you keep that data, and it renames Sigma's taxonomy fields (`Image`, `CommandLine`, `ParentImage`, `User`) to whatever your platform calls them. Run a rule through a backend with no suitable pipeline and you get, at best, a refusal - and at worst a syntactically valid query naming columns your schema does not have. Three consequences follow directly, and they are what interviewers are actually testing. First, **the same rule is two different queries on two platforms, and they are not guaranteed to be semantically identical**. Sigma's matching is defined as case-insensitive; backends differ on whether their operators are. Wildcard and escaping rules differ. A rule that behaves one way in one tenant can behave slightly differently in another without anybody editing it. Second, **what the backend cannot express is refused or dropped**. Search logic converts almost everywhere; anything that groups, counts or correlates across events does not, because many backends implement search only. Third, **the metadata is not part of the search**. `level`, `tags` and `falsepositives` are properties of the rule document, not of the compiled query. Unless your deployment tooling copies them onto the alert definition, the analyst who receives the hit at 03:00 gets a bare search result with no severity, no technique context and none of the author's notes about what benign activity looks like. That is a real triage cost, paid on every firing. ### What it is not Sigma is not YARA. YARA matches content - strings and byte patterns inside a file or a memory image - and is evaluated by a scanner against that content. Sigma matches fields in log records and is compiled into a search that a log platform runs. They answer different questions: *is this artefact the malware* versus *did this behaviour appear in my telemetry*. ### What portability actually buys It buys consumption. When a vendor or a community project publishes a rule for a signed Microsoft script host being used to fetch a remote scriptlet, you can take that rule the day it appears instead of reimplementing it from a blog post. It buys an exit if you ever change SIEM. What it does not buy is correctness in your data: a published Sigma rule is a claim about adversary behaviour, not a claim about your columns, your agents or your coverage. Turning the first into the second is the work.