A CEF mapping caps process command lines at 1023 characters — what does that destroy?

## What truncation does that dropping does not Of all the ways a mapping loses data, truncation is the most dangerous, because the field still looks like a value. A dropped field is visibly absent — a rule referencing it stops matching, an analyst sees a blank. A truncated field is present, plausible, and wrong. Everything downstream treats it as complete. CEF makes this easy to hit. Its extensions are flat `key=value` pairs drawn from a fixed dictionary, and the dictionary specifies maximum lengths — the custom string slots `cs1` through `cs6` are capped at 1023 characters. A command line is not a short string. Modern offensive tooling routinely produces command lines of several kilobytes, most of it encoded payload. A mapping that parks the command line in `cs1` keeps a prefix and discards the rest. ## The concrete failure Suppose six process-creation events over an afternoon, all launching the same interpreter with the same flags, differing only in a base64 payload that begins after the launcher and its options. After truncation, the normalised `process.command_line` values are byte-identical. Four things break at once: 1. **Deduplication and aggregation collapse them.** A pipeline or a dashboard that groups identical command lines now shows one entry with a count, or shows six rows that look like the same thing happening repeatedly. 2. **Rarity and stacking logic inverts.** Techniques that look for the unusual command line rely on distinctness; six unique executions that all appear as one common string are now the most ordinary-looking thing in the data set. 3. **Late-string matching never fires.** Any detection keyed on something that occurs after the cap — a flag, a URL, a marker inside the payload — cannot match, because those bytes are not in the record the rule reads. 4. **Command-line hashes collide.** Enrichments that fingerprint the command line to track "have we seen this exact invocation before" now say yes for executions that were never the same. The outcome that matters for an interview is the human one: the analyst sees a repeating, identical, boring string, concludes it is a scheduled job, and tunes it out. The tuning decision was made about the mapper's output, not about what the hosts actually ran. ## The direction-of-claim point Identical normalised values prove that **the mapper produced identical output**. They do not prove that identical things happened. Whenever a conclusion rests on sameness — dedup, rarity, "we have seen this before" — you must know whether the field carrying that sameness is complete. And note where the loss occurred. Sysmon Event ID 1 carries the full command line at collection time on the host, so the artefact existed and was captured; it was destroyed in transit by the schema mapping. That is a different problem from the native Windows Security 4688 process-creation event, which carries the command line only if audit policy was configured to include it — there the artefact was never collected at all. Distinguishing "never captured" from "captured and then discarded downstream" tells you whether the fix is an audit-policy change on hosts or a mapping change in the pipeline. ## The wider taxonomy of lossy mapping Truncation is one member of a family, and a strong answer names the others: - **Truncation** — a value longer than the field's cap keeps only its prefix. - **Dropping unmapped fields** — anything with no home in the target schema silently disappears at ingest. OCSF's `unmapped` object exists to prevent this; most flat formats have nowhere to put it. - **Flattening structure** — RFC 5424 structured-data elements, or a nested JSON object, squeezed into a single message string. The values survive as text but stop being addressable as fields, so a rule can no longer match on one of them cleanly. - **Enum collapse** — two source vocabularies mapped onto one target value set, so distinct meanings become the same token. - **Type coercion** — a string forced to an integer loses leading zeros; a high-precision timestamp rounded loses ordering within a burst. All of them are silent. None produces a parse error. ## Making it visible You cannot fix what you cannot see, so instrument the mapper: - Emit the **original length** of any capped field, plus a boolean truncated flag. Now truncation is queryable and countable. - Chart the **distribution of value lengths per source**. A spike stacked exactly at the cap is the signature; a healthy field has a smooth tail. - **Retain the raw event.** This fixes investigation, not detection: rules run over the normalised copy, so a truncated field still cannot match. What raw retention buys is the ability to go back and re-derive, and to re-normalise history under a corrected mapping — within the retention window and no further. - Where the field genuinely needs the room, move it out of a capped slot: carry the event in a model that has no such cap, or split the value across fields deliberately with an explicit marker rather than losing the tail by accident. ## What good sounds like "The prefix survived and the discriminating bytes did not, so six different executions became one repeated string and got tuned out. It is silent, so I would measure it: original length, a truncated flag, and a length histogram per source."