In a hunt over auditd logs, how do you confirm execve records exist on a Linux fleet?
answer
- the audit subsystem is rule-driven
- no matching rule means no record at all
- sample per build image, not per fleet
- SYSCALL and EXECVE pair share an event id
- unset auid means no login session
basics
~20 sSample a representative host per build image and confirm an execve-matching audit rule is loaded and producing SYSCALL/EXECVE record pairs in the search index. Without a matching rule the kernel writes nothing, so the source is silent rather than empty.
solid answer
~50 sThe Linux audit subsystem is rule-driven: it records the syscalls a loaded rule matches, and nothing else. If no rule matches `execve`, there is no partial record and no error — there is simply no execution telemetry, and every query over it returns zero rows. So the check is twofold. First, confirm the rule is loaded on hosts of each build image, not on "a host" — a fleet assembled from mixed images will often have execution auditing on newer builds and nothing on the older ones. Second, confirm the records reach the search index with the fields the hunt filters on: the executed image, the arguments, the parent, and the timestamp. Pulling one real `SYSCALL`/`EXECVE` pair per image is the fastest proof. That pair proves coverage for that host at that moment, not for the fleet, so the preflight output is a denominator: which images are covered, how many hosts that is, and which are dark.
code
text · 4 linestype=SYSCALL msg=audit(1717082400.412:88231): arch=c000003e syscall=59 success=yes exit=0
ppid=1 pid=20481 auid=4294967295 uid=0 gid=0 euid=0 comm="sh" exe="/usr/bin/dash" key="exec"
type=EXECVE msg=audit(1717082400.412:88231): argc=3 a0="/bin/sh" a1="-c" a2="/opt/bin/collect.sh"
...go deeper
Know that Linux audit records only what a loaded rule selects, so an execution hunt needs an execve rule in place first. Be able to say that a missing rule produces silence, not an error.
Be ready to explain the SYSCALL and EXECVE record pair, why the argument vector matters for a scheduled payload, and why coverage must be sampled per build image on a mixed fleet.
Show that you verify records in the search index rather than host configuration, and that you convert the sampling into a stated denominator that travels with the hunt result.
Be able to argue for a minimum audit baseline as an image property rather than a per-host fix, so coverage is inherited by new builds instead of chased after each hunt.
## Why auditd needs an explicit coverage check The Linux audit subsystem does not log by default in any useful sense. The kernel emits audit records for the events that **loaded rules** select; `auditd` is the userspace daemon that writes them out. Rules come in two families that matter to a hunter: - **syscall rules**, which select a system call and optional filters — the execution one is `execve`, and it is the rule that makes "what did this host run" answerable at all; - **watch rules**, which select a file or directory path and the access types to record, and are what make "did a new unit file or cron entry appear" answerable. When no rule matches, nothing is written. There is no truncated record, no counter, no warning in the query result. This is the property that makes a preflight necessary: an uninstrumented host and a quiet host produce identical output. ## What the check actually looks like Take the hypothesis — persistence installed as a `systemd` timer or a user cron entry, executing a payload on a schedule. The records that would betray it are execution records for whatever the timer runs, and write records against the unit directories and the cron spool. For each, confirm three things: **Is the rule loaded, per build image?** Fleets are rarely uniform. Mixed distro builds, images from different eras, hosts that pre-date the hardening baseline: execution auditing may be present on the current image and absent on the older ones, which are frequently the hosts you care about most. Sampling one host per image, rather than one host, is the difference between a real coverage figure and an anecdote. Older images may also carry no EDR sensor at all, so audit records are the only execution telemetry available there — and if the rule is missing, that host contributes nothing. **Do records arrive in the index?** A rule can be loaded on the host while the records never reach the search platform. Confirm by finding a genuine record for that host in the index within the last day, not by trusting the host-side configuration. **Do they carry the fields the hunt filters on?** An `execve` event lands as a pair of records sharing one event identifier: a `SYSCALL` record with the process metadata and a companion `EXECVE` record holding the argument vector. The arguments are the point — a scheduled payload is usually a shell invoked with a script path, so a source that gives you only a process name answers a much weaker question. Check the field names after normalisation too, since a mapping step can rename or drop them on the way in. ## Reading the sample record correctly A few field-level facts change what a record supports: - `syscall=59` on x86_64 is `execve`; the `arch` field tells you which syscall table applies, so a 32-bit process is a different number and rules must cover both architectures to be complete. - `auid` is the login uid. When it is unset it appears as `4294967295` — the unsigned form of -1 — which is normal for anything started by init or a timer rather than by a login session. It means you **cannot attribute the execution to a user from this record**; that is a real limitation to state in a hunt write-up, not a corrupt record. - `key` is the tag from the rule that matched. Its presence is itself useful preflight evidence: it tells you which ruleset is in force on that host. - `ppid` gives the parent process id, but the parent's identity has to be resolved from other records — by the time you look, the parent may be gone. ## What the check produces The honest output is a coverage statement with a denominator and a shape: which images carry execution auditing, which carry the directory watches, how many hosts each covers, and which are dark. That statement then travels with the hunt result, so the negative is read as "not seen across the covered subset" rather than "not present in the fleet". Where an image is dark, the gap is written up as a collection requirement naming the exact records needed and the hunt they unblock — and, in the meantime, the hunter looks for compensating evidence on those hosts: configuration-management inventory of unit and cron files, file-integrity data, and resolver query logs, which are collected centrally and therefore unaffected by the host's audit configuration.
- You pull one good execve record from a host. What does it license you to say about the fleet?That this host had a matching rule loaded and shipping at that moment. Nothing more. Rulesets follow build images and local configuration drift, so the statement generalises only to hosts you have sampled from the same image, and even then only until someone changes the ruleset. Coverage is a figure you measure, not one you infer from a single sample.
- Your hypothesis is about a scheduled job installed on disk. Why is execve auditing not sufficient on its own?Execution records show the payload running, but only from the moment the rule was loaded and only while the timer actually fires. A unit or cron file that was planted and has not yet run leaves no execution record. Directory write coverage — an audit watch on the unit and cron paths, or file-integrity data — is what makes the installation itself visible.
- The rules are loaded but no records for that host appear in the search platform. What does the preflight conclude?That the host is uncovered for hunting purposes. Host-side configuration is not coverage; only records reaching the index are searchable. The hunt scope excludes that host, and the discrepancy is handed to whoever owns the collection path, because a rule loaded and a rule delivering are two different claims.
saying these in an interview costs you the question
- Assumes auditd records everything by default
- Samples one host and calls it fleet coverage
- Reads unset auid as a corrupt or root-owned record
- Trusts host-side config instead of records in the index
- Expects a partial record when no rule matches the syscall