What must be true on a Linux host for an auditd watch on authorized_keys to produce records?

middleshould knowfreq 41%

answer

a file on disk is not loaded
ordering and immutability decide what loads
architecture and permission filters narrow it
the tag proves the watch is live
keys are only there if sshd looks there

basics

~20 s

The watch must be loaded into the running kernel ruleset, not just present as a file; the audit daemon running and collected; the rule unshadowed by earlier rules; and the fields the detection reads arriving with those names.

solid answer

~50 s

Several things, and only one of them is the rule file. The watch has to be compiled and loaded into the running kernel ruleset - a file dropped into the audit rules directory does nothing until the ruleset is rebuilt and applied, so a host that has not restarted the audit service is silently uncovered. Nothing earlier in the ruleset may suppress it: a never rule ahead of yours, or the configuration being made immutable before your rule loads, kills it with no error anywhere. Architecture filtering matters too, since a rule scoped to 64-bit syscalls misses a 32-bit process doing the same write. The daemon must be running, its records shipped, and the fields your logic reads must arrive with the names you used. Finally, the path is an assumption: keys are only there if the SSH daemon looks there.

Several things, and only one of them is the rule file. The watch has to be compiled and loaded into the *running* kernel ruleset - a file dropped into the audit rules directory does nothing until the ruleset is rebuilt and applied, so a host that has not restarted the audit service is silently uncovered. Nothing earlier in the ruleset may suppress it: a `never` rule ahead of yours, or the configuration being made immutable before your rule loads, kills it with no error anywhere. Architecture filtering matters too, since a rule scoped to 64-bit syscalls misses a 32-bit process doing the same write. The daemon must be running, its records shipped, and the fields your logic reads must arrive with the names you used. Finally, the path is an assumption: keys are only there if the SSH daemon looks there.

code

text · 7 lines

text

type=SYSCALL msg=audit(1735689600.123:45211): arch=c000003e syscall=257 success=yes exit=3
  ppid=1412 pid=1490 auid=1001 uid=0 euid=0 comm="bash" exe="/usr/bin/bash" key="ssh_keys"
type=PATH msg=audit(1735689600.123:45211): item=0 name="/home/svc-deploy/.ssh"
  nametype=PARENT ...
type=PATH msg=audit(1735689600.123:45211): item=1 name="/home/svc-deploy/.ssh/authorized_keys"
  nametype=NORMAL ...
...

go deeper

for a junior

Know that Linux does not audit file writes by default: something has to be configured on the host first, and until it is, no amount of good search logic will find anything.

for a middle

Be ready to walk the chain out loud - rule loaded into the kernel, not shadowed by earlier rules, filters wide enough, daemon running, records shipped, fields populated - and to say which link fails silently.

for a senior

Show that you treat the audit configuration as part of the detection and negotiate it with whoever owns the fleet, rather than assuming the platform team's desired state is the host's real state.

for a principal

Frame this as one control owned by two teams. The interesting question is who is accountable when the infrastructure half silently regresses and the security half keeps reporting coverage.

The chain between a rule file and an alert

A detection for persistence by SSH key - an adversary appending their own public key to a service account's authorized_keys so they can log in whenever they like - depends on the Linux audit subsystem noticing the write. Between "we wrote an audit rule" and "the detection can fire" there is a chain, and every link is an assumption the detection quietly makes.

1. The watch is in the running kernel ruleset. Audit rules live as files, but they only take effect once they are loaded into the kernel. A file placed in the rules directory by config management does nothing on its own; the ruleset has to be rebuilt and applied, which normally happens when the audit service restarts. A host that took the file but has not restarted since is configured and uncovered at the same time, and nothing about it looks wrong.

2. Nothing earlier in the ruleset shadows it. The kernel evaluates audit rules in order. A broad suppression rule placed near the top of a hardened base configuration - the sort added to cut volume from a chatty daemon - can swallow the events your rule was meant to see. Worse, the ruleset can be made immutable, after which no further rules load until reboot. If the immutability directive is ordered ahead of your file, your watch is simply never installed. There is no error in either case; the host just does not report.

3. The filter actually covers the activity. Syscall rules are filtered by architecture. A rule scoped only to 64-bit syscalls will not see the same write performed by a 32-bit process. Similarly, the permission bits on a file watch decide whether writes, attribute changes, reads or executions are captured - a watch for writes will not see an adversary merely reading the file to enumerate existing keys.

4. The daemon is running and the output is collected. Records that are generated still have to reach you. The audit daemon can be stopped, its backlog can be exceeded under load and records lost, and the log agent that forwards them can be misconfigured for the older half of the fleet.

5. The fields survive with the names the logic uses. The raw record is a set of key/value pairs; your rule reads normalised names such as file.path, process.executable and user.effective.id. Those must be populated by the time the rule runs. Note what the raw record gives you and what it does not: it tells you which process wrote which path, under which user identity, at a moment in time. It does not tell you the content that was written, so "a key was appended" is an inference from the path plus context, not something the record states.

6. The path itself is an assumption. The whole detection presumes that authorized keys live where you are watching. If the SSH daemon is configured to read keys from a different location, or to obtain them from a helper command rather than a file, your watch covers a path that no longer matters. The rule's declared inputs should say which configuration it assumes, because that is as much a prerequisite as the audit rule.

Reading the record

A single write produces several correlated records sharing one event identifier: a syscall record carrying the process, its parent, the acting identities and the tag you gave the watch, and one or more path records carrying the file names touched. The tag is what makes the whole thing checkable later - it is the thing you can search for to prove that the watch is live on a given host, because any write to that path, including a perfectly legitimate one from config management, produces a record carrying it.

What this means for how you write the rule

Each link becomes a line in the rule's declared prerequisites: the watch tag it expects, the architecture coverage it assumes, the SSH configuration it assumes, and the fields it reads. Write them down and each one becomes checkable - by querying already-received records for the tag, or by having config management report the loaded ruleset as an inventory fact. Leave them implicit and the rule carries a set of beliefs about hundreds of hosts that nobody has ever tested.

The honest summary is that the audit rule and the detection rule are two halves of one control, owned by two different teams. The detection is only as real as the weakest link in a chain that mostly runs through infrastructure the detection engineer does not administer.

## The chain between a rule file and an alert A detection for persistence by SSH key - an adversary appending their own public key to a service account's `authorized_keys` so they can log in whenever they like - depends on the Linux audit subsystem noticing the write. Between "we wrote an audit rule" and "the detection can fire" there is a chain, and every link is an assumption the detection quietly makes. **1. The watch is in the running kernel ruleset.** Audit rules live as files, but they only take effect once they are loaded into the kernel. A file placed in the rules directory by config management does nothing on its own; the ruleset has to be rebuilt and applied, which normally happens when the audit service restarts. A host that took the file but has not restarted since is configured and uncovered at the same time, and nothing about it looks wrong. **2. Nothing earlier in the ruleset shadows it.** The kernel evaluates audit rules in order. A broad suppression rule placed near the top of a hardened base configuration - the sort added to cut volume from a chatty daemon - can swallow the events your rule was meant to see. Worse, the ruleset can be made **immutable**, after which no further rules load until reboot. If the immutability directive is ordered ahead of your file, your watch is simply never installed. There is no error in either case; the host just does not report. **3. The filter actually covers the activity.** Syscall rules are filtered by architecture. A rule scoped only to 64-bit syscalls will not see the same write performed by a 32-bit process. Similarly, the permission bits on a file watch decide whether writes, attribute changes, reads or executions are captured - a watch for writes will not see an adversary merely reading the file to enumerate existing keys. **4. The daemon is running and the output is collected.** Records that are generated still have to reach you. The audit daemon can be stopped, its backlog can be exceeded under load and records lost, and the log agent that forwards them can be misconfigured for the older half of the fleet. **5. The fields survive with the names the logic uses.** The raw record is a set of key/value pairs; your rule reads normalised names such as `file.path`, `process.executable` and `user.effective.id`. Those must be populated by the time the rule runs. Note what the raw record gives you and what it does not: it tells you which process wrote which path, under which user identity, at a moment in time. It does not tell you the content that was written, so "a key was appended" is an inference from the path plus context, not something the record states. **6. The path itself is an assumption.** The whole detection presumes that authorized keys live where you are watching. If the SSH daemon is configured to read keys from a different location, or to obtain them from a helper command rather than a file, your watch covers a path that no longer matters. The rule's declared inputs should say which configuration it assumes, because that is as much a prerequisite as the audit rule. ## Reading the record A single write produces several correlated records sharing one event identifier: a syscall record carrying the process, its parent, the acting identities and the tag you gave the watch, and one or more path records carrying the file names touched. The tag is what makes the whole thing checkable later - it is the thing you can search for to prove that the watch is live on a given host, because any write to that path, including a perfectly legitimate one from config management, produces a record carrying it. ## What this means for how you write the rule Each link becomes a line in the rule's declared prerequisites: the watch tag it expects, the architecture coverage it assumes, the SSH configuration it assumes, and the fields it reads. Write them down and each one becomes checkable - by querying already-received records for the tag, or by having config management report the loaded ruleset as an inventory fact. Leave them implicit and the rule carries a set of beliefs about hundreds of hosts that nobody has ever tested. The honest summary is that the audit rule and the detection rule are two halves of one control, owned by two different teams. The detection is only as real as the weakest link in a chain that mostly runs through infrastructure the detection engineer does not administer.

Config management applied the audit rule file to every host this morning. Is the fleet covered?
Not yet. The file is on disk; the kernel ruleset is what matters, and it is rebuilt when the audit service restarts. Until then those hosts are configured and blind simultaneously. Worse, if an earlier file in the ruleset makes the configuration immutable, the rule will not load even after a restart, and there is no error to notice.
Your watch captures writes only. What does that exclude, and does it matter?
It excludes reads. An adversary enumerating which keys already grant access, or copying a private key from the same directory, produces nothing. Whether that matters depends on the behaviour you claimed to detect: if the rule is labelled as covering SSH key persistence it is fine, but it must not be read as covering credential access in that directory.
The record shows a write to authorized_keys by an automation account. What can you conclude?
That a process wrote to that path under that identity. The record carries the path, the process image and the acting user, not the bytes written, so it does not say a key was added or whose it is. Confirming that needs the file's contents or a comparison against the previous known state, which is a second collection decision.

saying these in an interview costs you the question

Thinks a rule file on disk is a loaded audit rule
Ignores rule ordering and the immutable ruleset flag
Assumes a 64-bit syscall filter covers all processes
Claims the record shows the key that was written
Never questions whether sshd reads that path at all

What must be true on a Linux host for an auditd watch on authorized_keys to produce records?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

The chain between a rule file and an alert

Reading the record

What this means for how you write the rule

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions5 questions

▾related topics4 topics

The chain between a rule file and an alert

Reading the record

What this means for how you write the rule

answer

go deeper

context