As an ordinary non-root user on a Linux host, `perf record` fails with a permission error about performance monitoring operations. Which sysctl governs that, what do its values mean, and what additionally blocks perf inside a container?

## Why perf is privileged at all A CPU profiler samples the instruction pointer of whatever is running. On a shared machine, that is a side channel: sample often enough and you can infer what other users' processes are doing, and kernel-mode samples leak kernel addresses that defeat address-space layout randomisation. So the kernel gates access, and the gate is a single tunable. ## kernel.perf_event_paranoid ```bash cat /proc/sys/kernel/perf_event_paranoid sudo sysctl -w kernel.perf_event_paranoid=1 echo 'kernel.perf_event_paranoid=1' | sudo tee /etc/sysctl.d/99-perf.conf ``` The scale, from most to least restrictive: - **3** — not an upstream value; Debian and Ubuntu kernels carry a patch adding it. Unprivileged users get no perf event access at all. This is why the failure so often shows up first on Ubuntu. - **2** — the upstream default on modern kernels. An unprivileged user may measure their own processes in user space, but not profile the kernel. - **1** — additionally allows kernel profiling, which is what you need for `[k]` frames and any syscall-heavy investigation. - **0** — allows CPU-wide access, so system-wide recording with `perf record -a` becomes possible for unprivileged users. - **-1** — no restrictions. Because it is a plain sysctl, `sysctl -w` changes it for the running kernel and a drop-in under `/etc/sysctl.d/` persists it across reboots. On a developer box or a dedicated performance host, dropping to `1` is normal; on a shared multi-tenant machine it is a real security decision, not a formality. ## The capability route Rather than loosening the sysctl for everyone, you can privilege the profiler. Since Linux 5.8 the fine-grained capability `CAP_PERFMON` exists precisely for this: it grants performance monitoring access without the blanket power of `CAP_SYS_ADMIN`, which is what was required before. Running perf under `sudo` works too and is what most people do interactively, but on a host where a monitoring agent needs standing access, `CAP_PERFMON` on that one binary or service is the smaller blast radius. ## The second, quieter gate: kernel.kptr_restrict Even with sampling permitted, kernel symbol *names* can be missing. `kernel.kptr_restrict` controls whether `/proc/kallsyms` exposes real kernel addresses; when it is restrictive, unprivileged readers see zeros, and perf cannot map a kernel-mode sample to a function. The symptom is a profile that clearly has kernel time in it but shows unresolved addresses instead of function names. Setting `kernel.kptr_restrict=0` alongside a permissive paranoid level is the pair that makes kernel profiling actually readable. ## Inside a container This is where people get stuck, because the sysctl looks right and perf still fails. Three things are in play: 1. **`perf_event_paranoid` is host-wide, not namespaced.** There is no per-container value; the container inherits whatever the host kernel has. Changing it inside the container is either impossible or changes it for the whole machine. 2. **The default seccomp profile blocks `perf_event_open`.** Common container runtimes ship a seccomp allowlist that does not include the syscall perf depends on, so the call is rejected before any capability check matters. Relaxing that — running the container with an unconfined or customised seccomp profile — is required in addition to the capability. 3. **Capabilities must be granted explicitly.** A container process normally does not hold `CAP_PERFMON` (or `CAP_SYS_ADMIN`), so even with seccomp relaxed and a permissive sysctl, `perf_event_open` fails on the capability check. There is a fourth, practical point worth making in an interview: profiling a containerised process is frequently easier *from the host*. The process is an ordinary process in the host's PID namespace with a different PID, so `perf record -p <host-pid>` works with no container configuration at all. The one thing you must then solve is symbols — the binaries and their debug information live inside the container's filesystem, so perf needs to be pointed at them (perf's `--symfs` option exists for exactly this class of problem). ## Reading the error Modern perf builds print a diagnostic naming the sysctl and suggesting a value, which is a gift: read it rather than guessing. The distinction to keep straight is between *permission* failures (paranoid level, capability, seccomp) and *symbolization* failures (stripped binaries, missing debug packages, `kptr_restrict`). They look similarly like "perf isn't working" and have completely different fixes.