/proc, /sys and sysctl

## The idea: files as a kernel API Unix's design instinct is "everything is a file", and /proc is the clearest expression of it. Rather than inventing a system call for every piece of information a user might want about the running kernel, Linux exposes that information through a filesystem whose files are computed on demand. Any program that can `open()`, `read()` and parse text can therefore inspect the kernel — no library, no special API, no privileges beyond ordinary file permissions. The filesystem type is literally named `proc`, and you can see the mount: ``` $ grep ' /proc ' /proc/mounts proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 ``` It is mounted by the boot process (or by the container runtime, inside a container) and it is not a partition, not a disk image, and not a RAM disk such as tmpfs. tmpfs really does store bytes in memory; procfs stores nothing at all. ## Why the size is zero When you call `stat()` on an ordinary file, the kernel reads the length recorded in the inode. A procfs file has an inode, but there is no stored content whose length could be recorded — the text is produced by a kernel function at read time, and its length depends on the state of the machine at that instant. /proc/meminfo on a box with a lot of NUMA nodes is longer than on a laptop; /proc/<pid>/maps grows and shrinks as a process maps and unmaps memory. Rather than lie, the kernel reports `st_size` as 0 for most procfs files. This is why naive tools misbehave here. `du -sh /proc` reports nothing useful, a backup tool that trusts `st_size` may copy an empty file, and `cp /proc/meminfo backup.txt` works only because `cp` keeps reading until EOF rather than trusting the size. It is also why /proc never appears in `df` output as consuming disk. ``` $ stat -c '%s %n' /proc/meminfo 0 /proc/meminfo $ wc -c < /proc/meminfo 1543 ``` A useful contrast is sysfs: attribute files under /sys typically report a size of 4096 (one page), because the sysfs convention is that a single attribute fits in one page buffer. Same trick, different convention. ## What lives there Procfs holds two broad families of entries. The numeric directories — /proc/1, /proc/4711 — are one per running process, holding that process's command line, memory maps, open file descriptors, limits and status. /proc/self is a magic symlink that always resolves to the directory of whichever process is reading it, which is how a program inspects itself without knowing its own PID. The non-numeric entries are system-wide: /proc/meminfo, /proc/cpuinfo, /proc/loadavg, /proc/uptime, /proc/stat, /proc/mounts, /proc/cmdline (the kernel command line the machine booted with), /proc/interrupts. All of them are formatted fresh on each read. ## Reading versus writing Most of /proc is read-only: the kernel is reporting, not accepting input. The important exception is the /proc/sys subtree, which mirrors the kernel's tunable parameters. There, writing a value to a file changes kernel behaviour immediately: ``` # echo 1 > /proc/sys/net/ipv4/ip_forward ``` Nothing is stored in a file by that write — the kernel parses the bytes and flips an internal variable, which is why the change is lost at the next reboot. A handful of per-process files are also writable, such as /proc/<pid>/oom_score_adj. ## Snapshot semantics Because each read runs kernel code, every read is a fresh sample rather than a cached one. That is exactly what you want for monitoring, but it has two consequences worth internalising. First, the cost of reading is paid in the kernel, and it is not always trivial — files that must walk large internal structures are genuinely expensive. Second, there is no stability guarantee across reads: a process directory can disappear between the moment you list /proc and the moment you open a file inside it, and two successive reads of the same file legitimately return different numbers. ## What an interviewer is checking They want to hear that /proc is a kernel interface, not storage; that the zero size follows from content being generated on demand; and that /proc/sys is the writable corner where writes become kernel configuration. Candidates who describe /proc as "a folder where Linux keeps process information on disk" have the mental model backwards, and that mistake tends to resurface later when they try to persist a sysctl by editing a file under /proc.