skip to content

In Linux eBPF, at what point does the kernel's verifier examine your program, and what happens to a program that fails verification?

level: juniorimportance: must knowfreq 70%

answer

  1. checked before it ever runs
  2. load time, not attach time
  3. the bpf() syscall is the gate
  4. a log buffer explains the rejection
  5. all-or-nothing: nothing partially runs

basics

~20 s

The eBPF verifier runs inside the kernel at load time, when the bpf() syscall submits the program — before it is attached to anything and before a single instruction executes. A rejected program never runs: the load call fails and the kernel returns a verifier log saying why.

solid answer

~50 s

Verification is a **load-time gate**, not a runtime one. When a loader such as libbpf or `bpftool` calls the `bpf()` syscall to load a program, the kernel's verifier walks the bytecode and tries to prove it is safe: that control flow terminates, that every memory access is in bounds, and that every register holds the type the instruction expects. Only if that proof succeeds does the kernel accept the program, hand back a file descriptor, and (normally) JIT-compile it to native machine code. If the proof fails, the syscall returns an error — typically `EACCES` — and the kernel fills a caller-supplied log buffer with a trace of what it was doing when it gave up. Nothing partial happens: there is no half-loaded program and no instructions execute. Attaching to a hook is a separate, later step, so a program can verify fine and still fail to attach.

go deeper

for a junior

Be able to say plainly that the verifier runs in the kernel when the program is loaded, before it runs at all, and that a rejected program simply never executes.

for a middle

Explain the four-stage lifecycle — compile, load, attach, run — and place the verifier exactly at load, in the bpf() syscall. Know that the rejection reason arrives in a log buffer the loader prints, not in dmesg.

for a senior

Show that you have debugged real rejections: read the tail of the log, map the instruction index back to source, and account for the same object loading on one kernel version and failing on an older one across a mixed fleet.

for a principal

Own the consequence of the design: safety is proven once and enforced never, so the platform's risk sits in kernel version skew, verifier bugs and who is allowed to load at all — not in runtime guardrails that do not exist.

## The one-sentence model eBPF lets you run your own code inside the kernel. The reason that is not insane is the **verifier**: a static analyser inside the kernel that must prove your program is safe *before* the kernel will accept it. Verification happens once, at load; after that the program runs at native speed with no runtime supervisor watching it. ## Where in the lifecycle it sits An eBPF program's life has four distinct stages, and people routinely blur the middle two: 1. **Compile** — clang compiles C to eBPF bytecode in an ELF object (`clang -target bpf`). No kernel involvement; clang will happily produce bytecode the kernel will refuse. 2. **Load** — a loader (libbpf, bcc, `bpftool prog load`, a Go or Rust library) calls the `bpf()` syscall with the `BPF_PROG_LOAD` command, passing the instructions, the program type, and a log buffer. **This is where the verifier runs.** On success the kernel returns a file descriptor for the program and JIT-compiles it to native instructions. 3. **Attach** — the program fd is wired to a hook (a tracepoint, a kprobe, a network interface, a cgroup). Attachment can fail on its own — wrong program type for the hook, a missing symbol, insufficient privilege — and that is *not* a verifier failure. 4. **Run** — the hook fires and the JIT-compiled code executes in kernel context. So "it didn't load" and "it didn't attach" are different bugs with different error text, and saying which one you hit is the first thing an interviewer listens for. ## What failure looks like The `bpf()` syscall returns `-1` and sets `errno`; a verifier rejection is usually `EACCES` (other load problems, like a malformed instruction stream or a bad argument, come back as `EINVAL`). The interesting part is the **verifier log**: the caller passes a buffer and a log level, and the kernel writes a trace of the paths it explored, the register state at each step, and a final line naming the instruction it could not accept — for example an invalid memory access, an unreachable instruction, or an unbounded loop. That log is written to your buffer, not to `dmesg`. libbpf prints it to stderr when a load fails; if it is truncated, you raise the log level and enlarge the buffer. Reading the log is the actual skill here — the last few lines usually name the register and the instruction index, and mapping that index back to your C is the debugging loop. ## Why it is all-or-nothing A kernel that ran unproven code and caught mistakes at runtime would need bounds checks, fault handling and a scheduler for your code — in other words, a sandbox with a runtime cost on every hook, in paths that fire millions of times a second. eBPF instead pays the cost **once**, at load, and then runs unguarded native code. The consequence is that the guarantee has to be total: a program that might be unsafe on one path out of a thousand is rejected outright, because at runtime nobody is checking. The corollary that catches people: a program the verifier accepts is safe in the memory-and-termination sense, not *correct*. It cannot crash the kernel, but it can still count the wrong thing, drop the wrong packet, or be far too expensive to run in a hot path. ## The practical consequences - **Verification is per-load, not per-attach.** Attaching the same program fd to a second hook does not re-verify it. Loading the same object file again does. - **The verifier is conservative.** Rejection means "I could not prove this safe", not "this is definitely broken". Perfectly correct programs get rejected, and the fix is to restructure the code so the proof becomes easy. - **Results vary by kernel.** The verifier improves release to release, so an object that loads on a 6.x kernel may be rejected on an older one. "Works on my laptop, rejected on the fleet's older kernel" is a real and common failure. - **Privilege matters.** Loading normally requires root or `CAP_BPF` (plus `CAP_PERFMON` for tracing, `CAP_NET_ADMIN` for networking hooks); unprivileged loading is disabled on most distributions, and unprivileged programs face extra restrictions on top of the ordinary ones.

  • Where do you actually read the verifier's explanation when a load fails?
    The kernel writes it into a log buffer the loader passes to the `bpf()` syscall, not to `dmesg`. libbpf prints it to stderr on failure; if the output is truncated you raise the log level and grow the buffer. The useful part is the tail: the instruction index and register state at the point the verifier gave up.
  • If I attach one loaded program to three different hooks, is it verified three times?
    No. Verification happens once per load, against the declared program type. Attaching uses the already-verified program fd, so attach failures are about type or permission mismatches, not safety proofs. Re-loading the same object file, however, does run the verifier again — which is why an object can load on one kernel and be rejected on another.
  • Does passing the verifier mean the program is correct?
    No. It means the kernel could prove memory safety and termination. The program can still read the wrong field, account to the wrong key, drop traffic it should pass, or be expensive enough to hurt the hot path it is attached to. Safety and correctness are separate problems, and only the first one is automated.

saying these in an interview costs you the question

  • Thinks the verifier checks each time the hook fires
  • Confuses a load rejection with an attach failure
  • Believes clang refuses to emit unverifiable bytecode
  • Expects the verifier log in dmesg
  • Assumes verified means functionally correct

context