A colleague loaded an XDP program onto a production interface from a terminal that has since been closed, and traffic is still being affected. Why do some eBPF attachments survive the process that created them while others vanish with it, and how would you find and remove this one?

## Why this is a real incident and not a trivia question The eBPF object model has no owning process in the way people expect. Once a program is loaded and attached, it runs because something in the kernel holds a reference to it, and "the shell that started it" is often not that something. Engineers who assume otherwise kill the loader, see nothing change, and conclude the box is haunted. ## The reference model A program loaded with `bpf(BPF_PROG_LOAD)` returns a file descriptor. While at least one reference exists the program stays loaded; when the last one goes, it is freed. References come from three places: 1. **An open fd** in some process — typically the loader. 2. **A pin** in the bpf filesystem, usually mounted at `/sys/fs/bpf`. A pinned program or link survives every process on the box. 3. **An attachment**, if the attach mechanism takes its own reference. It is the third that produces the surprise, because the answer differs by mechanism. ## bpf_link: the attachment is an object with an fd The modern interface is `BPF_LINK_CREATE`, which returns a **link** fd representing the attachment itself. fentry/fexit, tracepoint, and modern XDP and cgroup attachments all work this way when the loader asks for them. The lifetime rule is simple and pleasant: the attachment lives exactly as long as the link fd. Close it — including by the process exiting or being killed — and the kernel detaches cleanly. This is why running a bpftrace or libbpf tool in the foreground and pressing Ctrl-C leaves nothing behind, and why link-based attachments are self-cleaning by default. Pin the link under `/sys/fs/bpf` and you have deliberately opted out of that. ## The legacy paths: the kernel holds the reference Several widely used attach paths pre-date links and take a reference of their own: - **XDP over netlink.** `ip link set dev eth0 xdp obj prog.o sec xdp` attaches to the net device. The netdev holds the program, and the program keeps running after the loader dies, across the loader's terminal closing, and across the interface going down and up. - **tc classic filters.** A filter added with `tc filter add dev eth0 ingress bpf ...` lives in the qdisc's filter list. - **cgroup attachments** made with `bpf(BPF_PROG_ATTACH)` rather than a cgroup link stay attached to the cgroup. - **Socket filters** attached with `setsockopt(SO_ATTACH_BPF)` are an intermediate case: they are tied to the socket, so they die when the socket is closed, which usually means when the owning process exits. For the first three, there is no process to signal. You must detach through the same subsystem that attached. ## Finding it ```bash # which interfaces carry an XDP program, and which program id? ip link show dev eth0 bpftool net show # what is program id 42 — type, name, when was it loaded? bpftool prog show id 42 # which attachments are held by links (and therefore self-cleaning)? bpftool link show ``` `ip link show` prints an `xdp` marker with the program id for an attached interface. `bpftool net show` is the one command that enumerates XDP and tc attachments across the host's interfaces, and it is the right first move because `bpftool prog show` alone tells you what is *loaded*, not *where it is attached* — a distinction that trips people up under pressure. ## Removing it ```bash ip link set dev eth0 xdp off # detach XDP tc filter del dev eth0 ingress # remove a tc filter bpftool cgroup detach /sys/fs/cgroup/... ... # detach a cgroup program ``` If the program was pinned, the pin under `/sys/fs/bpf` is a separate reference and has to be removed too, or the program stays loaded even after detachment. Note what does *not* work: killing the loader, deleting the ELF object from disk, or flapping the link with `ip link set dev eth0 down && up`. The attachment lives on the netdev, and the object file was only ever the source of the bytecode. ## The habit to take away When you attach anything on a production host, know in advance which of the two lifetimes you have chosen, and write the detach command down before you run the attach. "It stops when I press Ctrl-C" is true for a link-based tool and false for `ip link set ... xdp`, and the difference is only visible if you know to look for it.