Give an overview of Flannel, Calico and Cilium as Kubernetes pod network implementations: what each is built on, and what would push you toward one rather than another?

## How to compare implementations at all Every pod network plugin answers four questions, and comparing them is easiest along those lines: 1. **Datapath** — how does a packet get from pod to pod (overlay, routed, eBPF-accelerated)? 2. **IPAM** — where do pod addresses come from and how flexibly? 3. **Enforcement** — can it enforce policy, and with what mechanism? (Policy *semantics* are a separate subject; here only capability matters.) 4. **Operability** — kernel requirements, observability, upgrade story, ecosystem features like encryption or multi-cluster. ## Flannel The deliberately small option, from the CoreOS lineage. A per-node agent assigns each node a subnet from the cluster CIDR and programs a datapath — by default `vxlan`, optionally `host-gw` (plain routes when all nodes share a subnet) or provider-specific backends. Configuration is a single ConfigMap; there is little to tune and little to break. What it does not do: enforce network policy. Historically deployments paired it with a policy component (the "Canal" combination) to fill the gap. It also offers no fancy IPAM, no encryption, no built-in visibility. **Choose it when** the cluster is small, you want the fewest moving parts, and isolation and observability are handled elsewhere or not yet needed. ## Calico The broadly deployed general-purpose plugin, and the one that popularised routed pod networking. Its node agent (Felix) programs the node's datapath while a routing component distributes pod routes. - **Datapath options**: native routing with **BGP** (peer directly with top-of-rack switches, or use route reflectors so you do not need a full mesh at scale), or encapsulation with VXLAN or IP-in-IP. Crucially it supports *cross-subnet* mode: encapsulate only when source and destination nodes are in different subnets, route natively otherwise. - **Dataplane**: the traditional iptables/nftables implementation, or an eBPF dataplane for lower overhead and better service handling. - **IPAM**: multiple pools, per-namespace or per-node pool selection, block sizes you control — useful when address space is constrained or when certain workloads need routable addresses. - **Enforcement**: a mature policy engine, including its own richer policy resources beyond the core Kubernetes objects. **Choose it when** you run on-premises with a network you can peer with, when you need address-space flexibility, or when you want a well-understood default with strong policy support and a long operational track record. ## Cilium Cilium's premise is that the kernel's programmable **eBPF** hooks are a better place to build the datapath than long chains of iptables rules. Programs attached at the network interface and socket layers handle forwarding, address translation and policy directly. What that unlocks: - **kube-proxy replacement**: Service virtual IPs are handled by eBPF programs (often at the socket layer for local traffic), which scales better than rule chains that grow with the number of services and backends. - **Identity-based enforcement**: workloads get an identity derived from their labels, and policy is evaluated on identity rather than on ephemeral IPs — which suits churn-heavy clusters. - **Protocol awareness**: policy can consider application-layer attributes such as HTTP paths or Kafka topics, not just address and port. - **Observability**: Hubble exposes per-flow records with pod-level identity, which is a real advantage on an encapsulated datapath where captures would otherwise show only tunnels. - **Extras**: transparent WireGuard or IPsec encryption, multi-cluster meshing, egress gateway, bandwidth management. Costs: a modern kernel is required, and the more features you turn on the more concepts and control-plane components you operate. Debugging shifts from `iptables-save` to Cilium's own tooling. **Choose it when** you have large service counts, need identity or protocol-aware enforcement, want built-in flow observability or encryption, and can guarantee the kernel baseline. ## Managed and provider plugins On managed clusters, the default is often a provider plugin that gives pods **real VPC addresses** from secondary interfaces or alias ranges. Then there is no overlay at all: cloud load balancers can target pods directly, cloud security groups and flow logs apply to pods, and latency is native. The tradeoff is address-space pressure — pods consume real subnet IPs — plus per-instance limits on how many addresses a node can hold, which caps pod density. Several providers now let you run Cilium or Calico in place of, or layered on, the default. ## Answering well Avoid ranking them absolutely. State each one's datapath and enforcement story in a sentence, then choose against the constraints of the environment described: underlay control, address space, kernel version, service count, observability and encryption needs, and how much operational surface the team can carry.