Services and Networking

The old Endpoints object crammed every backend of a Service into one resource, so one pod change rewrote the whole object and pushed it to every watcher - painful at thousands of endpoints. EndpointSlices shard the same data into capped chunks (100 endpoints by default), so a change updates one small slice, and they add per-endpoint conditions and topology fields.

The Service spec has only a few portable load-balancer fields, so provider-specific settings such as internal-only, balancer flavour or TLS are set with the provider's annotations on the Service. spec.loadBalancerClass hands the Service to a different load-balancer implementation instead of the default one.

The service controller adds a cleanup finalizer, has the provider build a balancer for the eligible nodes, writes the address into status, and updates or deletes the balancer as things change. Classic balancers forward to each node's NodePort.

It makes the Service headless: no virtual IP is allocated and no load balancing happens. Cluster DNS instead returns the individual pod IPs for the Service name, letting clients see and address each backing pod directly.

An overlay wraps each pod packet inside a node-to-node UDP packet, so the underlay never needs to know pod addresses — but it costs header bytes (MTU) and hides traffic. A routed network advertises each node's pod range as real routes, so packets travel unwrapped at native MTU, but the underlay must accept those routes.

Read the pod events and note the node, then check that node: is the network plugin's DaemonSet pod healthy there, is a valid config present in /etc/cni/net.d, are the binaries in /opt/cni/bin, does the node have a pod CIDR assigned, and is its address pool exhausted or full of stale allocations?

Roughly five seconds is the C library resolver's retry timeout, so a query is being dropped rather than answered slowly. Look for dropped UDP queries from parallel A and AAAA lookups colliding in connection tracking, and for overloaded or CPU-throttled DNS pods. Confirm with DNS server metrics, then deploy a node-local cache.

Attachment is two-sided. The route names the Gateway in parentRefs (with its namespace); the Gateway's listener declares allowedRoutes.namespaces as Same, All, or Selector plus permitted route kinds. Both must agree, and hostnames must intersect, or the route's status shows it was not accepted.

Cluster DNS publishes only ready endpoints, so replicas waiting on a quorum cannot resolve each other: a deadlock. publishNotReadyAddresses makes the EndpointSlice controller mark every endpoint ready, so peer names resolve as soon as pods have IPs.

The Ingress spec only models host and path routing plus TLS, so everything else had to go somewhere. Annotations are untyped metadata strings: unvalidated, undiscoverable, controller-specific, and in a shared proxy some of them let one tenant influence configuration affecting everyone.

Each controller installs an IngressClass whose spec.controller holds its own identifier. An Ingress selects one via spec.ingressClassName. If it is omitted, only a class annotated as default applies; with no default nothing serves the object, and with two defaults or a mismatch you get silent non-service or accidental public exposure.

A 404 from the controller means no rule matched. Check that the Ingress has the right class and was claimed, that the Host header matches spec.rules[].host exactly, that the path and pathType actually cover the request, and that the backend Service and its EndpointSlices resolve. Confirm the request reached the expected controller.

Cluster forwards external traffic to a pod on any node, adding a hop and rewriting the source IP to the node's. Local only serves traffic from pods on the receiving node, preserving the real client IP but dropping traffic on pods-less nodes and risking uneven load.

Verify the CNI actually enforces policy, then observe real traffic before restricting it. Roll out per namespace: ship DNS and platform allowances first, derive workload allowances from observed flows, then apply default-deny ingress, then egress. Make the default-deny part of namespace provisioning so new namespaces are born closed, and test deny paths in CI.

Set spec.hostname to a DNS label and spec.subdomain to the name of a headless Service in the same namespace that selects the pod. Once the pod is ready, cluster DNS answers <hostname>.<subdomain>.<namespace>.svc.<cluster-domain> with its IP.

Flannel is the minimal option — usually a VXLAN overlay, simple, no policy enforcement. Calico offers routed (BGP) or encapsulated modes with a mature policy engine on iptables or eBPF. Cilium builds its datapath on eBPF, adds identity-aware and layer-7 aware policy, can replace kube-proxy, and brings its own flow observability.

Set spec.sessionAffinity: ClientIP, optionally tuning sessionAffinityConfig.clientIP.timeoutSeconds (default 10800). It hashes the client source IP to a pod. It is L4 only, breaks behind NAT or a proxy, and gives no stickiness when the pod dies or the pod set changes.

internalTrafficPolicy: Local is a hard rule: ClusterIP traffic goes only to endpoints on the caller's node, and is dropped if there are none. trafficDistribution: PreferSameNode is a preference: it uses a same-node endpoint when one is ready and otherwise falls back to any ready endpoint.

MetalLB's controller assigns an address from an IPAddressPool to each LoadBalancer Service. Its speakers then attract traffic for that address. In Layer 2 mode one node answers ARP/NDP for it; in BGP mode nodes advertise it to routers, which spread traffic across nodes with ECMP.

Treat cluster DNS as tier-0: model query volume from pods times lookups per request times search-list amplification, add a node-local cache to absorb repeats, scale and protect the DNS Deployment with requests, anti-affinity, a critical priority class and a disruption budget, and constrain tenants from bypassing or inflating it.

Decide on measured convergence and failure behaviour, not benchmarks. Keep iptables while endpoint-sync latency stays inside your rollout error budget; move to nftables or IPVS when sync time is the bottleneck; drop kube-proxy for eBPF only when you accept coupling the Service dataplane to one CNI and can staff kernel-level debugging.