Networking & Proxy Concepts

A layer 4 proxy sees only connection data — source and destination IP and port, and for TLS the cleartext SNI hostname — and copies bytes through untouched. A layer 7 proxy parses each request, so it can route on path, method, headers or cookies.

Each instance keeps session state in its own memory, so round-robin sends the next request to an instance that never saw the login and treats the user as anonymous. At the proxy layer the fix is session affinity: pin a client to one backend.

A forward proxy acts for the clients and is configured on the client side, so the origin server sees only the proxy. A reverse proxy acts for the service owner, is deployed in front of the backends, and the client never knows it exists.

Terminating TLS means the load balancer holds the certificate's private key and completes the HTTPS handshake itself, so the browser-to-load-balancer hop is encrypted and the load-balancer-to-application hop is plain HTTP unless you separately encrypt it.

Active probes are synthetic and cheap to run, so they catch a dead or unreachable instance before users do, but they only ever test the probe path. Passive detection judges an instance from real request outcomes, catching failures probes never touch — at the cost of real failed requests.

Round-robin equalises request counts, not work. When request costs or backend capacities differ, equal counts produce unequal load. Least-connection routes by in-flight requests so busy backends are skipped; weighting fixes only known, static capacity differences.

DNS-only discovery supplies naming and resolution: a stable name mapping to a set of addresses. It supplies no health state, no push updates and no real per-request balancing, and its TTL is a hint that clients routinely ignore.

A service mesh applies retries, timeouts, per-hop identity and uniform telemetry to service-to-service traffic through proxies beside each workload, in any language and without code changes. An in-process library still sees call intent — fallbacks, semantics — that a proxy cannot.

Termination puts the key on the proxy and lets it read and route HTTP; passthrough leaves the key on every backend and reduces the proxy to forwarding bytes; re-encryption terminates, routes, then opens a second TLS connection to the backend, so both hops are encrypted at the cost of two handshakes and two certificate sets to manage.

One hop's timeout fired at 30 seconds and synthesised the 504 while the application kept working to 45. Proxy hops run independent fixed timers with no shared deadline, so the effective timeout is the shortest hop, not the sum.

A reverse proxy runs two independent connections per request, client-side and upstream-side, so client keep-alive says nothing about the backend leg. Upstream reuse needs an explicit pool, HTTP/1.1 on that leg, and no Connection: close forwarded through.

Roughly 22 to 32 seconds: up to one interval before the next probe, then three probes ten seconds apart, the last costing its two-second timeout. Tightening the numbers shortens that window but multiplies probe load and makes brief pauses eject healthy backends.

A layer 7 proxy terminates TLS, parses and validates every request, holds per-request state and buffers, and maintains its own upstream connection pool. That costs handshake and parsing CPU, memory per in-flight request, added latency, and a new component that can reject or time out requests itself.

Modulo hashing remaps most keys when N changes, so nearly the whole cache misses at once and the origin takes the load. Consistent hashing maps keys and nodes onto a ring, so removing a node moves only that node's share — roughly 1/N of keys.

All three are reverse proxies; the words differ in emphasis. Load balancer stresses spreading traffic over a pool, API gateway stresses per-caller concerns such as authentication and quotas, and reverse proxy is the plain role underneath both. One process can be all three.

Two: the caller's sidecar and the callee's sidecar, so a chain of N calls crosses 2N proxies. Latency is paid per hop and shows up worst in the tail; sidecar memory tracks how much mesh configuration the proxy holds, not how much traffic it carries.

Marking a backend down only stops new requests. Draining also waits out requests already dispatched and connections already pooled to it, bounded by a deadline after which the proxy force-closes. Long-lived connections never end on their own and must be cut.

Worst case is 30 seconds of proxy attempts against a 15-second client, so later attempts run after the client has already left. The backends do up to three times the work per user request, and slow backends turn retries into a load multiplier.

Overload makes checks time out, ejection moves that instance's traffic onto the survivors, the survivors slow further and are ejected in turn. The loop is closed because ejection increases the load that caused it. Safeguards: fail open below a healthy floor, cap the ejectable fraction, and never eject on latency alone.

A connection-level balancer picks a backend once per connection, and a gRPC client keeps one long-lived multiplexed connection carrying every call. Load follows connections, not requests, so a handful of pinned connections concentrate traffic and new replicas never receive one.

Affinity pins existing sessions, so a new instance can only receive brand-new sessions — during a spike most traffic is already pinned and scale-out adds unusable capacity. A deploy destroys every session pinned to each replaced instance at once, producing error bursts.

TLS ended at the proxy, so the app's own connection is plain HTTP and its force-https rule fires forever. Fix both hops: the proxy must send X-Forwarded-Proto, and the app must honour that header only on connections coming from the proxy.

Deregistration is a write, not a stop: the removal must replicate, every caller must refresh its cached member list, and open connections outlive the entry. Keep serving after deregistering, for longer than that whole window, then exit.

The runtime and the connection pool are holding the old address, not DNS. A JVM caches lookups under its own networkaddress.cache.ttl setting, ignoring the record's TTL, and an already-open pooled connection never re-resolves at all.

Traffic redirection is in place before the sidecar proxy is serving, so the application's earliest calls are steered into a proxy that has no listener or no configuration yet and are refused. Fix it by making the proxy start and become ready before the application container runs.

A per-pod sidecar isolates each workload and handles L7 itself, but enrolling or upgrading means restarting the pod and paying memory for every pod. A node-shared proxy avoids the restart and scales with nodes, at the cost of a shared blast radius.

The proxy verifies the client certificate and then asserts the resulting identity to the application in a request header, because the certificate itself cannot survive termination. That assertion is defeated whenever the header is not stripped from inbound requests or the application is reachable without going through the proxy.

Choose by the decisions you need to make per request. If routing, rewriting, per-request retries or quotas depend on reading the request, you need layer 7. If the payload is not HTTP, the proxy tier may not hold the key, or throughput and latency dominate, layer 4 is correct.

Start from what the control actually says and which network segments it covers, then pick the cheapest topology that satisfies it: plaintext inside a genuinely bounded segment, re-encryption where wires leave a trust boundary, and passthrough only when the edge must never hold the key or see plaintext.