skip to content

Reverse Proxy and Upstream

proxy_pass plus an upstream block, with a load-balancing method, keepalive connections and forwarded headers. Interviewers ask which headers you must set — Host, X-Forwarded-For, X-Forwarded-Proto — because the app behind you depends on them.

on this pageshow

questions

6

An app sitting behind nginx `proxy_pass` logs every client IP as the proxy's address, sees an upstream group name in its Host header, and builds http:// links although users arrive over HTTPS. Which `proxy_set_header` directives fix this, and what is nginx doing by default?

level: juniorimportance: must knowfreq 78%

answer

  1. the backend only sees nginx
  2. nginx sends almost nothing by default
  3. Host defaults to the upstream name
  4. three headers: name, address, scheme
  5. the app must also trust them

basics

~10 s

Set Host, X-Forwarded-For and X-Forwarded-Proto with proxy_set_header. Nginx defaults Host to $proxy_host, the proxied server's name, and sends no forwarded headers at all, so the app sees the proxy's IP and assumes plain HTTP.

solid answer

~50 s

All three symptoms are the same root cause: nginx only forwards what you tell it to. Its built-in defaults are `proxy_set_header Host $proxy_host;` and `proxy_set_header Connection close;` — and nothing else, so the app's Host is the upstream name, its remote address is the proxy, and it has no idea TLS was ever involved. The fix is three lines in the proxying location or server block: `proxy_set_header Host $host;` so virtual-host routing and generated links use the real name, `proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;` so the client address travels, and `proxy_set_header X-Forwarded-Proto $scheme;` so the app knows it was HTTPS. Then the app has to be configured to trust those headers — nginx setting them does nothing on its own. One gotcha: `proxy_set_header` is only inherited from an outer block if the current block defines none of its own, so adding a single header in a location silently drops the ones you set in the server block.

code

nginx · 13 lines
nginx
server {
    listen 443 ssl;
    server_name app.example.com;

    location / {
        proxy_pass http://backend;

        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

go deeper

for a junior

Memorise the three lines and what each fixes: Host for the real hostname, X-Forwarded-For for the client IP, X-Forwarded-Proto for the scheme. Be able to say that nginx sends none of them unless you ask.

for a middle

Explain the defaults — Host is $proxy_host, Connection is close — and what $proxy_add_x_forwarded_for expands to. Know that proxy_set_header inheritance is all-or-nothing per level.

for a senior

Connect the config to symptoms you have debugged: redirect loops from a missing X-Forwarded-Proto, rate limits keyed on the proxy's own IP, and the fact that the framework's trusted-proxy setting must be enabled before any of it takes effect.

for a principal

Treat the forwarded-header set as a platform contract rather than per-service config: one included snippet at the edge, one documented trust boundary, and a rule about who may set these headers, so that services do not each invent their own client-identity convention.

## Why the app loses all this information A reverse proxy terminates the client's connection and opens a new one to the backend. From the backend's point of view the peer is nginx: its remote address is nginx's address, the connection is whatever nginx used (usually plain HTTP on a private network), and the Host header is whatever nginx chose to send. Everything about the original client is gone unless nginx re-attaches it as request headers. Nginx's built-in defaults for proxied requests are minimal: ```nginx proxy_set_header Host $proxy_host; proxy_set_header Connection close; ``` `$proxy_host` is the name and port from the `proxy_pass` value — for `proxy_pass http://backend;` that is literally `backend`. That is why an application behind an unconfigured proxy reports a Host nobody has ever heard of, and why virtual-host selection, absolute-URL generation and cookie domains all misbehave. ## The three headers that matter ```nginx location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } ``` **Host.** `$host` is the host from the request line, or the Host header, or the matching `server_name`, lowercased and with the port stripped. `$http_host` is the raw header including any port. Use `$host` unless the backend genuinely needs the port back. **X-Forwarded-For.** Nginx adds no forwarded header on its own. `$proxy_add_x_forwarded_for` expands to the client's incoming X-Forwarded-For with `$remote_addr` appended, or just `$remote_addr` when the client sent none. `$remote_addr` alone is a useful alternative and is discussed below. **X-Forwarded-Proto.** `$scheme` is `http` or `https` for the connection the client made to nginx. Frameworks use it to decide whether to emit `https://` links, whether to set the Secure cookie flag, and whether to issue an HTTP-to-HTTPS redirect — which is how a missing X-Forwarded-Proto turns into an infinite redirect loop: the app sees http, redirects to https, nginx terminates TLS and proxies as http again. `X-Real-IP` is not standard; it is an nginx convention carrying a single address, convenient because it needs no parsing. ## Nginx setting them is only half the job These are ordinary request headers. The application must be told to trust them, and nearly every framework and app server has an explicit switch for that with a list of trusted proxy addresses. Until that switch is on, the app keeps using the socket peer address. This is the most common reason "I added the headers and nothing changed". ## The inheritance trap `proxy_set_header` directives are inherited from an outer configuration level **only when no `proxy_set_header` directive is defined at the current level**. Adding one header inside a `location` therefore discards every header set in the enclosing `server` or `http` block: ```nginx server { proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto $scheme; location /api/ { proxy_set_header X-Api-Key $api_key; # Host and X-Forwarded-Proto are now gone here proxy_pass http://backend; } } ``` The robust pattern is to put the common set in a separate file and `include` it in each location that proxies, so the full set is always re-declared. ## Related mechanics worth knowing Setting a header to the empty string removes it from the proxied request: `proxy_set_header Accept-Encoding "";` is the standard way to stop backends compressing responses you want nginx to post-process. And `proxy_pass_request_headers off;` stops the client's headers being forwarded at all, leaving only what you set explicitly. ## What a good answer sounds like Name the three headers, say what each one unblocks in the application, state that nginx's Host default is `$proxy_host` and that no forwarded header is added automatically, and finish by pointing out that the backend must be configured to trust the headers. That is the complete loop; stopping at "copy these three lines" is the shallow version.

  • What is the difference between $host and $http_host in an nginx configuration?
    `$http_host` is the raw Host request header exactly as the client sent it, port included, and is empty if the client sent none. `$host` is normalised: the host from the request line, else the Host header, else the matching `server_name`, lowercased with the port removed. `$host` is the safer default for `proxy_set_header Host`; use `$http_host` only when the backend genuinely needs the original port.
  • You add one proxy_set_header inside a location and the headers you set in the server block stop arriving. Why?
    `proxy_set_header` is inherited from an outer level only when the current level defines none of its own. Declaring a single header inside the location replaces the whole inherited set rather than adding to it. Keep the common headers in an included snippet and include it in every proxying location so the set is always complete.
  • Why can a missing X-Forwarded-Proto produce an infinite redirect loop?
    The app sees a plain HTTP request, applies its force-HTTPS rule and redirects the browser to the https:// URL. The browser comes back to nginx over TLS, nginx terminates it and proxies as plain HTTP again, so the app redirects once more. Setting `X-Forwarded-Proto $scheme` and enabling the app's trusted-proxy handling breaks the cycle.

saying these in an interview costs you the question

  • Assumes nginx forwards X-Forwarded-For automatically
  • Thinks the Host header passes through unchanged by default
  • Believes setting the headers in nginx is enough without configuring the app
  • Adds one header in a location and expects the outer ones to be inherited
  • Confuses X-Real-IP with a standard header the backend must honour

context

open as a page

In nginx, inside `location /api/ { ... }`, what is the difference between `proxy_pass http://backend;` and `proxy_pass http://backend/;`, and what URI does the upstream receive in each case?

level: middleimportance: must knowfreq 72%

basics

~20 s

Nginx replaces the matched location prefix with the URI in proxy_pass whenever one is present. With the trailing slash, /api/users reaches the upstream as /users; with no URI at all, the full /api/users is passed unchanged.

open as a page

You added `keepalive 32;` to an nginx `upstream` block, but connections to the backend are still opened and closed for every request. What else must the proxying location set, and what does that number actually count?

level: middleimportance: should knowfreq 52%

basics

~20 s

The keepalive directive alone is not enough: the location must also set proxy_http_version 1.1 and clear the Connection header with proxy_set_header Connection "". The number is the count of idle connections each worker caches, not a connection limit.

open as a page

In open-source nginx, a backend in an `upstream` group crashes and some clients still get 502s for the next several seconds. How do `max_fails`, `fail_timeout` and `proxy_next_upstream` decide that, and what can open-source nginx not do here?

level: seniorimportance: should knowfreq 44%

basics

~10 s

Open-source nginx marks backends down only passively, from real requests: max_fails failures within fail_timeout take a server out for fail_timeout seconds. Failures are whatever proxy_next_upstream counts, and requests already partly answered cannot be retried.

open as a page

An nginx config forwards `X-Forwarded-For $proxy_add_x_forwarded_for` and the backend reads the first address in that header to identify clients. A user forges the header and evades per-client limits. How do you make nginx produce a client address the backend can trust?

level: seniorimportance: should knowfreq 46%

basics

~20 s

Stop letting the backend parse a client-supplied header. Use nginx's realip module: set_real_ip_from lists the proxy addresses you trust, real_ip_header names the header, and real_ip_recursive on makes $remote_addr the last address that did not come from a trusted hop.

open as a page

An nginx `upstream` block uses `ip_hash;` and one backend receives most of the traffic from a large corporate customer while the others sit idle. What exactly does nginx hash, and why does that produce the skew?

level: middleimportance: nice to knowfreq 36%

basics

~10 s

Nginx's ip_hash keys on the first three octets of the client's IPv4 address, or the whole IPv6 address. Everyone behind one corporate NAT egress address therefore hashes identically and lands on a single backend.

open as a page