skip to content

Why does http.Get fail against an httptest.NewTLSServer URL, and what fixes it?

level: middleimportance: nice to knowfreq 32%

answer

  1. the URL scheme is not http
  2. nobody signed that certificate
  3. the server can hand you a client
  4. srv.Client() trusts exactly one certificate
  5. skipping verification is the wrong repair

basics

~20 s

The test server serves an https URL with its own certificate, which no system trust store knows, so the default client rejects it. Use the client the server hands you: srv.Client() already trusts that certificate.

solid answer

~40 s

`httptest.NewTLSServer` starts a real HTTPS server on loopback and signs its own certificate, so `srv.URL` is an `https://127.0.0.1:port` address whose issuer is not in the system trust store. `http.Get` therefore fails verification. The fix is `srv.Client()`, which returns an `*http.Client` whose transport is preconfigured to trust exactly that server's certificate — so TLS verification stays on and only this one server is trusted. Do not reach for `InsecureSkipVerify`: it disables verification globally for that transport, which means your test would keep passing if the code under test silently dropped certificate checking. If the code under test builds its own client, inject the transport or the root pool — `srv.Certificate()` gives you the leaf certificate to put in an `x509.CertPool`. Always `defer srv.Close()` so the listener and idle connections go away with the test.

code

go · 13 lines
go
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusPaymentRequired)
	io.WriteString(w, `{"error":{"code":"card_declined"}}`)
}))
defer srv.Close()

sdk := payments.New(srv.URL, srv.Client()) // base URL and client are injected

_, err := sdk.Charge(context.Background(), "ch_1")
if err == nil {
	t.Fatal("want an error for a declined card")
}

go deeper

for a junior

Recall that a TLS test server's URL is https and its certificate is self-signed, and that the server itself gives you a client that already trusts it.

for a middle

Explain what srv.Client() changes — one extra trusted root on that client's transport, with verification still enabled — and contrast it with disabling verification outright.

for a senior

Demonstrate that testability here is a design property: an outbound client must accept its base URL and HTTP client. Also cover server lifecycle and why a skipped Close leaks listeners across a package's tests.

for a principal

Own the standard: no InsecureSkipVerify anywhere in the repository including tests, injectable transports for every outbound client, and a small separate suite that talks to the real endpoint rather than pretending a test server proves it.

## What the TLS test server actually starts `httptest.NewTLSServer(handler)` is `httptest.NewServer` plus TLS. It: 1. listens on `127.0.0.1` on a free port, 2. generates/uses a certificate the `httptest` package ships, valid for `example.com`, `127.0.0.1` and `::1`, 3. serves your handler over HTTPS, and 4. returns a `*httptest.Server` whose `URL` field starts with `https://`. That certificate is signed by nothing a machine trusts. So the moment a default client dials it: ``` Get "https://127.0.0.1:53211/v1/charges": tls: failed to verify certificate: x509: certificate signed by unknown authority ``` This is TLS working, not TLS broken. The client has no reason to believe a self-signed certificate from a random port. ## The right fix: `srv.Client()` `(*httptest.Server).Client()` returns an `*http.Client` whose transport is already configured with the server's certificate as a trusted root. Verification is still fully on — hostname checking, expiry, chain building — it simply has one extra root that applies to this client only. Nothing global is mutated and no other server becomes trusted. ```go srv := httptest.NewTLSServer(handler) defer srv.Close() res, err := srv.Client().Get(srv.URL + "/v1/charges/ch_1") ``` There is a second accessor for the case where you cannot use that client wholesale: `(*httptest.Server).Certificate()` returns the server's `*x509.Certificate`, which you can add to an `x509.CertPool` and hand to your own transport's `tls.Config.RootCAs`. ## The wrong fix, and why it matters The reflex is `tls.Config{InsecureSkipVerify: true}`. It makes the test pass, and it makes the test worthless in one specific way: the test can no longer tell the difference between a client that verifies certificates correctly and one that does not. If the SDK under test is *supposed* to pin a CA, or is supposed to fail closed on a bad certificate, a test wired with `InsecureSkipVerify` will happily pass either way. Worse, the pattern travels: the same snippet gets copied out of the test into the production dialer. A related smell is mutating `http.DefaultTransport` or `http.DefaultClient` inside a test to trust the test server. That is global state in a package that other tests share, and it survives into whatever runs next in the same binary — including tests running in parallel. ## Designing the code under test so this is easy A client wrapper around a third-party API should take two things from its caller: a **base URL** and an **`*http.Client`** (or an `http.RoundTripper`). With those, the test becomes trivial: ```go sdk := payments.New(srv.URL, srv.Client()) ``` If the wrapper hardcodes the vendor's hostname or builds its own client internally, no amount of `httptest` cleverness helps and you end up rewriting DNS or patching globals. Reviewers should treat "base URL and HTTP client are injectable" as a testability requirement for any outbound client, not a nicety. ## Lifecycle: `Close` is not optional `defer srv.Close()` shuts the listener down and blocks until outstanding requests have completed, then closes idle connections. Leaving it out leaks a listener and goroutines for the life of the test binary, and in a package with many tests that is a slow accumulation you eventually notice as flakiness or exhausted file descriptors. If you need to configure the server before it starts — a custom `tls.Config`, HTTP/2 — use `httptest.NewUnstartedServer(handler)`, adjust `srv.TLS` or `srv.Config`, then call `srv.StartTLS()`. ## What this does and does not prove A test through a TLS test server proves your client can talk HTTPS to *something*, that it sends the right method, path, headers and body, and that it handles the status codes and payloads the handler returns. It does not prove anything about the vendor's real certificate chain, their TLS version support, or their behaviour under load. Those belong to a contract or smoke test against the real endpoint, run somewhere other than your unit suite.

  • Why is InsecureSkipVerify a bad way to make that test pass?
    It switches off certificate verification for that transport, so the test can no longer distinguish a client that validates certificates from one that does not — including the code under test regressing. It also gets copied out of tests into production dialers. `srv.Client()` keeps verification on and trusts exactly one certificate.
  • The SDK under test builds its own *http.Client internally. How do you test it against a TLS test server?
    You change the SDK: accept a base URL and an `*http.Client` or `http.RoundTripper` from the caller. Testability of an outbound client is a design property, not something a test helper can retrofit. Failing that, add `srv.Certificate()` to an `x509.CertPool` and inject the transport.
  • How do you customise a test server's TLS configuration before it starts serving?
    Use `httptest.NewUnstartedServer(handler)`, which returns a server that is not listening yet. Adjust `srv.TLS` or `srv.Config`, then call `srv.StartTLS()` (or `srv.Start()` for plain HTTP). Still `defer srv.Close()`.

The test server is a shop with a handwritten ID card. The default client refuses it, as it should; srv.Client() is a visitor badge that says "trust this one shop", not a blindfold that says "trust everyone".

saying these in an interview costs you the question

  • Setting InsecureSkipVerify to make the test pass
  • Mutating http.DefaultTransport so the test server is trusted globally
  • Assuming the test server's certificate is in the system trust store
  • Hardcoding a vendor hostname so no test server can stand in
  • Leaving out defer srv.Close() and leaking listeners