skip to content

Parsing and Escaping URLs

url.Parse gives a URL whose Path is already decoded while RawPath keeps the escapes, and Query returns a copy you must re-encode after editing. Open-redirect questions live on that distinction.

part ofGo (Golang)overview, primer and where to startread it →
on this pageshow

questions

4

In Go, why build a query string with url.Values.Encode instead of concatenating the parameters yourself?

level: juniorimportance: must knowfreq 60%

answer

  1. a value can contain your separators
  2. who escapes the ampersand and the equals
  3. one map, several values per key
  4. output order is sorted, not insertion
  5. a space in a query is not %20

basics

~10 s

url.Values.Encode escapes every key and value with url.QueryEscape, so an ampersand or equals sign inside a value cannot invent a new parameter. It also carries repeated keys and emits the pairs sorted by key.

solid answer

~40 s

`url.Values` is a `map[string][]string` with `Set`, `Add`, `Get`, `Del` and `Encode`. `Encode` renders the map as a query string, percent-escaping each key and each value with `url.QueryEscape` and joining pairs with `&`. That matters because a value is arbitrary text: if a user's `state` value contains `&role=admin`, hand-built concatenation silently creates a second parameter, while `Encode` writes `%26role%3Dadmin` inside the one value. `Encode` also handles a repeated key (`Add` twice) and writes its output sorted by key, so the same values always produce the same string. You attach it with `u.RawQuery = v.Encode()`; `u.String()` then emits `RawQuery` exactly as you set it. The one surprise is that `QueryEscape` writes a space as `+`, not `%20` — correct in a query, wrong in a path, where `url.PathEscape` is the right function.

code

go · 7 lines
go
u := &url.URL{Scheme: "https", Host: "api.example.test", Path: "/callbacks/return"}
v := url.Values{}
v.Set("state", "a&b=c")
v.Set("note", "héllo wörld")
u.RawQuery = v.Encode()
fmt.Println(u.String())
// https://api.example.test/callbacks/return?note=h%C3%A9llo+w%C3%B6rld&state=a%26b%3Dc

go deeper

for a junior

Be ready to build a query in front of the interviewer: make a url.Values, Set the pairs, assign Encode's output to u.RawQuery. Know that Encode escapes each key and value for you.

for a middle

Explain the mechanics: Encode sorts by key, escapes each part separately with url.QueryEscape, and writes a space as a plus. Say why per-part escaping is the only correct order of operations.

for a senior

Show the judgment about when Encode is wrong: it canonicalises, so re-encoding somebody else's query can change bytes you were relying on. Know that Query() is lenient and drops malformed pairs silently.

for a principal

Own the convention across services: URLs your teams build should always come from url.Values so cache keys and logged URLs are stable, and any signing scheme should define its canonical form once rather than assuming Encode's ordering forever.

## What `url.Values` actually is `url.Values`, from the standard library's `net/url` package, is a named map type: `map[string][]string`. The value is a slice because a query string is allowed to repeat a key — `tag=go&tag=http` is two values under `tag`. The API on it is deliberately tiny: - `Get(key)` returns the **first** value for a key, or `""` if the key is absent — it cannot distinguish "missing" from "present but empty". - `Set(key, value)` replaces every value for the key with one value. - `Add(key, value)` appends another value under the key. - `Del(key)` removes the key entirely. - `Has(key)` reports whether the key is present at all, which is how you tell an empty value from a missing one. - `Encode()` renders the whole map as a query string. Because it is a plain map, you may also index it directly (`v["tag"]`) to read every value under a key. ## What `Encode` does that concatenation does not `Encode` walks the keys in **sorted order** and, for each value under each key, writes `url.QueryEscape(key)`, then `=`, then `url.QueryEscape(value)`, joining the pairs with `&`. Three consequences follow. **Escaping happens per component.** The characters that give a query its structure — `&`, `=`, `?`, `#`, `+` — are exactly the characters that must be escaped when they appear *inside* a value. If you write `"?state=" + state` and the value is `abc&role=admin`, you have not sent one parameter, you have sent two, and the second one is whatever the caller chose. Escaping the assembled string afterwards is not a fix either: that escapes your own separators too, turning the whole query into one meaningless value. Escaping is a per-part operation, and `Encode` is what performs it per part. **The output is deterministic.** Map iteration order in Go is randomised, so a naive `for k, v := range values` would produce a different string on every run. `Encode` sorts the keys before writing, which is what makes rendered URLs stable across runs — good for tests, for cache keys and for logs. The flip side is that it will not reproduce the order in which you called `Set`, and it will not reproduce the order a string you parsed originally used. **Non-ASCII text is UTF-8 percent-encoded.** `QueryEscape` encodes the *bytes* of the UTF-8 string: `é` (U+00E9) becomes `%C3%A9`. Only the unreserved characters — letters, digits, `-`, `_`, `.` and `~` — survive unescaped. ## The plus sign `url.QueryEscape` encodes a space as `+`. That comes from HTML form encoding (`application/x-www-form-urlencoded`), and it is correct inside a query string: `url.QueryUnescape` turns `+` back into a space. It is **not** correct inside a path. `url.PathUnescape` deliberately does not decode `+`, so a `+` you put into a path segment stays a literal plus character forever. That is why the package ships two escape functions rather than one: use `url.QueryEscape` (or, better, `Values.Encode`) for query values and `url.PathEscape` for a single path segment, where a space becomes `%20`. ## Attaching values to a URL The idiomatic build is to construct or parse a `*url.URL`, then assign the encoded query: ```go u := &url.URL{Scheme: "https", Host: "api.example.test", Path: "/callbacks/return"} v := url.Values{} v.Set("state", state) u.RawQuery = v.Encode() ``` Two details are worth knowing here. `u.String()` writes `RawQuery` **verbatim** — it does not re-escape it, so whatever `Encode` produced is what goes on the wire. And `u.Query()` *parses* `RawQuery` and returns a fresh `url.Values`; mutating that map changes nothing about the URL until you assign `u.RawQuery = v.Encode()` again. `Query()` is also lenient: it silently discards pairs it cannot parse rather than reporting an error. ## Where `Encode` is the wrong tool Because `Encode` sorts and re-escapes, it is a *canonicaliser*, not a faithful copier. If you are verifying a signature or a hash over a URL somebody else produced, running their query through `Query()` and `Encode()` can change the bytes — different order, `+` where they sent `%20` — and the comparison fails. For that job, work with the raw query string you received. For building URLs, which is nearly always what you are doing, `Encode` is the correct and safe default.

  • What does url.Values.Encode do with a space inside a value?
    It writes a `+`, because `url.QueryEscape` uses form encoding. `url.QueryUnescape` decodes `+` back to a space, so the round trip inside a query is fine. In a path segment you want `url.PathEscape`, which writes `%20`; `url.PathUnescape` never decodes `+`, so a plus placed in a path stays a literal plus.
  • How do you send the same key twice, and how do you read all of its values back?
    Call `Add` once per value — `Set` replaces everything under the key. On the way back, `Get` returns only the first value, so index the map directly: `v["tag"]` gives the whole `[]string`. `Has` tells you whether the key was present at all, which `Get` cannot, since a missing key and an empty value both return `""`.
  • After building url.Values, how do you attach them to a *url.URL?
    Assign `u.RawQuery = v.Encode()`. `u.String()` writes `RawQuery` verbatim, so that string is exactly what goes on the wire. Note that `u.Query()` returns a freshly parsed copy — mutating the map it returns has no effect on the URL until you encode it back into `RawQuery`.

url.Values.Encode is to a query string what parameter binding is to SQL: you hand the library the values separately and it does the quoting, so a value can never turn into structure.

saying these in an interview costs you the question

  • Builds query strings with fmt.Sprintf and user-supplied values
  • Escapes the whole assembled query once at the end
  • Says Encode preserves the order keys were set in
  • Thinks url.QueryEscape and url.PathEscape are interchangeable
  • Believes mutating the map from u.Query() updates the URL
  • Uses Get and concludes a key is missing when its value is empty
open as a page

What do url.URL.Path, url.URL.RawPath and the EscapedPath method each hold after url.Parse?

level: middleimportance: should knowfreq 42%

basics

~20 s

Path holds the decoded path. RawPath holds the original escaped path, but only when it differs from the default encoding of Path. EscapedPath returns RawPath when it is a valid encoding of Path, otherwise it re-encodes Path itself.

open as a page

Your validator calls url.Parse on a callback URL and compares u.Host to an allowlist — what slips through, and how does url.ParseRequestURI differ?

level: seniorimportance: should knowfreq 45%

basics

~20 s

url.Parse accepts relative references, so a string with no scheme and no host parses without error and leaves u.Host empty — the allowlist comparison then never runs against anything. url.ParseRequestURI accepts only an absolute URI or an absolute path.

open as a page

A round trip through url.Parse and url.Values.Encode broke our signed callback URLs — what does it not preserve?

level: seniorimportance: nice to knowfreq 30%

basics

~20 s

Parsing a query into url.Values and re-encoding it canonicalises the bytes: parameter order becomes sorted by key, a space sent as %20 comes back as +, unnecessary escapes are dropped, and a valueless key gains a trailing equals sign. A signature over the original bytes then fails.

open as a page