skip to content

Data Encoding and Storage

Turning Go values into bytes and rows and back: json tags and streaming, the marshaler interfaces, XML, CSV, gob and binary, database/sql's pool and transactions, and decoding hostile input.

part ofGo (Golang)overview, primer and where to startread it →
on this pageshow

explore

questions

119 · 5 sections

What concrete Go types does json.Unmarshal produce when the target is an `any`?

level: juniorimportance: must knowfreq 72%
basics
~10 s

Six concrete types, and no others: map[string]any for a JSON object, []any for an array, string for a string, float64 for every number, bool for true and false, and an untyped nil for null.

open as a page

In Go, what does implementing MarshalJSON on a type change about how encoding/json encodes it?

level: juniorimportance: must knowfreq 62%
basics
~10 s

Implementing MarshalJSON() ([]byte, error) makes a type satisfy json.Marshaler, so encoding/json calls that method instead of walking the type's fields, and splices the JSON bytes it returns into the output.

open as a page

In encoding/json, what happens to the emitted JSON key when you rename a struct field?

level: juniorimportance: must knowfreq 70%
basics
~20 s

Without a json tag, encoding/json emits the Go field name as the key, so renaming the field renames the key and old readers see nothing. A json tag pins the wire name, freeing the Go field to be renamed.

open as a page

In encoding/json, what does the omitempty struct tag option omit, and what does it leave in?

level: juniorimportance: must knowfreq 78%
basics
~20 s

In encoding/json, omitempty skips a field whose value is false, 0, an empty string, a nil pointer or interface, or an empty array, slice or map. It never skips a zero-valued struct such as time.Time{}, and it only affects encoding.

open as a page

When should you use json.NewDecoder(r).Decode(&v) instead of json.Unmarshal on a []byte?

level: juniorimportance: must knowfreq 72%
basics
~20 s

Use json.NewDecoder when the JSON arrives as a stream, such as an HTTP body or a file, so it is decoded as bytes arrive. Use json.Unmarshal when you already hold the whole document in memory as a []byte.

open as a page

Why does adding a String() method to a type leave json.Marshal's output unchanged?

level: juniorimportance: must knowfreq 60%
basics
~10 s

fmt.Stringer is consulted only by the fmt package. encoding/json never looks for String; it looks for json.Marshaler and then encoding.TextMarshaler, and a type with neither is encoded from its structure, field by field.

open as a page

Why does a time.Duration field marshal to a bare number in encoding/json, and how do you get "1m30s"?

level: middleimportance: must knowfreq 62%
basics
~20 s

time.Duration is a named int64 counting nanoseconds and implements no marshaler interface, so encoding/json encodes the underlying integer: 90*time.Second becomes 90000000000. To emit "1m30s", define your own duration type with MarshalText and an UnmarshalText that calls time.ParseDuration.

open as a page

What does implementing encoding.TextMarshaler and TextUnmarshaler change about a type's encoding/json output?

level: juniorimportance: should knowfreq 52%
basics
~20 s

encoding/json writes the type as one JSON string built from the bytes MarshalText returns, instead of encoding its fields or its underlying number. Decoding hands the unquoted bytes back to UnmarshalText. json adds the quotes and escaping itself.

open as a page

In encoding/json, what may a map's key type be, and what does MarshalText have to do with it?

level: middleimportance: should knowfreq 38%
basics
~20 s

JSON object keys are strings, so encoding/json accepts a map key type only if it is a string kind, an integer kind, or implements encoding.TextMarshaler. Anything else fails with an unsupported-type error. Decoding needs the mirror: a string, an integer, or TextUnmarshaler.

open as a page

Why does fmt.Println(v) skip a String() method declared with a pointer receiver?

level: middleimportance: should knowfreq 44%
basics
~20 s

Only the pointer type's method set contains a pointer-receiver method, so the value copy stored in fmt's interface argument does not satisfy fmt.Stringer. fmt cannot take the address of that copy, so it falls back to default formatting.

open as a page

Why does a []byte struct field marshal to a base64 string in encoding/json?

level: juniorimportance: must knowfreq 55%
basics
~20 s

JSON has no type for raw bytes, so encoding/json marshals a []byte field as a base64 string using the standard padded alphabet. A nil []byte becomes null and an empty one becomes an empty string.

open as a page

What do binary.BigEndian.Uint32 and binary.LittleEndian.Uint32 return for the bytes 00 00 01 00?

level: juniorimportance: must knowfreq 45%
basics
~10 s

binary.BigEndian.Uint32 reads the most significant byte first and returns 256. binary.LittleEndian.Uint32 reads the least significant byte first and returns 65536. Same four bytes, opposite orders, different numbers.

open as a page

Why must a gzip.Writer be closed before the file is valid, and what breaks if its Close error is dropped?

level: juniorimportance: must knowfreq 48%
basics
~20 s

A gzip.Writer buffers compressed data and writes the gzip trailer, a CRC32 plus the uncompressed length, only on Close. Skip Close and the file is truncated; ignore Close's error and a failed final write vanishes silently.

open as a page

In Go's encoding/csv, when would you use Reader.Read in a loop instead of Reader.ReadAll?

level: juniorimportance: must knowfreq 55%
basics
~20 s

ReadAll builds the whole file as one [][]string in memory and returns nil records if any row fails to parse. Read hands back one record per call and ends with io.EOF, so large or partly malformed files stay workable.

open as a page

What is Go's encoding/gob package, and how do you encode and decode a value with it?

level: juniorimportance: must knowfreq 45%
basics
~20 s

encoding/gob is Go's own binary serialization format. Wrap a writer with gob.NewEncoder and call Encode to send a value; wrap a reader with gob.NewDecoder and call Decode with a pointer to receive it. Only exported fields travel.

open as a page

In a Go HTTP handler, what does http.MaxBytesReader do when a client sends a body larger than the limit?

level: juniorimportance: must knowfreq 58%
basics
~20 s

http.MaxBytesReader wraps a request body so any read past the given byte limit fails with a *http.MaxBytesError instead of continuing. It caps what one request can allocate, but the handler must still write the 413 response itself.

open as a page

Why does json.Unmarshal leave a struct field at zero when a key in the JSON is misspelled, and return no error?

level: juniorimportance: must knowfreq 60%
basics
~20 s

encoding/json skips any JSON object key that matches no field of the destination struct. A misspelled key is discarded, the field keeps its zero value, and Unmarshal returns nil. Silence is the documented default, not a bug.

open as a page

Why does json.Unmarshal succeed when a required field is missing from the JSON input?

level: juniorimportance: must knowfreq 74%
basics
~20 s

encoding/json has no required-field concept. A key absent from the input is simply never assigned, so the struct field keeps its zero value and Unmarshal returns nil. Requiredness is a rule your own code must check after decoding.

open as a page

Why is checking r.ContentLength not a safe size cap on an untrusted HTTP request body in Go?

level: middleimportance: should knowfreq 45%
basics
~20 s

r.ContentLength only reports what the client declared. A chunked request declares nothing and arrives as -1, so a length check silently passes exactly the requests whose size is unknown. Only a cap on bytes actually read bounds the allocation.

open as a page

What does json.Decoder.DisallowUnknownFields change about a Decode call that reads an unexpected key?

level: middleimportance: should knowfreq 50%
basics
~20 s

It makes a key that matches no field of the destination struct an error instead of a silent skip. Decode then returns an error naming the first such key, for example: json: unknown field "maxConnnections".

open as a page

Why does scanning a SQL NULL into a Go string variable fail in database/sql, and what do you scan into instead?

level: juniorimportance: must knowfreq 70%
basics
~20 s

A Go string has no value that means absent, so Scan returns an error saying converting NULL to string is unsupported. Declare the variable as sql.NullString or as *string, then check Valid or check for nil before using it.

open as a page

In Go's database/sql, why pass values as arguments after the query string instead of fmt.Sprintf-ing them into the SQL?

level: juniorimportance: must knowfreq 82%
basics
~20 s

database/sql never splices arguments into the SQL text. The statement and the values reach the driver separately, so the database parses the query before it sees any value. Anything you fmt.Sprintf in becomes SQL syntax rather than data.

open as a page

Does sql.Open actually connect to the database, and how do you prove connectivity?

level: juniorimportance: must knowfreq 72%
basics
~10 s

No. sql.Open only validates its arguments and hands back a connection-pool handle; it usually dials nothing. Call db.PingContext with a timeout to force a real connection and prove the database is reachable.

open as a page

In Go's database/sql, what does db.PrepareContext return and why must you Close it?

level: juniorimportance: must knowfreq 52%
basics
~20 s

It returns a *sql.Stmt, a handle to a statement the database server has already parsed and is holding open. Stmt.Close discards it on every pooled connection it was prepared on; skip that Close and the server-side state stays.

open as a page

In Go's database/sql, how do you run a multi-row query and read every row?

level: juniorimportance: must knowfreq 80%
basics
~10 s

Call db.QueryContext, defer rows.Close(), then loop while rows.Next() calling rows.Scan with a pointer per selected column, and check rows.Err() after the loop. Next returning false means end of data or a failure.

open as a page