skip to content

In database/sql, how do you Scan rows whose column list is only known at runtime?

level: seniorimportance: nice to knowfreq 24%

answer

  1. ask the result set what it returned
  2. Scan is variadic, so build the arguments
  3. two slices: values, and pointers to them
  4. no conversion means driver types come through
  5. reused destinations must be copied out

basics

~10 s

Call rows.Columns() for the names, allocate a []any of values plus a second []any holding a pointer to each value, and pass that second slice to rows.Scan with a spread. Type-switch on what lands.

solid answer

~50 s

`(*sql.Rows).Columns() ([]string, error)` gives you the result set's column names after the query has run, and `Scan` is variadic over `any`, so the dynamic form is to build the destination slice yourself: allocate `vals := make([]any, len(cols))` for the values and `ptrs := make([]any, len(cols))` where `ptrs[i] = &vals[i]`, then call `rows.Scan(ptrs...)` each iteration. After each `Scan`, `vals[i]` holds the driver's value for `cols[i]`. Scanning into `*any` performs no conversion, so what you get is one of the driver value types — typically `int64`, `float64`, `bool`, `string`, `[]byte`, `time.Time` or `nil` — and you type-switch to render it. When you need more than the name, `rows.ColumnTypes()` returns `*sql.ColumnType` values carrying `DatabaseTypeName` and `ScanType`. This is the right tool for a generic export or an admin query runner, and the wrong one for ordinary application code.

code

go · 20 lines
go
cols, err := rows.Columns()
if err != nil {
	return nil, err
}
vals := make([]any, len(cols))
ptrs := make([]any, len(cols))
for i := range vals {
	ptrs[i] = &vals[i]
}

var out [][]any
for rows.Next() {
	if err := rows.Scan(ptrs...); err != nil {
		return nil, err
	}
	row := make([]any, len(cols))
	copy(row, vals) // vals is reused on the next iteration
	out = append(out, row)
}
return out, rows.Err()

go deeper

for a junior

Know that rows.Columns() reports the result set's column names and that rows.Scan takes a variable number of pointers, so the destinations can be built as a slice and spread with three dots.

for a middle

Explain the two-slice construction and why one slice is not enough: Scan writes through pointers, so each element of the argument slice must be the address of an element of the value slice.

for a senior

Show the traps you would catch in review: reused destinations that must be copied per row, driver values arriving as []byte rather than string, ColumnTypes metadata being advisory, and the fact that rows.Err still has to be checked.

for a principal

Own the decision to allow this at all. A tool that runs arbitrary queries is an access-control and auditing question first, and you should be able to say where the dynamic path is permitted and where typed scanning stays mandatory.

## When the select list is not known until runtime Most application code knows its columns at compile time and scans into named variables. A few things do not: a CSV or JSON exporter over an arbitrary `SELECT`, an internal admin query runner, a migration verification tool, a test helper that dumps a table. For those, `database/sql` has exactly the API you need — it is just less obvious, because `Scan` is variadic. ### `Columns` and the two-slice trick `(*sql.Rows).Columns() ([]string, error)` returns the column names of the result set. It is callable after the query has run and errors if the rows are already closed. With the count in hand: ``` cols, err := rows.Columns() if err != nil { return err } vals := make([]any, len(cols)) ptrs := make([]any, len(cols)) for i := range vals { ptrs[i] = &vals[i] } for rows.Next() { if err := rows.Scan(ptrs...); err != nil { return err } // vals[i] is now the value of column cols[i] for this row } return rows.Err() ``` Two slices are needed because `Scan` writes **through** pointers. `vals` holds the values; `ptrs[i]` is `&vals[i]`, a `*any`, boxed into the `any` element type of `ptrs`. The `ptrs...` spread turns the slice into the variadic argument list. A single slice will not do: passing `vals...` hands `Scan` the values, not their addresses, and it rejects them as unsupported destinations. Note that both slices can be allocated once outside the loop and reused, since each `Scan` overwrites every element. If you keep the row — appending it to a result set — you must **copy** `vals` per iteration, or every stored row ends up aliasing the same backing array and holding the final row's values. That is the single most common bug in this pattern. ### What lands in an `*any` Scanning into `*any` asks for no conversion: you receive the value the driver produced. In practice that is one of a small set — `int64`, `float64`, `bool`, `string`, `[]byte`, `time.Time`, or `nil` for SQL NULL. Byte slices are copied into your destination rather than aliasing the driver's own buffer, so a value you keep after the next `Next` is safe. Because the concrete type varies by driver and by column, rendering means a type switch: ``` switch v := vals[i].(type) { case nil: // NULL case []byte: // text or binary the driver did not decode case int64, float64, bool, string, time.Time: _ = v } ``` The `[]byte` case is the one that surprises people: several drivers hand back raw bytes for text columns rather than a `string`, so a generic exporter that only handles `string` will print byte-slice noise for half its columns. ### When names are not enough: `ColumnTypes` `(*sql.Rows).ColumnTypes() ([]*sql.ColumnType, error)` returns richer metadata per column: `Name()`, `DatabaseTypeName()` (the driver's own type name, such as an integer or timestamp type), `ScanType()` (the Go type the driver would prefer), `Nullable()` and length/precision information where the driver supports it. Not every driver populates every method — several return a zero value and a `false` "ok" — so treat the metadata as advisory. Use it when you need to format output faithfully (dates versus numbers) rather than dumping everything as text. ### Everything else about the loop still applies The dynamic form changes only the destinations. You still check the error from `QueryContext`, still `defer rows.Close()`, still check `rows.Err()` after the loop, and `Scan` still errors if the number of destinations does not match the number of columns — which here cannot happen by construction, since you sized the slice from `Columns()`. That is a small, genuine advantage of the pattern over a hand-written select list. ### When not to reach for it This is deliberately unidiomatic for ordinary code. You lose compile-time type checking, you lose the discipline of an explicit select list, and you gain a type switch at every use site. If a caller knows its columns, it should scan into named variables. Reserve the dynamic form for tools whose whole purpose is to handle a query nobody wrote in advance — and be aware that an endpoint accepting arbitrary SQL is an authorisation decision long before it is a scanning problem.

  • Why can't you pass the value slice directly to Scan with a spread?
    Because `Scan` writes through its arguments and needs addresses. Spreading a `[]any` of values hands it the values themselves, which it rejects as unsupported destination types. The second slice exists solely so each element is a `*any` pointing into the first, which is what makes `rows.Scan(ptrs...)` legal and useful.
  • What concrete Go types can appear in a value scanned into *any?
    The driver's own value types, with no conversion applied: typically `int64`, `float64`, `bool`, `string`, `[]byte`, `time.Time`, or `nil` for NULL. `[]byte` is the one that catches people out, since several drivers return raw bytes for text columns, so a generic renderer must handle it explicitly rather than assuming `string`.
  • When would rows.ColumnTypes be worth calling instead of rows.Columns?
    When the output has to be formatted faithfully rather than dumped as text — telling a timestamp from a number, or deciding a column's width. `ColumnTypes` returns `*sql.ColumnType` values exposing `DatabaseTypeName`, `ScanType` and `Nullable`. Treat it as advisory: several drivers leave parts of it unpopulated and report the "ok" result as false.

saying these in an interview costs you the question

  • Passes the value slice to Scan instead of pointers to its elements
  • Appends the reused value slice per row without copying it
  • Assumes text columns always scan into a string, never []byte
  • Thinks database/sql maps result columns onto struct fields
  • Uses the dynamic form in ordinary code that knows its columns
  • Skips rows.Err after the loop because the pattern looks generic