skip to content

A partner CSV fails in Go with csv.ErrBareQuote. What does setting Reader.LazyQuotes change, and what does it cost?

level: seniorimportance: should knowfreq 36%

answer

  1. the reader can be told to stop objecting
  2. two quote sentinels, one switch
  3. you trade an error for a guess
  4. a runaway quote can swallow the next rows

basics

~20 s

LazyQuotes makes csv.Reader accept a quote inside an unquoted field and an undoubled quote inside a quoted field instead of failing. The cost is the signal: a malformed file now parses into fields that may be split wrongly, silently.

solid answer

~50 s

By default `csv.Reader` rejects two quoting mistakes: a quote appearing in a field that did not start with one, reported as `csv.ErrBareQuote`, and a stray or missing quote inside a quoted field, reported as `csv.ErrQuote`. Both arrive wrapped in a `*csv.ParseError` carrying `StartLine`, `Line` and `Column`. Setting `LazyQuotes = true` tells the reader to take those bytes literally instead of erroring. What you give up is the only warning you had that the file is broken: an unterminated quote read leniently can absorb the following lines into one field, and the row you import is quietly wrong rather than loudly rejected. My preference is to keep the errors, quarantine the failing records with their line numbers, and get the export fixed upstream. If leniency is unavoidable, I keep `FieldsPerRecord` enforcing a column count so a swallowed row still surfaces as `csv.ErrFieldCount`.

code

go · 20 lines
go
r := csv.NewReader(f)
r.TrimLeadingSpace = true // the usual real cause, not LazyQuotes

for {
	rec, err := r.Read()
	if err == io.EOF {
		break
	}
	var pe *csv.ParseError
	if errors.As(err, &pe) {
		log.Printf("rejected record at line %d, col %d: %v", pe.StartLine, pe.Column, pe.Err)
		continue
	}
	if err != nil {
		return err
	}
	if err := importRow(rec); err != nil {
		return err
	}
}

go deeper

for a junior

Know that the reader rejects certain quoting mistakes by default and that a flag exists to relax that. Do not set it without understanding what the file actually contains.

for a middle

Name the two sentinels the flag suppresses, and explain that relaxing them makes the reader guess at field boundaries rather than repair anything.

for a senior

Show the diagnosis first: read the position out of the *csv.ParseError, look at the bytes, distinguish a leading-space problem from a genuine quoting problem, and quarantine failing records instead of aborting the import.

for a principal

Own the stance on bad partner data: what the pipeline accepts silently, what it quarantines, what it rejects outright, and how a rejects report gets the upstream exporter fixed rather than absorbing the breakage forever.

## The two errors this knob suppresses `encoding/csv` defines sentinel errors for quoting problems: - `csv.ErrBareQuote` — a quote character appears inside a field that did not begin with a quote. - `csv.ErrQuote` — an extraneous or missing quote inside a quoted field. Each is delivered inside a `*csv.ParseError`, whose fields are `StartLine` (the line where the record began), `Line` and `Column` (where the failure was detected), and `Err` (the sentinel). Because `*csv.ParseError` implements `Unwrap`, `errors.Is(err, csv.ErrBareQuote)` works, and `errors.As(err, &pe)` gets you the positions. `LazyQuotes bool` on the `Reader` is the single switch that turns both of those into non-events: with it set, a quote may appear in an unquoted field, and a non-doubled quote may appear in a quoted field, and the reader keeps the characters rather than complaining. ## What the leniency actually does to your data This is the part to be precise about, because "it just works now" is the wrong summary. When the reader stops objecting it does not repair anything — it makes a guess about where fields end. Quotes that were meant as data end up embedded in values. Worse, an unbalanced quote read leniently can keep consuming past the end of the line, pulling the following record or records into a single field. The import then contains one row where there should have been three, and each has moved into the wrong column. Nothing is logged, because you asked for silence. So the trade is explicit: an error you can act on, exchanged for a value you cannot verify. ## The defensive combination If you must enable it — a long-standing partner whose exporter you cannot influence, and a deadline — pair it with the shape check. Leave `FieldsPerRecord` at 0 or set it to the known column count. A row swallowed by a runaway quote produces a record with the wrong number of fields, and that surfaces as `csv.ErrFieldCount` even though quoting complaints are switched off. Leniency about quoting plus strictness about shape catches the dangerous cases while letting the merely ugly ones through. ## Diagnose before you reach for it When a customer's export fails, the first job is to see the actual bytes at the reported position, not to loosen the parser. `*csv.ParseError` gives you the line, and `Reader.FieldPos(i)` gives the line and column where field *i* of the record you just read began, which is what you put in a rejects report the partner can act on. And check whether `LazyQuotes` is even the right knob. The most common false diagnosis is a file written with a space after every comma: ``` id, "name, with comma", amount ``` Because the field starts with a space rather than a quote, the reader treats it as unquoted, and the quote inside it is a bare quote — `csv.ErrBareQuote`. The right fix is `TrimLeadingSpace = true`, which makes the quote start the field and the record parse correctly. Turning on `LazyQuotes` here would "work" in the sense of not erroring, while leaving literal quote characters in your values and leaving the embedded comma splitting the field in two. Two knobs, similar symptom, opposite outcomes. Other knobs worth checking in the same pass: `Comma` if the file is semicolon- or tab-separated, and `Comment` if the export prefixes metadata lines with `#`. ## Streaming so you can quarantine All of this presumes you are driving `Read` in a loop. `ReadAll` gives you one verdict for the whole file, so a single bad record aborts an otherwise good import and you learn nothing about the rest. Reading record by record lets you keep the good rows, write the failures to a rejects file with `StartLine` and the sentinel, and report both counts at the end. That report is what actually gets a partner's exporter fixed — which is the only permanent resolution, since every lenient parse is a decision made on your side about data you cannot see. ## The position to hold `LazyQuotes` is not a repair, a compatibility mode, or a sensible default. It is a decision to stop being told that a file is malformed. Turn it on for a specific input, with a comment naming that input and why, and with a shape check behind it — never globally in a shared reader constructor where the next caller inherits the silence.

  • The file has a space after every comma and quoted fields fail. Is LazyQuotes the right knob?
    No — that is TrimLeadingSpace. With leading whitespace the field does not begin with a quote, so the reader sees a bare quote inside an unquoted field. Setting TrimLeadingSpace = true makes the quote start the field and the record parses properly. LazyQuotes would only silence the complaint, leaving literal quotes in the values and the embedded comma still splitting the field.
  • How do you report exactly where a quoting failure happened?
    Use errors.As to get the *csv.ParseError: StartLine is where the record began, Line and Column locate the failure, and Err is the sentinel — csv.ErrBareQuote or csv.ErrQuote. Reader.FieldPos(i) gives the line and column of field i in the record you just read. Those numbers are what make a rejects report actionable for whoever produced the file.
  • What stops LazyQuotes from silently merging rows?
    Keep the shape check on. An unterminated quote read leniently can absorb following lines into one field, and the resulting record then has the wrong number of fields, which surfaces as csv.ErrFieldCount even with quote errors suppressed. Pair leniency about quoting with strictness about column count, and quarantine whatever fails.

saying these in an interview costs you the question

  • Enables LazyQuotes globally as a default
  • Says LazyQuotes repairs or fixes the file
  • Confuses a leading-space failure with a quoting failure
  • Cannot say where the failing line number comes from
  • Aborts the whole import for one malformed record