skip to content

Why can many goroutines call os.File.ReadAt on one handle but not Seek then Read?

level: middleimportance: should knowfreq 44%

answer

  1. where does the position live
  2. argument versus shared state
  3. two calls are not one operation
  4. the buffer must be filled or an error returned
  5. the write side adds a non-overlap condition

basics

~20 s

ReadAt takes the offset as an argument and neither reads nor moves the file's cursor, so parallel calls cannot disturb each other. Seek then Read is two operations over one shared cursor, and concurrent callers interleave them.

solid answer

~50 s

`io.ReaderAt` declares `ReadAt(p []byte, off int64) (n int, err error)`, and its contract says two things that matter here: `ReadAt` neither affects nor is affected by any underlying seek offset, and clients may execute parallel `ReadAt` calls on the same source. `*os.File` honours that by using the positional read system call, `pread`, which carries the offset with the request. `Seek` plus `Read` cannot make that promise: the cursor is one piece of state on the open file, so between one goroutine's `Seek` and its `Read` another goroutine can move the cursor, and both read the wrong bytes. `ReadAt` also fills the buffer — it must return a non-nil error whenever `n < len(p)`, so there is no short-read loop to write, and at a clean end of file with the buffer exactly filled the error is nil. `io.WriterAt` mirrors it: parallel `WriteAt` calls are defined only when the byte ranges do not overlap.

code

go · 12 lines
go
const recordSize = 64

// Safe to call from many goroutines on one *os.File:
// ReadAt neither uses nor moves f's cursor, and it fills buf
// or returns a non-nil error.
func readRecord(f *os.File, i int64) ([]byte, error) {
	buf := make([]byte, recordSize)
	if _, err := f.ReadAt(buf, i*recordSize); err != nil {
		return nil, err
	}
	return buf, nil
}

go deeper

for a junior

Know that ReadAt takes the offset as a parameter while Seek followed by Read relies on a position stored in the file, and that only the first is safe to call from several goroutines at once.

for a middle

Explain the contract clauses: independent of the cursor, parallel calls allowed, and a non-nil error whenever the buffer is not filled. Be able to contrast that last one with a short Read, which is normal and not an error.

for a senior

Argue the design choice — positional reads versus a lock around a cursor — in throughput terms, and reach for a per-caller io.SectionReader when a consumer demands a plain reader. Know that io.WriterAt's guarantee is conditional on non-overlapping ranges.

for a principal

Decide what the component publishes: a handle with a cursor invites callers to serialise, a positional interface lets them fan out. That choice sets the concurrency ceiling for everyone who imports the package, and it is expensive to revisit.

## Two ways to read from a position ```go // stateful: move the cursor, then read from it f.Seek(off, io.SeekStart) f.Read(buf) // stateless: the position travels with the request f.ReadAt(buf, off) ``` The difference is where the position lives. In the first form it is state on the open file, shared by everybody holding that `*os.File`. In the second it is an argument, so it is private to the call. ## What io.ReaderAt actually promises ```go type ReaderAt interface { ReadAt(p []byte, off int64) (n int, err error) } ``` The documented contract has four clauses worth memorising: 1. **It fills the buffer.** When `ReadAt` returns `n < len(p)`, it must return a non-nil error explaining why. This is the opposite of `Read`, which may legitimately return 3 bytes with a nil error and nothing wrong. Callers of `ReadAt` therefore do not write a retry loop. 2. **It is independent of the cursor.** `ReadAt` does not use and does not move any seek offset the source may have. 3. **Parallel calls are allowed** on the same source. This is the clause that makes one open file serve many concurrent readers. 4. **A short read at the end** returns `io.EOF`; a read that fills `p` exactly up to the last byte of the file returns a nil error, with `io.EOF` arriving on the next call. `*os.File` implements this on Unix with `pread(2)`, the positional read: the kernel takes the offset as a parameter and never touches the file description's offset. `*bytes.Reader`, `*strings.Reader` and `*io.SectionReader` implement it over content they can index directly. ## Why the stateful pair cannot be made safe by accident Even though each individual `Read` on an `*os.File` is internally serialised, that serialisation is per call. `Seek` and `Read` are two calls, and nothing binds them into one atomic unit. With two goroutines the interleaving `SeekA, SeekB, ReadA, ReadB` is entirely ordinary, and then goroutine A reads from B's offset and B reads from the end of what A wanted. The bytes are not corrupt, which is what makes it nasty: each caller gets a perfectly well-formed record, just the wrong one. You can fix the pair with a mutex covering both calls, and that is correct — but it serialises every lookup on a resource, the file, that the operating system is perfectly happy to serve in parallel. On an index file being probed by many requests at once, that mutex becomes the throughput ceiling for no reason. ## io.SectionReader: a cursor of your own When the consumer insists on an `io.Reader` — a parser, a decoder, something that reads sequentially — you do not have to hand it the shared file: ```go sec := io.NewSectionReader(f, off, length) ``` `io.NewSectionReader(r io.ReaderAt, off, n int64) *io.SectionReader` builds a view over a byte range of `r`. The section reader has **its own** offset, so reading and seeking it never touches `f`'s cursor, and it reports `Size()` and returns `io.EOF` at the end of the section rather than the end of the file. Create one per lookup or per goroutine: they are cheap, they need no extra file descriptor, and they turn a shared mutable cursor into per-caller state. What a single `*io.SectionReader` does *not* support is two goroutines calling its `Read` at once — it has a cursor, and cursors are for one caller. ## The writing side `io.WriterAt` is the mirror image: `WriteAt(p []byte, off int64) (n int, err error)`, independent of the cursor, and it must return a non-nil error when `n < len(p)`. Its concurrency clause is narrower than `ReadAt`'s: parallel `WriteAt` calls on the same destination are defined only if **the ranges being written do not overlap**. Two goroutines filling disjoint fixed-width record slots is exactly the safe case; two goroutines writing the same slot is not, and no amount of positional writing makes it well defined. ## Choosing Use `ReadAt` when the position is computed — index entries, fixed-width records, a range request, a footer. Use the cursor when you genuinely stream front to back and there is exactly one reader. If you find yourself putting a mutex around a `Seek`/`Read` pair, that is usually a sign the position wanted to be an argument all along. ## What an interviewer is checking That you can articulate the shared-state difference rather than reciting "ReadAt is thread-safe", that you know the fill-the-buffer clause distinguishing `ReadAt` from `Read`, and that you know `io.WriterAt`'s parallel guarantee is conditional on non-overlapping ranges.

  • os.File.ReadAt returns n less than len(p). What does the contract guarantee?
    That the error is non-nil. `ReadAt` must explain any short result, so a partial read at the end of a file comes back with `io.EOF`. Unlike `Read`, you never loop to top up the buffer — a nil error means the buffer is full.
  • What does io.WriterAt promise for two goroutines writing the same file at once?
    Only that parallel `WriteAt` calls are safe when the byte ranges do not overlap. Disjoint fixed-width slots are fine; two writers over the same range are undefined, and positional writes do not make them atomic or ordered.
  • Can two goroutines share one *io.SectionReader?
    Not for `Read` or `Seek` — a section reader keeps its own cursor, so sharing one reproduces exactly the interleaving problem you were escaping. Give each goroutine its own section reader over the same `io.ReaderAt`; they are cheap and need no extra descriptor. Its `ReadAt` method is parallel-safe.
  • When is a mutex around Seek and Read the right answer anyway?
    When the source only implements `io.ReadSeeker` and you cannot get positional reads at all, or when lookups are rare enough that serialising them costs nothing. Be aware you are capping concurrency on a resource the operating system would serve in parallel.

A cursor is a single sticky note on the filing cabinet saying which drawer to open next; passing an offset per call is telling each clerk the drawer number in their own instructions.

saying these in an interview costs you the question

  • Says ReadAt is safe because os.File locks internally
  • Writes a retry loop around ReadAt to fill the buffer
  • Claims parallel WriteAt is safe for overlapping ranges
  • Thinks ReadAt leaves the cursor at off plus n
  • Shares one io.SectionReader between goroutines