skip to content

Why does reading a regular file in Go tie up an OS thread when reading a socket does not?

level: middleimportance: nice to knowfreq 30%

answer

  1. not every descriptor can be watched
  2. the kernel refuses to poll it
  3. a regular file is always reported ready
  4. so the read is a genuine blocking call
  5. one in-flight read, one thread

basics

~20 s

Readiness notification does not work for regular files — the kernel will not usefully poll them, and non-blocking mode has no effect. So Go issues a real blocking read that occupies its OS thread, while sockets are registered with the poller and only park a goroutine.

solid answer

~50 s

Go's poller is built on readiness notification, and readiness is meaningless for a regular file: epoll refuses to watch one, and opening it non-blocking does not make a read return early — the kernel just does the disk work. So when `os.Open` produces a regular file, the runtime cannot register it and falls back to blocking mode; each `Read` is a genuine blocking system call that holds its OS thread until the data is in memory. Pollable descriptors — sockets, pipes, terminals — do get registered, so waits on those park only a goroutine. The practical consequence is that file-heavy concurrency is thread-heavy concurrency: launching a goroutine per file across a large tree can push the thread count far above `GOMAXPROCS`. The fix is not a runtime setting but a concurrency cap on the file work itself.

code

go · 12 lines
go
sem := make(chan struct{}, 8) // at most 8 file reads in the kernel at once
for _, name := range names {
	sem <- struct{}{}
	go func(n string) {
		defer func() { <-sem }()
		data, err := os.ReadFile(n) // blocking: holds an OS thread
		if err != nil {
			return
		}
		process(data)
	}(name)
}

go deeper

for a junior

Remember the split: waiting on a socket is cheap, waiting on a file costs a thread. Do not assume goroutines make disk reads free.

for a middle

Explain why: readiness notification is meaningless for a regular file, so the runtime cannot register it and uses a plain blocking read instead.

for a senior

Show the operational consequence — thread count that scales with storage latency — and reach for a concurrency cap on the file work rather than a runtime knob.

for a principal

Own the sizing call: how much parallel file or object I/O a service is allowed to have in flight, and whether that limit lives in the library, the job, or the platform.

This is one of the sharpest edges in Go's I/O model, and it surprises people who have internalised "goroutines make blocking calls free". They do — for network I/O. Disk I/O is a different story. ## Readiness versus completion Go's poller is a **readiness** mechanism. It asks the kernel: tell me when this descriptor can be read without waiting. That question is well-defined for a socket (data has arrived or it has not) and for a pipe or a terminal. It is not well-defined for a regular file: from the kernel's point of view an ordinary file is *always* ready, because the read will always eventually succeed. Waiting on the disk is not something readiness notification models. On Linux, `epoll_ctl` simply rejects a regular file. And `O_NONBLOCK` on a regular file does not make `read` return early either — it is effectively ignored. ## What the runtime does about it When `os.Open` creates a file, the runtime attempts to hand the descriptor to the poller. For a pipe, a socket, or a character device such as a terminal, that succeeds and waits on the descriptor park a goroutine like any network wait. For a regular file the attempt fails and the descriptor is used in plain blocking mode. From there, every `Read` or `Write` is a real system call that stays in the kernel until it is done, occupying its OS thread the whole time. That is not a bug or an oversight — it is the trade-off of building on portable readiness APIs. Kernel interfaces for asynchronous disk I/O exist, but the Go runtime does not use them for the `os` package as of the current release, so the model stands. ## Why it does not stall your program The scheduler handles this the same way it handles any blocking call: the goroutine's scheduling context is handed to another thread, which keeps running the rest of the program. Correctness is preserved. What you spend is threads. ## Where this bites The classic shape is a job that walks a large directory tree and starts a goroutine per entry to read or stat it. The goroutines are cheap; the in-flight blocking calls are not. With no cap, an operation intended to be "a bit parallel" can put dozens or hundreds of threads in the kernel at once, each with its own stack and kernel bookkeeping. The symptom is a thread count many multiples of `GOMAXPROCS` and a machine spending time in the kernel rather than in your code. A second, quieter shape is a slow or contended filesystem — a network mount, a throttled cloud volume, a disk under pressure. The same code that behaved on a local SSD parks far more calls in the kernel when each one takes tens of milliseconds, so thread count scales with latency rather than with request rate. ## What to actually do Cap the concurrency of the file work with a small counting limit — a buffered channel used as a semaphore is the idiomatic form — sized to what the storage can absorb rather than to how many files you have. That bounds threads directly, because in-flight blocking calls are the thing that creates them. Reading files sequentially is often faster anyway on a single spindle or a throttled volume, so the cap frequently costs nothing. ## The mental model to keep Ask of any descriptor: *can the kernel tell Go when this is ready?* If yes — sockets, pipes, terminals — waiting costs a parked goroutine. If no — regular files, and anything behind a cgo call — waiting costs an OS thread. Almost every surprising Go thread count traces back to the second category.

  • Which non-socket descriptors does the Go runtime still manage with the poller?
    Anything the kernel can report readiness for: pipes, including the ones behind `os.Pipe` and a subprocess's stdout, terminals and other character devices, and Unix domain sockets. Those behave exactly like network connections — a wait parks the goroutine only. Regular files are the notable exclusion.
  • How would you keep a directory-walking job from creating hundreds of OS threads?
    Bound the number of file operations in flight rather than the number of goroutines you are willing to create. A buffered channel used as a counting semaphore around each read or stat is the usual form, sized to what the storage can actually absorb. Since threads come from in-flight blocking calls, capping the calls caps the threads.

saying these in an interview costs you the question

  • Assumes goroutines make disk I/O non-blocking too
  • Says os.Open descriptors go through the netpoller
  • Thinks setting a file non-blocking would fix it
  • Believes more goroutines make file reads faster
  • Confuses always-ready for polling with never blocking