skip to content

Threads, sysmon and LockOSThread

The runtime creates and parks OS threads on demand, keeps a sysmon thread running without a P, and lets LockOSThread nail one goroutine to one thread. Interviewers ask when that pinning is mandatory.

part ofGo (Golang)overview, primer and where to startread it →
on this pageshow

questions

5

If a goroutine calls runtime.LockOSThread and exits without unlocking, what happens to that OS thread?

level: middleimportance: should knowfreq 36%

answer

  1. the counter is still above zero
  2. the thread is not handed back
  3. the runtime cannot undo what you changed
  4. destruction is the cleanup, and it is intended
  5. one goroutine per operation, never unlocked

basics

~20 s

The runtime terminates that OS thread instead of reusing it. That is deliberate: a thread whose per-thread state was changed must not go on to run other goroutines, so exiting while still locked is the documented way to discard it.

solid answer

~40 s

`runtime.LockOSThread` and `runtime.UnlockOSThread` maintain a per-goroutine counter. If the goroutine returns while the count is still above zero, the runtime does not hand the thread back for other goroutines to use; it terminates it. That is a feature, not a leak. Once code has changed thread-owned state such as a Linux namespace, credentials, or a C library's thread-local context, the runtime has no way to restore it, so destroying the thread is the only safe cleanup. The idiom is therefore to lock, do the operation, and deliberately never unlock, running the whole thing on a goroutine created for that one operation. The price is a thread creation and teardown per operation, plus a live thread for as long as the operation runs, so it belongs in a bounded, non-hot path.

code

go · 11 lines
go
func runPinned(work func() error) error {
	errc := make(chan error, 1)
	go func() {
		runtime.LockOSThread() // deliberately never unlocked
		// change thread-owned state here, then:
		errc <- work()
		// returning ends this goroutine; the runtime
		// terminates the now-dirty OS thread with it
	}()
	return <-errc
}

go deeper

for a junior

Remember the pairing rule first: as many UnlockOSThread calls as LockOSThread calls, and know that a goroutine which ends while still locked takes its OS thread down with it.

for a middle

Explain the per-goroutine counter and why termination rather than reuse is the safe choice when thread-owned state cannot be restored. Be able to write the lock-work-exit shape without unlocking.

for a senior

Talk about the cost side: threads per concurrent operation, where you bound that concurrency, and how you would review a defer UnlockOSThread that hands back a thread whose namespace or credentials were changed.

for a principal

Frame it as a capacity decision. A design that burns a thread per in-flight operation converts request concurrency into process threads, and the runtime's ceiling turns overload into an abort rather than backpressure, so the limit belongs upstream.

## The mechanism Every goroutine carries a small counter of outstanding `runtime.LockOSThread` calls. While the counter is above zero the goroutine is wired to one OS thread and that thread runs nothing else. `runtime.UnlockOSThread` decrements it; reaching zero unwires the pair and returns the thread to normal service. If the goroutine instead **returns while the counter is above zero**, the runtime destroys the thread. The documented wording is that the thread will be terminated. It is not parked, not put back in a pool of reusable threads, and not reset: the kernel thread ends. ## Why the runtime chooses destruction The only reason to pin a goroutine is that something outside the Go heap is attached to the thread. That could be: - Linux namespace membership, credentials, or capability sets, all of which are per-thread; - a C library's thread-local handle, error slot, or session context; - a graphics context that is current on exactly one thread. The runtime cannot enumerate that state, let alone undo it. Reusing such a thread would let an unrelated goroutine inherit a namespace or a half-configured library context, which is a far worse failure than the cost of one thread. So the runtime takes the conservative option, and Go programmers turned that into an idiom: when you have deliberately dirtied a thread, **do not unlock** — end the goroutine and let the thread die with it. ## The shape that follows from it Because the cleanup is the goroutine's death, the unit of work has to be a goroutine created for the job: 1. start a goroutine for the operation; 2. lock it to its thread; 3. change the thread state and do the work; 4. send the result back over a channel and return, without unlocking. Everything that must observe the thread state has to run on that goroutine. Anything it starts with `go` is not pinned and will run on a different thread, so a helper goroutine spawned inside the locked one will not see the state. ## What it costs Each operation of this shape costs a thread creation and a thread teardown, plus a kernel stack and runtime bookkeeping for as long as the operation runs. That is cheap in absolute terms and expensive relative to a goroutine, so the pattern scales with the number of **concurrent** operations, not with their total number. Two consequences worth stating in an interview: - Concurrency has to be bounded above this code. Nothing in the pattern applies backpressure by itself. - The runtime has a hard ceiling on live threads (10000 by default, adjustable with `runtime/debug.SetMaxThreads`). Crossing it is not an error you can handle: the runtime prints a line about exceeding the thread limit and aborts the process with a fatal thread-exhaustion error. ## The alternative and when it is wrong The symmetrical `defer runtime.UnlockOSThread()` is correct only when the thread is exactly as it was before: you locked to guarantee a sequence of calls hit one thread, and you changed nothing durable. If you changed something durable, unlocking is actively harmful, because it puts a polluted thread back into general service where unrelated goroutines will run on it. Reviewing this code, the question to ask is not "is the lock balanced?" but "can this thread safely run somebody else's goroutine next?" ## Interaction with the scheduler While the goroutine is locked, the runtime may need to give its thread a scheduling context to run it, parking whichever other thread found the goroutine runnable. The thread also cannot be reused to run other goroutines while the lock is held, so a locked goroutine that blocks for a long time keeps a thread allocated but idle. None of this is fatal, but it is why pinning belongs on a dedicated, comparatively coarse-grained operation rather than around a single fast function call.

  • Why does the runtime destroy the thread rather than reset and reuse it?
    It has no idea what was changed. Namespace membership, credentials, and a C library's thread-local context are invisible to the runtime and cannot be enumerated or undone. Handing such a thread to an unrelated goroutine would leak that state into work that knows nothing about it, so the runtime takes the safe option and ends the thread.
  • What does the never-unlock idiom cost if it runs on a hot path?
    A thread creation and teardown per operation, and one live OS thread with its own kernel stack for the duration. The cost scales with concurrent operations, so unbounded concurrency turns into unbounded threads. Something above this code has to limit how many run at once; the runtime's thread ceiling aborts the process rather than pushing back.
  • When is deferring runtime.UnlockOSThread the right thing instead?
    When you locked only to guarantee that a sequence of calls hits one thread and you left nothing durable behind on it. Then the thread is genuinely clean and returning it to general service is free. The review question is whether that thread can safely run somebody else's goroutine next.

saying these in an interview costs you the question

  • Thinks the thread returns to a reusable pool
  • Calls it a leak rather than the intended cleanup
  • Says forgetting to unlock leaks the goroutine, not the thread
  • Assumes the runtime resets per-thread state automatically
  • Treats one pinned thread per operation as free
open as a page

A Go tool calls setns(2) to enter a namespace, then its work sometimes runs in the wrong namespace. Why, and what is the fix?

level: seniorimportance: should knowfreq 30%

basics

~20 s

The setns call changes only the calling OS thread, and the Go scheduler may resume the goroutine on a different thread after a syscall or a preemption. Pin with runtime.LockOSThread before entering, and let that goroutine exit rather than unlocking.

open as a page

Should a shared library call runtime.LockOSThread internally, or push thread pinning onto its callers?

level: principalimportance: should knowfreq 20%

basics

~20 s

Decide by how far the thread affinity reaches. If the whole affected operation happens inside one of your calls, pin internally and hide it. If the affinity outlives the call or caller code runs inside the pinned region, make pinning a documented obligation on the caller.

open as a page

What does runtime.LockOSThread do, and when does a Go program actually need it?

level: juniorimportance: nice to knowfreq 28%

basics

~20 s

runtime.LockOSThread wires the calling goroutine to the OS thread it is running on: that goroutine runs only there, and no other goroutine runs on that thread. You need it for APIs that keep state per OS thread.

open as a page

What is the Go runtime's sysmon thread, and why does it run without holding a P?

level: middleimportance: nice to knowfreq 24%

basics

~20 s

sysmon is a monitoring thread the Go runtime starts at program start and runs for the process lifetime. It holds no P, so GOMAXPROCS does not bound it and it can still act when every P is busy: polling the network, retaking Ps and forcing a periodic collection.

open as a page