skip to content

Why do Go's log and slog packages offer no file rotation, and what should a service do instead?

level: middleimportance: nice to knowfreq 33%

answer

  1. the packages only ever see an io.Writer
  2. no size, no path, nothing to roll
  3. retention belongs to the deployment
  4. a descriptor follows the inode, not the name
  5. somebody has to reopen after the rename

basics

~20 s

Their whole contract is an io.Writer, so they never learn whether the destination is a file and have nothing to cap, roll or compress. A service writes to a process stream and lets its supervisor own retention.

solid answer

~50 s

`log.New` and `slog.NewJSONHandler` both take an `io.Writer` and nothing else — no path, no size, no retention policy — so the standard library has no way to know a file is involved, let alone how big it may grow. That is deliberate: how long logs are kept is a deployment decision that differs per environment, and baking it into the binary couples the program to one answer. The Go-side answer is to write to `os.Stdout` and let the supervisor collect and expire the output. If your deployment genuinely needs files, the UNIX split is an external rotator that renames the file and signals the process; your program must then handle that signal and reopen the path, because the open descriptor keeps pointing at the renamed inode. Taking an in-process rotating writer instead means owning concurrency, roll-boundary partial writes and files stranded by a hard exit.

go deeper

for a junior

Know that Go's logging packages just take an io.Writer, so they cannot cap or roll anything, and that a service normally writes to os.Stdout rather than to a file it manages.

for a middle

Explain why the io.Writer contract makes rotation inexpressible, and what a program must do if an external rotator renames a file it holds open.

for a senior

Show the operational reasoning: retention is a deployment property, in-process rotation hides output from the platform, and a hard exit can strand a partly written file.

for a principal

Own the boundary as policy — services emit a stream and the platform owns retention uniformly, so no team encodes a retention decision in a binary the operator cannot change.

## The contract is one interface Everything in Go's logging surface bottoms out at `io.Writer`: ```go log.New(w io.Writer, prefix string, flag int) *log.Logger slog.NewJSONHandler(w io.Writer, opts *slog.HandlerOptions) *slog.JSONHandler slog.NewTextHandler(w io.Writer, opts *slog.HandlerOptions) *slog.TextHandler ``` `io.Writer` has exactly one method, `Write([]byte) (int, error)`. It cannot report a size, cannot be reopened, cannot be asked what path it came from. So the packages could not implement rotation even if they wanted to: they never learn whether they are writing to a file, a pipe, a network connection or a test buffer. `slog.HandlerOptions` carries a level, an `AddSource` flag and a `ReplaceAttr` hook — nothing about retention. ## Why that is the right boundary Retention is not a property of the program; it is a property of where the program runs. The same binary might run on a host where an operations team already rotates everything under a common policy, in a container where the runtime collects the output stream and expires it centrally, or on a developer's laptop where nobody wants files at all. A binary that caps its own log at 100 MB has hard-coded one of those environments, and the operator who needs a different answer has to redeploy code to get it. The corollary is the design rule for a service: **the process's responsibility ends at the file descriptor.** Format the record, write it to the stream, and stop. Collection, retention, compression and search live outside. ## What a service does instead Write to `os.Stdout` (or `os.Stderr`, consistently) and let the supervisor take it from there. In a container that means the runtime's collector; under a host process supervisor it means the supervisor's journal. Both already implement size caps and expiry, uniformly, for every process on the machine — which is exactly the property you want and exactly the property a per-service policy destroys. Writing to a file inside a container is the anti-pattern this replaces. The file sits in the container's writable layer where no collector looks, it grows until something fills up, and it is destroyed with the container. You have converted a solved platform problem into an unsolved application problem. ## If you genuinely must write files Some deployments — a long-lived host process, an air-gapped appliance — really do want files on disk. Then you have two options, and both cost you something. **External rotator plus reopen.** The classic UNIX arrangement: a scheduled tool renames `service.log` to `service.log.1`, creates a fresh `service.log`, and signals the process. The signal matters. An open file descriptor refers to an *inode*, not to a name, so after the rename your process keeps appending to the renamed file; the new one stays empty and the disk space is not reclaimed until you close the old descriptor. Your Go program has to subscribe to the signal with `os/signal`, open the path again, and swap the writer the handler is using. That swap has to be safe against records being written concurrently, which is real work you now own. **An in-process rotating writer.** Some programs take on a dependency that implements `io.Writer` and rolls by size or age underneath. It removes the signal handling, and hands you a different set of obligations: the writer must be safe for concurrent use by every goroutine that logs; a roll happening mid-record must not split a line across two files; and a hard exit — `os.Exit`, `log.Fatal`, SIGKILL — can leave a partially written file behind, with no flush having run. ## What to say in an interview Three beats. First, the mechanism: the packages accept an `io.Writer`, so rotation is not expressible in their contract. Second, the intent: retention is a deployment decision and the standard library refuses to make it for you. Third, the practical consequence: for a service or a job, write the stream and let the platform own retention; only reach for files when the deployment demands them, and then know that renaming a file does not move an open descriptor.

  • An external tool renames a Go service's open log file and creates a new one. Where do the next writes go?
    Into the renamed file. The process holds a descriptor on the inode, and renaming does not touch it, so the service keeps appending to the old file while the new one stays empty — and the disk space is not reclaimed until the descriptor closes. That is why the convention is for the rotator to signal the process so it reopens the path.
  • Why is writing log files inside a container worse than the same choice on a long-lived host?
    Nothing collects the file, nothing caps it, and it is destroyed when the container is replaced, so the records exist only while the thing that needs debugging is still alive. On a host there is at least an operations story around the file. In a container you have hidden the output from the platform that was already going to collect it for free.
  • What does slog.HandlerOptions let you configure about the output destination?
    Nothing about the destination. It carries the minimum level, an AddSource flag that records the call site, and a ReplaceAttr hook for rewriting attributes. The destination is the io.Writer passed as the constructor's first argument, and the handler never inspects it.

saying these in an interview costs you the question

  • Claims slog rolls a file once it passes a size limit
  • Thinks the process picks up a new file automatically after a rename
  • Writes log files inside a container so they survive a restart
  • Assumes the log package reopens its output on SIGHUP by itself
  • Treats retention policy as something to hard-code in the binary