When would you use logging.handlers.WatchedFileHandler instead of RotatingFileHandler?

### Two models of who owns rotation A long-running Python process writing to a file has to answer one question: **who rolls the file over?** There are exactly two defensible answers, and the handler class you choose is the answer. * **The application owns it.** `logging.handlers.RotatingFileHandler` or `logging.handlers.TimedRotatingFileHandler` renames the file when its own policy says so. * **Something outside owns it.** A scheduled rotation tool on the host, or an orchestrator that collects files, renames or removes the file on its own schedule. Then the application should not rotate at all — it should just keep writing to the right file, which is what `logging.handlers.WatchedFileHandler` is for. Choosing both is the failure mode: the external tool renames `app.log` to `app.log.1` while the handler is also producing its own `app.log.1`, and one silently overwrites the other. ### What WatchedFileHandler actually does It subclasses `logging.FileHandler` and adds one behaviour. Before writing each record it calls `reopenIfNeeded()`, which does an `os.stat` on the configured path and compares the file's device and inode numbers with those recorded when the current stream was opened. Identical: write and move on. Different, or the path no longer exists: flush the old stream, close it, open the path again, and record the new identity. The comparison is on **file identity, not on the name**. On Unix a name is a directory entry pointing at an inode, and an open file object refers to the inode, not the name. That is why renaming a file out from under a writer changes nothing for the writer: it keeps appending to the same inode, now reachable only under the new name. `WatchedFileHandler` is the handler that notices. ### The failure it prevents Without it, an external rotation produces a distinctive and confusing symptom set: * The file at the expected path is present but stays at zero bytes (or was recreated by the rotation tool and never grows). Your monitoring says "no logs". * The renamed file keeps growing, so the archive from an hour ago is the one with live content. * If the rotation tool *deleted* the old file rather than renaming it, the space is not returned to the filesystem while the process holds the descriptor open. `df` shows a full disk while `du` shows a nearly empty log directory — the classic "deleted but still open" disk leak, and it lasts until the process restarts or closes the handler. A rotation tool that instead copies the file and then truncates it in place keeps the same inode, so a plain `logging.FileHandler` survives it: files opened in append mode always write at the current end, so the writer follows the truncation down to zero rather than leaving a hole. The cost of that style is a small window between the copy and the truncate in which records are lost. Rename-and-create has no such window, and that is the style `WatchedFileHandler` exists to support. ### Costs and limits * **One extra stat syscall per record.** Cheap relative to the write, but it is per-record, so at very high volume it is measurable. If that matters, put the file write behind a queue so the stat happens on one background thread instead of on every worker thread. * **Unix-shaped.** The whole design assumes a file can be renamed or unlinked while a process has it open, which Windows does not generally allow. On Windows the external rotator fails instead, so the handler has nothing useful to do. * **It rotates nothing.** If nobody outside rotates, the file grows forever. The handler is not a safety net; it is one half of a contract. ### How it interacts with several processes Because no process attempts a rename, having several processes each hold a `WatchedFileHandler` on one path is far safer than having each hold a `RotatingFileHandler` there: when the external tool rotates, every process independently notices and reopens the new file. The records still interleave in arrival order, and there is still no ordering guarantee between processes, but no writer's data is stranded in a file another writer is about to delete. Where a single ordered stream matters, route the processes to one writer instead. ### Rule of thumb Application-owned rotation for a service you deploy as a plain process and whose disk you must bound from inside; `WatchedFileHandler` when the host, the packaging or the platform already runs rotation and you merely need to not fight it. Write down which one it is — the bug appears months later, when somebody adds the other half.