Rotation and Queue Handlers

### The shared machinery Both classes live in `logging.handlers`, and both extend `logging.handlers.BaseRotatingHandler`, which itself extends `logging.FileHandler`. The base class turns every `emit()` into two steps: call `shouldRollover(record)` to ask whether *this* record trips the rollover condition, and if the answer is yes call `doRollover()` before writing the line. That single fact answers most rotation questions in an interview: **rotation is driven by records arriving, not by a background timer**. A process that logs nothing between 02:00 and 08:00 performs no rotation in that window, whichever handler it uses. With timed rotation the effect is visible as a missing or oddly-dated file for a quiet period. ### RotatingFileHandler — roll by size `RotatingFileHandler(filename, maxBytes=0, backupCount=0)` decides by measuring. `shouldRollover` formats the pending record, adds its length to the stream's current position, and returns true when the total reaches `maxBytes` — so the file rolls *before* it crosses the limit, and a single line is never split across two files. `doRollover()` then shifts the numbered backups downward: `app.log.2` becomes `app.log.3`, `app.log.1` becomes `app.log.2`, the live `app.log` becomes `app.log.1`, and whatever fell past `backupCount` is deleted. Two defaults bite people: * `maxBytes=0` disables the size check entirely — the handler behaves like a plain `logging.FileHandler` and the file grows until the disk fills. * `backupCount=0` produces no backup name at all, so nothing is preserved. Both must be non-zero for rotation to happen. The number an operator actually wants is the worst-case footprint, roughly `maxBytes * (backupCount + 1)`; choose the pair from a disk budget, not from a feeling. ### TimedRotatingFileHandler — roll by clock `TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, utc=False, atTime=None)` computes a next-rollover instant when it opens the file and recomputes it after each rollover. `when` selects the unit — seconds, minutes, hours, days, `'midnight'`, or `'W0'`–`'W6'` for a given weekday — and `interval` multiplies it, so `when='h', interval=6` gives four files a day. The rotated file keeps the base name plus a **timestamp suffix** (`app.log.2026-09-04`) instead of a number. That is the practical reason to prefer timed rotation for anything a human reads: "yesterday's export log" is a filename, not an archaeology exercise across shifting numbers. `backupCount` here deletes the oldest matching files rather than renaming anything. Two knobs matter in production. `utc=False` is the default, so boundaries are computed in local time; a daylight-saving change therefore yields one short and one long file per year, and correlating logs across regions gets harder — set `utc=True` for a fleet. `atTime` moves the midnight or weekly boundary to a chosen time of day, which is how you keep rotation out of your busiest minute. ### They are alternatives, not two knobs on one handler You cannot ask one handler for "daily, but also cap each file at 100 MB" — the size and time policies live in different classes. Attaching both handlers to the same logger does not combine the policies either; it writes every record twice, to two different files. Pick the policy that matches how the file is consumed — size when the only constraint is disk, time when something downstream slices by day — and enforce the other bound with an operational alarm. ### Details that come up in follow-ups * `delay=True` postpones opening the file until the first record. Useful when a configuration declares many handlers of which few ever fire. * `encoding` should be set explicitly (`encoding="utf-8"`); leaving it to the platform default is how a log file becomes unreadable on another machine. * `logging.handlers.BaseRotatingHandler.namer` and `logging.handlers.BaseRotatingHandler.rotator` are hooks: `namer` chooses the rotated filename, `rotator` performs the move. Setting `rotator` to a function that compresses the file is the supported way to get gzipped archives without a second tool. * Every handler holds a `threading.Lock`, so several *threads* in one process rotate safely. Several *processes* pointed at one file do not — each has its own file object and rotates on its own. ### A worked shape ```python import logging import logging.handlers handler = logging.handlers.TimedRotatingFileHandler( "etl-export.log", when="midnight", backupCount=14, utc=True, encoding="utf-8" ) handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(name)s %(message)s")) logging.getLogger("etl.export").addHandler(handler) ``` Fourteen daily files, UTC boundaries, explicit encoding, timestamped names. If the same service had unpredictable bursts and a hard 500 MB budget, the size class with `maxBytes=100_000_000, backupCount=4` would be the honest choice instead.