In PHP, a nightly job must delete generated PDF invoices older than 90 days from a directory with a million files; how do you write it safely?
answer
- stream the listing, don't build an array
- DirectoryIterator with isDot() and isFile()
- filemtime() or getMTime() against a cutoff
- unlink() returns false with a warning
- skip symlinks and half-written files
basics
~20 sIterate the directory with DirectoryIterator instead of glob() or scandir(), skip dot entries, symlinks and non-PDF names, compare getMTime() with a cutoff, and check unlink()'s return. Files still being written should live under temp names so the job never sees them.
solid answer
~50 s`glob('*.pdf')` and `scandir()` build an array of every name before you delete anything, which costs memory proportional to a million entries; `DirectoryIterator` reads one entry at a time. For each entry I skip `isDot()`, anything that is not a regular file or is a link, and names that do not match the invoice pattern, then compare `getMTime()` with a cutoff such as `time() - 90 * 86400`. `unlink()` returns `false` with an `E_WARNING` if the file has already gone or permissions forbid it, so I count failures instead of assuming success. The generator should write under a temporary name and `rename()` into place, so the cleaner never sees a half-written PDF. I would add a dry-run switch, log counts, pin the base path with `realpath()`, and run it as the user that owns the files.
go deeper
Recall that DirectoryIterator walks a directory entry by entry, that isDot() skips . and .., and that unlink() returns a bool.
Explain why glob() and scandir() build arrays, how getMTime() compares against a cutoff, and why unlink() needs directory write permission.
Show the production guards: skip links, match a strict filename pattern, dry-run first, count failures, and keep half-written files invisible through rename.
Weigh file retention against policy: who owns the retention period, how deletions are audited, and whether invoices belong in object storage with lifecycle rules instead.
## The requirements A generator writes invoice PDFs into `/var/app/invoices`. Every night a CLI job must delete the ones older than 90 days. The directory can hold a million files, the generator may be writing new ones while the job runs, and deleting the wrong file is costly. ## Listing without an array | Approach | Memory | Notes | |---|---|---| | `glob('/var/app/invoices/*.pdf')` | an array of every matching path | sorted by default (`GLOB_NOSORT` skips that); returns `false` on error | | `scandir($dir)` | an array of every name | sorted too; includes `.` and `..` | | `opendir()` / `readdir()` | one name at a time | procedural, easy to forget `closedir()` | | `new DirectoryIterator($dir)` | one entry at a time | object with `isDot()`, `isFile()`, `isLink()`, `getMTime()`, `getPathname()` | With a million entries, the array-building approaches hold every name before deleting anything. `DirectoryIterator` keeps memory flat and gives each entry as an `SplFileInfo`. Its constructor throws `UnexpectedValueException` if the directory cannot be opened, which is a better failure than an empty loop. ## The job ```php <?php declare(strict_types=1); $base = realpath('/var/app/invoices') ?: throw new RuntimeException('no invoice dir'); $cutoff = time() - 90 * 86400; $dryRun = in_array('--dry-run', $argv, true); $deleted = $failed = 0; foreach (new DirectoryIterator($base) as $entry) { if ($entry->isDot() || $entry->isLink() || !$entry->isFile()) { continue; } if (!preg_match('/^INV-\d{4}-\d+\.pdf$/', $entry->getFilename())) { continue; } if ($entry->getMTime() >= $cutoff) { continue; } if ($dryRun || @unlink($entry->getPathname())) { $deleted++; } else { $failed++; } } fwrite(STDERR, "deleted={$deleted} failed={$failed}\n"); ``` What each guard is for: 1. **`isDot()`** skips `.` and `..`. 2. **`isLink()`** skips symlinks, so a link planted in the directory cannot make the job delete or inspect something elsewhere. `isFile()` alone follows links and would report the target. 3. **A strict filename pattern** means the job only ever touches files it recognises; a stray upload or `.gitkeep` survives. 4. **`getMTime()` against a cutoff** uses the modification time. Invoice date in the name is an alternative when files may be touched later. 5. **`unlink()` checked, not assumed.** It returns `false` with an `E_WARNING` when the file is gone, is a directory, or permissions forbid it. The `@` keeps a million-line warning flood out of the log while the counter records the failure. ## Files being written A PDF still being rendered has a fresh mtime, so the age check protects it. It is still better to make the producer invisible until done: - The generator writes to a temporary name (for example with `tempnam()` in the same directory, or a `.tmp` suffix the pattern excludes). - It calls `rename()` to the final `INV-….pdf` name when complete, which on one file system is atomic. - The cleaner's pattern never matches temp names, so it cannot delete a file mid-write. Orphaned temp files from crashed renders need their own, more conservative age rule. ## Testing it `touch(string $filename, ?int $mtime = null, ?int $atime = null)` sets a file's modification time, which makes the age rule testable without waiting 90 days: - Create a fixture directory with a few `INV-….pdf` files, `touch()` some of them to 100 days ago and some to yesterday. - Add a symlink and a non-matching file to prove the guards skip them. - Run the job with `--dry-run`, assert the counts, then run it for real and assert which files remain. ## Running it in production - **Run as the owning user.** `unlink()` needs write permission on the **directory**, not on the file. A job running as a different user fails on every file. - **Dry run first.** The `--dry-run` switch counts what would go, so a wrong cutoff or pattern shows up before anything is deleted. - **Bounded work per run.** If the backlog is huge, stop after N deletions and let the next night continue, which keeps I/O spikes and run time predictable. - **Stat cache.** Each entry is a different path, so the single-entry stat cache never returns stale data here; `clearstatcache()` is not needed. - **Exit status.** Return a non-zero exit code when `$failed` is above a threshold so the scheduler reports it. - **Pin the base path.** `realpath()` resolves the configured directory once; if the path comes from configuration, refuse to run when it resolves to something unexpected like `/`.
- Why does the cleanup check isLink() before isFile()?`isFile()` follows symlinks and reports on the target, so a link named like an invoice would pass and the job would act on a file it does not own. Checking `isLink()` first skips links entirely. `unlink()` on a link removes the link, not the target, but reading `getMTime()` through it still leaks decisions to wherever it points.
- unlink() fails with Permission denied although the PDF file itself is mode 0666; why?Deleting a file removes an entry from its directory, so it needs write and execute permission on the directory, not on the file. If the job runs as a user who cannot write to `/var/app/invoices`, every `unlink()` fails. Run the job as the owning user or fix the directory's group permissions.
- When would you pick glob() over DirectoryIterator for this job?When the directory is small and bounded, `glob()` with a pattern is shorter and readable. It returns an array of every match, sorted unless you pass `GLOB_NOSORT`, and brace patterns need `GLOB_BRACE`, which is only defined on platforms whose glob supports it. For a directory that grows without limit, a streaming iterator keeps memory flat.
saying these in an interview costs you the question
- glob() streams matches one at a time, so it is fine for a million files.
- unlink() needs write permission on the file itself, not on its directory.
- isFile() returns false for a symlink that points to a regular file.
- A cleanup script needs clearstatcache() before every filemtime() call.
- unlink() throws an exception when the file has already been deleted.