skip to content

A Node.js CLI walks a project tree: what do readdir's withFileTypes and recursive options give you, and where do they fall short?

level: middleimportance: should knowfreq 30%

answer

  1. types without a stat per entry
  2. Dirent name and parentPath
  3. one big array, no pruning
  4. links are listed, not followed

basics

~20 s

withFileTypes returns fs.Dirent objects that carry each entry's type without a stat call; recursive lists the whole tree in one array. Recursive cannot prune node_modules and does not follow directory links, so walk manually or use fs.glob with exclude.

solid answer

~40 s

`readdir(dir, { withFileTypes: true })` returns `fs.Dirent` objects with `name`, `parentPath` and `isFile()`, `isDirectory()` and `isSymbolicLink()`, using the type the operating system reports while reading the directory, so you avoid one `stat` per entry. `recursive: true`, added in v20.1.0, lists every file and directory beneath `dir` in a single array. That is convenient for small trees, but it has no filter, so it reads all of `node_modules` and `.git` before you can skip them, it holds the whole listing in memory, and it lists a symbolic link to a directory without descending into it. For a project walker, recurse yourself with `withFileTypes` and skip names as you go, or use `fs.glob` with `exclude`, stable in Node 24. Use `dirent.parentPath`: `dirent.path` was removed in v24.0.0.

code

javascript · 18 lines
javascript
import { readdir } from 'node:fs/promises';
import path from 'node:path';

const SKIP = new Set(['node_modules', '.git']);

async function* walk(dir) {
  for (const entry of await readdir(dir, { withFileTypes: true })) {
    const full = path.join(entry.parentPath, entry.name);
    if (entry.isDirectory()) {
      if (SKIP.has(entry.name)) continue; // pruned before it is read
      yield* walk(full);
    } else if (entry.isFile()) {
      yield full;                         // links are neither followed nor yielded
    }
  }
}

for await (const file of walk(process.argv[2] ?? '.')) console.log(file);

go deeper

for a junior

Recall that withFileTypes returns Dirent objects with isFile and isDirectory, and that recursive lists a whole tree in one call.

for a middle

Explain why Dirents save a stat per entry, why recursive cannot prune and holds everything in memory, and how links are reported rather than followed.

for a senior

Choose the walk for the tree: a pruning generator or opendir for big repositories, fs.glob with exclude for pattern searches, and cycle detection when links must be followed.

for a principal

Weigh a hand-written walker against fs.glob for a team's tooling: fewer moving parts versus control over pruning, links and memory on large repositories.

## Reading a directory in Node `fsPromises.readdir(path, options)` returns the entries of one directory, excluding `.` and `..`. By default the entries are plain name strings, so a tool that needs to know which ones are directories must call `stat` or `lstat` on each name: one extra system call, on the threadpool, per entry. Two options change that. ## withFileTypes: types without a stat per entry With `withFileTypes: true` the result is an array of **`fs.Dirent`** objects. A Dirent combines the name with the type the operating system reported while reading the directory: - `dirent.name`: the entry's own name, with no directory part. - `dirent.parentPath`: the directory that contains the entry, added in v21.4.0 and marked stable in v24.0.0. - `dirent.isFile()`, `isDirectory()`, `isSymbolicLink()` and the device, FIFO and socket checks. Two details matter in practice: 1. **A symbolic link reports as a link.** `isDirectory()` is `false` for a link that points at a directory; `isSymbolicLink()` is `true`. Follow it deliberately with `stat` if you mean to. 2. **The type comes from the file system.** The docs note some file systems report a type that differs from `fs.lstat()`; Node calls `lstat` only when the reported type is unknown. Use `lstat` when you need certainty. ## recursive: one call, one array `recursive: true` (added in v20.1.0 and v18.17.0) lists every file, subdirectory and nested entry beneath the starting directory. Combined with `withFileTypes`, each Dirent's `parentPath` tells you where it lives. Looking at how Node 24 implements it shows the trade-offs: - It reads directories **one after another** and returns only when the whole tree is done, as **one array** held in memory. - It has **no filter**. A project tree's `node_modules` or `.git` is read in full before your code sees a single entry. - It descends only into entries whose `isDirectory()` is true, so **links to directories are listed but not followed**. | Approach | Prunes while walking | Memory | Follows directory links | |---|---|---|---| | `readdir(dir, { recursive: true, withFileTypes: true })` | no | whole tree in one array | no | | manual walk with `readdir(..., { withFileTypes: true })` | yes | one directory at a time | only if you code it | | `opendir(dir)` iterated with `for await` | yes, per directory | `bufferSize` entries (32 by default) | only if you code it | | `fs.glob(pattern, { exclude })` | yes | streamed through an async iterator | only with `followSymlinks`, added in v24.16.0 | ## Walking a project tree properly For a CLI that must skip dependency and VCS directories: 1. Read one directory with `withFileTypes: true`. 2. For each Dirent, build the full path from `parentPath` and `name` with `path.join`. 3. Skip a directory by name **before** reading it, which is the point of walking by hand. 4. Recurse into directories, yield files, and decide explicitly what to do with links. An async generator keeps memory flat and lets the caller stop early. For very large directories, `opendir()` returns an `fs.Dir` that reads a few entries at a time and closes itself when a `for await` loop ends. When the job is really "find files matching a pattern", `fs.glob` is shorter: it was marked stable in v24.0.0 and its `exclude` option accepts a function or glob patterns. ## Errors during a walk A real tree changes while you read it, and not every directory is readable: - A directory deleted between listing its parent and reading it rejects with `ENOENT`. - A directory the process may not read rejects with `EACCES`. - With `recursive: true`, any such error rejects the **whole** call and discards everything already listed; a hand-written walker can catch per directory, log, and continue. Bound the concurrency if you parallelise across directories: each `readdir` takes a threadpool thread, and a walker that fans out without limit competes with every other async fs call in the process. ## The Node 24 change to know `dirent.path`, an older spelling of the parent directory, was deprecated in the documentation in v21.5.0, made a runtime deprecation in v23.0.0, and reached End-of-Life in v24.0.0 (`DEP0178`). Code written against it reads `undefined` on Node 24; use `dirent.parentPath`.

  • Your walker also has to follow symbolic links to directories. What new problem must it handle?
    Cycles. A link can point to an ancestor, so a naive walk loops forever. Resolve each followed directory with `realpath` and keep a set of visited real paths, skipping any you have seen. `fs.glob` offers `followSymlinks`, added in v24.16.0, and the docs say it does not traverse detected link cycles.
  • When is readdir with recursive: true the right choice after all?
    For a small, known tree with nothing to prune, such as a fixtures folder in a test or a generated output directory you must list completely. There the single call is clearer than a hand-written walker and the one-array result costs nothing that matters.

  • recursive: true follows symbolic links into the directories they point to
  • withFileTypes still performs a stat call for every entry
  • dirent.path is the way to find an entry's directory on Node 24
  • recursive readdir streams entries, so it is fine on huge trees
  • Dirent.isDirectory() is true for a link that points at a directory

Assumes Node 24: dirent.parentPath is stable and dirent.path is removed (v24.0.0); fs.glob is stable since v24.0.0; recursive readdir dates from v20.1.0.