skip to content

A macOS build machine shows mds_stores consuming CPU and hammering the disk whenever a build runs. What is that process doing, and how do you control it for specific volumes or directories?

level: middleimportance: nice to knowfreq 26%

answer

  1. it is a metadata index, not a filename list
  2. change notifications drive the work
  3. volume switch versus directory marker
  4. erasing the index makes load worse first
  5. excluded means unsearchable

basics

~20 s

That is Spotlight's metadata indexer reacting to the thousands of files a build creates and deletes. Use mdutil to check status and disable indexing per volume (mdutil -i off), and keep individual directories out of the index by naming them with a .noindex suffix or adding them to the Spotlight privacy list.

solid answer

~50 s

`mds` and its helpers `mds_stores` and `mdworker` are Spotlight's metadata indexing service. They watch filesystem change notifications and run importer plugins over new and modified files to extract searchable metadata, which is exactly the wrong workload on a machine that churns through a large object tree every build. Control it at two granularities. Per volume: `mdutil -s /Volumes/Build` shows indexing status, `sudo mdutil -i off /Volumes/Build` turns indexing off, and `sudo mdutil -E /Volumes/Build` erases and rebuilds the index — useful when it has become corrupt, but it triggers a full reindex, so never run it to "fix" load. Per directory: a folder whose name ends in `.noindex` is skipped, and a volume can be excluded by placing a `.metadata_never_index` file at its root, or through the privacy list in Spotlight settings. Note the trade-off — `mdfind` and Spotlight search stop working for anything you exclude.

code

bash · 12 lines
bash
# Is this volume being indexed at all?
mdutil -s /Volumes/Build
mdutil -a -s                       # every known volume

# Stop indexing the whole scratch volume
sudo mdutil -i off /Volumes/Build

# Or exclude just the churning output tree, by name
mkdir -p /Users/ci/work/artifacts.noindex

# Only for a genuinely corrupt index: erase and rebuild (expensive)
sudo mdutil -E /Volumes/Build

go deeper

for a junior

Know that mds and mdworker are Spotlight's indexing processes and that mdutil is the command that reports and changes indexing status for a volume.

for a middle

Explain what the indexer does — importer plugins over filesystem change notifications — and distinguish turning indexing off for a volume from erasing the index, which forces a full and expensive rebuild.

for a senior

Show the surgical fix on a build host: exclude the churn directory with a .noindex name or the privacy list, verify with mdutil -s, check mounted external and network volumes, and state the searchability trade-off you are accepting.

for a principal

Own it as fleet configuration: index exclusions for build and cache paths belong in managed policy and machine images, not in per-host tinkering, and the search capability being given up should be a recorded decision.

## What the processes are Spotlight is a metadata index, not a filename index. `mds` is the server; `mds_stores` maintains the index stores; `mdworker` and `mdworker_shared` are short-lived workers that run **importer plugins** to extract metadata from individual files — text content, EXIF data, document properties, and so on, depending on the file type. The indexer is driven by filesystem change notifications: when files appear, change or disappear, work is queued. On a laptop that is invisible. On a build machine that writes and deletes tens of thousands of intermediate objects per run, it means a second workload racing the build for CPU and disk, and an index that is stale the moment it is written. This is the classic "why is my CI Mac slow" answer after the build itself. ## Controlling it per volume `mdutil` is the management tool for indexing state: ```bash mdutil -s /Volumes/Build # status: indexing enabled or disabled sudo mdutil -i off /Volumes/Build # stop indexing this volume sudo mdutil -i on /Volumes/Build # resume indexing sudo mdutil -E /Volumes/Build # erase the index and rebuild it mdutil -a -s # status for all volumes ``` The distinction that matters is `-i off` versus `-E`. Turning indexing off stops the work. Erasing the index throws away the store and starts a **full reindex**, which is the heaviest thing you can ask Spotlight to do. `-E` is the right move for a genuinely corrupt index producing wrong search results; it is the wrong move for high load, where it makes the symptom worse for the next hour. Also note that the boot volume is not an ordinary target. macOS protects system content, and Apple's supported ways to exclude content there are the privacy list and per-directory markers rather than switching the system volume's indexing off wholesale. ## Controlling it per directory Two file-level markers do the job without disabling a whole volume: - **`.noindex` suffix on a directory** — name the directory `artifacts.noindex` and its contents are skipped. This is the surgical option for a build output tree, and it survives being recreated by the build if the build creates it with that name. - **`.metadata_never_index`** — an empty file at the **root of a volume** tells Spotlight not to index that volume. It is a volume-level marker, not a per-folder one; dropping it inside a subdirectory does nothing, which is a common misunderstanding. The graphical equivalent is the Spotlight privacy list in system settings, which is what you use for a path you cannot rename. On a managed fleet the same exclusions can be delivered centrally as policy rather than configured per machine. ## The cost of excluding An excluded path is invisible to Spotlight, and therefore to `mdfind` — the command-line query interface — and to anything built on the metadata index. `mdls` shows the attributes Spotlight has recorded for a file, and on an excluded path it will show essentially nothing. On a build machine nobody cares; on a shared workstation, silently excluding a user's project directory turns into a support ticket about search being broken. Say the trade-off out loud when you propose the exclusion. ## A related trap: network and external volumes Spotlight also indexes mounted external and network volumes when enabled, which is how a NAS mount ends up generating sustained traffic and CPU on a machine nobody is searching from. `mdutil -s` on the mount point tells you whether that is happening, and the same `-i off` or `.metadata_never_index` at the volume root stops it. ## The shape of a good answer Name the processes and say what they are actually doing — importer plugins over change notifications, not a filename scan. Separate the volume-level control (`mdutil -i off`) from the directory-level markers (`.noindex`, the privacy list) and from the destructive rebuild (`-E`) that people reach for by mistake. Close on the trade-off: everything you exclude leaves the searchable index.

  • Someone suggests running mdutil -E to fix the high load. Why is that wrong?
    Because it erases the index and forces a complete rebuild, which is the heaviest Spotlight workload there is — the machine gets slower, not faster, for as long as the reindex runs. `-E` is the remedy for a corrupt index giving wrong or missing search results. For load, you stop the work: turn indexing off for the volume, or exclude the churning directory.
  • How do you keep one build output directory out of the index without disabling indexing for the whole volume?
    Give the directory a name ending in `.noindex`, so Spotlight skips its contents, and have the build create it under that name so it stays excluded when regenerated. Alternatively add the path to the Spotlight privacy list, which is the option when you cannot rename it. A `.metadata_never_index` file does not help here — it is a volume-root marker, not a per-directory one.
  • What breaks for users once a path is excluded from indexing?
    Anything that relies on the metadata index: Spotlight search, `mdfind` queries, and the metadata attributes `mdls` would otherwise report for those files. Applications that use Spotlight to locate content will not find it either. On a build machine that is a non-issue; on a shared workstation it should be a deliberate, communicated decision rather than a silent fix.

saying these in an interview costs you the question

  • Thinking Spotlight only indexes file names
  • Running the index erase to reduce load
  • Placing .metadata_never_index inside a subdirectory and expecting it to apply
  • Killing mds and assuming it stays dead
  • Excluding a user's directories without telling them search will miss them

context