skip to content

You add an fstab entry for a new data disk, reboot, and the machine drops to an emergency shell instead of coming up. Why can one fstab line stop the whole boot, and which mount options let a non-critical filesystem fail without taking the system down?

level: seniorimportance: should knowfreq 55%

answer

  1. fstab lines become boot dependencies
  2. required by local-fs.target
  3. the option that makes it best-effort
  4. and the 90-second device wait

basics

~20 s

Every fstab entry becomes a boot-time dependency: on a systemd machine the fstab generator turns each line into a mount unit that local-fs.target requires, so a missing device or failed mount fails the target and drops you to emergency mode. Mark optional filesystems nofail.

solid answer

~50 s

On a systemd system, `systemd-fstab-generator` reads `/etc/fstab` at boot and turns each line into a `.mount` unit, and by default those units are pulled in as *required* by `local-fs.target`. If the device never appears, or the mount or its fsck fails, that target fails and the boot stops in emergency mode asking for the root password — which on a headless server means it is simply unreachable. The options that change this are `nofail`, which makes the mount best-effort so a failure is logged and ignored; `noauto`, which stops it being mounted at boot at all; `_netdev`, which declares the filesystem needs the network and orders it after the network is up; and `x-systemd.device-timeout=`, which shortens the default 90-second wait for a device that may not be there. My rule is that anything that is not root or genuinely required by a service gets `nofail`, and I validate with `findmnt --verify` and `mount -a` before ever rebooting.

go deeper

for a junior

Know that a wrong line in /etc/fstab can stop a machine from booting, and that you should run mount -a to test an entry before you reboot.

for a middle

Explain that fstab entries become mount units required by local-fs.target, and name the options that relax that: nofail, noauto, _netdev and a shortened device timeout.

for a senior

Show the operational habit — nofail by default on non-essential disks, findmnt --verify plus mount -a before any remote reboot, and a concrete recovery path from the bootloader when a machine is already stuck in emergency mode.

for a principal

Own the policy: which filesystems are allowed to be boot-critical at all, whether data disks are mounted via automount instead, and how machines are recovered out-of-band when console access is not available.

## What an fstab line becomes at boot On any current distribution, `/etc/fstab` is not read by a shell script at boot; `systemd-fstab-generator` parses it early and emits a `.mount` unit per entry, plus an `.automount` unit where you asked for one. The unit is named after the escaped mount point — `/data` becomes `data.mount`. Ordinary local filesystems get pulled into `local-fs.target`, and remote ones into `remote-fs.target`. Because those units are *required*, not merely wanted, a single failing mount fails the target that ordinary services depend on. When that happens, systemd stops the normal boot and starts `emergency.target`: a single-user shell on the console that prompts for the root password. On a laptop this is an inconvenience. On a rack machine or a cloud instance it is an outage that a reboot will not fix, because the shell waits forever and nothing else starts — no SSH, no network. This is why an unrelated fstab typo is one of the classic "the server never came back" incidents. ## Which failures reach that state - **The device does not exist.** The mount unit waits for the device unit to appear (default timeout 90 seconds), then fails. Typos in a UUID land here, as do disks that were detached, and disks whose UUID changed because someone reformatted them. - **The filesystem will not mount.** Wrong `fstype`, an option the filesystem rejects, or corruption. - **fsck fails.** The sixth field, the pass number, controls this: `1` for root, `2` for other locally checked filesystems, `0` to skip. A filesystem that needs manual repair fails its `systemd-fsck@` unit and takes the mount with it. ## The options that make a mount optional **`nofail`** — the important one. The mount unit is still generated and still attempted, but it is no longer a hard requirement of `local-fs.target`. Boot proceeds; the failure is visible in the logs. Every filesystem that is not required for the machine to be reachable should carry it. **`noauto`** — not mounted at boot at all. The entry exists so that `mount /data` works with just the mount point, and `mount -a` skips it. Useful for removable media and for filesystems a service mounts itself. **`_netdev`** — declares that the filesystem needs the network, which orders it after the network is up and puts it under `remote-fs.target` instead of `local-fs.target`. Required for iSCSI, NFS-backed loopbacks and network block devices; without it the mount is attempted before an interface has an address. **`x-systemd.device-timeout=`** — how long to wait for the backing device before giving up. The default is 90 seconds, and combined with a missing device that is 90 seconds of a boot that looks hung. Setting `x-systemd.device-timeout=10s` alongside `nofail` makes an absent optional disk cost ten seconds instead of a minute and a half. **`x-systemd.automount`** (usually with `x-systemd.idle-timeout=`) — creates an automount point instead of a mount: nothing is mounted until something first touches the path, at which point the mount is performed. This removes the filesystem from the boot path entirely, which is the strongest form of "do not let this block me". ```bash # optional data disk that must never block the boot UUID=6f3c1e6a-06a1-4c39-9ad3-2f9f2b7f8a11 /data ext4 defaults,nofail,x-systemd.device-timeout=10s 0 2 ``` ## The discipline around editing fstab Editing the file changes nothing by itself. Run `systemctl daemon-reload` so the generator re-runs and the new units exist, then `mount -a` to prove the entries actually work on the running system. `findmnt --verify` parses fstab and reports unreachable sources, unknown filesystem types and duplicate mount points before you find them the hard way. Only after both are clean should a remote machine be rebooted. ## Recovering a machine that is already stuck If you have console access, the emergency shell can be logged into with the root password, and the fix is to remount root writable (`mount -o remount,rw /`), correct the entry and reboot. If you do not, the recovery is out-of-band: boot with `systemd.unit=emergency.target` or `init=/bin/sh` appended to the kernel command line from the bootloader, or attach the disk to a rescue instance and edit fstab there. That asymmetry — trivially fixable at the console, painful without one — is exactly the argument for `nofail` as a default habit.

  • What is the difference between nofail and noauto?
    `nofail` still attempts the mount at boot but does not fail the boot if it does not work — the filesystem is mounted when the device is present. `noauto` does not attempt it at all: it is skipped by `mount -a` and by boot, and exists only so an operator or a service can run `mount /data` and have the options come from fstab. Use `nofail` for a disk that should normally be there, `noauto` for one that normally is not.
  • Why does an NFS or iSCSI mount need _netdev, and what happens without it?
    `_netdev` tells the generator this filesystem depends on the network, so the mount is ordered after the network is configured and grouped under remote-fs.target rather than local-fs.target. Without it the mount is attempted during early local filesystem setup, before any interface has an address, so it fails or blocks until its timeout. NFS mounts are usually recognised by their type, but block-device-backed network storage such as iSCSI needs the option explicitly.

saying these in an interview costs you the question

  • Thinks fstab is only read by a manual mount -a
  • Assumes a bad entry just gets skipped at boot
  • Adds nofail to the root filesystem entry
  • Reboots a remote server without testing with mount -a
  • Believes the boot hang is a hardware fault rather than a mount dependency

context