skip to content

Design a write path that must survive a power loss. Where do flush(), close(), and fsync fit, and what guarantees does each give?

level: principalimportance: nice to knowfreq 22%

answer

  1. Three layers: JVM buffer -> OS page cache -> device
  2. flush=JVM->OS, close=flush+release, fsync(force)=OS->device
  3. close() survives JVM crash, NOT power loss; fsync survives power loss
  4. Durable pattern: write temp, fsync, rename atomically, fsync dir
  5. fsync is costly -> batch / group commit

basics

~20 s

Writing data passes through layers: your JVM buffer, then the operating system's cache, then the physical disk. flush() moves data from the JVM to the OS, close() flushes and releases the handle, but only an OS sync (fsync) forces the OS to write to the disk so it survives a power loss.

solid answer

~50 s

Durability is layered. Your application buffer (BufferedWriter) holds bytes in the JVM; flush() pushes them to the OS via a write() syscall. close() flushes then releases the descriptor. But the OS keeps written data in its page cache and reports success before it hits the device, so a power loss after flush/close can still lose data. For durability you must force the OS to persist: FileChannel.force(true) or FileOutputStream.getFD().sync(), which issue fsync. The 'true' on force also persists file metadata (size, timestamps). A robust write path: write to a temp file, flush, fsync the file, close, atomically rename it over the target, then fsync the parent directory so the rename itself is durable. This is the write-temp-then-rename pattern used by databases and config writers. Trade-off: fsync is expensive (waits for the device), so batch it or use group commit rather than syncing per record.

code

java · 19 lines
java
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;

Path target = Path.of("config.json");
Path tmp = Path.of("config.json.tmp");
byte[] data = json.getBytes(StandardCharsets.UTF_8);

try (FileChannel ch = FileChannel.open(tmp,
        StandardOpenOption.CREATE, StandardOpenOption.WRITE,
        StandardOpenOption.TRUNCATE_EXISTING)) {
    ch.write(java.nio.ByteBuffer.wrap(data));
    ch.force(true);                 // fsync data + metadata to the device
}                                   // close() releases the handle

// Atomic publish: readers see old or new file, never a partial one
Files.move(tmp, target, StandardCopyOption.ATOMIC_MOVE,
        StandardCopyOption.REPLACE_EXISTING);
// (For full durability, also fsync the parent directory.)

go deeper

for a junior

Recognizes there is a difference between writing and saving, and that close() handles flushing.

for a middle

Knows flush moves data to the OS and that durability needs something more, even if hazy on the exact API.

for a senior

Names FileChannel.force/FD.sync, distinguishes JVM-crash vs power-loss safety, and knows the write-temp-then-rename idea.

for a principal

Designs the full durable, atomic write path (temp, fsync, atomic rename, dir fsync), reasons about group-commit trade-offs, device write caches, and per-system durability policy.

## The layered model of a write When your Java program writes data, it travels through several stages, each with its own buffer and its own failure window: 1. **JVM/application buffer** — e.g. the `char[]`/`byte[]` inside a `BufferedWriter`/`BufferedOutputStream`. `write()` calls just fill this. 2. **OS page cache** — when the JVM buffer is flushed, the JVM issues a `write()` **system call**, copying data into the operating system's in-memory cache. The OS returns 'success' immediately, *before* the data is on the device. This is **write-back caching** and is why I/O feels fast. 3. **Storage device** — the disk/SSD. Data only survives a power loss once it is here (and even then, device caches add nuance). Each Java operation acts on a specific layer, and that's the crux of designing for durability. ## What each call guarantees - **`flush()`** — moves data from the **JVM buffer to the OS** (layer 1 -> 2). After flush, a *JVM crash* won't lose the data (the OS has it), but a *power loss* still can. - **`close()`** — flushes once, then releases the file descriptor. Same durability level as flush: it does **not** by itself force the device. - **`FileChannel.force(boolean)`** / **`FileDescriptor.sync()`** — issues **fsync**, telling the OS to push its cache to the **device** (layer 2 -> 3) and wait until done. This is what survives power loss. `force(true)` also persists **metadata** (file length, mtime); `force(false)` may persist only the data (`fdatasync`-like). So the durability ladder is: write -> flush/close (crash-safe vs JVM) -> fsync (power-safe vs OS). ## Why close() is not enough A frequent senior-vs-principal distinction: 'I closed the file, so it's saved.' close() guarantees the bytes left the JVM and the handle is freed, but the OS may still hold them in cache. Pull the plug and they're gone. Durable writers must call fsync. ## The atomic, durable write pattern (write-temp-then-rename) Naively overwriting a file in place is dangerous: a crash mid-write leaves a half-written, corrupt file. The canonical robust recipe — used by databases, package managers, and careful config writers: 1. Write the new content to a **temporary file** in the same directory. 2. **flush** and **fsync** the temp file (its data is now on the device). 3. **close** the temp file. 4. **Atomically rename** the temp file over the target (`Files.move(tmp, target, ATOMIC_MOVE, REPLACE_EXISTING)`). On POSIX, rename is atomic: a reader sees either the old or the new file, never a partial one. 5. **fsync the parent directory** so the rename (a directory metadata change) is itself durable — otherwise a crash can lose the rename even though the file data is safe. This yields **atomicity** (all-or-nothing visibility) and **durability** (survives power loss). ## Cost and batching fsync is **expensive**: it blocks until the device confirms, which can be milliseconds — orders of magnitude slower than a buffered write. Calling it per record destroys throughput. Real systems use **group commit**: accumulate many writes, then a single fsync amortizes the cost across them, trading a bounded amount of latency/durability-window for throughput. The durability *policy* (how many records may be lost on crash) is a deliberate design knob. ## Caveats at the edge - Some disks have **volatile on-device write caches**; without barriers/FUA even fsync's guarantee can be undermined on misbehaving hardware. Enterprise storage and proper barriers address this. - Network filesystems (NFS) have weaker/again-different fsync semantics. - `FileChannel` (NIO) is the modern handle for `force()`; you can get it from `FileOutputStream.getChannel()` or `RandomAccessFile`. ## Mapping back to the original topic This is the principal-level extension of 'flush vs close vs encoding': flush and close are about getting bytes out of the JVM correctly; fsync is about making them *survive*. Knowing which layer each touches is what lets you design a write path with explicit, defensible guarantees.

  • If close() flushes, why isn't close() enough for durability?
    close() flushes the JVM buffer to the OS and releases the descriptor, but the OS may still hold the bytes in its page cache and report success before writing to the device. A power loss then loses the data. Only fsync (FileChannel.force / FD.sync) forces the OS to persist to the device.
  • Why fsync the parent directory after an atomic rename?
    A rename is a change to the directory's metadata. The file's data may be durable, but if the directory entry pointing to the new file isn't fsynced, a crash can lose the rename, leaving the old file. Syncing the directory makes the rename itself durable.

Think of mailing a letter. flush() = dropping it in your office outbox (it left your desk). close() = the mailroom took it. fsync = the postal service confirming delivery to the recipient's hand. Only the last one survives 'your building burning down' (power loss); the letter sitting in the mailroom (OS cache) does not.

saying these in an interview costs you the question

  • Believing close() (or flush()) guarantees data survives power loss
  • Overwriting a file in place instead of write-temp-then-rename
  • Calling fsync per record and tanking throughput with no batching
  • Forgetting to fsync the parent directory after the rename
  • Assuming all hardware honors fsync regardless of volatile device caches

context

resolving links…