A script refreshes a config file with `curl -sS https://example.com/config.json > /etc/app/config.json`, and readers occasionally load an empty or half-written file. What is happening, and what is the standard shell fix?

middleshould knowfreq 38%

answer

the target dies before the data arrives
a failed download destroys the good copy
stage it beside the destination
one rename, not a stream of bytes
same filesystem or nothing is gained

basics

~20 s

The redirection truncates the destination the moment it opens it, long before any bytes arrive, so readers see a growing file — or nothing at all if the download fails. Write to a temp file in the same directory and rename it over the destination.

solid answer

~50 s

> opens the target with truncate: the old contents are destroyed as the pipeline starts, and the new contents dribble in over the next few hundred milliseconds. Any reader in that window sees a truncated file, and if curl fails halfway or the network drops, the good configuration is gone permanently — you have turned a transient network problem into a broken service. The fix is to publish rather than overwrite: create a temp file in the same directory as the destination with tmp=$(mktemp /etc/app/config.json.XXXXXX), write and validate it there, set the mode readers expect (mktemp makes it 0600), and finish with mv -f "$tmp" /etc/app/config.json. A rename within one filesystem replaces the name in a single step, so every reader sees either the whole old file or the whole new one. Remove the temp file on the failure path via your cleanup.

`>` opens the target with truncate: the old contents are destroyed as the pipeline starts, and the new contents dribble in over the next few hundred milliseconds. Any reader in that window sees a truncated file, and if `curl` fails halfway or the network drops, the good configuration is gone permanently — you have turned a transient network problem into a broken service. The fix is to publish rather than overwrite: create a temp file in the *same directory* as the destination with `tmp=$(mktemp /etc/app/config.json.XXXXXX)`, write and validate it there, set the mode readers expect (mktemp makes it 0600), and finish with `mv -f "$tmp" /etc/app/config.json`. A rename within one filesystem replaces the name in a single step, so every reader sees either the whole old file or the whole new one. Remove the temp file on the failure path via your cleanup.

code

bash · 12 lines

bash

#!/usr/bin/env bash
set -euo pipefail

dest=./config.json
tmp=$(mktemp "$dest.XXXXXX")
trap 'rm -f -- "$tmp"' EXIT

printf '{"ok":true}\n' > "$tmp"
[[ -s $tmp ]]
chmod 0644 "$tmp"
mv -f -- "$tmp" "$dest"
trap - EXIT

go deeper

for a junior

Know that > empties the target file immediately, before the command produces output, so a slow or failing command leaves the destination empty. The safe habit is to write elsewhere first and move the finished file into place.

for a middle

Explain the whole pattern: mktemp beside the destination, validate the content, set the mode before moving, then mv — and say why a rename within one filesystem replaces the name in a single step while a cross-filesystem move does not.

for a senior

Show the failure analysis a reviewer wants: a bad refresh must leave the last known-good file live, an error page must not be published as valid config, and readers holding the old descriptor need an explicit reload path.

for a principal

Own the distinction between atomic visibility, durability and multi-file consistency, and decide where each guarantee should live — in the shell, in the application that writes the file, or in a delivery mechanism that publishes a whole versioned directory at once.

What the redirection actually does

cmd > file is two things: the shell opens file for writing, creating it if absent and truncating it to zero length if present, and then runs cmd with that descriptor as stdout. The truncation happens first, before the command has produced a single byte. For a fast local command the window is milliseconds; for a network fetch it is as long as the transfer takes.

Three distinct failures come out of that:

Torn reads. Anything reading the file mid-transfer gets a prefix of the new content. A JSON or YAML parser fails on it; a shell script sourcing it may execute half a line.
Destruction on failure. If curl exits non-zero, or the server returns an error page, or the process is killed, the destination is left empty or wrong — and the last known-good copy is already gone.
Success that is not success. Without --fail, curl writes a 404 body to the file and exits 0, so the file is complete, valid-looking and wrong.

Publish instead of overwrite

The pattern has four steps, and each one matters:

bash

dest=/etc/app/config.json
tmp=$(mktemp "$dest.XXXXXX")
trap 'rm -f -- "$tmp"' EXIT

curl -sS --fail -o "$tmp" https://example.com/config.json
[[ -s $tmp ]]                      # non-empty
chmod 0644 "$tmp"
mv -f -- "$tmp" "$dest"

Write somewhere else. The destination keeps its old content, valid and readable, for the entire duration of the download.

Validate before publishing. At minimum check the command's exit status and that the file is non-empty; better, parse it with whatever will consume it. This is the step that turns "the config was replaced with an error page" into "the refresh failed and the old config is still live", which is nearly always the behaviour you want.

Fix the metadata first. mktemp creates mode 0600 for good reasons, but the destination usually needs to be readable by a service account. Set the mode (and owner, if you are root) on the temp file before the rename — after the rename, the destination carries the temp file's metadata, and the old file's permissions are gone.

Rename last. mv within one filesystem replaces the destination name in a single step. There is no instant at which the name refers to a partial file: a reader that opens it gets either the complete old content or the complete new content.

Same directory, or the property evaporates

This is the detail people miss. If the temp file lives in /tmp and the destination is on a different filesystem, mv cannot rename — it falls back to copying the bytes across and then removing the source. That copy has exactly the same truncate-then-fill behaviour you were trying to avoid, so the whole exercise buys nothing. Creating the temp file next to the destination (mktemp "$dest.XXXXXX") guarantees the same filesystem, and as a bonus fails early if the destination directory is not writable rather than after the download.

A process that has the old file already open keeps reading the old content after the rename — it holds the file, not the name. That is usually what you want, but it means readers need to reopen (or be signalled) to pick up the change.

What this still does not give you

Durability. The rename makes the switch all-or-nothing to other processes, but it does not promise the bytes have reached the disk. A power cut immediately afterwards can leave the new name in place with unflushed content. The shell has no per-file flush; sync is the blunt instrument, and true durability is a job for the program writing the file.

Multi-file consistency. Two files renamed one after the other are two separate switches, and a reader can catch the pair mid-way. Where a set of files must change together, stage them in a new directory and swap a symlink in one operation — build releases/2026-08-21, point a temporary symlink at it, and rename that symlink over current (with GNU mv -T, so the symlink is replaced rather than moved inside the directory it points to). Note ln -sfn is not an equivalent atomic swap; it removes and recreates the link.

Concurrency between writers. Two scripts publishing the same destination will both succeed and the last rename wins. If that is not acceptable, the writers need a lock as well.

## What the redirection actually does `cmd > file` is two things: the shell opens `file` for writing, creating it if absent and **truncating it to zero length** if present, and then runs `cmd` with that descriptor as stdout. The truncation happens first, before the command has produced a single byte. For a fast local command the window is milliseconds; for a network fetch it is as long as the transfer takes. Three distinct failures come out of that: 1. **Torn reads.** Anything reading the file mid-transfer gets a prefix of the new content. A JSON or YAML parser fails on it; a shell script sourcing it may execute half a line. 2. **Destruction on failure.** If `curl` exits non-zero, or the server returns an error page, or the process is killed, the destination is left empty or wrong — and the last known-good copy is already gone. 3. **Success that is not success.** Without `--fail`, `curl` writes a 404 body to the file and exits 0, so the file is complete, valid-looking and wrong. ## Publish instead of overwrite The pattern has four steps, and each one matters: ```bash dest=/etc/app/config.json tmp=$(mktemp "$dest.XXXXXX") trap 'rm -f -- "$tmp"' EXIT curl -sS --fail -o "$tmp" https://example.com/config.json [[ -s $tmp ]] # non-empty chmod 0644 "$tmp" mv -f -- "$tmp" "$dest" ``` **Write somewhere else.** The destination keeps its old content, valid and readable, for the entire duration of the download. **Validate before publishing.** At minimum check the command's exit status and that the file is non-empty; better, parse it with whatever will consume it. This is the step that turns "the config was replaced with an error page" into "the refresh failed and the old config is still live", which is nearly always the behaviour you want. **Fix the metadata first.** `mktemp` creates mode 0600 for good reasons, but the destination usually needs to be readable by a service account. Set the mode (and owner, if you are root) on the temp file *before* the rename — after the rename, the destination carries the temp file's metadata, and the old file's permissions are gone. **Rename last.** `mv` within one filesystem replaces the destination name in a single step. There is no instant at which the name refers to a partial file: a reader that opens it gets either the complete old content or the complete new content. ## Same directory, or the property evaporates This is the detail people miss. If the temp file lives in `/tmp` and the destination is on a different filesystem, `mv` cannot rename — it falls back to copying the bytes across and then removing the source. That copy has exactly the same truncate-then-fill behaviour you were trying to avoid, so the whole exercise buys nothing. Creating the temp file next to the destination (`mktemp "$dest.XXXXXX"`) guarantees the same filesystem, and as a bonus fails early if the destination directory is not writable rather than after the download. A process that has the *old* file already open keeps reading the old content after the rename — it holds the file, not the name. That is usually what you want, but it means readers need to reopen (or be signalled) to pick up the change. ## What this still does not give you **Durability.** The rename makes the switch all-or-nothing to other processes, but it does not promise the bytes have reached the disk. A power cut immediately afterwards can leave the new name in place with unflushed content. The shell has no per-file flush; `sync` is the blunt instrument, and true durability is a job for the program writing the file. **Multi-file consistency.** Two files renamed one after the other are two separate switches, and a reader can catch the pair mid-way. Where a set of files must change together, stage them in a new directory and swap a symlink in one operation — build `releases/2026-08-21`, point a temporary symlink at it, and rename that symlink over `current` (with GNU `mv -T`, so the symlink is replaced rather than moved inside the directory it points to). Note `ln -sfn` is *not* an equivalent atomic swap; it removes and recreates the link. **Concurrency between writers.** Two scripts publishing the same destination will both succeed and the last rename wins. If that is not acceptable, the writers need a lock as well.

Why must the temp file live in the destination's own directory rather than /tmp?
So the final `mv` is a rename within one filesystem, which swaps the name in a single step. If the temp file is on a different filesystem, `mv` degrades into copy-then-delete, and the copy exposes exactly the partially written destination you were trying to prevent. Creating it beside the destination also fails fast when that directory is not writable.
A service already has the old config file open when you rename the new one into place. What does it see?
The old content, for as long as it keeps that descriptor open — it holds the file, not the name. That is usually desirable, since a running process is not yanked out from under itself, but it means the new configuration takes effect only when the reader reopens the path or is signalled to reload. Plan the reload step alongside the publish step.
Does the rename guarantee the new content survives a power cut?
No. Atomicity for other processes and durability on disk are different properties: the rename can be visible while the file's data is still buffered. The shell has no targeted flush, so if durability matters the writing program should fsync before the rename, or you accept the exposure. Say this out loud in an interview — conflating the two is the common error.
How do you publish several files that must change together?
Do not rename them one by one; a reader can observe the half-updated set. Stage the whole set in a new directory and switch a single symlink to it in one rename — with GNU `mv -T` over the existing link, since `ln -sfn` removes and recreates the link and so has a visible gap. One switch, one observable change.

saying these in an interview costs you the question

Thinks the redirection only writes when data arrives
Blames the reader and adds a retry loop
Puts the temp file in /tmp and renames across filesystems
Renames first and fixes permissions afterwards
Believes the rename also flushes the data to disk

A script refreshes a config file with `curl -sS https://example.com/config.json > /etc/app/config.json`, and readers occasionally load an empty or half-written file. What is happening, and what is the standard shell fix?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

What the redirection actually does

Publish instead of overwrite

Same directory, or the property evaporates

What this still does not give you

▾common follow-ups4 questions

▾red flags5 flags

§ context

▾related questions7 questions

▾related topics3 topics

What the redirection actually does

Publish instead of overwrite

Same directory, or the property evaporates

What this still does not give you

answer

go deeper

context