An application team says 'docker logs' returns nothing for their container even though the app is clearly writing log files. Explain how 'docker logs' gets its data and what the team must change.

middlemust knowfreq 65%

answer

driver captures PID 1 stdout/stderr only
json-file at /var/lib/docker/containers/<id>/
ln -sf /dev/stdout for legacy apps
buffering hides logs before a crash
exec output never logged

basics

~20 s

docker logs only replays what the container's main process wrote to stdout and stderr, captured by the logging driver. Logs written to a file inside the container are invisible to it. The app must log to stdout/stderr, or the file must be symlinked to /dev/stdout.

solid answer

~50 s

At container start the daemon attaches to the main process's stdout and stderr and hands every line to the configured logging driver. With the default json-file driver each line is stored as a JSON record on the host under /var/lib/docker/containers/<id>/<id>-json.log, tagged with a timestamp and the stream name. docker logs reads that back; -f, --tail, --since and --timestamps filter it.

So anything written to /var/log/app.log inside the container never reaches the driver. Three consequences: nothing in docker logs, nothing forwarded to a central log system, and the file grows in the container's writable layer until the host disk fills — and it vanishes when the container is removed, exactly when you needed it.

Fix by following the twelve-factor convention: configure the console appender so logs go to stdout, or symlink the file to /dev/stdout. Also check for output buffering, which commonly hides logs before a crash.

At container start the daemon attaches to the main process's stdout and stderr and hands every line to the configured logging driver. With the default `json-file` driver each line is stored as a JSON record on the host under `/var/lib/docker/containers/<id>/<id>-json.log`, tagged with a timestamp and the stream name. `docker logs` reads that back; `-f`, `--tail`, `--since` and `--timestamps` filter it. So anything written to `/var/log/app.log` inside the container never reaches the driver. Three consequences: nothing in `docker logs`, nothing forwarded to a central log system, and the file grows in the container's writable layer until the host disk fills — and it vanishes when the container is removed, exactly when you needed it. Fix by following the twelve-factor convention: configure the console appender so logs go to stdout, or symlink the file to `/dev/stdout`. Also check for output buffering, which commonly hides logs before a crash.

code

dockerfile · 3 lines

dockerfile

FROM nginx:1.27-alpine
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
 && ln -sf /dev/stderr /var/log/nginx/error.log

go deeper

for a junior

Say that only the main process's stdout and stderr are captured, and that the app must log there instead of to a file.

for a middle

Add the json-file storage path, the /dev/stdout symlink trick, and buffering as a common cause of missing output.

for a senior

Connect it to operations: disk growth in the writable layer, lost evidence after container removal, and driver-dependent availability of docker logs.

for a principal

Make stdout logging a platform contract so routing, retention and shipping are owned by the runtime rather than each service.

The capture path

Container logging in Docker is deliberately narrow. At start, the runtime gives the main process a pipe (or a pty when -t is used) for stdout and stderr. The daemon reads from it and passes each message — tagged with a timestamp and whether it came from stdout or stderr — to the container's logging driver. docker logs is a client of that driver, not of the process.

With the default json-file driver, records land in /var/lib/docker/containers/<container-id>/<container-id>-json.log, one JSON object per line: {"log":"...\n","stream":"stdout","time":"..."}. That file backs docker logs, --tail, --since and --timestamps.

Why file logging breaks the model

If the application writes to a file inside the container, the daemon never sees those bytes. Symptoms: docker logs is empty or shows only startup noise; any centralised pipeline built on the logging driver receives nothing; and the file accumulates in the container's writable layer, which sits on the host disk and is not separately sized, so it can fill /var/lib/docker. Worse, that file is destroyed when the container is removed, so a crashed container's evidence disappears.

The fixes, in order of preference

Log to stdout/stderr from the app. Configure the console appender (Logback or Log4j2 ConsoleAppender, Python StreamHandler, access_log /dev/stdout in nginx). Structured JSON on stdout is ideal for downstream parsing.
Symlink the file to the process's stream: RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log. This is what the official nginx image does; /dev/stdout inside the container resolves to the main process's stream.
Only if the app cannot be changed: a log-shipping sidecar reading a shared volume. This is a fallback — it reintroduces rotation and lifecycle management you would otherwise get for free.

Avoid running a file-tailing helper alongside the real service inside the same container: it works, but it adds a supervisor and hides process failures.

Details people get wrong

Only PID 1's streams are captured. Output from docker exec, or from a child whose stdout was redirected to a file, never appears.
Buffering. Many runtimes buffer stdout when it is a pipe rather than a TTY, so logs appear in bursts or are lost before a crash. PYTHONUNBUFFERED=1, stdbuf -oL, or a framework-level flush setting fixes it. This is a very common 'my logs are missing' cause that has nothing to do with Docker itself.
stdout and stderr are both captured and both shown by docker logs; the stream field distinguishes them.
docker logs is not universally available: it works with json-file, local and journald. With syslog, fluentd, gelf or awslogs it returns an error, because the daemon keeps no local copy.
Timestamps recorded by the driver are when Docker read the line, not when the app produced it; under load these diverge, so keep the application's own timestamp inside the message.

Why the convention is right

Writing to stdout puts log routing in the runtime's hands rather than the application's. The same image can then run with json-file locally, journald on a VM, or a cluster-level collector elsewhere, without touching application configuration — and rotation, retention and shipping become platform concerns instead of per-service code.

## The capture path Container logging in Docker is deliberately narrow. At start, the runtime gives the main process a pipe (or a pty when `-t` is used) for stdout and stderr. The daemon reads from it and passes each message — tagged with a timestamp and whether it came from stdout or stderr — to the container's **logging driver**. `docker logs` is a client of that driver, not of the process. With the default **json-file** driver, records land in `/var/lib/docker/containers/<container-id>/<container-id>-json.log`, one JSON object per line: `{"log":"...\n","stream":"stdout","time":"..."}`. That file backs `docker logs`, `--tail`, `--since` and `--timestamps`. ## Why file logging breaks the model If the application writes to a file inside the container, the daemon never sees those bytes. Symptoms: `docker logs` is empty or shows only startup noise; any centralised pipeline built on the logging driver receives nothing; and the file accumulates in the container's writable layer, which sits on the host disk and is not separately sized, so it can fill `/var/lib/docker`. Worse, that file is destroyed when the container is removed, so a crashed container's evidence disappears. ## The fixes, in order of preference 1. **Log to stdout/stderr from the app.** Configure the console appender (Logback or Log4j2 ConsoleAppender, Python StreamHandler, `access_log /dev/stdout` in nginx). Structured JSON on stdout is ideal for downstream parsing. 2. **Symlink the file to the process's stream**: `RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log`. This is what the official nginx image does; `/dev/stdout` inside the container resolves to the main process's stream. 3. **Only if the app cannot be changed**: a log-shipping sidecar reading a shared volume. This is a fallback — it reintroduces rotation and lifecycle management you would otherwise get for free. Avoid running a file-tailing helper alongside the real service inside the same container: it works, but it adds a supervisor and hides process failures. ## Details people get wrong - **Only PID 1's streams are captured.** Output from `docker exec`, or from a child whose stdout was redirected to a file, never appears. - **Buffering.** Many runtimes buffer stdout when it is a pipe rather than a TTY, so logs appear in bursts or are lost before a crash. `PYTHONUNBUFFERED=1`, `stdbuf -oL`, or a framework-level flush setting fixes it. This is a very common 'my logs are missing' cause that has nothing to do with Docker itself. - **stdout and stderr are both captured** and both shown by `docker logs`; the `stream` field distinguishes them. - **`docker logs` is not universally available**: it works with json-file, local and journald. With syslog, fluentd, gelf or awslogs it returns an error, because the daemon keeps no local copy. - **Timestamps** recorded by the driver are when Docker read the line, not when the app produced it; under load these diverge, so keep the application's own timestamp inside the message. ## Why the convention is right Writing to stdout puts log routing in the runtime's hands rather than the application's. The same image can then run with json-file locally, journald on a VM, or a cluster-level collector elsewhere, without touching application configuration — and rotation, retention and shipping become platform concerns instead of per-service code.

An app logs to stdout but nothing appears until it crashes, and then some lines are missing. What is happening?
Standard output is block-buffered when it is a pipe rather than a terminal, so the runtime holds lines until the buffer fills and loses whatever is unflushed when the process dies. Disable buffering or force line buffering — for example PYTHONUNBUFFERED=1, stdbuf -oL, or the logging framework's immediate-flush setting.
Why does 'docker logs' fail with an error on some containers?
Because docker logs can only replay output the daemon stored locally. That holds for the json-file, local and journald drivers. If the container uses a forwarding driver such as syslog, fluentd, gelf or awslogs, there is no local copy and the command reports that the driver does not support reading logs; you query the destination system instead.

saying these in an interview costs you the question

Believing docker logs reads log files from inside the container
Writing application logs only to a file in the container
Ignoring stdout buffering when logs appear missing
Assuming docker logs works with every logging driver
Expecting logs to survive after the container is removed

An application team says 'docker logs' returns nothing for their container even though the app is clearly writing log files. Explain how 'docker logs' gets its data and what the team must change.

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

The capture path

Why file logging breaks the model

The fixes, in order of preference

Details people get wrong

Why the convention is right

▾common follow-ups2 questions

▾red flags5 flags

§ context

▾related questions7 questions

▾related topics3 topics

The capture path

Why file logging breaks the model

The fixes, in order of preference

Details people get wrong

Why the convention is right

answer

go deeper

context