skip to content

Engine Operations

The dockerd daemon itself: how daemon.json configures it, how the CLI reaches a local or remote engine, where container stdout and image data land on the host, and what to reclaim when disk fills. Most production Docker incidents are host incidents.

part ofDockeroverview, primer and where to startread it →
on this pageshow

questions

page 1 of 2

What is the Docker Engine API, and how does the docker CLI use it?

level: juniorimportance: must knowfreq 62%

answer

  1. The CLI is not where the work happens
  2. Something is listening on a socket
  3. Ordinary HTTP over /var/run/docker.sock
  4. Version-prefixed paths, JSON bodies
  5. docker run is create plus start

basics

~20 s

The Docker Engine API is the JSON-over-HTTP API that dockerd serves, by default on the unix socket /var/run/docker.sock. The docker CLI holds no container logic: each command becomes an HTTP request such as POST /containers/create.

solid answer

~50 s

`dockerd` is an HTTP server. The Docker Engine API is the REST-style API it serves, by default on the unix socket `/var/run/docker.sock`, and the `docker` CLI is a thin client over it: `docker ps` is `GET /containers/json`, and `docker run` is really `POST /containers/create` followed by `POST /containers/{id}/start`. Paths carry a version prefix (`/v1.43/containers/json`), request and response bodies are JSON, and long-running calls such as `/events`, `/containers/{id}/logs?follow=1`, image pulls and builds stream instead of returning once. You can call it by hand: `curl --unix-socket /var/run/docker.sock http://localhost/v1.43/containers/json` — the hostname is ignored because the transport is the socket. There is no login on that API; on Linux the socket's ownership (`root:docker`) is the access control. Everything that automates Docker — Compose, CI agents, monitoring agents, the Go and Python SDKs — is another client of the same API.

code

bash · 14 lines
bash
# List running containers
curl -s --unix-socket /var/run/docker.sock \
  http://localhost/v1.43/containers/json

# Create and start a PHP-FPM container, the two calls the CLI makes
id=$(curl -s --unix-socket /var/run/docker.sock \
  -H 'Content-Type: application/json' \
  -d '{"Image":"php:8.3-fpm","Cmd":["php-fpm"]}' \
  -X POST 'http://localhost/v1.43/containers/create?name=pdfsign' \
  | sed 's/.*"Id":"\([^"]*\)".*/\1/')

curl -s -o /dev/null -w '%{http_code}\n' -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost/v1.43/containers/$id/start"

go deeper

for a junior

Be able to say in one sentence that dockerd serves an HTTP API on /var/run/docker.sock and the docker command is just a client of it. Naming one endpoint, such as GET /containers/json, is enough at this level.

for a middle

Be ready to decompose a CLI command into its calls — create, then start, with a pull in between when the image is missing — and to run one of them with curl --unix-socket, including the JSON body and Content-Type header.

for a senior

Show that you reach for the API when the CLI is misleading: reading the exact HostConfig a tool sent, or handling the streaming endpoints correctly so a build that failed mid-stream is not reported as a success.

for a principal

Own the decision of how your platform's tooling talks to the engine — an SDK against a pinned API version versus shelling out to a CLI — and the blast radius that follows from any process being able to open that socket.

### The daemon is a web server Docker is a client/server system. `dockerd` is the long-running daemon that owns images, containers, networks and volumes; the `docker` binary you type at is a *client* that holds none of that logic. Between them sits the Docker Engine API: a JSON-over-HTTP API that `dockerd` serves, by default, on the unix socket `/var/run/docker.sock`. Every container you have ever started was started by an HTTP request. Two consequences follow immediately. First, the `docker` binary is not where the interesting behaviour lives — it is a small HTTP client, and its power comes entirely from being able to open that socket. Second, anything else that can open the socket has the same power: Compose, build tooling, CI agents, monitoring agents, IDE plugins and your own shell scripts are all peers of the CLI, speaking one API. ### The shape of a request Because the transport is a unix socket there is no host to address. `curl` needs `--unix-socket`, and then any hostname at all, which it ignores: ``` curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/containers/json ``` Paths are prefixed with an API version. Bodies are JSON and a POST needs `Content-Type: application/json` — omit it and the daemon rejects the request rather than guessing. Options a CLI user passes as flags appear as JSON fields or query parameters: `docker ps -a` is `GET /containers/json?all=true`, and a status filter is a URL-encoded JSON map in the `filters` parameter. The endpoint families track the CLI's nouns — `/containers`, `/images`, `/networks`, `/volumes`, `/exec` — plus `/version`, `/info`, `/_ping`, `/events` and `/build`. ### What `docker run` actually is `docker run -d php:8.3-fpm` looks atomic and is not. The client performs: 1. `POST /containers/create`, with the image, command, environment and a `HostConfig` object carrying the host-level settings (mounts, port bindings, resource limits). Success is `201 Created` with an `Id`. 2. If that returns `404` because the image is not present locally, the CLI issues `POST /images/create?fromImage=php&tag=8.3-fpm` and retries. The API never pulls implicitly; the automatic pull is a client convenience. 3. `POST /containers/{id}/start`. 4. In the foreground case, an attach or logs call plus `POST /containers/{id}/wait` to learn the exit code. Knowing this decomposition is what makes API-level debugging possible. When `docker run` fails you can say whether creation, the pull or the start failed, because they are separate calls with separate status codes — `201` created, `404` no such image, `409` name already in use, `500` when the runtime refuses to start the process. ### Streaming endpoints Some endpoints do not return once. `GET /events`, `GET /containers/{id}/logs?follow=1`, `GET /containers/{id}/stats`, `POST /images/create` and `POST /build` hold the response open and stream — newline-delimited JSON progress objects for pulls and builds, framed byte chunks for logs and attach. A subtlety that catches people writing their own clients: a pull or a build answers `200 OK` as soon as it *starts*, and a failure arrives later *inside* the stream, as an object carrying an `errorDetail` field. A client that only checks the HTTP status will report a failed build as a success. Attach and exec go further: after the response headers the connection is hijacked and stops being HTTP, becoming a bidirectional byte stream. ### Clients and SDKs Docker publishes SDKs for Go (`github.com/docker/docker/client`) and Python (`docker`), and community libraries exist for other languages; all of them are wrappers over these HTTP calls. Choosing an SDK over shelling out to the CLI buys structured errors, no output parsing, correct handling of the streaming endpoints, and version negotiation. Shelling out buys familiarity and adds a dependency on human-readable output that changes between releases. ### There is no login The API has no username and password. On a default Linux install the socket file is owned by `root` with group `docker` and group write permission, so group membership is the whole access-control story and requests arrive at the daemon unauthenticated. That is exactly why exposing this API is treated as a security subject in its own right. ### What an interviewer is testing That you know the CLI is a thin client, that the socket carries ordinary HTTP, and that you could reach for `curl` when the CLI is unavailable or is hiding something — for example to see the exact `HostConfig` JSON a tool sent, or when writing an agent that must not shell out to a binary.

  • What does POST /containers/create return, and what does a 404 from it mean?
    `201 Created` with a body containing `Id` and any `Warnings`. A `404` means the image is not present on that daemon — the API does not pull implicitly, so the CLI reacts by calling `POST /images/create` and retrying. `409` means the name is already taken, and `400` means the JSON body was malformed or the `Content-Type` header was missing.
  • Why can an image pull or a build return HTTP 200 and still have failed?
    Those endpoints stream. The daemon sends `200 OK` when the operation begins, then emits newline-delimited JSON progress objects while it runs; a failure shows up as an object with an `errorDetail` field near the end of the stream. A correct client reads the whole stream and inspects each object rather than trusting the status line.
  • When would you use an SDK rather than shelling out to the docker CLI?
    Whenever the caller is a program rather than a person. An SDK gives typed requests, structured errors instead of parsed stderr, proper handling of the streaming and hijacked endpoints, and version negotiation. Shelling out couples you to human-readable output that changes between releases and forces you to reimplement stream handling badly.

The docker CLI is a browser for exactly one website: all the behaviour lives on the server, and the client only knows which URLs to open and how to print what comes back.

saying these in an interview costs you the question

  • Thinks the docker CLI creates containers itself
  • Believes docker run is one API call
  • Says the API needs a username and password
  • Thinks the socket speaks a custom binary protocol
  • Assumes container create pulls a missing image
  • Cannot name a single endpoint under /containers

context

open as a page

Where does the Docker engine actually run when you use Docker Desktop on macOS or Windows?

level: juniorimportance: must knowfreq 64%

basics

~20 s

Docker Desktop boots a small Linux virtual machine and runs dockerd inside it. The docker command you type on macOS or Windows is only a client talking to that daemon, so every Linux container is a process inside the VM.

open as a page

What does `docker image prune` delete by default, and what changes with `-a`?

level: juniorimportance: must knowfreq 72%

basics

~10 s

docker image prune deletes only dangling images: untagged layers orphaned when a tag moved to a newer build. Adding -a deletes every image that no container references, including tagged images you still want.

open as a page

What does `docker inspect` return, and how do you pull a single field out of it with --format?

level: juniorimportance: must knowfreq 78%

basics

~20 s

docker inspect prints a JSON array holding the engine's full low-level record for each named object — State, Config, HostConfig, NetworkSettings, Mounts. --format (or -f) applies a Go template to each object, so docker inspect -f '{{.State.Status}}' web prints just that field.

open as a page

After installing Docker Engine, `docker ps` fails with permission denied on /var/run/docker.sock — why, and what are the post-install steps?

level: juniorimportance: must knowfreq 76%

basics

~20 s

The docker CLI reaches dockerd through the unix socket /var/run/docker.sock, which is owned by root and group docker. Add your user with sudo usermod -aG docker $USER, then open a new login session so the group takes effect.

open as a page

What is Docker's default log driver, and where does it write a container's stdout on the host?

level: juniorimportance: must knowfreq 72%

basics

~20 s

Docker's default log driver is json-file: the daemon captures whatever a container writes to stdout and stderr and appends it as JSON lines to /var/lib/docker/containers/<id>/<id>-json.log on the host. A per-container --log-driver flag overrides that default.

open as a page

Why does `docker pull` from a private registry fail with `x509: certificate signed by unknown authority`, and where does the CA go?

level: juniorimportance: must knowfreq 60%

basics

~20 s

The Docker daemon, not the CLI, verifies the registry's server certificate, and nothing in that chain is trusted on the daemon's host. Install the signing CA at /etc/docker/certs.d/<registry-host>:<port>/ca.crt, or in the host's system trust bundle.

open as a page

Which daemon.json changes take effect on a Docker daemon reload, and which need a restart?

level: middleimportance: must knowfreq 60%

basics

~20 s

A SIGHUP reload re-reads only part of daemon.json — registry mirrors, insecure registries, debug level, labels, live-restore, concurrency limits and runtimes. Structural settings such as storage-driver, data-root and the daemon's listen addresses are read only at start-up and need a full restart.

open as a page

On Docker Desktop, why doesn't `--network host` put a container on the Mac's network?

level: middleimportance: must knowfreq 56%

basics

~20 s

"The host" under Docker Desktop means its Linux VM, not macOS. --network host joins the VM's network namespace and binds the VM's interfaces, and macOS has no route in. Publish with -p instead, which Desktop forwards inward.

open as a page

How do the json-file driver's max-size and max-file log options stop a Docker host filling with logs?

level: middleimportance: must knowfreq 66%

basics

~20 s

max-size caps one log file and max-file caps how many rotated copies are kept, bounding a container at roughly max-size times max-file. The json-file driver applies neither by default, and max-file does nothing unless max-size is set.

open as a page

For a Windows container, how do --isolation=process and --isolation=hyperv differ?

level: middleimportance: must knowfreq 70%

basics

~20 s

With --isolation=process a Windows container's processes run directly on the host's Windows kernel, so it starts fast and uses little memory but its base image build must match the host's. With --isolation=hyperv each container gets its own lightweight virtual machine and kernel, costing startup time and memory but removing that matching requirement.

open as a page

Why does exposing dockerd on tcp://0.0.0.0:2375 hand out root, and what is the safe alternative?

level: seniorimportance: must knowfreq 56%

basics

~20 s

Port 2375 is the plain, unauthenticated Engine API, and that API is root-equivalent: any caller can start a privileged container mounting the host root filesystem. Prefer an ssh:// endpoint, or dockerd --tlsverify on 2376, never publicly reachable.

open as a page

A Docker host is out of disk mid-build. How do you reclaim space without disrupting running containers?

level: seniorimportance: must knowfreq 63%

basics

~20 s

Measure first with docker system df -v and du on the data root, then sweep in rising order of risk: stopped containers, dangling images, aged build cache, finally image prune -a with an until= filter.

open as a page

What does `docker context use` change, and how does DOCKER_HOST interact with it?

level: juniorimportance: should knowfreq 48%

basics

~20 s

A docker context is a saved name for one engine endpoint: a unix socket, a tcp:// URL or an ssh:// URL. docker context use points every later CLI command at that daemon, and DOCKER_HOST overrides the selected context.

open as a page

What is /etc/docker/daemon.json, and how do you apply a change you make to it?

level: juniorimportance: should knowfreq 48%

basics

~20 s

/etc/docker/daemon.json is the Docker daemon's own configuration file: host-wide settings such as the default log driver, the storage driver, registry mirrors and data-root. dockerd reads it only at start-up, so an edit needs a daemon reload or restart.

open as a page

Why does docker pull fail with "no matching manifest for windows/amd64" after switching to Windows containers?

level: juniorimportance: should knowfreq 50%

basics

~20 s

A Docker engine serves one container operating system at a time. Once it is switched to Windows containers it asks the registry for a windows/amd64 image, and the image you pulled publishes only Linux entries, so nothing matches.

open as a page

How does a Docker Engine API client negotiate which API version it uses?

level: middleimportance: should knowfreq 38%

basics

~20 s

Docker Engine API paths carry the version, as in /v1.43/containers/json. A client calls GET /_ping first; the daemon answers with an Api-Version response header, and the client downgrades to that version when its own is newer. Negotiation never upgrades.

open as a page

Against a remote Docker engine, where do `-v` bind mounts and `docker build`'s context resolve?

level: middleimportance: should knowfreq 52%

basics

~20 s

Both resolve on the daemon's host, not yours. A -v path is interpreted there, so a missing directory is created empty instead of mounting your files. The build context directory is read locally but uploaded to the daemon first.

open as a page

Why are bind mounts from macOS or Windows into a Docker Desktop container slow?

level: middleimportance: should knowfreq 47%

basics

~20 s

Files stay on the host filesystem, so every open, stat and read crosses the VM boundary over a file-sharing protocol. A native Linux bind mount is a kernel operation with no crossing, so metadata-heavy trees suffer most.

open as a page

How do you read `docker system df -v`, and why is its RECLAIMABLE column misleading?

level: middleimportance: should knowfreq 47%

basics

~10 s

docker system df totals four buckets: images, containers, local volumes, build cache. -v itemises them with each image's SHARED and UNIQUE size. RECLAIMABLE assumes the aggressive sweep, not what a default prune returns.

open as a page

How do you read the columns of `docker stats`, and when do you need --no-stream?

level: middleimportance: should knowfreq 61%

basics

~20 s

docker stats streams a live per-container table of CPU %, MEM USAGE / LIMIT, MEM %, NET I/O, BLOCK I/O and PIDS, refreshing until interrupted. --no-stream prints one sample and exits, which is what any script or CI step needs so the command terminates.

open as a page

Why does the docker CLI report "client version is too new" against a host's daemon, and what does DOCKER_API_VERSION do?

level: middleimportance: should knowfreq 48%

basics

~20 s

The docker CLI and dockerd speak a versioned REST API. The client normally negotiates down to the daemon's version; the refusal appears when that negotiation is bypassed or the daemon is too old. DOCKER_API_VERSION pins the client to a fixed version.

open as a page

`docker pull` hangs behind a corporate proxy though HTTP_PROXY is exported in your shell - where does the proxy setting belong?

level: middleimportance: should knowfreq 48%

basics

~20 s

The docker CLI only talks to the daemon; dockerd makes the registry connection and never sees your shell environment. Configure the proxy on the daemon itself - a systemd drop-in, or the proxies block in /etc/docker/daemon.json - then restart it.

open as a page

How do you build a Docker Engine API /events client that misses no container deaths?

level: seniorimportance: should knowfreq 34%

basics

~20 s

Treat GET /events as a long-lived stream that will break: reconnect with backoff, pass since set to the last event's time so the daemon replays the gap, and reconcile against GET /containers/json on every reconnect because that replay buffer is bounded.

open as a page

After an edit to daemon.json, dockerd will not start. How do you diagnose and recover?

level: seniorimportance: should knowfreq 54%

basics

~20 s

Read the daemon's journal with journalctl -u docker: dockerd prints the exact configuration error before exiting - malformed JSON, an unrecognised key, or an option set both in daemon.json and as a unit flag. Move the file aside to recover.

open as a page

How do you size the CPU, memory and disk Docker Desktop's Linux VM is allowed to use?

level: seniorimportance: should knowfreq 41%

basics

~20 s

Every container's ceiling is the VM's, not the laptop's. On macOS the CPU, memory, swap and virtual-disk limits come from Docker Desktop's resource settings; with the Windows WSL 2 backend they come from the .wslconfig file, because the memory belongs to WSL.

open as a page

How do you use `docker events --filter` to catch containers that die overnight?

level: seniorimportance: should knowfreq 44%

basics

~20 s

Run docker events filtered to the actions you care about — --filter event=die --filter event=oom --filter event=health_status — and capture it to a file, because the stream is live and the daemon's replay buffer is small. Each die event carries the container's exitCode as an attribute.

open as a page

How do you choose between the docker-ce repository packages, the get.docker.com script and Docker Desktop when provisioning build hosts?

level: seniorimportance: should knowfreq 42%

basics

~20 s

Use Docker's docker-ce repository packages with an explicit pinned version for servers and build hosts; the get.docker.com convenience script is for quick throwaway boxes, not production; Docker Desktop is a licensed developer workstation product, not a server install.

open as a page

Which Docker log drivers can serve logs back to docker logs, and what happens with the others?

level: seniorimportance: should knowfreq 44%

basics

~20 s

Only json-file, local and journald keep messages where the daemon can replay them. Shipping drivers send them away, so since Docker 20.10 the daemon writes a bounded local cache to answer reads; the none driver is never readable.

open as a page

A process-isolated Windows container stops starting after the host is upgraded to a newer Windows build. How do you diagnose it?

level: seniorimportance: should knowfreq 45%

basics

~20 s

Process isolation requires the image's Windows build to match the host's, so an upgraded host rejects an image built on the old base with "The container operating system does not match the host operating system." Compare the image's OsVersion with the host build, then rebuild on the matching base tag or run it under Hyper-V isolation meanwhile.

open as a page

showing 1–30 of 39