skip to content

Connectivity Troubleshooting

The ladder for "the container cannot reach it": prove whether it is name resolution or routing, read what the network really contains, get a shell into the right namespace, and check the host firewall. Asked in SRE loops because the answer is a method.

part ofDockerread it →
on this pageshow

questions

3

`docker run -p 9412:8080` fails with "port is already allocated" — how do you find what holds that host port?

level: juniorimportance: must knowfreq 64%

answer

  1. Two possible holders, two different messages
  2. Ask Docker's bookkeeping before the kernel's
  3. One command prints a container's published mappings
  4. Restarting containers still hold the port
  5. `sudo ss -ltnp` names the PID on :9412

basics

~20 s

Something already owns that host port. Ask Docker first: docker ps -a plus docker port <name> shows whether another container publishes 9412, including one stuck restarting. Then ask the host: sudo ss -ltnp | grep :9412 names the owning process.

solid answer

~50 s

Publishing a port makes the daemon claim a **host** port, so the failure means the port is taken — and the exact wording narrows the search. `Bind for 0.0.0.0:9412 failed: port is already allocated` means Docker's own allocator has already promised it to another container; `listen tcp4 0.0.0.0:9412: bind: address already in use` means the bind lost to a process Docker knows nothing about. So check Docker's side first with `docker ps -a --format '{{.Names}} {{.Status}} {{.Ports}}'`, confirming a suspect with `docker port <name>`; a container in `Restarting` is a live holder even though it is not in plain `docker ps`. Then check the host with `sudo ss -ltnp | grep :9412`, which prints the PID and command — often `docker-proxy` for another container, sometimes a service like nginx. Fix by removing the holder or publishing on a different host port; changing the container-side number changes nothing.

code

bash · 3 lines
bash
docker ps -a --format '{{.Names}} | {{.Status}} | {{.Ports}}'
docker port thumbs-worker
# 8080/tcp -> 0.0.0.0:9412

go deeper

for a junior

Recall that in -p 9412:8080 the first number is the host's and the clash is always there. Know two commands: docker port <name> for what a container publishes, and sudo ss -ltnp for what any host process is listening on.

for a middle

Explain why there are two distinct error messages — the daemon's port allocator versus a failed bind() syscall — and what each one tells you about where to look. Be able to say why a restarting container is a holder and a stopped one is not.

for a senior

Show a method rather than a guess: read the message, query Docker's state, then the kernel's, and only then act. Explain why restarting dockerd is the wrong first move on a host running other people's containers.

for a principal

Own the prevention: host ports are a shared, unmanaged namespace on a busy machine. Be ready to argue for allocated port ranges, publishing to a specific address rather than the wildcard, or removing host publishing entirely in favour of a shared network and a single ingress.

Publishing a port with `docker run -p 9412:8080` asks the Docker daemon to take ownership of a **host** port and forward traffic arriving there into the container. Two independent things must succeed. First, the daemon's own port allocator must find that host port free in its bookkeeping — it tracks every mapping it has handed out. Second, on a stock Linux engine the `docker-proxy` helper must actually bind that port with an ordinary `bind()` syscall, which the kernel refuses with `EADDRINUSE` if any other socket already owns the address. Each failure produces a different message, and reading which one you got is the fastest step of the whole diagnosis. **Message one — Docker's own bookkeeping.** Wording along the lines of `driver failed programming external connectivity ... Bind for 0.0.0.0:9412 failed: port is already allocated` means the daemon believes it has already promised that host port to another container. The holder is therefore something Docker knows about, and `docker` commands will find it. **Message two — the kernel's bookkeeping.** Wording along the lines of `Error starting userland proxy: listen tcp4 0.0.0.0:9412: bind: address already in use` means the bind lost to a socket Docker did not create — a service on the host such as a web server, a database, an SSH tunnel, or a leftover process. No `docker` command will show it; you have to ask the host. **The ladder.** 1. *Ask Docker first.* `docker ps -a --format '{{.Names}} {{.Status}} {{.Ports}}'` lists every container and its published mappings. Look for the port number in the Ports column, and look at Status: a container in `Restarting` is a live holder even though it is not "running" when you glance at `docker ps`. 2. *Confirm on the suspect.* `docker port <name>` prints just that container's published mappings, one line per mapping, in the form `8080/tcp -> 0.0.0.0:9412`. This is the authoritative answer for "does this container publish 9412?" — much less error-prone than eyeballing a truncated Ports column. 3. *Ask the kernel.* `sudo ss -ltnp | grep :9412` lists listening TCP sockets with the owning PID and command. Run it with root or the process column is blank. A line whose users field names `docker-proxy` is another container's published port; a line naming `nginx`, `postgres` or `java` is a plain host process. 4. *Decide.* Stop or remove the container that holds it, stop the host service, or publish on a different host port. Changing the **container-side** number (`-p 9412:8081`) does nothing — the clash is entirely on the left-hand side of the colon. **A worked example.** A photo-thumbnail pipeline runs a Java batch job on a distroless base image, published as `-p 9412:8080` so an operator dashboard can scrape it. After a config change the new container refuses to start with "port is already allocated". `docker ps` shows nothing on 9412, which looks like a ghost — but `docker ps -a` shows the previous `thumbs-worker` sitting in `Restarting (1) 4 seconds ago`. It exits on the bad config, its `--restart unless-stopped` policy brings it back, and the JVM's roughly 6-second cold start means it holds 9412 for most of every cycle. The new container simply keeps losing the race. `docker rm -f thumbs-worker` frees the port permanently; the fix for the loop is a separate problem. **Traps worth naming.** - A **fully stopped** container releases its host port; only running or restarting ones hold it. If `docker ps -a` shows `Exited` and the port is still refused, the holder is elsewhere. - `EXPOSE` in the image reserves nothing. It is metadata; only `-p`/`-P` at run time claims a host port. - Binding a specific host IP (`-p 127.0.0.1:9412:8080`) only dodges the clash if the other listener is bound to a *different specific* address. A listener on the wildcard `0.0.0.0:9412` still conflicts. - If the daemon is configured without the userland proxy there is no `docker-proxy` socket for `ss` to find, so an "already allocated" error can coexist with a quiet `ss` output; trust Docker's message and hunt with `docker` commands. - Reaching for a daemon restart as the first move is a classic weak answer: it is disruptive, it restarts every container that is not `--restart=no`, and it hides the cause. Keep it as a last resort for the rare case where Docker insists a port is allocated but neither `docker ps -a` nor `ss` can find any holder.

  • The container that published the port is not running — can it still be the cause?
    A fully stopped container releases its host port, so an `Exited` container is not the holder. A container in `Restarting` is, though: each restart re-claims the port, and if it crashes in a loop it owns the port for most of every cycle. `docker ps` hides it; `docker ps -a` and its Status column show it. Remove it with `docker rm -f <name>` and the port frees permanently.
  • Does publishing on 127.0.0.1 instead of the wildcard avoid the conflict?
    Only if the existing listener is bound to a *different specific* address. A socket on `0.0.0.0:9412` covers every address on the host, so `-p 127.0.0.1:9412:8080` still fails. If the other process is bound to one specific interface address, publishing on a different specific address does work. Binding to loopback is a security control — it limits who can reach the mapping — not a way around a port clash.
  • `ss` shows nothing on the port but Docker still says it is allocated. What now?
    Two cases. If the daemon runs without the userland proxy there is no `docker-proxy` socket to find, so a quiet `ss` is expected and Docker's own bookkeeping is authoritative — hunt with `docker ps -a` and `docker port`. Otherwise you may have an orphaned mapping from an unclean shutdown: remove the container that owns it, and only as a last resort restart the daemon, which rebuilds the table but also restarts every container with a restart policy.

Two different desks can hand out the same meeting room: Docker's own booking sheet, and the building's. The error message tells you which desk refused you, and that is the desk to go and read.

  • Says the container's internal port is in use, not the host's
  • Assumes only Docker can hold a host port
  • Fixes it by changing the right-hand container port number
  • Thinks EXPOSE in the image reserves a host port
  • Restarts the whole daemon before looking at ss
  • Checks only `docker ps` and misses a restarting container

context

open as a page

Inside a Docker container, a connection to another service fails. How do you prove whether name resolution or the network path is at fault?

level: middleimportance: must knowfreq 71%

basics

~20 s

Resolve the name first: getent hosts <name> inside the container. No answer means a resolution problem — read /etc/resolv.conf and check which network the container is on. An answer followed by a refusal or a hang means the path or the peer's listener, not DNS.

open as a page

What does `docker network inspect` tell you when a container cannot reach the rest of its network?

level: seniorimportance: should knowfreq 46%

basics

~20 s

It answers attachment and addressing: the Containers map lists the running containers on that network on this host with their IPv4 addresses, IPAM.Config gives the subnet and gateway, and Internal reveals a network with no outside route. A missing container is the finding.

open as a page