What does Docker's log-opt mode=non-blocking change, and what does max-buffer-size control?

## The failure mode A geospatial tile service — Django under gunicorn — shipped its logs with a remote driver. The log endpoint stalled for about forty-seven seconds during a maintenance window. In that window the tile service stopped answering: workers that would normally turn a request round in milliseconds were parked inside a `write()` to `stdout`, the health endpoint timed out, and instances were recycled with the usual six-second cold start each. Nothing was wrong with the application, the database, or the network path to the users. The log driver had applied back-pressure and the application had inherited it. This is the default behaviour, and it is a deliberate one: **blocking** delivery means no message is silently lost. ## How blocking delivery propagates The container's `stdout` is a pipe read by the daemon. The daemon takes a message off that pipe and hands it to the logging driver; only when the driver accepts it does the daemon come back for the next. If the driver is slow to accept, the daemon stops draining, the pipe's kernel buffer fills, and the next `write(2)` from the process inside the container blocks. From the application's point of view, printing a line has become a blocking network operation against a system nobody thought of as a dependency. The severity depends on the concurrency model. A worker-per-request server stalls one worker per blocked write and can wedge its whole pool; an event-loop server can stall everything on one thread. It also has nothing to do with which driver you chose — a shipping driver that cannot reach its endpoint is the common case, but a json-file driver on a filesystem that has gone read-only or is out of space blocks in exactly the same way. ## What non-blocking mode does `mode=non-blocking` puts an intermediate in-memory ring buffer between the daemon's read side and the driver. The message is appended to the buffer and the read side continues immediately; a separate consumer drains the buffer into the driver at whatever pace the driver manages. The application's write never waits on the driver. When the driver falls behind far enough that the buffer is full, the buffer **drops log messages**. That is the entire trade: you have chosen to lose logs rather than to lose the workload. It is usually the right choice for a request-serving service and usually the wrong one for an audit trail. `max-buffer-size` sets the buffer's capacity and defaults to `1m`. It is only meaningful with `mode=non-blocking`; in blocking mode nothing is buffered on that path. ``` docker run -d --log-driver=fluentd \ --log-opt mode=non-blocking --log-opt max-buffer-size=4m \ tileserv:2.4 ``` ## Sizing the buffer The buffer must cover *log volume × the stall you expect to ride out*. Take the service's log rate in bytes per second and multiply by the longest driver stall you are willing to survive without loss. A service emitting roughly 180 KB/s of access logs needs about 8.5 MB to cover a 47-second stall; the default `1m` covers under six seconds of it. The reflex to "just make it large" has a cost too — the buffer is daemon memory, held per container, and a large buffer on many containers is real RSS on the host, spent to make loss less likely during an outage in which the logs are usually least valuable. Also be honest about what the buffer is not. It is memory inside `dockerd`. It does not survive a daemon restart, it is not an ordered delivery guarantee, and it is not a substitute for buffering in the collector tier, which is where durability belongs. ## Choosing a default A reasonable posture for a fleet is: request-serving workloads get `mode=non-blocking` with a buffer sized from their measured log rate, so a logging problem can never become a user-facing one; anything whose output is compliance-relevant stays blocking, and its logging path is made reliable enough that blocking never triggers. Whichever you pick, pick it explicitly — the default is blocking, and a team that has never thought about this has implicitly decided that the log backend is a hard dependency of the application. One more reason to decide it deliberately: the option is per container and is frozen when the container is created, like every other log setting. You cannot switch a wedged fleet to non-blocking during the incident that taught you why you wanted it — the change lands only as containers are recreated. That makes this a default to set once, in the same place the fleet's driver and rotation bounds are set, rather than a knob to reach for under pressure.