skip to content

When MLflow's scoring server answers POST /invocations with 415, 400 or 504, what does each status tell you to fix?

level: middleimportance: should knowfreq 11%

answer

  1. each status names a layer
  2. 415 means the header
  3. 400 covers body, schema and model
  4. 504 means the clock ran out
  5. sixty seconds unless changed

basics

~20 s

From MLflow's scoring server, 415 means the Content-Type header was refused, 400 means the body, the signature check or the model's own predict call failed, and 504 means scoring outlasted the request timeout, 60 seconds by default.

solid answer

~40 s

415 comes from the header check: a media type other than `application/json`, `text/csv` or `application/vnd.apache.parquet`, a charset other than UTF-8, or an extra parameter such as a 1.x client's `format=pandas-split`. 400 covers the failures after the header check: a JSON list instead of an object, zero or two wrapper keys, a malformed `dataframe_split`, a CSV pandas cannot read, a schema mismatch, and, easy to miss, an exception raised inside the model's `predict`, which the server reports as 400 with a stack trace rather than as 500. 504 comes from the server's timeout middleware, set by `--timeout` or `MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT` and 60 seconds by default. The JSON body's `error_code` and `message` name the specific cause.

go deeper

for a junior

Recall the mapping: 415 for the header, 400 for the body, schema or model, 504 for the timeout, and that the timeout defaults to 60 seconds.

for a middle

Explain the order of the checks and which header parameters, JSON shapes and schema failures land in each status, including the SCHEMA_ENFORCEMENT_FAILED error class.

for a senior

Show the operational traps: model crashes reported as 400, input data echoed into error messages, timed-out work still running, and params ignored behind a 200.

for a principal

Decide how a platform should classify scoring errors for alerting and client contracts when the server folds caller faults and model faults into one status code.

## Three statuses, three layers MLflow's scoring server — the FastAPI app that `mlflow models serve` and the `build-docker` image both run — checks a request in a fixed order: the header, then the body and the model's signature, then the model call, all under a timeout. Each layer has its own status, so the status alone tells you where to look. | Status | Raised by | Typical cause | |---|---|---| | **415** | the header check | wrong media type, a non-UTF-8 charset, an unknown header parameter | | **400** | parsing, schema enforcement or the model call | wrong JSON shape, a missing column, an exception inside `predict` | | **504** | the timeout middleware | scoring took longer than the request timeout | ## 415 — the header was refused The server splits `Content-Type` into a media type and parameters, then refuses the request with **415** when: 1. the media type is not `application/json`, `text/csv` or `application/vnd.apache.parquet` (`application/csv` and `text/plain` both fail); 2. the `charset` parameter is anything other than UTF-8; 3. any parameter other than `charset` is present. The third case catches clients written for MLflow 1.x, which sent `application/json; format=pandas-split`. The 415 text says the scoring protocol changed in MLflow 2.0. The fix is in the client: send a plain media type and move the format into the body's wrapper key. ## 400 — the body, the schema or the model The failures after the header check are reported as **400**, with a JSON body holding `error_code` (such as `BAD_REQUEST` or `INVALID_PARAMETER_VALUE`) and `message`: - **JSON shape** — a top-level list, zero or two of `dataframe_split` / `dataframe_records` / `instances` / `inputs`, or a `dataframe_split` without `data`. - **Unreadable CSV or Parquet** — the message says pandas could not parse the body. - **Schema enforcement** — a missing column or a value that cannot be converted to the signature's type. The body carries `error_class` `SCHEMA_ENFORCEMENT_FAILED`. - **An exception inside `predict`** — a bug in custom model code, or a library missing from the serving environment, comes back as a 400 whose message says an unexpected error occurred while evaluating the model, with the stack trace in the response body. That last case surprises people: a model crash is not a 500. Before blaming the caller for a 400, read the `message` and the stack trace. Error messages also **echo the input**. A schema failure includes the data and the schema; `MLFLOW_DISABLE_SCHEMA_DETAILS=true` replaces that inner detail with a generic sentence, but the server still prefixes a failed prediction's message with the data it tried to score. For a valuation form that may carry personal fields, keep error bodies out of client-facing logs. ## 504 — the request timeout A timeout middleware wraps every request. It is set by `--timeout` (`-t`) on `mlflow models serve` or the `MLFLOW_SCORING_SERVER_REQUEST_TIMEOUT` environment variable and defaults to **60 seconds**. When it fires, the client gets 504 with "Request processing time exceeded limit". The model call runs in a worker thread, and Python cannot cancel a thread, so the 504 releases the client while the scoring work itself keeps running. Typical fixes: - split an oversized batch into smaller requests; - raise the timeout deliberately for a batch caller; - move the nightly run off HTTP to a batch command such as `mlflow models predict`. ## Statuses that do not come from MLflow In the `build-docker` image, nginx sits in front of the scoring server, and some statuses never reach MLflow's code: - a path other than `/ping` or `/invocations` gets nginx's **404**, `/health` included; - a body over the image's 100 MB limit is refused by nginx before the model sees it. Knowing which process answered saves a search through the wrong logs: an MLflow error carries a JSON body with `error_code`, a proxy error does not. ## A triage order 1. **415** — fix the client's header first; nothing else has been looked at yet. 2. **400 with a parse message** — fix the JSON shape or the CSV. 3. **400 with `SCHEMA_ENFORCEMENT_FAILED`** — compare the columns and types sent against the model's signature. 4. **400 with a stack trace** — debug the model or its environment, not the request. 5. **504** — measure how long scoring takes for that batch size before changing the timeout. A 200 is not proof of correctness either: `params` sent to a model without a params schema are ignored with only a log warning.

  • A custom pyfunc model raises `KeyError` inside `predict`. What status does the MLflow scoring server return, and why does it matter for alerting?
    It returns 400, not 500: the server wraps any non-MLflow exception from `predict` in an MLflow error with a bad-request code and puts the stack trace in the body. An alert that counts only 5xx as server faults will miss model crashes, so dashboards should split 400s by message or `error_class` rather than treating every 400 as a caller mistake.
  • After a 504 from the MLflow scoring server, is the scoring for that request still running?
    Usually yes. The timeout middleware stops waiting and answers 504, but the model call runs in a worker thread that Python cannot cancel, so it runs to completion and its result is discarded. Repeated retries of an oversized batch can therefore pile up work on a server that already looks timed out.

  • A 400 always means the caller sent a malformed request
  • An exception inside predict comes back as a 500
  • The scoring server accepts the 1.x format=pandas-split header parameter
  • The 504 timeout stops the model computation for that request
  • The scoring server has no request timeout unless one is configured
  • A 200 proves every field of the request was used