What request body does MLflow's scoring server accept at POST /invocations, and what does it send back?
answer
- the header picks the parser
- a JSON object, never a bare list
- exactly one wrapper key
- split orientation keeps column order
- the reply is wrapped too
basics
~10 sMLflow's scoring server takes a JSON object with exactly one of dataframe_split, dataframe_records, instances or inputs (plus optional params), or a text/csv or Parquet body, and replies with JSON shaped {"predictions": ...}.
solid answer
~40 s`POST /invocations` picks its parser from the `Content-Type` header: `application/json`, `text/csv` or `application/vnd.apache.parquet`. A JSON body must be an object holding exactly one of `dataframe_split` (pandas split orientation: `data` plus optional `columns` and `index`), `dataframe_records` (a list of row objects), `instances` or `inputs` (tensor-style input), and it may add a `params` object for a model whose signature declares params. CSV and Parquet bodies become a pandas DataFrame directly and carry no params. With a signature, the server casts the parsed input to the declared types before `predict` runs. The reply is JSON with the output under `predictions`. In MLflow 3.16 this FastAPI app is the only pyfunc serving backend; MLServer support was removed in 3.13.
code
json · 9 lines{
"dataframe_split": {
"columns": ["living_area_m2", "bedrooms", "district", "year_built"],
"data": [
[84.0, 2, "D04", 1998],
[132.5, 4, "D11", 2015]
]
}
}go deeper
Recall the four JSON wrapper keys, that exactly one is allowed, the three content types, and that the answer sits under predictions.
Explain how the Content-Type selects the parser, why dataframe_split preserves column order, and how a signature casts and checks the parsed input before predict runs.
Show where a client goes wrong silently: params ignored on a model without a params schema, a 1.x body rejected after an upgrade, and reading the reply without checking the status code.
Weigh JSON against CSV or Parquet for large batch scoring through the HTTP server: payload size and parse cost against losing params, and when batch scoring should not go over HTTP at all.
## What answers at `/invocations` `mlflow models serve -m <model-uri>` loads a model through its **`python_function` (pyfunc)** flavor and starts a small **FastAPI** application under **uvicorn**. The app defines four routes: - `POST /invocations` — scoring. - `GET /ping` and `GET /health` — the same handler, 200 once the model is loaded. - `GET /version` — the MLflow version of the serving process. Since **MLflow 3.13** this built-in app is the only pyfunc serving backend: the deprecated `enable_mlserver` option was removed, so a 2.x setup that served through MLServer gets the built-in server. For a property-valuation estimator, the internal web form sends one listing per request and the nightly batch sends thousands, but both go through the same route and the same rules. ## The Content-Type picks the parser | `Content-Type` | Body | Parsed into | |---|---|---| | `application/json` | a JSON object with one wrapper key | a DataFrame, an array or a dict, depending on the key | | `text/csv` | pandas-style CSV with a header row | a pandas DataFrame | | `application/vnd.apache.parquet` | Parquet bytes | a pandas DataFrame | The only parameter the header may carry is `charset`, and only UTF-8 is accepted, so `application/json; charset=utf-8` is fine while `charset=latin-1` is not. Any other media type is refused with **HTTP 415**. One docs page still shows `application/csv`; the 3.16 source recognises only `text/csv`. The choice is not cosmetic: JSON keeps per-request `params` and nested values, while CSV and Parquet are plain tables whose column types come from the signature or from pandas' own inference. ## The four JSON wrapper keys A JSON body must be an **object** (a top-level list is rejected) and must hold **exactly one** of these keys: - **`dataframe_split`** — pandas `split` orientation: a required `data` list of rows, an optional `columns` list and an optional `index`. The docs recommend it over records because it keeps column order. - **`dataframe_records`** — a list of `{column: value}` objects; the docs warn it is not guaranteed to preserve column ordering. - **`instances`** — tensor-style input in the TF Serving request convention, cast to NumPy arrays. - **`inputs`** — the same convention under another key. A model whose signature was built from Python type hints must receive `inputs`. The two tensor keys are not quite interchangeable for a tabular model: the server converts an `instances` list of row objects into a column dictionary, and when schema enforcement then fails it logs a hint to send `inputs` instead to avoid that conversion. No wrapper key, or two of them, is a **400** whose message lists the allowed fields. That includes the 1.x shape `{"columns": [...], "data": [...]}` at the top level; the 2.0 protocol change moved those fields inside `dataframe_split`. A separate path accepts unwrapped chat, completion and embedding bodies for LLM models; that GenAI surface is outside this answer. ## `params` and the signature Next to the wrapper key, a JSON body may carry **`params`**, an object of inference-time settings passed to `predict(..., params=...)`. They take effect only if the model's signature declares a params schema; for a model logged without one, the server logs a warning and **ignores** them. CSV and Parquet bodies have nowhere to put params. If the model was logged with a **signature**, the server parses the payload against its input schema: CSV columns are read with the declared pandas dtypes, base64 strings become bytes for a binary column, and ISO-8601 strings become datetimes. pyfunc then enforces the schema before the model runs. What the signature records belongs to model packaging; what matters at the server is that a missing required column becomes a 400 instead of a wrong price. ## What comes back A successful call with one of these wrapped or tabular bodies returns **200** with `{"predictions": ...}`: - a NumPy array becomes nested JSON lists; - a pandas DataFrame or Series becomes a list of row objects (records orientation); - anything else is serialised as it is. Errors come back as JSON with `error_code` and `message` fields, so a client should check the status code before reading `predictions`. ## Writing the request by hand 1. Send `Content-Type: application/json`. 2. Wrap the rows in `dataframe_split`, with `columns` named as the signature names them. 3. Put inference settings in `params`, never as extra top-level keys. 4. Read the result from `predictions`, not from the top level of the reply. For the nightly batch, a CSV or Parquet body avoids building a large JSON document, at the cost of losing `params`.
- Why do the docs steer you to `dataframe_split` rather than `dataframe_records` for tabular models?`dataframe_records` is a list of row objects, and the docs warn it is not guaranteed to preserve column ordering. `dataframe_split` sends an explicit `columns` list alongside `data`, so the DataFrame the server builds has the columns in the order you sent. It is also more compact, because column names are not repeated on every row.
- A client sends `{"dataframe_split": {...}, "params": {"currency": "EUR"}}` to a model logged without a params schema. What happens to `currency`?The rows are scored and `currency` has no effect. When a model's signature declares no params schema, MLflow logs a warning that the provided params are being ignored and calls the model with empty params. Nothing in the response says so, which is why the model needs a params schema in its signature before a client relies on params.
- A JSON list of row objects at the top level is a valid body
- Sending both dataframe_split and inputs lets the server pick one
- CSV bodies must be sent as application/csv
- Predictions come back as a bare JSON array with no wrapper
- MLflow 3 can still serve through MLServer with enable_mlserver
- Params on a model without a params schema raise an error
Assumes MLflow 3.16; since 3.13 the built-in FastAPI scoring server is the only pyfunc serving backend because enable_mlserver was removed.