skip to content

How do you use the `mlflow models predict` CLI to check that a request file scores the way MLflow's scoring server will score it?

level: middleimportance: should knowfreq 8%

answer

  1. the server's parser without HTTP
  2. input path and content type
  3. json unless told otherwise
  4. CSV needs its type named
  5. the output is always JSON

basics

~20 s

Run mlflow models predict -m <uri> -i request.json, or -i listings.csv -t csv: it parses the file with the scoring server's own JSON or CSV code inside the model's environment and writes the {"predictions": ...} JSON the server would return.

solid answer

~40 s

`mlflow models predict` takes the model URI (`-m`), `--input-path` (`-i`), `--content-type` (`-t`) and `--output-path` (`-o`). The content type defaults to `json` and can only be `json` or `csv`, with no Parquet. A JSON file must use the same wrapper keys as `/invocations`, because the command calls the scoring server's own parsing functions. It runs inside an environment built by `--env-manager`, `virtualenv` by default, so a dependency problem shows up before deployment; `--pip-requirements-override` (`-r`) tries a different pin without re-logging, and `--env key=value` passes environment variables. The result is JSON wrapped in `predictions`, written to stdout or to `-o`. It checks the payload, the environment and the model, not the HTTP layer: headers, the timeout and the container's proxy are not exercised.

code

bash · 3 lines
bash
mlflow models predict -m models:/property-value/3 \
  -i nightly-listings.csv -t csv -o nightly-predictions.json \
  --env-manager uv

go deeper

for a junior

Recall the four options that matter: -m for the model, -i for the input, -t for json or csv, -o for the output.

for a middle

Explain why the command mirrors the server's parsing, why CSV needs -t csv, and how --env-manager and -r change the environment it runs in.

for a senior

Show what a clean run does and does not prove, and wire it into CI as a payload regression check while knowing which HTTP-layer failures it cannot catch.

for a principal

Decide which scoring checks belong before deployment and which only a deployed endpoint can answer, and where a CLI check is enough for a batch job without any server.

## What the command is for `mlflow models predict` scores a file with a logged model from the command line. Its value as a pre-deployment check comes from one detail: for JSON and CSV it calls the **same parsing functions** as the scoring server's `/invocations` route, after loading the model through pyfunc in the same kind of environment. If a request file scores here, the same body will parse the same way on the server. The Python call `mlflow.models.predict` runs the same machinery; using it to prove a model's environment and signature at logging time belongs to model packaging. This answer is the CLI form, used as a scoring check on real payloads. ## The options | Option | Default | What it does | |---|---|---| | `-m`, `--model-uri` | required | the model to load | | `-i`, `--input-path` | stdin | the request file | | `-t`, `--content-type` | `json` | `json` or `csv`; nothing else is accepted | | `-o`, `--output-path` | stdout | where the JSON result is written | | `--env-manager` | `virtualenv` | `virtualenv`, `uv`, `conda` or `local` | | `-r`, `--pip-requirements-override` | none | comma-separated pins to try instead of the logged ones | | `--env` | none | extra `key=value` environment variables, repeatable | | `--install-mlflow` | off | install the CLI's MLflow version into the environment | Two restrictions: `--pip-requirements-override` is refused with `--env-manager local`, and `--env` works only with `virtualenv`, `conda` or `uv`. ## The traps in the content type - **JSON is the default.** A CSV file passed with `-i listings.csv` and no `-t csv` is parsed as JSON and fails. The docs' own batch example, `mlflow models predict -m models:/<model_id> -i input.csv -o output.csv`, omits `-t csv` and trips exactly this. - **The JSON must be a server body.** A file holding a bare list of rows fails just as it would on the server; it needs one of `dataframe_split`, `dataframe_records`, `instances` or `inputs`, with an optional `params` object. - **There is no Parquet mode**, although the server accepts `application/vnd.apache.parquet`. A Parquet batch has to be checked another way. - **The output is JSON**, `{"predictions": ...}`, even when `-o` ends in `.csv`. ## What it proves and what it does not What a clean run shows: - the request file has the shape the server expects; - the signature accepts the columns and types in it; - the model loads and predicts in an environment built from its recorded requirements; - a candidate pin fixes a failure (try it with `-r`, then update the model's requirements, which for a registered model means a new version). What it does not exercise, because there is no HTTP request: - the `Content-Type` header check and its 415 refusals; - the request timeout and its 504; - worker count, host binding or the container's nginx routes; - latency under concurrent requests. Environment variables follow a simple rule: the variables of the shell that runs the CLI are passed into the model's environment by default, and `--env key=value` adds or overrides one. That is how you hand a model the address of a lookup service it reads at prediction time without exporting it globally. With no `--input-path`, the request is read from standard input, so a payload can be piped in from another command. ## Using it in the valuation pipeline 1. Keep a small set of real request files — a typical listing, a listing with missing optional fields, a CSV extract of the nightly batch. 2. In CI, after logging a candidate, run `mlflow models predict -m <candidate-uri> -i form-request.json -o form-out.json` and `-i nightly.csv -t csv -o nightly-out.json`. 3. Compare the predictions against stored expected values within a tolerance. 4. Use `--env-manager uv` to keep the environment build short in CI; reserve `local` for runners that already installed the model's requirements. A failure here costs a CI run; the same failure found in production costs a broken form.

  • A request file scores cleanly with `mlflow models predict`, yet the same body gets 415 from the deployed server. How is that possible?
    The CLI reads the file directly and never sees an HTTP header, so it cannot reproduce the server's `Content-Type` check. A 415 means the client's header is wrong — an unsupported media type such as `application/csv`, a non-UTF-8 charset, or an extra parameter. Fix the client's header; the body was fine.
  • The logged model fails in `mlflow models predict` because of a library version. How do you test a fix without re-logging?
    Pass `--pip-requirements-override` (`-r`) with the candidate pins, for example `-r scikit-learn==<version>`, using a non-local environment manager. If the prediction now succeeds, update the model's requirements; for a registered model that means registering a new version, because registered versions are immutable.

  • mlflow models predict guesses the content type from the file extension
  • A clean CLI run proves the HTTP server will accept the request
  • Naming the output file .csv makes the command write CSV
  • The CLI can score Parquet files like the server
  • The CLI accepts any JSON list of rows as input