skip to content

How does MLflow's `mlflow deployments` CLI send one logged model to different managed endpoints, and what does its `-t` target select?

level: middleimportance: nice to knowfreq 6%

answer

  1. one command shape, many destinations
  2. the target URI picks a plug-in
  3. an entry-point group finds plug-ins
  4. -C carries target-only settings
  5. help -t explains a target

basics

~20 s

mlflow deployments is a plug-in front end: -t names a target URI whose scheme selects an installed deployment plug-in, and create, update, predict and the other subcommands call that plug-in's client with the same model URI plus target-specific -C settings.

solid answer

~40 s

Every subcommand — `create`, `update`, `delete`, `list`, `get`, `predict`, `run-local`, `help` and the `*-endpoint` commands — resolves `-t`/`--target` to a deployment client. MLflow registers the `sagemaker` target in its own code; other destinations come from installed packages that register a plug-in under the `mlflow.deployments` entry-point group, and a target with no plug-in fails with a "No plugin found" error naming the target. `create` takes `--name`, `-m` and an optional `--flavor` it otherwise infers; target-specific settings go through repeatable `-C name=value` pairs that only the plug-in interprets. `update` points an existing deployment at a new model URI, and `predict` needs exactly one of `--name` or `--endpoint` plus an `-I` JSON file. What carries over between targets is the model URI and the command shape; instance types and scaling stay each target's own configuration.

go deeper

for a junior

Recall that -t names the target, that create and update take a model URI, and that help -t lists a target's options.

for a middle

Explain how the target URI resolves to a plug-in through the mlflow.deployments entry-point group, and how -C settings pass through to it untouched.

for a senior

Separate what is portable — model URI, command shape — from what is not: config keys, request format, scaling and traffic, and verify a deployment with predict before trusting it.

for a principal

Judge whether one plug-in-based deployment interface across targets reduces lock-in in practice, or just hides target-specific configuration that the team still has to own.

## A front end, not a server `mlflow models serve` runs MLflow's own scoring server. `mlflow deployments` does something different: it hands a model to **someone else's** serving system through a **deployment plug-in**. The CLI itself knows the command shape; the plug-in knows the destination. That split is what makes "the same model, a different endpoint" a one-flag change on the MLflow side. ## How `-t` finds a plug-in `-t` / `--target` is required on every subcommand and takes a **target URI**. MLflow parses the URI's scheme and looks the name up in a plug-in registry: - **`sagemaker`** is registered by MLflow's own code; its URI form is `sagemaker:/<aws-cli-profile-name>`, or plain `sagemaker`. - Other targets come from packages that declare an entry point in the **`mlflow.deployments`** group. Installing such a package is what makes its target name resolvable. The docs name `azureml-mlflow` for Azure ML and community plug-ins for other platforms. - A name with no plug-in raises "No plugin found for managing model deployments to ..." with a pointer to the plug-in list. A valid plug-in module must provide `target_help` and `run_local` and exactly one subclass of `BaseDeploymentClient`; the CLI calls that client's methods. ## The commands | Command | What it asks the plug-in to do | |---|---| | `create` | deploy `-m <model-uri>` as `--name`, with an optional `--flavor` (inferred if omitted) | | `update` | point the deployment at a new `-m`, or change its flavor or config | | `delete`, `list`, `get` | manage existing deployments | | `predict` | score an `-I` JSON file against `--name` or `--endpoint` (exactly one) | | `run-local` | run the deployment locally the way the target would | | `help` | print the target's URI format and its `-C` options | | `create-endpoint`, `update-endpoint`, `delete-endpoint`, `list-endpoints`, `get-endpoint` | manage endpoints on targets that have them | Target-specific settings travel as **`-C name=value`** (`--config`), repeatable; a repeated name is rejected. MLflow passes them through as a dictionary; only the plug-in knows what `instance-type` or `region_name` means. `mlflow deployments help -t <target>` is the authoritative list for a target. ## What is portable and what is not The portable part: 1. the **model URI** — the same `models:/property-value/3` goes to any target; 2. the **command shape** — `create`, `update`, `predict` read the same everywhere; 3. the **model format** — targets consume the pyfunc model MLflow logged. The non-portable part: - each target's `-C` keys, defaults and limits; - how the target serves the model (some build an image from MLflow's container, others their own runtime); - the request format the endpoint accepts, which the target decides, not MLflow; - traffic splitting, scaling and authentication, which live in the target's own configuration. So "switch clouds by changing `-t`" is true of the MLflow commands and untrue of the deployment: a migration still means re-learning the target's config keys and re-testing the endpoint. ## The Python form `mlflow.deployments.get_deploy_client(target_uri)` returns the same client the CLI uses, with `create_deployment`, `update_deployment`, `predict` and the rest. Without an argument it reads the target set by `set_deployments_target` or `MLFLOW_DEPLOYMENTS_TARGET`. A pipeline promoting the valuation model can call this client from code instead of shelling out. ## Where the docs and the 3.16 CLI disagree - The group's help text says `mlflow deployments help --target-name <name>`; the option the 3.16 `help` command actually defines is `-t` / `--target`. - A docs page runs `mlflow deployments run-local ... -p 5000`; in 3.16 the `run-local` command takes `--name`, `-t`, `-m`, `--flavor` and `-C`, so a port has to go through the target's own `-C` settings if the plug-in supports one. When a page and `--help` disagree, `--help` on the installed version wins. ## A deployment that is reviewable 1. Run `mlflow deployments help -t <target>` and pin the `-C` keys you use in version control. 2. `create` with an explicit `--name`, so a later `update` addresses the same deployment. 3. Smoke-test with `mlflow deployments predict -t <target> --name <name> -I listing.json`. 4. Roll forward with `update -m <new-uri>`; how traffic moves between versions is a separate design question.

  • `mlflow deployments create -t mytarget ...` fails with "No plugin found". What is missing?
    MLflow could not resolve `mytarget` in its plug-in registry. Only `sagemaker` is registered by MLflow's own code here; any other name must come from an installed package that declares an entry point in the `mlflow.deployments` group. Install the target's plug-in into the same environment as the CLI, then run `mlflow deployments help -t mytarget` to confirm it loads.
  • Why does `mlflow deployments predict` insist on exactly one of `--name` or `--endpoint`?
    A plug-in can address a deployment by name or, on targets with endpoints, an endpoint by name, and the two are different objects. The CLI refuses an ambiguous call when both or neither are given, reads `-I` as JSON with pandas, and passes the frame to the plug-in client's `predict`, printing the JSON result or writing it to `-O`.

  • mlflow deployments runs MLflow's scoring server on the target itself
  • Every cloud target works after installing MLflow alone
  • The -C settings are validated by MLflow against a fixed schema
  • Changing -t is all a migration between clouds needs
  • mlflow deployments predict accepts both --name and --endpoint together