skip to content

MLflow

MLflow logs runs to a tracking server, registers model versions under aliases, packages them as MLmodel flavors and serves them over REST. Interviewers treat it as the default open-source choice.

on this pageshow

questions

23

In MLflow, when do you record a value with log_param rather than log_metric, and what happens if a param is logged twice?

level: juniorimportance: must knowfreq 36%

answer

  1. inputs versus results
  2. written once per run
  3. step and history on metrics
  4. same key, different value refused
  5. tags stay overwritable

basics

~20 s

log_param records a run's configuration once per key; logging that key again with a different value raises "Changing param values is not allowed". log_metric records numeric results with a step and keeps every value, so anything that changes is a metric.

solid answer

~40 s

A param is an input to the run — `n_estimators`, `learning_rate`, the feature window — stored as a string and written once. Logging the same key again with the **same** value is accepted; a **different** value fails with an `MlflowException`, "Changing param values is not allowed". A metric is a numeric output: `mlflow.log_metric("val_rmse", v, step=i)` appends a point (step defaults to 0, timestamp to now), the whole series is kept, and the runs table and `mlflow.search_runs` show the latest point — highest step, then most recent timestamp. So a per-round loss, or a learning rate a scheduler changes, is a metric; a text label that changes, such as a status, is a tag set with `mlflow.set_tag`, which overwrites. `log_params` and `log_metrics` take dicts and batch the calls.

code

python · 16 lines
python
import mlflow

mlflow.set_experiment("store-demand")

with mlflow.start_run(run_name="store-demand-weekly"):
    mlflow.log_params({"max_depth": 6, "learning_rate": 0.05})
    mlflow.log_param("max_depth", 6)  # same value: accepted

    best = float("inf")
    for step, val_rmse in enumerate([12.4, 11.9, 11.7, 11.8]):
        mlflow.log_metric("val_rmse", val_rmse, step=step)  # history kept
        best = min(best, val_rmse)
    mlflow.log_metric("best_val_rmse", best)
    mlflow.set_tag("status", "evaluated")  # tags can be overwritten

    # mlflow.log_param("max_depth", 8)  # would raise: changing param values is not allowed

go deeper

for a junior

Recall the split: params are a run's inputs, written once; metrics are numeric outputs with a step and a kept history. Name the error a changed param raises.

for a middle

Explain what the store does on a repeated param key, how step and timestamp decide which metric point counts as latest, and why search treats params as strings.

for a senior

Show how the rules bite in real scripts: retries that recompute params, schedulers that change values mid-run, rankings that read the last point instead of the best, labels that belong in tags.

for a principal

Weigh which values a team standardises as params, metrics or tags so that runs from many projects stay searchable, and where the write-once rule saves a comparison from silent edits.

## Two kinds of record in an MLflow run An MLflow **run** is one execution of training code — one `python train.py`, or one trial of a sweep. While it is open, the code records small pieces of metadata in a **backend store** (a database) and files in an **artifact store**. Two of the metadata kinds look alike and behave very differently: - a **param** — a configuration value the run was started with; - a **metric** — a numeric value the run produced, possibly many times. Picking the wrong one is the most common first mistake with the tracking API, and MLflow enforces the difference rather than leaving it to convention. ## Params: configuration, written once `mlflow.log_param(key, value)` and its batch form `mlflow.log_params({...})` record inputs such as `max_depth`, `learning_rate` or the name of the feature set a store-demand forecaster was trained on. - The value is **string-ified** before it is stored, so `0.05` becomes `"0.05"`. Built-in backend stores accept param values up to 6,000 characters and keys up to 250. - A key is **written once per run**. Logging the same key again with an **identical** value is a no-op; logging it with a **different** value raises an `MlflowException`: "Changing param values is not allowed. Param with key='…' was already logged with value='…'". `log_params` checks the whole batch and lists every conflicting key. - Because params are strings, `mlflow.search_runs` filters on them with string comparators only — `=`, `!=`, `LIKE`, `ILIKE`, and `IS NULL` / `IS NOT NULL`. A range filter such as `params.learning_rate > 0.01` is not one of them. The write-once rule is what lets anyone trust that a run's param list describes the configuration it actually ran with. Why a run record needs that guarantee is a design question in its own right; the tool simply refuses the overwrite. ## Metrics: numbers with a step and a history `mlflow.log_metric(key, value, step=None, timestamp=None, ...)` and `mlflow.log_metrics({...}, step=...)` record numeric results. - `step` defaults to **0**; `timestamp` defaults to the current time in milliseconds. - Every call **appends** a point. Nothing is overwritten, so the whole curve of a boosting run's validation loss is kept. - `MlflowClient().get_metric_history(run_id, key)` returns every point. The runs table, `run.data.metrics` and `search_runs` show only the **latest** point per key: the one with the highest step, then the most recent timestamp. - Values are floats. A string label is not a metric. "Latest" is not "best": if validation error bottoms out at round 140 and rises to round 199, the run's `metrics.val_rmse` is the round-199 value unless you also log the score you intend to rank on. ## Choosing between param, metric and tag | What you are recording | Call | Can it change within a run? | |---|---|---| | A hyperparameter fixed for the run | `log_param` / `log_params` | No — a different value raises | | A loss or score per round or epoch | `log_metric(..., step=i)` | Yes — every point kept | | A value a scheduler changes during training | `log_metric(..., step=i)` | Yes | | A text label such as a status or owner | `set_tag` / `set_tags` | Yes — the last write wins | | A file: a plot, a CSV of predictions | `log_artifact` / `log_artifacts` | Files are added to the artifact store, not the database | Tags are the escape hatch for labels that legitimately change: the backend store merges a tag on write, so the last value wins. ## A typical training script 1. Open the run with `with mlflow.start_run(run_name="store-demand-weekly"):` so it ends cleanly even on an exception. 2. Log the configuration once with `mlflow.log_params(config)`. 3. Inside the training loop, log per-round values with `mlflow.log_metric("val_rmse", v, step=i)`. 4. After training, log the number you will rank runs on — for example `best_val_rmse` — as its own metric. 5. Attach files such as a residuals plot with `mlflow.log_artifact("residuals.png")`. ## Traps worth naming - Logging a per-epoch value as a param (`loss_epoch_3`) turns one curve into hundreds of keys and loses the step axis. - Recomputing a param on retry — a path containing a timestamp, a seed drawn at call time — makes the second attempt fail with the changing-param error. - Treating `metrics.val_rmse` in the runs table as the run's best value rather than its last one.

  • How do you get a metric's whole curve when the MLflow runs table shows only one number?
    Call `MlflowClient().get_metric_history(run_id, "val_rmse")`, which returns every logged point with its step, timestamp and value. The runs table, `run.data.metrics` and `search_runs` keep only the latest point per key — highest step, then most recent timestamp — so the full series comes from the history call or the run page's chart.
  • A retry wrapper re-runs the param-logging code inside the same MLflow run. Will it fail?
    Only if a value differs. The store compares each key with what is already recorded: an identical value is skipped, a different one raises "Changing param values is not allowed". A retry that recomputes a param nondeterministically — a timestamped output path, a seed drawn at call time — fails on the second attempt, so compute such values once before logging.
  • Why does a filter like params.learning_rate > 0.01 not work in mlflow.search_runs?
    Params are stored as strings, and the search grammar only allows string comparators on them: `=`, `!=`, `LIKE`, `ILIKE`, plus `IS NULL` and `IS NOT NULL`. Numeric comparators apply to metrics. If a team needs range queries on a configuration value, it records that value as a metric as well as a param.

A param is the recipe card written before the oven goes on — once baking starts you cannot amend it, only start another bake. A metric is the oven thermometer's log: a reading every minute, all of them kept, with the latest one shown on the display.

  • log_param overwrites the earlier value, like assigning to a dictionary key
  • log_metric keeps only the last value, so there is no curve to look at
  • Logging a param a second time with the same value raises an error
  • Params are stored as numbers, so params.learning_rate > 0.01 works in search_runs
  • A learning rate changed by a scheduler should be re-logged with log_param each epoch
open as a page

In MLflow, how does passing `registered_model_name=` to `log_model` differ from calling `mlflow.register_model` later, and how are versions numbered?

level: juniorimportance: must knowfreq 34%

basics

~20 s

registered_model_name in log_model logs and registers in one call; mlflow.register_model registers an already-logged model later. Both create the registered model if it is missing and add the next version number under that name, starting at 1.

open as a page

What request body does MLflow's scoring server accept at POST /invocations, and what does it send back?

level: juniorimportance: must knowfreq 28%

basics

~10 s

MLflow's scoring server takes a JSON object with exactly one of dataframe_split, dataframe_records, instances or inputs (plus optional params), or a text/csv or Parquet body, and replies with JSON shaped {"predictions": ...}.

open as a page

In MLflow, what does a logged model's MLmodel file declare, and how does it let one scikit-learn model load through both mlflow.sklearn and mlflow.pyfunc?

level: juniorimportance: must knowfreq 30%

basics

~20 s

MLmodel is the YAML manifest at the root of a logged MLflow model directory, listing its flavors, signature, input example and environment files. The sklearn flavor loads the native Pipeline; python_function names mlflow.sklearn as its loader_module, so pyfunc gets one generic predict.

open as a page

In MLflow, how do you log a nightly sweep's gradient-boosting trials as nested runs and then find the best trial from code?

level: middleimportance: must knowfreq 22%

basics

~10 s

Open a parent with mlflow.start_run, open each trial inside it with mlflow.start_run(nested=True), then call mlflow.search_runs filtered on tags.mlflow.parentRunId and ordered by the validation metric. Search ranks each run by that metric's latest logged value.

open as a page

In the MLflow Model Registry, how do aliases such as `@champion` promote and roll back a `room-rate-prod` model version?

level: middleimportance: must knowfreq 22%

basics

~20 s

MlflowClient.set_registered_model_alias(name, alias, version) points an alias such as champion at exactly one version; calling it again moves the alias, so promotion and rollback are the same call with a different version. A version may carry several aliases.

open as a page

What does MLflow's pyfunc predict check against a logged model signature, and what happens to malformed input when no signature was logged?

level: middleimportance: must knowfreq 18%

basics

~20 s

With a signature, pyfunc predict rejects missing required columns and unsafe type conversions, drops extra columns and applies params defaults. Without one, MLflow passes the input through untouched, so a bad request fails deep inside the model or scores wrongly.

open as a page

With mlflow.autolog() enabled for a scikit-learn regressor, what does MLflow record on its own, and what must you still log yourself?

level: middleimportance: should knowfreq 15%

basics

~20 s

On fit(), scikit-learn autologging records the estimator's get_params, scores computed on the training data, class-name tags and the fitted model with a signature. A held-out score your own code computes, and settings outside the estimator, you log yourself.

open as a page

Why do a scheduled job's MLflow runs land in a local mlflow.db instead of the team's shared tracking server, and how do you fix it?

level: middleimportance: should knowfreq 13%

basics

~20 s

The MLflow client uses mlflow.set_tracking_uri, else MLFLOW_TRACKING_URI, else a default — sqlite:///mlflow.db in the working directory since MLflow 3.7. A scheduler that doesn't pass the variable gets that default; set it in the job's environment or code.

open as a page

What does `mlflow models build-docker --model-uri` put into the image, and how does the container it builds serve requests?

level: middleimportance: should knowfreq 12%

basics

~20 s

mlflow models build-docker --model-uri copies the model into /opt/ml/model and installs its dependencies while the image builds; the container then runs nginx on port 8080 in front of uvicorn workers, one per CPU unless MLFLOW_MODELS_WORKERS is set.

open as a page

When MLflow's scoring server answers POST /invocations with 415, 400 or 504, what does each status tell you to fix?

level: middleimportance: should knowfreq 11%

basics

~20 s

From MLflow's scoring server, 415 means the Content-Type header was refused, 400 means the body, the signature check or the model's own predict call failed, and 504 means scoring outlasted the request timeout, 60 seconds by default.

open as a page

How do you use the `mlflow models predict` CLI to check that a request file scores the way MLflow's scoring server will score it?

level: middleimportance: should knowfreq 8%

basics

~20 s

Run mlflow models predict -m <uri> -i request.json, or -i listings.csv -t csv: it parses the file with the scoring server's own JSON or CSV code inside the model's environment and writes the {"predictions": ...} JSON the server would return.

open as a page

How does an mlflow.pyfunc.PythonModel package a scikit-learn pipeline with custom pre-filter code, and how does it find the pipeline file when loaded?

level: middleimportance: should knowfreq 14%

basics

~20 s

Subclass mlflow.pyfunc.PythonModel: load_context loads the pipeline once, predict applies the keyword rules and calls it. Log with artifacts={"pipeline": path}; MLflow copies the file into the model, and context.artifacts maps each name to its local path.

open as a page

Which environment files does MLflow write with a logged model, and how do you fix a missing dependency and prove it before deploying?

level: middleimportance: should knowfreq 11%

basics

~10 s

MLflow writes requirements.txt, python_env.yaml and conda.yaml, inferred from what the model imports. Add a missed package with extra_pip_requirements, or test one with mlflow.models.predict's pip_requirements_override, which scores in a fresh environment built from those files.

open as a page

When you run mlflow server for a team, how do you separate the backend store from the artifact store, and when should clients write artifacts through the server rather than straight to the bucket?

level: seniorimportance: should knowfreq 8%

basics

~20 s

Point --backend-store-uri at PostgreSQL or MySQL for metadata and --artifacts-destination at a bucket for files. Artifact serving is on by default, so clients upload through the server with its credentials; --no-serve-artifacts with --default-artifact-root makes clients write the bucket directly.

open as a page

A long-running pricing service loaded `models:/room-rate-prod@champion` from the MLflow registry; after on-call moves `champion` back a version, why does it still score with the newer one?

level: seniorimportance: should knowfreq 9%

basics

~20 s

An MLflow alias is resolved when the URI is loaded: load_model asks which version champion names, downloads it and keeps that object. Moving the alias changes only what the next load gets, so the service must reload or restart.

open as a page

On a fresh VM, why does `mlflow models serve` fail on a missing pyenv or take minutes to start, and what does `--env-manager` change?

level: seniorimportance: should knowfreq 6%

basics

~20 s

mlflow models serve defaults to --env-manager virtualenv: before starting, it builds an environment for the model's recorded Python version and requirements, using pyenv for the Python. uv builds it faster, conda uses conda, and local skips building entirely.

open as a page

In MLflow, what does `MlflowClient.copy_model_version` create when promoting `room-rate-staging@candidate` into a separate `room-rate-prod` registered model?

level: middleimportance: nice to knowfreq 5%

basics

~20 s

copy_model_version resolves the models:/ source to one version and adds a new version to the destination, creating that registered model if needed. The copy gets the destination's next number and keeps the source's tags, description and run, but no aliases.

open as a page

How does MLflow's `mlflow deployments` CLI send one logged model to different managed endpoints, and what does its `-t` target select?

level: middleimportance: nice to knowfreq 6%

basics

~20 s

mlflow deployments is a plug-in front end: -t names a target URI whose scheme selects an installed deployment plug-in, and create, update, predict and the other subcommands call that plug-in's client with the same model URI plus target-specific -C settings.

open as a page

What breaks when an MLflow 2 training script that logs a model with artifact_path= and loads it from the run's artifacts runs on MLflow 3?

level: seniorimportance: nice to knowfreq 6%

basics

~20 s

In MLflow 3 a logged model is a LoggedModel with its own model_id, stored outside the run's artifacts. artifact_path= still works but warns to use name=, and loading via mlflow.get_artifact_uri("model") fails; load from ModelInfo.model_uri or models:/<model_id>.

open as a page

Code written for MLflow 2.x calls `transition_model_version_stage` and loads `models:/room-rate/Production`; what replaces each when moving to MLflow 3.16?

level: seniorimportance: nice to knowfreq 6%

basics

~20 s

Registry stages have been deprecated since MLflow 2.9.0 and still are in 3.16. Replace the Production stage with an alias such as champion loaded by models:/name@champion, stage-as-status with model version tags, and stage-as-environment with separate registered models.

open as a page

After upgrading to MLflow 3.14 or later, mlflow.sklearn.log_model fails on a pipeline with a custom transformer class; why, and what are the options for logging it?

level: seniorimportance: nice to knowfreq 5%

basics

~20 s

MLflow 3.14 made skops the default serialization_format for mlflow.sklearn, and skops refuses to load types it was not told to trust. List the class in skops_trusted_types, move the custom step into a models-from-code wrapper, or opt back into cloudpickle.

open as a page