skip to content

MLOps & Model Lifecycle

Open-source tools that put a model's lifecycle under control: run tracking and a registry, data versioning, Kubernetes pipelines and drift reports. Interviewers ask which tool closes which gap.

on this pageshow

explore

questions

94 · 7 sections

In MLflow, when do you record a value with log_param rather than log_metric, and what happens if a param is logged twice?

level: juniorimportance: must knowfreq 36%
basics
~20 s

log_param records a run's configuration once per key; logging that key again with a different value raises "Changing param values is not allowed". log_metric records numeric results with a step and keeps every value, so anything that changes is a metric.

open as a page

In MLflow, how does passing `registered_model_name=` to `log_model` differ from calling `mlflow.register_model` later, and how are versions numbered?

level: juniorimportance: must knowfreq 34%
basics
~20 s

registered_model_name in log_model logs and registers in one call; mlflow.register_model registers an already-logged model later. Both create the registered model if it is missing and add the next version number under that name, starting at 1.

open as a page

What request body does MLflow's scoring server accept at POST /invocations, and what does it send back?

level: juniorimportance: must knowfreq 28%
basics
~10 s

MLflow's scoring server takes a JSON object with exactly one of dataframe_split, dataframe_records, instances or inputs (plus optional params), or a text/csv or Parquet body, and replies with JSON shaped {"predictions": ...}.

open as a page

In MLflow, what does a logged model's MLmodel file declare, and how does it let one scikit-learn model load through both mlflow.sklearn and mlflow.pyfunc?

level: juniorimportance: must knowfreq 30%
basics
~20 s

MLmodel is the YAML manifest at the root of a logged MLflow model directory, listing its flavors, signature, input example and environment files. The sklearn flavor loads the native Pipeline; python_function names mlflow.sklearn as its loader_module, so pyfunc gets one generic predict.

open as a page

In MLflow, how do you log a nightly sweep's gradient-boosting trials as nested runs and then find the best trial from code?

level: middleimportance: must knowfreq 22%
basics
~10 s

Open a parent with mlflow.start_run, open each trial inside it with mlflow.start_run(nested=True), then call mlflow.search_runs filtered on tags.mlflow.parentRunId and ordered by the validation metric. Search ranks each run by that metric's latest logged value.

open as a page

In DVC, after `dvc add data/camera-traps`, what do you commit to Git, and where do the 1.4 million photos actually live?

level: juniorimportance: must knowfreq 30%
basics
~20 s

You commit data/camera-traps.dvc and data/.gitignore. The pointer records the directory's MD5 hash, size, file count and path; the photos move into DVC's content-addressed cache under .dvc/cache and are linked or copied back into the workspace.

open as a page

In a DVC pipeline, what do `dvc.yaml` and `dvc.lock` each record, and why do both get committed to Git?

level: juniorimportance: must knowfreq 24%
basics
~20 s

dvc.yaml declares each stage's command, dependencies, outputs and tracked parameters; dvc.lock records what the last run actually used, with content hashes of dependencies and outputs and the parameter values. Committing both ties a commit to exact pipeline results.

open as a page

In DVC, why does a fresh `git clone` of a project lack its tracked data, and how do `dvc push` and `dvc pull` move it?

level: juniorimportance: must knowfreq 30%
basics
~20 s

Git carries only the small .dvc and dvc.lock pointer files; the bytes stay in each machine's DVC cache. dvc push uploads the cached objects a remote lacks; dvc pull downloads them and places them in the workspace.

open as a page

In DVC, after `git checkout` of last spring's tag, why are the workspace photos still this month's, and what restores the spring version?

level: middleimportance: must knowfreq 18%
basics
~20 s

git checkout only swaps the small .dvc pointer files; Git never tracked the photos. Running dvc checkout reads the restored pointers and rebuilds the workspace from the local cache, and dvc install adds a Git hook that runs it after every git checkout.

open as a page

In DVC, an S3 secret key set with `dvc remote modify` ended up in a Git commit — why, and where should remote credentials live?

level: middleimportance: must knowfreq 16%
basics
~20 s

Without a level flag, dvc remote modify writes .dvc/config, the project file that Git tracks. Secrets belong in the Git-ignored .dvc/config.local (written with --local) or outside DVC config entirely: environment variables, a credentials file or profile, SSH config.

open as a page

In the KFP v2 SDK, how do @dsl.component, @dsl.pipeline and the compiler turn Python functions into a run on a Kubeflow Pipelines backend?

level: juniorimportance: must knowfreq 22%
basics
~20 s

@dsl.component makes a typed function a container step, @dsl.pipeline wires steps into a graph through their outputs, Compiler().compile writes that graph as IR YAML, and kfp.Client submits it as a run in which each step gets its own container.

open as a page

In Kubeflow Trainer v2, what does a TrainJob's runtimeRef point at, and which part of the training job does that runtime own?

level: juniorimportance: must knowfreq 14%
basics
~20 s

A TrainJob's runtimeRef names a ClusterTrainingRuntime (the default kind) or a same-namespace TrainingRuntime. That platform-owned runtime holds the JobSet template and mlPolicy; the TrainJob adds only run values such as image, command, node count and resources.

open as a page

In a KFP v2 pipeline, when should a step pass data as an Input[Dataset] or Output[Model] artifact instead of a str parameter, and why?

level: middleimportance: must knowfreq 16%
basics
~20 s

Parameters (str, int, float, bool, list, dict) are small JSON values passed with a task's inputs. Datasets, models and metrics belong in artifacts: written to .path, then copied by the backend to object storage with lineage.

open as a page

In a KFP v2 pipeline, how do you run the deploy step only when the new model beats the current one, and what replaced dsl.Condition?

level: middleimportance: should knowfreq 8%
basics
~20 s

Have the evaluate step return both scores as parameter outputs and call deploy inside a with dsl.If(...) block that compares them; the backend evaluates the condition at run time. dsl.If, with dsl.Elif and dsl.Else, replaced the deprecated dsl.Condition.

open as a page

Why does a KFP v2 lightweight @dsl.component that relies on a module-level import and a same-file helper fail on the cluster with NameError?

level: middleimportance: should knowfreq 10%
basics
~20 s

A lightweight component ships only its own function source, run in base_image after KFP's own imports, so module-level imports and helpers never arrive. Import inside the body, list packages in packages_to_install, pass helpers through additional_funcs, or build a containerized component.

open as a page

With Feast, how do `get_historical_features` and `get_online_features` differ in what you pass, which store each reads, and which value each row receives?

level: juniorimportance: must knowfreq 18%
basics
~20 s

get_historical_features takes entity rows with timestamps, reads the offline store and returns each row's latest value at or before its timestamp within ttl; get_online_features takes keys only and returns the latest value loaded into the online store.

open as a page

Compared with self-hosting Feast, what do Databricks Feature Store, SageMaker Feature Store and Agent Platform Feature Store each run for you?

level: juniorimportance: must knowfreq 14%
basics
~20 s

Feast layers over storage you run: your online database, your materialization schedule, your feature server. Each managed service provisions its online store, keeps it in sync with its offline store, and serves reads through its own API.

open as a page

With Feast, where does `feast materialize-incremental` take its start time from, and why can it skip warehouse rows that landed late?

level: middleimportance: should knowfreq 9%
basics
~20 s

feast materialize-incremental END_TS starts each feature view at the latest end recorded in the registry, or at now minus ttl on a first run. Rows stamped before that start are never loaded, however late they land.

open as a page

Why does a Feast training set from `get_historical_features` show nulls for a nightly `spend_7d` feature, and how does the FeatureView's `ttl` cause it?

level: middleimportance: should knowfreq 12%
basics
~20 s

A FeatureView's ttl caps how far back from each entity row's timestamp the historical join looks; a nightly value older than ttl at that instant is dropped, leaving a null. Set ttl above the refresh interval plus load delay.

open as a page

In Databricks Feature Store, why can a model logged with FeatureEngineeringClient.log_model be scored with only primary keys, and what breaks that?

level: middleimportance: should knowfreq 8%
basics
~20 s

fe.log_model stores the training set's FeatureLookup specifications with the model, so score_batch and Model Serving fetch the features by primary key themselves. Anything you did to load_df's DataFrame before training is not replayed at inference.

open as a page

With TensorFlow Data Validation (TFDV), what do `generate_statistics_from_tfrecord`, `infer_schema` and `validate_statistics` each produce when you check a training batch?

level: juniorimportance: must knowfreq 12%
basics
~10 s

generate_statistics_from_tfrecord returns a DatasetFeatureStatisticsList summarising each feature; infer_schema turns those statistics into a draft Schema you review; validate_statistics checks a later batch's statistics against it and returns an Anomalies proto, empty when clean.

open as a page

In Evidently 0.7, how do you build a drift report with pass/fail tests for a weekly churn batch, and what does running it return?

level: juniorimportance: must knowfreq 14%
basics
~10 s

Wrap DataDriftPreset() in Report([...], include_tests=True) and call report.run(current, reference). The call returns a Snapshot holding per-column drift results plus pass/fail tests, which you save as HTML or JSON or upload to an Evidently workspace.

open as a page

Should a weekly batch churn model use Evidently or NannyML in its scoring job, Arize AX or Fiddler, or a cloud monitor, and what decides it?

level: seniorimportance: must knowfreq 10%
basics
~20 s

The choice turns on data residency, who operates storage and alerting, and label latency: libraries run in the job, managed platforms hold history and joins. SageMaker Model Monitor is closed to new customers; Google Model Monitoring v2 is in Preview.

open as a page

In Deequ, how do a `VerificationSuite`'s Error-level and Warning-level checks decide the run's status, and does `run()` fail the Spark job?

level: middleimportance: should knowfreq 10%
basics
~20 s

A failing Check reports Error or Warning according to its CheckLevel, and the VerificationResult status is the worst status across checks. run() only returns that result, so the job must test the status and fail itself.

open as a page

In TFDV, when is a feature's `skew_comparator` evaluated versus its `drift_comparator`, and why can an `infinity_norm` threshold miss a numeric feature?

level: middleimportance: should knowfreq 8%
basics
~20 s

The skew_comparator runs only when validate_statistics receives serving_statistics (training against serving); the drift_comparator only when it receives previous_statistics (current against an earlier batch). An infinity_norm threshold reads categorical value counts, so numeric features need jensen_shannon_divergence.

open as a page

In the shap library, which explainer fits a LightGBM regressor, a PyTorch network and an opaque predict function, and why?

level: juniorimportance: must knowfreq 38%
basics
~20 s

Use TreeExplainer for the LightGBM model, DeepExplainer or GradientExplainer for the PyTorch network, and PermutationExplainer or KernelExplainer for the opaque function: the more of the model an explainer can read, the faster and more exact it is.

open as a page

How does `shap.plots.waterfall` build one patient's readmission explanation from the Explanation's `.base_values`, `.values` and `.data`?

level: juniorimportance: must knowfreq 38%
basics
~20 s

A shap waterfall takes one row of an Explanation: it starts at base_values, the average model output E[f(X)], adds each feature's SHAP value from .values as a bar labelled with its .data value, and ends at that patient's output f(x).

open as a page

A shap waterfall for a binary XGBoost readmission model shows f(x) = -0.85 while `predict_proba` gives 0.30 — why, and how do you reconcile them?

level: middleimportance: must knowfreq 18%
basics
~20 s

By default TreeExplainer explains a binary XGBoost model on its raw log-odds margin, so base value plus SHAP values equals -0.85 in log-odds. The logistic function maps it to 0.30, the predicted probability; individual bars have no fixed probability equivalent.

open as a page

When shap.Explainer runs with algorithm="auto", how does it decide which explainer to build for a given model and masker?

level: middleimportance: should knowfreq 16%
basics
~20 s

shap.Explainer tries the linear, tree and additive explainers in turn; failing those, a callable model gets exact or permutation by masker type and feature count, or partition for text and image maskers. Kernel, Deep and Gradient are never chosen.

open as a page

When shap.Explainer is handed a 20,000-row DataFrame as its masker, how many rows does it use, and how should you size that background?

level: middleimportance: should knowfreq 14%
basics
~10 s

It uses 100 rows: the DataFrame is wrapped in shap.maskers.Independent, whose max_samples=100 sub-samples it with shap.utils.sample. Size the background on purpose, with max_samples or shap.sample, because model-agnostic cost grows linearly with it.

open as a page

In lime, what does LimeTabularExplainer.explain_instance return for one row of a classifier, and how do you read its as_list output?

level: juniorimportance: must knowfreq 38%
basics
~20 s

It returns an Explanation object. as_list(label=1) gives (feature condition, weight) pairs sorted by absolute weight: coefficients of a local linear surrogate fitted to the classifier's probability for that label, so positive weights push that class up.

open as a page

When choosing between the lime and shap packages to explain individual predictions of a production model, which conditions decide the choice?

level: seniorimportance: must knowfreq 28%
basics
~20 s

The conditions decide, not the brand: a tree ensemble shap can read gets exact, additive attributions without sampling; a black box means sampling in either package; lime gives readable bin conditions without additivity, and its last release was in 2020.

open as a page

In lime's LimeTabularExplainer, what do categorical_features and discretize_continuous change about an explanation, and how must categorical columns be encoded?

level: middleimportance: should knowfreq 11%
basics
~20 s

categorical_features lists integer-coded columns that lime samples from their training frequencies and explains as column=value. discretize_continuous=True, the default, bins continuous columns at training quartiles, so a weight describes being in the row's bin, not a per-unit slope.

open as a page

In lime 0.2.0.1's tabular explainer, which settings go on the LimeTabularExplainer constructor and which on explain_instance, and why does the split matter?

level: middleimportance: should knowfreq 14%
basics
~20 s

The constructor fixes everything derived from training data: kernel_width, discretisation, categorical columns, feature_selection and random_state. explain_instance takes per-row choices: num_features, num_samples, labels or top_labels, distance_metric and model_regressor. Changing a constructor setting means building a new explainer.

open as a page

A lime 0.2.0.1 tabular explanation shows exp.score near 0.3 and exp.local_pred far from the model's probability; what do they measure and what do you change?

level: seniorimportance: should knowfreq 6%
basics
~20 s

exp.score is the weighted R squared of lime's ridge surrogate on its perturbed samples, and exp.local_pred is that surrogate's prediction for the row. A low score or a distant local_pred means the weights describe the model poorly around this row.

open as a page