skip to content

On a stock KServe 0.21 install, which deployment mode does an InferenceService without a deploymentMode annotation get, and what does each mode bring?

level: middleimportance: should knowfreq 10%

answer

  1. one annotation per service
  2. cluster default lives in a ConfigMap
  3. shipped value uses the legacy name
  4. revisions and scale-to-zero vs Deployment
  5. fixed once the service exists

basics

~20 s

Knative mode. The shipped inferenceservice-config ConfigMap sets defaultDeploymentMode to Serverless, which KServe 0.21 reads as Knative: revisions, scale-to-zero and canaryTrafficPercent. Standard mode, chosen per service by annotation, runs a plain Deployment with HPA or KEDA.

solid answer

~30 s

The mode comes from the `serving.kserve.io/deploymentMode` annotation, or, when it is absent, from `defaultDeploymentMode` in the `deploy` entry of the `inferenceservice-config` ConfigMap. KServe 0.21 ships that as `"Serverless"`, the legacy name it normalizes to `Knative`, so a stock install runs Knative mode, although the code's own empty-value default is `Standard`. Knative mode gives revisions, KPA autoscaling, scale-to-zero and `canaryTrafficPercent`. `Standard` gives a Deployment, Service and HPA (or KEDA), no scale-to-zero, `deploymentStrategy`, and since 0.20 its own `spec.canary` list. `RawDeployment` and `Serverless` still parse but are deprecated names, and the webhook refuses changing the mode of an existing InferenceService.

code

yaml · 12 lines
yaml
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: playlist-blurbs
  annotations:
    serving.kserve.io/deploymentMode: Standard
spec:
  predictor:
    model:
      modelFormat:
        name: huggingface
      storageUri: hf://example-org/blurb-7b-instruct

go deeper

for a junior

Recall the annotation name, the two 0.21 mode names, and that the stock default is Knative.

for a middle

Explain where the default lives (the inferenceservice-config ConfigMap), how legacy names are normalized, and what objects each mode creates.

for a senior

Pick the mode per workload knowing it is fixed at creation, and verify the cluster default instead of trusting a constant or an old doc.

for a principal

Decide whether the platform needs Knative at all: it buys scale-to-zero and revisions at the cost of another control plane and cold starts.

## Two deployment modes, one annotation In KServe 0.21 an InferenceService runs in one of two **deployment modes**: - **`Knative`**: the controller creates a Knative Service. Every change to the spec makes a new immutable **revision**, Knative's autoscaler (KPA) scales on request concurrency or rps, and pods can **scale to zero**. - **`Standard`**: the controller creates a plain Kubernetes Deployment and Service, plus an HPA (or a KEDA ScaledObject) for scaling. No Knative needed, no revisions, no scale-to-zero. The mode is chosen per InferenceService by the annotation `serving.kserve.io/deploymentMode`. When the annotation is absent, the cluster default comes from the `deploy` entry of the **`inferenceservice-config`** ConfigMap in the `kserve` namespace, key `defaultDeploymentMode`. ## What a stock 0.21 install actually does The source sends a mixed signal, and that is where confident answers go wrong: 1. `pkg/constants/constants.go` defines `DefaultDeployment = Standard`, used when an empty mode string is parsed. 2. The shipped ConfigMap (`config/configmap/inferenceservice.yaml`) sets `"defaultDeploymentMode": "Serverless"`. 3. `Serverless` is the legacy name for `Knative`; the config loader normalizes it, so the cluster default is **Knative**. So on a stock install an InferenceService with no annotation runs in **Knative mode**, and needs Knative Serving to be installed. A cluster without Knative must either change the ConfigMap's `defaultDeploymentMode` to `Standard` or annotate each InferenceService. ## The names changed | Older name | 0.21 name | What 0.21 does with the old one | |---|---|---| | `Serverless` | `Knative` | constants mark it deprecated; the webhook rewrites the annotation to `Knative` | | `RawDeployment` | `Standard` | constants mark it deprecated; the webhook rewrites the annotation to `Standard` | Old manifests keep working. Reach for the new names, and expect status, docs and validation messages to use them. ## What each mode brings or withholds | Capability | Knative | Standard | |---|---|---| | Underlying objects | Knative Service and revisions | Deployment, Service, HPA or KEDA | | Scale to zero | yes, with `minReplicas: 0` | not supported | | Default autoscaling | KPA on `concurrency` | HPA on `cpu` (`serving.kserve.io/autoscalerClass: hpa`) | | Canary | `canaryTrafficPercent` between revisions | `spec.canary` list (since 0.20) | | Rollout tuning | revisions | `deploymentStrategy` (rejected in Knative mode) | | Extra per-pod container | Knative queue proxy | none | Knative suits bursty traffic and services that may sit idle. Standard avoids cold starts and the Knative dependency; KServe's docs recommend it for LLM workloads. ## The mode is fixed at creation The validating webhook compares the annotation with the mode recorded in the InferenceService status and refuses a change: `update rejected: deploymentMode cannot be changed from 'Knative' to 'Standard'`. Switching modes means creating a new InferenceService (or deleting and re-creating) and moving clients over, so it is a decision to get right up front. ## Choosing per workload on one cluster On a cluster with Knative installed, the two modes can run side by side, because the annotation is per InferenceService: - A small tabular model with a steady daytime pattern and quiet nights, such as a track-skip classifier, is a natural Knative service. It can scale to zero overnight, and new boosters roll out through revisions and `canaryTrafficPercent`. - A GPU-hungry generative model, such as a 7B blurb writer, is often put in `Standard` mode. Its cold start is long anyway, it gains nothing from the queue proxy, and an HPA or KEDA on a custom metric fits it better. - A team that needs a weighted canary in `Standard` mode uses the `spec.canary` list (0.20 and later). It has to annotate the service with `Standard` explicitly, because canary validation reads the annotation rather than the cluster default. Write the annotation on every production InferenceService even when it matches the cluster default. The default is a ConfigMap value that an administrator can change, and the explicit annotation keeps each service's behaviour independent of that edit. ## Checking a cluster - Read the ConfigMap: `kubectl get configmap inferenceservice-config -n kserve -o jsonpath='{.data.deploy}'`. - Read the InferenceService: the annotation, if set, and `status.deploymentMode`. - Look at what was created: a Knative Service and revisions, or a Deployment and HPA.

  • A team edits a running KServe InferenceService's deploymentMode annotation from Knative to Standard. What happens?
    The validating webhook rejects the update with `deploymentMode cannot be changed`, because the mode recorded in the InferenceService status differs from the new annotation. To move a service between modes you create a new InferenceService in the target mode, shift clients to it, and delete the old one.
  • Why does reading KServe's constants.go alone mislead about the stock default?
    It defines `DefaultDeployment = Standard`, which applies when an empty mode string is parsed. A stock install never reaches that case for new services, because the shipped ConfigMap sets `defaultDeploymentMode` to `Serverless`, which is normalized to `Knative`. The ConfigMap, not the constant, decides what an unannotated InferenceService gets.

  • A stock KServe install runs every InferenceService in Standard mode
  • Serverless and RawDeployment are the current KServe mode names
  • Standard mode scales an idle InferenceService to zero
  • You can flip a running InferenceService's mode by editing the annotation
  • canaryTrafficPercent is how Standard mode splits traffic

Assumes KServe 0.21, where the modes are Knative and Standard and the Serverless and RawDeployment names are deprecated aliases the webhook rewrites.