On a stock KServe 0.21 install, which deployment mode does an InferenceService without a deploymentMode annotation get, and what does each mode bring?
answer
- one annotation per service
- cluster default lives in a ConfigMap
- shipped value uses the legacy name
- revisions and scale-to-zero vs Deployment
- fixed once the service exists
basics
~20 sKnative mode. The shipped inferenceservice-config ConfigMap sets defaultDeploymentMode to Serverless, which KServe 0.21 reads as Knative: revisions, scale-to-zero and canaryTrafficPercent. Standard mode, chosen per service by annotation, runs a plain Deployment with HPA or KEDA.
solid answer
~30 sThe mode comes from the `serving.kserve.io/deploymentMode` annotation, or, when it is absent, from `defaultDeploymentMode` in the `deploy` entry of the `inferenceservice-config` ConfigMap. KServe 0.21 ships that as `"Serverless"`, the legacy name it normalizes to `Knative`, so a stock install runs Knative mode, although the code's own empty-value default is `Standard`. Knative mode gives revisions, KPA autoscaling, scale-to-zero and `canaryTrafficPercent`. `Standard` gives a Deployment, Service and HPA (or KEDA), no scale-to-zero, `deploymentStrategy`, and since 0.20 its own `spec.canary` list. `RawDeployment` and `Serverless` still parse but are deprecated names, and the webhook refuses changing the mode of an existing InferenceService.
code
yaml · 12 linesapiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: playlist-blurbs
annotations:
serving.kserve.io/deploymentMode: Standard
spec:
predictor:
model:
modelFormat:
name: huggingface
storageUri: hf://example-org/blurb-7b-instructgo deeper
Recall the annotation name, the two 0.21 mode names, and that the stock default is Knative.
Explain where the default lives (the inferenceservice-config ConfigMap), how legacy names are normalized, and what objects each mode creates.
Pick the mode per workload knowing it is fixed at creation, and verify the cluster default instead of trusting a constant or an old doc.
Decide whether the platform needs Knative at all: it buys scale-to-zero and revisions at the cost of another control plane and cold starts.
## Two deployment modes, one annotation In KServe 0.21 an InferenceService runs in one of two **deployment modes**: - **`Knative`**: the controller creates a Knative Service. Every change to the spec makes a new immutable **revision**, Knative's autoscaler (KPA) scales on request concurrency or rps, and pods can **scale to zero**. - **`Standard`**: the controller creates a plain Kubernetes Deployment and Service, plus an HPA (or a KEDA ScaledObject) for scaling. No Knative needed, no revisions, no scale-to-zero. The mode is chosen per InferenceService by the annotation `serving.kserve.io/deploymentMode`. When the annotation is absent, the cluster default comes from the `deploy` entry of the **`inferenceservice-config`** ConfigMap in the `kserve` namespace, key `defaultDeploymentMode`. ## What a stock 0.21 install actually does The source sends a mixed signal, and that is where confident answers go wrong: 1. `pkg/constants/constants.go` defines `DefaultDeployment = Standard`, used when an empty mode string is parsed. 2. The shipped ConfigMap (`config/configmap/inferenceservice.yaml`) sets `"defaultDeploymentMode": "Serverless"`. 3. `Serverless` is the legacy name for `Knative`; the config loader normalizes it, so the cluster default is **Knative**. So on a stock install an InferenceService with no annotation runs in **Knative mode**, and needs Knative Serving to be installed. A cluster without Knative must either change the ConfigMap's `defaultDeploymentMode` to `Standard` or annotate each InferenceService. ## The names changed | Older name | 0.21 name | What 0.21 does with the old one | |---|---|---| | `Serverless` | `Knative` | constants mark it deprecated; the webhook rewrites the annotation to `Knative` | | `RawDeployment` | `Standard` | constants mark it deprecated; the webhook rewrites the annotation to `Standard` | Old manifests keep working. Reach for the new names, and expect status, docs and validation messages to use them. ## What each mode brings or withholds | Capability | Knative | Standard | |---|---|---| | Underlying objects | Knative Service and revisions | Deployment, Service, HPA or KEDA | | Scale to zero | yes, with `minReplicas: 0` | not supported | | Default autoscaling | KPA on `concurrency` | HPA on `cpu` (`serving.kserve.io/autoscalerClass: hpa`) | | Canary | `canaryTrafficPercent` between revisions | `spec.canary` list (since 0.20) | | Rollout tuning | revisions | `deploymentStrategy` (rejected in Knative mode) | | Extra per-pod container | Knative queue proxy | none | Knative suits bursty traffic and services that may sit idle. Standard avoids cold starts and the Knative dependency; KServe's docs recommend it for LLM workloads. ## The mode is fixed at creation The validating webhook compares the annotation with the mode recorded in the InferenceService status and refuses a change: `update rejected: deploymentMode cannot be changed from 'Knative' to 'Standard'`. Switching modes means creating a new InferenceService (or deleting and re-creating) and moving clients over, so it is a decision to get right up front. ## Choosing per workload on one cluster On a cluster with Knative installed, the two modes can run side by side, because the annotation is per InferenceService: - A small tabular model with a steady daytime pattern and quiet nights, such as a track-skip classifier, is a natural Knative service. It can scale to zero overnight, and new boosters roll out through revisions and `canaryTrafficPercent`. - A GPU-hungry generative model, such as a 7B blurb writer, is often put in `Standard` mode. Its cold start is long anyway, it gains nothing from the queue proxy, and an HPA or KEDA on a custom metric fits it better. - A team that needs a weighted canary in `Standard` mode uses the `spec.canary` list (0.20 and later). It has to annotate the service with `Standard` explicitly, because canary validation reads the annotation rather than the cluster default. Write the annotation on every production InferenceService even when it matches the cluster default. The default is a ConfigMap value that an administrator can change, and the explicit annotation keeps each service's behaviour independent of that edit. ## Checking a cluster - Read the ConfigMap: `kubectl get configmap inferenceservice-config -n kserve -o jsonpath='{.data.deploy}'`. - Read the InferenceService: the annotation, if set, and `status.deploymentMode`. - Look at what was created: a Knative Service and revisions, or a Deployment and HPA.
- A team edits a running KServe InferenceService's deploymentMode annotation from Knative to Standard. What happens?The validating webhook rejects the update with `deploymentMode cannot be changed`, because the mode recorded in the InferenceService status differs from the new annotation. To move a service between modes you create a new InferenceService in the target mode, shift clients to it, and delete the old one.
- Why does reading KServe's constants.go alone mislead about the stock default?It defines `DefaultDeployment = Standard`, which applies when an empty mode string is parsed. A stock install never reaches that case for new services, because the shipped ConfigMap sets `defaultDeploymentMode` to `Serverless`, which is normalized to `Knative`. The ConfigMap, not the constant, decides what an unannotated InferenceService gets.
- A stock KServe install runs every InferenceService in Standard mode
- Serverless and RawDeployment are the current KServe mode names
- Standard mode scales an idle InferenceService to zero
- You can flip a running InferenceService's mode by editing the annotation
- canaryTrafficPercent is how Standard mode splits traffic
Assumes KServe 0.21, where the modes are Knative and Standard and the Serverless and RawDeployment names are deprecated aliases the webhook rewrites.