How do a Kubernetes ResourceQuota's scopes and scopeSelector fields narrow which pods it counts, and what do the Terminating and BestEffort scopes match?

middlenice to knowfreq 29%

answer

filters, all must match
deadline, not deletion
BestEffort counts pods only
scopeSelector with In/NotIn
limitedResources makes quota mandatory

basics

~20 s

Scopes make a ResourceQuota count only matching pods: Terminating means spec.activeDeadlineSeconds is set, BestEffort means no CPU or memory requests or limits, and PriorityClass matches priorityClassName. A pod must match every listed scope and selector.

solid answer

~40 s

By default a ResourceQuota counts every object of the kinds it caps. spec.scopes and spec.scopeSelector filter that set, and a pod must match all of them to be counted. Terminating matches pods with spec.activeDeadlineSeconds set, NotTerminating those without it. BestEffort matches pods whose containers declare no CPU or memory requests or limits, and it can only cap the pods count; NotBestEffort can also cap CPU and memory. PriorityClass goes in scopeSelector with In, NotIn, Exists or DoesNotExist, which lets you give each priority tier its own budget. Opposite scopes such as Terminating and NotTerminating cannot be combined. A cluster admin can also configure the API server so that pods using a given class are rejected unless a quota covering that class exists in their namespace.

By default a `ResourceQuota` counts every object of the kinds it caps. `spec.scopes` and `spec.scopeSelector` filter that set, and a pod must match **all** of them to be counted. `Terminating` matches pods with `spec.activeDeadlineSeconds` set, `NotTerminating` those without it. `BestEffort` matches pods whose containers declare no CPU or memory requests or limits, and it can only cap the `pods` count; `NotBestEffort` can also cap CPU and memory. `PriorityClass` goes in `scopeSelector` with `In`, `NotIn`, `Exists` or `DoesNotExist`, which lets you give each priority tier its own budget. Opposite scopes such as `Terminating` and `NotTerminating` cannot be combined. A cluster admin can also configure the API server so that pods using a given class are rejected unless a quota covering that class exists in their namespace.

code

yaml · 14 lines

yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: flags-high-priority
  namespace: flags
spec:
  hard:
    pods: "5"
    requests.cpu: 3100m
  scopeSelector:
    matchExpressions:
    - scopeName: PriorityClass
      operator: In
      values: ["tenant-high"]

go deeper

for a junior

Recall that scopes filter which pods a quota counts, and that Terminating is about activeDeadlineSeconds, not deletion.

for a middle

Explain the AND semantics of scopes and scopeSelector, why BestEffort quotas can only count pods, and which operators each scope accepts.

for a senior

Show how you would give each priority tier its own per-namespace budget and debug a scoped quota whose Used value never moves.

for a principal

Weigh opt-in quotas against limitedResources, which makes a class unusable until the platform team grants a namespace a covering quota.

Why scopes exist

A plain ResourceQuota counts every pod in its namespace. That is too blunt when one namespace runs different kinds of work: long-running services, short batch pods and pods at different priority tiers. Scopes are filters that make one quota count only a subset of objects, so each subset can get its own budget.

There are two ways to write them, and they are ANDed:

spec.scopes: a plain list of scope names. Every listed scope must match.
spec.scopeSelector.matchExpressions: a list of {scopeName, operator, values} requirements. Every expression must match.

If a quota sets both, a pod has to satisfy both. The kubectl create quota command exposes the plain form as --scopes.

The pod scopes and what they match

Scope	Matches	Keys it may cap
`Terminating`	pods with `spec.activeDeadlineSeconds` set to 0 or more	`pods` and compute keys
`NotTerminating`	pods with no `activeDeadlineSeconds`	`pods` and compute keys
`BestEffort`	pods whose QoS class is BestEffort (no CPU or memory requests or limits in any container)	`pods` only
`NotBestEffort`	every other pod	`pods` and compute keys
`PriorityClass`	pods by `spec.priorityClassName`	`pods` and compute keys
`CrossNamespacePodAffinity`	pods whose affinity terms reach other namespaces	`pods` and compute keys

There is also a VolumeAttributesClass scope for PersistentVolumeClaims, which can cap claim counts and requests.storage.

Points worth getting exactly right:

Terminating does not mean "being deleted". It means "has an active deadline", which is typical of Job pods. A pod shutting down has a deletionTimestamp, and that is unrelated to this scope.
BestEffort quotas can only count pods. Such pods declare no CPU or memory, so there is nothing to sum; putting requests.cpu in a BestEffort quota fails validation.
Conflicting pairs are rejected. Terminating with NotTerminating, or BestEffort with NotBestEffort, in one quota is invalid.
Operators are restricted. For Terminating, NotTerminating, BestEffort, NotBestEffort and CrossNamespacePodAffinity the operator must be Exists. PriorityClass accepts In, NotIn, Exists and DoesNotExist, and In/NotIn need a non-empty values list.

Budgeting by priority tier

The most common real use of scopes is a separate budget for each priority tier. How priority values and preemption work belongs to the scheduler; here the class name is just a label to match on.

Create one quota per tier, each with a scopeSelector that uses scopeName: PriorityClass, operator: In and that tier's class names.
Give the high tier a small pods and requests.cpu budget, so a team cannot mark all its workloads high-priority.
Optionally add a catch-all quota using NotIn or DoesNotExist for everything else.

A pod that matches several quotas must fit under all of them.

Making a class unusable without quota

A per-namespace quota only restricts namespaces that have one. To make a priority class unusable by default, the cluster admin configures the ResourceQuota admission plugin itself. The file passed to kube-apiserver with --admission-control-config-file holds a ResourceQuotaConfiguration whose limitedResources entries list matchScopes. A pod matching those scopes is then rejected with insufficient quota to match these scopes unless its namespace has a quota covering that scope. That turns quota into an allow-list: only namespaces the platform team gave a high-tier quota can run high-tier pods.

A worked split for one namespace

Suppose the flags namespace runs the long-lived flag-eval service and short cache-warming Jobs whose pods set activeDeadlineSeconds: 420. Three quotas give each kind of work its own budget:

Quota	Filter	Caps
`flags-services`	`scopes: [NotTerminating, NotBestEffort]`	`requests.cpu: 6800m`, `pods: "13"`
`flags-batch`	`scopes: [Terminating, NotBestEffort]`	`requests.cpu: 1900m`, `pods: "11"`
`flags-unsized`	`scopes: [BestEffort]`	`pods: "2"`

A burst of Jobs can now use up only the batch budget and never blocks a flag-eval rollout. The NotBestEffort scope on both compute quotas matters: without it, an unsized pod would match a quota that caps requests.cpu and be rejected for not declaring CPU. With it, an unsized pod matches only flags-unsized, which allows at most two.

Checking what a scoped quota counts

kubectl describe resourcequota lists the quota's scopes and its Used/Hard values.
If Used stays at zero while matching pods run, check the filter first: an activeDeadlineSeconds mistake or a misspelled class name in values is the usual cause.
Remember the rule that containers must declare CPU and memory once a quota caps them applies only to pods the quota matches, so a scoped compute quota affects only its subset.

## Why scopes exist A plain **ResourceQuota** counts every pod in its namespace. That is too blunt when one namespace runs different kinds of work: long-running services, short batch pods and pods at different **priority** tiers. **Scopes** are filters that make one quota count only a subset of objects, so each subset can get its own budget. There are two ways to write them, and they are **ANDed**: - `spec.scopes`: a plain list of scope names. Every listed scope must match. - `spec.scopeSelector.matchExpressions`: a list of `{scopeName, operator, values}` requirements. Every expression must match. If a quota sets both, a pod has to satisfy both. The `kubectl create quota` command exposes the plain form as `--scopes`. ## The pod scopes and what they match | Scope | Matches | Keys it may cap | |---|---|---| | `Terminating` | pods with `spec.activeDeadlineSeconds` set to 0 or more | `pods` and compute keys | | `NotTerminating` | pods with no `activeDeadlineSeconds` | `pods` and compute keys | | `BestEffort` | pods whose QoS class is BestEffort (no CPU or memory requests or limits in any container) | `pods` only | | `NotBestEffort` | every other pod | `pods` and compute keys | | `PriorityClass` | pods by `spec.priorityClassName` | `pods` and compute keys | | `CrossNamespacePodAffinity` | pods whose affinity terms reach other namespaces | `pods` and compute keys | There is also a `VolumeAttributesClass` scope for PersistentVolumeClaims, which can cap claim counts and `requests.storage`. Points worth getting exactly right: - **Terminating does not mean "being deleted".** It means "has an active deadline", which is typical of Job pods. A pod shutting down has a `deletionTimestamp`, and that is unrelated to this scope. - **BestEffort quotas can only count pods.** Such pods declare no CPU or memory, so there is nothing to sum; putting `requests.cpu` in a `BestEffort` quota fails validation. - **Conflicting pairs are rejected.** `Terminating` with `NotTerminating`, or `BestEffort` with `NotBestEffort`, in one quota is invalid. - **Operators are restricted.** For `Terminating`, `NotTerminating`, `BestEffort`, `NotBestEffort` and `CrossNamespacePodAffinity` the operator must be `Exists`. `PriorityClass` accepts `In`, `NotIn`, `Exists` and `DoesNotExist`, and `In`/`NotIn` need a non-empty `values` list. ## Budgeting by priority tier The most common real use of scopes is a separate budget for each **priority tier**. How priority values and preemption work belongs to the scheduler; here the class name is just a label to match on. 1. Create one quota per tier, each with a `scopeSelector` that uses `scopeName: PriorityClass`, `operator: In` and that tier's class names. 2. Give the high tier a small `pods` and `requests.cpu` budget, so a team cannot mark all its workloads high-priority. 3. Optionally add a catch-all quota using `NotIn` or `DoesNotExist` for everything else. A pod that matches several quotas must fit under **all** of them. ## Making a class unusable without quota A per-namespace quota only restricts namespaces that have one. To make a priority class unusable by default, the cluster admin configures the `ResourceQuota` admission plugin itself. The file passed to kube-apiserver with `--admission-control-config-file` holds a `ResourceQuotaConfiguration` whose `limitedResources` entries list `matchScopes`. A pod matching those scopes is then rejected with `insufficient quota to match these scopes` unless its namespace has a quota covering that scope. That turns quota into an allow-list: only namespaces the platform team gave a high-tier quota can run high-tier pods. ## A worked split for one namespace Suppose the `flags` namespace runs the long-lived `flag-eval` service and short cache-warming Jobs whose pods set `activeDeadlineSeconds: 420`. Three quotas give each kind of work its own budget: | Quota | Filter | Caps | |---|---|---| | `flags-services` | `scopes: [NotTerminating, NotBestEffort]` | `requests.cpu: 6800m`, `pods: "13"` | | `flags-batch` | `scopes: [Terminating, NotBestEffort]` | `requests.cpu: 1900m`, `pods: "11"` | | `flags-unsized` | `scopes: [BestEffort]` | `pods: "2"` | A burst of Jobs can now use up only the batch budget and never blocks a `flag-eval` rollout. The `NotBestEffort` scope on both compute quotas matters: without it, an unsized pod would match a quota that caps `requests.cpu` and be rejected for not declaring CPU. With it, an unsized pod matches only `flags-unsized`, which allows at most two. ## Checking what a scoped quota counts - `kubectl describe resourcequota` lists the quota's scopes and its `Used`/`Hard` values. - If `Used` stays at zero while matching pods run, check the filter first: an `activeDeadlineSeconds` mistake or a misspelled class name in `values` is the usual cause. - Remember the rule that containers must declare CPU and memory once a quota caps them applies only to pods the quota **matches**, so a scoped compute quota affects only its subset.

Why would you pair a BestEffort-scoped quota with a NotBestEffort one?
BestEffort pods declare nothing, so a compute quota cannot measure them; a count is the only lever. A `BestEffort` quota with a small `pods` value limits how many unsized pods a namespace can run, while a `NotBestEffort` quota caps the CPU and memory of everything that does declare resources. Together they cover the whole namespace without forcing every pod into one budget.
What error does a pod get when a limitedResources rule matches it and its namespace has no covering quota?
The ResourceQuota admission plugin rejects the create with 403 Forbidden and a message saying there is insufficient quota to match those scopes. The fix is for whoever owns quotas to create a ResourceQuota in that namespace whose `scopeSelector` covers the class, which is how a platform team grants a tier to a tenant.

saying these in an interview costs you the question

The Terminating scope counts pods that are shutting down after deletion.
A BestEffort-scoped quota can cap requests.cpu for unsized pods.
A pod is counted if it matches any one of the listed scopes.
Scopes can filter pods by arbitrary labels like app or team.
A PriorityClass-scoped quota restricts the class in every namespace automatically.

How do a Kubernetes ResourceQuota's scopes and scopeSelector fields narrow which pods it counts, and what do the Terminating and BestEffort scopes match?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Why scopes exist

The pod scopes and what they match

Budgeting by priority tier

Making a class unusable without quota

A worked split for one namespace

Checking what a scoped quota counts

▾common follow-ups2 questions

▾red flags5 flags

Why scopes exist

The pod scopes and what they match

Budgeting by priority tier

Making a class unusable without quota

A worked split for one namespace

Checking what a scoped quota counts

answer

go deeper