skip to content

Clustering and Dimensionality Reduction

You will learn how to find structure without labels — partitioning, density, and mixture views of clustering — and how PCA compresses features while keeping variance. Interviewers probe k-means assumptions, how you pick k, and when PCA helps versus when it destroys interpretability.

on this pageshow

explore

questions

page 1 of 2

In DBSCAN, what makes a point a core, border, or noise point?

level: juniorimportance: must knowfreq 68%

answer

  1. Two parameters, three labels
  2. Count neighbours inside a radius
  3. A threshold on that count
  4. Reachability from a dense point decides the rest

basics

~20 s

DBSCAN calls a point core when at least minPts points lie within radius eps of it. A non-core point that sits inside some core point's eps-radius is a border point. Anything else is noise and gets no cluster.

solid answer

~50 s

DBSCAN has two parameters: a radius `eps` and a count `minPts`. For every point you count how many points fall within `eps` of it. If that count reaches `minPts`, the point is a **core point** — it sits inside a dense region. A point that misses the threshold but still lies within `eps` of some core point is a **border point**: it joins that core point's cluster but cannot be used to grow the cluster further. A point that is neither is **noise**, and DBSCAN leaves it unassigned. Clusters are then the maximal groups of core points chained together through each other's neighbourhoods, plus the border points hanging off them. That chaining is why DBSCAN needs no cluster count and can trace a cluster of any shape — taxi pickups strung along a curved waterfront come out as one hotspot rather than being cut into round blobs.

code

python · 19 lines
python
import math

pts = [(0, 0), (0.4, 0.1), (0.2, 0.5), (0.8, 0.3),
       (-0.9, 0.2), (5.0, 5.0), (2.6, 2.6)]
eps, min_pts = 1.0, 3  # min_pts counts the point itself

def nbrs(i):
    return [j for j, q in enumerate(pts) if math.dist(pts[i], q) <= eps]

core = {i for i in range(len(pts)) if len(nbrs(i)) >= min_pts}

for i, p in enumerate(pts):
    if i in core:
        label = 'core'
    elif any(j in core for j in nbrs(i)):
        label = 'border'
    else:
        label = 'noise'
    print(p, len(nbrs(i)), label)

go deeper

for a junior

Be ready to name the two parameters and say which is a distance and which is a count, then define core, border and noise in one sentence each. Say plainly that DBSCAN is not told how many clusters to find.

for a middle

Explain the reachability chain: only core points extend a cluster, border points attach without extending, and a cluster is the maximal density-connected set. Then say what changes when you move eps up or minPts up.

for a senior

Show you know where the labels bite in practice: noise is a deliverable you have to act on, border assignment is order-dependent, and eps is meaningless until features are on a comparable scale. Tie the label counts back to a decision someone downstream makes.

for a principal

Own the framing question of whether a hard partition with an explicit noise bucket is the right output at all for the problem, versus a soft or hierarchical assignment. Argue the operational cost of a label that shifts run to run for points on cluster rims.

## The setup DBSCAN — Density-Based Spatial Clustering of Applications with Noise — clusters points by asking a purely local question of each one: *is it crowded around here?* It never computes a cluster centre, never assumes a cluster is round, and never asks you how many clusters exist. It asks you for two numbers instead. - **eps** (often written as the Greek epsilon) is a **distance**: the radius of the neighbourhood drawn around each point. - **minPts** is a **count**: how many points must fall inside that radius for the region to count as dense. The *eps-neighbourhood* of a point is the set of all points whose distance to it is at most `eps`. Everything below is a statement about the size of that set. ## The three labels **Core point.** A point whose eps-neighbourhood contains at least `minPts` points. It sits in the interior of a dense region. Conventions differ on whether the point itself counts toward the total — the original formulation includes it — so when you quote a `minPts` value in an interview, say which convention you are using. Core points are the only points that can *grow* a cluster. **Border point.** A point that fails the core test (fewer than `minPts` neighbours within `eps`) but lies within `eps` of at least one core point. It is on the rim of a dense region: close enough to belong, not dense enough to extend. Think of pickups on the outer edge of a taxi hotspot — clearly part of it, but the crowd thins out there. **Noise point.** A point that is neither core nor within `eps` of any core point. DBSCAN gives it no cluster at all. This is one of the algorithm's defining features: noise is a **first-class output**, not a failure. A lone pickup on a road shoulder between two hotspots is genuinely not part of either. ## How labels become clusters The glue is *reachability*. Point B is **directly density-reachable** from point A if A is a core point and B is inside A's eps-neighbourhood. B is **density-reachable** from A if you can walk from A to B through a chain of such steps, where every point in the chain except possibly the last is core. Two points are **density-connected** if some core point reaches both. A DBSCAN cluster is a maximal set of density-connected points. The practical consequence is the shape freedom. Because a cluster grows by hopping from core point to core point, its outline is whatever the dense region's outline happens to be. Two interlocking crescents stay two crescents; concentric rings stay separate rings — neither is recoverable by a method that assigns each point to the nearest of k centres, because such a method carves space into straight-edged, convex pieces and the crescent's own centre lies outside the crescent. ## Consequences worth knowing **Border points are the one soft spot.** A border point can sit within `eps` of core points belonging to two different clusters. It is density-reachable from both, but it can only be labelled once, so the classic algorithm gives it to whichever cluster reaches it first. Shuffle the input rows and that assignment can flip. Core-point and noise labels are stable; only border assignment is order-dependent. **The two parameters push in opposite directions.** Raising `eps` with `minPts` fixed enlarges every neighbourhood: more points clear the core test, clusters swell and eventually merge, and the noise count falls. Raising `minPts` with `eps` fixed makes the density test stricter: fewer core points, clusters shrink or split at their thin parts, and the noise count rises. Setting `minPts` to 1 degenerates the algorithm entirely — every point is core and every point is its own cluster. **eps is measured in the units of your feature space.** If one column runs in the thousands and another in the tenths, the large column dominates the distance and `eps` effectively measures only that column. Scale features before you begin. **Density must be roughly comparable across clusters.** One global `eps` and one global `minPts` encode a single notion of "dense enough". When one region is much tighter than another, no single pair of values serves both — that is DBSCAN's best-known limitation and the reason density-based hierarchies exist. ## What an interviewer is checking That you can state the two parameters and say which is a distance and which is a count; that you know noise is a deliberate label rather than a bug; that you do not confuse `minPts` with a number of clusters; and that you can explain, without hand-waving, why chaining through core points is what buys arbitrary cluster shapes.

  • Why can DBSCAN recover two interlocking crescent-shaped clusters when a centroid-based method cannot?
    Because a DBSCAN cluster grows by hopping from core point to core point through overlapping neighbourhoods, so its outline is just the outline of the dense region. A method that assigns each point to the nearest of k centres cuts space into convex pieces, and a crescent's own centre lies outside the crescent, so the two crescents get sliced across each other instead of separated.
  • Can the same border point end up in a different cluster if you shuffle the input rows?
    Yes. A border point can lie within eps of core points from two different clusters, and it can carry only one label, so the classic algorithm awards it to whichever cluster expands to it first — which depends on processing order. Core points and noise points are unaffected: those labels follow only from neighbourhood counts, so they are stable across runs.
  • What happens to the labels if you raise minPts and leave eps unchanged?
    The density test gets stricter, so fewer points clear the core threshold. Clusters shrink, thin low-density bridges between them break, some small clusters disappear entirely, and the share of points labelled noise rises. It is the usual knob for smoothing away spurious micro-clusters, at the cost of discarding genuinely sparse structure.

Picture taxi pickups dropped on a city map. A pickup surrounded by many others is deep inside a hotspot, one on the hotspot's fading rim still belongs to it, and a single fare on an empty road shoulder belongs to nothing at all.

saying these in an interview costs you the question

  • Says DBSCAN needs the number of clusters up front
  • Swaps the parameters: calls eps a count and minPts a distance
  • Treats noise points as errors to delete rather than a label
  • Claims every point ends up assigned to some cluster
  • Says border points can grow a cluster the same way core points do
  • Assumes DBSCAN clusters are round because a distance is involved

context

open as a page

In a Gaussian mixture model, what does it mean for a point to have responsibilities 0.6 and 0.4?

level: juniorimportance: must knowfreq 64%

basics

~20 s

The point is split softly between the two components: given the fitted model, there is a 60 percent posterior chance the first component generated it and 40 percent the second. It is an ambiguity score, not a label.

open as a page

What does each iteration of Lloyd's algorithm for k-means do?

level: juniorimportance: must knowfreq 84%

basics

~10 s

Each Lloyd iteration does two things: assign every point to its nearest centroid, then move each centroid to the mean of the points assigned to it. The loop repeats until assignments stop changing.

open as a page

What does a dendrogram show, and how does agglomerative clustering build one?

level: juniorimportance: must knowfreq 68%

basics

~20 s

A dendrogram is a tree recording the order and distance at which clusters merge. Agglomerative clustering starts with every observation as its own cluster and repeatedly merges the two closest ones, drawing each merge at the distance it happened.

open as a page

What does PCA maximise when it picks the first principal component?

level: juniorimportance: must knowfreq 78%

basics

~20 s

PCA picks the first principal component as the unit-length direction in the mean-centred data along which the projected points have the largest variance. Each later component maximises the remaining variance while staying orthogonal to all earlier ones.

open as a page

In an RFM customer segmentation, what do recency, frequency and monetary value each measure?

level: juniorimportance: must knowfreq 58%

basics

~20 s

Recency is how long since a customer's last purchase, frequency is how many purchases they made in a fixed window, and monetary is how much they spent in it. Recent, frequent, high-spending customers are the top tier.

open as a page

On a t-SNE map, why is the gap between two clusters not a real distance?

level: juniorimportance: must knowfreq 70%

basics

~20 s

t-SNE only preserves each point's near neighbours. It minimises a divergence that punishes tearing neighbours apart but barely punishes moving distant points, so between-cluster gaps and island sizes are artefacts of the layout, not measured distances.

open as a page

How does an isolation forest use a point's average path length to score it as an anomaly?

level: middleimportance: must knowfreq 65%

basics

~20 s

An isolation forest splits data at random until each point sits alone. Anomalies stand apart, so random cuts isolate them in very few splits. The score is the average number of splits across many trees, normalised: short paths mean anomalous.

open as a page

Why does k-means inertia fall monotonically with k, and how does the elbow method cope?

level: middleimportance: must knowfreq 72%

basics

~20 s

Inertia is the within-cluster sum of squared distances to centroids, and adding a cluster can only shrink it - at k equal to the number of points it reaches zero. So the elbow method looks for diminishing returns, never for a minimum.

open as a page

How does linear discriminant analysis choose its projection axes differently from PCA?

level: middleimportance: must knowfreq 58%

basics

~20 s

Linear discriminant analysis uses the labels. It picks axes that maximise the spread between class means relative to the spread inside each class. PCA ignores labels and picks directions of largest total variance, which need not separate classes at all.

open as a page

What does the adjusted Rand index correct for that the raw Rand index does not?

level: middleimportance: must knowfreq 63%

basics

~20 s

The adjustment removes the agreement two labelings reach by chance. The raw Rand index counts agreeing point pairs and stays high even for unrelated labelings; the adjusted version rescales it to about 0 for random, 1 for identical.

open as a page

What happens in the E step and the M step when EM fits a Gaussian mixture?

level: middleimportance: must knowfreq 72%

basics

~20 s

The E step fixes the parameters and computes each point's responsibility: the posterior probability that each component generated it. The M step fixes those responsibilities and re-estimates every component's weight, mean and covariance as responsibility-weighted averages.

open as a page

How is the silhouette coefficient computed for a single point, and what does a negative value mean?

level: middleimportance: must knowfreq 74%

basics

~20 s

For a point, a is its mean distance to the other members of its own cluster and b its mean distance to the points of the nearest other cluster; s = (b - a) / max(a, b). Negative s means the point sits closer to that other cluster.

open as a page

Why can two k-means runs on the same data return different clusterings?

level: middleimportance: must knowfreq 68%

basics

~20 s

k-means starts from randomly chosen centroids and only converges to a local minimum of the within-cluster sum of squares, so a different start can settle in a different clustering. Multiple restarts and k-means++ seeding reduce the spread.

open as a page

Why is k-medoids more robust to outliers than k-means, and what does that robustness cost?

level: middleimportance: must knowfreq 60%

basics

~20 s

k-medoids uses an actual data point as each cluster centre and minimises the sum of distances to it, so one extreme value cannot drag the centre the way a mean can. The price is a far slower, roughly quadratic search.

open as a page

How do single, complete, average and Ward linkage decide which clusters merge next?

level: middleimportance: must knowfreq 74%

basics

~20 s

Each rule collapses the distances between two clusters' members into one number, and the closest pair merges. Single takes the nearest cross-pair, complete the farthest, average the mean; Ward merges whichever pair adds least to within-cluster sum of squares.

open as a page

How do you choose how many principal components to keep?

level: middleimportance: must knowfreq 68%

basics

~20 s

Read the explained-variance ratios - each component's share of total variance. Common rules are a cumulative threshold such as 95%, the elbow of a scree plot, or tuning the count against the downstream model's score.

open as a page

How do you profile and name customer segments once a clustering run has produced them?

level: middleimportance: must knowfreq 64%

basics

~20 s

Build a profile table: each segment's mean on every feature indexed against the overall mean, plus headcount and revenue share. Name each segment from the two or three dimensions where it differs most from the base.

open as a page

How does latent Dirichlet allocation model a document as a mixture of topics?

level: middleimportance: must knowfreq 60%

basics

~20 s

Latent Dirichlet allocation treats each document as a probability distribution over K topics, and each topic as a probability distribution over vocabulary words. Fitting infers both, so one support ticket comes out as 0.6 billing, 0.3 login, 0.1 shipping.

open as a page

In t-SNE, what does the perplexity setting actually control?

level: middleimportance: must knowfreq 58%

basics

~20 s

Perplexity sets the effective number of near neighbours each point is fitted to. Each point's Gaussian kernel width is tuned by search until its neighbour distribution has that perplexity, so the setting chooses the scale at which structure is preserved.

open as a page

Why can cluster purity not be reported on its own as a clustering quality score?

level: juniorimportance: should knowfreq 44%

basics

~20 s

Purity credits each cluster only for its majority true class, so splitting a cluster can never lower it. Push the number of clusters up and purity climbs toward 1.0, reaching it when every point sits alone.

open as a page

Why is it wrong to compare k-means inertia between two runs built on different feature sets?

level: juniorimportance: should knowfreq 41%

basics

~20 s

Inertia is the total within-cluster sum of squared distances from points to their cluster centroid. It is an unnormalised quantity whose magnitude grows with feature count, feature scale and row count, so two runs over different feature sets are simply on different scales.

open as a page

How do you cluster a table of purely categorical attributes, where a mean is undefined?

level: juniorimportance: should knowfreq 40%

basics

~20 s

Use k-modes: each cluster centre is the most frequent value of every attribute, and two records are compared by counting the attributes on which they disagree. Averaging category codes is meaningless, so plain k-means does not apply.

open as a page

What does the local outlier factor capture that a single global density cut-off misses?

level: middleimportance: should knowfreq 50%

basics

~20 s

Local outlier factor compares a point's density with that of its own nearest neighbours. It flags a point sitting in a sparser pocket than its surroundings, even where one global density threshold would call it perfectly normal.

open as a page

How do you choose DBSCAN's eps parameter from a sorted k-distance curve?

level: middleimportance: should knowfreq 52%

basics

~20 s

Compute every point's distance to its minPts-th nearest neighbour, sort those distances, and plot them. The curve stays flat where points have close neighbours and turns sharply upward where they do not. Set eps at that knee.

open as a page

In a Gaussian mixture, what do spherical, diagonal and full covariance each buy you?

level: middleimportance: should knowfreq 46%

basics

~20 s

Spherical forces round components of one width; diagonal allows axis-aligned stretching; full allows tilt and correlation. Each step up fits shape better but costs more parameters, so full covariance needs far more data per component.

open as a page

How do you choose where to cut a dendrogram to get a flat set of clusters?

level: middleimportance: should knowfreq 57%

basics

~20 s

A horizontal cut at some height keeps every merge below it and undoes the rest; equivalently, stopping after n minus k merges leaves k clusters. Cut inside a wide gap between consecutive merge heights, then check the groups against outside constraints.

open as a page

When would you prefer NMF over latent semantic analysis for extracting topics?

level: middleimportance: should knowfreq 45%

basics

~10 s

Prefer non-negative matrix factorization when people must read the topics: non-negativity makes every topic an additive list of words. Latent semantic analysis produces signed components that capture synonymy well but rarely read as themes.

open as a page

How do you set an assumed contamination rate, or a one-class SVM's nu, with no labels?

level: seniorimportance: should knowfreq 45%

basics

~20 s

Set the contamination rate from review capacity, not from a guess at the truth: it only chooses where to cut a score ranking, so pick the cut that yields an alert volume your team can actually work through.

open as a page

A k-means elbow reads k=4 but the silhouette-versus-k sweep peaks at k=7 - how do you choose?

level: seniorimportance: should knowfreq 56%

basics

~20 s

The two routes optimise different things, so disagreement is normal rather than a contradiction. Cross-tabulate the two solutions to see whether the seven nest inside the four, check the size distribution behind the silhouette peak, and let the intended use pick the granularity.

open as a page

showing 1–30 of 57