A dense core and a diffuse halo break DBSCAN's single eps — how does HDBSCAN fix that?
answer
- One radius, two densities
- The core and the halo want different radii
- Do not choose a radius — sweep all of them
- Keep whatever survives the longest span of density levels
basics
~20 sOne global eps encodes a single notion of dense, so it cannot be tight enough for a core and loose enough for a halo. HDBSCAN sweeps all radii instead and keeps the clusters that persist longest.
solid answer
~50 sDBSCAN applies a single radius everywhere. On a galaxy catalogue with a tight core and a sparse halo, a small `eps` resolves the core beautifully and dumps the entire halo into noise; a large `eps` picks up the halo but swallows the core into one undifferentiated blob and starts bridging to neighbouring structures. There is no value in between that serves both, because the two regions are dense at different scales. HDBSCAN removes the choice: it inflates distances in sparse regions using each point's own neighbour distance, builds a cluster tree across *all* radii, and then selects, branch by branch, the clusters that survive the longest span of density levels. A dense core can be selected at one density and the halo at another, in the same run. What remains is a minimum cluster size, a more interpretable knob than a radius, and noise is still an output.
go deeper
Be able to say that DBSCAN applies one radius everywhere, so data with a tight region and a loose region cannot be served well by any single value. Knowing that a hierarchical variant exists to address this is enough here.
Explain both failure directions concretely — a small radius discards the sparse region as noise, a large one dissolves the dense region and bridges to its neighbours — and describe HDBSCAN as building results for all radii at once rather than committing to one.
Show the mechanism: a per-point core distance, mutual reachability inflating sparse separations, a hierarchy built by removing longest edges, and selection by how long a cluster persists. Then state honestly what you pay in runtime and in explainability.
Own the call between an interpretable global radius a domain expert can challenge and a stability-selected hierarchy that fits the data better but is harder to defend. Argue which failure your organisation can actually absorb and what happens when densities drift over time.
## The failure first DBSCAN's two parameters together define exactly one threshold: `minPts` points inside radius `eps` means dense. That threshold is global. It is applied identically at every location in the feature space. Take a galaxy catalogue with a tightly packed core and a wide, thin halo around it. - Pick `eps` small enough that the core resolves into its real substructure, and no halo point has `minPts` neighbours that close. The entire halo is labelled noise — thousands of real objects thrown away. - Pick `eps` large enough that halo points find company, and every core point now has hundreds of neighbours. The core's internal structure vanishes into one mass, and if any other structure lies nearby, the enlarged neighbourhoods bridge across and merge them too. - Anything in between does a mediocre job of both. This is not a tuning failure. It is structural: one radius encodes one density, and the data has two. Recognising that — rather than promising to tune harder — is the substance of the answer. ## What HDBSCAN changes HDBSCAN keeps DBSCAN's definition of density and drops the commitment to a single radius. **Core distance.** For each point, take the distance to its k-th nearest neighbour, for a chosen k. A point in a dense region has a small core distance; a point in a sparse region has a large one. This is a *local* measure of how crowded that point's own surroundings are. **Mutual reachability distance.** Between two points a and b, define it as the largest of three quantities: a's core distance, b's core distance, and the plain distance between them. Points in dense regions keep their true separation, because their core distances are small. Points in sparse regions get pushed apart, because their large core distances dominate. The effect is to make sparse points harder to connect without changing anything about dense ones. **A hierarchy over all radii.** Build a minimum spanning tree over the mutual reachability distances and then remove edges from longest to shortest. Each removal splits the data further. Sweeping the removal threshold from large to small is exactly equivalent to running DBSCAN at every `eps` from large to small and recording what happens — so the tree encodes the entire family of DBSCAN results in one structure, instead of one member of it. **Condense, then select by stability.** Not every split is meaningful; many just shed a few points. HDBSCAN condenses the tree using a **minimum cluster size**: a split into two groups where one is smaller than that size is treated as the parent losing points, not as a real split. What remains is a compact tree of candidate clusters, each of which is born at some density level and dies at another. HDBSCAN then scores each candidate by **stability** — roughly, how much density range it survives, weighted by how many points survive with it — and selects a set of non-overlapping candidates maximising that total. The consequence is the fix: the selected clusters can come from *different* levels of the tree. The dense core is selected at a high density level, the diffuse halo at a low one, in a single run, with no global radius reconciling them. ## What you still have to decide, and what you give up - **A minimum cluster size** remains, and it is a genuine modelling choice — it states how many members a group needs before you would call it a group. Most people find this easier to reason about with a domain expert than a radius in scaled feature units. - **A neighbour count for the core distance** also remains, controlling how aggressively sparse regions are pushed apart. Larger values smooth the result and push more points toward noise. - **Noise still exists.** HDBSCAN does not assign everything; points that never join a stable cluster stay unassigned, and each assigned point additionally carries a membership strength saying how firmly it belongs. - **Cost goes up.** You are building a spanning tree and a hierarchy rather than doing local neighbourhood queries with one threshold, so expect it to be heavier than DBSCAN on the same data. - **Interpretability goes down in one specific way.** DBSCAN's `eps` is a physical quantity — so many metres, so many scaled units — that a domain expert can argue about directly. "The clusters that were most stable across density levels" is harder to defend in a room. If your domain has a defensible radius, DBSCAN's explicitness can be worth more than HDBSCAN's flexibility. - **The shared limits stay shared.** Both depend on a meaningful distance metric, both need scaled features, and both degrade when dimensionality is high enough that distances concentrate. ## The alternative you should be able to reject A reasonable interviewer will ask why you do not simply run DBSCAN twice with two radii and combine. You can, *if* you can partition the space in advance into regions of comparable density and defend the split — core versus halo, say, by radius from a centre. When you cannot, you are hand-executing a coarse version of the sweep HDBSCAN does properly, and you inherit the problem of reconciling points that two runs label differently.
- What do you give up by moving from DBSCAN to HDBSCAN?Speed and explicitness. Building a hierarchy over all density levels costs more than thresholded neighbourhood queries. More importantly, DBSCAN's eps is a physical radius a domain expert can argue about directly, whereas selecting clusters by stability is much harder to defend in a review. When your domain supplies a defensible radius, DBSCAN's transparency can outweigh HDBSCAN's flexibility.
- Why not just run DBSCAN twice with two different eps values and combine the results?That works when you can partition the space in advance into regions of comparable density and defend the partition — core versus halo by radius, for instance. When you cannot, you are hand-executing a two-point version of the sweep HDBSCAN performs continuously, and you still have to reconcile points that the two runs label differently, which has no principled answer.
- Does HDBSCAN eliminate the noise label?No. Points that never join a cluster that is stable across a meaningful density range remain unassigned, exactly as in DBSCAN. What it adds is a membership strength for each assigned point, so rim members can be distinguished from firmly interior ones rather than both being flat cluster members.
- How would you sanity-check HDBSCAN's clusters when you have no labels?Look at the stability scores and sizes of the selected clusters and at the share of points left unassigned, then re-run on a resample and see whether the same groups reappear. Beyond that, pull sample members from each cluster and have a domain expert say whether the grouping corresponds to anything real — that review is what earns an unsupervised result trust.
saying these in an interview costs you the question
- Says DBSCAN handles varying density fine because it needs no k
- Claims more tuning of eps would eventually fix a two-density dataset
- Thinks HDBSCAN is parameter-free
- Says HDBSCAN assigns every point and never reports noise
- Asserts HDBSCAN is faster than DBSCAN because it skips the eps search
- Believes raising eps fixes a core that has merged into its surroundings