Silhouette scores stay flat from k=2 to k=30 on document embeddings — what does that mean?

## What silhouette actually measures For each point, let *a* be its mean distance to the other members of its own cluster and *b* its mean distance to the members of the nearest *other* cluster. The point's silhouette is (b − a) / max(a, b), which lands in [−1, 1]: near 1 means it sits comfortably inside its own group, near 0 means it is on the boundary between two, and negative means it is closer on average to a neighbouring cluster than to its own. The reported score is the mean over all points, so it is a compactness-versus-separation summary that assumes the distance metric you used is the right one. ## Reading a flat sweep A sweep from k=2 to k=30 that hovers in a narrow band with no meaningful peak carries a real message: at no granularity does the partition buy you separation. Two distinct causes produce the same picture. **The corpus has no partition structure.** Broad text corpora frequently form overlapping continua — support tickets shade from billing into account access into refunds, and there is no distance at which those regions detach. Silhouette is behaving correctly; it is telling you the shape of the data. **The algorithm's model does not fit.** Silhouette computed on a k-means partition inherits k-means' assumptions. Elongated, nested or wildly unequal clusters can be real yet score poorly under a mean-distance criterion. Before concluding the data is structureless, re-run the sweep with a different clustering family — hierarchical with average or Ward linkage, or a density method with a minimum-cluster-size sweep — and see whether the flatness persists. A related trap is the absolute scale. On high-dimensional text embeddings, mean silhouette values in the 0.05–0.2 range are ordinary even for partitions that human reviewers find useful, because the spread between within- and between-cluster distances is compressed. Anyone quoting "below 0.5 means bad clustering" is importing a threshold from low-dimensional tabular data. ## Why other internal indices rarely rescue you Davies–Bouldin (lower is better), Calinski–Harabasz (higher is better), inertia with the elbow heuristic and the gap statistic are all worth computing — they are cheap — but they measure variations on the same compactness/separation theme under similar geometric assumptions. When silhouette is flat they usually agree, and when they disagree you have no principled way to arbitrate. Treat unanimity as mild confirmation and disagreement as a signal that the structure is weak either way. Internal indices are weak evidence by construction: they never see anything outside the vectors and the metric you handed them. ## Getting external evidence This is where a stalled analysis actually gets unstuck. - **Held-out human labelling.** Sample 200–400 documents and have someone assign them to categories without seeing the clusters. Then compare the cluster assignments to those labels with the **adjusted Rand index** or **normalized mutual information**, both of which correct for chance agreement, and look at per-cluster purity. This answers a question no internal index can: do the machine's groups correspond to distinctions a person cares about? It can also reveal the opposite finding — that the human labels themselves do not separate in the embedding space, which is a legitimate and useful result. - **Stability selection.** Draw bootstrap or subsample replicates of the corpus, re-cluster each at the same k, and measure agreement on the overlapping points (adjusted Rand index between runs works well). Structure that is real tends to survive resampling; a k whose assignments reshuffle every replicate is not supported by the data, whatever any index says. Stability is often the single most informative check when internal indices are uninformative. - **Downstream utility.** Ultimately clusters exist to serve something — a triage queue, a taxonomy, a feature in a model, a report a human reads. Score them by that: does a reviewer find the clusters nameable and distinct? Does adding the cluster id as a feature move the target metric? A partition that is mediocre by silhouette but obviously useful to the team is a good partition. ## What to do when nothing supports a partition Accept the finding rather than shopping for a k that looks defensible on a chart. Practical exits: - **Drop the partition.** If the corpus is a continuum, nearest-neighbour retrieval serves most of the use cases people wanted clustering for, and it makes no false claim of discrete categories. - **Go hierarchical and let a human cut it.** A dendrogram lets a domain expert choose granularity where it is meaningful, which is more honest than an automatically selected k. - **Use density clustering and keep the noise.** Extracting a handful of genuinely dense pockets and admitting the rest is unstructured is often the true answer. - **Seed the clustering.** If you have a few labelled anchors, constrained or semi-supervised clustering can align groups with the distinctions you care about instead of the dominant variance in the space. - **Change the representation.** A general-purpose embedding model may not separate your domain's distinctions at all; a domain-adapted model, or embedding a normalized field rather than raw free text, can change the geometry more than any clustering hyperparameter will. And report the flat sweep itself. "There is no natural number of clusters here" is a finding, and stating it beats presenting a k chosen from noise as though it were discovered.