First Off, What Is t-SNE?
Imagine you have a spreadsheet with thousands of rows and hundreds of columns—a high-dimensional dataset. Your mission is to see which data points are similar to each other. It's impossible to visualize in 100 dimensions, so you use an algorithm to plot
it in 2D. That's where t-SNE (t-Distributed Stochastic Neighbor Embedding) comes in. Its goal is to take points that are close together in that complex, high-dimensional space and make sure they're close together on a simple 2D map. It’s a powerful tool for exploring data, but it achieves its beautiful results by following rules that aren't immediately obvious, often leading to major misinterpretations.
Surprise #1: Cluster Sizes Are Meaningless
Your first t-SNE plot finishes, and you see a few distinct clusters. One is a tight little ball, and another is a big, sprawling shape. The immediate, intuitive assumption is that the big cluster contains more data points or is more varied. This is the first and most common trap. The size of clusters in a t-SNE plot means nothing. The algorithm is designed to spread points out to avoid overlap, meaning it naturally expands dense clusters and contracts sparse ones to make everything visible. A tiny, dense group and a large, sparse group from your original data might end up looking exactly the same size on the final plot. Treat clusters as groups, but ignore their relative size and density entirely.
Surprise #2: The Space Between Clusters Is Also Meaningless
The next logical step is to measure the distance between those clusters. If cluster A is far from cluster B but close to cluster C, that must mean A and B are very different, right? Wrong again. The global distances between separated clusters in a t-SNE plot are not reliable. The algorithm's primary focus is on preserving local structure—keeping immediate neighbors together. It gives up on accurately representing larger, global distances to achieve this. Two clusters that appear far apart might not be any more different than two clusters that are close. Think of it as organizing books by author on a shelf (local structure) but not necessarily arranging the authors alphabetically in the library (global structure).
Surprise #3: 'Perplexity' Completely Changes the Picture
Unlike some plug-and-play algorithms, t-SNE has a crucial setting called “perplexity.” In simple terms, perplexity is your guess at how many neighbors each point has. This single parameter, typically set between 5 and 50, dramatically influences the final visualization. A low perplexity value tells the algorithm to focus only on the most immediate neighbors, which can result in a plot that looks like a fragmented collection of tiny, meaningless clumps. A high perplexity value encourages the algorithm to consider broader, more global patterns. Because the output is so sensitive to this setting, running t-SNE with just one perplexity value is a common mistake; you have to experiment to see which setting reveals the most stable and logical structure for your specific dataset.
So, What Is It Good For?
After learning that cluster sizes and the distances between them are misleading, it’s easy to feel like t-SNE is useless. But that's not the case. It is an unparalleled tool for exploratory data analysis and generating hypotheses. A t-SNE plot can reveal underlying patterns you didn't know existed, showing you that your data does, in fact, have distinct groups. The key is to use it as a starting point, not a final destination. Once t-SNE shows you that clusters exist, you can use other methods, like a dedicated clustering algorithm, to validate and define those groups. It’s also wise to pre-process your data, often using a technique like PCA to reduce initial dimensions, which can speed up t-SNE and reduce noise.
