t-SNE and UMAP are non-linear dimensionality reduction methods intended primarily for visualising high-dimensional data in two or three dimensions. Unlike PCA, which tries to preserve global variance, they concentrate on preserving local structure – on keeping points that were close in the original space close on the resulting map. t-SNE converts distances into neighbourhood probabilities and minimises the Kullback-Leibler divergence between the distributions in the original and target spaces, using a heavy-tailed Student’s t-distribution in the target space so that clusters push apart. UMAP builds on Riemannian geometry and the topology of the neighbourhood graph; it is orders of magnitude faster, scales to millions of points and preserves more global structure as well. A crucial caveat for interpretation: cluster sizes and the distances between them have no reliable meaning, the result depends strongly on the perplexity or number-of-neighbours setting, and the methods are not well suited to projecting new points. They are thus an exploratory tool rather than an input to another model – commonly used to display clusters of embeddings.
Imagine having to flatten a globe onto a sheet of paper. It cannot be done without distortion – something has to be sacrificed. t-SNE and UMAP choose to sacrifice large distances and preserve neighbourhoods: they guarantee that cities next to each other stay next to each other on the map, but they care nothing about how far London is from Tokyo. On such a map you can reliably say “here is a clear cluster of countries”, but you must never read off that one cluster is twice as far away as another. It is a tool for the eye, not for the ruler.