Dimensionality reduction is the process of transforming data from a high-dimensional space into a space of lower dimension in such a way that meaningful information is preserved. In machine learning it is a set of techniques that reduce the number of input variables, or features, in a dataset. The aim is to simplify the model, lower computational cost, remove redundant or irrelevant features and forestall the curse of dimensionality, which can degrade model performance. The methods divide into feature selection, where a subset of the original features is chosen, and feature extraction, which creates new, fewer features as combinations of the original ones.
Imagine a table of data about houses with a hundred columns – from the colour of the facade through the type of door handles to the age of the roof covering. Working out anything useful from such a mass of data is hard, and many of the columns are pointless. Dimensionality reduction is like having an expert look at that table and say: “To determine a house price we really only need to know its floor area, its location and its overall condition.” So instead of a hundred columns you suddenly need only the three most important ones, which are far easier to work with – and yet you have not lost the crucial information.