A random forest is an ensemble model that combines the predictions of many decision trees trained on different random variants of the same dataset. It builds on the principle of bagging: each tree learns from a bootstrap sample, that is, a random selection with replacement, and their results are merged by voting in classification or averaging in regression. A random forest adds a second source of diversity – at each node split only a random subset of features is offered, which prevents all the trees from building on the same dominant variable and thereby resembling one another. The theoretical core is the fact that averaging many uncorrelated models reduces variance without increasing bias; the individual trees may therefore be deep and overfitted while the whole still generalises excellently. Practical advantages are minimal hyperparameter tuning, resistance to outliers, easy parallelisation, and a free error estimate on the samples left out of the bootstrap. The price is loss of transparency and higher memory use. On tabular data it remains, alongside gradient boosting, among the strongest methods there are.
It is the transfer of the old insight that a group often decides better than an individual. You assemble a hundred doctors, but give each only part of the medical file and moreover let each look at only some of the examinations – one sees blood tests, another the scans, a third the case history. Each on his own occasionally gets it badly wrong. But when all hundred vote on a diagnosis, their individual errors cancel out and the result is better than the best of them. The price is obvious: from one doctor you learn the reason, from a hundred voters only the outcome.