Regularization is the collective name for techniques that limit the effective capacity of a model and thereby improve its ability to generalize to new data. Its purpose is to suppress overfitting, the state in which a model memorizes the training samples along with their noise. The most common form is adding a penalty term to the loss function. The L2 penalty, known as weight decay, adds the sum of squared weights and pushes them evenly towards zero, penalizing models that rely on a few extremely strong connections. The L1 penalty adds the sum of absolute values and zeroes many weights outright, so it doubles as feature selection. Alongside explicit penalties there is a range of implicit mechanisms: dropout switches neurons off at random, early stopping ends training the moment the validation error starts to rise, data augmentation expands the training set with transformations of itself, and label smoothing prevents overconfident predictions. Even the batch size and the noise in stochastic gradients have a regularizing effect. Finding the right amount is always a matter of balance: regularize too hard and you end up underfitting.
Regularization is like the advice you give a student before an exam: “Do not recite the definitions from the three books you read – say it in your own words, simply.” You are forcing them to give up the detail and hold on to the essentials, because the essentials also work on a question that was not in the books. In practice you might forbid them more than ten technical terms, cover half their notes now and then, or stop them before they manage to memorize anything. And as with any advice, it can be taken too far: a student forbidden from everything ends up saying nothing at all.