Pooling is an operation that reduces the spatial resolution of a feature map in convolutional networks by aggregating values within small windows. The most widespread are max pooling, which keeps only the maximum value from each window, and average pooling, which computes the mean. The operation has no learnable parameters. There are three benefits. It reduces the number of values the following layers must process, and with it the computational and memory cost. It enlarges the receptive field of neurons in deeper layers, so the network gradually sees a larger part of the image. And it introduces local invariance to translation – shifting an edge by one pixel does not change the maximum, which is desirable, because we are usually interested in the presence of a feature rather than its pixel-exact position. That very property, however, is a problem in segmentation, since it loses the information needed for outlines; hence in the U-Net decoder it is combined with skip connections. Modern architectures often replace pooling with strided convolution, which performs the reduction in a learned way, and at the end of the network global average pooling is used before the classification layer.
It is like writing down only the single most important sentence from each page of a long report. The text shrinks to a tenth, becomes far easier to work with, and the main idea survives. At the same time it stops mattering whether that sentence stood in the first or the second paragraph – which is often an advantage, because what interests you is that the topic came up, not exactly where. The price is obvious: the details are irretrievably gone. If somebody needs a quotation pinned to the exact line, you cannot afford this shortcut.