A variational autoencoder (VAE) is a generative model that extends the autoencoder with a probabilistic view of the latent space. The encoder does not emit a single point but the parameters of a distribution – a mean and a variance – from which the latent vector is only then sampled at random. The loss function has two terms: the reconstruction error and the Kullback-Leibler divergence, which pushes the resulting distribution towards a standard normal distribution. This makes the latent space continuous and densely occupied, so any point can be sampled from it and the decoder will turn it into a meaningful output – which is not true of an ordinary autoencoder, because that one leaves “empty spots” between the training points. The key technical trick is reparameterization, which moves the randomness into a separate noise input and thus allows the gradient to be pushed through the sampling step. A VAE produces somewhat blurrier outputs than other generative approaches, but it trains stably and offers a well-structured latent space. Today it often forms part of diffusion models, where it compresses the image into the latent form in which the diffusion itself takes place.
Recall the editor who boils an article down to a text message. An ordinary autoencoder files each article at one specific spot in the archive – and the space between the drawers stays empty, so if you open a drawer in between, nonsense falls out. A VAE deliberately blurs the archive: instead of storing an article at a point, it spreads it over a small region, and those regions touch one another. The whole archive is then “inhabited”. You can reach in anywhere, even halfway between two articles, and get something new but still meaningful – a text sitting exactly midway between a review and a reportage, say.