You can picture diffusion models as an artist who learns to create by first destroying a perfect image, gradually adding noise until nothing remains but random grain. Then it learns to reverse the process – reconstructing the original image from random noise step by step. Once it has learned this, it can generate from any new noise an entirely original yet realistic image, piece of music or text resembling the data it trained on.
Diffusion models are a class of generative probabilistic model in machine learning. Their principle rests on progressively destroying the data structure in the training data through a forward diffusion process that systematically adds Gaussian noise. The model then learns to reverse this process by means of a reverse process, in which a neural network, typically of the U-Net architecture, is trained to remove noise and reconstruct the original data from its noisy version.
New data samples are generated by feeding pure noise to the model’s input and having the model apply the learned reverse process iteratively, thereby shaping a coherent output step by step. This mechanism makes it possible to generate high-quality, diverse samples resembling the training dataset. Diffusion models are used above all in image, audio and video generation tasks, where they achieve state-of-the-art results in metrics of quality and fidelity. Their name derives from the concept of diffusion in thermodynamics, which describes the gradual mixing of particles.