The transformer architecture is a type of deep learning model originally designed for natural language processing tasks. Its basic building block is the attention mechanism, which allows the model to weight the importance of different parts of the input data when producing its output. Unlike the recurrent and convolutional neural networks used previously, a transformer processes whole input sequences at once, which permits substantial parallelisation of computation and more efficient training on large data. The architecture consists of an encoder part and a decoder part, both containing attention layers and feed-forward neural networks.
Imagine translating a long sentence. Rather than looking at each word in turn one after another, your brain naturally weighs which words in the sentence matter most to the meaning and how they relate to one another. That is exactly how the transformer architecture works. It is a clever system able to “look” at all the data at once and decide for itself which parts to give greater “attention” to, in order to accomplish the given task as well as possible, whether that is translating text, answering questions or generating images.