The Different Types of Neural Networks in AI and What They Are Good For

To get a better grasp of the various types of neural network and what each is actually used for, let us look together at some simple examples and at the differences in how they are conceived and how they approach processing data.

Multilayer perceptrons (MLP)

Multilayer perceptrons (MLPs) are the basic form of neural network. They are used for classification tasks such as recognising handwritten digits, where each input image is transformed through a series of hidden layers into an output layer that decides which digit the image represents.

How MLPs recognise handwritten digits

Every image of a handwritten digit is converted into a set of numerical values representing the intensity of the individual pixels. Those values form the input layer of the neural network. MLPs contain one or more hidden layers, which allow the network to learn more complex patterns in the data. The larger the model, the more hidden layers it needs. Each neuron in a hidden layer computes a weighted sum of its inputs and applies an activation function to it. That process lets the network capture non-linear relationships between inputs and outputs. At the end of the MLP is an output layer containing one neuron for each possible category (for digit recognition, 10 neurons for the numbers 0 to 9). The output layer turns the results of the hidden layers into probabilities indicating how likely it is that the input image represents each digit. During training the MLP learns to optimise the weights of the connections between neurons so that the error between the predicted and the actual output is as small as possible. This is done with the backpropagation algorithm together with an optimisation method such as stochastic gradient descent.

Picture an MLP as a factory where the input image is the raw material and the finished product is the recognised digit. The raw material passes through various processing lines (the hidden layers), each of which adds a certain value or transformation based on the previous steps. At the end of the process an inspection station (the output layer) determines which product has been made on the basis of the values added in the earlier stages. Learning and optimising the process then amounts to adjusting the production line so that the error between the expected and the actual product is as small as possible.

Long short-term memory networks (LSTM)

Long short-term memory networks (LSTMs) are an extension of RNNs that solves the problem of long-range dependencies. They are used, for instance, in machine translation, where holding the context of a whole paragraph matters for translating the individual sentences correctly. Thanks to their unique structure, LSTMs are able to retain information for longer, which makes them ideal for applications where understanding context across an entire sequence is important.

How LSTMs work

The forget gate: LSTM modules contain what are called forget gates, which decide which information from the past is no longer needed and can be removed from the cell state. That helps the model shed stale information which is not relevant to current or future computations. The input gate decides which new information is relevant and should be added to the cell state. That allows an LSTM to update its state with new, important information. The cell state works as a kind of “long-term memory” for the network, holding information for long-term use. Thanks to the gates that control the flow of information, this state can retain important information effectively over a long period. Finally, the output gate controls which information from the cell state will be used to produce the network’s current output. That lets an LSTM generate relevant outputs on the basis of both long-retained and recently acquired information.

Do you want to learn a great deal more about AI tools? Exclusively and before anyone else? Join my community on Patreon. A regular dose of tips, tricks and how-tos is waiting for you…

This comes in handy in machine translation, for example, where holding the context of a whole paragraph or even a whole document matters for translating the individual sentences correctly. LSTMs make that possible thanks to their ability to retain relevant information about the context of a text over a long period, which improves translation quality.

It is like reading a complex story and trying to remember the key details so that you follow how it develops. Just as you remember the important information and let irrelevant details go, LSTM networks “decide” what to keep for future reference and what is no longer needed. This ability to retain and correctly use relevant information makes LSTMs especially useful for applications where context and long-term memory are crucial.

Recurrent neural networks (RNNs)

Recurrent neural networks (RNNs) are ideal for sequential data such as text or time series. They can, for example, generate a continuation of a given text or predict share prices from historical data, because they retain information from earlier inputs for use in later computations.

How RNNs handle sequential data

Unlike traditional neural networks, which process inputs independently, RNNs process data sequentially, taking into account not only the current input but also the information gained from previous steps. That allows the network to “remember” information from the past and use it in the current computation. In RNNs every node (neuron) in the sequential layer has a “state” which is updated at every step of the sequential processing. That state is handed from one step to the next, which lets the network retain information about previous inputs. For training RNNs a variant of backpropagation called “backpropagation through time” is used, which updates the weights on the basis of errors produced at different time steps.

RNNs can be trained on large text corpora so that they learn the patterns of a language, and then used to generate new text that follows the language structures they have learned. After training on a collection of short stories, for example, an RNN can generate new stories.

In economics or finance, RNNs can predict future trends in share prices from historical data. The network learns from temporal patterns and can identify potential future price movements.

Think of an RNN as a group of theatre actors performing a long play, where each actor has to remember not only their own lines but also what happened in the earlier scenes so that they can react correctly as the story develops. In the same way an RNN “remembers” earlier inputs and their context, which lets it respond effectively to new information and produce meaningful outputs in a continuous stream of data. This model of “memory” and “continuity” in RNNs allows very accurate modelling and generation of sequential data, which is key for tasks where context and the order of information matter.

Convolutional neural networks (CNNs)

Convolutional neural networks (CNNs) specialise in processing images. They are used for recognising objects in pictures or video, where the convolutional layers automatically and efficiently identify and exploit spatial hierarchies of features (edges, shapes, patterns and so on).

How CNNs recognise objects

The foundation of a CNN is its convolutional layers, which apply filters (or kernels) to the input image. These filters automatically detect important features such as edges, shapes and textures. Each filter produces its own “feature map” representing the presence of a given feature in different regions of the image. The convolutional layers are often followed by pooling (downsampling) layers, which reduce the dimensionality of the feature maps and make them more robust to small shifts and distortions in the image. The most common is max pooling, which takes the maximum value from each small patch of a feature map. At the end of a CNN architecture are the fully connected layers, which take the flattened outputs of the previous layers and perform the final classification or regression. This part of the network interprets the combined features detected in the image and decides whether particular objects are present. A CNN learns through backpropagation and optimisation algorithms such as stochastic gradient descent. During training the weights of the filters and of the neurons in the layers are adjusted so as to minimise the error between the predicted and the actual image labels.

Imagine a CNN as a team of forensic detectives examining a scene. Each detective (a filter in a convolutional layer) specialises in a particular type of evidence: one looks for fingerprints (edges), another for hairs (textures) and so on. Once the evidence has been gathered (the feature maps), the team boils down its findings (the pooling layers) so as to concentrate only on the most important ones. Finally they put all the information together (the fully connected layers) and decide who the culprit was or what they looked like (classifying the object).

This approach allows a CNN to recognise and classify objects in pictures or video automatically and efficiently, by learning to recognise and combine key visual features.

Transformers

Transformers are a breakthrough in natural language processing thanks to their ability to capture long-range dependencies between words in a text without having to process the data sequentially. They are used for advanced tasks such as generating text, translating languages or summarising documents, where the model “understands” the overall context and can produce coherent, relevant answers.

How transformers work

Unlike RNNs and LSTMs, which process data sequentially (one element after another), transformers allow whole sequences of data to be processed in parallel. That brings a significant improvement in efficiency and in the time needed for training and inference. At the heart of transformers is the attention mechanism, which lets the model “concentrate” on the relevant parts of the input sequence as it generates each word of the output. In this way the model can capture long-range dependencies and context regardless of how far apart the words are in the text. Many transformers use an encoder-decoder structure, where the encoder processes the input sequence and the decoder generates the output. That structure is particularly useful for tasks such as machine translation, where the model needs to understand the whole input text before generating any part of the output.

Transformers can generate coherent, relevant sequences of text, for example continuations of stories, answers to questions or articles on a given topic. They can also summarise long documents into short, coherent summaries while preserving the key information and the context of the original text. The best known examples are ChatGPT from OpenAI, Claude from Anthropic and the text GPT models from Google, Bard, Gemini and others.

Thanks to their ability to capture nuances of meaning and context across different languages, transformers have become the foundation of the most modern machine translation systems, whether that is Google Translate or the German translator DeepL.

If we carry on the detective analogy from the CNN section, transformers are like a team of crime analysts working on a complicated case in which every member of the team has to weigh up all the evidence (the texts) at the same time, regardless of when or where it was found. Instead of going through the individual witness statements one after another, the team can identify the connections and the key moments of the whole case, which can lead to a faster and more accurate solution.

Transformers therefore provide an extraordinarily powerful tool for various NLP applications by combining the efficiency of parallel processing with an advanced attention mechanism for capturing long-range dependencies and context.