Types of ANNs
There are many types of ANNs, but they can be broadly classified into two categories: feedforward and recurrent.
Feedforward ANNs are the simplest and most common type of ANNs. They have a fixed number of layers and connections that do not form cycles. The information flows in one direction, from the input layer to the output layer. Feedforward ANNs can learn to approximate any function, given enough hidden units and training data. Some examples of feedforward ANNs are perceptron, multilayer perceptron (MLPs), and convolutional neural networks (CNNs).
Recurrent ANNs are more complex and powerful than feedforward ANNs. They have connections that form cycles, allowing them to store and process temporal information. The information flows in both directions, from the input layer to the output layer and back. Recurrent ANNs can learn to model sequential data, such as natural language, speech, and video. Some examples of recurrent ANNs are Hopfield Networks, long short-term memory (LSTM), and gated recurrent units (GRU).
Following diagram shows the schematic representation of Recurrent ANNs
The architecture of an ANN refers to the number, size, and arrangement of its layers and units. The architecture determines the complexity and capacity of the ANN, as well as its suitability for different tasks. There are many possible architectures of ANNs, but some of the most common ones are:
Perceptron: A perceptron is the simplest feedforward ANN. It consists of a single layer of binary units that are connected to the input and output. A perceptron can learn to perform linear classification, such as AND, OR, and XOR operations.
Following image represents Perceptron Architecture
Multilayer Perceptron (MLP): An MLP is a feedforward ANN that consists of multiple layers of units, usually with a nonlinear activation function like ReLU (Rectified Linear Unit). ReLU is given by f(x) = max(0,x). An MLP can learn to perform nonlinear classification and regression, such as handwritten digit recognition and function approximation etc.
Convolution Neural Network (CNN): A CNN is a feedforward ANN that consists of multiple layers of units, some of which are convolutional and some of which are fully connected. A convolutional layer applies a set of filters to the input, producing a set of feature maps. A fully connected layer connects all the units in one layer to all the units in the next layer. A CNN can learn to perform image recognition, object detection, and segmentation. The CNN will be discussed in detailed later
For the time being, consider the following diagram, which schematically represents Convolution Neural Network
Hopfield network: A hopfield network is a recurrent ANN that consists of a single layer of binary units that are fully connected to each other. A hopfield network can store and retrieve patterns, such as images and words, as stable states of the network.
Below is the generalized structure of Hopfield Neural Network.
The inputs to the neural network can either be -1 or 1 and the outputs will also be either one of those two values.
Though LSTM will be discussed in details, still following diagram gives a brief of the LSTM structure
