Artificial Neural Network (ANN) - An Introduction

In this blogpost, you are going to learn what is Artificial Neural Network (ANN)? How did it evolve? and why it is important to learn about ANN.

Let's Start

Have you ever wondered how your brain works? How do you recognize faces, learn languages, play games, or solve problems? Your brain is an amazing organ that can perform these tasks and many more, thanks to its billions of cells called neurons. Neurons are the basic units of information processing in your brain. They communicate with each other through electrical and chemical signals, forming complex networks that enable you to think, feel, and act.

But what if we could create artificial systems that mimic the structure and function of the brain? What if we could build machines that can learn from data and perform tasks that are difficult or impossible for humans? This is the goal and challenge of artificial intelligence (AI) and machine learning (ML), two fields that aim to create intelligent agents that can understand and interact with the world.

What is Artificial Neural Network?

Artificial neural network (ANN) is a computational model that consists of a collection of artificial neurons, also called nodes or units, that are connected by weighted links, also called edges or connections. Each node can receive inputs from other nodes or external sources, and produce an output based on a mathematical function, also called an activation function. The output of a node can then be transmitted to other nodes or external destinations, forming a network of information flow.

The structure and behaviour of an ANN can be described by three main aspects: the network architecture, the network parameters, and the network function. The network architecture refers to the number and arrangement of nodes and links in the network, such as the number of layers, the number of nodes per layer, and the pattern of connections between layers. The network parameters refer to the numerical values of the weights and biases associated with each link and node, respectively. The network function refers to the mathematical operation that each node performs on its inputs, such as linear, sigmoid, tanh, relu, softmax, etc.

The network architecture and parameters determine the capacity and complexity of the network, while the network function determines the nonlinearity and flexibility of the network. Together, they define the input-output relationship of the network, which can be represented by a mathematical equation or a graphical diagram. For example, the following figure shows a simple ANN with three input nodes, two hidden nodes, and one output node, along with its corresponding equation:

The equation for the output node is:

How Artificial Neural Network has been inspired by Human Brain?

You must be wondering that where did the idea come from?

To answer this question, we need to look at the most powerful and complex computer in the world: the human brain. The human brain is made up of billions of tiny cells called neurons. Neurons are like the building blocks of the brain. They communicate with each other by sending and receiving electrical signals through connections called synapses. The signals carry information that help us think, feel, and act.

Scientists have been fascinated by the brain for a long time, and they wanted to understand how it works and how it can be imitated. In the 1940s, two scientists named Norman Wiener and Warren McCulloch came up with the idea of using mathematics and logic to model the brain’s behavior. They proposed that the brain can be seen as a network of simple units that can process information and make decisions based on rules. They called this network a neural network, and they hoped that it could help them solve problems that are too hard for normal computers.

However, their idea was not very practical, because they did not have the technology to implement it. It was not until the 1950s that another scientist named Frank Rosenblatt invented a device that could actually simulate a neural network. He called it a perceptron, and it was the first real attempt to mimic human intelligence. A perceptron was a machine that had a layer of input units, a layer of output units, and a set of weights that determined how the inputs affected the outputs. The perceptron could learn from examples by adjusting the weights based on the feedback it received. For example, if the perceptron was given an image of a cat and told that it was a cat, it would increase the weights that led to the correct output and decrease the weights that led to the wrong output. This way, the perceptron could learn to recognize cats and other objects.

However, the perceptron had a major limitation: it could only learn linear functions, which are functions that can be drawn as straight lines. This means that the perceptron could not learn more complex functions, such as those that involve curves or twists. For example, the perceptron could not learn to recognize the letter X, because it is not a linear function. This limitation was discovered by two mathematicians named Marvin Minsky and Seymour Papert, who wrote a book that criticized the perceptron and discouraged further research on neural networks.

But the story did not end there. In the 1980s, a new generation of researchers revived the interest in neural networks, and they came up with a solution to the perceptron’s problem. They realized that by adding more layers of units between the input and the output, they could create a network that could learn any function, no matter how complex. They called this network a multilayer perceptron, or a feedforward neural network. A feedforward neural network is a network that has a fixed number of layers and connections that do not form cycles. The information flows in one direction, from the input layer to the output layer. A feedforward neural network can learn to approximate any function, given enough hidden units and training data. Some examples of feedforward neural networks are multilayer perceptrons, convolutional neural networks, and deep neural networks.

But feedforward neural networks are not the only type of neural networks. There is another type of neural networks that are more complex and powerful than feedforward neural networks. They are called recurrent neural networks, and they have connections that form cycles, allowing them to store and process temporal information. The information flows in both directions, from the input layer to the output layer and back. Recurrent neural networks can learn to model sequential data, such as natural language, speech, and video. Some examples of recurrent neural networks are hopfield networks, long short-term memory, and gated recurrent units.

As you can see, artificial neural networks are inspired by the human brain, but they are not exactly the same. They are simplified and abstracted versions of the brain, and they have their own advantages and disadvantages. Artificial neural networks are faster and more accurate than the brain in some tasks, but they are slower and more prone to errors in other tasks. Artificial neural networks can learn from data and perform tasks that are usually done by humans, but they cannot explain how they do it or why they do it. Artificial neural networks are powerful and useful tools for artificial intelligence, but they are not the ultimate solution for intelligence. They are still far from achieving the level of complexity and creativity that the human brain possesses.

Why is Artificial Neural Network important?

Artificial neural network is important because it can learn from data and perform tasks that are difficult or impossible for conventional algorithms or human experts. ANN can learn from data by adjusting its parameters based on the feedback or error it receives from the desired output or goal. This process is called training or learning, and it can be done using various methods and techniques, such as gradient descent, backpropagation, stochastic gradient descent, etc.

By learning from data, ANN can discover patterns, features, and relationships that are hidden or unknown in the data, and use them to make predictions, classifications, recommendations, or decisions. For example, ANN can learn to recognize faces, objects, or emotions from images, to translate languages, summarize texts, or generate captions from natural language, to play games, control robots, or drive cars from reinforcement signals, and to create music, art, or stories from generative models.

ANN can perform these tasks better than conventional algorithms or human experts because it can handle large and complex data, deal with uncertainty and noise, adapt to changing environments, and generalize to new situations. ANN can also overcome some of the limitations and biases of human cognition, such as memory, attention, or emotion. For these reasons, ANN is widely used and applied in various domains and fields, such as computer vision, natural language processing, speech recognition, recommender systems, healthcare, education, finance, etc.

In the next blogpost, we will discuss components, structures and types of ANN.