Overview
Artificial Neural Networks are the computational models inspired by the human brain. Many of the recent advancements have been made in the field of Artificial Intelligence, including Voice Recognition, Image Recognition, Robotics using Artificial Neural Networks.
These biological methods of computing is considered to be the next major advancement in the Computing Industry.
What is Neural Network?
The term ‘Neural’ is derived from the human (animal) nervous system’s basic functional unit ‘neuron’ or nerve cells which are present in the brain and other parts of the human (animal) body.
Parts of Neuron and their Functions
The typical nerve cell of human brain comprises of four parts -
Image Source - cs231n.github.io
Dendrite
It receives signals from other neurons.
Soma (cell body)
It sums all the incoming signals to generate input.
Axon
When the sum reaches a threshold value, neuron fires and the signal travels down the axon to the other neurons.
Synapses
The point of interconnection of one neuron with other neurons. The amount of signal transmitted depend upon the strength (synaptic weights) of the connections.
The connections can be inhibitory (decreasing strength) or excitatory (increasing strength) in nature.
So, neural network, in general, is a highly interconnected network of billions of neuron with trillion of interconnections between them.
Difference Between Computer and Human Brain
What is Artificial Neural Network(ANN)?
Artificial Neural Networks are the biologically inspired simulations performed on the computer to perform certain specific tasks like clustering, classification, pattern recognition, etc.
Artificial Neural Networks, in general - is a biologically inspired network of artificial neurons configured to perform specific tasks.
Difference Between Artificial Neural Networks (ANN) and Biological Neural Networks (BNN)
Characteristics
|
Artificial Neural Network
|
Biological(Real) Neural Network
|
Speed
|
Faster in processing information. Response time is in nanoseconds.
|
Slower in processing information. The response time is in milliseconds.
|
Processing
|
Serial processing.
|
Massively parallel processing.
|
Size & Complexity
|
Less size & complexity. It does not perform complex pattern recognition tasks.
|
Highly complex and dense network of interconnected neurons containing neurons of the order of 1011 with 1015 of interconnections.
|
Storage
|
Information storage is replaceable means new data can be added by deleting an old one.
|
Highly complex and dense network of interconnected neurons containing neurons of the order of 1011 with 1015 of interconnections.
|
Fault tolerance
|
Fault intolerant. Information once corrupted cannot be retrieved in case of failure of the system.
|
Information storage is adaptable means new information is added by adjusting the interconnection strengths without destroying old information
|
Control Mechanism
|
There is a control unit for controlling computing activities
|
No specific control mechanism external to the computing task.
|
Similarity of Artificial Neural Networks with Biological Neural Network
Neural Networks resemble the human brain in the following two ways -
- A neural network acquires knowledge through learning.
- A neural network's knowledge is stored within inter-neuron connection strengths known as synaptic weights.
VON NEUMANN ARCHITECTURE BASED COMPUTING | ANN BASED COMPUTING |
Serial processing - processing instruction and problem rule one at time (sequential)
|
Parallel processing - several processors perform simultaneously (multitasking)
|
Function logically with set of if & else rules - rule-based approach
|
Function by learning pattern from given input (image, text or video, etc.)
|
Programmable by higher level languages such as C, Java, C++, etc.
|
ANN is in essence program themselves.
|
Requires either big or error prone parallel processors
|
Use of application specific multi-chips.
|
Comparison Between Artificial Neural Network (ANN) With Biological Neural Network (BNN)
The dendrites in the Biological Neural Network are analogous to the weighted inputs based on their synaptic interconnection in the Artificial Neural Network.
The cell body is comparable to the artificial neuron unit in the Artificial Neural Network which also comprises of summation and threshold unit.
Axon carries output that is analogous to the output unit in case of Artificial Neural Network. So, ANN is modeled using the working of basic biological neurons.
How Does Artificial Neural Network Works?
Artificial Neural Networks can be viewed as weighted directed graphs in which artificial neurons are nodes, and directed edges with weights are connections between neuron outputs and neuron inputs.
The Artificial Neural Network receives information from the external world in the form of pattern and image in vector form. These inputs are mathematically designated by the notation x(n) for n number of inputs.
Each input is multiplied by its corresponding weights. Weights are the information used by the neural network to solve a problem. Typically weight represents the strength of the interconnection between neurons inside the Neural Network.
The weighted inputs are all summed up inside computing unit (artificial neuron). In case the weighted sum is zero, bias is added to make the output not- zero or to scale up the system response. Bias has the weight and input always equal to ‘1'.
The sum corresponds to any numerical value ranging from 0 to infinity. To limit the response to arrive at the desired value, the threshold value is set up. For this, the sum is passed through activation function.
The activation function is set to the transfer function used to get the desired output. There are linear as well as the nonlinear activation function.
Some of the commonly used activation function is - binary, sigmoidal (linear) and tan hyperbolic sigmoidal functions(nonlinear).
- Binary - The output has only two values either 0 and 1. For this, the threshold value is set up. If the net weighted input is greater than 1, an output is assumed one otherwise zero.
- Sigmoidal Hyperbolic - This function has ‘S’ shaped curve. Here tan hyperbolic function is used to approximate output from net input. The function is defined as - f (x) = (1/1+ exp(-𝝈x)) where 𝝈 - steepness parameter.
You May also Love to Read Overview of Artificial Intelligence & Role of Natural Language Processing in Big Data
Artificial Neural Network Architecture
A typical Neural Network contains a large number of artificial neurons called units arranged in a series of layers.
In typical Artificial Neural Network, comprises different layers -
- Input layer - It contains those units (Artificial Neurons) which receive input from the outside world on which network will learn, recognize about or otherwise process.
- Output layer - It contains units that respond to the information about how it's learned any task.
- Hidden layer - These units are in between input and output layers. The job of hidden layer is to transform the input into something that output unit can use in some way.
Most Neural Networks are fully connected that means to say each hidden neuron is fully linked to the every neuron in its previous layer(input) and to the next layer (output) layer.
Types of Neural Networks in Artificial Intelligence
Parameter
|
Types
|
Description
|
Based on connection pattern
|
FeedForward, Recurrent
|
|
Based on the number of hidden layer
|
Single layer, Multi-Layer
|
|
Based on nature of weights
|
Fixed, Adaptive
|
|
Based on Memory unit
|
Static, Dynamic
|
|
Neural Network Architecture Types
Perceptron Model in Neural Networks
Neural Network is having two input units and one output units with no hidden layers. These are also known as ‘single layer perceptrons.
Radial Basis Function Neural Network
These networks are similar to the feed forward Neural Network except radial basis function is used as activation function of these neurons.
Multilayer Perceptron Neural Network
These networks use more than one hidden layer of neurons, unlike single layer perceptron. These are also known as Deep Feedforward Neural Networks.
Recurrent Neural Network
Type of Neural Network in which hidden layer neurons has self-connections. Recurrent Neural Networkspossess memory. At any instance, hidden layer neuron receives activation from the lower layer as well as it previous activation value.
Long Short-Term Memory Neural Network (LSTM)
Type of Neural Network in which memory cell is incorporated inside hidden layer neurons is called LSTM network.
Hopfield Network
A fully interconnected network of neurons in which each neuron is connected to every other neuron. The network is trained with input pattern by setting a value of neurons to the desired pattern. Then its weights are computed. The weights are not changed. Once trained for one or more patterns, the network will converge to the learned patterns. It is different from other Neural Networks.
Boltzmann Machine Neural Network
These networks are similar to Hopfield network except some neurons are input, while other are hidden in nature. The weights are initialized randomly and learn through back propagation algorithm.
Convolutional Neural Network
Get a complete overview of Convolutional Neural Networks through our blog Log Analytics with Machine Learning and Deep Learning.
Modular Neural Network
It is the combined structure of different types of the neural network like multilayer perceptron, Hopfield Network, Recurrent Neural Network, etc. which are incorporated as a single module into the network to perform independent subtask of whole complete Neural Networks.
Physical Neural Network
In this type of Artificial Neural Network, electrically adjustable resistance material is used to emulate the function of synapse instead of software simulations performed in the neural network.
Learning Techniques in Artificial Neural Networks
The neural network learns by adjusting its weights and bias (threshold) iteratively to yield desired output. These are also called free parameters. For learning to take place, the Neural Network is trained first. The training is performed using a defined set of rules also known as the learning algorithm.
Continue Reading: XenonStack/Blog
No comments:
Post a Comment