What is Deep Learning?

Deep learning is a method in artificial intelligence (AI) that teaches computers to

process data in a way that is inspired by the human brain. Deep learning models
can recognize complex patterns in pictures, text, sounds, and other data to produce
accurate insights and predictions. You can use deep learning methods to automate
tasks that typically require human intelligence, such as describing images or
transcribing a sound file into text.
Why is deep learning important?
Artificial intelligence (AI) attempts to train computers to think and learn as
humans do. Deep learning technology drives many AI applications used in
everyday products, such as the following:

 Digital assistants
 Voice-activated television remotes
 Fraud detection
 Automatic facial recognition
It is also a critical component of emerging technologies such as self-driving cars,
virtual reality, and more.
Deep learning models are computer files that data scientists have trained to
perform tasks using an algorithm or a predefined set of steps. Businesses use deep
learning models to analyze data and make predictions in various applications.
What are the uses of deep learning?
Deep learning has several use cases in automotive, aerospace, manufacturing,
electronics, medical research, and other fields. These are some examples of deep
learning:

 Self-driving cars use deep learning models to automatically detect road signs and
pedestrians.
 Defense systems use deep learning to automatically flag areas of interest in satellite
images.
 Medical image analysis uses deep learning to automatically detect cancer cells for
medical diagnosis.
 Factories use deep learning applications to automatically detect when people or
objects are within an unsafe distance of machines.
 You can group these various use cases of deep learning into four broad categories
—computer vision, speech recognition, natural language processing (NLP), and
recommendation engines.

Computer vision

Computer vision is the computer's ability to extract information and insights from
images and videos. Computers can use deep learning techniques to comprehend
images in the same way that humans do. Computer vision has several applications,
such as the following:

 Content moderation to automatically remove unsafe or inappropriate content from

image and video archives
 Facial recognition to identify faces and recognize attributes like open eyes, glasses,
and facial hair
 Image classification to identify brand logos, clothing, safety gear, and other image
details

Speech recognition

Deep learning models can analyze human speech despite varying speech patterns,
pitch, tone, language, and accent. Virtual assistants such as Amazon Alexa
and automatic transcription software use speech recognition to do the following
tasks:

 Assist call center agents and automatically classify calls.

 Convert clinical conversations into documentation in real time.
 Accurately subtitle videos and meeting recordings for a wider content reach.

Natural language processing

Computers use deep learning algorithms to gather insights and meaning from text
data and documents. This ability to process natural, human-created text has several
use cases, including in these functions:

 Automated virtual agents and chatbots

 Automatic summarization of documents or news articles
 Business intelligence analysis of long-form documents, such as emails and forms
 Indexing of key phrases that indicate sentiment, such as positive and negative
comments on social media

Recommendation engines

Applications can use deep learning methods to track user activity and
develop personalized recommendations. They can analyze the behavior of various
users and help them discover new products or services. For example, many media
and entertainment companies, such as Netflix, Fox, and Peacock, use deep learning
to give personalized video recommendations.
How does deep learning work?
Deep learning algorithms are neural networks that are modeled after the human
brain. For example, a human brain contains millions of interconnected neurons that
work together to learn and process information. Similarly, deep learning neural
networks, or artificial neural networks, are made of many layers of artificial
neurons that work together inside the computer.
Artificial neurons are software modules called nodes, which use mathematical
calculations to process data. Artificial neural networks are deep learning
algorithms that use these nodes to solve complex problems.
What are the components of a deep learning network?
The components of a deep neural network are the following.

Input layer

An artificial neural network has several nodes that input data into it. These nodes
make up the input layer of the system.

Hidden layer

The input layer processes and passes the data to layers further in the neural
network. These hidden layers process information at different levels, adapting their
behavior as they receive new information. Deep learning networks have hundreds
of hidden layers that they can use to analyze a problem from several different
angles.
For example, if you were given an image of an unknown animal that you had to
classify, you would compare it with animals you already know. For example, you
 would look at the shape of its eyes and ears, its size, the number of legs, and its fur
pattern. You would try to identify patterns, such as the following:

 The animal has hooves, so it could be a cow or deer.

 The animal has cat eyes, so it could be some type of wild cat.
The hidden layers in deep neural networks work in the same way. If a deep
learning algorithm is trying to classify an animal image, each of its hidden layers
processes a different feature of the animal and tries to accurately categorize it.

Output layer

The output layer consists of the nodes that output the data. Deep learning models
that output "yes" or "no" answers have only two nodes in the output layer. On the
other hand, those that output a wider range of answers have more nodes.
What is deep learning in the context of machine learning?
Deep learning is a subset of machine learning. Deep learning algorithms emerged
in an attempt to make traditional machine learning techniques more efficient.
Traditional machine learning methods require significant human effort to train the
software. For example, in animal image recognition, you need to do the following:

 Manually label hundreds of thousands of animal images.

 Make the machine learning algorithms process those images.
 Test those algorithms on a set of unknown images.
 Identify why some results are inaccurate.
 Improve the dataset by labeling new images to improve result accuracy.
This process is called supervised learning. In supervised learning, result accuracy
improves only when you have a broad and sufficiently varied dataset. For instance,
the algorithm might accurately identify black cats but not white cats because the
training dataset had more images of black cats. In that case, you would need to
label more white cat images and train the machine learning models once again.
 What are the benefits of deep learning over machine learning?
A deep learning network has the following benefits over traditional machine
learning.

Efficient processing of unstructured data

Machine learning methods find unstructured data, such as text documents,

challenging to process because the training dataset can have infinite variations. On
the other hand, deep learning models can comprehend unstructured data and make
general observations without manual feature extraction. For instance, a neural
network can recognize that these two different input sentences have the same
meaning:

 Can you tell me how to make the payment?

 How do I transfer money?

Hidden relationships and pattern discovery

A deep learning application can analyze large amounts of data more deeply and
reveal new insights for which it might not have been trained. For example,
consider a deep learning model that is trained to analyze consumer purchases. The
model has data only for the items you have already purchased. However, the
artificial neural network can suggest new items that you haven't bought by
comparing your buying patterns to those of other similar customers.

Unsupervised learning

Deep learning models can learn and improve over time based on user behavior.
They do not require large variations of labeled datasets. For example, consider a
neural network that automatically corrects or suggests words by analyzing your
typing behavior. Let's assume it was trained in the English language and can spell-
check English words. However, if you frequently type non-English words, such
as danke, the neural network automatically learns and autocorrects these words too.

Volatile data processing

Volatile datasets have large variations. One example is loan repayment amounts in
a bank. A deep learning neural network can categorize and sort that data as well,
such as by analyzing financial transactions and flagging some of them for fraud
detection.
What are the challenges of deep learning?
As deep learning is a relatively new technology, certain challenges come with its
practical implementation.

Large quantities of high-quality data

Deep learning algorithms give better results when you train them on large amounts
of high-quality data. Outliers or mistakes in your input dataset can significantly
affect the deep learning process. For instance, in our animal image example, the
deep learning model might classify an airplane as a turtle if non-animal images
were accidentally introduced in the dataset.
To avoid such inaccuracies, you must clean and process large amounts of data
before you can train deep learning models. The input data preprocessing requires
large amounts of data storage capacity.

Large processing power

Deep learning algorithms are compute-intensive and require infrastructure with

sufficient compute capacity to properly function. Otherwise, they take a long time
to process results.
What are the benefits of deep learning in the cloud?
Running deep learning algorithms on cloud infrastructure can overcome many of
these challenges. You can use deep learning in the cloud to design, develop, and
train deep learning applications faster.

Speed

You can train deep learning models faster by using clusters of GPUs and CPUs to
perform the complex mathematical operations that your neural networks require.
You can then deploy these models to process large amounts of data and produce
increasingly relevant results.

Scalability

With the wide range of on-demand resources available through the cloud, you can
access virtually unlimited hardware resources to tackle deep learning models of
any size. Your neural networks can take advantage of multiple processors to
seamlessly and efficiently distribute workloads across different processor types and
quantities.
 Difference Between Artificial Intelligence vs Machine Learning
vs Deep Learning



Artificial Intelligence is basically the mechanism to incorporate human

intelligence into machines through a set of rules(algorithm). AI is a combination of
two words: “Artificial” meaning something made by humans or non-natural things
and “Intelligence” meaning the ability to understand or think accordingly. Another
definition could be that “AI is basically the study of training your
machine(computers) to mimic a human brain and its thinking capabilities”.
AI focuses on 3 major aspects(skills): learning, reasoning, and self-
correction to obtain the maximum efficiency possible.
Machine Learning:
Machine Learning is basically the study/process which provides the
system(computer) to learn automatically on its own through experiences it had and
improve accordingly without being explicitly programmed. ML is an application
or subset of AI. ML focuses on the development of programs so that it can access
data to use it for itself. The entire process makes observations on data to identify
the possible patterns being formed and make better future decisions as per the
examples provided to them. The major aim of ML is to allow the systems to
learn by themselves through experience without any kind of human
intervention or assistance.
Deep Learning:
Deep Learning is basically a sub-part of the broader family of Machine Learning
which makes use of Neural Networks(similar to the neurons working in our brain)
to mimic human brain-like behavior. DL algorithms focus on information
processing patterns mechanism to possibly identify the patterns just like our
human brain does and classifies the information accordingly. DL works on larger
sets of data when compared to ML and the prediction mechanism is self-
administered by machines.
Below is a table of differences between Artificial Intelligence, Machine Learning
and Deep Learning:
Artificial Intelligence Machine Learning Deep Learning

AI stands for Artificial ML stands for Machine DL stands for Deep Learning,
Intelligence, and is Learning, and is the study that and is the study that makes
 Artificial Intelligence Machine Learning Deep Learning

basically the
use of Neural
study/process which
uses statistical methods Networks(similar to neurons
enables machines to
enabling machines to improve present in human brain) to
mimic human behaviour
with experience. imitate functionality just like
through particular
a human brain.
algorithm.

AI is the broader family

consisting of ML and DL as ML is the subset of AI. DL is the subset of ML.
it’s components.

DL is a ML algorithm that
AI is a computer
ML is an AI algorithm which uses deep(more than one
algorithm which exhibits
allows system to learn from layer) neural networks to
intelligence through
data. analyze data and provide
decision making.
output accordingly.

If you are clear about the

If you have a clear idea about math involved in it but don’t
the logic(math) involved in have idea about the features,
Search Trees and much behind and you can visualize so you break the complex
complex math is involved the complex functionalities functionalities into
in AI. like K-Mean, Support Vector linear/lower dimension
Machines, etc., then it defines features by adding more
the ML aspect. layers, then it defines the DL
aspect.

It attains the highest rank in

The aim is to basically The aim is to increase
terms of accuracy when it is
increase chances of accuracy not caring much
trained with large amount of
success and not accuracy. about the success ratio.
data.
 Artificial Intelligence Machine Learning Deep Learning

DL can be considered as
neural networks with a large
Three broad
number of parameters layers
categories/types Of AI
Three broad categories/types lying in one of the four
are: Artificial Narrow
Of ML are: Supervised fundamental network
Intelligence (ANI),
Learning, Unsupervised architectures: Unsupervised
Artificial General
Learning and Reinforcement Pre-trained Networks,
Intelligence (AGI) and
Learning Convolutional Neural
Artificial Super
Networks, Recurrent Neural
Intelligence (ASI)
Networks and Recursive
Neural Networks

The efficiency Of AI is Less efficient than DL as it

More powerful than ML as it
basically the efficiency can’t work for longer
can easily work for larger
provided by ML and DL dimensions or higher amount
sets of data.
respectively. of data.

Examples of AI
applications include: Examples of ML applications Examples of DL applications
Google’s AI-Powered include: Virtual Personal include: Sentiment based
Predictions, Ridesharing Assistants: Siri, Alexa, Google, news aggregation, Image
Apps Like Uber and Lyft, etc., Email Spam and Malware analysis and caption
Commercial Flights Use an Filtering. generation, etc.
AI Autopilot, etc.

AI refers to the broad field ML is a subset of AI that DL is a subset of ML that

of computer science that focuses on developing focuses on developing deep
focuses on creating algorithms that can learn neural networks that can
intelligent machines that from data and improve their automatically learn and
can perform tasks that performance over time extract features from data.
would normally require without being explicitly
human intelligence, such
 Artificial Intelligence Machine Learning Deep Learning

as reasoning, perception,
programmed.
and decision-making.

ML algorithms can be
categorized as supervised,
unsupervised, or
AI can be further broken
reinforcement learning. In DL algorithms are inspired
down into various
supervised learning, the by the structure and function
subfields such as robotics,
algorithm is trained on of the human brain, and they
natural language
labeled data, where the are particularly well-suited to
processing, computer
desired output is known. In tasks such as image and
vision, expert systems,
unsupervised learning, the speech recognition.
and more.
algorithm is trained on
unlabeled data, where the
desired output is unknown.

DL networks consist of
In reinforcement learning, the multiple layers of
AI systems can be rule- algorithm learns by trial and interconnected neurons that
based, knowledge-based, error, receiving feedback in process data in a hierarchical
or data-driven. the form of rewards or manner, allowing them to
punishments. learn increasingly complex
representations of the data.

AI vs. Machine Learning vs. Deep Learning Examples:

Artificial Intelligence (AI) refers to the development of computer systems that can
perform tasks that would normally require human intelligence.
Some examples of AI include:
There are numerous examples of AI applications across various industries. Here
are some common examples:
 Speech recognition: speech recognition systems use deep learning algorithms
to recognize and classify images and speech. These systems are used in a variety
of applications, such as self-driving cars, security systems, and medical imaging.
 Personalized recommendations: E-commerce sites and streaming services like
Amazon and Netflix use AI algorithms to analyze users’ browsing and viewing
history to recommend products and content that they are likely to be interested
in.
 Predictive maintenance: AI-powered predictive maintenance systems analyze
data from sensors and other sources to predict when equipment is likely to fail,
helping to reduce downtime and maintenance costs.
 Medical diagnosis: AI-powered medical diagnosis systems analyze medical
images and other patient data to help doctors make more accurate diagnoses and
treatment plans.
 Autonomous vehicles: Self-driving cars and other autonomous vehicles use AI
algorithms and sensors to analyze their environment and make decisions about
speed, direction, and other factors.
 Virtual Personal Assistants (VPA) like Siri or Alexa – these use natural
language processing to understand and respond to user requests, such as playing
music, setting reminders, and answering questions.
 Autonomous vehicles – self-driving cars use AI to analyze sensor data, such as
cameras and lidar, to make decisions about navigation, obstacle avoidance, and
route planning.
 Fraud detection – financial institutions use AI to analyze transactions and
detect patterns that are indicative of fraud, such as unusual spending patterns or
transactions from unfamiliar locations.
 Image recognition – AI is used in applications such as photo organization,
security systems, and autonomous robots to identify objects, people, and scenes
in images.
 Natural language processing – AI is used in chatbots and language translation
systems to understand and generate human-like text.
 Predictive analytics – AI is used in industries such as healthcare and marketing
to analyze large amounts of data and make predictions about future events, such
as disease outbreaks or consumer behavior.
 Game-playing AI – AI algorithms have been developed to play games such as
chess, Go, and poker at a superhuman level, by analyzing game data and making
predictions about the outcomes of moves.
Examples of Machine Learning:
Machine Learning (ML) is a subset of Artificial Intelligence (AI) that involves the
use of algorithms and statistical models to allow a computer system to “learn” from
data and improve its performance over time, without being explicitly programmed
to do so.
Here are some examples of Machine Learning:
 Image recognition: Machine learning algorithms are used in image recognition
systems to classify images based on their contents. These systems are used in a
variety of applications, such as self-driving cars, security systems, and medical
imaging.
 Speech recognition: Machine learning algorithms are used in speech
recognition systems to transcribe speech and identify the words spoken. These
systems are used in virtual assistants like Siri and Alexa, as well as in call
centers and other applications.
 Natural language processing (NLP): Machine learning algorithms are used in
NLP systems to understand and generate human language. These systems are
used in chatbots, virtual assistants, and other applications that involve natural
language interactions.
 Recommendation systems: Machine learning algorithms are used in
recommendation systems to analyze user data and recommend products or
services that are likely to be of interest. These systems are used in e-commerce
sites, streaming services, and other applications.
 Sentiment analysis: Machine learning algorithms are used in sentiment analysis
systems to classify the sentiment of text or speech as positive, negative, or
neutral. These systems are used in social media monitoring and other
applications.
 Predictive maintenance: Machine learning algorithms are used in predictive
maintenance systems to analyze data from sensors and other sources to predict
when equipment is likely to fail, helping to reduce downtime and maintenance
costs.
 Spam filters in email – ML algorithms analyze email content and metadata to
identify and flag messages that are likely to be spam.
 Recommendation systems – ML algorithms are used in e-commerce websites
and streaming services to make personalized recommendations to users based on
their browsing and purchase history.
 Predictive maintenance – ML algorithms are used in manufacturing to predict
when machinery is likely to fail, allowing for proactive maintenance and
reducing downtime.
 Credit risk assessment – ML algorithms are used by financial institutions to
assess the credit risk of loan applicants, by analyzing data such as their income,
employment history, and credit score.
 Customer segmentation – ML algorithms are used in marketing to segment
customers into different groups based on their characteristics and behavior,
allowing for targeted advertising and promotions.
  Fraud detection – ML algorithms are used in financial transactions to detect
patterns of behavior that are indicative of fraud, such as unusual spending
patterns or transactions from unfamiliar locations.
 Speech recognition – ML algorithms are used to transcribe spoken words into
text, allowing for voice-controlled interfaces and dictation software.
Examples of Deep Learning:
Deep Learning is a type of Machine Learning that uses artificial neural networks
with multiple layers to learn and make decisions.

Difference between ANN and BNN





Do you ever think of what it’s like to build anything like a brain, how these things
work, or what they do? Let us look at how nodes communicate with neurons and
what are some differences between artificial and biological neural networks.
1. Artificial Neural Network: Artificial Neural Network (ANN) is a type of
neural network that is based on a Feed-Forward strategy. It is called this because
they pass information through the nodes continuously till it reaches the output
node. This is also known as the simplest type of neural network. Some advantages
of ANN :
 Ability to learn irrespective of the type of data (Linear or Non-Linear).
 ANN is highly volatile and serves best in financial time series forecasting.
Some disadvantages of ANN :
 The simplest architecture makes it difficult to explain the behavior of the
network.
 This network is dependent on hardware.
2. Biological Neural Network: Biological Neural Network (BNN) is a structure
that consists of Synapse, dendrites, cell body, and axon. In this neural network, the
processing is carried out by neurons. Dendrites receive signals from other neurons,
Soma sums all the incoming signals and axon transmits the signals to other cells.
Some advantages of BNN :
 The synapses are the input processing element.
 It is able to process highly complex parallel inputs.
Some disadvantages of BNN :
 There is no controlling mechanism.
 Speed of processing is slow being it is complex.
Differences between ANN and BNN :
Biological Neural Networks (BNNs) and Artificial Neural Networks (ANNs) are
both composed of similar basic components, but there are some differences
between them.
Neurons: In both BNNs and ANNs, neurons are the basic building blocks that
process and transmit information. However, BNN neurons are more complex and
diverse than ANNs. In BNNs, neurons have multiple dendrites that receive input
from multiple sources, and the axons transmit signals to other neurons, while in
ANNs, neurons are simplified and usually only have a single output.
Synapses: In both BNNs and ANNs, synapses are the points of connection
between neurons, where information is transmitted. However, in ANNs, the
connections between neurons are usually fixed, and the strength of the connections
is determined by a set of weights, while in BNNs, the connections between neurons
are more flexible, and the strength of the connections can be modified by a variety
of factors, including learning and experience.
Neural Pathways: In both BNNs and ANNs, neural pathways are the connections
between neurons that allow information to be transmitted throughout the network.
However, in BNNs, neural pathways are highly complex and diverse, and the
connections between neurons can be modified by experience and learning. In
ANNs, neural pathways are usually simpler and predetermined by the architecture
of the network.
Parameters ANN BNN

input dendrites
weight synapse
Structure
output axon
hidden layer cell body

very precise structures and

Learning they can tolerate ambiguity
formatted data

complex simple
Processor high speed low speed
one or a few large number
 separate from a processor integrated into processor
Memory localized distributed
non-content addressable content-addressable

centralized distributed
Computing sequential parallel
stored programs self-learning

Reliability very vulnerable robust

numerical and symbolic perceptual

Expertise
manipulations problems

Operating well-defined poorly defined

Environment well-constrained un-constrained

performance degraded even on

Fault Tolerance the potential of fault tolerance
partial damage

Overall, while BNNs and ANNs share many basic components, there are
significant differences in their complexity, flexibility, and adaptability. BNNs are
highly complex and adaptable systems that can process information in parallel, and
their plasticity allows them to learn and adapt over time. In contrast, ANNs are
simpler systems that are designed to perform specific tasks, and their connections
are usually fixed, with the network architecture determined by the designer.

Single Layer Perceptron in TensorFlow




single-layer perceptron and its implementation in Python using the TensorFlow

library. Neural Networks work in the same way that our biological neuron works.
 Structure of a biological neuron

Biological neuron has three basic functionality

 Receive signal from outside.
 Process the signal and enhance whether we need to send information or not.
 Communicate the signal to the target cell which can be another neuron or gland.
In the same way, neural networks also work.

Neural Network in Machine Learning

What is Single Layer Perceptron?

It is one of the oldest and first introduced neural networks. It was proposed
by Frank Rosenblatt in 1958. Perceptron is also known as an artificial neural
 network. Perceptron is mainly used to compute the logical gate like AND, OR,
and NOR which has binary input and binary output.
The main functionality of the perceptron is:-
 Takes input from the input layer
 Weight them up and sum it up.
 Pass the sum to the nonlinear function to produce the output.

Single-layer neural network

Here activation functions can be anything like sigmoid, tanh, relu Based on the
requirement we will be choosing the most appropriate nonlinear activation
function to produce the better result. Now let us implement a single-layer
perceptron.
IMPLEMENTATION OF SINGLE-LAYER PERCEPTRON
Let us now implement a single-layer perceptron using the “MNIST” dataset using
the TensorFlow library.
Step1: Import necessary libraries
 Numpy – Numpy arrays are very fast and can perform large computations in a
very short time.
 Matplotlib – This library is used to draw visualizations.
 TensorFlow – This is an open-source library that is used for Machine Learning
and Artificial intelligence and provides a range of functions to achieve complex
functionalities with single lines of code.

Multi-layer Perceptron
 Multi-layer perception is also known as MLP. It is fully connected dense layers,
which transform any input dimension to the desired dimension. A multi-layer
perception is a neural network that has multiple layers. To create a neural
network we combine neurons together so that the outputs of some neurons are
inputs of other neurons.
A gentle introduction to neural networks and TensorFlow can be found here:
 Neural Networks
 Introduction to TensorFlow
A multi-layer perceptron has one input layer and for each input, there is one
neuron(or node), it has one output layer with a single node for each output and it
can have any number of hidden layers and each hidden layer can have any
number of nodes. A schematic diagram of a Multi-Layer Perceptron (MLP) is
depicted below.

In the multi-layer perceptron diagram above, we can see that there are three
inputs and thus three input nodes and the hidden layer has three nodes. The
output layer gives two outputs, therefore there are two output nodes. The nodes in
the input layer take input and forward it for further process, in the diagram above
the nodes in the input layer forwards their output to each of the three nodes in the
hidden layer, and in the same way, the hidden layer processes the information and
passes it to the output layer.
Every node in the multi-layer perception uses a sigmoid activation function. The
sigmoid activation function takes real values as input and converts them to
numbers between 0 and 1 using the sigmoid formula.

Artificial Neural Networks and its Applications





As you read this article, which organ in your body is thinking about it? It’s the
brain of course! But do you know how the brain works? Well, it has neurons or
nerve cells that are the primary units of both the brain and the nervous system.
These neurons receive sensory input from the outside world which they process
and then provide the output which might act as the input to the next neuron.
Each of these neurons is connected to other neurons in complex arrangements at
synapses. Now, are you wondering how this is related to Artificial Neural
Networks? Well, Artificial Neural Networks are modeled after the neurons in the
human brain. Let’s check out what they are in detail and how they learn
information.

Artificial Neural Networks

Artificial Neural Networks contain artificial neurons which are called units. These
units are arranged in a series of layers that together constitute the whole Artificial
Neural Network in a system. A layer can have only a dozen units or millions of
units as this depends on how the complex neural networks will be required to learn
the hidden patterns in the dataset. Commonly, Artificial Neural Network has an
input layer, an output layer as well as hidden layers. The input layer receives data
from the outside world which the neural network needs to analyze or learn about.
Then this data passes through one or multiple hidden layers that transform the
input into data that is valuable for the output layer. Finally, the output layer
provides an output in the form of a response of the Artificial Neural Networks to
input data provided.
In the majority of neural networks, units are interconnected from one layer to
another. Each of these connections has weights that determine the influence of one
unit on another unit. As the data transfers from one unit to another, the neural
network learns more and more about the data which eventually results in an output
from the output layer.
 Neural Networks Architecture

The structures and operations of human neurons serve as the basis for artificial
neural networks. It is also known as neural networks or neural nets. The input layer
of an artificial neural network is the first layer, and it receives input from external
sources and releases it to the hidden layer, which is the second layer. In the hidden
layer, each neuron receives input from the previous layer neurons, computes the
weighted sum, and sends it to the neurons in the next layer. These connections are
weighted means effects of the inputs from the previous layer are optimized more or
less by assigning different-different weights to each input and it is adjusted during
the training process by optimizing these weights for improved model performance.
Artificial neurons vs Biological neurons
The concept of artificial neural networks comes from biological neurons found in
animal brains So they share a lot of similarities in structure and function wise.
 Structure: The structure of artificial neural networks is inspired by biological
neurons. A biological neuron has a cell body or soma to process the impulses,
dendrites to receive them, and an axon that transfers them to other neurons. The
input nodes of artificial neural networks receive input signals, the hidden layer
nodes compute these input signals, and the output layer nodes compute the final
output by processing the hidden layer’s results using activation functions.
 Biological Neuron Artificial Neuron

Dendrite Inputs

Cell nucleus or
Nodes
Soma

Synapses Weights

Axon Output

 Synapses: Synapses are the links between biological neurons that enable the
transmission of impulses from dendrites to the cell body. Synapses are the
weights that join the one-layer nodes to the next-layer nodes in artificial neurons.
The strength of the links is determined by the weight value.
 Learning: In biological neurons, learning happens in the cell body nucleus or
soma, which has a nucleus that helps to process the impulses. An action potential
is produced and travels through the axons if the impulses are powerful enough to
reach the threshold. This becomes possible by synaptic plasticity, which
represents the ability of synapses to become stronger or weaker over time in
reaction to changes in their activity. In artificial neural networks,
backpropagation is a technique used for learning, which adjusts the weights
between nodes according to the error or differences between predicted and actual
outcomes.
Biological Neuron Artificial Neuron

Synaptic plasticity Backpropagations

 Activation: In biological neurons, activation is the firing rate of the neuron

which happens when the impulses are strong enough to reach the threshold. In
artificial neural networks, A mathematical function known as an activation
function maps the input to the output, and executes activations.

How do Artificial Neural Networks learn?

Artificial neural networks are trained using a training set. For example, suppose
you want to teach an ANN to recognize a cat. Then it is shown thousands of
different images of cats so that the network can learn to identify a cat. Once the
 neural network has been trained enough using images of cats, then you need to
check if it can identify cat images correctly. This is done by making the ANN
classify the images it is provided by deciding whether they are cat images or not.
The output obtained by the ANN is corroborated by a human-provided description
of whether the image is a cat image or not. If the ANN identifies incorrectly
then back-propagation is used to adjust whatever it has learned during
training. Backpropagation is done by fine-tuning the weights of the connections in
ANN units based on the error rate obtained. This process continues until the
artificial neural network can correctly recognize a cat in an image with minimal
possible error rates.

What are the types of Artificial Neural Networks?

 Feedforward Neural Network: The feedforward neural network is one of the

most basic artificial neural networks. In this ANN, the data or the input provided
travels in a single direction. It enters into the ANN through the input layer and
exits through the output layer while hidden layers may or may not exist. So the
feedforward neural network has a front-propagated wave only and usually does
not have backpropagation.
 Convolutional Neural Network: A Convolutional neural network has some
similarities to the feed-forward neural network, where the connections between
units have weights that determine the influence of one unit on another unit. But a
CNN has one or more than one convolutional layer that uses a convolution
operation on the input and then passes the result obtained in the form of output
to the next layer. CNN has applications in speech and image processing which is
particularly useful in computer vision.
 Modular Neural Network: A Modular Neural Network contains a collection of
different neural networks that work independently towards obtaining the output
with no interaction between them. Each of the different neural networks
performs a different sub-task by obtaining unique inputs compared to other
networks. The advantage of this modular neural network is that it breaks down a
large and complex computational process into smaller components, thus
decreasing its complexity while still obtaining the required output.
 Radial basis function Neural Network: Radial basis functions are those
functions that consider the distance of a point concerning the center. RBF
functions have two layers. In the first layer, the input is mapped into all the
Radial basis functions in the hidden layer and then the output layer computes the
output in the next step. Radial basis function nets are normally used to model the
data that represents any underlying trend or function.
 Recurrent Neural Network: The Recurrent Neural Network saves the output of
a layer and feeds this output back to the input to better predict the outcome of
the layer. The first layer in the RNN is quite similar to the feed-forward neural
network and the recurrent neural network starts once the output of the first layer
is computed. After this layer, each unit will remember some information from
the previous step so that it can act as a memory cell in performing
computations.

Applications of Artificial Neural Networks

1. Social Media: Artificial Neural Networks are used heavily in Social Media. For
example, let’s take the ‘People you may know’ feature on Facebook that
suggests people that you might know in real life so that you can send them friend
requests. Well, this magical effect is achieved by using Artificial Neural
Networks that analyze your profile, your interests, your current friends, and also
their friends and various other factors to calculate the people you might
potentially know. Another common application of Machine Learning in social
media is facial recognition. This is done by finding around 100 reference points
on the person’s face and then matching them with those already available in the
database using convolutional neural networks.
2. Marketing and Sales: When you log onto E-commerce sites like Amazon and
Flipkart, they will recommend your products to buy based on your previous
browsing history. Similarly, suppose you love Pasta, then Zomato, Swiggy, etc.
will show you restaurant recommendations based on your tastes and previous
order history. This is true across all new-age marketing segments like Book
sites, Movie services, Hospitality sites, etc. and it is done by
implementing personalized marketing. This uses Artificial Neural Networks to
identify the customer likes, dislikes, previous shopping history, etc., and then
tailor the marketing campaigns accordingly.
3. Healthcare: Artificial Neural Networks are used in Oncology to train algorithms
that can identify cancerous tissue at the microscopic level at the same accuracy
as trained physicians. Various rare diseases may manifest in physical
characteristics and can be identified in their premature stages by using Facial
Analysis on the patient photos. So the full-scale implementation of Artificial
Neural Networks in the healthcare environment can only enhance the diagnostic
abilities of medical experts and ultimately lead to the overall improvement in the
quality of medical care all over the world.
4. Personal Assistants: I am sure you all have heard of Siri, Alexa, Cortana, etc.,
and also heard them based on the phones you have!!! These are personal
assistants and an example of speech recognition that uses Natural Language
 Processing to interact with the users and formulate a response accordingly.
Natural Language Processing uses artificial neural networks that are made to
handle many tasks of these personal assistants such as managing the language
syntax, semantics, correct speech, the conversation that is going on, etc.

BPN was discovered by Rumelhart, Williams & Honton in 1986. The core concept of
BPN is to backpropagate or spread the error from units of output layer to internal hidden layers
in order to tune the weights to ensure lower error rates. It is considered a practice of fine-tuning
the weights of neural networks in each iteration. Proper tuning of the weights will make a sure
minimum loss and this will make a more robust, and generalizable trained neural network.

What is Backpropagation?
Backpropagation is the essence of neural network training. It is the method of fine-tuning the
weights of a neural network based on the error rate obtained in the previous epoch (i.e.,
iteration). Proper tuning of the weights allows you to reduce error rates and make the model
reliable by increasing its generalization.

Backpropagation in neural network is a short form for “backward propagation of errors.” It is a

standard method of training artificial neural networks. This method helps calculate the gradient
of a loss function with respect to all the weights in the network.

Table of Content:
 









17.8M

156

10 Most Common Interview Questions and Answers

Next

Stay

How Backpropagation Algorithm Works

The Back propagation algorithm in neural network computes the gradient of the loss function for
a single weight by the chain rule. It efficiently computes one layer at a time, unlike a native
direct computation. It computes the gradient, but it does not define how the gradient is used. It
generalizes the computation in the delta rule.

Consider the following Back propagation neural network example diagram to understand:
How Backpropagation Algorithm Works

1. Inputs X, arrive through the preconnected path

2. Input is modeled using real weights W. The weights are usually randomly selected.
3. Calculate the output for every neuron from the input layer, to the hidden layers, to the
output layer.
4. Calculate the error in the outputs

Error = Actual Output – Desired Output

B

5. Travel back from the output layer to the hidden layer to adjust the weights such that the
error is decreased.

Keep repeating the process until the desired output is achieved

Why We Need Backpropagation?

Most prominent advantages of Backpropagation are:

 Backpropagation is fast, simple and easy to program

 It has no parameters to tune apart from the numbers of input
 It is a flexible method as it does not require prior knowledge about the network
 It is a standard method that generally works well
 It does not need any special mention of the features of the function to be learned.

What is a Feed Forward Network?

A feedforward neural network is an artificial neural network where the nodes never form a cycle.
This kind of neural network has an input layer, hidden layers, and an output layer. It is the first
and simplest type of artificial neural network.

Types of Backpropagation Networks

Two Types of Backpropagation Networks are:

 Static Back-propagation
 Recurrent Backpropagation

Static back-propagation
It is one kind of backpropagation network which produces a mapping of a static input for static
output. It is useful to solve static classification issues like optical character recognition.

Recurrent Backpropagation
Recurrent Back propagation in data mining is fed forward until a fixed value is achieved. After
that, the error is computed and propagated backward.

The main difference between both of these methods is: that the mapping is rapid in static back-
propagation while it is nonstatic in recurrent backpropagation.

History of Backpropagation
 In 1961, the basics concept of continuous backpropagation were derived in the context
of control theory by J. Kelly, Henry Arthur, and E. Bryson.
 In 1969, Bryson and Ho gave a multi-stage dynamic system optimization method.
 In 1974, Werbos stated the possibility of applying this principle in an artificial neural
network.
 In 1982, Hopfield brought his idea of a neural network.
 In 1986, by the effort of David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams,
backpropagation gained recognition.
 In 1993, Wan was the first person to win an international pattern recognition contest with
the help of the backpropagation method.

Backpropagation Key Points

 Simplifies the network structure by elements weighted links that have the least effect on
the trained network
 You need to study a group of input and activation values to develop the relationship
between the input and hidden unit layers.
 It helps to assess the impact that a given input variable has on a network output. The
knowledge gained from this analysis should be represented in rules.
 Backpropagation is especially useful for deep neural networks working on error-prone
projects, such as image or speech recognition.
 Backpropagation takes advantage of the chain and power rules allows backpropagation
to function with any number of outputs.
Best practice Backpropagation
Backpropagation in neural network can be explained with the help of “Shoe Lace” analogy

Too little tension =

 Not enough constraining and very loose

Too much tension =

 Too much constraint (overtraining)

 Taking too much time (relatively slow process)
 Higher likelihood of breaking

Pulling one lace more than other =

 Discomfort (bias)

Disadvantages of using Backpropagation

 The actual performance of backpropagation on a specific problem is dependent on the
input data.
 Back propagation algorithm in data mining can be quite sensitive to noisy data
 You need to use the matrix-based approach for backpropagation instead of mini-batch.

Summary
 A neural network is a group of connected it I/O units where each connection has a
weight associated with its computer programs.
 Backpropagation is a short form for “backward propagation of errors.” It is a standard
method of training artificial neural networks
 Back propagation algorithm in machine learning is fast, simple and easy to program
 A feedforward BPN network is an artificial neural network.
 Two Types of Backpropagation Networks are 1)Static Back-propagation 2) Recurrent
Backpropagation
 In 1961, the basics concept of continuous backpropagation were derived in the context
of control theory by J. Kelly, Henry Arthur, and E. Bryson.
 Back propagation in data mining simplifies the network structure by removing weighted
links that have a minimal effect on the trained network.
 It is especially useful for deep neural networks working on error-prone projects, such as
image or speech recognition.
 The biggest drawback of the Backpropagation is that it can be sensitive for noisy data.