Deep Learning Tutorial

Deep learning is based on the branch of machine learning, which is a subset of artificial intelligence. Since
neural networks imitate the human brain and so deep learning will do. In deep learning, nothing is
programmed explicitly. Basically, it is a machine learning class that makes use of numerous nonlinear
processing units so as to perform feature extraction as well as transformation. The output from each
preceding layer is taken as input by each one of the successive layers.
Deep learning models are capable enough to focus on the accurate features themselves by requiring a little
guidance from the programmer and are very helpful in solving out the problem of dimensionality. Deep
learning algorithms are used, especially when we have a huge no of inputs and outputs.
Since deep learning has been evolved by the machine learning, which itself is a subset of artificial
intelligence and as the idea behind the artificial intelligence is to mimic the human behavior, so same is "the
idea of deep learning to build such algorithm that can mimic the brain".
Deep learning is implemented with the help of Neural Networks, and the idea behind the motivation
of Neural Network is the biological neurons, which is nothing but a brain cell.
Deep learning is a collection of statistical techniques of machine learning for learning feature hierarchies
that are actually based on artificial neural networks.
So basically, deep learning is implemented by the help of deep networks, which are nothing but neural
networks with multiple hidden layers.
Example of Deep Learning

In the example given above, we provide the raw data of images to the first layer of the input layer. After
then, these input layer will determine the patterns of local contrast that means it will differentiate on the
basis of colors, luminosity, etc. Then the 1st hidden layer will determine the face feature, i.e., it will fixate
on eyes, nose, and lips, etc. And then, it will fixate those face features on the correct face template. So, in the
2nd hidden layer, it will actually determine the correct face here as it can be seen in the above image, after
which it will be sent to the output layer. Likewise, more hidden layers can be added to solve more complex
problems, for example, if you want to find out a particular kind of face having large or light complexions.
So, as and when the hidden layers increase, we are able to solve complex problems.
Architectures
o Deep Neural Networks
It is a neural network that incorporates the complexity of a certain level, which means several
numbers of hidden layers are encompassed in between the input and output layers. They are highly
proficient on model and process non-linear associations.
o Deep Belief Networks
A deep belief network is a class of Deep Neural Network that comprises of multi-layer belief
 networks.
Steps to perform DBN:
1. With the help of the Contrastive Divergence algorithm, a layer of features is learned from
perceptible units.
2. Next, the formerly trained features are treated as visible units, which perform learning of
features.
3. Lastly, when the learning of the final hidden layer is accomplished, then the whole DBN is
trained.
o Recurrent Neural Networks
It permits parallel as well as sequential computation, and it is exactly similar to that of the human
brain (large feedback network of connected neurons). Since they are capable enough to reminisce all
of the imperative things related to the input they have received, so they are more precise.
Types of Deep Learning Networks
1. Feed Forward Neural Network
A feed-forward neural network is none other than an Artificial
Neural Network, which ensures that the nodes do not form a
cycle. In this kind of neural network, all the perceptrons are
organized within layers, such that the input layer takes the input,
and the output layer generates the output. Since the hidden layers
do not link with the outside world, it is named as hidden layers.
Each of the perceptrons contained in one single layer is
associated with each node in the subsequent layer. It can be
concluded that all of the nodes are fully connected. It does not contain any visible or invisible connection
between the nodes in the same layer. There are no back-loops in the feed-forward network. To minimize the
prediction error, the backpropagation algorithm can be used to update the weight values.

Applications:
o Data Compression
o Pattern Recognition
o Computer Vision
o Sonar Target Recognition
o Speech Recognition
o Handwritten Characters Recognition
2. Recurrent Neural Network
Recurrent neural networks are yet another variation of feed-forward networks. Here each of the neurons
present in the hidden layers receives an input with a specific delay in time. The Recurrent neural network
mainly accesses the preceding info of existing iterations. For example, to guess the succeeding word in any
sentence, one must have knowledge about the words that were previously used. It not only processes the
inputs but also shares the length as well as weights crossways time. It does not let the size of the model to
increase with the increase in the input size. However, the only problem with this recurrent neural network is
that it has slow computational speed as well as it does not contemplate any future input for the current state.
It has a problem with reminiscing prior information.
Applications:
o Machine Translation
o Robot Control
o Time Series Prediction
o Speech Recognition
o Speech Synthesis
 o Time Series Anomaly Detection
o Rhythm Learning
o Music Composition
3. Convolutional Neural Network
Convolutional Neural Networks are a special kind of neural network mainly used for image classification,
clustering of images and object recognition. DNNs enable unsupervised construction of hierarchical image
representations. To achieve the best accuracy, deep convolutional neural networks are preferred more than
any other neural network.
Applications:
o Identify Faces, Street Signs, Tumors.
o Image Recognition.
o Video Analysis.
o NLP.
o Anomaly Detection.
o Drug Discovery.
o Checkers Game.
o Time Series Forecasting.
4. Restricted Boltzmann Machine
RBMs are yet another variant of Boltzmann Machines. Here the neurons present in the input layer and the
hidden layer encompasses symmetric connections amid them. However, there is no internal association
within the respective layer. But in contrast to RBM, Boltzmann machines do encompass internal connections
inside the hidden layer. These restrictions in BMs helps the model to train efficiently.
Applications:
o Filtering.
o Feature Learning.
o Classification.
o Risk Detection.
o Business and Economic analysis.
5. Autoencoders
An autoencoder neural network is another kind of unsupervised machine learning algorithm. Here the
number of hidden cells is merely small than that of the input cells. But the number of input cells is
equivalent to the number of output cells. An autoencoder network is trained to display the output similar to
the fed input to force AEs to find common patterns and generalize the data. The autoencoders are mainly
used for the smaller representation of the input. It helps in the reconstruction of the original data from
compressed data. This algorithm is comparatively simple as it only necessitates the output identical to the
input.
o Encoder: Convert input data in lower dimensions.
o Decoder: Reconstruct the compressed data.
Applications:
o Classification.
o Clustering.
o Feature Compression.
Deep learning applications
o Self-Driving Cars
In self-driven cars, it is able to capture the images around it by processing a huge amount of data,
and then it will decide which actions should be incorporated to take a left or right or should it stop.
So, accordingly, it will decide what actions it should take, which will further reduce the accidents
that happen every year.
o Voice Controlled Assistance
When we talk about voice control assistance, then Siri is the one thing that comes into our mind. So,
you can tell Siri whatever you want it to do it for you, and it will search it for you and display it for
you.
o Automatic Image Caption Generation
Whatever image that you upload, the algorithm will work in such a way that it will generate caption
accordingly. If you say blue colored eye, it will display a blue-colored eye with a caption at the
bottom of the image.
o Automatic Machine Translation
With the help of automatic machine translation, we are able to convert one language into another
with the help of deep learning.
8 practical examples of deep learning
Now that we’re in a time when machines can learn to solve complex problems without human
intervention, what exactly are the problems they are tackling? Here are just a few of the tasks that
deep learning supports today and the list will just continue to grow as the algorithms continue to
learn via the infusion of data.
1. Virtual assistants
Whether it’s Alexa or Siri or Cortana, the virtual assistants of online service providers use deep
learning to help understand your speech and the language humans use when they interact with them.
2. Translations
In a similar way, deep learning algorithms can automatically translate between languages. This can
be powerful for travellers, business people and those in government.
3. Vision for driverless delivery trucks, drones and autonomous cars
The way an autonomous vehicle understands the realities of the road and how to respond to them
whether it’s a stop sign, a ball in the street or another vehicle is through deep learning algorithms.
The more data the algorithms receive, the better they are able to act human-like in their information
processing—knowing a stop sign covered with snow is still a stop sign.
4. Chatbots and service bots
Chatbots and service bots that provide customer service for a lot of companies are able to respond in
an intelligent and helpful way to an increasing amount of auditory and text questions thanks to deep
learning.
5. Image colorization
Transforming black-and-white images into colour was formerly a task done meticulously by human
hand. Today, deep learning algorithms are able to use the context and objects in the images to colour
them to basically recreate the black-and-white image in colour. The results are impressive and
accurate.
6. Facial recognition
Deep learning is being used for facial recognition not only for security purposes but for tagging
people on Facebook posts and we might be able to pay for items in a store just by using our faces in
the near future. The challenges for deep-learning algorithms for facial recognition is knowing it’s the
same person even when they have changed hairstyles, grown or shaved off a beard or if the image
taken is poor due to bad lighting or an obstruction.
7. Medicine and pharmaceuticals
From disease and tumour diagnoses to personalised medicines created specifically for an individual’s
genome, deep learning in the medical field has the attention of many of the largest pharmaceutical
and medical companies.
8. Personalised shopping and entertainment
 Ever wonder how Netflix comes up with suggestions for what you should watch next? Or where
Amazon comes up with ideas for what you should buy next and those suggestions are exactly what
you need but just never knew it before? Yep, it’s deep-learning algorithms at work.
The more experience deep-learning algorithms get, the better they become. It should be an
extraordinary few years as the technology continues to mature.

Limitations
o It only learns through the observations.
o It comprises of biases issues.
Advantages
o It lessens the need for feature engineering.
o It eradicates all those costs that are needless.
o It easily identifies difficult defects.
o It results in the best-in-class performance on problems.
Disadvantages
o It requires an ample amount of data.
o It is quite expensive to train.
o It does not have strong theoretical groundwork.

Deep Learning Algorithms

What is Deep Learning Algorithm?
Deep learning can be defined as the method of machine learning and artificial intelligence that is intended to
intimidate humans and their actions based on certain human brain functions to make effective decisions. It is
a very important data science element that channels its modeling based on data-driven techniques
under predictive modeling and statistics. To drive such a human-like ability to adapt and learn and to
function accordingly, there have to be some strong forces which we popularly called algorithms.
Deep learning algorithms are dynamically made to run through several layers of neural networks, which are
nothing but a set of decision-making networks that are pre-trained to serve a task. Later, each of these is
passed through simple layered representations and move on to the next layer. However, most machine
learning is trained to work fairly well on datasets that have to deal with hundreds of features or columns. For
a data set to be structured or unstructured, machine learning tends to fail mostly because they fail to
recognize a simple image having a dimension of 800x1000 in RGB. It becomes quite unfeasible for a
traditional machine learning algorithm to handle such depths. This is where deep learning.
Importance of Deep Learning
Deep learning algorithms play a crucial role in determining the features and can handle the large number of
processes for the data that might be structured or unstructured. Although, deep learning algorithms can
overkill some tasks that might involve complex problems because they need access to huge amounts of data
so that they can function effectively. For example, there's a popular deep learning tool that recognizes
images namely Imagenet that has access to 14 million images in its dataset-driven algorithms. It is a highly
comprehensive tool that has defined a next-level benchmark for deep learning tools that aim images as their
dataset.
Deep learning algorithms are highly progressive algorithms that learn about the image that we discussed
previously by passing it through each neural network layer. The layers are highly sensitive to detect low-
level features of the image like edges and pixels and henceforth the combined layers take this information
and form holistic representations by comparing it with previous data. For example, the middle layer might
be programmed to detect some special parts of the object in the photograph which other deep trained layers
are programmed to detect special objects like dogs, trees, utensils, etc.
However, if we talk out the simple task that involves less complexity and a data-driven resource, deep
learning algorithms fail to generalize simple data. This is one of the main reasons deep learning is not
considered effective as linear or boosted tree models. Simple models aim to churn out custom data, track
fraudulent transactions and deal with less complex datasets with fewer features. Also, there are various cases
like multiclass classification where deep learning can be effective because it involves smaller but more
structured datasets but is not preferred usually.
Having said that, let's look understand some of the most important deep learning algorithms given below.
Deep Learning Algorithms
The Deep Learning Algorithms are as follows:
1. Convolutional Neural Networks (CNNs)
CNN's popularly known as ConvNets majorly consists of several layers and are specifically used for image
processing and detection of objects. It was developed in 1998 by Yann LeCun and was first
called LeNet. Back then, it was developed to recognize digits and zip code characters. CNNs have wide
usage in identifying the image of the satellites, medical image processing, series forecasting, and anomaly
detection.
CNNs process the data by passing it through multiple layers and extracting features to exhibit convolutional
operations. The Convolutional Layer consists of Rectified Linear Unit (ReLU) that outlasts to rectify the
feature map. The Pooling layer is used to rectify these feature maps into the next feed. Pooling is generally
a sampling algorithm that is down-sampled and it reduces the dimensions of the feature map. Later, the

result generated consists of 2-D arrays consisting of single, long, continuous, and linear vector flattened
in the map. The next layer i.e., called Fully Connected Layer which forms the flattened matrix or 2-
D array fetched from the Pooling Layer as input and identifies the image by classifying it.

2. Long Short Term Memory Networks (LSTMs)

LSTMs can be defined as Recurrent Neural Networks (RNN) that are programmed to learn and adapt for
dependencies for the long term. It can memorize and recall past data for a greater period and by default, it is
its sole behavior. LSTMs are designed to retain over time and henceforth they are majorly used in time
series predictions because they can restrain memory or previous inputs. This analogy comes from
their chain-like structure consisting of four interacting layers that communicate with each other differently.
Besides applications of time series prediction, they can be used to construct speech recognizers,
development in pharmaceuticals, and composition of music loops as well.
LSTM work in a sequence of events. First, they don't tend to remember irrelevant details attained in the
previous state. Next, they update certain cell-state values selectively and finally generate certain parts of the
cell-state as output. Below is the diagram of their operation.
3. Recurrent Neural Networks (RNNs)
Recurrent Neural Networks or RNNs consist of some directed connections that form a cycle that allow the
input provided from the LSTMs to be used as input in the current phase of RNNs. These inputs are deeply
embedded as inputs and enforce the memorization ability of LSTMs lets these inputs get absorbed for a
period in the internal memory. RNNs are therefore dependent on the inputs that are preserved by LSTMs
and work under the synchronization phenomenon of LSTMs. RNNs are mostly used in captioning the
image, time series analysis, recognizing handwritten data, and translating data to machines.
RNNs follow the work approach by putting output feeds (t-1) time if the time is defined as t. Next, the
output determined by t is feed at input time t+1. Similarly, these processes are repeated for all the input
consisting of any length. There's also a fact about RNNs is that they store historical information and there's
no increase in the input size even if the model size is increased. RNNs look something like this when
unfolded.

4. Generative Adversarial Networks (GANs)

GANs are defined as deep learning algorithms that are used to generate new instances of data that match the
training data. GAN usually consists of two components namely a generator that learns to generate false data
and a discriminator that adapts itself by learning from this false data. Over some time, GANs have gained
immense usage since they are frequently being used to clarify astronomical images and
simulate lensing the gravitational dark matter. It is also used in video games to increase graphics
for 2D textures by recreating them in higher resolution like 4K. They are also used in creating realistic
cartoons character and also rendering human faces and 3D object rendering.
GANs work in simulation by generating and understanding the fake data and the real data. During the
training to understand these data, the generator produces different kinds of fake data where the discriminator
quickly learns to adapt and respond to it as false data. GANs then send these recognized results for updating.
Consider the below image to visualize the functioning.

5. Radial Basis Function Networks (RBFNs)

RBFNs are specific types of neural networks that follow a feed-forward approach and make use of radial
functions as activation functions. They consist of three layers namely the input layer, hidden
layer, and output layer which are mostly used for time-series prediction, regression
testing, and classification.
RBFNs do these tasks by measuring the similarities present in the training data set. They usually have an
input vector that feeds these data into the input layer thereby confirming the identification and rolling out
results by comparing previous data sets. Precisely, the input layer has neurons that are sensitive to these
data and the nodes in the layer are efficient in classifying the class of data. Neurons are originally present in
the hidden layer though they work in close integration with the input layer. The hidden layer
contains Gaussian transfer functions that are inversely proportional to the distance of the output from the
neuron's center. The output layer has linear combinations of the radial-based data where the Gaussian
functions are passed in the neuron as parameter and output is generated. Consiider the given image below to
understand the process thoroughly.

6. Multilayer Perceptrons (MLPs)

MLPs are the base of deep learning technology. It belongs to a class of feed-forward neural networks having
various layers of perceptrons. These perceptrons have various activation functions in them. MLPs also have
connected input and output layers and their number is the same. Also, there's a layer that remains hidden
amidst these two layers. MLPs are mostly used to build image and speech recognition systems or some
other types of the translation software.
The working of MLPs starts by feeding the data in the input layer. The neurons present in the layer form a
graph to establish a connection that passes in one direction. The weight of this input data is found to exist
between the hidden layer and the input layer. MLPs use activation functions to determine which nodes are
ready to fire. These activation functions include tanh function, sigmoid and ReLUs. MLPs are mainly used
to train the models to understand what kind of co-relation the layers are serving to achieve the desired output
from the given data set. See the below image to understand better.

7. Self Organizing Maps (SOMs)

SOMs were invented by Teuvo Kohenen for achieving data visualization to understand the dimensions of
data through artificial and self-organizing neural networks. The attempts to achieve data visualization to
solve problems are mainly done by what humans cannot visualize. These data are generally high-
dimensional so there are lesser chances of human involvement and of course less error.
SOMs help in visualizing the data by initializing weights of different nodes and then choose random vectors
from the given training data. They examine each node to find the relative weights so that dependencies can
be understood. The winning node is decided and that is called Best Matching Unit (BMU). Later, SOMs
discover these winning nodes but the nodes reduce over time from the sample vector. So, the closer the node
to BMU more is the more chance to recognize the weight and carry out further activities. There are also
multiple iterations done to ensure that no node closer to BMU is missed. One example of such is the RGB
color combinations that we use in our daily tasks. Consider the below image to understand how they
function.

8. Deep Belief Networks (DBNs)

DBNs are called generative models because they have various layers of latent as well as stochastic variables.
The latent variable is called a hidden unit because they have binary values. DBNs are also
called Boltzmann Machines because the RGM layers are stacked over each other to establish
communication with previous and consecutive layers. DBNs are used in applications like video and image
recognition as well as capturing motional objects.
DBNs are powered by Greedy algorithms. The layer to layer approach by leaning through a top-
down approach to generate weights is the most common way DBNs function. DBNs use step by step
approach of Gibbs sampling on the hidden two-layer at the top. Then, these stages draw a sample from the
visible units using a model that follows the ancestral sampling method. DBNs learn from the values present
in the latent value from every layer following the bottom-up pass approach.

9. Restricted Boltzmann Machines (RBMs)

RBMs were developed by Geoffrey Hinton and resemble stochastic neural networks that learn from the
probability distribution in the given input set. This algorithm is mainly used in the field of
dimension reduction, regression and classification, topic modeling and are considered the building blocks
of DBNs. RBIs consist of two layers namely the visible layer and the hidden layer. Both of these layers are
connected through hidden units and have bias units connected to nodes that generate the output. Usually,
RBMs have two phases namely forward pass and backward pass.
The functioning of RBMs is carried out by accepting inputs and translating them to numbers so that inputs
are encoded in the forward pass. RBMs take into account the weight of every input, and the backward pass
takes these input weights and translates them further into reconstructed inputs. Later, both of these translated
inputs, along with individual weights, are combined. These inputs are then pushed to the visible layer where
the activation is carried out, and output is generated that can be easily reconstructed. To understand this
process, consider the below image.
Autoencoders
Autoencoders are a special type of neural network where inputs are outputs are found usually identical. It
was designed to primarily solve the problems related to unsupervised learning. Autoencoders are highly
trained neural networks that replicate the data. It is the reason why the input and output are generally the
same. They are used to achieve tasks like pharma discovery, image processing, and population
prediction.
Autoencoders constitute three components namely the encoder, the code, and the decoder. Autoencoders
are built in such a structure that they can receive inputs and transform them into various representations. The
attempts to copy the original input by reconstructing them is more accurate. They do this by encoding the
image or input, reduce the size. If the image is not visible properly they are passed to the neural network for
clarification. Then, the clarified image is termed a reconstructed image and this resembles as accurate as of
the previous image. To understand this complex process, see the below-provided image.