Introduction to Deep
Learning
Dr. Sushma Jain, Associate Professor
Evaluation Scheme
MST : 30 Syllabus upto CNN
EST : 40
LAB : 15 (Project + Coursera Course + Lab
Assignments)
QUIZ : 15 November (First Week)
Outline
• Introduction
• Deep Learning
• Deep Learning vs Machine Learning
• Hierarchical Representations
• Deep Learning Applications
• Deep Learning Frameworks
Introduction
Deep learning
Deep Learning is a subset of Machine Learning that uses mathematical
functions to map the input to the output. These functions can extract
non-redundant information, which enables them to form a relationship
between the input and the output
Conti…
The ANN is the heart of deep learning models. It mimic the working
of the neurons in the human brain.
Conti…
• Deep learning algorithms are similar to nervous
system structured where each neuron connected
each other and passing information.
• It deals with multi-layer neural network with more
neurons, more layers and more interconnectivity.
• The “Deep” in this learning is number of layers.
Machine Learning vs Deep Learning
Machine Learning Deep Learning
Feature Engineering Features need to be identified by expert & Learn features from data
hand coded
Data Dependencies ML algorithm can work with small amount Large data is required to
of data because of handcrafted rule understand problem perfectly
Hardware Dependencies Low end machine can used Large amount of matrix
multiplication operations, to
Optimize performance GPU is
required
Problem Solving Modular approach Solve problem end-to-end
Approach
Performance Less training time but more testing time More training time but testing
(large data set) time is less
Applications of deep learning
Conti…
• Speech recognition: Some of the familiar software like Apple’s Siri,
Google’s Alexa, and Microsoft Cortana
• Pattern recognition: Detection of Tumor cells in the CT scans, detect
fraudulent transactions
• NLP: GPT3, chatgpt
• Recommender systems: Social media platform such as Instagram,
YouTube and Netflix. These companies use a recommendation system
to recommend shows, videos, posts, and stories based on users'
activities.
Real life example of deep learning
Healthcare
Medical images such as CT scans, MRI, and X-rays can sometimes be
difficult to interpret
Surgical robotics: Robots have a superhuman ability to repeat exact
motions like that of a trained surgeon.
Transportation
Self-driving cars from Tesla
Smart cities can manage their resources efficiently and manage traffic,
public services, and disaster response
Conti….
Agriculture
Robot Picking: To classify and pick crops
Crop and soil monitoring to estimate yield
Plant disease and pest detection :
To classify plants suffering from the disease from healthy plants
To detect pest infestation
Weeds control
Summary
Machine Learning Limitations:
• It is costly to develop hand-crafted features
• The hand-crafted features are usually highly dependents on one application,
and cannot be transferred easily to other applications
• Deep learning seeks to learn rich hierarchical representations automatically
through multiple stage of feature learning process.
Image recognition
Pixel → edge → corners → part → object
Deep Learning Frameworks
• TensorFlow
• Theano
• Keras
• PyTorch :- Computer Vision and NLP
• Caffe:- Computer Vision, Image classification & Segmentation
• Deeplearning4j:-Deeplearning4J is a set of projects for building
JVM-based deep learning applications
• CNTK:- Microsoft Cognitive Toolkit (CNTK) is an open-source deep-
learning toolkit for building, training, and evaluating neural networks
Deep Learning Models
• Deep Feedforward Network (MLP)
• Convolution Neural Network (CNN)
• Recurrent Neural Network (RNN)
• Auto-Encoder (AE)
• Generative Adversarial Network (GAN)
Perceptron
Linear & Non-linear Classification
Multi Layer Perceptron
Convolution Neural Network (CNN)
[Link]
Convolution Neural Network (Cont.….)
• Input can have very high dimension. MLP is fully-connected neural
network would need a large amount of parameters.
• CNNs are a special type of neural network whose hidden units are
only connected to local receptive field.
• Less number of parameters needed by CNNs.
Convolution Neural Network Applications
• Image Classification
• Object Detection
• Video Recognition
• Medical Image Analysis
• Recommender Systems
Recurrent Neural Network (RNN) (Cont..)
CNN limitation:
Rely on examples being vectors of fixed length
All inputs are independent of each other.
After each data point is processed, the entire state of the network is
lost
Recurrent Neural Network used to process variable length sequences of
inputs
Recurrent Neural Network (RNN)
[Link]
Recurrent Neural Network (RNN) (Cont..)
• RNNs are called recurrent because they perform the same task
for every element of a sequence, with the output being depended on
the previous computations.
• RNN have a memory which captures information about what has
been calculated so far.
Recurrent Neural Network Applications
• Machine Translation
• Sentiment Classification
• Video Activity Recognition
• DNA Sequence Analysis
• Speech Recognition
• Music Generation
Auto-Encoder (AE)
[Link]
Auto-Encoder (AE)
• An autoencoder is a feed-forward neural net used to reproduced the
input
• Map high-dimensional data to lower dimensions for visualization
• Learn abstract features in an unsupervised way so these can apply
them to a supervised task
Auto-Encoder Applications
• Dimensionality Reduction
• Image Compression
• Image Denoising
• Feature Extraction
Generative Adversarial Network (GAN)
[Link]
Generative Adversarial Network (Cont..)
• Create new data examples that resemble training data.
• GANs can create images that look like photographs of human faces,
even though the faces don't belong to any real person.
• GANs achieve this level of realism by pairing a generator, which learns
to produce the target output, with a discriminator, which learns to
distinguish true data from the output of the generator. The generator
tries to fool the discriminator, and the discriminator tries to keep
from being fooled
Generative Adversarial Network
Applications
• Image Editing
• Synthetic data Generation
• Game Design
Summary
• CNN highly effective in learning good feature representation from
data in an “end-to-end” manner.
• RNN memorize previous input for prediction of next.
• AE used to encode data in lower dimension and regenerate it again
in decoding process.
• GAN mainly used for synthetic data generation.
Resources
[Link]
introduction-to-deep-learning-10048099be51