AD-601 Deep Learning (May 2023) - Expanded
Exam Notes (v2)
Q1(a) Deep Learning & Biological Neuron
Definition: Deep Learning is a subset of Machine Learning that uses Artificial Neural Networks
(ANNs) with multiple hidden layers to learn complex patterns from data automatically.
Characteristics: Automatic feature extraction, high accuracy, supports image, text and speech
processing, requires large datasets and GPUs.
Applications: Face recognition, medical diagnosis, autonomous vehicles, NLP, recommendation
systems.
Biological Neuron: Dendrites receive signals, Soma processes information, Axon carries the
impulse, Myelin sheath increases transmission speed, Synapse transfers signals to the next
neuron.
Q1(b) Machine Learning vs Deep Learning
Machine Learning: Requires manual feature engineering, works well with small and medium
datasets, comparatively lower computational cost.
Deep Learning: Learns features automatically, performs better on unstructured data, needs
powerful hardware and large datasets.
Examples: ML – Decision Tree, SVM. DL – CNN, RNN, LSTM.
Q2(a) Gradient Descent
Gradient Descent minimizes the loss function by updating weights iteratively.
Formula: W(new)=W(old)−η×Gradient.
Algorithm: Initialize weights → Forward propagation → Calculate loss → Compute gradients →
Update weights → Repeat until convergence.
Types: Batch GD, Stochastic GD, Mini-batch GD.
Advantages: Simple, efficient, widely used.
Q2(b) MLP & Backpropagation
MLP consists of Input, Hidden and Output layers. Hidden layers use activation functions to learn
non-linear relationships.
Backpropagation computes error at the output and propagates it backwards to update weights
using gradient descent.
Q3(a) LeNet
LeNet is one of the earliest CNN architectures developed by Yann LeCun for handwritten digit
recognition. Architecture: Input → Conv → Pool → Conv → Pool → Fully Connected → Output.
Advantages: fewer parameters, good accuracy, efficient image classification.
Q3(b) Convolution
Convolution uses a kernel/filter to scan the input image and generate feature maps. It extracts
edges, textures and important visual features while reducing manual feature engineering.
Q4(a) CNN Variants
LeNet, AlexNet, VGGNet, GoogLeNet and ResNet differ in depth, architecture and performance.
ResNet uses residual connections to train very deep networks.
Q4(b) RNN & Bidirectional RNN
RNN is suitable for sequential data because it maintains previous hidden states. Bidirectional RNN
processes data in forward and backward directions for better contextual understanding.
Q5(a) Deep RNN
Deep RNN contains multiple recurrent hidden layers and is used in speech recognition, NLP,
language translation and time-series forecasting.
Q5(b) DBM
Deep Boltzmann Machine is an unsupervised probabilistic generative model with multiple hidden
layers used for feature learning and representation.
Q5(c) DBN & Supervised Learning
DBN is formed by stacking Restricted Boltzmann Machines (RBMs). Supervised learning trains
models using labelled input-output data.
Q6(a) Deep Learning in Data Science
Applications include fraud detection, predictive analytics, recommendation systems, medical image
analysis, sentiment analysis and anomaly detection.
Q6(b) Neural Network
Components include neurons, weights, bias and activation functions. Data flows from input to
output through hidden layers.
Q7(a) Gated RNN
LSTM and GRU use gates to preserve long-term dependencies and overcome the vanishing
gradient problem.
Q7(b) Deep Generative Models
GAN, VAE, DBN and DBM generate realistic synthetic data for image generation, augmentation
and simulation.
Q8(a) Gibbs Sampling
A Markov Chain Monte Carlo (MCMC) algorithm used to sample from complex probability
distributions in Bayesian models.
Q8(b) Feedforward Neural Network
The simplest ANN where information flows only in one direction: Input → Hidden → Output.
Q8(c) AI vs Deep Learning
AI is the broad field, ML is its subset, and DL is a subset of ML using deep neural networks.
Q8(d) Final Revision
Revise Gradient Descent, Backpropagation, CNN, RNN, LSTM, LeNet, DBN, DBM, Gibbs
Sampling and Deep Generative Models.