UNIT – 1: Fundamentals of Deep Learning

---

Q1. Explain Overfitting and Underfitting in Machine Learning.

Definition:

Overfitting and underfitting are common problems in machine learning where the model fails to
generalize properly on unseen data.

---

Step-by-Step Explanation:

1. Underfitting:

The model is too simple to capture the pattern in the data.

Both training and test errors are high.

Example: Linear regression for a curved dataset.

2. Overfitting:

The model is too complex and memorizes training data.

Training error is low, but test error is high.

Example: Deep neural network trained with little data.

3. Causes of Overfitting:

Too many parameters, not enough data, long training time.

4. Prevention Techniques:

Regularization (L1, L2), Dropout, Early Stopping.

Using more data or data augmentation.

5. Causes of Underfitting:
Simple model, insufficient training, wrong features.

6. Prevention Techniques:

Use more complex model, increase epochs, improve feature quality.

7. Good Fit:

A model that balances bias and variance is called a “good fit.”

---

Diagram:

Accuracy

Overfitting
/ \
/\
_____________/_____Good Fit_______\_________
Underfitting
____________________________________________ Model Complexity

---

Example:

Overfitting: Deep model on small dataset.

Underfitting: Simple model on complex dataset.

---

Conclusion:

The goal of training is to avoid both underfitting and overfitting so the model can perform well on
unseen data.

---

Keywords:
Overfitting, Underfitting, Bias, Variance, Regularization, Cross-validation.

---

---

Q2. Explain the Random Forest Algorithm with a neat diagram.

Definition:

Random Forest is an ensemble method that builds multiple decision trees using random subsets
of data and features, and combines their results for accurate prediction.

---

Step-by-Step Working:

1. Dataset Sampling:

Create multiple random subsets (bootstrap samples) from the dataset.

2. Build Decision Trees:

Train each subset independently to build a tree.

3. Feature Selection:

Each tree uses a random subset of features for splitting nodes.

4. Voting / Averaging:

Classification: majority voting among trees.

Regression: average prediction from all trees.

5. Out-of-Bag Error (OOB):

Samples not used in training are tested for validation.

6. Advantages:
High accuracy, handles missing data, resists overfitting.

7. Limitations:

Slower for large number of trees.

---

Diagram:

Dataset

Tree 1 Tree 2 Tree 3

Majority Vote Final Output

---

Example:

Predicting whether a student passes or fails using several random subsets of data.

---

Conclusion:

Random Forest combines many weak decision trees to form a strong, stable, and accurate
model.

---

Keywords:

Bagging, Ensemble Learning, Decision Tree, Bootstrap, Majority Vote.

---

---

Q3. Explain Gradient Boosting Algorithm.

 Definition:

Gradient Boosting is a sequential ensemble method that builds trees one by one, each correcting
the previous tree’s errors.

---

Step-by-Step Explanation:

1. Train first weak learner (tree) get prediction ŷ .

2. Compute residuals = (actual – predicted).

3. Train next tree on residuals ŷ .

4. Combine models Final prediction = ŷ + α·ŷ (α = learning rate).

5. Repeat multiple rounds until convergence.

6. Each new tree minimizes loss using gradient descent.

7. Result = strong model from many weak learners.

---

Diagram:

Data Tree 1 Residuals Tree 2 Residuals Tree 3 Final Model

---

Example:

Used in XGBoost, LightGBM — best for tabular and Kaggle competitions.

---

Conclusion:

Gradient Boosting builds strong models sequentially and improves prediction accuracy gradually.
---

Keywords:

Boosting, Residuals, Gradient Descent, Weak Learner, Learning Rate.

---

---

Q4. Differentiate between AI, ML, and DL.

Definition:

Artificial Intelligence (AI) is the broad field of creating intelligent machines.

Machine Learning (ML) is a subset of AI where systems learn from data.
Deep Learning (DL) is a subset of ML that uses neural networks.

---

Comparison Table:

Feature AI ML DL

Concept Machines acting smart Learn from data Neural networks learn automatically
Data need Less Moderate Huge data
Feature extraction Manual Manual Automatic
Examples Chatbots, Robotics SVM, Decision Tree CNN, RNN, GAN
Compute Low Medium High (GPU needed)

---

Diagram:

Artificial Intelligence

Machine Learning

Deep Learning

---

Example:
AI Self-driving car

ML Spam email filter

DL Face recognition using CNN

---

Conclusion:

AI is the umbrella field; ML and DL are the core techniques enabling AI in the modern era.

---

Keywords:

AI, ML, DL, Feature Learning, Neural Networks.

---

---

Q5. What are Kernel Methods in Deep Learning?

Definition:

Kernel methods map data into higher-dimensional feature spaces using kernel functions to make
it linearly separable.

---

Step-by-Step Explanation:

1. Map data x Φ(x) in higher-dimensional space.

2. Compute kernel function k(x, y) = Φ(x),Φ(y) .

3. Use kernel trick — compute dot products without explicit mapping.

4. Examples: RBF, Polynomial, Sigmoid kernels.

5. Useful in SVMs and hybrid DL models.

6. Improves non-linear separation.

7. High computational cost for large datasets.

---

Diagram:

Low-D space: inseparable points

Kernel Function
High-D space: linearly separable

---

Example:

RBF kernel SVM for handwritten digit classification.

---

Conclusion:

Kernel methods provide flexibility for complex boundaries, though rarely used directly in modern
DL.

---

Keywords:

Kernel Trick, RBF, Feature Space, SVM.

---

---

Q6. Write about the Scope and Future of AI.

Definition:

AI is the technology that enables machines to perform tasks requiring human intelligence. Its
scope covers all industries.

---
 Step-by-Step Points:

1. Healthcare: Predict diseases, robotic surgeries.

2. Finance: Fraud detection, stock market forecasting.

3. Education: Personalized learning.

4. Transportation: Self-driving cars.

5. Agriculture: Crop monitoring via drones.

6. Entertainment: Chatbots, recommendations.

7. Future Trends: Explainable AI, Edge AI, Responsible AI.

---

Diagram:

AI Applications:
Healthcare
Education
Finance
Transport
Industry

---

Example:

Google’s DeepMind for protein folding prediction (scientific AI).

---

Conclusion:

The future of AI is limitless — it will transform every sector but must be developed responsibly.

---
 Keywords:

AI Applications, Future Scope, Edge AI, Responsible AI.

---

---
UNIT – 2: Training and Optimization in Deep Learning

---

Q7. Explain different types of Optimizers in Deep Learning. (SGD, Adam, RMSProp, Adagrad)

---

Definition:

Optimizers are algorithms that adjust the model parameters (weights and biases) to minimize
the loss function during training.

---

Step-by-Step Explanation:

1. Goal of Optimizer:

Reduce error/loss between predicted and actual values.

Update weights efficiently using gradients.

2. Types of Optimizers:

a) Gradient Descent (GD):

Updates all weights using the full dataset.

Slow but accurate.

Formula:

w=w-α* L(w)

where α = learning rate, L(w)= gradient.

b) Stochastic Gradient Descent (SGD):

Updates weights after every sample (faster, noisy).

Formula same as GD, but per-sample.

c) Mini-Batch Gradient Descent:

Uses small batches for better speed and stability.

d) Adagrad:

Adapts learning rate per parameter.

Works well for sparse data.

e) RMSProp:

Uses moving average of squared gradients.

Prevents oscillations, faster convergence.

f) Adam:

Combines momentum + RMSProp.

Most widely used in DL.

Formula uses running averages of gradients (m, v).

---

Diagram:

Loss
\
\
\
\___ SGD Faster convergence
\__ Adam Adaptive step
_____________________ Epochs

---

Example:

Training CNN or RNN models — Adam optimizer used by default in Keras/TensorFlow.

---

Conclusion:

Optimizers control the learning process; Adam is the most effective for deep learning tasks.

---

Keywords:

Optimizer, Learning Rate, Gradient, Adam, RMSProp, SGD, Adagrad.

---

---

Q8. Explain the Vanishing and Exploding Gradient Problem.

---

Definition:

In deep networks, as gradients are backpropagated through layers, they may become extremely
small (vanish) or extremely large (explode), affecting training stability.

---

Step-by-Step Explanation:

1. Backpropagation:

Gradients are propagated backward layer by layer.

Each layer’s gradient depends on the product of previous derivatives.

2. Vanishing Gradient:

Gradients shrink towards zero in deeper layers.

Weights stop updating network stops learning.

Common in sigmoid/tanh activations.

3. Exploding Gradient:

Gradients grow too large unstable weights.

Leads to NaN or infinite loss values.

4. Causes:

Deep architectures with improper initialization.

Improper learning rates.

5. Solutions:

Use ReLU activation instead of Sigmoid.

Gradient Clipping (limit max gradient value).

Proper weight initialization (He/Xavier).

Batch Normalization.

---

Diagram:

Layer Depth

\
\
\__ Vanishing (gradient 0)

Exploding (gradient ∞)

Training Depth

---

Example:

Training deep RNNs — gradients vanish, so LSTM/GRU were developed to fix it.

---
 Conclusion:

Vanishing/exploding gradients hinder deep model training; using ReLU, normalization, and LSTM
resolves the issue.

---

Keywords:

Gradient Clipping, ReLU, Backpropagation, Initialization, LSTM.

---

---

Q9. Explain Backpropagation Algorithm with suitable diagram.

---

Definition:

Backpropagation is a supervised learning algorithm used to train neural networks by minimizing

the error through gradient calculation.

---

Step-by-Step Explanation:

1. Forward Pass:

Input hidden output layers.

Compute output and loss.

2. Compute Loss:

L = (Target – Output)² / 2

3. Backward Pass:

Compute derivative of loss w.r.t weights.

4. Gradient Calculation:
Use Chain Rule to find error gradients for each layer.

5. Weight Update:

w = w – α * ∂L/∂w

6. Repeat:

Continue until loss is minimized.

---

Diagram:

Input Hidden Output

Error Backpropagation

---

Example:

Used in every deep learning model (CNN, RNN, MLP) for training.

---

Conclusion:

Backpropagation is the foundation of deep learning; it allows weight updates through gradient
computation.

---

Keywords:

Error Propagation, Gradient, Weight Update, Chain Rule, Learning Rate.

---

---
Q10. Explain Loss Functions used in Deep Learning.

---

Definition:

A loss function measures the difference between predicted and actual values and guides model
optimization.

---

Types of Loss Functions:

1. Mean Squared Error (MSE):

For regression tasks.

Formula:

L = (1/n) Σ (y_pred - y_true)²

2. Mean Absolute Error (MAE):

Measures average absolute difference.

Less sensitive to outliers.

3. Binary Cross-Entropy:

For binary classification.

Formula:

L = -[y log(p) + (1 - y) log(1 - p)]

4. Categorical Cross-Entropy:

For multi-class problems.

5. Hinge Loss:

Used in SVMs.
6. KL Divergence:

Measures difference between probability distributions.

---

Diagram:

Loss vs Epochs
\
\
\__ Model learning Loss
______________________________ Epochs

---

Example:

In image classification (CNN), categorical cross-entropy is used.

---

Conclusion:

Choosing the right loss function is key for proper model learning and accuracy.

---

Keywords:

Loss, Error, MSE, Cross-Entropy, Regression, Classification.

---

---

Q11. Explain Learning Rate and its effect on model training.

---

Definition:
Learning rate (α) controls how much to update model weights in each iteration during
optimization.

---

Step-by-Step Explanation:

1. Small learning rate slow convergence.

2. Large learning rate unstable training.

3. Optimal learning rate smooth loss reduction.

4. Adaptive learning rates (Adam, RMSProp) adjust automatically.

5. Learning rate scheduling reduces rate over epochs.

---

Diagram:

Loss
\
\
\__ Too high Diverges
\__ Too low Slow
\__ Optimal Fast convergence
______________________________ Epochs

---

Example:

In CNN training, using 0.001 is a good learning rate for Adam optimizer.

---

Conclusion:

Choosing an appropriate learning rate is crucial for fast and stable convergence.

---
 Keywords:

Learning Rate, Convergence, Scheduler, Stability, Optimization.

---

---

Q12. What is Regularization in Deep Learning? Explain its types.

---

Definition:

Regularization techniques reduce overfitting by adding penalty terms or modifying training

behavior.

---

Step-by-Step Explanation:

1. L1 Regularization (Lasso):

Adds |w| term to loss makes weights sparse.

2. L2 Regularization (Ridge):

Adds w² term keeps weights small and smooth.

3. Dropout:

Randomly drops neurons during training.

4. Early Stopping:

Stops training before overfitting starts.

5. Data Augmentation:

Creates variations of data (rotation, crop, flip).

6. Batch Normalization:

Normalizes layer outputs stable gradients.

---

Diagram:

Train Accuracy
\
\__ Overfitting
\
\__ Regularized Model (Good Fit)
________________ Epochs

---

Example:

In CNN, dropout rate of 0.5 is commonly used.

---

Conclusion:

Regularization improves generalization and prevents overfitting in deep models.

---

Keywords:

L1, L2, Dropout, BatchNorm, Early Stopping, Generalization.

---

---

Q13. Explain Batch Normalization and its advantages.

---

Definition:
Batch Normalization normalizes inputs of each layer to stabilize and accelerate training.

---

Step-by-Step Explanation:

1. Compute mean (μ) and variance (σ²) for each batch.

2. Normalize each input:

x' = (x - μ) / √(σ² + ε)

3. Scale (γ) and shift (β) parameters applied.

4. Helps maintain stable gradients.

5. Reduces dependency on weight initialization.

6. Acts as regularizer, reducing overfitting.

7. Works well with deep CNNs and RNNs.

---

Diagram:

Input Batch Norm Layer Activation Output

---

Example:

Used in almost every CNN model like ResNet, VGG.

---

Conclusion:

Batch Normalization speeds up convergence and makes training more stable.

---
 Keywords:

Batch Norm, Normalization, Stability, Gradient, Regularization.

---

---

UNIT–2 Complete (7 Long Answers)

Optimizers
Vanishing/Exploding Gradient
Backpropagation
Loss Functions
Learning Rate
Regularization
Batch Normalization

UNIT – 3: Neural Network Architectures and Frameworks

---

Q14. Explain the architecture and working of an Artificial Neural Network (ANN).

---

Definition:

An Artificial Neural Network (ANN) is a computational model inspired by the human brain that
consists of interconnected neurons (nodes) arranged in layers.

---

Step-by-Step Working:

1. Input Layer:

Accepts features or data from the dataset.

Example: image pixels, text embeddings.

2. Hidden Layers:

Perform computations by applying weights and activation functions.

3. Output Layer:

Produces final predictions (e.g., class labels, numeric outputs).

4. Weighted Connections:

Each input has an associated weight that determines its influence.

5. Activation Function:

Introduces non-linearity (Sigmoid, ReLU, Tanh).

6. Forward Propagation:

Data moves forward from input output through hidden layers.

7. Backpropagation:

Error is propagated backward to adjust weights and minimize loss.

---

Diagram:

Input Layer Hidden Layer Output Layer

OOOOOOOOO

Connections (Weights)

---

Example:

Predicting house price based on size, location, and rooms.

---

Conclusion:

ANNs can model complex relationships by learning from data, forming the foundation for deep
learning models.

---

Keywords:

Neuron, Layers, Activation, Forward Pass, Backpropagation.

---

---

Q15. Explain different types of Neural Networks.

---

Definition:

Neural networks are categorized based on how neurons are connected and how data flows
through layers.

---

Types:

1. Feedforward Neural Network (FNN):

Data flows in one direction: input output.

No feedback loops.

2. Convolutional Neural Network (CNN):

Used for image processing.

Has convolution, pooling, and fully connected layers.

3. Recurrent Neural Network (RNN):

Has feedback connections.

Used for sequential data like text or speech.

4. Generative Adversarial Network (GAN):

Two networks (Generator + Discriminator) compete to generate realistic data.

5. Autoencoders:

Used for feature extraction and dimensionality reduction.

6. Radial Basis Function (RBF) Network:

Uses radial functions as activation for classification tasks.

---

Diagram:

Input Hidden Output (Feedforward)

Feedback (RNN)

---

Example:

CNN Image recognition

RNN Text prediction

GAN Image generation

---

Conclusion:

Each neural network type is specialized for different data types and problems, forming the core
of modern AI.

---

Keywords:

CNN, RNN, GAN, Autoencoder, Feedforward.

---

---

Q16. Explain the architecture of a Convolutional Neural Network (CNN).

---

Definition:

A CNN is a deep learning architecture designed primarily for image recognition and visual data
processing.

---

Layers in CNN:

1. Input Layer:

Takes image data (height × width × channels).

2. Convolution Layer:

Applies filters to extract features (edges, patterns).

3. Activation (ReLU):

Removes negative values improves non-linearity.

4. Pooling Layer:

Reduces dimensions using max or average pooling.

5. Fully Connected Layer:

Combines features to make final decision.

6. Output Layer:
Gives class probabilities using Softmax.

---

Diagram:

Input Image Conv ReLU Pool FC Output

---

Example:

CNN models like VGG16, ResNet are used in object detection, facial recognition, and handwriting
detection.

---

Conclusion:

CNNs are efficient for spatial data analysis and are widely used in all computer vision tasks.

---

Keywords:

Convolution, Pooling, ReLU, Softmax, Feature Maps.

---

---

Q17. Explain the architecture and working of Recurrent Neural Networks (RNN).

---

Definition:

RNNs are neural networks with feedback connections that can process sequential data such as
text or speech.

---

Step-by-Step Working:
1. Sequential Input:

Takes time-series or sequence data.

2. Hidden State (Memory):

Stores previous outputs to influence next inputs.

3. Weights Sharing:

Same weights used across time steps.

4. Backpropagation Through Time (BPTT):

Updates weights considering sequence dependencies.

5. Limitations:

Suffers from vanishing gradients hard to learn long dependencies.

---

Diagram:

X1 H1 Y1

X2 H2 Y2

X3 H3 Y3

---

Example:

Used in next-word prediction, time-series forecasting, and speech recognition.

---

Conclusion:
RNNs efficiently capture sequential patterns, forming the base for LSTM and GRU models.

---

Keywords:

Sequence, Hidden State, Memory, BPTT, Temporal Data.

---

---

Q18. Explain Keras architecture and its workflow.

---

Definition:

Keras is a high-level deep learning framework built on TensorFlow used for building and training
neural networks easily.

---

Workflow Steps:

1. Import Libraries:

import keras or from tensorflow import keras.

2. Load Dataset:

Example: MNIST, CIFAR-10.

3. Model Building:

Sequential model or Functional API.

4. Add Layers:

Dense, Conv2D, LSTM, Dropout etc.

5. Compile Model:

Choose optimizer, loss, and metrics.

6. Train Model:

.fit() method for training.

7. Evaluate & Predict:

.evaluate() and .predict() for testing and predictions.

---

Diagram:

Data Model Compile Train Evaluate Predict

---

Example:

Building a CNN using Keras to classify handwritten digits from MNIST dataset.

---

Conclusion:

Keras simplifies deep learning with a clean API, modular design, and TensorFlow backend for
flexibility.

---

Keywords:

Sequential API, [Link](), [Link](), TensorFlow Backend.

---

---
Q19. Explain TensorFlow architecture.

---

Definition:

TensorFlow is an open-source deep learning framework developed by Google for training and
deploying machine learning models.

---

Architecture Components:

1. Tensor:

Multi-dimensional array representing data.

2. Graph:

Represents computation flow.

3. Session:

Executes graph operations.

4. Variable:

Holds parameters of the model.

5. Operation (Op):

Represents a computation (add, multiply).

6. Eager Execution:

Immediate evaluation of operations (modern mode).

7. TensorFlow Serving / Lite:

For deploying models on servers or mobile devices.

---

Diagram:

Tensor Graph Session Execution Output

---

Example:

Training a CNN for object detection using TensorFlow and exporting the model using TensorFlow
Lite.

---

Conclusion:

TensorFlow provides flexibility and scalability for building, training, and deploying deep learning
models.

---

Keywords:

Tensor, Graph, Session, Variable, TensorFlow Lite.

---

---

Q20. Explain PyTorch architecture and its advantages.

---

Definition:

PyTorch is a deep learning framework developed by Facebook, known for dynamic computation
graphs and easy debugging.

---
 Architecture Components:

1. Tensors:

Multi-dimensional arrays (like NumPy but GPU-supported).

2. Autograd:

Automatic differentiation for backpropagation.

3. NN Module:

Provides ready-made layers and loss functions.

4. Optimizer Module:

Provides Adam, SGD, RMSProp optimizers.

5. Dynamic Computation Graphs:

Graphs created at runtime easier debugging.

6. TorchScript:

Converts model into deployable form.

---

Diagram:

Input Model ([Link]) Loss Optimizer Output

---

Example:

Training a CNN for image classification using PyTorch Lightning framework.

---

Conclusion:

PyTorch’s flexibility, GPU support, and dynamic graph make it ideal for research and fast
development.

---

Keywords:

Autograd, [Link], Tensors, Dynamic Graph, GPU.

---

---

UNIT–3 Complete (Q14–Q20)

ANN Architecture
Types of Neural Networks
CNN
RNN
Keras Workflow
TensorFlow Architecture
PyTorch Architecture
UNIT – 4: Deep Architectures (CNN, RNN, LSTM, GRU)

---

Q21. Explain the architecture and working of a Convolutional Neural Network (CNN).

---

Definition:

A Convolutional Neural Network (CNN) is a deep learning model specially designed for analyzing
visual data such as images and videos.
It automatically detects important features like edges, textures, and patterns.

---

Step-by-Step Working:

1. Input Layer:

Takes image data (e.g., 28×28 pixels × 3 color channels).

2. Convolution Layer:

Applies filters (kernels) to extract features like edges, curves.

Output = feature maps.

3. Activation Function (ReLU):

Removes negative values, introducing non-linearity.

4. Pooling Layer:

Reduces spatial size (max/average pooling) reduces computation.

5. Flattening:

Converts 2D feature maps into a 1D vector.

6. Fully Connected (Dense) Layer:

Combines all features for classification.

7. Output Layer:

Produces final result (class probabilities using Softmax).

---

Diagram:

Input Image Conv Layer ReLU Pooling Flatten Fully Connected Output

---

Example:

Used in object detection (YOLO), face recognition, and handwriting digit classification (MNIST).
---

Conclusion:

CNNs are powerful in capturing spatial relationships in images and are the backbone of
computer vision.

---

Keywords:

Convolution, Pooling, Feature Map, ReLU, Softmax, Flattening.

---

---

Q22. Explain different layers used in CNN.

---

Definition:

Each layer in CNN has a specific purpose in transforming the image into meaningful features for
classification.

---

Types of Layers:

1. Convolution Layer:

Extracts local patterns.

Filter slides over image and calculates dot products.

2. ReLU Layer:

ReLU(x) = max(0, x) Removes negative values.

3. Pooling Layer:

Reduces feature size using Max/Avg pooling.

4. Fully Connected Layer:

Combines features for final output.

5. Dropout Layer:

Randomly drops some neurons to avoid overfitting.

6. Softmax Layer:

Converts raw scores to class probabilities.

---

Diagram:

Conv ReLU Pool FC Softmax

---

Example:

In LeNet-5, 2 convolution layers + 2 pooling layers + 1 FC layer are used.

---

Conclusion:

Each layer plays a vital role — from extracting features to classification, enabling CNNs to learn
hierarchical patterns.

---

Keywords:

Convolution, Pooling, Dropout, ReLU, Softmax.

---
---

Q23. Explain the architecture and working of a Recurrent Neural Network (RNN).

---

Definition:

RNNs are a type of neural network where connections between nodes form a directed cycle,
allowing information to persist across time steps.

---

Step-by-Step Working:

1. Sequential Input:

Takes input data in time steps (t t, …

, t ).

2. Hidden State:

Stores previous information (memory) and passes it to next step.

3. Weight Sharing:

Same weights are used at every time step.

4. Output Generation:

Each hidden state produces an output based on current and past data.

5. Backpropagation Through Time (BPTT):

Updates weights for all time steps simultaneously.

6. Limitations:

Suffers from vanishing gradients can’t handle long sequences.

---

Diagram:

X1 H1 Y1

X2 H2 Y2

X3 H3 Y3

---

Example:

Text prediction (next-word prediction), stock price forecasting, and speech recognition.

---

Conclusion:

RNNs capture temporal patterns but struggle with long dependencies, leading to the
development of LSTM and GRU.

---

Keywords:

Hidden State, Sequence, Memory, Temporal Data, BPTT.

---

---

Q24. Explain the architecture and working of Long Short-Term Memory (LSTM) network.

---

Definition:

LSTM is a special kind of RNN capable of learning long-term dependencies using memory cells
and gates.

---

Step-by-Step Working:
1. Cell State:

Acts as long-term memory.

Information flows through the cell with minimal changes.

2. Forget Gate:

Decides which information to remove.

f = σ(Wf·[h x , ]+ bf)

3. Input Gate:

Decides which new information to store.

i = σ(Wi·[h x , ]+ bi)

4. Cell Update:

C = f *C + i *C̃

5. Output Gate:

Decides the output for the current step.

o = σ(Wo·[h x , ]+ bo)

6. Final Output:

h = o * tanh(C )

---

Diagram (Simplified LSTM Cell):

Cell State Ct

Forget Input Output Gates

 Update & Produce Hidden State

---

Example:

Used in text translation, sentiment analysis, and speech recognition (Google Voice).

---

Conclusion:

LSTMs effectively remember long-term sequences and solve the vanishing gradient problem.

---

Keywords:

Forget Gate, Input Gate, Output Gate, Memory Cell, Sequence Learning.

---

---

Q25. Explain the architecture and working of GRU (Gated Recurrent Unit).

---

Definition:

GRU is a simplified version of LSTM that combines the forget and input gates into a single
update gate, reducing complexity.

---

Step-by-Step Working:

1. Update Gate (z ):

Controls how much past information to retain.

2. Reset Gate (r ):
Controls how much past memory to forget.

3. New Memory Content:

h̃ = tanh(W·[r *h x ])
,

4. Final Hidden State:

h = (1 - z )*h + z *h̃

5. Advantages over LSTM:

Fewer parameters faster training.

Works well for moderate sequence lengths.

---

Diagram (Simplified GRU Cell):

Reset Update

Combine Output h

---

Example:

Used in speech-to-text and video caption generation.

---

Conclusion:

GRUs are lightweight alternatives to LSTMs with similar performance but faster convergence.

---
 Keywords:

Reset Gate, Update Gate, Hidden State, Sequence Data.

---

---

Q26. Differentiate between RNN, LSTM, and GRU.

---

Comparison Table:

Feature RNN LSTM GRU

Gradient Problem Yes Solved Solved

Gates Used None 3 (Forget, Input, Output) 2 (Reset, Update)
Memory Cell No Yes Yes
Complexity Low High Moderate
Training Time Fast Slow Faster
Accuracy Medium High High
Use Case Short sequences Long-term memory Medium sequences

---

Conclusion:

RNNs are simple but limited; LSTM and GRU are advanced versions that handle long-term
dependencies efficiently.

---

Keywords:

Gates, Memory, Sequence, Gradient, Performance.

---

---

UNIT–4 Complete (Q21–Q26)

CNN Working
CNN Layers
RNN
 LSTM
GRU
Comparison Table
UNIT – 5: Applications of Deep Learning (GAN, Autoencoders, NLP, Transfer Learning,
Reinforcement Learning)

---

Q27. Explain the architecture and working of Generative Adversarial Networks (GANs).

---

Definition:

A Generative Adversarial Network (GAN) is a deep learning model consisting of two networks —
a Generator and a Discriminator — that compete with each other to create realistic synthetic
data.

---

Step-by-Step Working:

1. Generator (G):

Takes random noise (z) as input.

Generates fake data samples similar to real data.

2. Discriminator (D):

Takes both real and fake samples.

Tries to classify them as real or fake.

3. Adversarial Training:

Generator improves to fool Discriminator.

Discriminator improves to detect fakes.

4. Objective:

Minimize Generator loss, maximize Discriminator accuracy.

5. Result:

Generator eventually produces highly realistic outputs.

---

Diagram:

Noise (z) Generator Fake Image Discriminator Real / Fake

Real Data --------------------

---

Example:

DeepFake videos

Generating new human faces ([Link])

Art or music generation

---

Conclusion:

GANs are powerful generative models that can create realistic synthetic data and are widely used
in AI creativity and image synthesis.

---

Keywords:

Generator, Discriminator, Adversarial Training, Fake Data, Real Data.

---

---

Q28. Explain the architecture and working of Autoencoders.

---
 Definition:

An Autoencoder is a type of neural network that learns to compress (encode) data into a smaller
representation and then reconstruct (decode) it back to the original form.

---

Step-by-Step Working:

1. Encoder:

Compresses input data into a lower-dimensional form (latent space).

2. Latent Space (Code):

The compressed representation of data.

3. Decoder:

Reconstructs data from the latent representation.

4. Loss Function:

Measures difference between input and reconstructed output (MSE loss).

5. Training:

Minimize reconstruction loss using backpropagation.

---

Diagram:

Input Encoder Latent Space Decoder Reconstructed Output

---

Example:
Noise reduction in images

Dimensionality reduction

Feature extraction for anomaly detection

---

Conclusion:

Autoencoders are unsupervised neural networks used for compression and noise removal.

---

Keywords:

Encoder, Decoder, Latent Space, Reconstruction Loss, Unsupervised.

---

---

Q29. Explain Variational Autoencoder (VAE).

---

Definition:

A Variational Autoencoder (VAE) is an advanced type of autoencoder that learns probability

distributions instead of fixed encodings, allowing it to generate new data.

---

Step-by-Step Working:

1. Encoder:

Maps input x to mean (μ) and variance (σ²).

2. Sampling:

Samples latent vector z = μ + σ·ε (ε ~ Normal(0,1)).

3. Decoder:

Reconstructs data from z.

4. Loss Function:

Combines reconstruction loss + KL divergence (for regularization).

5. Output:

Generates realistic samples from learned distribution.

---

Diagram:

Input Encoder (μ, σ) Sampling (z) Decoder Output

---

Example:

Generating synthetic medical images

Creating new faces or 3D models

---

Conclusion:

VAEs combine generative power with stability, providing control over data generation unlike
traditional autoencoders.

---

Keywords:

Latent Space, Sampling, KL Divergence, Probabilistic Model.

---
---

Q30. Explain Natural Language Processing (NLP) and its applications.

---

Definition:

Natural Language Processing (NLP) is a branch of AI that helps computers understand, interpret,
and generate human language.

---

Major Components:

1. Tokenization:

Breaking text into words or tokens.

2. Stemming and Lemmatization:

Reducing words to their root form.

3. Stop Word Removal:

Removing common words (is, the, of, etc.).

4. Vectorization:

Converting text to numerical data (TF-IDF, Word2Vec).

5. Model Building:

Using RNN, LSTM, or Transformers for understanding text.

6. Evaluation:

Measuring model accuracy using metrics like BLEU or F1.

---

Diagram:

Text Tokenization Vectorization Model Output

---

Example:

Google Translate

Chatbots

Sentiment Analysis (positive/negative reviews)

---

Conclusion:

NLP bridges human language and computers, making tasks like translation, summarization, and
sentiment detection possible.

---

Keywords:

Tokenization, Vectorization, Word2Vec, RNN, Transformer, Sentiment.

---

---

Q31. Explain Word Embeddings (Word2Vec / GloVe).

---

Definition:

Word embeddings are vector representations of words that capture their meaning, context, and
relationship with other words.

---
 Working:

1. Word2Vec:

Based on two models:

CBOW: predicts a word from surrounding context.

Skip-Gram: predicts surrounding words from a target word.

2. GloVe (Global Vectors):

Learns word relationships based on global word co-occurrence statistics.

3. Similarity Measurement:

Cosine similarity used to find related words.

4. Result:

Similar words have closer vectors in space.

---

Diagram:

Word Embedding Layer Vector (Continuous Space)

---

Example:

king - man + woman ≈ queen

Used in chatbots, sentiment models, and search engines.

---

Conclusion:
Word embeddings enable deep learning models to understand the semantic meaning of
language effectively.

---

Keywords:

CBOW, Skip-Gram, GloVe, Cosine Similarity, Embedding Vector.

---

---

Q32. Explain Transfer Learning in Deep Learning.

---

Definition:

Transfer Learning is a technique where a model trained on one task is reused (fine-tuned) for
another related task.

---

Step-by-Step Process:

1. Pretraining:

Use large datasets (e.g., ImageNet) to train base model.

2. Feature Extraction:

Use pretrained layers as fixed feature extractors.

3. Fine-Tuning:

Retrain only last few layers for new task.

4. Advantages:

Saves time and computation.

Works well even with small datasets.

---

Diagram:

Pretrained Model Remove Last Layer Add New Layer Fine-Tune New Task Output

---

Example:

Using VGG16 trained on ImageNet for medical image classification.

---

Conclusion:

Transfer learning allows deep models to reuse learned knowledge, reducing cost and improving
performance.

---

Keywords:

Pretrained, Fine-tuning, Feature Extraction, Reuse, ImageNet.

---

---

Q33. Explain Reinforcement Learning and its components.

---

Definition:

Reinforcement Learning (RL) is a learning technique where an agent learns to take actions in an
environment to maximize reward.

---

Components:
1. Agent: Learns and makes decisions.

2. Environment: The world where the agent operates.

3. State (S): Current situation of the agent.

4. Action (A): Possible moves taken by agent.

5. Reward (R): Feedback for action taken.

6. Policy (π): Strategy for choosing actions.

---

Diagram:

Agent Action Environment Reward Agent

---

Example:

AlphaGo game AI

Self-driving cars

Robotic arm control

---

Conclusion:

Reinforcement Learning teaches systems to make decisions autonomously through experience

and feedback.

---

Keywords:

Agent, Environment, Reward, Policy, Q-Learning.

---

---

Q34. Explain Q-Learning Algorithm in Reinforcement Learning.

---

Definition:

Q-Learning is a model-free RL algorithm that learns the value of actions to maximize long-term
rewards.

---

Algorithm Steps:

1. Initialize Q(s, a) = 0 for all states and actions.

2. Choose an action using ε-greedy policy.

3. Perform action observe reward (R) and next state (S’).

4. Update Q-value:

Q(s, a) = Q(s, a) + α [R + γ * max Q(s’, a’) - Q(s, a)]

where
α = learning rate, γ = discount factor.

5. Repeat until convergence.

---

Diagram:

State Action Reward Update Q-Table Next State

---

Example:

Training a robot to navigate a maze efficiently.

---

Conclusion:

Q-Learning helps agents learn optimal policies without needing a model of the environment.

---

Keywords:

Q-Value, Policy, Reward, Learning Rate, Discount Factor.

---

---

Q35. Explain Applications of Deep Learning in real life.

---

Applications:

1. Healthcare:

Disease detection, X-ray analysis.

2. Finance:

Fraud detection, stock predictions.

3. Agriculture:

Crop health monitoring using drones.

4. Transportation:

Self-driving cars and route optimization.

5. Security:
Face recognition and anomaly detection.

6. Entertainment:

Music generation, personalized recommendations.

7. Language:

Speech recognition, translation (NLP).

---

Diagram:

Deep Learning
Vision
Speech
Text
Robotics
Healthcare

---

Example:

ChatGPT (language model)

Tesla Autopilot

Google Photos automatic tagging

---

Conclusion:

Deep Learning is transforming every domain by providing automation, intelligence, and better
decision-making.

---

Keywords:
Automation, Vision, NLP, AI Applications, Healthcare.

---

---

UNIT–5 Complete (Q27–Q35)

GANs
Autoencoders
VAEs
NLP
Word Embeddings
Transfer Learning
Reinforcement Learning + Q-Learning
Real-life Applications