Deep Learning – Topper Level Exam Notes (B.
Tech
III Year, II Sem)
Comprehensive 10-mark answers with definitions, diagrams (ASCII), equations, steps, and key
points for high scoring.
UNIT I – Neural Foundations & Perceptron
Q1. Biological Neuron: Structure, Functioning & ANN Inspiration
Structure: Dendrites (input), Soma (integration), Axon (output), Synapse (connection).
Neat Diagram (ASCII):
Dendrites → [ Soma ] → Axon → Terminals → Synapse → Next neuron
Functioning:
• Reception: Dendrites receive electrochemical signals
• Integration: Soma sums inputs
• Threshold: Fires if potential exceeds threshold
• Transmission: Signal travels along axon
• Synaptic transfer: Neurotransmitters pass signal forward
ANN Inspiration: Inputs ↔ dendrites, weights ↔ synapses, summation ↔ soma, activation ↔
threshold, output ↔ axon.
Key Equation: y = f(Σ w■x■ + b)
Q2. Computational Unit in ANN
An artificial neuron computes weighted sum of inputs plus bias and passes it through activation
function.
• Inputs: x1, x2, …, xn
• Weights: w1, w2, …, wn
• Summation: net = Σ w■x■ + b
• Activation: y = f(net)
• Output: forwarded to next layer
Q3. McCulloch–Pitts Neuron
Binary threshold neuron model.
• Assumes binary inputs and outputs
• Uses threshold function
• Implements logic gates
Examples: AND, OR, NOT using weights and threshold.
Q4. Linear Perceptron
Single-layer classifier performing linear decision boundary.
Decision boundary: w·x + b = 0
Classifies data into two linearly separable classes.
Q5. Perceptron Learning Algorithm (PLA)
• Initialize weights
• For each training example, compute output
• Update rule if misclassified: w = w + η y x
• Repeat until convergence
Q6. Perceptron Convergence Theorem
If data is linearly separable, PLA converges in finite steps.
• Requires linear separability
• Learning rate positive
• Finite convergence guarantee
UNIT II – Neural Networks & Learning
Q1. Feedforward Neural Network (FNN)
Multi-layer network with no cycles.
• Layers: Input, Hidden, Output
• Unidirectional signal flow
• Used for regression/classification
Q2. Multilayer Perceptron (MLP)
FNN with one or more hidden layers enabling non-linear learning.
• Hidden layers introduce non-linearity
• Activation functions: ReLU, Sigmoid, Tanh
• Universal function approximator
Q3. Backpropagation
Uses chain rule to compute gradients.
• Forward pass computes output
• Backward pass computes gradients
• Weights updated via gradient descent
Q4. Empirical Risk Minimization (ERM)
Minimizes average loss over training data.
• Loss function measures error
• Hypothesis space defines models
• Goal: generalization to unseen data
Q5. Regularization
• L1: promotes sparsity
• L2: penalizes large weights
• Dropout: random neuron removal
• Early stopping: prevent overfitting
Q6. Autoencoders & Training Challenges
Autoencoder: Encoder → Latent space → Decoder
Challenges: Vanishing gradients, exploding gradients, overfitting, high computation.
UNIT III – Optimization & Training Issues
Q1. Advanced Optimizers
• Adagrad: adaptive learning rate
• RMSProp: decaying average
• Adam: momentum + adaptive learning
Q2. Second-order Methods
Use curvature information (Hessian).
• Newton's method
• Quasi-Newton (BFGS)
• Faster convergence but expensive
Q3. Saddle Point Problem
Flat regions where gradient ≈ 0 but not minima.
• Common in high dimensions
• Slows training
Q4. Overcoming Saddle Points
• Use momentum-based optimizers
• Add noise
• Adaptive learning rates
Q5. Dropout
Randomly drops neurons during training.
• Prevents overfitting
• Improves generalization
Q6. Batch Normalization
Normalizes inputs of each layer.
• Stabilizes training
• Reduces internal covariate shift
• Helps gradient flow
UNIT IV – CNNs, RNNs & Generative Models
Q1. RNN
Processes sequential data using hidden state.
• Maintains memory
• Used in NLP, speech
• Issue: vanishing gradient
Q2. CNN
• Convolution: feature extraction
• Pooling: dimension reduction
• Fully connected: classification
Q3. RNN vs LSTM vs CNN
• RNN: simple sequential
• LSTM: handles long-term dependencies
• CNN: spatial features
Q4. Bidirectional RNN
Processes sequence in both directions.
Q5. BPTT & GRU
BPTT unfolds RNN; GRU uses gates (update/reset).
Q6. RBM & DBM
Energy-based models for unsupervised learning.
UNIT V – Modern Deep Learning
Q1. Variational Autoencoder (VAE)
Encodes input into probabilistic latent space.
• Uses KL divergence
• Generative model
Q2. VAE Working
Encoder → latent distribution → sampling → decoder.
Q3. Transformers
Use self-attention instead of recurrence.
• Parallel processing
• Better long-range dependency handling
Q4. GPT
Transformer-based language model.
• Pretraining on large data
• Fine-tuning for tasks
Q5. AI Trends
VAEs, Transformers, GPT revolutionizing AI.
Q6. Multimodal AI
Combines vision, text, speech using transformers.