DEEP LEARNING(MSDS202)

MODULE-5
Practical Methodology: Performance Metrics, Default Baseline Models,
Determining Whether to Gather More Data, Selecting Hyperparameters,
Debugging Strategies, Example: Multi-Digit Number Recognition.
Applications: Vision, NLP, Speech.

VTU [Link] (Data Science / AI & ML) Notes

Module: Practical Methodology and Applications of Deep Learning

Prepared as per VTU [Link] CBCS Syllabus (10–15 Marks

MODULE: Practical Methodology and Applications of Deep

Introduction

Deep learning models achieve high performance only when they are
designed, trained, and evaluated using a proper practical methodology.
Practical methodology involves selecting suitable performance metrics,
choosing baseline models, tuning hyperparameters, debugging errors, and
deciding whether additional data is required.

These practices help build models that are accurate, efficient, robust,
and generalize well to unseen data.

Learning Objectives

After studying this module, students should be able to:

 Explain performance evaluation metrics.

 Describe baseline models.

 Decide when additional data is needed.

 Explain hyperparameter selection techniques.

 Discuss debugging strategies.

 Explain multi-digit number recognition.

 Describe applications of deep learning in Vision, NLP, and Speech.

Definition

A Performance Metric is a quantitative measure used to evaluate how

The choice of metric depends on the problem (classification, regression,

Need for Performance Metrics

 Evaluate model quality.

 Compare different models.

 Detect overfitting and underfitting.

 Improve decision making.

 Select the best algorithm.

Classification Metrics

1. Accuracy

Measures the percentage of correctly classified samples.

Formula

[
\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}
]

Where:

 TP = True Positive

 TN = True Negative

 FP = False Positive

 FN = False Negative

2. Precision

Measures the proportion of predicted positives that are actually positive.

3. Recall (Sensitivity)

Measures the proportion of actual positives correctly identified.

[
\text{Recall}=\frac{TP}{TP+FN}
]

4. F1-Score

Harmonic mean of Precision and Recall.

[
F1=\frac{2PR}{P+R}
]

5. Confusion Matrix

Shows prediction results in tabular form.

Actual / Positi Negati

Positive TP FN

Negative FP TN

Regression Metrics

Mean Squared Error (MSE)

[
MSE=\frac{1}{N}\sum(y-\hat y)^2
]

Root Mean Squared Error (RMSE)

Mean Absolute Error (MAE)

[
MAE=\frac{1}{N}\sum|y-\hat y|
]

Advantages

 Objective evaluation.

 Easy model comparison.

 Measures prediction quality.

2. Default Baseline Models

Definition

A Baseline Model is the simplest model used as a reference for

A new model should perform better than the baseline.

Purpose

 Benchmark performance.

 Detect implementation errors.

 Measure improvement.

Types

Random Predictor

Outputs random predictions.

Majority Class Predictor

Always predicts the most frequent class.

Uses simple linear relationships.

Logistic Regression

Baseline for classification tasks.

Advantages

 Easy to implement.

 Fast training.

 Provides performance reference.

Disadvantages

 Low accuracy.

 Limited learning capability.

3. Determining Whether to Gather More Data

Why More Data?

Deep learning models improve with larger and more diverse datasets.

Indicators That More Data Is Needed

High Variance (Overfitting)

Training accuracy is high but testing accuracy is low.

Solution: Gather more training data.

Poor Generalization

Model performs poorly on unseen data.

Imbalanced Dataset
One class has significantly fewer samples.

High Noise

Data contains many incorrect labels.

Benefits of More Data

 Reduces overfitting.

 Improves generalization.

 Increases accuracy.

 Learns better feature representations.

Limitations

 Costly to collect.

 Time-consuming.

 Storage requirements increase.

4. Selecting Hyperparameters

Definition

Hyperparameters are parameters set before training begins and are not
learned automatically during training.

Examples

 Learning rate

 Batch size

 Number of epochs

 Number of hidden layers

 Number of neurons

 Activation function

 Optimizer
  Dropout rate

Hyperparameter Selection Methods

Manual Search

Trial-and-error approach.

Grid Search

Tests every combination of predefined values.

Advantages

 Finds optimal combination.

Disadvantages

 Computationally expensive.

Random Search

Randomly samples combinations.

Advantages

 Faster than Grid Search.

Bayesian Optimization

Uses previous results to guide future searches.

Advantages

 Efficient.

 Faster convergence.

Hyperparameter Tuning Process

Select Hyperparameters

Train Model

│
Evaluate Performance

Adjust Hyperparameters

Repeat

5. Debugging Strategies

Definition

Debugging is the process of identifying and correcting errors that reduce

Common Problems

Underfitting

 High training error.

 High testing error.

Solutions

 Increase model complexity.

 Train longer.

 Add features.

Overfitting

 Low training error.

 High testing error.

Solutions

 Dropout.

 Regularization.

 Early stopping.

 More training data.

Gradients become very small.

Solutions

 ReLU activation.

 LSTM.

 Proper weight initialization.

Exploding Gradient

Gradients become very large.

Solutions

 Gradient clipping.

 Lower learning rate.

Data Errors

 Incorrect labels.

 Missing values.

 Duplicate records.

Debugging Checklist

 Verify dataset quality.

 Check preprocessing.

 Inspect learning curves.

 Validate loss function.

 Test different hyperparameters.

 Compare with baseline model.

6. Example: Multi-Digit Number Recognition

Definition
Multi-digit number recognition is the task of recognizing complete
numbers containing multiple digits from an image.

Example:

Input Image:

5839

Output:

5839

Architecture

Input Image

CNN Feature Extraction

Sequence Model (LSTM/RNN)

Digit Prediction

Recognized Number

Working

1. Input image is captured.

2. CNN extracts visual features.

3. LSTM learns digit sequence.

4. Output layer predicts each digit.

5. Entire number is recognized.

Applications

 Postal code recognition.

 Bank cheque processing.

 Utility meter reading.

 Passport and document processing.

Advantages

 High accuracy.

 Automatic feature extraction.

 Handles variable-length numbers.

7. Applications of Deep Learning

A. Vision

Definition

Computer Vision enables computers to understand images and videos

Applications

 Face Recognition

 Image Classification

 Object Detection

 Image Segmentation

 Medical Imaging

 Self-driving Cars

 Satellite Image Analysis

Common Models

 CNN

 ResNet

 YOLO
  Vision Transformer (ViT)

B. Natural Language Processing (NLP)

Definition

NLP enables computers to understand and generate human language.

Applications

 Machine Translation

 Sentiment Analysis

 Chatbots

 Text Summarization

 Question Answering

 Spam Detection

 Document Classification

Common Models

 RNN

 LSTM

 GRU

 Transformer

 BERT

 GPT

C. Speech Processing

Definition

Speech processing enables computers to recognize, understand, and

Applications
  Speech Recognition

 Voice Assistants

 Speaker Identification

 Voice Search

 Speech Translation

 Text-to-Speech

Common Models

 RNN

 LSTM

 GRU

 Transformer

 Conformer

Comparison of Applications

Applicati
Input Common Models Example
on

Images/
Vision CNN, ResNet, ViT Face Recognition
Videos

LSTM, Transformer, Machine

LSTM, GRU,
Speech Audio Voice Assistant
Conformer

Advantages of Practical Methodology

 Improves model accuracy.

 Reduces overfitting.

 Optimizes computational resources.

 Enhances model reliability.

 Supports reproducible experiments.

 Hyperparameter tuning is time-consuming.

 Requires computational resources.

 Data collection can be expensive.

 Debugging deep models is complex.