🧹 ML & Deep Learning — Week 2

Data Pre-processing — Complete Exam Notes

Dr. Nabanita Choudhury • Symbiosis International (Deemed University)

PART 1: WHAT IS DATA PRE-PROCESSING?

Definition: The process of transforming raw, messy data into a clean, structured, and meaningful format suitable
for ML models.

Why is it needed? Real-world data is incomplete, noisy, inconsistent, and unscaled. Feeding such data directly
into models leads to poor learning, biased predictions, and unstable performance.

💡 Data pre-processing is a MANDATORY first step in every ML pipeline — garbage in =

garbage out!

Objectives of Data Pre-processing

Objective Why it Matters
Improve data quality Ensures reliable and accurate input for models
Reduce noise & inconsistencies Prevents misleading patterns being learned
Handle missing/incorrect values Avoids errors during model training
Convert to numerical form ML algorithms only understand numbers
Scale features to comparable ranges Prevents features with large values dominating
Enhance model accuracy & Better real-world performance on unseen data
generalization

PART 2: THE 8 MAJOR STEPS OF DATA PRE-PROCESSING

💡 Memory trick for 8 steps: C-M-O-T-E-F-F-S → 'Clever ML Operators Transform Every Feature
For Success'

A. Data Cleaning
Removes errors and inconsistencies from raw data. The foundation of all pre-processing.
• Handling missing values
• Removing duplicate records
• Correcting data types
 • Eliminating outliers
• Fixing inconsistent labels (e.g., 'Male' vs 'M' vs 'male')

Impact: Prevents misleading learning and improves reliability.

B. Handling Missing Data

Missing values must be dealt with — leaving them causes errors or biased models.
Strategy When to Use Example
Deletion (row/column When missing % is very small Drop rows with NaN values
removal) (<5%)
Mean imputation Numerical data, no major Replace missing Age with avg age
outliers
Median imputation Numerical data with outliers Replace missing salary with median
salary
Mode imputation Categorical data Replace missing Gender with most
frequent value
Forward/Backward fill Time-series data Fill missing stock price with previous
day's value
Model-based imputation Complex datasets, small Use regression to predict missing
missing % values

C. Outlier Detection and Treatment

Outlier: An extreme value that differs significantly from the rest of the data.

Detection Method How it works

Z-Score If |Z| > 3, the point is an outlier. Uses mean and standard deviation.
IQR (Interquartile Range) Outlier if value < Q1 - 1.5×IQR or > Q3 + 1.5×IQR
Box Plot Visual method — points outside whiskers are outliers

Treatment When to Use

Remove Clearly erroneous values (e.g., age = 999)
Cap / Winsorization Replace outliers with boundary values (Q1 or Q3 limits)
Transform Apply log/sqrt to compress extreme values

Impact: Prevents model bias and instability.

D. Data Transformation (Scaling)

Converts features to a common scale so no single feature dominates model training.
 Method Formula Output Range Best For
Min-Max x' = (x - xmin) / (xmax - [0, 1] KNN, K-Means, Neural
Normalization xmin) Networks
Z-Score Z = (X - μ) / σ Mean=0, SD=1 SVM, PCA, Neural
Standardization Networks
Log Transformation x' = log(x) Compressed Skewed data, large value
range ranges
Power Transformation x' = x^n Varies Non-linear relationships

E. Encoding Categorical Data

ML models require numerical input — text categories must be converted to numbers.

Encoding Type How it Works When to Use Example

Label Encoding Assigns integer to each Ordinal data (ordered Low=0, Medium=1,
category (0,1,2...) categories) High=2
One-Hot Encoding Creates binary column for Nominal data (no Color: [Red, Blue,
each category order) Green] → 3 columns
Ordinal Encoding Assigns numbers Ordered categories Education:
respecting natural order School<UG<PG →
1,2,3

💡 Use One-Hot for categories with NO order (e.g., City, Gender). Use Label/Ordinal for ordered
categories (e.g., Grade, Size).

F. Feature Selection
Selecting only the most relevant features — removing irrelevant or redundant ones.
• Reduces dimensionality and computation time
• Improves model performance and reduces overfitting

Method Type Technique How it Works

Filter Methods Correlation, Chi-Square Rank features by statistical relevance — no
model needed
Wrapper Methods Forward/Backward Use model performance to iteratively
Selection add/remove features
Embedded Methods LASSO, Tree-based Feature selection happens during model
Importance training itself

G. Feature Engineering
Creating new, meaningful features from existing data to improve model performance.
• Extracting day/month/year from a date column
• Creating 'BMI' from height and weight columns
 • Combining 'Distance' and 'Time' to create 'Speed'

Impact: Enhances predictive power — often the most impactful step for model improvement.

H. Data Splitting
Divide dataset into separate subsets to enable fair model evaluation.
Split Typical Size Purpose
Training Set 60–80% Model learns patterns from this data
Validation Set (optional) 10–20% Tune hyperparameters during development
Testing Set 10–20% Final unbiased evaluation of model performance

Impact: Enables unbiased model evaluation and prevents overfitting.

PART 3: MIN-MAX NORMALIZATION — FORMULA & SOLVED

EXAMPLE

Formula
Min-Max Normalization Formula:
x' = (x - x_min) / (x_max - x_min)

Where: x = original value, x_min = minimum value in feature, x_max = maximum value in feature
Output range is always [0, 1]. The minimum value becomes 0, maximum becomes 1.

✅ Solved Example (From PDF — Exam-Ready)

Question: Normalize the following Age values using Min-Max Scaling:
Ages: A=18, B=20, C=22, D=24, E=26

Step 1: Find Min and Max

x_min = 18, x_max = 26, Range = 26 - 18 = 8

Step 2: Apply formula: x' = (x - 18) / 8

Student Age (x) Calculation Normalized Value (x')

A 18 (18-18)/8 = 0/8 0.00
B 20 (20-18)/8 = 2/8 0.25
C 22 (22-18)/8 = 4/8 0.50
D 24 (24-18)/8 = 6/8 0.75
 E 26 (26-18)/8 = 8/8 1.00

💡 Interpretation: Min value → 0, Max value → 1. All values proportionally spread between 0
and 1. Useful for KNN, K-Means, Neural Networks.

📝 Practice Numerical (Electricity Data)

Q: Normalize electricity consumption data: 120, 150, 180, 200, 220, 250, 275, 300, 320, 350 kWh

Step 1: x_min = 120, x_max = 350, Range = 350 - 120 = 230

Consumption (x) Calculation Normalized (x')

120 (120-120)/230 0.00
150 (150-120)/230 0.13
180 (180-120)/230 0.26
200 (200-120)/230 0.35
220 (220-120)/230 0.43
250 (250-120)/230 0.57
275 (275-120)/230 0.67
300 (300-120)/230 0.78
320 (320-120)/230 0.87
350 (350-120)/230 1.00

PART 4: Z-SCORE STANDARDIZATION — FORMULA & SOLVED

EXAMPLE

Formula
Z-Score Standardization Formula:
Z = (X - μ) / σ

Where: X = original value, μ (mu) = mean of feature, σ (sigma) = standard deviation

After standardization: Mean of Z-scores = 0, Standard deviation = 1

💡 Key difference from Normalization: Standardization does NOT restrict to [0,1]. It centers
data around 0. Better for data with outliers or Gaussian distribution.
✅ Solved Example (From PDF — Exam-Ready)
Question: Standardize quiz scores X = [50, 60, 70, 80, 90] using Z-Score

Step 1: Compute Mean (μ)

μ = (50 + 60 + 70 + 80 + 90) / 5 = 350 / 5 = 70

Step 2: Compute Standard Deviation (σ)

Formula: σ = √[ Σ(X - μ)² / n ]

X X-μ (X - μ)²
50 50 - 70 = -20 (-20)² = 400
60 60 - 70 = -10 (-10)² = 100
70 70 - 70 = 0 (0)² = 0
80 80 - 70 = +10 (+10)² = 100
90 90 - 70 = +20 (+20)² = 400
SUM — Σ = 1000

σ = √(1000/5) = √200 = 14.14

Step 3: Compute Z-Scores (Z = (X - 70) / 14.14)

X Z-Score Calculation Z-Score Interpretation

50 (50-70)/14.14 -1.41 1.41 std devs BELOW average
60 (60-70)/14.14 -0.71 0.71 std devs below average
70 (70-70)/14.14 0.00 Exactly AT the mean
80 (80-70)/14.14 +0.71 0.71 std devs above average
90 (90-70)/14.14 +1.41 1.41 std devs ABOVE average

Final Standardized Data: Z = [-1.41, -0.71, 0.00, +0.71, +1.41]

💡 Negative Z → below average. Positive Z → above average. Z=0 → exactly at mean.

📝 Practice Numerical (Electricity Standardization)

Q: Standardize electricity consumption X = [180, 220, 260, 300, 340] kWh

Step 1: μ = (180+220+260+300+340)/5 = 1300/5 = 260

X X-μ (X - μ)²
180 -80 6400
 220 -40 1600
260 0 0
300 +40 1600
340 +80 6400
SUM — 16000

Step 2: σ = √(16000/5) = √3200 = 56.57

Step 3: Apply Z = (X - 260) / 56.57

X Z-Score
180 (180-260)/56.57 = -1.41
220 (220-260)/56.57 = -0.71
260 (260-260)/56.57 = 0.00
300 (300-260)/56.57 = +0.71
340 (340-260)/56.57 = +1.41

Final: Z = [-1.41, -0.71, 0.00, +0.71, +1.41]

PART 5: NORMALIZATION vs STANDARDIZATION — KEY

COMPARISON

Feature Min-Max Normalization Z-Score Standardization

Formula x' = (x - xmin)/(xmax - xmin) Z = (X - μ) / σ
Output Range Always [0, 1] No fixed range; centered at 0
Mean after transform Not fixed Always 0
Std Dev after transform Not fixed Always 1
Effect of Outliers Sensitive — outliers compress More robust to outliers
other values
Best for When you need bounded range Data with Gaussian/normal
[0,1] distribution
Used with KNN, K-Means, Neural Networks SVM, PCA, Neural Networks,
KNN
Preserves distribution Yes Yes
shape?
 PART 6: MASTER QUICK REFERENCE TABLE — ALL 8 STEPS

S Name Purpose Example Technique Impact

t
e
p
A Data Cleaning Remove errors & Duplicate removal, Reliable learning
inconsistencies type correction
B Missing Data Fill or remove Mean/Median No training errors
Handling gaps imputation
C Outlier Treatment Handle extreme Z-score, IQR, Box Prevents model bias
values plot
D Data Scale features Min-Max, Z-Score Equal feature contribution
Transformation equally
E Encoding Convert text to One-Hot, Label ML-readable format
numbers Encoding
F Feature Selection Keep only useful Correlation, LASSO Reduces dimensionality
features
G Feature Create new Date extraction, BMI Boosts predictive power
Engineering features calc
H Data Splitting Divide train/test 80-20 or 70-10-20 Fair model evaluation
sets split

PART 7: RAPID FIRE Q&A — EXAM PREPARATION

Question Answer
What is data pre-processing? Transforming raw data into clean, structured, ML-
ready format
Why is pre-processing mandatory? Raw data is incomplete, noisy, inconsistent &
unscaled — causes poor model performance
Name 5 steps of data cleaning? Handle missing values, remove duplicates,
correct data types, eliminate outliers, fix
inconsistent labels
When to use Mean vs Median imputation? Mean = no outliers; Median = when outliers are
present
What is an outlier? An extreme value that significantly differs from the
rest of the data
How does Z-Score detect outliers? If |Z| > 3, the value is considered an outlier
What is IQR method for outliers? Outlier if < Q1 - 1.5*IQR or > Q3 + 1.5*IQR
Min-Max Normalization formula? x' = (x - xmin) / (xmax - xmin) → Output: [0,1]
Z-Score Standardization formula? Z = (X - μ) / σ → Output: Mean=0, SD=1
When to use Normalization vs Normalization: bounded range needed.
Standardization? Standardization: outliers present or Gaussian
data
What is Label Encoding? Assigns integer to each category (e.g., Low=0,
Medium=1, High=2)
What is One-Hot Encoding? Creates separate binary column for each category
(for nominal/no-order data)
What is Feature Selection? Selecting only relevant features and removing
redundant ones to reduce dimensionality
Difference: Feature Selection vs Feature Selection = remove existing features. Engineering
Engineering? = CREATE new features from existing data
Typical train/test split ratio? 80% training, 20% testing (or 70/20/10 with
validation)
Why normalize before using KNN? KNN uses distance — unscaled features with
large ranges will dominate distance calculations
Z-score of value exactly at the mean? Z=0
Negative Z-score means? The value is BELOW the average
What does winsorization do? Caps outliers at boundary values (Q1 or Q3 limits)
instead of removing them
Name 3 filter-based feature selection Correlation analysis, Chi-Square test, Information
methods? Gain

All the best for your exam! 🎯

Week 2 — Data Pre-processing | ML & Deep Learning | Dr. Nabanita Choudhury