Machine Learning Interview Notes –

Preprocessing Topics
✅ 1. Handling Missing Values (In-Depth)
1. Drop Rows or Columns
Use when missing data is very small or random.

Suitable for large datasets with few nulls.

Risk: You may lose useful data, leading to bias if not random.

# Drop rows
[Link]()

# Drop columns
[Link](axis=1)

2. Impute with Mean

Use when data is numerical and normally distributed.

Avoid if outliers are present.

df['Age'] = df['Age'].fillna(df['Age'].mean())

3. Impute with Median

Use when data has outliers or is skewed.

df['Age'] = df['Age'].fillna(df['Age'].median())

4. Impute with Mode

Best for categorical features.

df['City'] = df['City'].fillna(df['City'].mode()[0])
5. Fill with Constant
Use for categorical fields like Gender , City .

Helps preserve rows with a tag like 'Unknown'.

df['City'] = df['City'].fillna('Unknown')

6. KNN Imputer
Uses K nearest rows to fill missing value.

Good for numeric features, needs scaling.

from [Link] import KNNImputer

from [Link] import StandardScaler

scaler = StandardScaler()
df_scaled = scaler.fit_transform(df)

imputer = KNNImputer(n_neighbors=2)
df_imputed = imputer.fit_transform(df_scaled)

1. Label Encoding
Good for tree-based models

Not suitable for linear models (imposes order)

from [Link] import LabelEncoder

le = LabelEncoder()
df['City_Label'] = le.fit_transform(df['City'])

2. One-Hot Encoding
Suitable for nominal categorical features

df = pd.get_dummies(df, columns=['City'], drop_first=True)

3. Ordinal Encoding
Use for ordered categories like education level
 df['Education'] = df['Education'].map({
'High School': 0,
'Graduate': 1,
'Postgraduate': 2
})

✅ 3. Handling Outliers
1. IQR Method

Q1 = df['Age'].quantile(0.25)
Q3 = df['Age'].quantile(0.75)
IQR = Q3 - Q1

lower = Q1 - 1.5 * IQR

upper = Q3 + 1.5 * IQR

df_filtered = df[(df['Age'] >= lower) & (df['Age'] <= upper)]

2. Z-Score Method

from [Link] import zscore

df['zscore'] = zscore(df['Age'])
df_filtered = df[df['zscore'].abs() <= 3]

3. Trimming (Remove Outliers)

df_trimmed = df[(df['Feature'] >= lower_bound) & (df['Feature'] <= upper_bound)]

###

4. Use Robust Models

Tree models like Random Forest are naturally robust to outliers.

🧠 Interview Tips
Use mean only when no outliers and data is symmetric.

Use median when data has outliers.

KNN and Iterative Imputers use feature relationships.

Label Encoding can confuse linear models.

IQR is more robust than Z-score for skewed data.

from pathlib import Path

✅ 4. Handling Categorical Variables (In

Depth for Interviews)
Categorical variables are features that take on limited, discrete values. Examples include Gender, City,
Department, etc.

Since most machine learning algorithms can't work with text labels directly, we convert them into
numeric representations using encoding techniques.

🔸 Why Encode Categorical Data?

ML models expect numerical input.

Without encoding, models like Logistic Regression, SVM, and KNN will throw errors.

Encoding helps the model understand and work with non-numeric data.

✅ 1. Label Encoding
🔹 What it does:
Assigns each unique category a unique integer label.

🔹 Example:
City Encoded

Delhi 0

Mumbai 1

Chennai 2

✔ When to Use:
For tree-based models (e.g., Decision Tree, Random Forest, XGBoost)

When the feature has ordinal meaning (e.g., Small < Medium < Large)

⚠ When to Avoid:
For linear models (implies an artificial order)
✅ Code:
from [Link] import LabelEncoder

le = LabelEncoder()
df['City_Label'] = le.fit_transform(df['City'])

✅ 2. One-Hot Encoding
🔹 What it does:
Converts each category into a binary column (0 or 1)

Avoids imposing any ordinal relationship

🔹 Example:
City Delhi Mumbai Chennai

Delhi 1 0 0

Mumbai 0 1 0

✔ When to Use:
For nominal categorical data (no order)

Preferred for linear models, SVM, KNN

⚠ Caution:
Can cause dimensionality explosion if too many unique categories

✅ Code:
# One-hot encode and drop first to avoid dummy variable trap
df = pd.get_dummies(df, columns=['City'], drop_first=True)

✅ 3. Ordinal Encoding
🔹 What it does:
Converts categories with an inherent order into integers
🔹 Example:
Education Encoded

High School 0

Graduate 1

Postgraduate 2

✔ When to Use:
For ordered categorical features

Where the order carries meaning (e.g., experience levels)

✅ Code:
df['Education'] = df['Education'].map({
'High School': 0,
'Graduate': 1,
'Postgraduate': 2
})

🧠 Interview Questions
Q1: What is one-hot encoding?
A: One-hot encoding transforms a categorical column into multiple binary columns. Each column
represents a category, marked with 0 or 1.

Q2: Why can label encoding be problematic?

A: Label encoding introduces ordinal relationships. Linear models might assume Mumbai > Chennai >
Delhi , which may not be true and can bias results.

Q3: When would you use label encoding?

A: Label encoding is useful for tree-based models, which are not affected by label order. Also suitable
when categories are naturally ordered.
Q4: How do you handle a categorical feature with many unique
values (high cardinality)?
A: Options include:

Frequency Encoding

Target Encoding (with caution to avoid leakage)

Hash Encoding (in libraries like CategoryEncoders)

Dimensionality reduction before encoding

Q5: Can you combine multiple encoding techniques?

A: Yes. Often you apply label encoding to ordinal columns and one-hot encoding to nominal ones in the
same dataset.

✅ End of notes on Handling Categorical Variables.

📘 Feature Scaling for Machine Learning

(Interview Style)
Feature scaling is a crucial preprocessing technique used in many machine learning (ML) workflows. It
ensures that features contribute equally to model training, especially for algorithms sensitive to feature
magnitude.

✅ Why Feature Scaling?

Many ML algorithms are distance-based (e.g., KNN, SVM) or gradient-based (e.g., Logistic
Regression, Neural Networks).

Features with larger ranges can dominate smaller ones, skewing results.

Helps gradient descent converge faster.

Improves model stability and interpretability.

✅ 1. Standardization (Z-score Normalization)

🔹 Formula:
z = (x - mean) / std

Transforms data to have zero mean and unit variance.

Useful when data is normally distributed.

🔹 Code Example:
from [Link] import StandardScaler

scaler = StandardScaler()
df_scaled = scaler.fit_transform(df[['feature1', 'feature2']])

✔ When to Use:
Data is approximately Gaussian.

Algorithms like:

Linear / Logistic Regression

SVM

KNN

PCA

Neural Networks

✅ 2. Min-Max Normalization
🔹 Formula:
x_scaled = (x - min) / (max - min)

Scales features to a fixed [0, 1] range.

🔹 Code Example:
from [Link] import MinMaxScaler

scaler = MinMaxScaler()
df_scaled = scaler.fit_transform(df[['feature1', 'feature2']])

✔ When to Use:
When you need bounded features.

Ideal for:

Neural Networks

Image pixel data (e.g., 0–255)

📌 Key Points to Remember

❗ Always fit the scaler on training data only, then transform both train and test sets.
 ❌ Never scale the target variable, unless the model specifically requires it (e.g., regression with
neural networks).

🌳 No need to scale for tree-based models like:

Decision Trees

Random Forest

XGBoost

📊 PCA needs scaling as it is variance and distance-based.

⚡ Faster convergence with scaled features in gradient descent.

🧠 Interview Questions
Q1: Why is feature scaling important?
A: To ensure features contribute equally to the model, improve training stability, and accelerate
convergence in optimization.

Q2: Which models are sensitive to feature scaling?

A: SVM, KNN, Linear Regression, Logistic Regression, PCA, and Neural Networks.

Q3: When should you use standardization over normalization?

A: Use standardization when data follows a Gaussian distribution. Use normalization when features
need to be in a fixed [0,1] range.

Q4: Do tree-based models require feature scaling?

A: No. Tree-based models are not affected by feature magnitudes.

📎 Extra Tips
🔍 Use RobustScaler (not covered here) if data has many outliers — it uses median and IQR.

📦 Save your scaler using joblib or pickle if deploying the model.

🔄 Avoid data leakage: Never fit scaler on test data!

📘 Handling Imbalanced Data (Interview

Style Guide)
In classification problems, when one class significantly outnumbers the others, the dataset is said to
be imbalanced.

📌 Example:
Fraud Detection – 98% Legitimate, 2% Fraudulent.

🔸 Why Is It a Problem?
Models may predict the majority class all the time and still show high accuracy.

Accuracy becomes misleading in imbalanced settings.

 Model performance on the minority class suffers (e.g., fraud, disease, spam).

✅ 1. Resampling Techniques
🔹 A. Undersampling (Reduce Majority Class)
Randomly removes samples from the majority class.

⚠ Risk: Losing valuable data and diversity.

from imblearn.under_sampling import RandomUnderSampler

rus = RandomUnderSampler()
X_res, y_res = rus.fit_resample(X, y)

🔹 B. Oversampling (Duplicate Minority Class)

Randomly duplicates samples from the minority class.

⚠ Risk: Overfitting to repeated samples.

from imblearn.over_sampling import RandomOverSampler

ros = RandomOverSampler()
X_res, y_res = ros.fit_resample(X, y)

✅ 2. SMOTE (Synthetic Minority Oversampling

Technique)
Creates synthetic samples of the minority class using KNN logic.

Better than naive oversampling.

from imblearn.over_sampling import SMOTE

smote = SMOTE()
X_res, y_res = smote.fit_resample(X, y)

✅ Works well for continuous features

⚠ Not suitable for categorical features unless modified

✅ 3. Class Weight Adjustment

Assigns higher weights to minority class during training.

Avoids changing the dataset size.

 from sklearn.linear_model import LogisticRegression

model = LogisticRegression(class_weight='balanced')

📌 Key Points to Remember

❗ Always check class distribution before training.
🧪 Use Stratified Train-Test Split to maintain class ratios.
🤖 SMOTE is powerful but requires careful validation.
⚠ Be cautious of overfitting with oversampling techniques.
🔍 Always monitor minority class performance (recall, precision, etc.).

🧠 Interview Questions
Q1: What is class imbalance and why is it a problem?
A: When one class dominates others. Models become biased and fail to detect important minority
instances.

Q2: What techniques do you use to handle imbalanced data?

A: Resampling (over/under), SMOTE, adjusting class weights, stratified splits.

Q3: Why is accuracy not a good metric for imbalanced datasets?

A: A model may predict only the majority class and still achieve high accuracy while failing on
minority cases.

Q4: What are the pros and cons of SMOTE?

A: Pros: Better generalization via synthetic data.
Cons: Risk of overfitting, not ideal for categorical data.

Q5: How do tree-based models like Random Forest handle imbalance?

A: Use class_weight='balanced' or resample data manually. Class balancing still improves
performance.

🧠 Feature Engineering (In-Depth for

Interviews)
Feature Engineering is the process of creating new features or modifying existing ones to
improve a model’s performance. It helps the model learn better by extracting more signal from raw
data using domain knowledge and transformation techniques.

✅ Why Is Feature Engineering Important?

Good features can dramatically improve accuracy

Helps models generalize better

Captures hidden relationships in data

 Essential when data is limited, noisy, or lacks structure

✅ 1. Log Transformations
🔹 What It Does:
Reduces right-skewness

Makes relationships more linear

Stabilizes variance

🔹 When to Use:
For heavily skewed data like:

Income

House prices

Number of users

🔹 Example (Python):
import numpy as np
df['Log_Price'] = np.log1p(df['Price']) # log(1 + x) avoids log(0)

✅ 2. Date/Time Features
🔹 Why:
Date variables often contain patterns like:

Weekends

Holidays

Seasonality

🔹 Example (Python):
df['Year'] = df['Date'].[Link]
df['Month'] = df['Date'].[Link]
df['DayOfWeek'] = df['Date'].[Link]
df['IsWeekend'] = df['DayOfWeek'] >= 5

✅ 3. Domain-Specific Features
Use domain knowledge to extract features that capture real-world behavior.
 Domain Feature Example

Real Estate Price_per_sqft = Price / Area

Banking Credit_Utilization = Balance / Credit_Limit

E-commerce Days_Since_Last_Purchase

Health BMI = Weight / (Height^2)

📌 Key Points to Remember

🚀 Feature Engineering Improves All Models:
Linear models benefit from crafted features (e.g., interactions, log)

Tree-based models (Random Forest, XGBoost) capture interactions, but insightful features
boost performance

🧪 Validating Feature Usefulness

🔹 1. Correlation:
Check how strongly features relate to the target.

[Link]()['target'].sort_values()

🔹 2. Feature Importance:
Use built-in functions from models like RandomForest or XGBoost.

model.feature_importances_

🔹 3. Cross-Validation:
Ensure feature performs well across data splits.

from sklearn.model_selection import cross_val_score

cross_val_score(model, X, y, cv=5).mean()

⚠️ Avoid Data Leakage

Data leakage occurs when a feature contains future information about the target.
❌ Example:
Using total_transaction_amount in fraud prediction when it includes post-fraud data.

✅ Rule:
Only use features available at prediction time.

🔥 Bonus: Tree-Based Models Still Love Good

Features
Tree models can capture feature interactions, but engineered features often make models
simpler and more accurate.

🧠 Interview Questions & Sample Answers

Q1: What is feature engineering?
A: Creating or modifying input features to improve model learning and accuracy.

Q2: Can you give an example of a feature you created?

A: I created Price_per_sqft for a real estate model, which better reflected property value than
price or size alone.

Q3: Why would you use log transformation?

A: To reduce skewness and make data more normally distributed.

Q4: What is data leakage?

A: When a feature leaks information about the target or uses data unavailable at prediction time.

📘 Train-Test Split & Cross-Validation

(Interview Style Guide)
In machine learning, the goal is to build a model that generalizes well to new, unseen data.
If we train and test on the same data, the model may memorize instead of learning patterns —
leading to overfitting.

🔸 1. Train-Test Split
📌 Concept:
Splits the dataset into:

Training Set – Used to train the model.

Test Set – Used to evaluate performance.

Prevents the model from being tested on data it has already seen.
🔹 Code Example:
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, random_state=42
)

🔸 2. K-Fold Cross-Validation (CV)

📌 Concept:
Splits the data into K equal folds.

Trains the model K times, each time using a different fold as test set.

Gives a more robust and reliable estimate of model performance.

Very useful when data is limited.

🔹 Code Example:
from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
scores = cross_val_score(model, X, y, cv=5)
print("Average CV Score:", [Link]())

❓ Common Interview Questions

Q1: Why is it important to split the dataset into training and test sets?
→ To simulate how the model will perform on unseen data and prevent overfitting.

Q2: What is the advantage of K-Fold Cross-Validation over a simple train-test split?
→ It gives a more robust performance estimate, especially when the dataset is small.

Q3: When would you use Stratified K-Fold over normal K-Fold?
→ In classification tasks with imbalanced classes, to preserve class ratios in all folds.

Q4: Why can’t we use normal K-Fold on time series data?

→ It breaks temporal order. Instead, use TimeSeriesSplit which respects sequence.

🧠 Key Points to Remember

⚠️ Always split data before any preprocessing (scaling, encoding, etc.)
🧪 Use StratifiedKFold for imbalanced classification problems.
🕒 Use TimeSeriesSplit for time-dependent datasets.
🧠 K-Fold CV avoids bias from a single random train-test split.
 ❌ Don’t evaluate on the same data used for training — it gives misleading results.

✅ End of Interview Notes on Train-Test Split & Cross-Validation

✅ Evaluation Metrics – Complete

Summary (Interview Style Guide)
Understanding evaluation metrics is essential for selecting the right model and ensuring it's working
as expected — especially in real-world problems like fraud detection, healthcare, or forecasting.

🔷 1. Confusion Matrix
Predicted Positive Predicted Negative

Actual Positive ✅ True Positive (TP) ❌ False Negative (FN)

Actual Negative ❌ False Positive (FP) ✅ True Negative (TN)

🔷 2. Classification Metrics
✅ Accuracy
Formula:

Accuracy = (TP + TN) / (TP + TN + FP + FN)

✅ Use when: Classes are balanced

⚠️ Problematic when: Class imbalance exists

✅ Precision
Formula:

Precision = TP / (TP + FP)

✅ Use when False Positives are costly

(e.g., predicting cancer when it's not present)
✅ Recall (Sensitivity)
Formula:

Recall = TP / (TP + FN)

✅ Use when False Negatives are costly

(e.g., missing a real cancer case)

✅ F1-Score
Formula:

F1 = 2 * (Precision * Recall) / (Precision + Recall)

✅ Use when:
Data is imbalanced

Both Precision and Recall matter

🔷 3. Classification Report
from [Link] import classification_report
print(classification_report(y_true, y_pred))

Metric Description

Precision Of predicted positives, how many are correct

Recall Of actual positives, how many were found

F1-Score Harmonic mean of precision & recall

Support Number of actual samples in each class

🧠 Example Output:
precision recall f1-score support
0 0.90 0.93 0.91 100
1 0.85 0.79 0.82 60
🔷 4. Regression Metrics
✅ MAE (Mean Absolute Error)
Formula:

MAE = (1/n) * Σ|y_true - y_pred|

✅ Easy to interpret
⚠️ Doesn’t penalize large errors heavily

✅ MSE (Mean Squared Error)

Formula:

MSE = (1/n) * Σ(y_true - y_pred)^2

✅ Penalizes large errors more

⚠️ Not in the same unit as the target

✅ RMSE (Root Mean Squared Error)

Formula:

RMSE = √MSE

✅ Same unit as target

⚠️ More sensitive to outliers

✅ R² Score (Coefficient of Determination)

Formula:

R² = 1 - (Σ(y_true - y_pred)² / Σ(y_true - y_mean)²)

✅ Explains variance captured by the model

Range: (-∞ to 1)

1 = perfect prediction

0 = model predicts the mean

<0 = worse than the mean

🎯 Interview Questions Cheat Sheet
Question Suggested Answer

When to prefer F1-score over When data is imbalanced or both precision and recall
Accuracy? matter

Difference between Precision and Precision = exactness (FP matters), Recall =

Recall? completeness (FN matters)

Why is RMSE sometimes preferred RMSE penalizes larger errors more — helpful when big
over MAE? mistakes are risky

What does R² = 0.80 mean? 80% of target variance is explained by the model

What does the Classification Report

Precision, Recall, F1-score, and Support — per class
show?

✅ End of Interview Notes on Evaluation Metrics

✅ 3. Pipeline and Preprocessing Order

(with Code + Interview Focus)

🔹 Why It’s Important

Ensures data leakage is avoided

Makes reproducibility and deployment easier

Keeps preprocessing steps consistent for training and testing

✅ Recommended Preprocessing Order

Step Task

1️⃣ Split data (train/test or cross-validation)

2️⃣ Handle missing values (imputation)

3️⃣ Encode categorical features

4️⃣ Scale/normalize numerical data

5️⃣ Train the model

6️⃣ Apply same steps to test set

🔹 Code Example: Without Pipeline (Manual
Handling)
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
from [Link] import SimpleImputer

# 1. Split first
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

# 2. Impute missing values

imputer = SimpleImputer(strategy='mean')
X_train = imputer.fit_transform(X_train)
X_test = [Link](X_test) # ❗Only transform test set

# 3. Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test) # ❗Only transform test set

🔹 Code Example: Using Pipeline (Recommended)

from [Link] import Pipeline
from [Link] import RandomForestClassifier
from [Link] import SimpleImputer
from [Link] import StandardScaler

pipeline = Pipeline([
('imputer', SimpleImputer(strategy='mean')),
('scaler', StandardScaler()),
('model', RandomForestClassifier())
])

# Pipeline handles fitting and transformation properly

[Link](X_train, y_train)
preds = [Link](X_test)

⚠️ Data Leakage Example

Mistake:

scaler = StandardScaler()
X = scaler.fit_transform(X) # ❌ Fit on full data before splitting

Why it's bad?

The test set influences scaling parameters (mean, std)

 Leads to inflated performance during evaluation

❓ Common Interview Questions

What is the correct order of preprocessing steps?
→ Split → Impute → Encode → Scale → Train

Why should we split the data before scaling or imputing?

→ To avoid data leakage from the test set

What are the benefits of using a pipeline?

→ Prevents leakage, automates steps, and improves reproducibility

How do you apply the same preprocessing to the test set?

→ Use .transform() (not .fit_transform() ) on test data

🧠 Points to Remember
✅ Always split your data first
✅ Use .fit() only on training data; use .transform() for test/val

✅ Pipelines ensure clean, consistent, and production-ready workflows

⚠️ Never leak target-related info into feature engineering or scaling steps