Master Machine Learning Question Bank

Comprehensive Assessment for [Link] Students

Covers Data Preprocessing, SVM, PCA, Ensemble Learning and Imbalanced Data

INDEX
Section 1: Data Preprocessing, CV & Matrix Transformations
Section 2: Ensemble Learning (Bagging, Boosting, RF, XGBoost)
Sub-Section 2.1: Bagging & Random Forest (Q1 - Q10)
Sub-Section 2.2: AdaBoost (Q11 - Q20)
Sub-Section 2.3: Gradient Boosting Machines (GBM) (Q21 - Q30)
Sub-Section 2.4: XGBoost, LightGBM & CatBoost (Q31 - Q40)
Sub-Section 2.5: Advanced Comparisons & Hyperparameter Strategy (Q41 - Q50)
Section 3: Support Vector Machines (SVM)
Sub-Section 3.1: Geometric Intuition & Basics
Sub-Section 3.2: The Mathematics of SVM
Sub-Section 3.3: Soft Margin & Slack Variables
Sub-Section 3.4: The Kernel Trick
Sub-Section 3.5: Support Vector Regression (SVR)
Sub-Section 3.6: Assumptions, Pros, Cons & Use Cases
Sub-Section 3.7: Scikit-Learn Implementation
Section 4: Dimensionality Reduction (PCA & t-SNE)
Section 5: Imbalanced Data & SMOTE
Sub-Section 5.1: Core Concepts & The Accuracy Paradox
Sub-Section 5.2: Rules & Basic Resampling
Sub-Section 5.3: Advanced Undersampling
Sub-Section 5.4: Advanced Oversampling (SMOTE Family)
Sub-Section 5.5: Hybrid Methods & Math
Section 6: Mixed Advanced Practice [All Numericals]
Section 1: Data Preprocessing, CV & Matrix Transformations
Instructions: Choose the best answer for each question.
Q1. At what stage in the machine learning pipeline is it generally recommended to handle duplicate
rows?
A) After feature scaling
B) Early in the pipeline
C) Immediately before model training
D) During hyperparameter tuning

Q2. You are processing a highly volatile cryptocurrency dataset. You notice identical consecutive
trades executed by distinct users at the exact same millisecond. Should these duplicates be removed?
A) Yes, to reduce dataset size and speed up training.
B) Yes, because all identical rows negatively impact model accuracy.
C) No, because they represent mathematically or logically distinct events.
D) No, but they should be modified to have slight variations.

Q3. What is a primary reason to handle duplicated data from a scraped dataset of used car prices?
A) To prevent models from giving unwarranted weight to repeated data.
B) To increase the variance of the target variable.
C) To allow PCA to function correctly.
D) To convert nominal data to numerical data.

Q4. Which of the following scenarios is an example of when to AVOID dropping duplicate rows?
A) A database join accidentally duplicates user profiles.
B) A sensor transmits the same packet twice due to network latency.
C) Two distinct customers buy the identical car model at the exact same price.
D) A web scraper accidentally scrapes page 1 of a forum twice.

Q5. What is the primary goal of handling 'Mixed Data Types' in a single column?
A) To ensure columns have a uniform and appropriate data type.
B) To compress the physical file size of the dataset.
C) To standardize the mathematical range of the features.
D) To handle missing NaN values effectively.

Q6. A column representing cryptocurrency trading volume contains values like '1500' and 'USD 1500'.
What is the immediate consequence if this is fed into an XGBoost model?
A) The model will automatically drop the 'USD' text.
B) The model will convert the column to NaN values.
C) The algorithm will fail due to parsing errors.
D) The model will apply One-Hot Encoding.

Q7. When downcasting data types (e.g., float64 to int32) to save memory, what is the primary risk?
A) It introduces multicollinearity.
B) It can lead to the loss of necessary mathematical precision.
C) It automatically converts data into categorical formats.
D) It prevents the use of distance-based algorithms.

Q8. You are cleaning a 'Mileage' column in a used car dataset and find the string '12,000 miles'. What is
the correct preprocessing step?
A) Apply One-Hot Encoding to the column.
B) Remove the text 'miles' and parse the string to a numeric format.
C) Delete the entire row containing the string.
D) Apply Standard Scaling directly.

Q9. Which of the following acronyms represents a mechanism for Missing Values?
A) SMOTE
 B) MCAR
C) PCA
D) VIF

Q10. Why do nearly all machine learning algorithms fail if datasets contain NaN values?
A) NaN values cause an explosion in dimensionality.
B) Algorithms cannot perform mathematical operations on null data.
C) NaN values automatically skew the dataset.
D) They force the model to overfit.

Q11. If your dataset is very small, why should you avoid outright deletion of rows with missing values?
A) It leads to a massive explosion in dimensions.
B) It leads to a critical loss of information.
C) It makes the dataset too highly correlated.
D) It prevents the use of SMOTE.

Q12. A dataset regarding crypto trading has a heavily skewed distribution of transaction amounts. If
dealing with missing values in this column, which simple imputation method should be avoided?
A) Median Imputation
B) Mode Imputation
C) Mean Imputation
D) Predictive Imputation

Q13. You have a car price distribution heavily skewed to the right by luxury vehicles. To impute a
missing price, what is the best simple imputation metric?
A) Mean
B) Median
C) Standard Deviation
D) Variance

Q14. What defines an outlier using the Interquartile Range (IQR) method?
A) Values within 1 standard deviation of the mean.
B) Values that deviate significantly using Q1 and Q3.
C) Values that match the dataset's variance.
D) Values that cause parsing errors.

Q15. For which of the following algorithms is handling outliers considered CRUCIAL due to high
sensitivity to skewness?
A) Random Forest
B) XGBoost
C) Decision Trees
D) Linear/Logistic Regression

Q16. Why is handling outliers often considered UNNECESSARY for tree-based models like Random
Forest?
A) They automatically delete outliers during training.
B) They split nodes based on thresholds regardless of scale.
C) They use Z-scores internally to scale data.
D) They are designed specifically for anomaly detection.

Q17. In which specific task should you actively AVOID removing outliers?
A) Predicting the average price of a sedan.
B) Fraud detection.
C) Linear Regression modeling.
D) Customer segmentation using K-Means.
Q18. Numerical Challenge: In a dataset of car prices, the 25th percentile (Q1) is $10,000 and the 75th
percentile (Q3) is $20,000. Using the standard 1.5 * IQR rule, what is the upper boundary for detecting
an outlier?
A) $25,000
B) $30,000
C) $35,000
D) $40,000

Q19. Numerical Challenge: A cryptocurrency feature has a mean of 50 and a standard deviation of 10.
You are using Z-scores to identify outliers. What is the Z-score for a data point with a value of 80?
A) 2.0
B) 3.0
C) 4.0
D) 0.3

Q20. Which of the following is an algorithm that relies heavily on distance calculations and thus
requires careful handling of outliers?
A) Decision Trees
B) Random Forest
C) K Means
D) Naive Bayes

Q21. What is the primary purpose of Encoding Categorical Variables?

A) To standardize the range of numerical columns.
B) To convert text labels into a machine-readable numeric format.
C) To remove correlated independent variables.
D) To group continuous data into discrete categories.

Q22. When should you explicitly AVOID using Label Encoding?

A) When the variable has an inherent logical order (e.g., Low, Medium, High).
B) When the variable has high cardinality.
C) When dealing with nominal variables (categories with no logical order like colors).
D) When using tree-based models.

Q23. Why should One-Hot Encoding be avoided if a column has high cardinality (thousands of unique
categories)?
A) It will cause the model to misinterpret numbers as having weight.
B) It destroys detailed data granularity.
C) It will cause a massive explosion in dimensions.
D) It inherently creates extreme outliers.

Q24. Numerical Challenge: Your [Link] students have a dataset with 5,000 rows. There is a categorical
column for 'Car_Brand' with 30 unique brands. If they apply standard One-Hot Encoding (without
dropping the first column), how many total cells (data points) are added to the matrix by this single
feature?
A) 150,000
B) 30,000
C) 5,000
D) 145,000

Q25. You have a categorical column 'Education Level' with values: High School, Bachelors, Masters,
PhD. Which encoding technique is most mathematically appropriate?
A) One-Hot Encoding
B) Label/Ordinal Encoding
C) Data Discretization
D) Standard Scaling
Q26. What is the core function of Data Scaling and Normalization?
A) Removing identical rows from a dataset.
B) Standardizing the range of features so all numerical columns are on a similar scale.
C) Identifying independent variables that are highly correlated.
D) Converting continuous data into discrete intervals.

Q27. Why is scaling essential for an algorithm like Support Vector Machines (SVM) or K-Nearest
Neighbors (KNN)?
A) Because they cannot process categorical text data.
B) Because they split nodes based on thresholds.
C) Because they rely on distance calculations.
D) Because they are immune to multicollinearity.

Q28. For Neural Networks and Linear Regression, what is a primary benefit of data scaling?
A) It guarantees the removal of outliers.
B) It ensures faster convergence during gradient descent.
C) It automatically balances skewed class distributions.
D) It converts non-linear relationships into linear ones.

Q29. For which family of algorithms is data scaling considered unnecessary?

A) Distance-based algorithms (KNN)
B) Gradient descent-based algorithms (Neural Networks)
C) Tree-based algorithms (Decision Trees, Random Forests)
D) Linear models (Logistic Regression)

Q30. Numerical Challenge: You apply a MinMaxScaler to a feature. The original feature has a minimum
value of 20 and a maximum of 100. What will be the scaled value for an original data point of 40?
A) 0.20
B) 0.25
C) 0.40
D) 0.50

Q31. What is 'Multicollinearity' in the context of data preprocessing?

A) When target variables are heavily skewed.
B) When independent variables are highly correlated with each other.
C) When a dataset has thousands of unique categorical values.
D) When identical rows exist in the dataset.

Q32. In predicting car prices, you have columns for 'Engine_Size_Liters' and 'Engine_Size_CC'. Why is
it crucial to handle this multicollinearity if using Linear Regression?
A) To keep the model's coefficients stable and interpretability high.
B) Because Random Forest models will fail to execute.
C) To prevent an explosion of dimensionality.
D) To standardize the range of the features.

Q33. When is handling multicollinearity considered LESS critical?

A) When absolute interpretability of coefficients is required.
B) When using linear regression for hypothesis testing.
C) When using complex, non-linear models like ensemble trees or deep learning purely for predictive
accuracy.
D) When dealing with missing values.

Q34. Which of the following techniques is explicitly mentioned for combining variables into new
components for dimensionality reduction?
A) VIF
 B) SMOTE
C) PCA (Principal Component Analysis)
D) IQR

Q35. What problem does Feature Selection and Dimensionality Reduction primarily aim to mitigate?
A) Parsing errors from mixed types.
B) The Curse of Dimensionality.
C) Class imbalance in target variables.
D) Loss of mathematical precision during downcasting.

Q36. Why should you AVOID using PCA (Principal Component Analysis) when explaining a car price
model to a business stakeholder?
A) It massively increases training times.
B) It requires nominal categorical encoding first.
C) PCA transforms columns into mathematical components that humans cannot easily map to real-world
metrics.
D) PCA is highly sensitive to class imbalance.

Q37. What is the definition of Handling Imbalanced Data?

A) Standardizing the mathematical ranges of inputs.
B) Adjusting class distributions for classification problems using oversampling or undersampling.
C) Removing identical rows from a dataset.
D) Grouping continuous data into discrete categories.

Q38. In a cryptocurrency fraud detection model, 99.5% of transactions are normal and 0.5% are
fraudulent. What happens if you do NOT handle this imbalanced data?
A) The model will crash due to NaN generation.
B) The model will suffer from multicollinearity.
C) The model will likely just predict the majority class (Normal) every time.
D) The model will perfectly overfit the minority class.

Q39. Which of the following is a specific technique used for oversampling minority classes?
A) PCA
B) StandardScaler
C) SMOTE
D) One-Hot Encoding

Q40. What is the absolute golden rule regarding the application of balancing techniques (like SMOTE)
on dataset splits?
A) Never balance your testing/validation datasets.
B) Always balance both training and testing datasets.
C) Only apply SMOTE to the validation dataset.
D) Balance the test data, but not the training data.

Q41. What is the primary action of Data Discretization (Binning)?

A) Converting categorical labels into continuous numbers.
B) Grouping continuous data into discrete categories/intervals.
C) Synthetically generating new data points.
D) Extracting the principal components of a matrix.

Q42. When might you intentionally use binning on a numerical column?

A) When exact numerical precision is essential for the prediction task.
B) When you want to increase the dimensionality of the dataset.
C) When a non-linear relationship can be more easily modeled by categorizing the data.
D) When you need to perform gradient descent.
Q43. Why is binning rarely used when exact numerical precision is required?
A) It fundamentally destroys detailed data granularity.
B) It causes the model to overfit heavily.
C) It introduces multicollinearity.
D) It prevents the use of One-Hot Encoding.

Q44. Numerical Challenge: A standard normal distribution has a mean of 0 and a standard deviation of
1. If you apply a StandardScaler to a dataset, what will the new mean of that transformed column be?
A) 1
B) 0
C) -1
D) It depends on the original data.

Q45. In a cryptocurrency dataset, you find the 'Volume' column is highly correlated with the
'Market_Cap' column. Which preprocessing technique addresses this directly?
A) Handling Missing Values
B) Handling Multicollinearity and Correlation
C) Data Scaling
D) Handling Outliers

Q46. If you have a dataset with 50 numerical columns and apply PCA to extract 5 principal
components, what have you primarily achieved?
A) Dimensionality Reduction
B) Data Discretization
C) Categorical Encoding
D) Imputation

Q47. Why is handling missing values (via imputation) required BEFORE applying algorithms like PCA?
A) Because PCA automatically drops columns with NaN.
B) Because PCA relies on covariance matrices which cannot compute with NaN values.
C) Because NaN values cause PCA to generate strings.
D) PCA actually handles missing values natively.

Q48. When building an XGBoost model for car prices, why might you skip extensive Data Scaling?
A) XGBoost requires categorical text data.
B) Tree-based ensembles are generally robust to varying scales because they split on thresholds.
C) Scaling will cause the model to overfit.
D) XGBoost automatically applies MinMax internally.

Q49. A student accidentally applies One-Hot Encoding to a 'Car_ID' column containing 10,000 unique
identifier strings. What is the immediate consequence?
A) The model's interpretability increases.
B) The data becomes perfectly scaled.
C) A massive explosion in dimensions (Curse of Dimensionality).
D) Multicollinearity is eliminated.

Q50. You are processing survey data where users typed their income. You find entries like '50000',
'50k', and '50,000'. Which preprocessing step is required?
A) Handling Outliers
B) Handling Missing Values
C) Handling Mixed Data Types
D) Data Discretization

Q51. Which preprocessing technique serves as the bridge between raw text categories and machine
learning models?
A) Data Scaling
 B) Dimensionality Reduction
C) Encoding Categorical Variables
D) SMOTE

Answer Key & Rationales

Q1: Early in the pipeline
Rationale: Duplicates should be removed early to ensure each observation is unique and models don't give
unwarranted weight to repeated data.
Q2: No, because they represent mathematically or logically distinct events.
Rationale: When duplicates represent legitimate, distinct events, removing them alters the true reality of the
dataset.
Q3: To prevent models from giving unwarranted weight to repeated data.
Rationale: Duplicates act like a multiplier for specific instances, biasing the model.
Q4: Two distinct customers buy the identical car model at the exact same price.
Rationale: These are distinct, legitimate real-world events that happen to have identical feature values.
Q5: To ensure columns have a uniform and appropriate data type.
Rationale: Machine learning models require uniform numerical matrices; strings mixed with ints will cause
parsing errors.
Q6: The algorithm will fail due to parsing errors.
Rationale: Models require purely numeric inputs; string characters introduce inconsistencies.
Q7: It can lead to the loss of necessary mathematical precision.
Rationale: Converting a float (which has decimals) to an integer truncates the decimal, losing precision.
Q8: Remove the text 'miles' and parse the string to a numeric format.
Rationale: Removing the string characters ensures the column has a uniform numerical type.
Q9: MCAR
Rationale: MCAR stands for Missing Completely At Random.
Q10: Algorithms cannot perform mathematical operations on null data.
Rationale: Matrix multiplication and distance calculations require actual numerical values to compute.
Q11: It leads to a critical loss of information.
Rationale: In small datasets, every observation is vital; deleting rows reduces the model's ability to learn
generalizable patterns.
Q12: Mean Imputation
Rationale: The mean is highly sensitive to extreme values in skewed distributions, leading to inaccurate
imputations.
Q13: Median
Rationale: The median represents the 50th percentile and is unaffected by extreme high-value outliers.
Q14: Values that deviate significantly using Q1 and Q3.
Rationale: IQR is calculated as Q3 - Q1, and outliers are typically those beyond 1.5 * IQR from the quartiles.
Q15: Linear/Logistic Regression
Rationale: Linear models fit a line of best fit, which is heavily pulled by extreme, distant outlier values.
Q16: They split nodes based on thresholds regardless of scale.
Rationale: A split at x > 10 treats a value of 11 the same as 11,000, rendering the exact magnitude of the
outlier irrelevant.
Q17: Fraud detection.
Rationale: In anomaly detection tasks like fraud, the outliers are the exact target you are trying to find.
Q18: $35,000
Rationale: IQR = 20k - 10k = 10k. 1.5 * 10k = 15k. Upper bound = Q3 + 15k = $35,000.
Q19: 3.0
Rationale: Z = (Value - Mean) / StdDev = (80 - 50) / 10 = 30 / 10 = 3.0.
Q20: K Means
Rationale: K-Means measures the Euclidean distance to cluster centroids, meaning outliers can drastically
pull the centroid away from the true center.
Q21: To convert text labels into a machine-readable numeric format.
Rationale: Algorithms require numerical matrices; encoding bridges the gap from text to numbers.
Q22: When dealing with nominal variables (categories with no logical order like colors).
Rationale: Assigning 1=Red, 2=Blue, 3=Green implies to the model that Green > Red, which is mathematically
false.
Q23: It will cause a massive explosion in dimensions.
Rationale: Creating a new column for 10,000 unique categorical values creates 10,000 new columns, leading
to the Curse of Dimensionality.
Q24: 150,000
Rationale: 30 unique brands = 30 new columns. 30 columns * 5,000 rows = 150,000 total cells added.
Q25: Label/Ordinal Encoding
Rationale: Because the categories have a strict logical order, mapping them to 1, 2, 3, 4 preserves that
hierarchy for the model.
Q26: Standardizing the range of features so all numerical columns are on a similar scale.
Rationale: Scaling ensures that features ranging from 1-10 and 1,000-10,000 are mathematically comparable
by the algorithm.
Q27: Because they rely on distance calculations.
Rationale: Without scaling, a feature with a range of 10,000 will mathematically dominate a feature with a
range of 1 when calculating geometric distance.
Q28: It ensures faster convergence during gradient descent.
Rationale: Uniform scales create a smoother error surface, allowing the gradient descent algorithm to find the
minimum much faster.
Q29: Tree-based algorithms (Decision Trees, Random Forests)
Rationale: Trees split based on thresholds (e.g., is X > 5?). The scale of the number doesn't affect the model's
ability to find that optimal splitting point.
Q30: 0.25
Rationale: Formula: (X - Min) / (Max - Min). (40 - 20) / (100 - 20) = 20 / 80 = 0.25.
Q31: When independent variables are highly correlated with each other.
Rationale: It means two or more predictor variables are providing the exact same, redundant information to
the model.
Q32: To keep the model's coefficients stable and interpretability high.
Rationale: If two inputs are identical, the linear regression algorithm struggles to assign the correct weight
(coefficient) to each, making interpretation impossible.
Q33: When using complex, non-linear models like ensemble trees or deep learning purely for predictive
accuracy.
Rationale: These black-box models can internally ignore or downweight redundant features without breaking
their predictive power.
Q34: PCA (Principal Component Analysis)
Rationale: PCA mathematically transforms and combines multiple correlated variables into fewer, uncorrelated
principal components.
Q35: The Curse of Dimensionality.
Rationale: Having too many columns (features) leads to overfitting, massive noise, and exponentially slower
training times.
Q36: PCA transforms columns into mathematical components that humans cannot easily map to real-world
metrics.
Rationale: A business user understands 'Mileage'; they do not understand 'Principal Component 1', which is a
mathematical blend of Mileage, Age, and Engine Size.
Q37: Adjusting class distributions for classification problems using oversampling or undersampling.
Rationale: It forces the model to look at the minority class by synthetically balancing the ratio of outcomes in
the training set.
Q38: The model will likely just predict the majority class (Normal) every time.
Rationale: A model can achieve 99.5% accuracy simply by guessing 'Normal' for every transaction, utterly
failing to detect any fraud.
Q39: SMOTE
Rationale: Synthetic Minority Over-sampling Technique (SMOTE) generates synthetic examples of the
minority class.
Q40: Never balance your testing/validation datasets.
Rationale: Balancing should strictly be applied to the training data. The test data must reflect real-world
distributions to evaluate true performance.
Q41: Grouping continuous data into discrete categories/intervals.
Rationale: It takes an infinite range of numbers (e.g., ages 1-100) and chunks them into a few distinct groups
(e.g., Child, Adult, Senior).
Q42: When a non-linear relationship can be more easily modeled by categorizing the data.
Rationale: If risk spikes only between ages 16-25, binning 'Age' allows linear models to capture that specific
bracket's risk without complex polynomials.
Q43: It fundamentally destroys detailed data granularity.
Rationale: By changing $49,999 and $50,001 into the same 'Medium Income' bin, you lose the exact
mathematical difference between them.
Q44: 0
Rationale: Standard scaling centers the data around a mean of exactly 0.
Q45: Handling Multicollinearity and Correlation
Rationale: This specific step identifies and removes redundant, highly correlated independent variables.
Q46: Dimensionality Reduction
Rationale: You reduced the width of the dataset from 50 features down to 5, mitigating the curse of
dimensionality.
Q47: Because PCA relies on covariance matrices which cannot compute with NaN values.
Rationale: The mathematical foundation of PCA requires complete numerical vectors to calculate variance
and covariance.
Q48: Tree-based ensembles are generally robust to varying scales because they split on thresholds.
Rationale: The decision tree algorithm inside XGBoost doesn't calculate distances; it just finds optimal
numerical split points.
Q49: A massive explosion in dimensions (Curse of Dimensionality).
Rationale: The dataset immediately gains 10,000 new columns of mostly zeroes, severely crippling algorithm
performance.
Q50: Handling Mixed Data Types
Rationale: These strings must be parsed, cleaned of text characters, and unified into a single numerical type.
Q51: Encoding Categorical Variables
Rationale: Encoding translates textual concepts into the numeric language algorithms understand.
Instructions: Choose the best answer for each question.
Q52. You are training an automated trading bot on daily stock market data from 2021 to 2025. If you
evaluate your model using standard, randomized 5-fold cross-validation, what critical logical error are
you introducing?
A) Multicollinearity, because the temporal features will become highly correlated.
B) Data leakage, because randomized folds will allow future market data to predict past market movements.
C) Explosion of dimensions, because each fold creates a new set of categorical variables.
D) Class imbalance, because standard folds cannot handle financial data.

Q53. Numerical Challenge: You are preprocessing a used car dataset with exactly 5,000 records to
predict prices. You configure an 8-fold cross-validation strategy. In each iteration of the cross-
validation process, exactly how many records are allocated for training and how many for validation?
A) 4,000 for training; 1,000 for validation
B) 4,500 for training; 500 for validation
C) 4,375 for training; 625 for validation
D) 625 for training; 4,375 for validation

Q54. You are building a classification model for PSU sector stocks, categorizing them as 'Buy', 'Hold',
or 'Sell'. The 'Sell' instances make up only 4% of your total data. If you use standard K-Fold CV instead
of Stratified K-Fold CV, what is the most likely negative consequence?
A) The model will execute much slower.
B) A validation fold might randomly contain zero 'Sell' instances, making it impossible to evaluate the
model's ability to predict them.
C) The standard K-Fold will synthetically generate 'Sell' data using SMOTE automatically.
D) The independent variables will lose their original scaling.
Q55. In a survey given to crypto traders, the 'Annual Portfolio Losses' column has 30% missing values.
You realize that traders who lost the most money simply felt embarrassed and refused to answer. What
type of missing data mechanism is this, and why is mean imputation highly dangerous here?
A) MCAR (Missing Completely At Random); mean imputation will over-estimate the losses.
B) MNAR (Missing Not At Random); mean imputation will severely underestimate the true average loss
because the highest values are the ones missing.
C) MAR (Missing At Random); mean imputation is the correct approach.
D) MCAR; mean imputation will cause an explosion of dimensionality.

Q56. Numerical Challenge: A feature representing the 'Number of Previous Owners' for a sample of
used cars has the following values: [1, 1, 1, 2, 2, 3, NaN, 15]. The 15 represents a massive outlier (a
corporate fleet vehicle). If you must impute the NaN value, what is the precise numerical difference
between using Mean Imputation versus Median Imputation?
A) 1.57
B) 2.00
C) 3.57
D) 0.57

Q57. A used car dataset has missing values in the 'Engine_CC' column. Instead of filling the blanks
with the overall average engine size, you write a script that uses a Random Forest regressor to predict
the missing 'Engine_CC' based on the car's 'Brand', 'Price', and 'Mileage'. What is this preprocessing
technique formally called?
A) Synthetic Minority Over-sampling (SMOTE)
B) Data Discretization
C) Multivariate / Predictive Imputation
D) Principal Component Analysis (PCA)

Q58. Advanced Pipeline Logic: During preprocessing, a student applies a StandardScaler to their
entire dataset, forcing the mean to 0 and variance to 1. Immediately after, they apply a 10-Fold Cross-
Validation to train and evaluate their model. What critical methodological flaw has occurred?
A) Data Leakage: The training folds contain statistical information (the global mean and variance) that was
influenced by the validation fold data.
B) Curse of Dimensionality: Standard scaling prior to CV expands the matrix exponentially.
C) Information Loss: K-Fold CV cannot process negative values generated by the StandardScaler.
D) There is no flaw; this is the standard sequence of operations.

Answer Key & Rationales

Q52: Data leakage, because randomized folds will allow future market data to predict past market movements.
Rationale: In time-series data, standard CV randomly mixes past and future. You must use Time-Series CV
(forward-chaining) to ensure the model only ever learns from the past to predict the future.
Q53: 4,375 for training; 625 for validation
Rationale: 5000 total records / 8 folds = 625 records per validation fold. 5000 - 625 = 4,375 records used for
training in that iteration.
Q54: A validation fold might randomly contain zero 'Sell' instances, making it impossible to evaluate the
model's ability to predict them.
Rationale: Stratified CV ensures that the 4% distribution is maintained across every single fold. Standard CV
might randomly group all the 'Sell' instances into one fold, leaving others empty.
Q55: MNAR (Missing Not At Random); mean imputation will severely underestimate the true average loss
because the highest values are the ones missing.
Rationale: When data is missing precisely *because* it is an extreme value, imputing the mean of the
remaining 'safe' data creates a massive downward bias.
Q56: 1.57
Rationale: Mean (excluding NaN): (1+1+1+2+2+3+15)/7 = 25/7 = 3.57. Median (excluding NaN): The ordered
list is 1, 1, 1, 2, 2, 3, 15. The middle value is 2. Difference: 3.57 - 2 = 1.57.
Q57: Multivariate / Predictive Imputation
Rationale: Using other variables (multivariate) to train a model that predicts and fills in the missing values is
known as Predictive Imputation or MICE (Multivariate Imputation by Chained Equations).
Q58: Data Leakage: The training folds contain statistical information (the global mean and variance) that was
influenced by the validation fold data.
Rationale: Scaling must occur *inside* the cross-validation loop. If you scale the whole dataset first, the
validation data secretly influences the scaled values of the training data, leading to overly optimistic validation
scores.
Instructions: Choose the best answer for each question.
Q59. A student is working on the Car Dekho dataset, which contains 10,000 rows and 15 columns. They
discover that 500 rows have a missing 'Mileage' value, and 200 rows have a missing 'Engine_CC' value.
Upon closer inspection, exactly 100 rows are missing BOTH 'Mileage' and 'Engine_CC'. If the student
applies Listwise Deletion (dropping any row with at least one NaN value), what will be the final shape of
the dataset matrix (Rows, Columns)?
A) (9300, 15)
B) (9400, 13)
C) (9400, 15)
D) (9500, 15)

Q60. You are building a classification model for a cryptocurrency dataset with 20,000 total
transactions. The target variable is 'Fraud', where fraudulent transactions make up exactly 1% of the
dataset. You apply standard SMOTE to oversample the minority class until the dataset is perfectly
balanced (50/50). What is the total number of rows in your final dataset?
A) 40,000
B) 39,600
C) 20,000
D) 39,800

Q61. A dataset containing [Link] student project data has 5,000 rows and 12 total columns. There are
10 numeric columns and 2 categorical columns: 'Department' (containing 4 unique values) and 'State'
(containing 20 unique values). If a student applies One-Hot Encoding to the categorical features and
does NOT use a drop-first (dummy variable trap) mechanism, how many columns will the resulting
dataset have?
A) 34
B) 36
C) 32
D) 24

Q62. You have an extremely imbalanced stock market trading dataset with 100,000 records: 95,000
labelled 'Hold', 4,000 labelled 'Buy', and 1,000 labelled 'Sell'. To optimize training time without losing
minority data, you build a pipeline that FIRST undersamples the majority class ('Hold') down to 10,000
records, and THEN applies SMOTE to make the 'Buy' and 'Sell' classes equal to the new 'Hold' count.
What is the final total row count of the training set?
A) 100,000
B) 30,000
C) 285,000
D) 15,000

Q63. Your initial dataset shape is (8000, 12). You decide to use simple Mean Imputation on the 'Income'
column which has missing values. To preserve the knowledge of which rows were originally missing,
you add a 'Missing_Indicator' boolean column. Afterwards, you apply a MinMaxScaler to all numerical
features. What is the final shape of the dataset matrix?
A) (8000, 12)
B) (8000, 13)
 C) (7500, 13)
D) (8000, 24)

Q64. A dataset has 15,000 rows and 10 columns. You apply equal-frequency binning (e.g., pandas qcut)
to the 'Price' column, converting it into 5 discrete categories, completely replacing the original numeric
'Price' column. Next, you apply One-Hot Encoding strictly to this new binned column, ensuring you use
'drop_first=True' to avoid multicollinearity. What is the final shape of your dataset?
A) (15000, 15)
B) (15000, 14)
C) (15000, 13)
D) (15000, 10)

Answer Key & Rationales

Q59: (9400, 15)
Rationale: Total rows with at least one missing value = 500 (Mileage) + 200 (Engine) - 100 (Intersection) =
600 rows to drop. 10,000 - 600 = 9,400 rows. Columns remain unchanged. Shape is (9400, 15).
Q60: 39,600
Rationale: 1% of 20,000 = 200 Fraud cases. This leaves 19,800 Normal cases. SMOTE generates synthetic
Fraud cases until they equal the Normal cases (19,800). Total rows = 19,800 (Normal) + 19,800 (Fraud) =
39,600.
Q61: 34
Rationale: Original numeric columns = 10. 'Department' becomes 4 columns. 'State' becomes 20 columns.
Total columns = 10 + 4 + 20 = 34.
Q62: 30,000
Rationale: Step 1: 'Hold' is reduced to 10,000. Current state is (10k Hold, 4k Buy, 1k Sell). Step 2: SMOTE
brings 'Buy' up to 10,000 and 'Sell' up to 10,000. Final state is (10k Hold, 10k Buy, 10k Sell) = 30,000 rows.
Q63: (8000, 13)
Rationale: Rows remain 8000 since you imputed, not deleted. You started with 12 columns and explicitly
added 1 new binary column for the missing indicator. The MinMaxScaler scales values in place and does not
change the matrix shape. 12 + 1 = 13 columns.
Q64: (15000, 13)
Rationale: Start with 10. Replace 'Price' with a binned version (still 10 columns). Apply OHE with
drop_first=True to the 5 categories. This creates 4 new binary columns. The single categorical column is
removed. Remaining original columns = 9. 9 + 4 = 13 columns.
Section 2: Ensemble Learning (Bagging, Boosting, RF, XGBoost)
Module 6: Comprehensive Ensemble Learning MCQ Bank

Sub-Section 2.1: Bagging & Random Forest (Q1 - Q10)

1. Which of the following is the primary statistical objective of Bagging (Bootstrap Aggregating)?
A) Decreasing the bias of the base estimator
B) Decreasing the variance of the base estimator
C) Decreasing both bias and variance simultaneously
D) Increasing the complexity of the base estimator
Answer: B. Bagging averages multiple independent, high-variance models (like deep unpruned trees)
to smooth out predictions, drastically reducing variance without significantly affecting bias.
2. In a Random Forest, how does the algorithm ensure that the individual decision trees are
decorrelated?
A) By training each tree sequentially on the errors of the previous tree
B) By assigning different sample weights to the training data for each tree
C) By considering only a random subset of features at each split node
D) By applying L1 regularization to the leaf nodes
Answer: C. Feature subsampling at each split (usually square root of total features) prevents dominant
features from being selected at the root of every tree, forcing the trees to learn different data
representations.
3. [Numerical] A dataset has 1000 rows. When creating a bootstrap sample (sampling with
replacement) of size 1000, approximately what percentage of the original unique data points will be
included in a single bootstrap sample?
A) 100%
B) 85.2%
C) 63.2%
D) 36.8%
Answer: C. The probability of a sample not being picked is (1-1/N)^N. As N approaches infinity, this
equals 1/e ≈ 0.368. Therefore, 1 - 0.368 = 0.632, meaning ~63.2% of unique samples are included (the In-
Bag samples), leaving ~36.8% as Out-Of-Bag (OOB) samples.
4. Which statement about the Out-Of-Bag (OOB) error in Random Forest is true?
A) It requires a separate validation dataset to be held out during training.
B) It evaluates each tree using the ~36.8% of data points it did not see during training.
C) It is only applicable to regression problems, not classification.
D) It represents the error rate on the training data.
Answer: B. OOB error acts as a free validation metric, evaluating trees only on the specific samples
that were omitted from their respective bootstrap datasets.
5. If you increase the 'n_estimators' in a Random Forest from 100 to 1000, what is the most likely
outcome?
A) The model will severely overfit the training data.
B) The model's test variance will stabilize, but training time will increase linearly.
C) The model's bias will decrease significantly.
D) The OOB error will increase.
Answer: B. Unlike boosting, adding more trees to a bagging algorithm does not cause overfitting. The
performance simply plateaus/stabilizes, at the cost of computational time.
6. Why are Decision Trees the preferred base estimator for Random Forests instead of Logistic
Regression models?
A) Trees have high bias, which forests correct.
B) Logistic regression cannot be trained on subset data.
C) Trees are highly sensitive to data variations (high variance), which makes bootstrap sampling effective at
creating diverse models.
D) Trees compute faster than linear combinations.
Answer: C. Bagging only works well when the base learners are unstable (high variance). Linear
models are highly stable; a bootstrap sample won't change a regression line much, negating the
benefit of the ensemble.
7. In a Random Forest regressor, how is the final output determined for a new test instance?
A) By taking the median of all tree predictions.
B) By a weighted average based on tree training accuracy.
C) By taking the simple, unweighted mean of all individual tree predictions.
D) By selecting the prediction of the tree with the lowest OOB error.
Answer: C. Random forest relies on uniform voting. For regression, it takes the unweighted average of
all trees. (Unlike Boosting, where voting is weighted).
8. Extremely Randomized Trees (ExtraTrees) differ from standard Random Forests in what specific
way?
A) They do not use bootstrap sampling, and they choose split thresholds completely at random.
B) They use sequential error tracking.
C) They utilize second-order gradients.
D) They only train on categorical variables.
Answer: A. ExtraTrees randomly select split thresholds rather than conducting an exhaustive search
for the optimal Gini/Entropy threshold, further increasing variance and computational speed.
9. Which hyperparameter is most effective at controlling the individual complexity of a Random Forest
base tree to prevent extreme RAM consumption?
A) learning_rate
B) max_depth
C) n_estimators
D) subsample
Answer: B. max_depth (or min_samples_split/leaf) truncates tree growth, keeping memory usage down
on massive datasets.
10. [Numerical] You have a dataset with 144 features. By default, how many features will a Random
Forest Classifier evaluate at each node split?
A) 144
B) 72
C) 12
D) 1
Answer: C. The default heuristic for classification in Random Forest is the square root of the total
features. sqrt(144) = 12.

Sub-Section 2.2: AdaBoost (Q11 - Q20)

11. What is the fundamental mechanism AdaBoost uses to focus on difficult instances?
A) It calculates the pseudo-residuals of the previous predictions.
B) It exponentially increases the sample weight of misclassified data points.
C) It deletes correctly classified points from the dataset.
D) It increases the depth of the decision trees.
Answer: B. AdaBoost dynamically alters the probability distribution of the data, forcing subsequent
weak learners to prioritize heavily weighted, misclassified instances.
12. [Numerical] During Iteration 1 of AdaBoost, the base stump misclassifies 10 out of 50 samples.
What is the weighted error rate? (Assume uniform initial weights).
A) 0.10
B) 0.20
C) 0.25
D) 0.50
Answer: B. Initial weights are uniform (1/50 = 0.02). The error is the sum of weights of misclassified
instances: 10 * 0.02 = 0.20 (or simply 10/50).
13. [Numerical] Based on the previous question (error = 0.20), what is the Stage Weight (alpha) for this
stump? Formula: alpha = 0.5 * ln((1-error)/error)
A) 0.5 * ln(0.25)
B) 0.5 * ln(4)
C) 0.5 * ln(0.8)
D) 0.5 * exp(4)
Answer: B. alpha = 0.5 * ln((1-0.20)/0.20) = 0.5 * ln(0.80/0.20) = 0.5 * ln(4). This positive stage weight
gives the stump voting power.
14. What happens mathematically in AdaBoost if a weak learner achieves a weighted error rate of
exactly 0.50?
A) The stage weight becomes 1.
B) The stage weight becomes 0, and its vote is ignored.
C) The algorithm enters an infinite loop.
D) The sample weights are doubled.
Answer: B. alpha = 0.5 * ln((1-0.5)/0.5) = 0.5 * ln(1) = 0. A model with 50% error is essentially random
guessing, so it receives zero voting power.
15. [Numerical] A correctly classified sample currently has a weight of 0.10. The stump that just
classified it earned a stage weight of alpha = 0.70. What is the unnormalized updated weight of this
sample? (Note: y*h(x) = +1 for correct).
A) 0.10 * exp(0.70)
B) 0.10 * exp(-0.70)
C) 0.10 + 0.70
D) 0.10 * ln(0.70)
Answer: B. The update formula is w * exp(-alpha * y * h(x)). Since it was correct, y*h(x) is +1, so the
weight decreases: 0.10 * exp(-0.70).
16. Why is AdaBoost considered highly sensitive to outliers and noisy labels?
A) Outliers cause the learning rate to drop to zero.
B) Outliers are consistently misclassified, causing their sample weights to grow exponentially across iterations,
hijacking the model's focus.
C) Outliers reduce the depth of the decision stumps.
D) AdaBoost uses Squared Error loss, which squares outlier values.
Answer: B. Because AdaBoost exponentially increases weights for errors, an un-cleanable noisy label
will continuously attract weight until the model ruins its general boundaries trying to fit it.
17. Which loss function does standard binary AdaBoost explicitly minimize?
A) Log Loss (Cross Entropy)
B) Squared Error Loss
C) Exponential Loss
D) Hinge Loss
Answer: C. AdaBoost minimizes Exponential Loss: L(y, F(x)) = exp(-y * F(x)).
18. What is the difference between SAMME and SAMME.R algorithms in AdaBoost?
A) SAMME.R uses Regression, SAMME uses Classification.
B) SAMME.R uses class probability estimates (soft votes), while SAMME uses discrete class labels (hard
votes).
C) SAMME.R cannot handle multi-class problems.
D) There is no difference; they are aliases.
Answer: B. The 'R' stands for Real. SAMME.R calculates updates using logarithmic probabilities, which
generally converges much faster than discrete hard-label updates.
19. If you decrease the 'learning_rate' in AdaBoost from 1.0 to 0.1, what corresponding change should
you typically make to maintain capacity?
A) Increase the max_depth of the stumps.
B) Increase the n_estimators.
C) Decrease the n_estimators.
D) Switch to Bagging.
Answer: B. Learning rate and n_estimators are inversely proportional. Shrinking the learning rate
means each tree contributes less, so you need more trees (iterations) to reach optimal complexity.
20. What is the architectural definition of the default "Weak Learner" in AdaBoost?
A) A fully grown unpruned decision tree.
B) A Support Vector Machine with a linear kernel.
C) A Decision Stump (A tree with max_depth=1).
D) A Logistic Regression model.
Answer: C. AdaBoost relies on Decision Stumps — a root node with exactly two leaves, splitting on a
single feature.
Sub-Section 2.3: Gradient Boosting Machines (GBM) (Q21 - Q30)
21. How does Gradient Boosting fundamentally differ from AdaBoost in handling errors?
A) GBM drops misclassified samples.
B) GBM increases sample weights, while AdaBoost uses gradients.
C) GBM fits the new model directly to the negative gradients (pseudo-residuals) of the loss function, rather
than altering sample weights.
D) GBM builds trees in parallel.
Answer: C. GBM is an optimization in function space. It calculates the numerical residual error of the
ensemble, and trains the next tree to predict that exact residual value.
22. [Numerical] You are training a GBM regressor with Squared Error Loss. The true target y = 100. The
current ensemble prediction F(x) = 80. What is the pseudo-residual (target value) the next tree will train
on for this instance?
A) 100
B) 20
C) -20
D) 8000
Answer: B. For Squared Error, the pseudo-residual simplifies exactly to (True - Predicted). 100 - 80 =
+20.
23. In the GBM algorithm, what is the role of the initial prediction F0(x)?
A) It is a random baseline tree.
B) It is a constant scalar value that minimizes the global loss function (e.g., the mean for squared error).
C) It is always zero.
D) It is an AdaBoost stump.
Answer: B. GBM initializes with a single optimal constant value before sequentially adding trees to
adjust that constant.
24. [Numerical] A GBM model predicts a residual of 15 for a leaf node. The learning rate (shrinkage) is
0.1. The previous ensemble prediction was 50. What is the updated ensemble prediction?
A) 65.0
B) 51.5
C) 48.5
D) 50.15
Answer: B. Update rule: F_new = F_old + (learning_rate * tree_prediction). 50 + (0.1 * 15) = 50 + 1.5 =
51.5.
25. What happens to the pseudo-residuals as a well-configured GBM model approaches convergence?
A) They increase exponentially.
B) They approach zero, mimicking white noise.
C) They converge to the global mean of the target.
D) They alternate between exactly +1 and -1.
Answer: B. As the ensemble learns the underlying pattern, the remaining errors (residuals) get smaller
and smaller, eventually representing only unlearnable random dataset noise.
26. Which of the following allows Stochastic Gradient Boosting to reduce variance and prevent
overfitting?
A) Setting 'subsample' < 1.0 (e.g., 0.8), forcing each tree to train on a random fraction of the data.
B) Squaring the residuals.
C) Using deep trees.
D) Setting learning rate to 1.0.
Answer: A. Friedman introduced stochasticity by subsampling rows without replacement for each tree,
marrying the variance-reduction benefits of Bagging with Boosting.
27. Why does GBM generally use slightly deeper trees (e.g., max_depth 3 to 6) compared to AdaBoost
(max_depth 1)?
A) Because function space optimization is immune to overfitting.
B) To capture multi-variable interaction effects that single-split stumps cannot see.
C) To slow down the execution speed intentionally.
D) Because gradients cannot be calculated on stumps.
Answer: B. A stump (depth 1) can only evaluate one feature per iteration. Depth 3+ allows the tree to
chain conditions (e.g., IF Age > 30 AND Salary > 50k), naturally modeling feature interactions.
28. Which loss function makes a GBM Regressor highly robust to extreme target outliers?
A) Squared Error Loss
B) Absolute Error Loss (L1)
C) Log Loss
D) Exponential Loss
Answer: B. Absolute Error (L1) calculates the median and uses the absolute difference, preventing the
massive gradient spikes that Squared Error (L2) generates for extreme outliers.
29. In a GBM pipeline, if you observe the training error decreasing to zero while the validation error
begins to rise sharply, what is happening?
A) Underfitting
B) Overfitting (The model has started fitting the noise residuals).
C) The learning rate is too low.
D) Gradient explosion.
Answer: B. This is classic overfitting. Sequential boosting models will perfectly memorize the training
data if M (n_estimators) is too high without early stopping.
30. Is GBM inherently parallelizable at the tree-building level during training?
A) Yes, all trees can be built simultaneously.
B) No, Tree (m) strictly requires the residual outputs from Tree (m-1) to begin training.
C) Yes, if using GPU acceleration.
D) No, because decision trees cannot be parallelized.
Answer: B. The core boosting loop is strictly sequential. (Note: XGBoost parallelizes the *node
splitting* inside the tree, but the *sequence* of trees remains serial).

Sub-Section 2.4: XGBoost, LightGBM & CatBoost (Q31 - Q40)

31. What is the major mathematical advancement of XGBoost over standard GBM?
A) It uses exponential loss instead of squared error.
B) It utilizes a Second-Order Taylor Expansion (incorporating Hessians alongside Gradients) for faster, more
accurate loss minimization.
C) It relies exclusively on categorical feature encoding.
D) It converts the sequential process into a fully parallel one.
Answer: B. By using both first-derivative (Gradient) and second-derivative (Hessian) information,
XGBoost accurately approximates complex loss functions and converges faster.
32. [Numerical] In XGBoost, the optimal leaf weight formula is W* = -(Sum of Gradients) / (Sum of
Hessians + Lambda). If a leaf has a sum of gradients = -0.5, a sum of Hessians = 0.25, and L2 penalty
lambda = 1.0, what is the optimal weight w* assigned to this leaf?
A) -2.0
B) 0.4
C) -0.4
D) 2.0
Answer: B. w* = -(-0.5) / (0.25 + 1.0) = 0.5 / 1.25 = 0.4
33. What does the 'gamma' hyperparameter control in XGBoost?
A) The learning rate shrinkage.
B) The L1 regularization of weights.
C) The minimum structure Gain required to make a further partition on a leaf node (Tree Pruning).
D) The fraction of columns used per split.
Answer: C. Gamma acts as a complexity penalty. If the calculated Gain of a potential split is less than
Gamma, XGBoost aborts the split (pruning).
34. How does XGBoost handle missing values natively?
A) It imputes them with the median of the column.
B) It throws an error and requires manual imputation.
C) It learns an optimal default direction (left or right branch) for missing values during the training split
evaluation.
D) It drops rows with missing values.
Answer: C. XGBoost's Sparsity-Aware Split Finding algorithm dynamically routes missing data to the
left, then the right, and permanently assigns them to whichever direction yields the highest Gain.
35. Which specific mechanism allows LightGBM to process massive datasets faster than XGBoost?
A) Ordered Boosting
B) GOSS (Gradient-based One-Side Sampling) and Histogram-based binning.
C) Deep Neural layers
D) Disabling regularization
Answer: B. LightGBM bins continuous features into histograms (reducing split points) and uses GOSS
to drop data points with small gradients, drastically reducing the required computational payload.
36. Unlike standard level-wise tree growth, how does LightGBM grow its trees?
A) Root-wise
B) Symmetric-wise
C) Leaf-wise (Best-first), splitting the leaf with the maximum delta loss regardless of depth level.
D) Randomly
Answer: C. Leaf-wise growth produces highly asymmetric trees that minimize loss faster, though it is
more prone to overfitting on small datasets without setting 'num_leaves' constraints.
37. CatBoost (Categorical Boosting) introduces "Ordered Boosting." What specific problem does this
solve?
A) High RAM consumption
B) Prediction Shift/Target Leakage caused by using the same data instances to calculate residuals and train
the model.
C) Missing values
D) Slow inference time
Answer: B. Ordered Boosting uses artificial time-step permutations, ensuring that the residual
calculation for a data point relies only on a model trained on data points that came *before* it in the
permutation.
38. Why is CatBoost exceptionally fast during the prediction (inference) phase in production
environments?
A) It uses linear regression under the hood.
B) It utilizes Oblivious (Symmetric) Trees, meaning the exact same split condition applies across an entire
depth level, allowing rapid bitwise evaluation.
C) It drops 50% of the trees.
D) It converts to AdaBoost during inference.
Answer: B. Symmetric trees are highly structured and balanced. A CPU can evaluate the path for a data
point simultaneously using array indices, making it vastly faster than traversing asymmetrical trees.
39. If you have a tabular dataset where columns are highly sparse and mutually exclusive (e.g., large
One-Hot Encoded text matrices), which LightGBM feature optimizes this?
A) EFB (Exclusive Feature Bundling)
B) GOSS
C) SAMME.R
D) Taylor Expansion
Answer: A. EFB merges mutually exclusive sparse features into a single dense feature, heavily
reducing feature dimensionality without losing information.
40. In XGBoost, if you increase the L2 regularization parameter 'reg_lambda', what is the mathematical
effect on the optimal leaf weight denominator?
A) It decreases the denominator, blowing up the weights.
B) It increases the denominator, forcibly shrinking the final output weight closer to zero.
C) It removes the denominator.
D) It shifts the Hessian to zero.
Answer: B. The denominator is (Sum Hessians + lambda). Increasing lambda inflates the denominator,
which naturally scales the calculated weight (w*) down, buffering against extreme outlier-driven
weights.

Sub-Section 2.5: Advanced Comparisons & Hyperparameter Strategy (Q41 - Q50)

41. You are dealing with a massively imbalanced dataset (e.g., 99% legitimate, 1% fraud). Which
XGBoost parameter is most critical to tune to adjust the loss function focus?
A) max_depth
B) scale_pos_weight
C) subsample
D) n_estimators
Answer: B. scale_pos_weight acts as a multiplier applied to the gradient/Hessian of the minority class,
forcing the algorithm to penalize minority misclassifications much harder.
42. When comparing Random Forest and Gradient Boosting, how does their relationship with tree
depth differ?
A) Both require shallow trees.
B) RF uses shallow trees to reduce bias; GBM uses deep trees to reduce variance.
C) RF requires deep, unpruned trees to maintain high variance for bagging; GBM requires shallow trees to
maintain high bias for sequential correction.
D) Both require deep trees.
Answer: C. Bagging relies on unstable (deep) models. Boosting relies on weak, highly stable (shallow)
models to slowly march toward the target.
43. Early Stopping is a critical mechanism in Boosting. Based on what metric is early stopping usually
triggered?
A) When training error hits zero.
B) When a hold-out Validation Error stops improving for a specified number of rounds.
C) When memory limit is reached.
D) When learning rate becomes negative.
Answer: B. Early stopping monitors a distinct validation set. If validation loss plateaus or rises while
training loss continues dropping (overfitting), the training loop breaks early.
44. Which algorithm provides the most robust, native support for raw categorical text columns (e.g.,
User IDs, City Names) without requiring manual pre-processing?
A) Standard scikit-learn AdaBoost
B) XGBoost
C) CatBoost
D) Random Forest
Answer: C. CatBoost natively calculates dynamic target statistics for categorical strings on the fly,
eliminating the need for vast One-Hot matrices.
45. What is the fundamental trade-off when tuning 'learning_rate' and 'n_estimators' in any GBM
framework?
A) They must add up to 1.0.
B) A lower learning rate provides a smoother, more generalized decision boundary but strictly requires a higher
n_estimators (more compute time) to converge.
C) Higher learning rates require higher n_estimators.
D) They do not affect each other.
Answer: B. Shrinking the step size (learning rate) means the algorithm takes smaller steps down the
gradient. To reach the minimum, it mathematically must take more steps (estimators).
46. [Numerical] In XGBoost classification, the final output for an instance is a margin log-odds score of
+2.0. Using the Sigmoid function, what is the probability output for this class? (Assume e^2 approx
7.39)
A) 0.50
B) 0.12
C) 0.88
D) 1.00
Answer: C. 1 / (1 + e^-2.0) = 1 / (1 + 1/7.39) = 1 / (1 + 0.135) = 1 / 1.135 = 0.88. The model predicts the
positive class with 88% confidence.
47. If you deploy a LightGBM model on a dataset with only 500 rows and use default settings, what is
the most severe risk?
A) The model will crash.
B) Severe overfitting due to unrestricted Leaf-wise tree growth perfectly memorizing the tiny dataset.
C) Underfitting.
D) EFB will delete all columns.
Answer: B. LightGBM's documentation explicitly warns against using it on small datasets (under
10,000 rows) because its aggressive leaf-wise growth will instantly overfit small samples.
48. Which ensemble algorithm utilizes an exponential penalty to structural complexity: Obj = Loss +
gamma*T + 0.5*lambda*Sum(w^2)?
A) AdaBoost
B) Random Forest
C) XGBoost
D) Standard GBM
Answer: C. This is the exact Regularized Objective Function equation engineered exclusively for
XGBoost.
49. What is "Column Subsampling" (colsample_bytree) in XGBoost conceptually equivalent to?
A) GOSS in LightGBM
B) AdaBoost sample weight updates
C) The random feature selection mechanism at node splits utilized by Random Forests.
D) Early stopping
Answer: C. Column subsampling introduces horizontal stochasticity, randomly hiding columns from
trees to force structural diversity and reduce overfitting, exactly mimicking Random Forest behavior.
50. You must deliver a highly interpretable model for a banking compliance board. Which of the
following is the EASIEST to natively unroll into plain-English IF-THEN rules without relying on SHAP
values?
A) LightGBM
B) XGBoost
C) Random Forest
D) AdaBoost (using depth-1 Decision Stumps)
Answer: D. Because AdaBoost relies on max_depth=1 stumps, every single weak learner is a basic 1-
variable rule. You can easily print the entire ensemble out as a list of simple weighted rules. Deep trees
(GBM/RF) are virtually impossible to interpret natively.
Section 3: Support Vector Machines (SVM)
Support Vector Machines (SVM): Comprehensive [Link] Exam

Sub-Section 1: Geometric Intuition & Basics

1. What is the primary objective of a Support Vector Machine in classification?
A) To minimize the log-loss of the training data
B) To maximize the margin between different classes
C) To calculate the probability of a data point belonging to a class
D) To group unlabelled data into distinct clusters
Answer: B
2. In a 2D feature space, the SVM decision boundary is a line. In an n-dimensional space, what is this
boundary called?
A) Hyperplane
B) Orthogonal plane
C) Support Vector
D) Margin gutter
Answer: A
3. What happens to the SVM decision boundary if you delete a massive cluster of data points that are
safely located far behind the margin?
A) The boundary shifts towards the deleted points.
B) The margin width increases.
C) The boundary remains completely unchanged.
D) The model requires retraining to avoid a crash.
Answer: C
4. Why do we typically label the two classes as +1 and -1 in SVM mathematics instead of 1 and 0?
A) It saves memory in the computer.
B) It prevents division by zero.
C) It allows us to combine the positive and negative constraints into a single mathematical inequality.
D) Scikit-learn requires all labels to be negative.
Answer: C
5. Which of the following best describes "Support Vectors"?
A) The vectors that contain the highest feature values.
B) The data points closest to the decision boundary that dictate its position.
C) The perpendicular weight vector w.
D) The points that are misclassified by the algorithm.
Answer: B
6. If the equation of the decision boundary is w^T x + b = 0, what is the geometric relationship of the
weight vector w to the boundary?
A) It is parallel to the boundary.
B) It is perfectly perpendicular (orthogonal) to the boundary.
C) It represents the length of the boundary line.
D) It acts as the intercept of the boundary.
Answer: B
7. In a Hard Margin SVM, what is the mathematical formula for the total width of the margin?
A) ||w||
B) 1/2 ||w||^2
C) 2 / ||w||
D) w^T x + b
Answer: C
8. Which algorithm does not explicitly care about maximizing the distance to the nearest points,
potentially resulting in a boundary dangerously close to one class?
A) Hard Margin SVM
B) Soft Margin SVM
C) Logistic Regression
D) Support Vector Regression
Answer: C
Sub-Section 3.2: The Mathematics of SVM
9. To maximize the margin mathematically, what does the SVM objective function try to minimize?
A) 1/2 ||w||^2
B) 2 / ||w||
C) Sum of alpha_i
D) The bias b
Answer: A
10. In the Primal constraint y_i(w^T x_i + b) >= 1, what does the 1 represent?
A) The number of misclassifications allowed.
B) A strict requirement that all points lie on or behind the margin gutters.
C) The distance from the origin.
D) The maximum value of the bias.
Answer: B
11. Why do we convert the Primal Problem into the Dual Formulation using Lagrange Multipliers
(alpha)?
A) To make the algorithm run slower but more accurately.
B) To remove the dependence on w and b during the optimization phase.
C) To convert the classification problem into a regression problem.
D) To allow the use of strings and text data directly.
Answer: B
12. In the Dual Formulation, how do the training data points (x_i and x_j) interact with each other?
A) Through Euclidean distance addition.
B) Through a dot product (x_i^T x_j).
C) Through matrix inversion.
D) They do not interact.
Answer: B
13. According to the KKT conditions, if the optimizer assigns a Lagrange Multiplier alpha_i = 0 to a
specific data point, what does this physically mean?
A) The point is a Support Vector.
B) The point is misclassified.
C) The point is safely behind the margin and exerts no force on the boundary.
D) The point is an outlier and caused the model to crash.
Answer: C
14. Once the optimizer finishes the Dual problem and finds the alpha values, how is the final global
weight vector w calculated?
A) w is randomly initialized.
B) By summing the product of alpha_i, y_i, and x_i for the Support Vectors.
C) By taking the inverse of the features matrix.
D) By calculating the dot product of all non-support vectors.
Answer: B
15. If a point sits exactly on the positive margin gutter, what will the equation w^T x + b evaluate to?
A) 0
B) +1
C) -1
D) Infinity
Answer: B

Sub-Section 3.3: Soft Margin & Slack Variables

16. What is the fatal flaw of a Hard Margin SVM?
A) It cannot handle text data.
B) It will fail to find a solution if a single outlier makes the data linearly inseparable.
C) It uses too much memory.
D) It always underfits the data.
Answer: B
17. What does the Slack Variable (xi_i) measure?
A) The physical distance of a point from the decision boundary.
B) The number of features in a dataset.
C) The exact amount a data point violates the margin rule.
D) The learning rate of the optimizer.
Answer: C
18. If a data point is correctly classified and sits safely behind its margin gutter, what is its Slack
Variable (xi_i) value?
A) -1
B) 0
C) 1
D) It equals the hyperparameter C.
Answer: B
19. If a positive class data point (+1) has a Slack Variable xi = 2.5, where is it physically located?
A) Safely in the positive territory.
B) Exactly on the positive gutter.
C) Inside the margin but still on the positive side.
D) It has crossed the central hyperplane and is completely misclassified.
Answer: D
20. What is the name of the loss function used to calculate the Slack penalty: max(0, 1 - y_i(w^T x_i +
b))?
A) Log Loss
B) Mean Squared Error
C) Hinge Loss
D) Cross-Entropy Loss
Answer: C
21. In the Soft Margin objective function (Minimize 1/2 ||w||^2 + C * Sum(xi_i)), what does setting a
VERY HIGH value for C do?
A) It heavily penalizes errors, leading to a narrow margin and potential overfitting.
B) It relaxes the penalty, leading to a wide margin and potential underfitting.
C) It forces all Slack variables to equal zero.
D) It changes the kernel to RBF automatically.
Answer: A
22. If you set C=0.01 (a very low value), how will the SVM handle a stubborn outlier?
A) It will aggressively twist the boundary to classify the outlier correctly.
B) It will accept the misclassification penalty to maintain a wider, stable margin.
C) It will crash because C must be > 1.
D) It will delete the outlier from the dataset.
Answer: B
23. How does the hyperparameter C appear in the Dual Equation constraints?
A) It acts as a ceiling for the Lagrange Multipliers: 0 <= alpha_i <= C.
B) It replaces the dot product of the data points.
C) It is added to the bias term b.
D) It does not appear in the Dual constraints at all.
Answer: A
24. In a Soft Margin SVM, which points are considered Support Vectors?
A) Only points exactly on the gutters.
B) Any point with alpha_i > 0 (including points on the gutters, inside the margin, or misclassified).
C) Only points that are misclassified.
D) All points in the training dataset.
Answer: B

Sub-Section 3.4: The Kernel Trick

25. When is it necessary to use the Kernel Trick?
A) When the dataset is perfectly linearly separable.
B) When there are more rows than columns.
C) When the classes overlap in a complex, non-linear pattern (e.g., a donut shape).
D) When we want to increase the speed of a linear classifier.
Answer: C
26. Which of the following best defines the "Kernel Trick"?
A) Manually engineering thousands of polynomial features before training.
B) Using a mathematical function to compute the dot product of points in a higher dimension without actually
transforming them.
C) Dropping features that have low correlation with the target.
D) Randomly shuffling the data to find a better boundary.
Answer: B
27. What is the most widely used Kernel function for non-linear data?
A) Linear Kernel
B) Polynomial Kernel
C) Sigmoid Kernel
D) Radial Basis Function (RBF) Kernel
Answer: D
28. To what dimensional space does the RBF Kernel conceptually map the data?
A) 3D space
B) 10D space
C) Infinite-dimensional space
D) It reduces it to 1D space
Answer: C
29. The RBF Kernel is defined by exp(-gamma ||x_i - x_j||^2). What does it fundamentally measure?
A) The slope of the line between two points.
B) The Euclidean distance (similarity) between two points.
C) The absolute sum of the features.
D) The cross-entropy of the labels.
Answer: B
30. In the RBF Kernel, what happens if you set the hyperparameter gamma very HIGH?
A) The influence of a single point reaches very far, creating a smooth, linear-like boundary.
B) The influence of a single point is tiny, creating jagged boundaries wrapping tightly around individual points
(Overfitting).
C) The model ignores all outliers.
D) The margin becomes infinitely wide.
Answer: B
31. If you have a dataset with 50,000 features (like TF-IDF vectors for text), which Kernel should you
default to?
A) RBF
B) Linear
C) Polynomial (Degree 5)
D) Sigmoid
Answer: B
32. What is the result of using an RBF Kernel with a massive C value and a massive gamma value?
A) A perfectly generalized model.
B) Extreme underfitting (a flat line).
C) Extreme overfitting (a model that memorized the training data).
D) A syntax error in Python.
Answer: C

Sub-Section 3.5: Support Vector Regression (SVR)

33. While SVC tries to keep the "street" empty, what is the geometric goal of Support Vector
Regression (SVR)?
A) To keep the street empty but curved.
B) To fit as many data points as possible inside the street (the tube).
C) To maximize the number of Support Vectors.
D) To separate the data into clusters.
Answer: B
34. What is the epsilon hyperparameter in SVR?
A) The learning rate of the optimizer.
B) A margin of tolerance around the prediction line where errors are ignored (considered zero).
C) The penalty applied to misclassified points.
D) The polynomial degree of the kernel.
Answer: B
35. In SVR, if a data point falls inside the epsilon-insensitive tube, what is its Slack penalty (xi)?
A) xi > 0
B) xi < 0
C) xi = 0
D) xi = epsilon
Answer: C
36. Why is SVR often preferred over standard Ordinary Least Squares (Linear Regression) for noisy
data?
A) It calculates faster.
B) It natively outputs probability distributions.
C) The epsilon-tube and C parameter prevent massive outliers from aggressively skewing the line of best fit.
D) It doesn't require feature scaling.
Answer: C
37. True or False: You can use the RBF Kernel with Support Vector Regression (SVR).
A) True
B) False
Answer: A
38. What does the SVR objective function attempt to minimize to keep the regression line smooth and
prevent overfitting?
A) The bias b
B) 1/2 ||w||^2
C) The number of features
D) The epsilon width
Answer: B

Sub-Section 3.6: Assumptions, Pros, Cons & Use Cases

39. Which preprocessing step is absolutely non-negotiable and mandatory before training an SVM?
A) Feature Engineering
B) Principal Component Analysis (PCA)
C) Feature Scaling (e.g., Standardization)
D) One-Hot Encoding of target labels
Answer: C
40. Why does unscaled data (e.g., Age 0-100 vs. Salary 0-1,000,000) destroy SVM accuracy?
A) Because the algorithm cannot handle large numbers.
B) Because SVM maximizes distance, and the feature with the largest magnitude will mathematically dominate
the distance calculations.
C) Because it causes division by zero.
D) Because scaling is required by Python syntax.
Answer: B
41. What is a major disadvantage of SVMs compared to Decision Trees?
A) They cannot handle non-linear data.
B) They are highly sensitive to unscaled data and operate as "black boxes" when using the RBF kernel.
C) They always get stuck in local minima.
D) They cannot be used for regression.
Answer: B
42. For which of the following dataset sizes is SVM generally a bad choice due to its O(n^2) to O(n^3)
complexity?
A) 1,000 rows
B) 10,000 rows
C) 50,000 rows
D) 5,000,000 rows
Answer: D
43. If an algorithmic trading bot uses SVM to predict Buy (+1) or Sell (-1) signals, how does the model
ensure a global minimum is reached during training?
A) By running multiple random starts.
B) Because the objective function is a convex quadratic programming problem.
C) By dropping half the dataset.
D) By relying entirely on the Moving Average feature.
Answer: B
44. Does a standard SVM natively output probabilities (e.g., "70% sure this is Class 1")?
A) Yes, inherently through the Sigmoid function.
B) No, it only outputs a hard boundary prediction (+1 or -1), and requires expensive Platt Scaling to estimate
probabilities.
C) Yes, through the Dual Formulation.
D) No, it cannot output probabilities under any circumstances.
Answer: B

Sub-Section 3.7: Scikit-Learn Implementation

45. If you are predicting stock market directions and have 800 "Hold" days and only 200 "Buy" days,
which Scikit-Learn parameter prevents the SVM from ignoring the "Buy" days?
A) kernel='linear'
B) probability=True
C) class_weight='balanced'
D) random_state=42
Answer: C
46. In Scikit-learn, what happens if you set probability=True in SVC()?
A) The model trains significantly faster.
B) The model runs an internal 5-fold cross-validation, slowing down training considerably.
C) The model switches to Logistic Regression.
D) The model disables the C parameter.
Answer: B
47. When using train_test_split on historical financial data (time-series) for an SVM trading model,
which parameter must be set?
A) stratify=y
B) shuffle=False
C) test_size=0.9
D) random_state=None
Answer: B
48. What is the risk of calling scaler.fit_transform() on the entire dataset before splitting it into train and
test sets?
A) Syntax error
B) Memory overflow
C) Data Leakage (The scaler calculates the mean using future test data, giving the model an unfair
advantage).
D) It drops all missing values automatically.
Answer: C
49. If your SVM optimizer hangs in an infinite loop while trying to fit incredibly noisy data, which Scikit-
Learn parameter acts as a failsafe to break the loop?
A) degree
B) max_iter
C) gamma
D) class_weight
Answer: B
50. When performing Hyperparameter Tuning for an RBF SVM using GridSearchCV, which two
parameters are almost always searched together?
A) C and kernel
B) C and gamma
C) epsilon and degree
D) max_iter and probability
Answer: B
Section 4: Dimensionality Reduction (PCA & t-SNE)
Dimensionality Reduction & PCA
1. Before calculating the covariance matrix for standard PCA, what is the most critical preprocessing
step?
A) Normalizing features to a range of 0 to 1
B) Mean-centering the data
C) Removing all outliers
D) Taking the natural logarithm of all features
Answer: B
Explanation: If data isn't centered at the origin, the first principal component will just point towards the mean of
the data rather than the direction of maximum variance.
2. The covariance matrix computed during PCA is always guaranteed to be:
A) Identity
B) Diagonal
C) Symmetric
D) Orthogonal
Answer: C
Explanation: The covariance matrix C is calculated as 1/(n-1) X^T X (where X is mean-centered). By definition
of matrix transposition, (X^T X)^T = X^T X, making it perfectly symmetric.
3. What do the eigenvectors of the covariance matrix represent in PCA?
A) Feature means
B) Directions of maximum variance
C) Projection errors
D) Data normalization factors
Answer: B
Explanation: Eigenvectors point in the directions where the data is most spread out.
4. What do the eigenvalues corresponding to each eigenvector represent?
A) The noise threshold of the dataset
B) The mean of the projected data
C) The magnitude of variance captured along that principal component
D) The correlation coefficient between original features
Answer: C
Explanation: The eigenvalue lambda_i tells you exactly how much variance is captured along its corresponding
eigenvector.
5. In Singular Value Decomposition (SVD) where A = U Sigma V^T, what do the singular values
(diagonal entries of Sigma) correspond to?
A) The eigenvectors of the covariance matrix
B) The square roots of the eigenvalues of A^T A
C) The feature weights of the original data
D) The magnitude of noise in the data
Answer: B
Explanation: In SVD, the singular values in Sigma are exactly the square roots of the eigenvalues of the
covariance matrix A^T A.
6. The principal components generated by PCA are mathematically guaranteed to be:
A) Sparse
B) Independent
C) Uncorrelated
D) Highly interpretable
Answer: C
Explanation: Because the eigenvectors of a symmetric matrix are strictly orthogonal, projecting data onto them
results in new features that have zero linear correlation.
7. The first principal component (PC1) is defined as the direction in space that:
A) Minimizes the variance of the projected data
B) Maximizes the variance of the projected data
C) Perfectly separates categorical classes
D) Eliminates all outliers
Answer: B
Explanation: This is the core mathematical objective function of PCA.
8. The second principal component (PC2) must satisfy which of the following conditions?
A) It must capture more variance than PC1
B) It must be completely parallel to PC1
C) It must be orthogonal to PC1 and capture the maximum remaining variance
D) It must have an eigenvalue equal to 1
Answer: C
Explanation: PCA solves a constrained optimization problem. Each subsequent component must capture the
maximum remaining variance while being completely perpendicular (orthogonal) to all previous ones.
9. Why is standardization (scaling features to unit variance) typically recommended before applying
PCA?
A) PCA is scale-invariant, so scaling speeds up computation
B) PCA is sensitive to the scale of the original features; larger scales will disproportionately dominate the
components
C) It automatically removes collinearity
D) It converts the dataset into a normal distribution
Answer: B
Explanation: Variance is a squared metric. If one feature is measured in millimeters and another in kilometers,
the millimeter feature's artificially high variance will dominate the eigenvectors.
10. What metric is most commonly used to determine how many principal components to keep?
A) Classification accuracy
B) Silhouette score
C) Explained variance ratio
D) Learning rate
Answer: C
Explanation: Calculated as lambda_i / Sum(lambda_j), this tells you the percentage of total information
retained by a given component.
11. Which visual tool is traditionally used to plot the explained variance against the number of principal
components?
A) Scatter plot
B) Scree plot
C) Box plot
D) Dendrogram
Answer: B
Explanation: A line plot of eigenvalues used to find the "elbow" where adding more components yields
diminishing returns.
12. If the original dataset has 10 features, what is the maximum number of principal components that
PCA can extract?
A) 5
B) 9
C) 10
D) Infinite
Answer: C
Explanation: The maximum rank of a covariance matrix is equal to the number of original features (assuming
the number of samples N > D).
13. In PCA, preserving the first k components explicitly minimizes which of the following?
A) Classification error
B) Model bias
C) Total dataset variance
D) Reconstruction error
Answer: D
Explanation: PCA mathematically minimizes the mean squared orthogonal distance from the original data
points to the projected lower-dimensional hyperplane.
14. Mathematically, the reconstruction error after applying PCA dimensionality reduction is equal to:
A) The sum of the retained eigenvalues
B) The mean squared distance between original and reconstructed points
C) The cross-entropy loss of the features
D) The sum of the feature means
Answer: B
Explanation: This is the literal definition of the reconstruction error.
15. A major drawback of using t-SNE instead of PCA as a preprocessing step for a downstream
machine learning classifier is that t-SNE:
A) Cannot reduce dimensions below 3
B) Requires labeled data to function
C) Is too fast, leading to underfitting
D) Does not learn a parametric mapping function to easily transform new, unseen test data
Answer: D
Explanation: t-SNE is a non-parametric visualization algorithm. It learns the low-dimensional embedding for the
training data but does not yield a function f(x) to apply to new, unseen test data.
16. How does PCA differ fundamentally from Linear Discriminant Analysis (LDA)?
A) PCA is unsupervised (ignores labels), while LDA is supervised (maximizes class separability)
B) PCA maximizes classification accuracy, while LDA minimizes reconstruction error
C) PCA works only on non-linear data, while LDA works on linear data
D) There is no difference; they are different names for the same algorithm
Answer: A
Explanation: PCA only looks at feature variance (ignores labels). LDA looks at labels to find axes that
maximize the distance between different classes.
17. If your data lies on a highly complex, non-linear manifold (like a rolled-up "Swiss Roll"), standard
PCA will likely fail to capture the true structure because:
A) PCA can only capture linear correlations
B) PCA requires the data to be exactly normally distributed
C) The covariance matrix cannot be calculated for complex data
D) Non-linear data has no variance
Answer: A
Explanation: Standard PCA is strictly a linear transformation and cannot "unroll" complex non-linear manifolds.
18. What happens if you apply PCA to a dataset where all original features are already perfectly
uncorrelated and have a variance of 1?
A) The algorithm will fail to converge
B) The resulting covariance matrix will be an identity matrix, and PCA will not provide any meaningful
dimensionality reduction
C) PCA will perfectly separate all classes
D) The eigenvalues will all be exactly 0
Answer: B
Explanation: If features are already perfectly uncorrelated with unit variance, the covariance matrix is an
Identity matrix, meaning no direction has more variance than any other.
19. When using PCA for image compression, what is the typical consequence of dropping the
components with the smallest eigenvalues?
A) The primary structure of the image is lost
B) The image becomes completely inverted
C) High-frequency details and noise are discarded, retaining the broader image structure
D) The image file size increases
Answer: C
Explanation: Components with the smallest eigenvalues represent the least variance, which in images usually
corresponds to minor details or background noise.
20. Which algorithm provides a non-linear alternative to PCA by using the "kernel trick"?
A) Truncated SVD
B) Kernel PCA
C) t-SNE
D) UMAP
Answer: B
Explanation: Maps data into a higher-dimensional space using a kernel function (like RBF) to make non-linear
data linearly separable.
21. If a dataset has N samples and D features, and N < D (e.g., microarray data), how many meaningful
principal components with non-zero eigenvalues can be extracted at most?
A) D
B) N - 1
C) N + D
D) D - N
Answer: B
Explanation: In "wide" datasets (like genomics), the rank of the mean-centered data matrix is at most N-1. Any
components beyond that will have an eigenvalue of 0.
22. Which matrix operation is generally preferred for computationally efficient PCA on large, sparse
datasets?
A) Eigendecomposition of the covariance matrix
B) Singular Value Decomposition (SVD) directly on the data matrix
C) Gaussian Elimination
D) Cholesky Decomposition
Answer: B
Explanation: Computing the massive D x D covariance matrix directly is computationally explosive. SVD
extracts the same components much more efficiently directly from the data matrix.
23. The total variance of the original dataset is mathematically equal to:
A) The trace (sum of diagonal elements) of the covariance matrix
B) The determinant of the covariance matrix
C) The largest eigenvalue alone
D) The product of all eigenvalues
Answer: A
Explanation: A beautiful property of linear algebra: the total variance of the data is the sum of the diagonal of
the covariance matrix, which is mathematically equal to the sum of all its eigenvalues.
24. If the first two principal components capture 98% of the total variance, what does this imply?
A) The data is completely random
B) The remaining components contain 98% noise
C) The dataset can be effectively represented in a 2D space with minimal loss of information
D) The dataset has exactly two features
Answer: C
Explanation: You can drop all other features and still retain 98% of the mathematical relationships in the
dataset.
25. PCA is fundamentally sensitive to:
A) The order of the rows in the dataset
B) Outliers, because they pull the direction of maximum variance toward themselves
C) The specific machine learning library being used
D) Whether the target variable is continuous or categorical
Answer: B
Explanation: Because variance relies on squared differences from the mean, extreme outliers drastically skew
the direction of maximum variance.
26. In t-SNE, the algorithm optimizes the lower-dimensional embedding by minimizing which specific
mathematical metric?
A) Mean Squared Error (MSE)
B) Cross-Entropy Loss
C) Kullback-Leibler (KL) Divergence
D) Frobenius Norm
Answer: C
Explanation: t-SNE converts similarities into conditional probabilities. It then minimizes the KL divergence,
which measures how one probability distribution diverges from a second reference probability distribution.
27. Is PCA considered a feature selection technique or a feature extraction technique?
A) Feature selection, because it selects the best original features.
B) Feature extraction, because it creates new features by combining the original ones.
C) Both, depending on the kernel used.
D) Neither; it is strictly a clustering algorithm.
Answer: B
Explanation: PCA does not pick and choose from existing features (selection); it calculates entirely new axes
that are linear combinations of the originals (extraction).
28. If w is an eigenvector of length 1, and C is the covariance matrix, PCA seeks to maximize which of
the following expressions?
A) w^T C w
B) C w
C) w^T w
D) C^T w
Answer: A
Explanation: The projected variance onto a vector w is given by w^T C w. PCA is the constrained optimization
problem of maximizing this term subject to w^T w = 1.
29. Which optimization technique is used to solve the constrained maximization problem in PCA
mathematically?
A) Gradient Descent
B) Lagrange Multipliers
C) Newton-Raphson Method
D) Simplex Algorithm
Answer: B
Explanation: To maximize variance subject to the unit-vector constraint, Lagrange multipliers are introduced,
which naturally leads to the eigenvalue equation C w = lambda w.
30. What does a "near-zero" eigenvalue in a PCA indicate about the original dataset?
A) The data is perfectly normally distributed.
B) The dataset is highly noisy.
C) There is high multicollinearity (linear dependence) among the original features.
D) The learning rate is too low.
Answer: C
Explanation: An eigenvalue of nearly 0 means there is a direction with almost no variance, indicating that one
or more features can be almost perfectly predicted by a linear combination of the others.
31. How does PCA typically handle categorical variables (e.g., "Red", "Blue", "Green")?
A) It assigns them higher eigenvalues.
B) It converts them into continuous variables using logarithms.
C) Standard PCA is not designed for categorical variables and they should generally be excluded or handled
with techniques like MCA.
D) It drops them automatically.
Answer: C
Explanation: PCA assumes continuous numeric data to calculate meaningful means, variances, and
covariances.
32. The concept of "Whitening" in PCA refers to:
A) Removing missing values from the dataset.
B) Scaling the principal components so that they all have a variance of exactly 1.
C) Deleting the components with the lowest eigenvalues.
D) Converting all negative values to positive values.
Answer: B
Explanation: Whitening transforms the data so the covariance matrix becomes the identity matrix, meaning all
new features are uncorrelated and have unit variance.
33. In t-SNE, what is the role of the "perplexity" hyperparameter?
A) It sets the maximum number of iterations for gradient descent.
B) It dictates the learning rate of the KL divergence minimization.
C) It balances attention between local and global aspects of the data, roughly acting as a guess for the number
of close neighbors each point has.
D) It controls the threshold for outlier rejection.
Answer: C
Explanation: Perplexity fundamentally changes the shape of the probability distributions in the high-
dimensional space, directly affecting how t-SNE balances local cluster tightness versus global data structure.
34. Which of the following best describes the asymmetry of KL Divergence used in t-SNE?
A) KL(P||Q) = KL(Q||P)
B) It heavily penalizes placing distant points close together, but does not heavily penalize placing close points
far apart.
C) It strictly measures Euclidean distance.
D) It can be negative.
Answer: B
Explanation: Because of how the KL divergence formula is structured (P * log(P/Q)), if P (high-D probability) is
large and Q (low-D probability) is small, the penalty is massive. If P is small and Q is large, the penalty is small.
35. If you have a dataset with 100 features, and you reduce it to 5 principal components, what is the
dimension of the resulting projection matrix (the weights)?
A) 5 x 5
B) 100 x 100
C) 100 x 5
D) 5 x 100
Answer: C
Explanation: You need a weight for each of the 100 original features to calculate each of the 5 new
components.
36. Why might Autoencoders be preferred over PCA for dimensionality reduction in certain deep
learning tasks?
A) Autoencoders are computationally faster than PCA.
B) Autoencoders do not require a loss function.
C) By using non-linear activation functions, Autoencoders can learn complex, non-linear mappings that PCA
cannot.
D) Autoencoders guarantee uncorrelated latent features.
Answer: C
Explanation: Standard PCA is mathematically equivalent to a single-layer linear autoencoder. Adding hidden
layers and non-linear activations allows autoencoders to capture much more complex data manifolds.
37. According to the Eckart-Young-Mirsky Theorem, PCA provides the optimal:
A) Margin of separation between two classes.
B) Low-rank approximation of a matrix in terms of the Frobenius norm.
C) Probability distribution of a dataset.
D) Learning rate schedule.
Answer: B
Explanation: This theorem mathematically guarantees that truncating SVD (which is what PCA does) gives the
best possible lower-rank matrix approximation with the smallest possible reconstruction error.
38. The eigenvectors in PCA are typically constrained to be "unit vectors". What does this mean?
A) Their sum equals 1.
B) They only contain 1s and 0s.
C) Their geometric length (L2 norm) is exactly 1.
D) They represent 1% of the variance.
Answer: C
Explanation: Without constraining the length of the vector w to 1 (||w|| = 1), you could trivially maximize the
variance (w^T C w) just by scaling up the values inside the vector w to infinity.
39. If PC1 and PC2 capture 40% and 30% of the variance respectively, what can be said about their
correlation?
A) They are 70% correlated.
B) They are 10% correlated.
C) They are negatively correlated.
D) They are completely uncorrelated (0 correlation).
Answer: D
Explanation: Regardless of how much variance they capture, all principal components are mathematically
enforced to be completely orthogonal, meaning their correlation is exactly zero.
40. What is a "Biplot" in the context of PCA?
A) A plot showing the error rates of two different machine learning models.
B) A graph that overlays both the projected data points and the original feature vectors (loadings) on the same
principal component axes.
C) A 3D plot of the first three components.
D) A plot used exclusively to find outliers.
Answer: B
Explanation: Biplots are highly useful for interpreting PCA because they show how each original feature
contributes to the principal components and how the samples group together.
41. Which algorithm is considered a modern, often faster alternative to t-SNE that better preserves
global data structure?
A) Truncated SVD
B) UMAP (Uniform Manifold Approximation and Projection)
C) Isomap
D) Independent Component Analysis (ICA)
Answer: B
Explanation: UMAP uses Riemannian geometry and algebraic topology to optimize the lower-dimensional
space, generally running faster than t-SNE and doing a better job at keeping distant clusters in their correct
relative positions.
42. What is the computational time complexity of calculating PCA via the eigendecomposition of the
covariance matrix for N samples and D features?
A) O(N log N)
B) O(D^2)
C) O(D^3)
D) O(N)
Answer: C
Explanation: Computing the covariance matrix takes O(N * D^2), and solving the eigendecomposition of a D x
D matrix takes O(D^3). This is why SVD is preferred for datasets with a massive number of features.
43. Independent Component Analysis (ICA) differs from PCA primarily because:
A) ICA looks for components that are statistically independent, not just linearly uncorrelated, making it useful
for blind source separation (like the cocktail party problem).
B) ICA is supervised while PCA is unsupervised.
C) ICA maximizes variance while PCA minimizes it.
D) ICA only works on text data.
Answer: A
Explanation: Uncorrelated (PCA) means no linear relationship. Independent (ICA) is a much stronger statistical
condition, meaning no relationship whatsoever.
44. "Robust PCA" alters the standard PCA algorithm specifically to deal with:
A) Small datasets.
B) Severe outliers and corrupted data by decomposing the matrix into a low-rank and a sparse component.
C) Categorical variables.
D) Time-series forecasting.
Answer: B
Explanation: Standard PCA uses the L2 norm (squared errors), which makes it highly sensitive to outliers.
Robust PCA modifies this to isolate sparse outlier noise.
45. In a pipeline, why should PCA be applied after splitting data into training and testing sets?
A) To speed up the train-test split function.
B) Because applying PCA on the whole dataset before splitting causes "data leakage," where information from
the test set influences the components used to train the model.
C) PCA cannot be applied to test sets.
D) It shouldn't; PCA must always be applied to the full dataset first.
Answer: B
Explanation: The principal components (the transformation matrix) must be learned exclusively from the
training data, and then that exact same transformation is applied to the test data.
46. If the variance of Feature A is 100, and the variance of Feature B is 1, without standardization, what
will happen in PCA?
A) PC1 will align almost perfectly with Feature B.
B) PC1 will align almost perfectly with Feature A.
C) PC1 will balance them equally at a 45-degree angle.
D) The algorithm will throw an error.
Answer: B
Explanation: PCA maximizes variance. Since Feature A has 100 times the variance, the algorithm will view it
as the most "important" direction by default, ignoring the actual underlying structure.
47. When reconstructing data from k principal components, the lost information is known to reside in
the:
A) Null space of the covariance matrix.
B) Orthogonal complement spanned by the discarded eigenvectors.
C) Trace of the retained eigenvectors.
D) Mean vector.
Answer: B
Explanation: The information you throw away lives exactly in the subspace defined by the components you
didn't keep.
48. Why does t-SNE suffer from the "crowding problem"?
A) High-dimensional data requires more space to represent distances accurately than a 2D map can provide,
causing points to crush together in the center.
B) The learning rate is usually set too high.
C) It cannot handle more than 1000 data points.
D) The KL divergence becomes negative.
Answer: A
Explanation: In high dimensions, the volume of space scales exponentially. When forcing that data into 2D,
there simply isn't enough "room" to represent all distances properly, so t-SNE uses Student's t-distribution to
help push clusters apart in the 2D space.
49. Truncated SVD is often preferred over standard PCA for which specific type of data?
A) Time-series data
B) Highly sparse data (like TF-IDF matrices in NLP) where mean-centering would destroy the sparsity.
C) Image data
D) Unscaled data
Answer: B
Explanation: Standard PCA requires mean-centering, which turns a sparse matrix (mostly zeros) into a dense
matrix, crashing computer memory. Truncated SVD operates directly on the sparse matrix without mean-
centering.
50. The sum of the explained variance ratios of all possible principal components in a dataset will
exactly equal:
A) 0
B) The number of features D
C) 1 (or 100%)
D) The trace of the matrix
Answer: C
Explanation: Because the components orthogonalize and partition the total variance of the dataset, their ratios
are fractions of the whole, which must always sum to 1.
Section 5: Imbalanced Data & SMOTE
Imbalanced Data, SMOTE & Practice Problems
Question 1: SMOTE Generation
You are analyzing a dataset of rare diseases. A minority class patient has blood metrics recorded as Point
P1(40,60). The nearest neighbor with the same disease is Point P2(70,90). If the SMOTE algorithm uses a
random multiplier of lambda=0.7, what will be the exact feature values of the newly generated synthetic
patient?
Answer: Difference = (70-40, 90-60) = (30, 30). Scaled d = 0.7 * (30,30) = (21,21). Synthetic Point t =
(40+21, 60+21) = (61,81).
Question 2: Feature Selection (Correlation Filter)
You are trying to predict a student's Final Grade (Y). You are evaluating a feature called Hours of Video
Games (X).
• Covariance between X and Y = -24
• Standard Deviation of X = 4
• Standard Deviation of Y = 8
A) Calculate the Pearson correlation coefficient (r) between the feature and the target.
B) Does this feature have a positive or negative linear relationship with the grade?
C) If your feature selection algorithm drops any feature whose absolute correlation (|r|) is less than 0.50, will
this feature be kept or dropped?
Answer: A) r = -24 / (4 * 8) = -24 / 32 = -0.75. B) Negative. C) |-0.75| = 0.75. Since 0.75 > 0.50, Kept.
Question 3: SMOTE in 3D (Advanced)
A minority class has three features (X, Y, Z).
Base Point = (10, 20, 30)
Nearest Neighbor = (15, 20, 45)
If SMOTE generates a synthetic point at (12, 20, 36), what was the random weight (lambda) chosen by the
algorithm?
Answer: Look at the X coordinate. Base is 10, Neighbor is 15. Difference is 5. Synthetic is 12. Distance
moved = 12 - 10 = 2. Lambda = Moved / Total Difference = 2 / 5 = 0.4.

Sub-Section 5.1: Core Concepts & The Accuracy Paradox

1. What defines an "Imbalanced Dataset" in machine learning?
A) A dataset where numerical features are on vastly different scales.
B) A dataset with significantly more columns (features) than rows (samples).
C) A classification dataset where one target class heavily outnumbers the other.
D) A dataset containing both categorical text and continuous numerical data.
Answer: C
2. Which of the following is the best real-world example of a highly imbalanced dataset?
A) Predicting whether a coin flip will land on Heads or Tails.
B) Predicting credit card fraud from a bank's daily transaction logs.
C) Predicting if a customer will buy a red shirt or a blue shirt.
D) Predicting housing prices based on square footage.
Answer: B
3. What is the "Accuracy Paradox"?
A) The more data you feed a model, the less accurate it becomes.
B) Models trained on imbalanced data can achieve 99% accuracy by simply guessing the majority class every
time, while failing to predict the minority class entirely.
C) Accuracy increases exponentially when you use Random Oversampling, but only on the test set.
D) Accuracy cannot be calculated if the minority class is less than 5% of the data.
Answer: B
4. If a model strictly guesses "Legitimate" for a dataset containing 990 Legitimate transactions and 10
Fraudulent transactions, what will its Recall for the Fraud class be?
A) 99%
B) 100%
C) 1%
D) 0%
Answer: D
5. Why is the F1-Score preferred over standard Accuracy for imbalanced datasets?
A) It strictly measures True Negatives.
B) It takes the harmonic mean of Precision and Recall, punishing the model if it completely ignores the minority
class.
C) It allows the model to train faster by dropping redundant data.
D) It converts categorical predictions into continuous probabilities.
Answer: B

Sub-Section 5.2: Rules & Basic Resampling

6. What is the "Golden Rule" of applying oversampling techniques like SMOTE?
A) Always apply SMOTE to both the Training and Testing datasets.
B) Apply SMOTE to the dataset before splitting it into Training and Testing sets.
C) Only apply SMOTE to the Training dataset after the train/test split.
D) Only apply SMOTE to the Testing dataset to see how it handles synthetic data.
Answer: C
7. What is the primary danger of using "Random Undersampling"?
A) It causes severe overfitting by duplicating exact rows.
B) It takes massive amounts of computational power.
C) It throws away a large portion of the majority class, potentially deleting critical learning patterns.
D) It creates synthetic noise along the decision boundary.
Answer: C
8. What is the primary danger of using "Random Oversampling"?
A) It causes severe overfitting because the model memorizes exact duplicate copies of the minority class.
B) It deletes important majority class data.
C) It cannot be used on tree-based models.
D) It shifts the cluster centroids away from the true mean.
Answer: A
9. When using the class_weight='balanced' parameter in Scikit-Learn, how does the algorithm adjust to
imbalanced data?
A) It randomly deletes rows from the majority class during training.
B) It generates synthetic data points behind the scenes.
C) It mathematically multiplies the error penalty (Loss Function) when the model misclassifies a minority point,
forcing the model to pay more attention to it.
D) It converts the classification problem into a regression problem.
Answer: C

Sub-Section 5.3: Advanced Undersampling

10. How does the "Tomek Links" technique identify data points to remove?
A) It randomly selects 10% of the majority class and drops it.
B) It finds a majority and minority point that are each other's absolute closest neighbors, and deletes the
majority point to clear the boundary.
C) It uses K-Means to find the center of the majority class and deletes everything outside of it.
D) It deletes any point that is more than 3 standard deviations from the mean.
Answer: B
11. Which undersampling technique uses K-Means clustering to replace the original majority class
points with mathematically generated "center" points?
A) Tomek Links
B) NearMiss
C) SMOTE
D) Cluster Centroids
Answer: D
12. How does the NearMiss algorithm select which majority points to keep?
A) It keeps the majority points that are furthest away from the minority class to prevent overlap.
B) It keeps the majority points whose average distance to the nearest minority points is the smallest, helping to
define a sharp decision boundary.
C) It keeps majority points completely at random.
D) It keeps only the majority points that share the exact same values as the minority class.
Answer: B

Sub-Section 5.4: Advanced Oversampling (SMOTE Family)

13. How does standard SMOTE generate a synthetic data point?
A) It randomly duplicates an existing minority row and adds minor Gaussian noise.
B) It picks a minority point, draws a vector line to one of its K-nearest minority neighbors, and places a new
point randomly along that line.
C) It finds the exact mathematical average of all minority points and places a synthetic point there.
D) It uses a neural network to generate fake images of the data.
Answer: B
14. What specific problem does Borderline-SMOTE solve compared to standard SMOTE?
A) Standard SMOTE cannot handle categorical variables.
B) Standard SMOTE is too computationally heavy.
C) Standard SMOTE treats all minority points equally, which can generate "bridges" of noise around extreme
minority outliers. Borderline-SMOTE only generates points along the decision boundary.
D) Borderline-SMOTE generates majority class points instead of minority class points.
Answer: C
15. How does the ADASYN algorithm decide how many synthetic points to generate for a specific
minority point?
A) It generates exactly 5 points for every original point.
B) It calculates a density ratio based on how many majority enemies surround the point; it generates more
synthetic points for minority points that are harder to learn.
C) It generates fewer points for minority points that are surrounded by majority points.
D) It asks the user to manually input the desired ratio.
Answer: B
16. If your imbalanced dataset contains both numerical values (like 'Income') and categorical strings
(like 'City'), which SMOTE variant MUST you use?
A) SMOTE-Tomek
B) Borderline-SMOTE
C) ADASYN
D) SMOTE-NC
Answer: D

Sub-Section 5.5: Hybrid Methods & Math

17. What is the core philosophy behind Hybrid Resampling Methods like SMOTE-Tomek?
A) To use multiple machine learning models at the same time.
B) To oversample the majority class while undersampling the minority class.
C) To use SMOTE to balance the total numbers, and then immediately use an undersampling technique to
clean up the blurry/noisy boundary created by SMOTE.
D) To use SMOTE on the training data and Tomek Links on the testing data.
Answer: C
18. Which Hybrid method is known for the most aggressive "deep cleaning", deleting any point whose
class contradicts the majority vote of its neighbors?
A) SMOTE-ENN (Edited Nearest Neighbors)
B) SMOTE-Tomek
C) Random-SMOTE
D) ADASYN-Centroids
Answer: A
19. A minority point (A) is located at coordinates (10, 20). Its nearest neighbor (B) is at (20, 30). If
SMOTE uses a random weight of lambda = 0.5, what are the coordinates of the newly generated
synthetic point?
A) (10, 20)
B) (15, 25)
C) (30, 50)
D) (5, 10)
Answer: B
20. A model evaluates 100 cancer scans. It predicts 5 people have cancer. Of those 5, 4 actually have
cancer, and 1 is a healthy person (false alarm). What is the Precision of this model?
A) 100%
B) 80%
C) 20%
D) 0%
Answer: B
Section 6: Mixed Advanced Practice
Section 6: Mixed Advanced Practice (Logistic, Regressions, KNN, Loss Functions)
Q.1 You are training a logistic regression model on an imbalanced dataset containing 5,000 total
samples. The dataset is distributed as follows: Class 0: 4,000 samples, Class 1: 1,000 samples. Using
the standard balanced weight formula, Calculate the weights that should be assigned to Class 0 and
Class 1.
A. Weight (Class 0) = 0.20, Weight (Class 1) = 0.80
B. Weight (Class 0) = 1.25, Weight (Class 1) = 5.00
C. Weight (Class 0) = 0.625, Weight (Class 1) = 2.50
D. Weight (Class 0) = 2.50, Weight (Class 1) = 0.625
Correct Answer: C
Q.2 Consider the following dataset with two features (x1, x2) and a target variable (y):
X1,X2,Y
1,2,10
2,1,20
You are training a multiple linear regression model y = beta1*x1 + beta2*x2 using Mean Squared Error (MSE)
loss. Given the initial parameters beta1 = 0, beta2 = 0, and a learning rate eta = 0.05, compute the updated
values of beta1 and beta2 after exactly one iteration of gradient descent.
A. beta1 = 1.25, beta2 = 1.25
B. beta1 = 2.50, beta2 = 2.00
C. beta1 = 5.00, beta2 = 4.00
D. beta1 = 2.00, beta2 = 2.50
Correct Answer: B
Q.3 Consider the binary classification cost function: (Log Loss / Cross-Entropy). Two samples are
classified:
Sample A: true label y=1, predicted probability p = 0.40
Sample B: true label y=0, predicted probability p = 0.60
Both predictions are incorrect (missed by a minor margin). What is the total cost (sum of costs for both
samples), rounded to two decimal places?
A. 0.92
B. 1.83
C. 0.46
D. 2.01
Correct Answer: B
Q.4 In this small dataset: [2, 5, 9, 12, 18, 150], for the original data point 12, what are its scaled values
after applying MinMaxScaler and RobustScaler respectively?
A. MinMax: 0.07, Robust: 0.12
B. MinMax: 0.12, Robust: 0.07
C. MinMax: 0.08, Robust: 0.15
D. MinMax: 0.07, Robust: 0.54
Correct Answer: A
Q.5 Assume we have the following data points in a 2D feature space: Minority Points (M): M1 = (0, 0)
and M2 = (10, 0). Majority Points (J): JA = (2, 0), JB = (3, 2), and JC = (9, 0). Using the NearMiss-1
(n_neighbors=2) algorithm, which majority point(s) will be retained if the algorithm is set to select the
two majority points that are closest to the minority cluster based on its specific distance rule?
A. JA and JB
B. JA and JC
C. JB and JC
D. All three points are equally close
Correct Answer: B
Q.6 Assume we have the following data points in a 2D feature space: Minority Points (M): M1 = (0, 0)
and M2 = (10, 0). Majority Points (J): JA = (2, 0), JB = (5, 1), and JC = (9, 0). Using the NearMiss-2
(n_neighbors=2) algorithm, which majority point will be retained if the algorithm is set to select only
one point that has the minimum average distance to its n farthest minority points?
A. JA
B. JB
C. JC
D. JA and JC tie
Correct Answer: B
Q.7 Consider the following dataset containing missing values (NaN):
Point,X1,X2,X3
P0,10,12,14
P1,20,22,NaN
P2,30,32,34
P3,40,42,44
P4,2,NaN,6
What is the imputed value for P1's missing X3 feature using k=2 (KNN Imputation)?
A. 14
B. 24
C. 34
D. 39
Correct Answer: B
Q.8 KD-tree is built from the following nine 2-dimensional points: (2,3), (5,4), (9,6), (4,7), (8,1), (7,2).
Which of the following point(s) is at level-01 (Assume root is level-0)?
A. (4, 7)
B. (8, 1)
C. (7, 2)
D. (5, 4)
Correct Answer: D
For Q.9-11 Imagine a simple scenario where a bank wants to predict if a customer will default on a loan based
on two features: Credit Score (High vs. Low) and Has Collateral (Yes vs. No). The dataset contains 8 total
samples:
ID,Credit Score (X1),Has Collateral (X2),Will Default (Y)
B1,High,Yes,No
B2,High,Yes,No
B3,High,No,No
B4,High,No,Yes
B5,Low,Yes,No
B6,Low,No,Yes
B7,Low,No,Yes
B8,Low,No,Yes
Q.9 Calculate the initial Entropy and Gini Impurity of the entire dataset (the root node) before any splits
are made.
A. 1.0 0.5
B. 0.5 1.0
C. 0.5 0.1
D. 1.0 1.0
Correct Answer: A. 1.0 0.5
Q.10 Calculate the information gain using the entropy if the split is made using the score credit score.
A. 1.98
B. 0.198
C. 0.189
D. 9.18
Correct Answer: C. 0.189
Q.11 Calculate the information gain using the gini impurity if the split is made using the score has
collateral.
A. 0.2
B. 0.5
C. 0.0
D. 0.3
Correct Answer: D. 0.3
Q.12 For the logits [0, ln(4)], where the first value corresponds to Class 0 and the second to Class 1,
what is the probability of Class 0?
A. 0.20
B. 0.25
C. 0.75
D. 0.80
Correct Answer: A. 0.20
Q.13 If a logistic regression model outputs a sigmoid probability of 0.4 for an instance where the true
label is 1, what is the Binary Cross-Entropy loss for this specific point? (Use ln(0.4) approx -0.92 and
ln(0.6) approx -0.51)
A. 0.40
B. 0.51
C. 0.92
D. 1.33
Correct Answer: C. 0.92
Q.14 Calculate the R2 score for the following dataset:
Actual (y),Predicted ( y )
2,3
4,4
6,5
A. 0.65
B. 0.75
C. 0.80
D. 0.25
Correct Answer: B. 0.75
Q.15 A regression model with 2 predictors (p=2) is trained on 11 observations (n=11). If the R2 score is
0.80, what is the Adjusted R2 value?
A. 0.70
B. 0.75
C. 0.78
D. 0.82
Correct Answer: B. 0.75
Q.16 Consider a single data point (x,y) = (2, 4) and a model y = wx (no bias). If the current weight w =
1.0 and the learning rate alpha = 0.05, what is the updated weight w after one step of Stochastic
Gradient Descent? (Loss function: L = (wx - y)^2)
A. 1.1
B. 1.2
C. 1.4
D. 1.8
Correct Answer: C. 1.4
Q.17 For the cost function J(w) = 3w^2 - 12w + 5, at what value of w will the gradient be zero, and what
is the gradient at w=3?
A. Zero at w=2; Gradient at w = 3 is 6
B. Zero at w=4; Gradient at w=3 is -6
C. Zero at w=2; Gradient at w=3 is -3
D. Zero at w=6; Gradient at w=3 is 0
Correct Answer: A. Zero at w=2; Gradient at w=3 is 6
Q.18 Determine the absolute difference between the L1 penalty and L2 penalty for a model where
weights are w1 = 3, w2 = 4. Use Lasso parameter lambda = 1 and Ridge parameter sigma = 0.5.
A. 1.5
B. 4.5
C. 5.5
D. 7.0
Correct Answer: C. 5.5
Q.19 If we apply Standard Scaling (Z-score normalization) to a feature with values {10, 20, 30}, what will
the transformed value of 10 be?
A. -1.22
B. -1.0
C. 0
D. 1.0
Correct Answer: A. -1.22
Q.20 Using SMOTE, a synthetic point is generated between P1(2, 3) and P2(5, 7). If the random number
lambda chosen is 0.4, what are the coordinates of the new point?
A. (3.2, 4.6)
B. (3.5, 5.0)
C. (2.8, 4.2)
D. (4.0, 6.0)
Correct Answer: A. (3.2, 4.6)
Q.21 A clustering analysis yields the following for Point A: average distance to points in its own cluster
(a) = 2.0, and average distance to the nearest neighboring cluster (b) = 5.0. What is the Silhouette
Coefficient for Point A?
A. 0.30
B. 0.40
C. 0.60
D. 1.50
Correct Answer: C. 0.60
Q.22 In a testing set of 200 samples, a model correctly identifies 120 True Positives and 50 True
Negatives. If there are 20 False Positives, how many False Negatives are there, and what is the Recall?
A. 10 FN; Recall = 0.92
B. 30 FN; Recall = 0.80
C. 10 FN; Recall = 0.85
D. 20 FN; Recall = 0.86
Correct Answer: A. 10 FN; Recall = 0.92
Q.23 A classification model for credit card fraud (Positive Class) was evaluated on 2,000 transactions.
Actual/Predicted:
Actual Positive -> TP = 40, FN = 10
Actual Negative -> FP = 50, TN = 1900
Calculate the Precision and the F1-Score for this model.
A. Precision = 0.80, F1 Score = 0.57
B. Precision = 0.44, F1 Score = 0.57
C. Precision = 0.44, F1 Score = 0.70
D. Precision = 0.80, F1 Score = 0.44
Correct Answer: B. Precision = 0.44, F1 Score = 0.57
Q.24 Calculate the L2 Penalty term for a model with three weights: w1 = 4, w2 = 0, w3 = -3. Assume the
regularization strength parameter (alpha or lambda) is set to 0.5.
A. 12.5
B. 25.0
C. 7.0
D. 3.5
Correct Answer: A. 12.5
Q.25 A SMOTE algorithm is generating a point between PA (10, 20) and its neighbor PB (30, 60). The
algorithm picks a random number gap lambda = 0.25. What are the coordinates of the synthetic point
Pnew?
A. (15, 30)
B. (20, 40)
C. (15, 70)
D. (25, 50)
Correct Answer: A. (15, 30)
Q.26 A model predicts a probability of 0.8 for the positive class (y = 1). What is the resulting Binary
Cross-Entropy loss for this observation? (Given: ln(0.8) approx -0.22, ln(0.2) approx -1.61)
A. 0.11
B. 0.22
C. 1.61
D. 0.80
Correct Answer: B. 0.22
Q.27 If you are using polynomial regression with two input features (x1, x2) and the degree is set to 2,
how many total terms will the model have (including the intercept/bias term)?
A. 4
B. 5
C. 6
D. 9
Correct Answer: C. 6
Q.28 A 1000-sample dataset has 30% missing in 1 feature. After applying dropna function of pandas,
what is the training set size for 5-fold CV per fold?
A. 140
B. 560
C. 350
D. None of these
Correct Answer: B. 560
Q.29 For a 100-sample dataset, KNN (k=5) has neighbors at distances [1.0, 1.0, 1.5, 2.0, 20] with output
[1, 1, 4, 6, 15]. If weights are defined as inverse of distance with no scaling, what is the approximated
weighted average output?
A. 2.62
B. 3.62
C. 8.42
D. 3.22
Correct Answer: A. 2.62
Q.30 A 2000-sample dataset has 50 features. After L2 regularization (Ridge, lambda=0.15), the L2
penalty is 0.075. If the MSE is 0.025, what is the new loss?
A. 0.05
B. 0.10
C. 0.075
D. 0.025
Correct Answer: B. 0.10
Q.31 For a 1000 sample dataset, 5 fold CV is used. If 15% of the data is missing in 1 feature, how many
samples are in the training set per fold after imputation
A. 200
B. 800
C. 750
D. None of these
Correct Answer: B. 800
Q.32 For a 1000 sample dataset (90% class-0 and 10% class-1), After SMOTE oversampling Class 1 to
25% of the total. What will be the size of data?
A. 200
B. 800
C. 750
D. None of these
Correct Answer: B. 800
Q.33 Consider a 1000-sample dataset.
Scenario 1 (Weighted KNN): k=5, neighbor distances: [0.1, 0.2, 0.3, 0.4, 0.5], labels: [A, B, A, B, B]. Weights =
inverse of distance (no scaling).
Scenario 2 (Unweighted KNN): k=5, neighbor distances: [1, 2, 3, 4, 5], labels: [A, B, A, B, B].
If the test sample is labeled A, what is the accuracy in each scenario?
A. Weighted KNN: 100% ; Unweighted KNN: 0%
B. Weighted KNN: 0% ; Unweighted KNN: 100%
C. Both: 100%
D. Both: 0%
Correct Answer: A. Weighted KNN: 100% ; Unweighted KNN: 0%
Q.34 A retail analytics team is analyzing daily sales of a stable product with no extreme promotional
spikes. The distribution of sales appears symmetric over time. Data collected over 5 days: [20, 25, 30,
NaN, 35]. What is the most appropriate value to replace the missing entry?
A. 25
B. 27.5
C. 30
D. 35
Correct Answer: B. 27.5
Q.35 A financial dataset records transaction amounts where a few very large transactions exist,
creating a right-skewed distribution. Observed values: [10, 15, 20, NaN, 200]. What should replace the
missing value?
A. 17.5
B. 49
C. 61.25
D. 200
Correct Answer: A. 17.5