1. A regression model has an R-squared of 0.90 but an adjusted R-squared of 0.70.
What does this
imply?
• a) The model fits the data well.
• b) The model has too many irrelevant features.
• c) The model is underfitting the data.
• d) The target variable is not suitable for regression.
Solution:
b) The model has too many irrelevant features.
Explanation:
The large difference between R-squared (0.90) and adjusted R-squared (0.70) suggests that many
features do not contribute to the model's predictive ability, indicating overfitting.
2. A dataset used for regression analysis contains missing values in some features. What methods
can be used to handle this issue?
• a) Drop rows with missing values.
• b) Use mean or median imputation for missing values.
• c) Use K-Nearest Neighbors (KNN) to impute missing values.
• d) Replace missing values with random numbers.
Solution:
a) Drop rows with missing values.
b) Use mean or median imputation for missing values.
c) Use K-Nearest Neighbors (KNN) to impute missing values.
Explanation:
• Dropping rows is simple but can lead to data loss.
• Mean/median imputation works well for numerical data.
• KNN imputes values based on similar data points.
Replacing missing values with random numbers introduces noise and is not recommended.
3.
A Ridge regression model is trained with (\lambda = 10), resulting in the following equation:
[ \text{Price} = 500 + 100 \times \text{Size} - 5 \times \text{Age} ]
If the original model (without regularization) had the coefficients for (\text{Size}) as 150 and for
(\text{Age}) as -20, what can you infer?
• a) Regularization reduced overfitting.
• b) The model is underfitting the data.
• c) Regularization increases feature coefficients.
• d) No significant change is observed.
Solution:
a) Regularization reduced overfitting.
Explanation:
Ridge regression penalizes large coefficients to reduce overfitting. Here, coefficients are smaller after
regularization, improving generalization.
4. You fit a 10th-degree polynomial regression model to predict stock prices and achieve almost
zero training error. On testing data, however, the error is very high. What is the most likely reason?
• a) The model is underfitting the data.
• b) The model is overfitting the data.
• c) The data has too many missing values.
• d) Polynomial regression cannot predict stock prices.
Solution:
b) The model is overfitting the data.
Explanation:
A high-degree polynomial fits training data perfectly but often fails to generalize, leading to high test
error.
5. In a Ridge regression model, increasing the penalty term (\lambda) causes the coefficient of a
feature to decrease from 25 to 5. What does this suggest?
• a) The feature is irrelevant.
• b) The model is more sensitive to this feature.
• c) The model penalizes large coefficients.
• d) The model underfits the data with higher (\lambda).
Solution:
c) The model penalizes large coefficients.
Explanation:
Ridge regression penalizes large coefficients to prevent overfitting, reducing their values as (\lambda)
increases.
6.
A regression model has been evaluated with the following metrics:
• Mean Absolute Error (MAE): 5
• Root Mean Squared Error (RMSE): 7
• R-squared: 0.65
Which of the following can be concluded?
• a) The predictions are relatively close to actual values on average.
• b) The model explains 65% of the variance in the target variable.
• c) RMSE is better than MAE for evaluating this model.
• d) The model is overfitting the data.
Solution:
a) The predictions are relatively close to actual values on average.
b) The model explains 65% of the variance in the target variable.
Explanation:
• MAE shows average error (5 units), while R-squared indicates the model explains 65% of
target variance.
• RMSE doesn't necessarily imply better evaluation here, and there’s no evidence of
overfitting.
7. A model predicting student grades finds high multicollinearity between hours studied, test
preparation hours, and attendance percentage. How can this issue be addressed?
• a) Drop one of the correlated features.
• b) Use Lasso regression.
• c) Perform PCA to reduce dimensionality.
• d) Increase the dataset size.
Solution:
a) Drop one of the correlated features.
b) Use Lasso regression.
c) Perform PCA to reduce dimensionality.
Explanation:
• Dropping features reduces redundancy.
• Lasso penalizes irrelevant features by reducing them to zero.
• PCA combines correlated features into independent components, mitigating
multicollinearity.
8. A bike rental company uses temperature, weather conditions, and time of the day to predict the
number of rentals. Engineers suggest adding an interaction term between temperature and
weather conditions. Why might this improve the model?
• a) It captures non-linear relationships between variables.
• b) It reduces the number of features.
• c) It helps model the combined impact of temperature and weather conditions.
• d) It ensures feature scaling is not required.
Solution:
c) It helps model the combined impact of temperature and weather conditions.
Explanation:
Interaction terms capture the combined influence of two features, improving the model's ability to
account for complex relationships.
9. A company is using a linear regression model to predict housing prices based on features like
area, number of rooms, and location. After training the model, the weights for some features are
excessively high. Which type of regularization is most effective to reduce the impact of these high
weights?
Options:
a) Lasso Regularization
b) Ridge Regularization
c) No Regularization
d) Elastic Net Regularization
Answer: b) Ridge Regularization
Explanation:
Ridge regularization (L2) adds a penalty term to the cost function, which helps to shrink the
coefficients of the features, especially those with excessively high values. This prevents overfitting by
reducing the influence of large weights, while still allowing the model to include all features.
10. A dataset contains 100 features, of which only 10 are relevant for predicting the target
variable. Which of the following methods would likely help improve the model's
performance? (Select all that apply)
Options:
a) Using L2 regularization
b) Using L1 regularization
c) Increasing the learning rate of gradient descent
d) Using Elastic Net Regularization with a high ratio of L1 penalty
Answer: b), d)
Explanation:
• b) L1 regularization (Lasso) helps by driving irrelevant feature coefficients to zero, effectively
performing feature selection.
• d) Elastic Net Regularization combines the benefits of both L1 and L2 regularization, and
using a higher L1 ratio helps focus on feature selection, making it effective when there are
many irrelevant features.
11. A telecom company uses linear regression to predict customer churn. The model performs well
on the training set but poorly on the validation set. What is the most likely reason for this
behavior?
Options:
a) Model underfitting due to insufficient features
b) Model overfitting due to lack of regularization
c) The cost function is incorrectly implemented
d) Gradient descent did not converge
Answer: b) Model overfitting due to lack of regularization
Explanation:
Overfitting occurs when a model learns the noise and random fluctuations in the training data, which
reduces its ability to generalize to new, unseen data. Regularization methods such as L1 or L2 can
help prevent overfitting by limiting the complexity of the model.
12. A team applies gradient descent to train a linear regression model on a dataset where one
feature has values ranging from 0 to 1,000,000 and another feature ranges from 0 to 1. What
challenges might arise, and how can they be addressed? (Select all that apply)
Options:
a) Slow convergence of gradient descent
b) Large differences in feature scales cause numerical instability
c) Applying feature standardization or normalization can mitigate the problem
d) Increasing the learning rate solves the problem
Answer: a), b), c)
Explanation:
• a) The gradient descent may converge slowly when the feature scales differ drastically, as the
gradients might be uneven, leading to inefficient updates.
• b) Numerical instability can occur because large values dominate smaller ones in the
gradient updates.
• c) Standardization or normalization can rescale the features to the same range, improving
the stability and speed of convergence.
13.
A linear regression model with L2 regularization has the cost function:
[ J(w) = \frac{1}{m} \sum_{i=1}^m (y_i - w \cdot x_i)^2 + \lambda \sum_{j=1}^n w_j^2 ]
Given ( m = 3 ), ( \lambda = 0.5 ), and the weights ( w = [1, 2] ), calculate the regularization term in
the cost.
Answer: ( 0.5 \cdot (1^2 + 2^2) = 2.5 )
Explanation:
The regularization term is calculated as:
[ \lambda \sum_{j=1}^n w_j^2 = 0.5 \cdot (1^2 + 2^2) = 2.5 ]
14. A fintech startup uses Elastic Net Regularization for feature selection in its credit scoring model.
Which statement about the behavior of the Elastic Net is correct?
Options:
a) It acts purely like Ridge regularization when ( \alpha = 1 ).
b) It combines the benefits of L1 and L2 regularization, helping in feature selection and reducing
multicollinearity.
c) It only removes features with zero coefficients like Lasso.
d) It applies a penalty proportional to the square root of the coefficients.
Answer: b) It combines the benefits of L1 and L2 regularization, helping in feature selection and
reducing multicollinearity.
Explanation:
Elastic Net combines L1 and L2 penalties. The L1 penalty aids in feature selection by shrinking some
coefficients to zero, while the L2 penalty reduces multicollinearity, stabilizing the model when
features are highly correlated.
15. A logistic regression model uses gradient descent for optimization. The training loss oscillates
and does not converge. Which actions could help resolve this issue? (Select all that apply)
Options:
a) Decrease the learning rate
b) Increase the learning rate
c) Use adaptive learning rates like Adam or RMSprop
d) Apply regularization to the cost function
Answer: a), c)
Explanation:
• a) A high learning rate can cause the loss to oscillate, and lowering it can help the model
converge smoothly.
• c) Adaptive learning rates, like those used in Adam or RMSprop, adjust the learning rate
during training, helping resolve oscillations and improving convergence.
16. A retail company uses batch gradient descent to optimize a linear regression model. The cost
decreases initially but plateaus at a high value. What could be the possible reasons? (Select all that
apply)
Options:
a) The learning rate is too high, causing oscillations.
b) The model is under-parameterized.
c) Features are not normalized.
d) The model is stuck in a local minimum.
Answer: b), c)
Explanation:
• b) An under-parameterized model might not have enough complexity to fit the data, causing
the cost to plateau.
• c) If the features are not normalized, the gradient updates might not be effective, causing the
model to converge slowly or get stuck.
17. A data science team is working on a project to predict customer churn based on user behavior
on an e-commerce platform. They have a high-dimensional dataset with over 500 features, many
of which are highly correlated. They decide to apply dimensionality reduction techniques. Which
of the following advanced methods will provide both feature reduction and capture most of the
variance while addressing multicollinearity?
Options:
a) Apply Principal Component Analysis (PCA) followed by feature selection using a Random Forest
model.
b) Use Independent Component Analysis (ICA) to capture independent features and separate sources
of variance.
c) Apply Kernel PCA with a non-linear kernel to handle non-linear correlations.
d) Use Non-negative Matrix Factorization (NMF) for dimensionality reduction in cases where all
features are positive.
Answer: a)
Explanation:
PCA is an effective method for reducing dimensionality while preserving the maximum variance in
the data by transforming correlated features into uncorrelated principal components. By applying
PCA, the team can handle multicollinearity and capture the most significant variance in the data. The
option of combining PCA with Random Forest feature selection can help further refine the features
for predictive tasks.
18. An agricultural research team is studying how different weather conditions, soil types, and crop
characteristics affect yield. They have collected sensor data with highly correlated features, leading
to multicollinearity. Which advanced dimensionality reduction technique will not only address
multicollinearity but also maximize the explanatory power for crop yield prediction?
Options:
a) Apply Canonical Correlation Analysis (CCA) to find relationships between weather and soil
features.
b) Use Linear Discriminant Analysis (LDA) to maximize class separability.
c) Implement Kernel PCA to model non-linear relationships in the data.
d) Apply Principal Component Analysis (PCA) for linear dimensionality reduction while capturing
maximum variance.
Answer: d)
Explanation:
PCA is well-suited for reducing dimensionality while preserving the variance in the data. It addresses
multicollinearity by transforming correlated features into uncorrelated principal components. This is
particularly beneficial in the case of highly correlated environmental data, helping to reduce the
feature space without sacrificing explanatory power for crop yield prediction.
19. A university research team applies PCA to analyze a dataset with 200 features, including
demographic information, study habits, and test scores. After applying PCA, they observe that the
first few components explain 95% of the variance. Which of the following statements is an
advanced interpretation of this result?
Options:
a) The retained principal components represent a compact but comprehensive summary of the
original data, preserving the majority of the variance.
b) The first few principal components are independent of each other and can be interpreted as
distinct underlying factors.
c) The remaining 5% of the variance is negligible and can be safely ignored.
d) The variance captured by the components is irrelevant as the data needs to be classified using
only the raw features.
Answer: a)
Explanation:
By retaining the first few principal components, the team preserves most of the original data’s
variance, which means that the new components provide a compact representation of the dataset.
This is useful for reducing the complexity of models and enhancing performance while retaining the
most significant information from the original features.
20. A team at an AI research lab is working on compressing high-resolution images using PCA. They
seek to reduce the dimensionality of image data while retaining important features. Which
advanced application of PCA in image compression will allow them to achieve this goal?
Options:
a) PCA can be used with a predefined threshold of variance to retain significant features while
reducing the size of the data.
b) PCA can be applied recursively, compressing the image into multiple layers of smaller components.
c) PCA inherently works with non-linear transformations, making it ideal for complex image features.
d) PCA is most effective when applied to raw pixel data directly without any pre-processing.
Answer: a)
Explanation:
PCA allows for the compression of images by selecting a subset of the principal components that
capture the maximum variance of the data. By setting a threshold for the variance (e.g., retaining
95% of the variance), the dimensionality of the image data can be reduced while maintaining most of
the important features, significantly reducing data size for storage or transmission.