Introduction To Machine Learning - Unit 5 - Week 2
Uploaded by
Gopinathan RIntroduction To Machine Learning - Unit 5 - Week 2
Uploaded by
Gopinathan R1/24/26, 5:01 PM Introduction to Machine Learning - - Unit 5 - Week 2
([Link] gopinathanr@[Link]
([Link]
NPTEL ([Link] » Introduction to Machine Learning (course)
Announcements (announcements) About the Course (preview) Q&A (forum) Progress (student/home) Mentor (student/mentor)
Review Assignment (assignment_review) Course Recommendations (/course_recommendations)
Click to register for
Certification exam
Week 2 : Assignment 2
([Link]
If already registered, click to Assignment not submitted Due date: 2026-02-04, 23:59 IST.
check your payment status 1) In building a linear regression model for a particular data set, you observe the coefficient of one of the features having a relatively high negative 1 point
value.
This suggests that
Course outline This feature has a strong effect on the model (should be retained)
This feature does not have a strong effect on the model (should be ignored)
About NPTEL ()
It is not possible to comment on the importance of this feature without additional information
How does an NPTEL
2) We have seen methods like ridge and lasso to reduce variance among the co-efficients. We can use these methods to do feature selection also. 1 point
online course work? ()
Which one of them is more appropriate?
Week 0 () Ridge
Lasso
Week 1 ()
3) Given a set of n data points, (x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ), the best least squares fit f (x) is obtained by minimization of: 1 point
Week 2 ()
Linear Regression (unit? Σ
n
[yi − f (x i )]
i=1
unit=32&lesson=33)
min(yi − f (x i ))
Multivariate Regression (unit?
unit=32&lesson=34) 2
n
Σ [yi − f (x i )]
i=1
Subset Selection 1 (unit?
unit=32&lesson=35) max(yi − f (x i ))
[Link] 1/3
1/24/26, 5:01 PM Introduction to Machine Learning - - Unit 5 - Week 2
Subset Selection 2 (unit? 4) During linear regression, with regards to residuals, which among the following is true? 1 point
unit=32&lesson=36)
Lower is better
Shrinkage Methods (unit?
Higher is better
unit=32&lesson=37)
Depends upon the data
Principal Components
None of the above
Regression (unit?
unit=32&lesson=38)
5) In the lecture on Multivariate Regression, you learn about using orthogonalization iteratively to obtain regression co-effecients. This method is 1 point
Partial Least Squares (unit? generally referred to as Multiple Regression using Successive Orthogonalization
unit=32&lesson=39) In the formulation of the method, we observe that in iteration k, we regress the entire dataset on z0 , z1 , . . . zk−1 . It seems like a waste of computation
Practice: Practice Assignment to recompute the coefficients for z0 a total of p times, z1 a total of p − 1 times and so on. Can we reuse the coefficients computed in iteration j for iteration
2 (assessment?name=353) j + 1 for zj−1 ?
Quiz: Week 2 : Assignment
2 (assessment?name=354) No. Doing so will result in the wrong γ matrix. and hence, the wrong β i ’s.
Text Transcripts () Yes. Since zj−1 is orthogonal to zj−l ∀l ≤ j1 , the multiple regression in each iteration is essentially a univariate regression on each of the previous
residuals. Since the regression coefficients for the previous residuals don’t change over iterations, we can reuse the coefficients for further iterations.
Download Videos ()
6) You decide to reduce the dimensionality of your data(N × p) using Best Subset Selection. The library you’re using has a function regress(X, Y ) 1 point
Books () that takes
in X and Y and regresses Y on X. What is the expected number of times regress(·, ·) will be called during your dimensionality reduction?
Problem Solving Session -
Jan 2026 ()
N
O(2 )
p
O(2 )
p
O(N )
2
O(p )
7) If the number of features is larger than the number of training data points, to identify a suitable subset of the features for use with linear 1 point
regression, we would prefer
Forward stepwise selection
Backward stepwise selection
8) Assume you have a five-dimensional input data for a three-class classification problem. Further assume that all five dimensions of the input 1 point
are independent to each other. In this scenario, is it possible for linear regression using lasso to result in one or more coefficients to become zero?
Yes
No
9) You are given the following five three-dimensional training data instances (along with one- dimensional output) 1 point
[Link] 2/3
1/24/26, 5:01 PM Introduction to Machine Learning - - Unit 5 - Week 2
• x1 = 5, x2 = 7, x3 = 3, y = 4
• x1 = 2, x2 = 4, x3 = 9, y = 8
• x1 = 3, x2 = 8, x3 = 1, y = 2
• x1 = 7, x2 = 7, x3 = 2, y = 3
• x1 = 1, x2 = 9, x3 = 7, y = 8
Using the K-nearest neighbour technique for performing regression, what will be the predicted y value corresponding to the query point
(x1 = 5, x2 = 3, x3 = 4), for K = 2?
3
2.5
3.5
2
10) For the dataset given in the previous question, what will be the predicted y value corresponding to the query point (x1 = 5, x2 = 3, x3 = 4), 1 point
for K = 3?
4.66
5
3
3.5
You may submit any number of times before the due date. The final submission will be considered for grading.
Submit Answers
[Link] 3/3
Common questions
Powered by AILasso (Least Absolute Shrinkage and Selection Operator) differs from Ridge regression in that it performs both coefficient shrinkage and variable selection by penalizing the absolute size of the regression coefficients. This can lead to coefficients being exactly zero, effectively selecting a subset of predictors. In contrast, Ridge regression only minimizes the coefficients without actually reducing them to zero, making Lasso more suitable for feature selection .
Forward stepwise selection is preferred over backward stepwise selection when the number of features exceeds the number of training data points because forward selection starts with no variables and adds one variable at a time. This is computationally feasible with large feature sets as it leverages criteria like AIC or BIC to determine the inclusion of each feature, reducing the risk of multicollinearity that backward selection might exacerbate .
Best Subset Selection during dimensionality reduction using a regression function like regress(X, Y) typically involves extensive computation with complexity of O(2^p), where p is the number of predictors. This high complexity arises because the method evaluates all possible combinations of predictors to determine the best subset that fits the model, making it computationally intensive for large p .
Principal Component Regression addresses multicollinearity by transforming the original predictors into a new set of uncorrelated variables called principal components, which are then used in regression. This process eliminates multicollinearity effects among the predictors, improves model stability, and is advantageous because it enables effective coefficient estimation and better generalization in scenarios with highly collinear data .
Yes, Lasso regression can result in zero coefficients even in datasets where features are independent of each other. This outcome occurs because Lasso performs variable selection by shrinking some coefficients to zero based on the penalty imposed, thus simplifying the model and enhancing interpretability by essentially excluding less influential features .
Recomputing coefficients for previously processed variables during multivariate regression using orthogonalization can be redundant because the orthogonalization process implies that each iteration effectively operates on uncorrelated components. Thus, the regression coefficients for residuals from previous iterations remain unchanged and can be reused, eliminating the need for recomputation .
A high negative coefficient in a linear regression model suggests that the feature has a strong inverse relationship with the dependent variable, contributing significantly to the model. It should be retained in the model as it indicates that as the feature value increases, the output variable tends to decrease, assuming linearity in the relationship .
Minimizing the sum of squared residuals in least squares linear regression is critical because it ensures that the line of best fit represents the data as closely as possible by reducing the discrepancies between observed and predicted values. This approach prioritizes small prediction errors, leading to the most accurate estimation of the relationship between variables within the linear framework .
Shrinkage methods like Ridge and Lasso involve trade-offs between model complexity and bias. Ridge regression effectively reduces model variance by shrinking coefficients but does not perform variable selection, possibly leading to less interpretable models. Lasso, on the other hand, introduces sparsity by reducing some coefficients to zero, which aids in model interpretability but may introduce bias by excluding potentially relevant features. Balancing these factors is crucial depending on the modeling goals .
In K-nearest neighbor regression, the choice of K directly influences the predicted output for a query point because it determines the number of closest data points considered for averaging. A smaller K may lead to overfitting as the model may become sensitive to noise, whereas a larger K might underfit by oversmoothing the predictions. Thus, selecting an optimal K requires balancing bias and variance .