SIMPLE LINEAR REGRESSION

Question Set

Question 1
For a simple linear regression model, the eigenvalues obtained from its Hessian matrix are
given as -4 and -3. Which of the following statement/statements are true about the model?
(a) A Global maxima exists for the model
(b) A Global minima is guaranteed by the Hessian matrix
(c) The cost function for the model must not be a squared error function
(d) The cost function curvature is non-convex in shape

Question 2
A simple linear-regression model ŷ = ωx + b is fitted on two datasets.
After fitting on Dataset A, you obtained:
sₓₓ(A) = 5.40, sₓᵧ(A) = 2.16
A new batch of observations is appended, giving Dataset B with:
sₓₓ(B) = 8.10, sₓᵧ(B) = 10.53
Here sₓₓ is the variance of the input variable x and sₓᵧ is the covariance between x and y.
The change in the slope parameter Δω is ____.
(Provide the answer correct to two decimal places.)

Question 3
In a dataset with 20 data examples, a simple linear regression best-fit line for an input x and
target y is described as follows:
ŷ = b₀ + b₁x
If the sum of squares for residuals is 28, the output variance is 37.5, and the covariance
between input and output is 13.4, then the absolute value of the coefficient b₁ is ____
(rounded off to 3 decimal points).

Question 4
For the same dataset with 20 data examples, if the centroid is given as (9.75, 13.3), and (4,
y(i)) is a data instance in the dataset with a squared error value of 56.87, the value of y (i) is
_________ (rounded off to 3 decimal points).

Question 5
In a simple linear regression model, the parameters b₀ and b₁ represent the intercept and
slope, respectively. The prediction variance Sŷŷ is defined as follows:
1
𝑠ŷŷ = ŷ( ) − ȳ
(𝑚 − 1)
where ŷ(i) is the predicted value from the regression model for one data example, and ȳ is the
mean of the predicted values. The input variance Sₓₓ is given by:
1
𝑠 = x ( ) − 𝑥̅
(𝑚 − 1)

Then, the ratio Sŷŷ / Sₓₓ is always:

Question 6
Kavya is interested to see if students who consume more caffeine tend to study more as well.
She randomly selected 25 students from her college and recorded their caffeine intake (mg)
and the number of hours spent studying.
A scatter plot of the data showed a linearly increasing relationship. The following table
describes the results of simple linear regression analysis on the data:
Predictor Coefficient value
Caffeine (mg) 0.189
Constant 3.45

Based on the given data, which of the following statement(s) is/are true?
(a) For each additional 1 mg of caffeine, the study time is predicted to increase by 0.189 hours.
(b) For each additional 1 hour of study time, the caffeine intake is predicted to increase by 0.189
mg.
(c) When the study time is 10 hours, the caffeine intake is predicted to be 34.5 mg.
(d) When the caffeine intake is zero mg, the study time is predicted to be 3.45 hours.

Question 7
In the Gradient Descent algorithm, the steps need to be followed in a specific order. Fixing
Step-1 as "Initialize Parameters" and Step-5 as "Repeat until convergence", which of the
following options correctly orders the intermediate steps?
(a) Compute the Gradient, Update Parameters, Compute the Cost Function
(b) Compute the Cost Function, Compute the Gradient, Update Parameters
(c) Update Parameters, Compute the Cost Function, Compute the Gradient
(d) Compute the Cost Function, Update Parameters, Compute the Gradient

Question 8
A simple linear regression model with three data points is given below:
f(xᵢ) = 2x + 5
x y
4 2
3 5
1 4

After one iteration of gradient descent algorithm, the intercept value of the model is changed
to 3 by the algorithm. The learning rate of the algorithm is ______.
Question 9
A simple linear regression model with initial parameters (ω₀, b₀) converges to (ω, β) in 20
iterations of the gradient descent algorithm. The gradient calculated for the initial model
parameter w₀ is g. If in every iteration the gradient g is reduced by a factor of 2, then the
learning rate 𝛼 is:
(a) 2(ω - ω₀) / g
(b) (ω₀ - ω) / 2g
(c) g / [2(ω - ω₀)]
(d) (ω₀ + ω) / 2g

Question 10
The image shows the trajectory of gradient descent applied to the cost function J(θ) = θ², with
the parameter θ on the X-axis and cost on the Y-axis. The black dot in the figure represents
the final point reached after convergence, using a well-tuned learning rate.

Assume:
• The cost function is convex and differentiable.
• Gradient descent is run until convergence.
• The initial value of θ is far from the minimum.
Based on this, which of the following statements are true?
(a) The point reached at the black dot is a saddle point and a local minimum.
(b) The derivative at the black dot is exactly zero.
(c) The cost value at the black dot will be strictly zero.
(d) Gradient descent with a sufficiently small learning rate guarantees convergence for convex
functions.
Question 11
Given data {(-1, 1), (2, -5), (3, 5)} of the form (x, y), we fit a model y = wx using linear
least-squares regression. The optimal value of w is ___ (Round off to three decimal places).
(GATE-2025, 1 Mark)

Question 12

Consider that for a supervised learning task, the objective function being minimized is
fw(x) = wx, where x  R is the input and w  R is the parameter. Stochastic Gradient
Descent with learning rate of 0.10 is used for parameter updates. Suppose that at the end
of iteration i, the value of w becomes 10.00. Let x = 10.00 be the input for iteration (i + 1).
The value of w at the end of iteration (i + 1) is . (Rounded off to two decimal places)
(GATE-2026-1M)