Machine Learning

Problem Set  Complete Solutions

University:
Instructor:
Benha National University

Subject:
Dr. Manar H. Lashin

Academic Year:
Machine Learning

2025/2026

Problem 1 & 2: Gradient Descent for Linear Regression (one iter-

Problems Covered

•
ation update of θ , θ )
0 1

• Problem 3: Simple Linear Regression  computing b , b , predic-

tion & residual
0 1

• Problem 4: Multiple Linear Regression  two predictors, matrix /

normal-equation approach
• Problem 5: Logistic Regression  two predictors, probability com-
putation, confusion matrix

Compiled solution document

Machine Learning  Problem Set Solutions Benha National University

Contents
1 Gradient Descent for Linear Regression  Background 2
2 Problem 1  Gradient Descent (First Iteration) 2
2.1 Step 1  Compute Predictions . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Step 2  Compute Gradients . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Step 3  Update Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 Problem 2  Gradient Descent, Dierent Dataset (α = 0.01) 3
3.1 Step 1  Compute Predictions (m = 6) . . . . . . . . . . . . . . . . . . . . . 4
3.2 Step 2  Compute Gradients . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 Step 3  Update Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4 Problem 3  Simple Linear Regression 5
4.1 Part (a)  Slope and Intercept . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.1.1 Computation Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.1.2 Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.1.3 Calculate b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.1.4 Calculate b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1

4.2 Part (b)  Prediction and Residual at x = 8 . . . . . . . . . . . . . . . . . . . 6

0

5 Problem 4  Multiple Linear Regression (Two Predictors) 6

5.1 Part (a)  Matrix (Normal Equation) Approach . . . . . . . . . . . . . . . . 7
5.1.1 Design Matrix and Target Vector . . . . . . . . . . . . . . . . . . . . . 7
5.1.2 Summation Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.1.3 X X and X y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
T T
7
5.1.4 Normal Equations System . . . . . . . . . . . . . . . . . . . . . . . . . 8
5.2 Part (b)  Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
6 Problem 5  Logistic Regression (Two Predictors) 8
6.1 Part (a)  Logistic Regression Model . . . . . . . . . . . . . . . . . . . . . . . 8
6.2 Part (b)  Classify 6 Customers . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.2.1 Detailed Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.2.2 Summary Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6.2.3 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6.2.4 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7 Complete Summary of All Problems 11

1
Machine Learning  Problem Set Solutions Benha National University

1. Gradient Descent for Linear Regression  Background

Core Formulas
Hypothesis (Linear Regression):
ŷ (i) = θ0 + θ1 x(i)

Cost Function (MSE):

m
1 X (i)
2
J(θ0 , θ1 ) = ŷ − y (i)
2m i=1

Update Rules (one iteration):

m
1 X (i)
ŷ − y (i)


θ0 := θ0 − α
m i=1

m
1 X (i)
ŷ − y (i) x(i)


θ1 := θ1 − α
m i=1

Note: θ1 is multiplied by x(i) because it is the partial derivative of ŷ with respect to θ1 :

∂ ŷ
∂θ1
= x.

2. Problem 1  Gradient Descent (First Iteration)

Problem Statement
Given:
X = [1, 2, 3, 4], Y = [1.5, 2, 2.5, 5]
Initial parameters: θ , . Learning rate: α = 0.23.
Find the updated values of and θ after one iteration.
0 = 0.5 θ1 = 0.5
θ0 1

2.1. Step 1  Compute Predictions

With ŷ (i)
= 0.5 + 0.5 x(i) :

2
Machine Learning  Problem Set Solutions Benha National University

i x(i) y (i) ŷ (i) ŷ (i) − y (i) (ŷ (i) − y (i) ) x(i)

1 1 1.5 0.5 + 0.5(1) = 1.0 −0.5 −0.5

2 2 2.0 0.5 + 0.5(2) = 1.5 −0.5 −1.0
3 3 2.5 0.5 + 0.5(3) = 2.0 −0.5 −1.5
4 4 5.0 0.5 + 0.5(4) = 2.5 −2.5 −10.0

Sum −4.0 −13.0

2.2. Step 2  Compute Gradients

∂J 1
= (−4.0) = −1.0
∂θ0 4
∂J 1
= (−13.0) = −3.25
∂θ1 4

2.3. Step 3  Update Parameters

θ0 := 0.5 − 0.23 × (−1.0) = 0.5 + 0.23
θ1 := 0.5 − 0.23 × (−3.25) = 0.5 + 0.7475

Final Answer  Problem 1

θ0 = 0.73 θ1 = 1.2475

3. Problem 2  Gradient Descent, Dierent Dataset (α =

0.01)

Problem Statement
Given:
X = [1, 2, 4, 3, 5, 6], y = [1, 3, 3, 2, 5, 4]
Same initial values: θ , . Learning rate: α = 0.01.
Find the updated values of and after one iteration.
0 = 0.5 θ1 = 0.5
θ0 θ1

3
Machine Learning  Problem Set Solutions Benha National University

3.1. Step 1  Compute Predictions (m = 6)

i x(i) y (i) ŷ (i) ŷ − y (ŷ − y) x

1 1 1 0.5 + 0.5(1) = 1.0 0.0 0.0

2 2 3 0.5 + 0.5(2) = 1.5 −1.5 −3.0
3 4 3 0.5 + 0.5(4) = 2.5 −0.5 −2.0
4 3 2 0.5 + 0.5(3) = 2.0 0.0 0.0
5 5 5 0.5 + 0.5(5) = 3.0 −2.0 −10.0
6 6 4 0.5 + 0.5(6) = 3.5 −0.5 −3.0

Sum −4.5 −18.0

3.2. Step 2  Compute Gradients

∂J 1
= (−4.5) = −0.75
∂θ0 6
∂J 1
= (−18.0) = −3.0
∂θ1 6

3.3. Step 3  Update Parameters

θ0 := 0.5 − 0.01 × (−0.75) = 0.5 + 0.0075
θ1 := 0.5 − 0.01 × (−3.0) = 0.5 + 0.03

Final Answer  Problem 2

θ0 = 0.5075 θ1 = 0.53

Comparison: Eect of Learning Rate

α ∆θ0 ∆θ1

Problem 1 0.23 +0.2300 +0.7475

Problem 2 0.01 +0.0075 +0.0300
A smaller α produces much smaller steps per iteration  the model converges more
slowly but more stably.

4
Machine Learning  Problem Set Solutions Benha National University

4. Problem 3  Simple Linear Regression

Problem Statement
Five observations on x (independent) and y (dependent):
x y
2 7
4 5
6 4
8 2
10 1
(a) Calculate the intercept b and slope b , and write down the estimated regression
equation.
0 1

(b) For x = 8, predict ŷ and calculate the residual.

4.1. Part (a)  Slope and Intercept
Formulas
P P
X ( xi )( yi )
xi yi −
b1 = Pn 2 b0 = ȳ − b1 x̄
X ( xi )
x2i −
n

4.1.1. Computation Table

xi yi x2i xi yi

2 7 4 14
4 5 16 20
6 4 36 24
8 2 64 16
10 1 100 10
P P P P
= 30 = 19 = 220 = 84

4.1.2. Means
30 19
x̄ = = 6, ȳ = = 3.8
5 5

5
Machine Learning  Problem Set Solutions Benha National University

4.1.3. Calculate b 1

30 × 19
84 − 84 − 114 −30
b1 = 5 = = = −0.75
900 220 − 180 40
220 −
5

4.1.4. Calculate b 0

b0 = 3.8 − (−0.75)(6) = 3.8 + 4.5 = 8.3

Regression Equation
ŷ = 8.3 − 0.75 x
The negative slope conrms: as x increases, y decreases.
4.2. Part (b)  Prediction and Residual at x = 8
ŷ = 8.3 − 0.75(8) = 8.3 − 6.0 = 2.3
e = y − ŷ = 2 − 2.3 = −0.3

Result
ŷ = 2.3 e = −0.3
The actual value is 0.3 units below the regression line.

5. Problem 4  Multiple Linear Regression (Two Predic-

tors)
Problem Statement
House price as a function of House Area (x ) and Lot Size (x ):
1 2

House Area (ft ) Lot Size (ft ) Price (USD)

2 2

1474 10000 150000

1560 9500 145000
1584 12000 160000
1645 13000 170000
1712 13500 190000
1870 13000 200000
Model: y = a + a x + a x
0 11 1 12 2

6
Machine Learning  Problem Set Solutions Benha National University

(a) Derive a , a , a using least squares.

(b) Predict price for Area = 1560 ft , Lot = 13750 ft .
0 11 12
2 2

5.1. Part (a)  Matrix (Normal Equation) Approach

Normal Equation
a = (X T X)−1 X T y
where the design matrix X has a column of ones, a column for x , and a column for
x.
1
2

5.1.1. Design Matrix and Target Vector

   
1 1474 10000 150000
1 1560 9500  145000
   
1 1584 12000 160000
X=
1
, y= 
 1645 13000
170000
 
1 1712 13500 190000
1 1870 13000 200000

5.1.2. Summation Table

x1 x2 y x21 x22 x1 x2 x1 y

1474 10000 150000 2172676 100000000 14740000 221100000

1560 9500 145000 2433600 90250000 14820000 226200000
1584 12000 160000 2509056 144000000 19008000 253440000
1645 13000 170000 2706025 169000000 21385000 279650000
1712 13500 190000 2930944 182250000 23112000 325280000
1870 13000 200000 3496900 169000000 24310000 374000000

9845 71000 1015000 16249201 854500000 117375000 1679670000

Also: P
x2 y = 12,172,500,000 .
5.1.3. X X and X y
T T

 
6 9845 71000
X T X =  9845 16249201 117375000
71000 117375000 854500000
 
1015000
X T y =  1679670000 
12172500000

7
Machine Learning  Problem Set Solutions Benha National University

5.1.4. Normal Equations System


6 a + 9845 a11 + 71000 a12 = 1,015,000
 0


9845 a0 + 16249201 a11 + 117375000 a12 = 1,679,670,000


71000 a0 + 117375000 a11 + 854500000 a12 = 12,172,500,000


Solving via Gaussian elimination / matrix inversion:

Estimated Regression Equation
a0 = −53307.8, a11 = 77.52, a12 = 6.15
ŷ = −53307.8 + 77.52 x1 + 6.15 x2
Every extra ft of house area adds $77.52; every extra ft of lot size adds $6.15.
2 2

5.2. Part (b)  Prediction

For x 1 = 1560 ft , x
2
2 = 13750 ft :
2

ŷ = −53307.8 + 77.52(1560) + 6.15(13750)

= −53307.8 + 120931.2 + 84562.5

Predicted House Price

ŷ ≈ $152,185.9

6. Problem 5  Logistic Regression (Two Predictors)

Problem Statement
Given parameters: β = −25.9382, β = 0.1109 (Income), β = 0.9638 (Lot_Size).
(a) Construct the logistic regression model.
0 1 2

(b) Using cuto p ≥ 0.75 ⇒ Owner; else Nonowner, classify 6 customers and build a
confusion matrix.
6.1. Part (a)  Logistic Regression Model
Sigmoid Function
1
p= , z = β0 + β1 x1 + β2 x2
1 + e−z
1
p=
1+ e−(−25.9382 + 0.1109·Income + 0.9638·Lot_Size)

8
Machine Learning  Problem Set Solutions Benha National University

6.2. Part (b)  Classify 6 Customers

6.2.1. Detailed Calculations
Customer 1  Income = 60.0, Lot_Size = 18.4
z = −25.9382 + 0.1109(60.0) + 0.9638(18.4) = −25.9382 + 6.654 + 17.7339 = −1.5503

p=
1
1+e 1.5503
=
1
1 + 4.7115
= 0.1751 ⇒ 0.1751 < 0.75 ⇒ Nonowner
Customer 2  Income = 64.8, Lot_Size = 21.6
z = −25.9382 + 0.1109(64.8) + 0.9638(21.6) = −25.9382 + 7.1863 + 20.8181 = 2.0662

p=
1+e
1
−2.0662
=
1
1 + 0.1267
= 0.8876 ⇒ 0.8876 ≥ 0.75 ⇒ Owner
Customer 3  Income = 84.0, Lot_Size = 17.6
z = −25.9382 + 0.1109(84.0) + 0.9638(17.6) = −25.9382 + 9.3156 + 16.9629 = 0.3403

p=
1
1 + e−0.3403
=
1
1 + 0.7114
= 0.5842 ⇒ 0.5842 < 0.75 ⇒ Nonowner
Customer 4  Income = 59.4, Lot_Size = 16.0
z = −25.9382 + 0.1109(59.4) + 0.9638(16.0) = −25.9382 + 6.5875 + 15.4208 = −3.9299

p=
1
1+e 3.9299
=
1
1 + 50.877
= 0.0193 ⇒ 0.0193 < 0.75 ⇒ Nonowner
Customer 5  Income = 108.0, Lot_Size = 17.6
z = −25.9382 + 0.1109(108.0) + 0.9638(17.6) = −25.9382 + 11.9772 + 16.9629 = 3.0019

p=
1+e
1
−3.0019
=
1
1 + 0.0498
= 0.9526 ⇒ 0.9526 ≥ 0.75 ⇒ Owner
Customer 6  Income = 75.0, Lot_Size = 19.6
z = −25.9382 + 0.1109(75.0) + 0.9638(19.6) = −25.9382 + 8.3175 + 18.8905 = 1.2698

p=
1+e
1
−1.2698
=
1
1 + 0.2808
= 0.7807 ⇒ 0.7807 ≥ 0.75 ⇒ Owner

9
Machine Learning  Problem Set Solutions Benha National University

6.2.2. Summary Table

Cust. Income Lot_Size z p Predicted Actual Correct?
1 60.0 18.4 −1.5503 0.1751 Nonowner Owner No
2 64.8 21.6 2.0662 0.8876 Owner Owner Yes
3 84.0 17.6 0.3403 0.5842 Nonowner Nonowner Yes
4 59.4 16.0 −3.9299 0.0193 Nonowner Nonowner Yes
5 108.0 17.6 3.0019 0.9526 Owner Owner Yes
6 75.0 19.6 1.2698 0.7807 Owner Nonowner No
6.2.3. Confusion Matrix
Actual \ Pre- Owner (Pred.) Nonowner
dicted (Pred.)
Owner (Actual) TP = 2 FN = 1
Nonowner (Ac- FP = 1 TN = 2
tual)
6.2.4. Performance Metrics
Accuracy = T PTotal
+ TN
=
2+2
6
4
= ≈ 66.7%
6
Precision = T PT+PF P = 23 ≈ 66.7%
Recall (Sensitivity) = T P T+PF N = 32 ≈ 66.7%
Key Observations
ˆ Customer 1 was misclassied as Nonowner (p = 0.175): low income and small
lot pushed the probability well below the threshold.
ˆ Customer 6 was misclassied as Owner (p = 0.781): its probability fell just
above the cuto 0.75 despite being a Nonowner.
ˆ Overall accuracy is 66.7% with the 0.75 cuto; lowering the threshold would
increase recall at the cost of precision.

10
Machine Learning  Problem Set Solutions Benha National University

7. Complete Summary of All Problems

All Results at a Glance
Problem 1  Gradient Descent (α = 0.23, m = 4)
θ0 = 0.73, θ1 = 1.2475

Problem 2  Gradient Descent (α = 0.01, m = 6)

θ0 = 0.5075, θ1 = 0.53

Problem 3  Simple Linear Regression

ŷ = 8.3 − 0.75x, ŷ|x=8 = 2.3, e = −0.3

Problem 4  Multiple Linear Regression

ŷ = −53307.8 + 77.52 x1 + 6.15 x2 , ŷ|x1 =1560, x2 =13750 ≈ $152,186

Problem 5  Logistic Regression (cuto 0.75)

1
p=
1 + e−(−25.9382+0.1109 x1 +0.9638 x2 )
Confusion Matrix: TP=2, FN=1, FP=1, TN=2 ⇒ Accuracy = 66.7%

End of Problem Set Solutions

11