2/27/2026

Unit 02
Supervised Learning: Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |1

Simple Linear Regression (SLR)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |2

1
 2/27/2026

Linear regression
Linear Regression:
 a statistical approach to modeling the relationship
between a dependent variable and a given set of
independent variables

 There is a positive linear

correlation(relationship) when the variable
on the x-axis increases as the variable on
the y-axis increases

 There is a negative linear correlation when

one variable increases as the other variable
decreases
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |3

Linear regression
Linear Regression:
 There is a non-linear
correlation when there is a
relationship between variables
but the relationship is
not linear (straight)

 There is no correlation when

there is no pattern that can be
detected between the variables

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |4

2
 2/27/2026

Simple Linear regression

 a statistical method for establishing the relationship between single
dependent variable and a single independent variable using a straight
line(e.g. Predicting a student’s marks based on hours studied)
 Dependent variable must be a continuous/real value
 Independent variable can be measured on continuous or categorical
values
 Objectives:
1. Model the relationship between the two variables – e.g. Income &
Expenditure, Experience & Salary etc.
2. Forecasting new observations - Weather forecasting according to
temperature
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |5

Simple Linear Regression

Equation of SLR:
y = a0+a1x + є
Where,
 y: Predicted Variable
 a0 : ‘y’ Intercept
a1
 a1: Coefficient of regression
 x: Independent variable a0
 Є : Error

 Goal: find optimum values of a0 and a1 (Best Fit Line)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |6

3
 2/27/2026

Simple Linear regression

Best Fit Line
 A regression line that
implies as minimum error
in actual and predicted
value as possible
 Points exactly on
regression line(Red dots) –
predicted values
 Points around the
regression line(Black dots)
– actual values
 Several lines may exists
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |7

Simple Linear regression

Best Fit Line
 A regression line that
implies as minimum error
in actual and predicted
value as possible
 Points exactly on
regression line – predicted
values
 Points around the
regression line – actual
values
 Several lines may exists
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |8

4
 2/27/2026

Simple Linear Regression

Equation for SLR:

y = a0 + a1X
Where,
 y: Predicted value
 a0: Y- intersect
 a1 : Slope of independent variable
 X : Independent variable

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) |9

Simple Linear regression

Assumptions of SLR:
1. Linearity:
 The independent and dependent variables have a linear relationship
 Changes in the dependent variable follow those in the independent
variable(s) in a linear fashion
 If the relationship is not linear, then linear regression will not be an
accurate model

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 10

5
 2/27/2026

Simple Linear regression

Assumptions of SLR:
2. Independence:
 The observations/actual values in the dataset are independent of each
other
 i. e. Value of the dependent variable for one observation does not depend
on the value of the dependent variable for another observation
 If the observations are not independent, then linear regression will not be
an accurate model
Height in ft Weight in Kg BMI
55 is independent of 60 and 75 (X) (y)
5 55 9.3
5.5 60 21.3
9.3 is independent of 60 and 75
6 75 22.4
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 11

Simple Linear regression

Assumptions of SLR:
3. Homoscedasticity (Constant Variance) :
 Across all levels of the independent
variable(s), the variance of the errors
is constant

 This indicates that the amount of the

independent variable(s) has no impact
on the variance of the errors

 If the variance of the errore is not

constant, then linear regression will
not be an accurate model
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 12

6
 2/27/2026

Simple Linear regression

Assumptions of SLR:
4. Normality :
 The residuals (Actual value –
Predicted value) should be normally
distributed
 If all the residuals (errors) are plotted
in a histogram, they should roughly
form a bell-shaped curve (normal
distribution)
 If the residuals are not normally
distributed, then linear regression
will not be an accurate model
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 13

Simple Linear regression

Example 1 - (Least Square Method ):
 Given a dataset, find the equation (using least
square method) for prediction of student's final
exam grade based on mid term grade in the
course Machine Learning.
 Predict the final exam grade of student who
obtained 70 grade in the mid term exam.

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 14

7
 2/27/2026

Simple Linear Regression

Equation for a0 and a1(Least Square Method):

Equation of SLR: y = a0 + a1X a0 = y̅ - a1X

Where,
 Xi: ‘i‘ th input variable
 yi: ‘i‘ th onput variable
 x̅ : Mean of all X values
 y̅ : Mean of all y values
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 15

Simple Linear Regression

Example (Least Square Method ):
 Here, Mid Term is independent variable and Final
Exam is dependent variable
 Equation of Simple Linear regression:
Final Exam = a0 + a1(Mid Term)

 Least Square Method:

a1

a0 = y̅ - a1x͞

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 16

8
 2/27/2026

Simple Linear Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 17

Simple Linear Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 18

9
 2/27/2026

Simple Linear Regression

-0.17 10 -1. 7 0.03

-22.17 11 243.87 491.51
8.83 3 26.49 77.97
1.83 4 7.32 3.35
21.83 16 349.28 476.55
13.83 1 13.83 91.27
-13.17 -25 329.25 173.45
10.83 5 54.15 117.29
-7.17 3 -21.51 51.41
-39.17 -22 861.74 1534.29
15.83 0 0 250.59
8.83 16 141.28 77.97

2004 3445.67

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 19

Simple Linear Regression

Example (Least Square Method ):
 Hence, SLR equation is:

Final Exam = 32.02 + 0.5816 x (Mid Term)

 When Mid Term is 70

Final Exam = 32.02 + 0.5816 x 70
= 72.74

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 20

10
 2/27/2026

Simple Linear Regression (SLR) Code

Example

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 21

Multiple Linear Regression

(MLR)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 22

11
 2/27/2026

Multiple Linear regression

Linear Regression:
 Simple Linear Regression − only addresses two features.
 Multiple Linear Regression − Deals with more than one feature at once
F1: size of the
house
Independent Variables
F2: No. of
Bedrooms
Size of No. of Locality Condition Price
the Bed of the
F3: Locality house rooms house

F4: Condition of
the house

F5: Price of
House ? Dependent Variable

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 23

Multiple Linear regression

Equation of MLR:

y = a0+a1x1+a2x2+a3x3+ …… anxn + є
Where,
 a0 : ‘y’ Intercept
 a1 , a2 , a3 , … an : Slope or Coefficient of regression
 x1 , x2 , x3 , … xn : Independent / Predictor variables
 Є : Error

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 24

12
 2/27/2026

Multiple Linear regression

Equation of MLR:
Size of No. of Locality Conditio Price
the Bed n of the
house rooms house
F1: size of the
house

F2: No. of
Bedrooms

F3: Locality

F4: Condition of
the house Price of House =
F5: Price of a0 + a1(Size of the house) + a2(No.
House ? of Bed rooms) + a3(Locality) +
a4(Condition of the house)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 25

Multiple Linear regression

Assumptions of Regression Model:

 Linearity: The relationship between dependent and

independent variables should be linear

 Multivariate normality: Multiple Regression assumes that the

residuals are normally distributed.

 Lack of Multi-colinearity: It is assumed that there is little or

no multicollinearity between the independent variables.

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 26

13
 2/27/2026

Multiple Linear regression

Numerical Example: TV Mobile
Weekly
Sales
(x1) (x2)
 Equation: (Y)

Weekly Sales = a0+a1(TV)+a2(Mobile) 1 4 1

 Matrices for TV and Mobile will be as: 2 5 6

3 8 8

4 2 12

6 10 ?

Bias Intersect

Coeff. Of
 Equation of Y-Intersect and Slopes will be TV

as: Coeff. Of
Mobile

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 27

Multiple Linear regression

Numerical Example: TV Mobile
Weekly
Sales
(x1) (x2)
 Matrix of coefficients is given as: (Y)
1 4 1

2 5 6

3 8 8

4 2 12

6 10 ?

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 28

14
 2/27/2026

Multiple Linear regression

Numerical Example: TV Mobile
Weekly
Sales
(x1) (x2)
 Matrix of coefficients is given as: (Y)
1 4 1

2 5 6

3 8 8

4 2 12

6 10 ?

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 29

Multiple Linear regression

Numerical Example: TV Mobile
Weekly
Sales
(x1) (x2)
 Matrix of coefficients is given as: (Y)
1 4 1

2 5 6

3 8 8

4 2 12

6 10 ?

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 30

15
 2/27/2026

Multiple Linear regression

Numerical Example: TV Mobile
Weekly
Sales
(x1) (x2)
 Matrix of coefficients is given as: (Y)
1 4 1

2 5 6

3 8 8

4 2 12

6 10 ?

Intersect

Coeff. Of
TV

Coeff. Of
Mobile

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 31

Multiple Linear regression

Numerical Example: TV Mobile
Weekly
Sales
(x1) (x2)
 Hence, the Multiple Linear Regression (Y)
model is as: 1 4 1

Weekly Sales = 2 5 6

-1.69+3.48(TV) - 0.05(Mobile) 3 8 8

4 2 12

= -1.69+3.48(6) - 0.05(10) 6 10 ?

= 18.69

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 32

16
 2/27/2026

Multiple Linear Regression

Multiple Linear Regression Code

Example

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 33

Evaluation metric: Mean Squared Error (MSE )

Evaluation metric: Mean Squared Error (MSE )
 Mean Squared Error is a loss function used in regression
 It measures the average squared difference between the predicted values and
the actual values

 Squaring makes all errors positive and penalizes large mistakes more than
small ones
 For example, being wrong by 10 (error² = 100) is much worse than being
wrong by 2 (error² = 4)
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 34

17
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Evaluation metric: Mean Squared Error (MSE )

Interpretation of MSE:
 MSE = 0 → Perfect predictions (no error).
 Small MSE → Predictions are close to actual values
(good model).
 Large MSE → Predictions are far from actual values
(bad model)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 35

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error (MSE)
Final Exam ( y) Predicted (ŷ)
Example 1:
84 73.903067
 From Final Exam Marks and
63 61.107865
Predicted marks calculate the Mean 77 79.137467
Squared Error. 78 75.066267
90 86.698268
75 82.045468
49 66.342266
79 80.300668
77 69.831866
52 51.220664
74 83.208668
90 79.137467

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 36

18
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error (MSE)
Final Exam ( y) Predicted (ŷ)
Example 1:
84 73.903067
63 61.107865
77 79.137467
78 75.066267
90 86.698268
75 82.045468
49 66.342266
79 80.300668
77 69.831866
52 51.220664
74 83.208668
90 79.137467

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 37

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Final ( y) Predicted (ŷ) Error e=(y−ŷ) Squared error (e2)
(MSE) Example 1: 84 73.903067
63 61.107865
77 79.137467
78 75.066267
90 86.698268
75 82.045468
49 66.342266
79 80.300668
77 69.831866
52 51.220664
74 83.208668
90 79.137467

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 38

19
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Final ( y) Predicted (ŷ) Error e=(y−ŷ) Squared error (e2)
(MSE) Example 1: 84 73.903067 10.096933
63 61.107865 1.892135
77 79.137467 -2.137467
78 75.066267 2.933733
90 86.698268 3.301732
75 82.045468 -7.045468
49 66.342266 -17.342266
79 80.300668 -1.300668
77 69.831866 7.168134
52 51.220664 0.779336
74 83.208668 -9.208668
90 79.137467 10.862533

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 39

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Final ( y) Predicted (ŷ) Error e=(y−ŷ) Squared error (e2)
(MSE) Example 1: 84 73.903067 10.096933 101.948056
63 61.107865 1.892135 3.580175
77 79.137467 -2.137467 4.568765
78 75.066267 2.933733 8.606789
90 86.698268 3.301732 10.901434
75 82.045468 -7.045468 49.638619
49 66.342266 -17.342266 300.75419
79 80.300668 -1.300668 1.691737
77 69.831866 7.168134 51.382145
52 51.220664 0.779336 0.607365
74 83.208668 -9.208668 84.799566
90 79.137467 10.862533 117.994623

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 40

20
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Final ( y) Predicted (ŷ) Error e=(y−ŷ) Squared error (e2)
(MSE) Example 1: 84 72.251717 11.748283 138.022153
63 61.478379 1.521621 2.315330
77 76.658992 0.341008 0.116286
78 73.231112 4.768888 22.742293
90 83.025056 6.974944 48.649844
75 79.107478 -4.107478 16.871376
49 65.885653 -16.885653 285.125277
79 77.638386 1.361614 1.853993
77 68.823837 8.176163 66.849641
52 53.153526 -1.153526 1.330622
Interpretation:
74 80.086872 -26.086872 680.524891
Since, MSE is high this model
is a bad model 90 76.658992 13.341008 177.982494
Σ 736.473465
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 41

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error (MSE) Code

Example

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 42

21
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Example2(Least Square Method ) Exercise:
Hours_Studied Exam_Score
 Given a dataset, find the equation (using least (X) (y)
square method) for prediction of student's final 1 35
Exam_Score based on Hours_Studied. 2 42

 Predict the Exam_Score of student who has 3 50

studied for 12 hours daily. 4 55

5 62
6 70
7 72
8 78
9 85
10 90
12 ?
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 43

Evaluation metric: Mean Squared Error (MSE )

Hours_Studied Exam_Score
Example 2 (Least Square Method ): (X) (y)
 Here, Hours_Studied is independent variable and 1 35
Exam_Score is dependent variable 2 42
 Equation of Simple Linear regression: 3 50
Exam_Score = a0 + a1(Hours_Studied ) 4 55
5 62
6 70
 Least Square Method:
7 72
8 78
9 85
10 90
12 ?

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 44

22
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Hours_Stud Exam_Score (𝑿 − 𝑿)*
(𝑿 − 𝑿) (𝒀 − 𝒀) (𝑿 − 𝑿)2
ied(X) (Y) (𝒀 − 𝒀)
1 35 -4.5 -28.9 130.05 20.25
2 42 -3.5 -21.9 76.65 12.25
3 50 -2.5 -13.9 34.75 6.25
4 55 -1.5 -8.9 13.35 2.25
a1 = 𝟒𝟗𝟕.𝟓
𝟖𝟐.𝟓
= 𝟔. 𝟎𝟑
5 62 -0.5 -1.9 0.95 0.25
6 70 0.5 6.1 3.05 0.25 a0 = 63.9 – (6.03)(5.5)
= 30.7
7 72 1.5 8.1 12.15 2.25
8 78 2.5 14.1 35.25 6.25 y = 30.7 + 6.0(Hours_studied)

9 85 3.5 21.1 73.85 12.25 When Hours_studied = 12

10 90 4.5 26.1 117.45 20.25
y = 30.7 + 6.0 (12)
𝑿 = 5.5 𝒀 = 63.9 Σ = 497.5 Σ = 82.5
y = 103
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 45

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error (MSE)
Hours_Studied Exam_Score Predicted_Final
Example 2:
1 35 36.763636
2 42 42.793939
3 50 48.824242
4 55 54.854545
5 62 60.884848
6 70 66.915152
7 72 72.945455
8 78 78.975758
9 85 85.006061
10 90 91.036364

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 46

23
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Exam_Score Predicted Error Squared
(y) (ŷ) e=(y−ŷ) error (e2)
(MSE) Example 2:
35 36.763636
42 42.793939
50 48.824242
55 54.854545
62 60.884848
70 66.915152
72 72.945455
78 78.975758
85 85.006061
90 91.036364

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 47

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Exam_Score Predicted Error Squared
(y) (ŷ) e=(y−ŷ) error (e2)
(MSE) Example 2:
35 36.763636 -1.763636
42 42.793939 -0.793939
50 48.824242 1.175758
55 54.854545 0.145455
62 60.884848 1.115152
70 66.915152 3.084848
72 72.945455 -0.945455
78 78.975758 -0.975758
85 85.006061 -0.006061
90 91.036364 -1.036364

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 48

24
 2/27/2026

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Exam_Score Predicted Error Squared
(y) (ŷ) e=(y−ŷ) error (e2)
(MSE) Example 2:
35 36.763636 -1.763636 3.110415
42 42.793939 -0.793939 0.630340
50 48.824242 1.175758 1.382398
55 54.854545 0.145455 0.021159
62 60.884848 1.115152 1.243559
70 66.915152 3.084848 9.512272
72 72.945455 -0.945455 0.893878
78 78.975758 -0.975758 0.952106
85 85.006061 -0.006061 0.000037
90 91.036364 -1.036364 1.074056

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 49

Evaluation metric: Mean Squared Error (MSE )

Mean Squared Error Exam_Score Predicted Error Squared
(y) (ŷ) e=(y−ŷ) error (e2)
(MSE) Example 2:
35 36.763636 -1.763636 3.110415
42 42.793939 -0.793939 0.630340
50 48.824242 1.175758 1.382398
55 54.854545 0.145455 0.021159
62 60.884848 1.115152 1.243559
70 66.915152 3.084848 9.512272
72 72.945455 -0.945455 0.893878
78 78.975758 -0.975758 0.952106

Interpretation: 85 85.006061 -0.006061 0.000037

Since, MSE is too small this 90 91.036364 -1.036364 1.074056

model is a good model
Σ 18.820219
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 50

25
 2/27/2026

Mean Squared Error (MSE) Code

Example 2

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 51

Mean Absolute Error(MAE)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 52

26
 2/27/2026

Evaluation metrics: Mean Absolute Error(MAE)

 A regression evaluation metric used to measure how far predictions are from
actual values on average
 It measures the average absolute difference between the predicted values
and the actual values

Why to take modulus?

 Some errors can be positive and some can be negative
 While computing total error some values can cancel each other – leads to
wrong result
 e.g. Errors: 5000 - 450 = 500, 5000 - 5500 = - 500; If we add both: 500 – 500 = 0
 But in reality the error was 500 which is too big to neglect
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 53

Evaluation metrics: Mean Absolute Error(MAE)

Advantages of MAE:
 The error value is same unit as that of target variable (e.g. inches, Celsius etc)
 MAE is very robust regarding outliers
 MAE focuses more on magnitude of the error
 Linear penalty (all errors treated equally)

When to Use MAE?:

Use MAE when:
 We want interpretable error
 Outliers should not dominate
 Error importance is uniform

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 54

27
 2/27/2026

Evaluation metrics: Mean Absolute Error(MAE)

Interpretation of MAE:
 MAE has no universal “good” value.
 It depends on the scale(range) of the target variable
 We must compare MAE to actual data values

Data Range MAE Interpretation

0 to 100 2 Very accurate
0 to 100 25 Poor accuracy
₹1 lakh to ₹2 lakh ₹5,000 Good
₹1 lakh to ₹2 lakh ₹60,000 Bad

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 55

Evaluation metrics: Mean Absolute Error(MAE)

Mean Absolute Error(MAE) Numerical
Actual Score 𝑦 Predicted Score (ŷ)
Example:
80 75
 From Actual Score and Predicted Score
60 70
calculate the Mean Absolute Error.
90 85
70 65
85 95

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 56

28
 2/27/2026

Evaluation metrics: Mean Absolute Error(MAE)

Mean Squared Error (MSE)
Example 1:
Actual Score 𝑦 Predicted Score (ŷ)

80 75
60 70
90 85
70 65
85 95

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 57

Evaluation metrics: Mean Absolute Error(MAE)

Mean Squared Error Actual( y) Predicted (ŷ) Error e=(y−ŷ) Absolute error
(MSE) Example 1: 80 75
60 70
90 85
70 65
85 95

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 58

29
 2/27/2026

Evaluation metrics: Mean Absolute Error(MAE)

Mean Squared Error Actual( y) Predicted (ŷ) Error e=(y−ŷ) Absolute error
(MSE) Example 1: 80 75 80−75 = 5
60 70 60−70 = −10
90 85 90-85 = 5
70 65 70-65 = 5
85 95 85 -95 = −10

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 59

Evaluation metrics: Mean Absolute Error(MAE)

Mean Squared Error Actual( y) Predicted (ŷ) Error e=(y−ŷ) Absolute error
(MSE) Example 1: 80 75 80−75 = 5 5
60 70 60−70 = −10 10
90 85 90-85 = 5 5
70 65 70-65 = 5 5
85 95 85 -95 = −10 10

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 60

30
 2/27/2026

Evaluation metrics: Mean Absolute Error(MAE)

Mean Squared Error Actual( y) Predicted (ŷ) Error e=(y−ŷ) Absolute error
(MSE) Example 1: 80 75 80−75 = 5 5
60 70 60−70 = −10 10
90 85 90-85 = 5 5
70 65 70-65 = 5 5
85 95 85 -95 = −10 10
No. of observations = 5
Σ 35

MAE = 35/5 = 7

Interpretation:
Since, MSE is lower this
model is a average model
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 61

Mean Absolute Error(MAE): Code

Example

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 62

31
 2/27/2026

R² (R-squared) - Coefficient of
Determination

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 63

R² (R-squared) - Coefficient of Determination

 A statistical measure that determines the
proportion of variance in the dependent variable
that is predictable from the independent variables
 Essentially, it’s a score that reflects how well the
data fit the regression model
 The value of R-squared lies between 0 and 1(0% -
Regression model R2=15%
100%):
 R² = 0 (0%)means the model explains none of the
variability;
 R² = 1(100%) means the model explains all of it.
 Usually, the larger the R2, the better the
regression model fits your observations Regression model R2=85%

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 64

32
 2/27/2026

R² (R-squared) - Coefficient of Determination

Calculation of R-squared:
Simple Linear Regression:
 R2 is calculated as the square of the Pearson correlation coefficient(r)
between the dependent and independent variables.
R² = r²

Multiple Regression Models:

Residual Sum of Squares(RSS/SSR)

Total Sum of Squares (TSS/SST)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 65

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual Predicted
No.
 Calculate R2 score for given data: (y) (ŷ)
1 10 9
2 12 11
3 15 14
4 18 17
5 20 19
Step 1: Mean of Actual Values
6 22 21
7 25 24
8 30 29

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 66

33
 2/27/2026

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual
(y)
 Calculate R2 score for given data:
10
12
15
18
Step 2: Total Sum of Squares (TSS) 20
22
25
30

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 67

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual
(y)
 Calculate R2 score for given data:
10 -9
12 -7
15 -4
18 -1
Step 2: Total Sum of Squares (TSS) 20 1
22 3
25 6
30 11

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 68

34
 2/27/2026

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual
(y)
 Calculate R2 score for given data:
10 -9 81
12 -7 49
15 -4 16
18 -1 1
Step 2: Total Sum of Squares (TSS) 20 1 1
22 3 9
25 6 36
30 11 121
Ʃ 314

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 69

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual Predicted Error
(y−ŷ)²
(y) (ŷ) (y−ŷ)
 Calculate R2 score for given data:
10 9
12 11
15 14
18 17
Step 3: Residual Sum of Squares (RSS): 20 19
22 21
25 24
30 29

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 70

35
 2/27/2026

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual Predicted Error
(y−ŷ)²
(y) (ŷ) (y−ŷ)
 Calculate R2 score for given data:
10 9 1
12 11 1
15 14 1
18 17 1
Step 3: Residual Sum of Squares (RSS): 20 19 1
22 21 1
25 24 1
30 29 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 71

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual Predicted Error
(y−ŷ)²
(y) (ŷ) (y−ŷ)
 Calculate R2 score for given data:
10 9 1 1
12 11 1 1
15 14 1 1
18 17 1 1
Step 3: Residual Sum of Squares (RSS): 20 19 1 1
22 21 1 1
25 24 1 1
30 29 1 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 72

36
 2/27/2026

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual Predicted Error
(y−ŷ)²
(y) (ŷ) (y−ŷ)
 Calculate R2 score for given data:
10 9 1 1
12 11 1 1
15 14 1 1
18 17 1 1
Step 3: Residual Sum of Squares (RSS): 20 19 1 1
22 21 1 1
25 24 1 1
30 29 1 1
Ʃ= 8

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 73

R² (R-squared) - Coefficient of Determination

Numerical Example: Actual Predicted Error
(y−ŷ)²
(y) (ŷ) (y−ŷ)
 Calculate R2 score for given data:
10 9 1 1
12 11 1 1
15 14 1 1
18 17 1 1
20 19 1 1
Step 4: Calculate R² 22 21 1 1
• Model explains 97.45% of 25 24 1 1
variation in actual data 30 29 1 1
• Predictions are very close to
Ʃ= 8
real values
• Excellent model fit

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 74

37
 2/27/2026

R² (R-squared): Code Example

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 75

R² (R-squared) - Coefficient of Determination

Interpreting R-Squared:
 A high R2 means the model explains a large portion of
the variance. However, it doesn’t mean our model is
necessarily correct or useful
 A low R-squared means little variance is explained by
the model. This isn’t always bad
 Especially in social sciences or noisy data, where a low
R-squared is often expected
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 76

38
 2/27/2026

R² (R-squared) - Coefficient of Determination

When to Use R-Squared:

R-squared can be very helpful if used in the right
situations. It’s appropriate for:
 Comparing models with the same number of
predictors
 Explaining how much variance in a dataset is
accounted for by the model
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 77

R² (R-squared) - Coefficient of Determination

When to Use R-Squared:

On the other hand, R-squared can be misleading in the
following scenarios:

 Adding irrelevant predictors, which results in overfitting.

 In this case, R-squared always increases even if the

predictive performance of the model doesn't improve.

 Comparing models with different numbers of predictors

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 78

39
 2/27/2026

Logistic Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 79

Logistic regression

 Linear regression predicts numerical response but not able to predict

categorical variables
 When response is categorical it is called classification
 Logistic regression predicts the output of a categorical dependent
variable. O/P : Categorical
 It can be either Yes or No, 0 or 1, true or False, etc.
 Instead of giving the exact value as 0 and 1, it gives the probabilistic
values which lie between 0 and 1
 “S” shaped logistic function is used, which predicts two maximum
values (0 or 1)

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 80

40
 2/27/2026

Logistic regression
Scenarios of predicting categorical variables:
1. Is email Spam or not? Answer: Yes or No ⇨ Categorical
response

2. Based on the CET marks if student should be admitted to

programme or not? Answer: Should admitted or
Shouldn’t admitted ⇨ Categorical response

3. Based on the marks secured, decide whether student is

passed or failed. Answers: Pass or Fail ⇨ Categorical
response
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 81

Logistic regression
How Logistic regression works?
 Suppose, a company wants to
determine employee’s salary hike
based on their performance
Salary Hike

 Linear regression model will solve

this problem
 Independent variable: Employees
performance rating, Dependent
Variable: Salary hike
 Plotting a best-fit regression line
will serve the purpose Employee Rating

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 82

41
 2/27/2026

Logistic regression
How Logistic regression works?
 Suppose, a company wants to
determine employee’s salary hike
based on their performance

Salary Hike
 Linear regression model will solve
this problem
 Independent variable: Employees
performance rating, Dependent
Variable: Salary hike
 Plotting a best-fit regression line
will serve the purpose Employee Rating

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 83

Logistic regression
How Logistic regression works?
 What if, a company wants to know
whether employee should be get
promotion or not?
Salary Hike

 Linear regression model wont solve

this problem

Employee Rating

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 84

42
 2/27/2026

Logistic regression
How Logistic regression works?
 What if, a company wants to know
whether employee should be get
promotion or not?

getting promotion
Probability of
 Linear regression model wont
solve this problem
 Instead of straight line, S- shaped
curve is fitted over data points
 If probability is 0 ⇨ Wont get
promotion
 If probability is 1 ⇨ Get promotion Employee Rating

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 85

Logistic regression
How Logistic regression works?
 But probability is not always 0 or 1
 What if it is between these two?
 A threshold value is decided
getting promotion
Probability of

 Based on the threshold values a

company can decide whether
employee should be promoted or
not
 E.g. Threshold is 0.5
 If Probability <0.5 ⇨ Wont
promoted Employee Rating
 If Probability >= 0.5 ⇨ Get
R. [Link]
Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 86

43
 2/27/2026

Logistic regression
How Logistic regression
works?
 Suppose employee rating is 2.8,

getting promotion
probability = 0.25 ⇒ wont get

Probability of
promotion

 If employee rating is 3.6,

probability ⇒ 0.75 ⇒ Will get
promotion
 ‘S’ shaped curve is called as Employee Rating
Sigmoid function
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 87

Logistic regression
Logistic Function (Sigmoid
Function):
 A mathematical function used to
map the predicted values to
probabilities
 It maps any real value into another
value within a range of 0 and 1
 Values above the threshold value
tends to 1, and a value below the
threshold values tends to 0
 Values is equal to exactly 0.5, value
tends to 1
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 88

44
 2/27/2026

Logistic regression
Assumptions of Logistic Regression:
 Independent observations: Each observation is independent of
the other. meaning there is no correlation between any input
variables.
 Binary dependent variables: The dependent variable must be
binary or meaning it can take only two values.
 Linear relationship between independent variables and log
odds
 No outliers: There should be no outliers in the dataset.
 Large sample size: The sample size is sufficiently large
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 89

Logistic regression
Equation of Logistic regression: y
 Odds of success:

a1

a0

x
 Values of Odds ranges form 0 to ∞, as P can
be between 0 and 1
 Consider the equation straight line:
y=a0+a1.x
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 90

45
 2/27/2026

Logistic regression
Equation of Logistic regression: y
 To find out Odds of success we need to take
logarithm of : i.e.
a1
 Here, P is x, as we want the probability of
happening of x then its corresponding y
a0
value. Hence equation becomes:
x

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 91

Logistic regression
Equation of Logistic regression:

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 92

46
 2/27/2026

Logistic regression

=1

=1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 93

Logistic regression
Numerical Example 1:
 Consider a dataset of CIBIL
scores of the bank customers.
A trained logistic regression
model have learned the
parameters as: a0 = 1 and a1 =
8.
 If a customer XYZ has a CIBIL
score of 60, predict whether
he/she will get a loan or not.
(Assume a threshold of 0.5) Since, probability of getting a loan is 1
(above threshold value), a customer
will get a loan
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 94

47
 2/27/2026

Logistic regression
Numerical Example 2: Study Hours Pass(1)/Fail(0)
 Consider a dataset that contains hours 29 0
of study and result of student. Apply 15 0
Logistic regression classifier and
33 1
answer the following questions:
28 1
1. What is probability of student to
pass who studied for 33 hours? 39 1

2. How many hours of study a students

requires to pass a course with
probability of more than 95% ?
Assume log(odds) as:
Log(odds) = -64+2*study hours
R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 95

Logistic regression
 In logistic regression sigmoid curve is given by:

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 96

48
 2/27/2026

Logistic regression
Numerical Example 2: Study Hours Pass(1)/Fail(0)
1. What is probability of student to pass who 29 0
studied for 33 hours? 15 0
33 1
28 1
39 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 97

Logistic regression
Numerical Example 2: Study Hours Pass(1)/Fail(0)
1. What is probability of student to pass who 29 0
studied for 33 hours? 15 0
33 1
28 1
39 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 98

49
 2/27/2026

Logistic regression
Numerical Example 2: Study Hours Pass(1)/Fail(0)
2. How many hours of study a students 29 0
requires to pass a course with probability 15 0
of more than 95% ?
33 1
28 1
39 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 99

Logistic regression
Numerical Example 2: Study Hours Pass(1)/Fail(0)
2. How many hours of study a students 29 0
requires to pass a course with probability 15 0
of more than 95% ?
33 1
28 1
39 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 100

50
 2/27/2026

Logistic regression
Numerical Example 2: Study Hours Pass(1)/Fail(0)
2. How many hours of study a students 29 0
requires to pass a course with probability 15 0
of more than 95% ?
33 1
28 1
39 1

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 101

Polynomial Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 102

51
 2/27/2026

Polynomial Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 103

Polynomial Regression

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 104

52
 2/27/2026

Polynomial Regression

n = No. of
elements

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 105

Polynomial Regression
Numerical Example:
 Apply Polynomial Regression on
given a dataset and find the
equation.
 Also predict the value of y when x =
7.

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 106

53
 2/27/2026

Polynomial Regression
Numerical Example:

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 107

Polynomial Regression
Numerical Example:

Equation:

When x = 7, y = -0.75+0.95(7)+0.75(49) = 42.65

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 108

54
 2/27/2026

Polynomial Regression
Advantages
 Captures non-linear relationships.
 More flexible than simple linear regression.

Disadvantages
 Overfitting if polynomial degree is too high.
 Sensitive to outliers.
 Interpretation becomes harder for high-degree polynomials.

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 109

Polynomial Regression

Polynomial Regression Code Example

R. B. Mannade, Asst. Professor Govt. College of Engineering, Aurangabad(MS) | 110

55