lOMoARcPSD|61364143
Linear Regression by Hand
[Link] (Dr. A.P.J. Abdul Kalam Technical University)
Scan to open on Studocu
Studocu is not sponsored or endorsed by any college or university
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PN notes
NOTES ON
LINEAR REGRESSION
Topics
linear regression equation
formulae test
Pearson's correlation coefficient
bendline
making
line
Evaluating R2
Regression using
R squared
co efficient of Determination
Adjusted R2
Exercises
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
simple linear regression punoees
linear allows to
Simple regression us
predict the linear relationship between
two variables
So think in this way
we are
dealing with two variables
x feature independent variable
label dependent variable
y
so in this case take the value of
re and predict y
s
x y
Cinputs
Engined
Coutput
Now let's see the formulaes behind SLR
Linear Regression function
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PNNOTES
t ba see
y
ybo
intercept of slope of the
y line Regression line for
the
M
To calculate the slope of the line BD
we have two
formulae
formula 1
standard deviation
bl y sign of y
Sze
T standard deviation
Pearson's of
correlation 24
Coefficient
where
Pearson's correlation co efficient G can
be calculated
of
raffarth
mean
[Link]
IA8ZCcn
D8 st3Cy y
ng I
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PN Noles
standard deviation of y and ne is
FEEL
Sy sa
n 1
number
of A
gamp
formula 2
bI ny Iy
at Caf
wheeze
n feature label
y
a mean of the feature
g mean of the label
line
calculate
y intercept of the
To
bo
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
the am's
I I mean of
meangbff
slope of bine
n
for
Lets solve one problem
01 3 Gien the below dataset calculate
and plot the regression line
i a
Temperature Electricity consumption
K KW
I
40 35
50 40
SOI we will solve this problem using
method I
using Pearson's correlation
coefficient
The Pearson's correlation coefficient
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
the Pearson s correlation coefficient
is
formula
r Z Cen I Cy g 3
Zen a
7
5 Ecy g 5
Lets calculate all the values required to
substitute in above formula
201 30
most of 52
440
35
y i
yn
101 35 275
i 402
254
Cm of y g
5c 35
y1
27
I
P T O
PNevotes
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PNNOTES
G D G YTG
nyx
n y y n
[Link]
t f
50 40 15 125 187.5 225 156.25
50 25
n
Zen Cy 53 p Zyl 55
zen ay
o t 00 0 97590
5M 55 0.976
ooot 0 cl
the result
from above we
say these
can
enists positive correlation between feature
and label
lets calculate the standard deination of a
n number of samples I
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
Sn
f
12 9
13.228
Sy
152 97
8
40.976 12
Sy l322
lets calculate the slope of the line
b Cslopeof
Saf
o
a r
a 6
YET
If
b
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
P N NGS
Lets calculate the intercept of line
to I m a
27.5 l 35
7
54
lbo SI
t 7
so
from the simple linear
regression equation will be
bot b a
y A
equation
eine
7.5 t d ni
y the
Now lets create a bendline
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PN notes
t T
or
A
yeadline
50 feet
pegosion
no V
0
30
[Link]
predicted
co
I
10 actual y
b b b to fo
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
ENDED
Evaluating the Regression Line
METHOD I R squared
co efficient of determination
The motto of R squared is to figure
out how good the line is filled ensuring
minimal errors of points cold
To find R2 need to ask the
you
following questions
WHAT PERCENTAGE OF THE VARIATION
IN IS DESCRIBED BY THE VARIATION
IN N
y
label
feature
Total variations in
y
Squared error
from mean of y
Cy g ft Cf Ift t Lyn 55
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
Porno
This is also called
Yi 5 OF SQUARES
7
SUM
2g OF RESIDUALS
variance
of y Totalvariationsing
n
Effi 5
L
T
K
what percentage
of total variation is
not described by variation in n
R2 I Sekine
SET
The above equation is also called
CO EFFLUENT OF DETERMINATION
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PNovoty
If R2 is greater value i e closer
to 1 then the line is the best
tilted line
[Link] is small line is a
good
11 fit
R2 is close to 1
Seine is
large line is not a
good
tit
H
R2 is close to 0
Lets see an example to calculate R2
y TT y 55
N Sarased error squared from
y Y with line mean y
2 3 2.1905 o 6553 10.5625
I I 1.2143 O 0459 1.5625
I 2 o 7387 l 5924 3 0625
4 3 3 6666 O 4464 7.5625
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
PNnotes
from table
Ly 55 2 738 SEeine
z y J 22 75 3 SET
R2 I serine
SET
i
zits
I O 1203
O 8797 o
88g
[Link]
since RZ is
very near to 1 therefore its
a
good fit line
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])
lOMoARcPSD|61364143
Arnott
PROBLEMS FOUND IN RZ metric
R2 increases with increase in features
predictions independent variables added
to the model This property of R2 will
MISLEAD the fact the model is best fit
The abore
misleading also happens when
Polynomial features are introduced
Solution
The above Ipsobleen can be soled using
ADJUSTED Rt metric
ADJUSTED R2 I n D
1572
cohese
number observations datapts
n is of
K is no of independent variables in
your model excluding the
constant
Downloaded by Harshit shukla (shuklaharshit2002megatron@[Link])