27-04-2025

1
 27-04-2025

2
 27-04-2025

3
 27-04-2025

4
 27-04-2025

10

5
 27-04-2025

11

12

6
 27-04-2025

13

14

7
 27-04-2025

15

16

8
 27-04-2025

17

18

9
 27-04-2025

19

20

10
 27-04-2025

21

22

11
 27-04-2025

23

24

12
 27-04-2025

25

26

13
 27-04-2025

27

28

14
 27-04-2025

29

30

15
 27-04-2025

31

32

16
 27-04-2025

33

34

17
 27-04-2025

35

36

18
 27-04-2025

37

38

19
 27-04-2025

39

40

20
 27-04-2025

41

42

21
 27-04-2025

43

44

22
 27-04-2025

45

46

23
 27-04-2025

47

48

24
 27-04-2025

49

50

25
 27-04-2025

51

52

26
 27-04-2025

53

54

27
 27-04-2025

55

56

28
 27-04-2025

57

58

29
 27-04-2025

59

60

30
 27-04-2025

61

62

31
 27-04-2025

63

64

32
 27-04-2025

65

66

33
 27-04-2025

67

34
 01-05-2025

1
 01-05-2025

2
 01-05-2025

3
 01-05-2025

4
 01-05-2025

10

5
 01-05-2025

11

12

6
 01-05-2025

13

14

7
 01-05-2025

15

16

8
 01-05-2025

17

18

9
 01-05-2025

19

20

10
 01-05-2025

21

22

11
 01-05-2025

23

24

12
 01-05-2025

25

26

13
 01-05-2025

27

28

14
 01-05-2025

29

30

15
 01-05-2025

31

32

16
 01-05-2025

33

34

17
 01-05-2025

35

36

18
 01-05-2025

37

38

19
 01-05-2025

39

40

20
 01-05-2025

41

42

21
 01-05-2025

43

44

22
 01-05-2025

45

46

23
 01-05-2025

47

48

24
 01-05-2025

49

50

25
 01-05-2025

51

52

26
 01-05-2025

53

54

27
 01-05-2025

55

56

28
 01-05-2025

57

58

29
 01-05-2025

59

60

30
 01-05-2025

61

62

31
 01-05-2025

63

64

32
 01-05-2025

65

66

33
 01-05-2025

67

68

34
 01-05-2025

69

70

35
 08-05-2025

Correlation and Regression Analysis

1
 08-05-2025

2
 08-05-2025

3
 08-05-2025

4
 08-05-2025

10

5
 08-05-2025

11

12

6
 08-05-2025

13

14

7
 08-05-2025

15

16

8
 08-05-2025

17

18

9
 08-05-2025

19

20

10
 08-05-2025

21

22

11
 08-05-2025

23

Significance Test for Correlation

 Hypotheses
H0: ρ = 0 (no correlation)
HA: ρ ≠ 0 (correlation exists)
r
 Test statistic t 
1  r 2 (with n – 2 degrees of freedom)
n  2

Here, the degree of freedom is taken to be n-2

because, two points can be joined by a straight line
surely

24

12
 08-05-2025

Example:

Is there evidence of a linear relationship between child

height and weight at the 0.05 level of significance?

H0: ρ = 0 (No correlation)

H1: ρ ≠ 0 (correlation exists)
 = 0.05 , df = 8 - 2 = 6

r .886
t    4.68
1 r 2
1  . 886 2
n2 82

25

Example:

At 5% level of significance, the critical value of tα/2, (n-2) = t.025, 6

=2.447.
The observed value > critical value;
So, we reject the null hypothesis.
That means correlation exists between height and weight at 5%
level of significance.

26

13
 08-05-2025

Regression Analysis
 Regression analysis is used to:
Predict the value of a dependent variable based on the value
of at least one independent variable.
Explain the impact of changes in an independent
variable on the dependent variable.
Controlling
 Dependent / Response variable: the variable we wish to explain.
in Simple linear regression it is always continuous variable
 Independent/explanatory variable: The variable used to
explain the dependent variable. In linear regression it could have
any measurement scale have any measurement scale.

27

Simple Linear Regression Model

 Only one independent variable, x

 Relationship between x and y is described

by a linear function

 Changes in y are assumed to be caused by

changes in x (E[y | x] )

28

14
 08-05-2025

Population Linear Regression

The population regression model:

Dependent Population Population Random
Variable Independent Error
y intercept Slope
Variable term, or
Coefficient
residual

y  β0  β1x  ε
Linear component Random Error
component

29

Linear Regression Assumptions

 The relationship between the two variables, x and y is Linear

 Independent observations

 Error values are Normally distributed for any given value of x

 The probability distribution of the errors has Equal variance

 Fixed independent variables (not random = non-stochastic = given

values = deterministic); the only randomness in the values of Y
comes from the error term 

 No autocorrelation of the errors (has some similarities with the 2nd)

 No outlier distortion

30

15
 08-05-2025

Assumptions viewed pictorially

LINE (Liner, Independent, Normal and Equal

variance) assumptions
Y

y|x= +  x

Identical normal
distributions of
errors, all centered on
~N(y|x y|x)
the regression line.

31

Population Linear Regression

y y  β0  β1x  ε
Observed
Value of y
for xi
εi Slope = β1
Predicted Random Error
Value of y for this x value
for xi

Intercept
= β0
xi x

32

16
 08-05-2025

Estimated Regression Model

The sample regression line provides an estimate of the

population regression line

Estimated Estimate of Estimate of the

(or predicted) the regression regression slope
y value intercept

Independent

ŷ i  b0  b1x variable

The individual random error terms ei have a mean of zero

33

Least Squares Criterion

e 2
 (y yˆ)2
 (y  (a+bx))2

34

17
 08-05-2025

Least Squares Regression

Given: xi x1 x2 …. xn
yi y1 y2 …. yn

The form of the function is assumed to be

known but the coefficients are unknown.

ei 2  ( yi  f ( xi )) 2  ( f ( xi )  yi ) 2
The difference is assumed to be the result of
experimental error.

35

Determine the Unknowns

We want to find 𝑎 and 𝑏 to minimize:

Φ(𝑎, 𝑏) = (𝑦 − (𝑎 + 𝑏𝑥 ))

How do we obtain 𝑎 and 𝑏 to

minimize: Φ(𝑎, 𝑏)?

36

18
 08-05-2025

Determine the Unknowns

Necessary condition for the minimum :

  ( a, b)
0
a
  ( a, b)
0
b

37

Determining the Unknowns

𝜕Φ(𝑎, 𝑏)
= 2 𝑦 − 𝑎 − 𝑏𝑥 (−1) = 0
𝜕𝑎

𝜕Φ(𝑎, 𝑏)
= 2 𝑦 − 𝑎 − 𝑏𝑥 (−𝑥 ) = 0
𝜕𝑏

38

19
 08-05-2025

Normal Equations

 n   n 
n a    xi  b    yi 
 i 1   i 1 
 n   n 2  n 
  xi  a    xi  b    xi y i 
     
 i 1   i 1   i 1 

39

Solving the Normal Equations

 n   n  n 
n   xi yi     xi    yi 
b   i 1   i 1   i 1 
2
 n 2  n 
n   xi     xi 
 i 1   i 1 

1   n   n 
a   y i   b   xi  
n   i 1   
 i 1  

40

20
 08-05-2025

Example 1: Linear Regression

Assume :
f(x)  a  bx x 1 2 3
y 5.1 5.9 6.3
Equations :
 n   n 
n a    xi  b    yi 
 
 i 1   i 1 
 n   n 2  n 
  xi  a    xi  b    xi y i 
     
 i 1   i 1   i 1 

41

Example 1: Linear Regression

i 1 2 3 sum
xi 1 2 3 6
yi 5.1 5.9 6.3 17.3
xi2 1 4 9 14
xi yi 5.1 11.8 18.9 35.8
Equations :
3 a  6 b  17.3
6a  14 b  35.8
Solving : a  4.5667 b  0.60

42

21
 08-05-2025

Regression equation of Y on X and X on Y

In the equation of y=a+bx;

The simplified formula of the two normal equations to get the
estimated values of a and b are
na+ b∑xi = ∑yi ….(1)
a∑xi + b∑xi2 = ∑xiyi …(2)

The estimated value of

∑ ̅ ( , )
𝑏= =
∑ ̅ ( )

43

Regression equation of Y on X and X on Y

na+ b∑xi = ∑yi ….(1)
a∑xi + b∑xi2 = ∑xiyi …(2)
Also, from (1), we get
a+bx = y
a=y - bx
Now, the regression equation of y on x is
Y= a+bx = y - bx + bx
( , )
y- y = b(x- x ), where b = ( )
b is called the regression coefficient of y on x. i.e., byx
( , )
Now, byx = ( )
= =r
So, the regression equation of y on x is y - y = byx(x - x) = r (x - x)

44

22
 08-05-2025

Regression equation of Y on X and X on Y

Similarly, the regression equation of x on y is
( , )
x- x = bxy(y- y ), where bxy =
( )
bxy is called the regression coefficient of x on y.
( , )
Now, bxy = = =r
( )

So, the regression equation of x on y is x - x = bxy(y - y ) = r (y - y )

45

Relationship between correlation coefficient and regression

coefficients
The regression equation of y on x is y - y = r (x - x)
And the regression equation of x on y is x - x = r (y - y )
r is the geometric mean between the two regression coefficients,
byx= r …(3)
bxy= r …(4)
Multiplying (3) and (4), we get
r2= byx x bxy
r =± byx x bxy .

46

23
 08-05-2025

Find the regression equations of both Y on X and X on Y from the data given
below. Hence find the correlation coefficient.

X 15 20 25 30 35
Y 12 14 18 25 31

Answer

47

X Y (X - 𝑥 ̅) (Y - y ̅) (X - 𝑥 ̅)2 (Y - y ̅)2 (X - 𝑥 ̅)(Y - y ̅)

15 12 -10 -8 100 64 80
20 14 -5 -6 25 36 30
25 18 0 -2 0 4 0
30 25 5 5 25 25 25
35 31 10 11 100 121 110

∑X = ∑Y= ∑(X - 𝑥 ̅)= ∑(Y - y ̅)= ∑(X - 𝑥 ̅)2= ∑(Y - y ̅)2 = ∑(X - 𝑥 ̅)(Y - y )̅ =
125 100 0 0 250 250 245

Now x = 125/5 = 25; y = 100/5 =20;

∑(X − 𝑥 ̅)(Y − y ̅)
byx = = = 0.98
∑(X − 𝑥 ̅)2

∑(X − 𝑥 ̅)(Y − y ̅)
bxy = = = 0.98
∑(Y − y ̅)2

48

24
 08-05-2025

Now x = 125/5 = 25; y = 100/5 =20;

∑(X − 𝑥 ̅)(Y − y ̅)
byx = = = 0.98
∑(X − 𝑥 ̅)2
∑(X − 𝑥 ̅)(Y − y ̅)
bxy = = = 0.98
∑(Y − y ̅)2
So, the regression equation of Y on X is y - y = byx(x - x)
y-20 = 0.98(x-25)
Y= - 4.5 + 0.98x
The regression equation of X on Y is x - x = bxy(y - y )
x-25 = 0.98 (y-20)
X= 5.4+ 0.98 y
So, r2= 0.98 x 0.98
r =± 0.98 x 0.98 = ± 0.98
Now, both regression coefficients are positive.
So, the correlation coefficient is positive, and it is 0.98.

49

Point of intersection between two regression lines

Let us assume that the regression equation of Y on X is Y=a+bX;
And the regression equation of X on Y is X=c+dY.

The intersection point of these regression lines is (𝑥̅ , 𝑦), that is, the mean. The point of
intersection of regression lines gives the mean. This is because the two lines coincide and
pass-through this common point. Therefore, this is the solution for both equations.

50

25
 08-05-2025

Interpretation of the Slope and the Intercept

 a is the estimated average value of y when the value of x is

zero (provided that x is inside the data range considered).
 Otherwise, it shows the portion of the variability of the
dependent variable left unexplained by the independent
variable considered

 b is the estimated change in the average value of y as a result of

a one-unit change in x

51

Example: Simple Linear Regression

 A researcher wishes to examine the relationship between

the amount of the daily average diets taken by a cohort of
20 sample children and the weight gained by them in one
month (both measured in kg). The content of the food is the
same for all of them.

 Dependent variable (y) = weight gained in one month

measured in kilogram

 Independent variable (x) = average weight of diet taken per

day by a child measured in Kilogram

52

26
 08-05-2025

Sample Data for child weight Model

Diet (x) Weight gained (y) Diet (x)

Weight gained (y)

0.4 0.65 0.86 1.1

0.46 0.66 0.89 1.12
0.55 0.63 0.91 1.20
0.56 0.73 0.93 1.32
0.65 0.78 0.96 1.33
0.67 0.76 0.98 1.35
0.78 0.72 1.02 1.42
0.79 0.84 1.04 1.1
0.80 0.87 1.08 1.5
0.83 0.97 1.11 1.3

53

Estimation using the computational formula

From the data we have:

Σx = 20.35, Σy = 16.27, Σxy = 17.58, Σx2 = 22.30
 xy   x y
b  n
(  x)2
 x2 
n

17.57  20.35*16.27 / 20
b  0.643
22.30  414.12 / 20

a  y  bx  0.81350.643*1.0175 0.160

54

27
 08-05-2025

Regression Using MS-Excel

Analyze/regression/linear….Coefficients

Coefficients Standard Error t Stat P-value

Intercept 0.159745041 0.077343133 2.0654069 0.053591

X Variable 1 0.642511017 0.073245577 8.772011112 6.44E-08

Weight gained = 0.16 +0.643(food weight)

55

Interpretation of the Intercept, a

Weight gained = 0.16 + 0.643(food weight)

 Here, no child has had 0 kilogram of food per day, so for

foods within the range of sizes observed, 0.16 Kg is the
portion of the weight gained not explained by food.

 Whereas b = 0.643 tells us that the average weight of a

child increases by 0.643, on average, for each additional
one kilogram of food taken each day

56

28
 08-05-2025

Explained and Unexplained Variation

Total variation is made up of two parts:

SST  SSR  SSE
Total sum of Sum of Squares Sum of Squares
Squares Regression Error

SST  (y  y)2 SSR  (yˆ  y)2 SSE  (y  ŷ)2

where:
y = Average value of the dependent variable
y = Observed values of the dependent variable
ŷ = Estimated value of y for the given x value

57

Explained and Unexplained …

y
yi  
SSE = (yi - yi )2 y
_
SST = (yi - y)2

Y  _
SSR =  (yi - y)2 _
_
y y

x
Xi

58

29
 08-05-2025

Coefficient of Determination, R2

 The coefficient of determination is the portion of the total

variation in the dependent variable that is explained by
variation in the independent variable

 The coefficient of determination is also called R-squared and

is denoted as R2

SSR s um of squares explained by regression

R2  
SST total s um of squares

where 0  R2 1

59

Coefficient of Determination, R2

In the single independent variable case, the coefficient

of determination is

R2  r2
Where:
R2 = Coefficient of determination
r = Simple correlation coefficient

60

30
 08-05-2025

Coefficient of Determination, R2 cont…

 The F-test tests the statistical significance of the

regression of the dependent variable on the
independent variable: H0: β = 0

 However, the reliability of the regression equation is very

commonly measured by the correlation coefficient R.

 Equivalently one can check the statistical significance of

R or R2 using F-test and can reach exactly the same F-
value as model coefficients’ test

61

Inference about the Slope: t-Test

 t test for a population slope
Is there a linear relationship between x and y?
 Null and alternative hypotheses
H0: β1 = 0 (no linear relationship)
H1: β1  0 (linear relationship does exist)
 Test statistic
b 1  β1
t  d.f.  n 2
– sb1
Where:
b1 = Sample regression slope (coefficient)
β1 = Hypothesized slope, usually 0

sb1 = Estimator of the standard error of the slope

62

31
 08-05-2025

Inference about the Slope :t Test

Estimated Regression Equation:

Weight gained = 0.16 +0.643(food)

 The slope of this model is 0.643

 Does weight of food taken per day affect

children’s weight?

 We have to test it statistically

63

Inferences about the Slope: t-Test Example

b1  0
b1 s b t 
1
sb1
Coefficients
Unstandardized
Coefficients
Model B Std. Error t Sig.
(Constant) 0.160 0.077 2.065 .054
Food weight 0.643 0.073 8.772 0.000

The calculated t-test is 8.772 Conclusion: There is

which is greater than the sufficient evidence that food
tabulated one 2.074 weight taken per day affects
Decision: Reject Ho children’s weight

64

32
 08-05-2025

65

Confidence Interval estimation

Confidence Interval Estimate of the Slope:
b 1  t  / 2 s b
df = n-2 = 18, t(0.025,18) = 2.101
1

Coefficients

Unstandardized Standardized 95% Confidence

Coefficients Coefficients Interval for B
Std. Lower Upper
Model B Error Beta t Sig. Bound Bound
(Constant) 0.160 0.077 2.065 .054 -0.003 0.322
Food weight 0.643 0.073 0.900 8.772 0.000 0.489 0.796

The 95% confidence interval for the slope is (0.489, 0.796).

Note also that this 95% confidence interval does not include 0.
Look the relationship between all the figures in the blue circles

66

33