Chapter 9
Regression
Analysis
University of Economics
Ho Chi Minh City
Dang Van Thac
Outline
•Correlation Analysis
•Simple Regression Analysis
•Multiple Regression Analysis
Dang Van Thac
Correlation Analysis
• Correlation analysis is used to measure the relationship between two
variables.
• We use r to denote the correlation coefficients between variables:
(𝑥𝑖 − 𝑥 )(𝑦𝑖 − 𝑦)
𝑟=
(𝑥𝑖 − 𝑥 )2 (𝑦𝑖 −𝑦)2
• r ranges from -1 to 1 (r=0 indicates no relation, r>0 indicates a positive
relation, r<0 indicates a negative relation.）
• The relationship between variables is “correlated”
Simple Regression Analysis
• Regression analysis: is a statistical methodology that analyzes the
statistical relation between two or more variables so that a dependent
variable can be predicted from other independent variables.
• Example 1: Sales of a product can be predicted by the amount of advertising
expenditures.
• Example 2: The performance of an employee on a job can be predicted by his or
her work attitudes.
• Example 3: The income of a person can be predicted by his or her level of
education.
• Example 4: The performance of a firm can be predicted by its product innovation.
• Example 5: The performance of a student can be predicted by his or her diligence.
Dang Van Thac
Simple Regression Analysis
• Simple regression model: 𝑌𝑖 = 𝛽0 + 𝛽1 𝑋𝑖 + 𝜀𝑖
𝒀𝒊 is the dependent variable in the ith trial (outcome variable, response variable).
𝑿𝒊 is the independent variable in the ith trial (predictor variable, exploratory
variable).
𝜷𝟎 and 𝜷𝟏 are parameters. 𝜷𝟎 is the intercept or constant term (when x=0, it
gives the mean of the probability distribution of Y at X=0, When x is not equal to
0, it does not have any meaning), 𝜷𝟏 is the slope or the change in the mean of
the probability distribution of Y per unit increase in X.
𝜺𝒊 is a random error term.
This model is said to be simple, linear in the parameters, and linear in the
predictor variables.
Dang Van Thac
Simple Regression Analysis
Sampling
Population Sample
Inference
𝒀𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝒊 + 𝜺𝒊 𝒀𝒊 = 𝒃𝟎 + 𝒃𝟏 𝑿𝒊
Y Y
X X
Dang Van Thac
Simple Regression Analysis
• Estimation of Regression Function:
𝒀𝒊 = 𝒃𝟎 + 𝒃𝟏 𝑿𝒊
𝑋𝑖 −𝑋 𝑌𝑖 −𝑌
• Method of Least Squares: 𝑏1 = , 𝑏0 = 𝑌 − 𝑏1 𝑋
𝑋𝑖 −𝑋 2
• Analysis of Variance:
Source of variance Sum of squares df Mean Square F
Regression 2 1 𝑀𝑆𝑅 = 𝑆𝑆𝑅/1 F= 𝑀𝑆𝑅/𝑀𝑆𝐸
𝑆𝑆𝑅 = 𝑌𝑖 − 𝑌
Error 2 n-2 𝑀𝑆𝐸 = 𝑆𝑆𝐸/(𝑛 − 2)
𝑆𝑆𝐸 = 𝑌𝑖 − 𝑌𝑖
Total 2 n-1
𝑆𝑆𝑇𝑂 = 𝑌𝑖 − 𝑌
Dang Van Thac
Simple Regression Analysis
1 𝑋2
• Variance and standard error of 𝒃𝟎 : 𝑠2 𝑏0 = 𝑀𝑆𝐸 + , 𝑆{𝑏𝑜 } = 𝑆 2 {𝑏0 }
𝑛 𝑋𝑖 −𝑋 2
• Test concerning 𝒃𝟎 : 𝑡 = 𝑏0 /𝑠{𝑏0 }
𝑀𝑆𝐸
• Variance and standard error of 𝒃𝟏 : 𝑠2 𝑏1 = , 𝑆{𝑏1 } = 𝑆 2 {𝑏1 }
𝑋𝑖 −𝑋 2
• Test concerning 𝒃𝟏 : 𝑡 = 𝑏1 /𝑠{𝑏1 }
𝑆𝑋
• Standardized coefficients 𝒃𝟏∗ : 𝑏1∗ = 𝑏1
𝑆𝑌
• R square: is the proportionate reduction of total variance of Y associated with
2 SSR 𝑆𝑆𝐸
the use of the predictor variable X. （𝑅 = =1 − )
𝑆𝑆𝑇𝑂 𝑆𝑆𝑇𝑂
1−𝑅 2 𝑛−1
• Adjusted R square: 𝐴𝑑𝑗𝑢𝑠𝑡𝑒𝑑 𝑅2 = 1 − ( ) (n is the sample size,
𝑛−𝑝−1
p is the number of independent variables in the model.
Dang Van Thac
Simple Regression Analysis
• Example: A researcher wanted to determine the
relationship between consumers’ perceived healthiness
and their purchase intention toward organic drinking
products. The researcher conducted a study to select
randomly 60 consumers at different organic drinking
stores. He used simple linear regression analysis to test
the proposed relationship.
• Demonstration by using SPSS to analyze the data of this
example.
Dang Van Thac
Multiple Regression Analysis
• Multiple regression analysis is a statistical technique that can be used
to analyze the relationship between a single dependent variable and
several independent variables.
• Example 1: A worker’s performance is determined by his or her skills, knowledge,
attitudes, and emotions.
• Example 2: The relationship between a firms’ financial performance and marketing
ability, human resource management, and product innovation.
• Example 3: In a medical study of short children, the response variable was the peak
plasma growth hormone level. There were 14 predictor variables, including age,
gender, height, weight, and 10 skinfold measurements.
• Example 4: An educator wants to know whether students’ English ability is
predicted by age, intelligence, family support, school support, and social
environment. Dang Van Thac
Multiple Regression Analysis
• Two predictor variables:
𝒀𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝟏 + 𝜷𝟐 𝑿𝟐 + 𝜺𝒊 𝒀𝒊 = 𝒃𝟎 + 𝒃𝟏 𝑿𝟏 + 𝒃𝟐 𝑿𝟐
• More than two predictor variables
𝒀𝒊 = 𝜷𝟎 + 𝜷𝟏 𝑿𝟏 + 𝜷𝟐 𝑿𝟐 + ⋯ + 𝜺𝒊 𝒀𝒊 = 𝒃𝟎 + 𝒃𝟏 𝑿𝟏 + 𝒃𝟐 𝑿𝟐 +…
• The dependent variable Y is always a quantitative variable.
The independent variable 𝑋𝑖 can be either quantitative or qualitative
variable.
Dang Van Thac
Multiple Regression Analysis
• Multicollinearity problem occurs when the
independent variables are highly correlated.
• We can detect and test the multicollinearity
problem by using:
(1)The correlation coefficient （r greater than 0.5)
(2)The variance inflation factor (VIF) (VIF<3)
Multiple Regression Analysis
• Qualitative independent variables in regression: we use
dummy variable to deal with qualitative independent variables.
Example:
If the independent variable is gender (with male and female):
Coding: 𝐷𝑖 = 0 𝑖𝑓 𝑎 𝑚𝑎𝑙𝑒 𝑟𝑒𝑠𝑝𝑜𝑛𝑑𝑒𝑛𝑡
= 1 𝑖𝑓 𝑎 𝑓𝑒𝑚𝑎𝑙𝑒 𝑟𝑒𝑠𝑝𝑜𝑛𝑑𝑒𝑛𝑡
Regression model: 𝑌𝑖 = 𝛽0 + 𝛽1 𝐷𝑖 + 𝜀𝑖
• Interpretation: the average difference in the dependent variable
between those with 1 and those with 0 values of that dummy
variable, holding other independent variables constant.
Multiple Regression Analysis
• Illustrative example: A researcher wanted to determine
the relationship between consumers’ perceived
healthiness, extrinsic motivation and their purchase
intention toward organic drinking products. The
researcher conducted a study to select randomly 60
consumers at different organic drinking stores. He used
multiple linear regression analysis to test the proposed
relationship.
• Using regression analysis to analyze the data of this
example.
Dang Van Thac
Examples and Practice