MULTIPLE REGRESSION

Prepared by: Dr Norfariza Mohd Radzi

• Is a reliable method of identifying which variables have impact on a topic of interest. The
process of performing a regression allows you to confidently determine which factors matter
most, which factors can be ignored, and how these factors influence each other.

• In order to understand regression analysis fully, it’s essential to comprehend the following
terms:

Dependent Variable: This is the main factor that you’re trying to understand or predict.
Independent Variables: These are the factors that you hypothesize have an impact on your
dependent variable.
LINEAR REGRESSION

3
 LINEAR REGRESSION
⊳ It is used when we want to predict the value of a variable based on the value of another
variable.
⊳ The variable we want to predict is called the dependent variable (or sometimes, the outcome
variable). The variable we are using to predict the other variable's value is called the
independent variable (or sometimes, the predictor variable).
⊳ For example, you could use linear regression to understand whether exam performance can
be predicted based on revision time; whether cigarette consumption can be predicted based
on smoking duration; and so forth.
⊳ If you have two or more independent variables, rather than just one, you need to use
multiple regression.

4
 ASSUMPTIONS
⊳ The two variables should be measured at the continuous level (i.e., they are
either interval or ratio variables).
⊳ There needs to be a linear relationship between the two variables
⊳ There should be no significant outliers.
⊳ Independence of observations
⊳ Homoscedasticity of data which is where the variances along the line of best fit remain
similar as you move along the line.

⊳ The residuals (errors) of the regression line are approximately normally distributed.
5
 EXAMPLE
A salesperson for a large car brand wants to determine whether there
is a relationship between an individual's income and the price they pay
for a car. As such, the individual's "income" is the independent variable
and the "price" they pay for a car is the dependent variable. The
salesperson wants to use this information to determine which cars to
offer potential customers in new areas where average income is
known.

6
 RUNNING THE TEST
⊳ Click Analyze > Regression > Linear...
⊳ In Linear Regression dialogue box, transfer specific variables to
Dependent and Independent boxes.
⊳ Need to check four of the assumptions discussed in
the Assumptions section above by clicking Statistics and Plots and
select the appropriate options.
⊳ Click OK

7
 OUTPUT
⊳ SPSS Statistics will generate quite a few tables of output for a linear regression.
The three main tables are:

⊳ This table provides the R and R2 values. The R value represents the simple
correlation and is 0.873 (the "R" Column), which indicates a high degree of
correlation.
⊳ The R2 value (the "R Square" column) indicates how much of the total variation in
the dependent variable, Price, can be explained by the independent
variable, Income. In this case, 76.2% can be explained, which is very large. 8
 OUTPUT

This indicates the statistical significance of the regression model that was run. Here, p <
0.0005, which is less than 0.05, and indicates that, overall, the regression model statistically
significantly predicts the outcome variable (i.e., it is a good fit for the data).

9
 OUTPUT

The Coefficients table provides us with the necessary information to predict price from income, as
well as determine whether income contributes statistically significantly to the model.
Furthermore, we can use the values in the "B" column under the "Unstandardized Coefficients"
column, Price = 8287 + 0.564(Income)

10
MULTIPLE REGRESSION

11
 MULTIPLE REGRESSION
 Multiple regression is an extension of simple linear regression.
 Multiple regression also allows you to determine the overall fit (variance
explained) of the model and the relative contribution of each of the predictors to
the total variance explained.
 It is used when we want to predict the value of a variable based on the value of
two or more other variables.
 The variable we want to predict is called the dependent variable (or sometimes,
the outcome, target or criterion variable).
 The variables we are using to predict the value of the dependent variable are
called the independent variables (or sometimes, the predictor, explanatory or
regressor variables).
12
 MULTIPLE REGRESSION
⊳ For example, you could use multiple regression to understand whether exam
performance can be predicted based on revision time, test anxiety, lecture
attendance and gender.
⊳ Alternately, you could use multiple regression to understand whether daily
cigarette consumption can be predicted based on smoking duration, age when
started smoking, smoker type, income and gender.

13
 ASSUMPTIONS
⊳ Dependent variable should be measured on a continuous scale (i.e., it is either
an interval or ratio variable).
⊳ Two or more independent variables, which can be either continuous (i.e.,
an interval or ratio variable) or categorical (i.e., an ordinal or nominal variable).
⊳ Independence of observations
⊳ There needs to be a linear relationship between (a) the dependent variable and each of
the independent variables, and (b) the dependent variable and the independent
variables collectively.
⊳ Homoscedasticity of data
⊳ Data must not show multicollinearity, which occurs when two or more independent
variables are highly correlated with each other.
⊳ No significant outliers, high leverage points or highly influential points
⊳ Residuals (errors) are approximately normally distributed 14
 EXAMPLES
A health researcher wants to be able to predict "VO2max", an indicator
of fitness and health. This can put off those individuals who are not
very active/fit and those individuals who might be at higher risk of ill
health. To this end, a researcher recruited 100 participants to perform a
maximum VO2max test, but also recorded their "age", "weight", "heart
rate" and "gender". The researcher's goal is to be able to predict
VO2max based on these four attributes.

15
 RUNNING THE TEST
⊳ Click Analyze > Regression > Linear...
⊳ In Linear Regression dialogue box, transfer specific variables to Dependent
and Independent boxes.
⊳ In Method box, choose Stepwise.
⊳ Click Statistics and check for Estimates and Model fit in the Linear regression:
Statistics dialogue box. Check for Confidence intervals too and leaving the
Level(%) at ‘95’. Click Continue and OK

16
 OUTPUT – MODEL SUMMARY
 The "R" column represents the value of R, the multiple correlation coefficient. R can be
considered to be one measure of the quality of the prediction of the dependent variable.
 The "R Square" column represents the R2 value (also called the coefficient of determination),
which is the proportion of variance in the dependent variable that can be explained by the
independent variables. The independent variables explain 57.7% of the variability of our
dependent variable.

17
 OUTPUT – STATISTICAL SIGNIFICANCE
 The F-ratio in the ANOVA table (see below) tests whether the overall regression
model is a good fit for the data. The table shows that the independent variables
statistically significantly predict the dependent variable, F(4, 95) = 32.393, p < .0005
(i.e., the regression model is a good fit of the data).

18
 OUTPUT – ESTIMATED MODEL COEFFICIENTS
 Unstandardized coefficients indicate how much the dependent variable varies
with an independent variable when all other independent variables are held
constant.
 Consider the effect of age in this example. The unstandardized coefficient, B1,
for age is equal to -0.165 (see Coefficients table). This means that for each one
year increase in age, there is a decrease in VO2max of 0.165 ml/min/kg.

19
 OUTPUT – STATISTICAL SIGNIFICANCE OF THE
INDEPENDENT VARIABLES
 We can see from the "Sig." column that all independent variable coefficients are
statistically significantly different from 0 (zero). Although the intercept, B0, is
tested for statistical significance, this is rarely an important or interesting finding.

20
 DECISION AND CONCLUSION
A multiple regression was run to predict VO2max from gender, age, weight and heart
rate. These variables statistically significantly predicted VO2max, F(4, 95) = 32.393, p <
.0005, R2 = .577. All four variables added statistically significantly to the
prediction, p < .05.

21