Regression
DR. REEM SAEED ALGHAMDI
Example
Example
• A researcher is interested in examining the
effects of four independent variables
simultaneously on the dependent variable
(the overall variance explained by the model).
• Also, the researcher is interested in
examining the unique contribution (strength
and direction) of each independent variable
on the dependent variable.
Similar to the previous example:
Subjective A researcher is interested in examining the
norms
effects of three independent variables on the
Intention dependent variable and the unique contribution
Self percieved of
behaviors behavior of each independent variable to the dependent
variable.
Attitudes
Building model
The conceptual models illuminate how some factors (i.e., the independent
variables) are related to the outcome variable (i.e., the dependent variable)
This model requires multivariate statistical analysis
Regression model is used to determine the strength and direction of an
association between two variables after adjusting for the influence
of other variables
There are various types of regression
linear regression
logistic regression
Regression
Regression is a multivariate analysis
Regression allows researchers to consider the effects of several
independent variables on one dependent variable in the same analysis.
Regression is important because:
It allows considering the fact that health outcomes are usually affected
by more than one variable (e.g., safety outcome is predicted by nurses’
perception of safety culture, number of staff, and supportive manager)
Regression
Itdetermines which factors are the most important or more strongly
associated with the outcome (e.g., the most influential factor of safety
outcome is perception of safety culture).
It discovers spurious results in a bivariate relation. (e.g., there is a
significant correlation between going to the gym and happiness.
However, physical health status could be associated with both. It means
that physical health leads to going to the gym and being happier. If we
control for physical health, the bivariate might not be significant.
Example
Body temperature
Heart rate
Anxiety level
Linear Regression
Linear regression describes the linear relationship between a dependent
variable (in a ratio scale) and one or more independent variables
Regression analysis gives:
(1)the overall variance explained by the model, which is called the
coefficient of determination (R2). It measures the percentage of variance
that the model explains.
(2) the unique contribution (strength and direction) of each independent
variable by using regression coefficients.
Assumptions of linear regression
It is a parametric test
The sample must be representative of the population
The dependent variable is roughly normally distributed.
Homoscedasticity (for every value of X, the distribution of Y scores must
have approximately equal variability).
The relationship between X and Y must be linear
There is no multicollinearity
Multicollinearity
Multicollinearity is a problem that can occur when predictors are too
highly intercorrelated
Can yield unstable and misleading regression results
Avoid using two predictors whose correlation is .85 or higher
Multicollinearity can be tested by computing a tolerance, which ranges
from .0 to 1.0
The higher the tolerance, the better; default tolerance for excluding variable in SPSS
= .0001
Multiple Regression Assumptions
Multiple regression used inferentially to estimate population values relies on
several assumptions
Multivariate normality—Each variable and all linear combinations of
them are assumed to be normally distributed
Linearity—That there is a straight-line relationship between pairs of
variables
Homoscedasticity—Variability in scores for one variable is similar at all
values of another variable.
Regression Assumptions (cont’d)
Independence of errors—Errors of prediction are assumed to be
independent of each other.
Main tool for exploring violations of assumptions: Residual scatterplots
that plot errors of prediction on one axis against predicted values of the
DV on the other.
Homogeneity of
Regression
Illustration
Residual
Scatterplots
When assumptions for
multiple regression are met,
residuals are distributed in an
approximate rectangle, with
heavy clustering of residuals
along a center line as in this
diagram
Residual
Scatterplots
Residual scatterplot when
the assumption of
multivariate normality is
violated →
Distribution of residuals is
skewed
Residual
Scatterplots
Residual scatterplot when the
assumption of linearity is
violated →
The relationship between
residuals and predicted values
of Y is not linear
Residual
Scatterplots
Residual scatterplot when
the assumption of
homoscedasticity is violated
→
Variation in error terms is
not consistent across all
values of Y’
Examples using SPSS
Researchers are interested in looking at the factors that predict
children's reading ability. Based on existing literature, the researcher
hypothesized that age, memory span, and IQ.
Note that N=20
Adjusted R-squared is 0.804
Explaining 80.4% of the
variance in reading ability
Our model significantly predicts
outcome (reading ability)
Interpretation
Only age is a significant predictor.
It indicates that as age increases by
one unit, the reading ability increases
by 0.47 units, controlling for IQ and
short-term memory span
(unstandardized B).
It also indicates that as age increases
by one standard deviation, the
reading ability increases by 0.73
standard deviation, controlling for IQ
and short-term memory span
(standardized Beta).
Results
Multiple regression was conducted to determine the best linear
combination of age, memory span, and IQ for predicting students’ reading
ability.
This combination of variables significantly predicted reading ability, F(3,16) =
26.98,p < .05, with only age variable significantly contributing to the
prediction. The adjusted R-squared value was .804. This indicates that the
model explained 80.4% of the variance in reading ability.
Logistic regression (logit analysis) analyzes the
relationship between one or more predictor
variables and a categorical dependent variable:
Binary logistic regression, used when the outcome
Logistic is dichotomous (e.g., sepsis, absence of sepsis)
Regression
Multinomial logistic regression, used when the
outcome has three or more categories (e.g., live
birth, miscarriage, abortion)
In logistic regression, predictors can
be:
Continuous variables (e.g., age)
Predictor
Dichotomous variables (e.g., sex)
Variables
Categorical variables (e.g., employment
status: not working (1), working part
time (2), working full time (3)
Logistic regression
Logistic regression is used when the dependent variable is dichotomous
(two categories) and the independent variables are on any measurement
scale.
Adjusted odds ratio is the central part of logistic regression
The odds ratio measures how much more likely (or less likely) an
outcome is to be present given certain conditions or exposures
Example of logistic regression
Isthere a combination of memory span, IQ test, and reading
ability in predicting whether students will succeed in the
preparation year?
The dependent variable is dichotomous (dummy variable):
success/not success
Success is the dependent outcome
variable and is coded 0 or 1.
Only the constant is in the equation.
The overall model is significant when all three
independent variables are entered.
These are similar to R2 and give a
rough estimate of the variance that can be
predicted from the combination of the four
variables. The Cox & Snell R is usually an
underestimate.
IQ is significant
predictor for
success
Exp(B) presents the odds
ratios.
Results
The Omnibus Tests of Model Coefficients table indicates that, when we consider all
three predictors together, the model or equation χ2=13.00, df=43, N=20, p<.05).
The result suggests that IQ significantly predicted students’ success. The odds ratio
and 95% confidence interval for the IQ test were 1.43 (95% CI: 1.06-1.93). It means
that as IQ increases by one unit, the change in the odds of success is 1.43.
Questions and discussion