Dependent Independent
Variable Variable
Last Week T-Test Continuous Categorical
Correlation
and Continuous Continuous
This Week Regression
Chi-Square Categorical Categorical
General Categorical Categorical
Next Week Linear or or
Model Continuous Continuous
Association Between Two Variables
• No association
• Linear association
– Positive association
– Negative association
• Curvilinear association
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Strength of Linear Association
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Strength of Linear Association
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Strength of Linear Association
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Quantifying the Strength
of Linear Correlation
• What does a positive Student SAT GPA
linear correlation # (X) (Y)
mean?
1 450 2.7
– Large numbers on one
variable go with large 2 520 3.1
numbers on the other
variable. 3 600 3.5
• How to decide what 4 470 2.6
are large and small 5 460 3.1
numbers? µ 500 3.0
– Relative to the means.
σ 55.5 0.32
3.6
3.4
3.2
3
350 400 450 500 550 600 650
2.8
2.6
2.4
Student SAT GPA X – µ X Y – µ Y (X – µ X)(Y – µ Y)
# (X) (Y)
1 450 2.7 -50 -0.3 15
2 520 3.1 20 0.1 2
3 600 3.5 100 0.5 50
4 470 2.6 -30 -0.4 12
5 460 3.1 -40 0.1 -4
Sum 2500 15.0 0 0 75 (Cross Product)
µ 500 3.0 0 0 15 (Covariance)
σ 55.5 0.32
Quantifying the Strength
of Linear Correlation
• Is 15 a large or smaller number?
• At least we know it is positive.
• Magnitude relative to the variance (or standard
deviation) of X and Y.
Co var iance σ XY
r= =
σ X ⋅σ Y σ X ⋅σ Y
• r = 15 / (55.5 x 0.32) = 0.84
Alternative Approach
• Standardize X and Y first (z-scores), then
calculate the covariance between the z-
scores.
r=
∑z X ⋅ zY
N
Student SAT GPA zX zY zX zY
# (X) (Y)
1 450 2.7 -0.90 -0.93 0.84
2 520 3.1 0.36 0.31 0.11
3 600 3.5 1.80 1.55 2.79
4 470 2.6 -0.54 -1.24 0.67
5 460 3.1 -0.72 0.31 -0.22
Sum 2500 15.0 0 0 4.19
µ 500 3.0 0 0 0.84 r
σ 55.5 0.32
Interpreting the Magnitude of
Correlations
• Always between -1 and +1
• Proportion of variance explained by the
other variable: r2
• r = .84, r2 = .71 = 71%
• A correlation of .8 is NOT two times
stronger than a correlation of .4.
– How much stronger?
– 4 times. (.8)2 = .64; (.4)2 = .16
Significance Testing
• The following has a t distribution:
r N −2
t=
1− r2
df = N – 2
r = .84, t = 2.68, df = 3, p = .075
Not significant at .05 level. Small sample size.
When There’s a Significant
Correlation
• Correlation and Causation
• X causes Y
• Y causes X
• Z causes both X and Y
When There’s No Significant
Correlation
• Small sample
• Other Noise
• Attenuation due to unreliability of
measurement
• Outliers
• Restriction in range
• Curvilinearity
From Correlation to Regression
• Correlation: to describe the relationship
between two variables
• Regression: to use one variable to predict
another variable
• The accuracy of prediction depends on the
strength of correlation
Strength of Linear Association
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
An Example
• Research Question: Does eating spinach
increase strength?
• Randomly sampled 20 individuals.
• IV: How many cans of spinach one consumed
in the past week.
• DV: How many push-ups one can do in a
minute.
70
60 r = .86
50
40
Pushup
30
20
10
0
0 5 10 15 20 25
Spinach
Coefficientsa
Unstandardized Standardized
Coefficients Coefficients
Model B Std. Error Beta t Sig.
1 (Constant) 19.443 3.494 5.565 .000
spinach 1.550 .220 .856 7.031 .000
a. Dependent Variable: pushup
Yˆ =19.48+1.55X
)
zY = (.856) z X
Understanding R2:
Proportion of Variance Explained, or
Proportion Reduction in Error
70
60
50
40
Pushup
30
20
10
0
0 5 10 15 20 25
Spinach
70
60
50
40
Pushup
30
20 When you don’t know X,
you can only use the mean
10 of Y to predict the Y score
of any individual.
0
0 5 10 15 20 25
Spinach
70
60
50
40
Pushup
30
20 Errors (or variance) are relatively
high when you use the mean of
10 Y as your prediction.
0
0 5 10 15 20 25
Spinach
70
60
50
40
Pushup
30
20
10
0
0 5 10 15 20 25
Spinach
70
60
50
40
Pushup
30
20 When you know X, and use X to
predict Y, the errors become
10 smaller.
0
0 5 10 15 20 25
Spinach
70
60
50
40
Pushup
30
Green
20
)
R2 =
∑ (Y − Y ) 2
10
∑ (Y − Y ) 2
Green and
Red
0
0 5 10 15 20 25
Spinach
# of push-ups
Spinach consumption
Association Between Two
Categorical Variables
• Angelina Jolie or Jennifer Aniston?
Test for Independence
• Null Hypothesis: There is no relationship
between JA/AJ preference and which side
you are sitting in the classroom.
• To rephrase: JA/AJ preference does not
depend on which side you are sitting in the
classroom.
• Another version: People sitting on the right
and people sitting on the left do not have
different JA/AJ preferences.
JA AJ Total
Observed
Left Expected
Right
Total
Expected Frequency
• Expected assuming the null hypothesis is
true, i.e., no association between the two
variables.
C⋅R
Expected =
N
• C: Column total, R: Row total,
N: Grand total
Chi-Square
(Observed − Expected ) 2
χ =∑
2
Expected
• Degree of Freedom
df = (# of Columns – 1)(# of Rows – 1)
• What is the df for a 2 x 2 table?
• The shape of Chi-Square distribution
depends on the degree of freedom
Chi-Square Distribution
Critical Region
Chi-Square
• The chi-square statistic is always positive.
Why?
• When df = 1, chi-square distribution is the
distribution of z2.
• Without looking up in a reference, what is
the alpha = .05 cutoff value for the chi-
square distribution (df = 1)?
– (1.96)2 = 3.84
Back to Angelina and Jennifer
• In SPSS.
If We Still Have Time…
Chi-Square Test
for Goodness of Fit
To test whether a distribution is
the same as a predetermined or
theoretical distribution.
Next Week
• Integrating t-test, correlation, regression,
and chi-square test for independence
• They are all special cases of the general
linear model
• Effect size and power for the above tests