Social Statistics

Lecture 5. Comparing Two Groups

Course instructor: Dr. Han, Sinn Won

SOC2103-01 [2025 1st Semester] 1

Any questions about the KGSS data?

SOC2103-01 [2025 1st Semester] 2

Quick Summary
• Confidence intervals and significance (hypothesis) tests are core tools
in inferential statistics –they serve slightly different purposes
• Both are used to draw conclusions about a population based on sample
data → Based on sampling distributions, use SE, and rely on the same
assumptions (i.e., independent sample, random sample, normality and large sample size)

• Key differences:
• CIs estimate a range of plausible values for a parameter
• The outcome is an interval (e.g., (2.5, 4.2)) → If the interval does not contain the
value under the null hypothesis, it suggests a statistically significant difference
• In contrast, significant tests test a claim (hypothesis) about a population parameter
• The outcome is a decision → If the p-value is less than alpha, reject the null
hypothesis
SOC2103-01 [2025 1st Semester] 3
Goals for this lecture
• Develop skills for constructing confidence intervals and
conducting significance tests
• To compare the population means or proportion of two
independent groups (the two samples must come from separate, unrelated groups)
• Example:
• Comparing the average test scores of students from two different schools

• To assess the average difference of matched pairs

• Example:
• Measuring a group’s blood pressure before and after a treatment (same
people measured twice)

SOC2103-01 [2025 1st Semester] 4

Moving from One Sample to Two
• In the last lecture, we learned how to conduct significance
tests that a population parameter (proportion or mean)
takes a particular value
• Examples: 𝐻0 : 𝜇 = 65; 𝐻0 : 𝑝 = 0.5
• Sometimes we have sample from two different populations
and want to test a hypothesis about the difference
between the population paramaters in the two populations
• Most commonly, we want to test 𝐻0 : 𝜇1 = 𝜇2 ; 𝐻0 : 𝑝1 = 𝑝2
• Notice this is the same as testing
𝐻0 : 𝜇1 − 𝜇2 = 0 or 𝐻0 : 𝑝1 − 𝑝2 = 0
SOC2103-01 [2025 1st Semester] 5
Comparing Two Proportions
• Suppose we have independent samples from two
populations, and we want to test 𝐻0 : 𝑝1 − 𝑝2 = 0
• Based on our sample, what is our best guess of the true
difference in the population proportions?
• 𝑝Ƹ 1 − 𝑝Ƹ 2
• In the one-sample case, 𝑝ො has a sampling distribution. In
the two-sample case, 𝑝ො1 − 𝑝ො2 has a sampling distribution.
• If you were to repeat your study of two groups many times, with
the same sample sizes, you would get a variety of differences
between the sample proportions
SOC2103-01 [2025 1st Semester] 6
CIs for the Difference between Two Proportions
ො
𝑝(1− ො
𝑝)
• Recall: With one sample, the CI is: 𝑝Ƹ ± 𝑧 ∗ ∗
𝑛
• In other words: sample statistic ±𝑧 ∗ (standard error)
• We use the same logic here: 𝑝Ƹ 1 − 𝑝Ƹ 2 ± 𝑧 ∗ ∗ 𝑆𝐸(𝑝Ƹ 1 − 𝑝Ƹ 2)
• Standard Error (SE) Formula:

∗
𝑝Ƹ 1(1 − 𝑝Ƹ 1) 𝑝Ƹ 2(1 − 𝑝Ƹ 2)
𝑝Ƹ 1 − 𝑝Ƹ 2 ± 𝑧 ∗ +
𝑛1 𝑛2

• Notice 𝑆𝐸 𝑝Ƹ 1 − 𝑝Ƹ 2 = 𝑆𝐸 𝑝Ƹ 1 2 + 𝑆𝐸 𝑝Ƹ 2 2

• Why do we add the standard errors (squared) rather than subtracting

them?
SOC2103-01 [2025 1st Semester] 7
CIs for the Difference between Two Proportions
• Why do we add the standard errors (squared) rather than subtracting
them when calculating the standard error for the difference between two
proportions?
• When you subtract two estimates, 𝑝Ƹ 1 − 𝑝Ƹ 2, the uncertainty (variability) in each
estimate doesn’t cancel out – it combines (if the samples are independent)
• We subtract proportions (because we care about the difference)
• But we add their variances (because each sample contributes uncertainty
independently, and we care about total uncertainty)

• Analogy: Imagine you’re measuring two things with separate rulers, an each with
little wiggle (uncertainty). Even if you’re subtracting the lengths, each ruler's wiggles
(variability) still affect the result – and they add up!

SOC2103-01 [2025 1st Semester] 8

Model Assumptions
• Under what circumstances, is it appropriate to use this formula?
• In the one sample case, we assumed a random sample from the
population
• Here, we assume the samples drawn from both populations are random and
independent
• Generally, samples from two different groups are assumed to be independent, unless
there’s a natural or deliberate link between individuals in the two groups (e.g., pairing, or
shared background that connects them)

• In the one sample case, we required at least 15 successes and 15

failures
• Here, we require at least 10 successes and 10 failures in each sample
• This ensures approximate normality (정규분포)

SOC2103-01 [2025 1st Semester] 9

Example: Nationality and Voting
• You are interested in whether Japanese or Koreans are more likely to
vote. You randomly sample 200 Japanese and 100 Koreans. In your
samples, 140 Japanese and 50 Koreans report having voted in the
last election. Construct a 95% CI for the Japanese voting rate
advantage over the Korean rate.
∗
𝑝Ƹ 1(1 − 𝑝Ƹ 1) 𝑝Ƹ 2(1 − 𝑝Ƹ 2)
𝑝Ƹ 1 − 𝑝Ƹ 2 ± 𝑧 ∗ +
𝑛1 𝑛2

0.7(1 − 0.7) 0.5(1 − 0.5)

0.7 − 0.5 ± 1.96 +
200 100

0.2 ± 1.96 0.060 = (𝟎. 𝟎𝟖, 𝟎. 𝟑𝟐)

SOC2103-01 [2025 1st Semester] 10
Interpreting the CI
• If we were to repeat the study many times, 95% of the appropriately
constructed CIs would contain the true difference in the population
proportions
• This doesn’t say anything about the individual population proportions
• In our example, the CI is entirely positive numbers: the range of
plausible values suggests that Japanese vote at higher rates than
Koreans
• What if I asked for the CI for the Korean advantage in voting rate?
• We just flip the sign: (−𝟎. 𝟎𝟖, −𝟎. 𝟑𝟐)
• It doesn’t matter which population you assign as first and which as second,
provided you correctly interpret the result

SOC2103-01 [2025 1st Semester] 11

Significance Tests for Two Proportions
• Significance tests follow a similar logic in two independent samples
as the logic in one sample
• 𝐻0 : 𝑝1 − 𝑝2 = 0
• 𝐻𝑎 : 𝑝1 − 𝑝2 > 0 or 𝐻𝑎 : 𝑝1 − 𝑝2 < 0 or 𝐻𝑎 : 𝑝1 − 𝑝2 ≠ 0
ො 0
𝑝−𝑝
• In one large sample 𝑧 =
𝑆𝐸0
sample statistic − value under H0
• In other words, 𝑧 =
standard error under H0
• Recall: this uses the mean and standard error of the sampling distribution
assuming 𝐻0 is true
𝑝ො1−𝑝ො2 −0
• In two samples: 𝑧 = 𝑆𝐸 (𝑝ො −𝑝ො )
0 1 2
SOC2103-01 [2025 1st Semester] 12
Finding 𝑆𝐸0 (𝑝Ƹ 1 − 𝑝Ƹ 2)
• Our general formula for the standard error of the difference between
two independent sample proportions is
𝑝Ƹ 1(1 − 𝑝Ƹ 1) 𝑝Ƹ 2(1 − 𝑝Ƹ 2)
𝑆𝐸 = +
𝑛1 𝑛2

• Remember that 𝑆𝐸0 is the standard error if 𝐻0 is true

• If 𝐻0 is true, 𝑝1 = 𝑝2 , so we have a single proportion
• We estimate this common 𝑝 by pooling information from both samples

𝑝ො1 𝑛 +𝑝ො2 𝑛 total number of successes in two samples

The pooled proportion 𝑝Ƹ = 1 2 =
𝑛1 +𝑛2 total sample size in two samples

𝑝Ƹ 1 − 𝑝Ƹ 𝑝Ƹ 1 − 𝑝Ƹ 1 1
𝑆𝐸0 𝑝Ƹ 1 − 𝑝Ƹ 2 = + = 𝑝Ƹ 1(1 − 𝑝Ƹ 1) +
𝑛1 𝑛2 𝑛1 𝑛2
SOC2103-01 [2025 1st Semester] 13
Model Assumptions
• Under what circumstances, is it appropriate to use this formula?
• Again, assume the samples drawn from both populations are
random and independent
• We require at least 10 successes and 10 failures in each sample, if
we are doing a two-sided test

SOC2103-01 [2025 1st Semester] 14

Example: Nationality and Voting
• Assess whether there is sufficient evidence to conclude at the 0.01
significance level that Japanese and Koreans have different voting rates
• Recall: 𝑝Ƹ 1 = 0.7, 𝑛1 = 200, 𝑝Ƹ 2 = 0.5, 𝑛2 = 100
• Step 1: Write down the hypotheses
• 𝐻0 : 𝑝1 − 𝑝2 = 0
• 𝐻𝑎 : 𝑝1 − 𝑝2 ≠ 0
• Step 2: Compute the test statistic (z)
140+50
• Find the pooled proportion (since 𝐻0 assumes no difference) → 𝑝:Ƹ = 𝟎. 𝟔𝟑
200+100

𝑝Ƹ 1 − 𝑝Ƹ 2 − 0 0.7 − 0.5 − 0 0.2

𝑍= = = = 𝟑. 𝟑𝟗
1 1 1 1 0.059
𝑝(1
Ƹ − 𝑝)Ƹ + 0.63(1 − 0.63) +
𝑛1 𝑛2 200 100
SOC2103-01 [2025 1st Semester] 15
Example: Nationality and Voting (cont.)
• Step 3: Draw conclusions
• Find the p-value
𝑃 𝑍 < −3.39 + 𝑃 𝑍 > 3.39 = 2𝑃(𝑍 < −3.39)
2 ∗ 0.00035 = 0.007
• Is the p-value less than the significance level?
• Yes!
• We can reject 𝑯𝟎 that Japanese and
Koreans have the same voting rate in
favor of 𝑯𝒂 that they have different
voting rates

SOC2103-01 [2025 1st Semester] 16

Homework: COVID Testing
• Suppose in January a random sample of 100 Yonsei students was
drawn and 15 of them said they had been tested for COVID in the
last month. In February, a new random sample of 120 students was
drawn. In this sample, 21 said they had been tested in the last
month. Is there sufficient evidence to conclude, at the 0.01
significance level, that the proportion of students tested for COVID
was higher in February than in January?

• Step 1: Write down the hypotheses

• Step 2: Compute the test statistic

• Step 3: Draw conclusions

SOC2103-01 [2025 1st Semester] 17
Comparing Two Means
• Again, we extend concepts from the one-sample case to the case of
two independent samples
• With one large sample, the CI is approximately:
∗
𝑠
𝑥ҧ ± 𝑧 ∗
𝑛
• In other words: sample statistic ± 𝑧 ∗ (standard error)
• We use the same logic here: 𝑥ҧ 1 − 𝑥ҧ 2 ± 𝑧 ∗ ∗ 𝑆𝐸(𝑥ҧ 1 − 𝑥ҧ 2)
• Model assumptions: independent random samples, both large
• If both sample sizes are large, CLT (중심극한이론) says the sampling distribution of the means
will be approximately normal
• Rule of thumb: If both 𝑛1 ≥ 30 and 𝑛2 ≥ 30 , you’re good to go!

SOC2103-01 [2025 1st Semester] 18

𝑆𝐸(𝑥ҧ 1 − 𝑥ҧ 2)
• Recall that, for sample proportions from independent samples:
𝑆𝐸 𝑝Ƹ 1 − 𝑝Ƹ 2 = 𝑆𝐸 𝑝Ƹ 1 2 + 𝑆𝐸 𝑝Ƹ 2 2

• For two samples from independent samples:

𝑆𝐸 𝑥ҧ 1 − 𝑥ҧ 2 = 𝑆𝐸 𝑥ҧ 1 2 + 𝑆𝐸 𝑥ҧ 2 2

2 2
𝑆1 𝑆2 𝑆12 𝑆22
= 𝑛1
+ 𝑛2
= 𝑛1
+𝑛
2

SOC2103-01 [2025 1st Semester] 19

Example: PISA Math Test Scores
• In 2012, 15-year-old students in the US had an average score of 481 on
PISA’s math literacy scale, with a standard error of 3.6. Students in the UK
had an average score of 494, with a standard error of 3.3.
• Construct a 95% CI for the difference in the countries’ population mean
scores (assume large samples)
𝑥ҧ1 − 𝑥ҧ2 ± 𝑧 ∗ 𝑆𝐸(𝑥ҧ1) 2 + 𝑆𝐸(𝑥ҧ2) 2

494 − 481 ± 1.96 3.3 2 + 3.6 2 = (𝟑. 𝟒, 𝟐𝟐. 𝟔)

• What can we conclude about how the two countries compare in score?
• We are 95% confident that the true difference in average scores (UK–US) lies
between 3.4 and 22.6 points.
• Alternatively, we could have computed the standard errors from the
sample sizes and estimated standard deviations
SOC2103-01 [2025 1st Semester] 20
Significance Tests for Two Means
• 𝐻0 : 𝜇1 − 𝜇2 = 0
• 𝐻𝑎 : 𝜇1 − 𝜇2 > 0 or 𝐻𝑎 : 𝜇1 − 𝜇2 < 0 or 𝐻𝑎 : 𝜇1 − 𝜇2 ≠ 0
ҧ 0
𝑥−𝜇
• In one large sample 𝑧 =
𝑆𝐸0
sample statistic − value under H0
• In other words, 𝑧 =
standard error under H0
• In two large independent samples:
𝑥ҧ 1 − 𝑥ҧ 2 − 0 𝑥ҧ 1 − 𝑥ҧ 2 − 0
𝑧= =
𝑆𝐸0 (𝑥ҧ 1 − 𝑥ҧ 2)
𝑠12 𝑠22
+
𝑛1 𝑛2
• Same model assumptions as for the CI
SOC2103-01 [2025 1st Semester] 21
Homework: Fish Lengths in Two Regions
• Suppose a sample of 200 fish in the East Sea (동해) has an average
length of 67cm and a sample standard deviation of 6cm. Suppose a
sample of 250 fish in the Yellow Sea (서해) has an average length of
68cm and a sample standard deviation of 5cm.
• Test whether there is sufficient evidence to conclude at the 0.05
significance level that the average length of fish in the East Sea is
less than the average length of fish in the Yellow Sea.
• Step 1: Write down the hypotheses

• Step 2: Compute the test statistic

• Step 3: Draw conclusions

SOC2103-01 [2025 1st Semester] 22
CIs and Significance Tests, Reprise
• Remember the link between confidence intervals and significance
tests for the mean in one sample
• We can reject 𝐻0 : 𝜇 = 𝜇0 at the 0.05 significance level if and only if the 95%
confidence interval does not contain 𝜇0
• Likewise for 0.01/99% and 0.10/90%
• The same holds for comparisons of means between two
independent samples
• We can reject 𝐻0 : 𝜇1 − 𝜇2 = 0 at the 0.05 significance level if and only if the
95% confidence interval does not contain 0
• What about for comparisons of proportions between two
independent samples?
• Same logic as with means – If the CI excludes 0 (which represents no
difference), the observed difference is statistically significant
SOC2103-01 [2025 1st Semester] 23
Mean of Differences in Matched Samples
• Up until now, we’ve been talking how to analyze two independent samples
• Sometimes, we want to analyze data from matched pairs
• Examples: pre-test and post-test; biological twins (esp. monozygotic twins)
• Why studying matched pairs is helpful
• Even if you randomly draw two independent samples (like a random pre-test group
vs. a random post-test group), you could get samples that are pretty different
• The differences between the two groups may come from:
1) Signal: The effect you’re trying to measure
2) Noise: Natural differences between people (e.g., age, ability, motivation, …)
• In a matched pairs design:
• You compare each subject to themselves (e.g., before vs. after)
• Or you match two very similar individuals (like twins)
➢ Matched pairs eliminate that noise and make it easier to detect real effects!
SOC2103-01 [2025 1st Semester] 24
Mean of Differences in Matched Samples (cont.)
• Analysis
• In matched pairs, you analyze 𝜇𝑑 – the mean difference within pairs
• Treat each pair of data points as if it provided just one data point: the
difference between the values in the pair
• You compute the difference for each pair, like: 𝑑𝑖 = 𝑋1𝑖 − 𝑋2𝑖
• Then, you treat that set of differences 𝑑1 , 𝑑2 , 𝑑3 , … , 𝑑𝑛 as your sample of interest
• Then, use one-sample methods (significance tests, CIs)
• Once you have the list of differences, you:
• Compute the mean of the differences: 𝑑ҧ
• Compute the standard deviation of the differences: 𝑆𝑑
• Use one-sample methods to analyze whether the mean of difference is
significantly different from zero (or any other value you’re testing)

SOC2103-01 [2025 1st Semester] 25

Example: Depression Scores
• Suppose a random sample of 316 adults tried a new drug treating
depression. If, on average, depression scores fell by 1 after starting
treatment and the standard deviation of the score change was 6, do
we have sufficient evidence to conclude at the 0.05 significance level
that depression levels were lower after treatment?
• Step 1: Write down the hypotheses
𝐻0 : 𝜇𝑑 = 0; 𝐻𝑎 : 𝜇𝑑 < 0; 𝜇𝑑 = 𝜇𝑝𝑜𝑠𝑡 − 𝜇𝑝𝑟𝑒
• Step 2: Find the test statistic
𝑥ҧ𝑑 − 0 𝑥ҧ𝑑 − 0 −1
𝑧= = 𝑠 = = −2.96
𝑆𝐸0 𝑑 6
𝑛 316
SOC2103-01 [2025 1st Semester] 26
Example: Depression Scores (cont.)
• Step 3: Draw conclusions
• Reject 𝐻0 if the test statistic is less than the 5th percentile of the
standard normal distribution
• Note that this is a one-tailed test at the 0.05 level
• On the standard normal curve, this is the z-score where the area to the left is 0.05
• From the z-table: −1.645
• −2.96 < −1.645 → We can reject 𝑯𝟎 of no change, on average, in
favor of 𝑯𝒂 that depression scores were lower post-treatment
• Q: Does this mean that the drug was
successful in treating depression?

SOC2103-01 [2025 1st Semester] 27

Comparing Proportions in Matched Samples:
McNemar’s Test
• McNemar’s Test is a go-to method when you are comparing proportions in
matched samples. McNemar’s Test is used when:
• You have paired (matched) data
• Each subject is measured twice (before & after) or in two related conditions
• The response is categorical with two possible outcomes (yes/no, correct/incorrect)
• Example: Pre/post studies with binary outcomes
(e.g., Did people vote before and after an ad campaign?)
• McNemar’s Test is an easy way to conduct a significance test for the
difference between population proportions in matched pair samples

SOC2103-01 [2025 1st Semester] 28

McNemar’s Test (cont.)
• McNemar’s Test data layout (2 x 2)
• Let’s say you’re comparing responses before and after some treatment:
After: Yes After: No Row Total
Before: Yes a b a+b
Before: No c d c+d
Column Total a+c b+d N
• Only b and c matter for McNemar’s Test: these are called discordant pairs (where the
response changed)
• Hypotheses:
• 𝐻0 : The proportions of “yes” and “no” did not change (i.e., b=c)
• 𝐻𝑎 : The proportions did change
𝑏−𝑐
• Test statistic (for large samples): 𝑧 = 𝑏+𝑐

SOC2103-01 [2025 1st Semester] 29