AUTHOR: ANH VU PHUONG

HO CHI MINH UNIVERSITY OF SCIENCE (HCMUS)

STATISTICS OVERVIEW

DATE: January 6, 2026

HO CHI MINH CITY, VIETNAM
 CONTENTS

1. OVERVIEW AND DESCRIPTIVE STATISTICS 1

1) Define Statistics, Population, Sample, and Process . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1).1 Statistics, Population, Sample, and Process . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1).2 Scales of Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2) Methods in descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2).1 Stem and leaf plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2).2 Histogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3) Measure of location (Mean, Median, Mode, Percentiles and Quartiles) . . . . . . . . . . . . 5
3).1 Comparison of Mean, Median, and Mode for different shapes of distributions . . . 7
3).2 Percentiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3).3 Quartiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4) Measure of variability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4).1 How to measure the variation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4).2 Meaning of variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4).3 Interquartile Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4).4 Coefficient of Variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2. PROBABILITY 10
5) Overview of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5).1 Elements of Probability: Sample Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5).2 Elements of Probability: Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6) Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7) Complementary Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7).1 DeMorgan’s Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8) Kolmogorov Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8).1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
9) Equiprobability Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
10) Interpretation of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
11) Counting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
12) Permutation and Combination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
12).1 Permutation: Ordered Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
12).2 Combinations: Unordered Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
12).3 Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
13) Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
14) Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
15) Partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
16) The Law of Total Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
17) Bayes Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3. DISCRETE RANDOM VARIABLE & PROBABILITY DISTRIBUTIONS 21

18) Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
18).1 Sample Space of Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
18).2 Type of Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

ii
 18).3 Probability Mass Functions of Discrete Variables . . . . . . . . . . . . . . . . . . . . . . 22
18).4 Cumulative Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
18).5 Expected Value of a Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
18).6 Expectation and Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
18).7 Variance and Standard deviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
19) Bernoulli Trial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
19).1 Expectiation and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
20) Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
20).1 Expectation and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
20).2 Binomial R function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
21) Geometric Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
21).1 The pmf of Geometric Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
21).2 Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
21).3 Geometric Distribution in a Different View . . . . . . . . . . . . . . . . . . . . . . . . . . 30
21).4 Geometric Distribution pmf & cdf R Code . . . . . . . . . . . . . . . . . . . . . . . . . . 31
22) Negative Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
22).1 Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
22).2 Negative Binomial Distribution (Textbook Version) . . . . . . . . . . . . . . . . . . . . 31
22).3 Negative Binomial Distribution pmf & cdf R Code . . . . . . . . . . . . . . . . . . . . . 32
23) Hypergeometric Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
23).1 Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
23).2 Hypergeometric Distribution in R Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
24) Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
24).1 Poisson Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
24).2 Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
24).3 Poisson Distribution in R Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
24).4 Conditions for a Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
24).5 Relationship Between Binomial and Poisson . . . . . . . . . . . . . . . . . . . . . . . . . 36

4. CONTINUOUS RANDOM VARIABLE & PROBABILITY DISTRIBUTION 37

25) Continuous Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
26) Probability Density Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
27) Cumulative Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
28) Mean and Variance of a Continuous Random Variable . . . . . . . . . . . . . . . . . . . . . . . 38
29) Relationship Between fY (y) and FY (y) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
30) Percentiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
31) Uniform Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
32) Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
32).1 Density of Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
32).2 Characteristics of Normal pdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
32).3 Empirical Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
32).4 Standard Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
32).5 Inverse Normal Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
33) Approximating the Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
33).1 Correction for Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

iii
 34) Skewness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
34).1 Types of Skewness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
34).2 Tests of Skewness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
34).3 Interpretation of Skewness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
34).4 Magnitude of Skewness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
34).5 Handling skewness in data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
34).6 Difference between Dispersion and Skewness . . . . . . . . . . . . . . . . . . . . . . . . 52
35) Exponential Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
35).1 Mean, Median and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
35).2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
35).3 Memoryless Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
36) Gamma Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
36).1 Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
37) Weibull Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
37).1 cdf of Weibull Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
37).2 Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
37).3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
38) Chi-squared Distribution (χ2 Distribution) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5. JOINT PROBABILITY DISTRIBUTIONS AND RANDOM SAMPLES 60

39) Joint pmf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
40) Joint cdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
41) Joint pdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
42) Marginal pmf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
43) Marginal pdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
44) Conditional pmf/pdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
44).1 Properties of Conditional pdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
44).2 Conditional Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
45) Independence of random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
46) Expected value of a Function of Two random variables . . . . . . . . . . . . . . . . . . . . . . 62
47) Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
47).1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
47).2 Independence and Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
48) Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
49) Linear functions of Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
50) Revision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
50).1 Terminology: Population/Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
50).2 Terminilogy: Random Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
50).3 Terminology: Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
50).4 Terminilogy: Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
50).5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
51) Propositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
52) Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
52).1 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

iv
6. POINT ESTIMATION 69
53) Point estimators and Sampling distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
53).1 Terminology: Sampling Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
53).2 Evaluate an Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
54) The Methods of Moment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
55) Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
55).1 The Invariance Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

7. STATISTICAL INTERVALS BASED ON A SINGLE SAMPLE 75

56) Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
57) Inference on Population Proportion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
58) Point Estimator of Proportion p . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
58).1 Property of p̂ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
58).2 Sampling Distribution of p̂ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
59) t Disitrbution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
59).1 t Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
60) Confidence Interval (CI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
60).1 Confidence Interval for proportion p . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
60).2 Interpretation of Confidence Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
61) Inference of Population Mean µ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
61).1 Sampling distribution of Ȳ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
61).2 CI for population mean µ assuming normality and known σ 2 . . . . . . . . . . . . . 81
61).3 CI for population mean µ assuming normality and unknown σ 2 . . . . . . . . . . . . 81
61).4 CI for a population variance σ 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
62) Prediction interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
63) Normal Quantile-Quantile (qq) Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
64) Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

8. TESTS OF HYPOTHESES BASED ON A SINGLE SAMPLES 86

65) Statistical Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
65).1 Defintion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
65).2 Relationship between confidence interval (CI) and hypothesis testing . . . . . . . . 86
65).3 Testing procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
66) Hypothesis Testing for the population proportion p . . . . . . . . . . . . . . . . . . . . . . . . 86
67) Method of Evaluating a Test: Type I and Type II Errors . . . . . . . . . . . . . . . . . . . . . . 90
67).1 Type I Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
67).2 Type II Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
67).3 Trade-off between Type I and Type II errors . . . . . . . . . . . . . . . . . . . . . . . . . 92
68) Hypothesis Testing for Population Mean µ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
68).1 Hypothesis Testing for the mean µ with known variance σ 2 . . . . . . . . . . . . . . 93
68).2 Hypothesis Testing for the mean µ with unknown variance σ 2 . . . . . . . . . . . . 95
69) Hypothesis Testing for the population variance σ 2 . . . . . . . . . . . . . . . . . . . . . . . . . 99

9. INFERENCES BASED ON TWO SAMPLES 102

70) Difference between two population means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
70).1 Basic assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

v
 71) Inference on µ1 and µ2 assuming known σ12 and σ22 . . . . . . . . . . . . . . . . . . . . . . . . . 103
71).1 Confidence Interval (CI) assuming known σ12 and σ22 . . . . . . . . . . . . . . . . . . . 103
71).2 Hypothesis Testing for µ1 and µ2 assuming known σ12 and σ22 . . . . . . . . . . . . . 103
72) Inference on µ1 and µ2 assuming unknown σ12 and σ22 : σ12 = σ22 . . . . . . . . . . . . . . . . 107
72).1 Confidence Interval (CI) assuming unknown σ12 and σ22 : σ12 = σ22 . . . . . . . . . . . 107
72).2 Hypothesis Testing for µ1 and µ2 assuming unknown σ12 and σ22 : σ12 = σ22 . . . . . 108
73) Inference on µ1 and µ2 assuming unknown σ12 and σ22 : σ12 ̸= σ22 . . . . . . . . . . . . . . . . 112
73).1 Confidence Interval (CI) assuming unknown σ12 and σ22 : σ12 ̸= σ22 . . . . . . . . . . . 112
73).2 Hypothesis Testing for µ1 and µ2 assuming unknown σ12 and σ22 : σ12 ̸= σ22 . . . . . 112
74) Inference on p1 and p2 : Independent samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
74).1 Confidence Interval (CI) of p1 and p2 : Independent samples . . . . . . . . . . . . . . . 116
74).2 Hypothesis Testing for p1 and p2 : Independent samples . . . . . . . . . . . . . . . . . 117
75) F Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
75).1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
75).2 Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
75).3 F Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
76) Inference on ratio of population variances σ22 /σ12 : Independent samples . . . . . . . . . . . 121
76).1 Confidence Interval (CI) σ22 /σ12 : Independent samples . . . . . . . . . . . . . . . . . . . 121
76).2 Hypothesis Testing for σ22 /σ12 : Independent samples . . . . . . . . . . . . . . . . . . . 122
77) Inference on µ1 and µ2 : Dependent samples (Matched-pairs) . . . . . . . . . . . . . . . . . . 125
77).1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
77).2 Hypothesis Testing for µ1 and µ2 : Dependent samples (Matched-pairs) . . . . . . . 127
78) One-way analysis of variance (ANOVA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

10. LINEAR REGRESSION 134

79) Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
80) Simple linear regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
80).1 Least squares estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
80).2 Model assumptions and properties of least squares estimators . . . . . . . . . . . . . 137
80).3 Estimating the error variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
80).4 Inference for β0 and β1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
80).5 Confidence and Prediction intervals for a given x = x0 . . . . . . . . . . . . . . . . . . 142
81) Multiple linear regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
81).1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
81).2 Least squares estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
81).3 Estimating the error variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
81).4 Analysis of variance for linear regression . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
81).5 Inference for individual regression parameters . . . . . . . . . . . . . . . . . . . . . . . 150
81).6 Confidence and prediction intervals for a given x = x0 . . . . . . . . . . . . . . . . . . 153
82) Model diagnostics (residual analysis) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
82).1 Model assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
82).2 Residuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
82).3 QQ-plot for Normality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
82).4 Residual plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
82).5 Common violations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

vi
List of Tables

4..1 Normal Distribution Table (Positive z-score) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4..2 Normal Distribution Table (Negative z-score) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4..3 Chi-squared (χ2 ) Distribution Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7..1 t-Distribution Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
10..1 Analysis of variance table for linear regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

vii
STATISTICS OVERVIEW

1. OVERVIEW AND DESCRIPTIVE STATISTICS

1) Define Statistics, Population, Sample, and Process

1).1 Statistics, Population, Sample, and Process
 Statistics is the science of problem solving using DATA. A statistical investigative process involves with collecting,
organizing, summarizing and analyzing information in order to draw conclusions or to make decisions.
 Descriptive Statistics deals with organizing, displaying, and summarizing the data collected.
 Inferential Statistics consists of methods that use sample results to help make decisions or predictions about a pop-
ulation.
 Census is a survey that includes every member of the population whose desired information can be obtained.
 Sample survey is a technique of collecting information from a portion of the population.
 Variables are the characteristics of the individuals (sampling units) within the population.
 Random Variables are variables that take values at random.
Key Point: Possible values of variables vary.
Consider the variable “height”. If all individuals had the same height, then obtaining the height of one individual would be
sufficient in knowing the heights of all individuals. Of course, this is not the case. As investigators, we wish to identify the
factors that influence variability

1).2 Scales of Measurement

 Scales of measurement include: Nominal, Interval, Ordinal, Ratio
 The scale determines the amount of information contained in the data
 The scale indicates the data summarization and statistical analyses that are most appropriate.

Scale Definition Key characteristics Statistical analyses

 Data are labels or names used to  Categories are mutually  Mode
identify an attribute of the element. exclusive  Frequency distribution
Nominal  A nonnumeric label or numeric  No inherent order or ranking  Chi-square test
code may be used  Cannot perform
mathematical operations
 The data have the properties of  Categories have a defined  Median
nominal data and the order or rank order  Percentiles
of the data is meaningful.  Differences between  Spearman’s rank correlation
Ordinal  A nonnumeric label or numeric categories are not  Mann-Whitney U test
code may be used quantifiable
 Can use greater than or less
than comparisons
 The data have the properties of  Equal intervals between  Mean
ordinal data, and the interval values  Standard deviation
Interval between observations is expressed  No true zero point  Pearson’s correlation
in terms of a fixed unit of measure.  Can perform addition and  t-tests
 Interval data are always numeric subtraction  ANOVA
 The data have all the properties of  Equal intervals between  All statistical tests
interval data and the ratio of two values applicable to interval data
Ratio values is meaningful.  True zero point  Geometric mean
 Variables such as distance, height,  Can perform all  Coefficient of variation
weight, and time use the ratio scale. mathematical operations
 This scale must contain a zero (addition, subtraction,
value that indicates that nothing multiplication, division)
exists for the variable at the zero
point.

Page 1 of 160
STATISTICS OVERVIEW

Nominal Ordinal Interval Ratio

Categorizes and labels variables ✓ ✓ ✓ ✓
Ranks categories in order ✓ ✓ ✓
Has known, equal intervals ✓ ✓
Has a true or meaningful zero ✓

 The list of observations a variable assumes is called data.

Qualitative data are observations corresponding to a qualitative variable.
− Gender is a variable. The possible values for Gender are male or female ⇒ the values: Male or Female are data.
Quantitative data are observations corresponding to a quantitative variable.
− Weight is a variable. A weight of 154 lbs is an observation.
 Discrete data are observations corresponding to a discrete variable.
 Continuous data are observations corresponding to a continuous variable.

2) Methods in descriptive statistics

A statistical table can be used to display data graphically as a data distribution, consists of Class, Class Frequency,
Relative Frequency or Percentage.
 For qualitative data, three measurements are available for the list of categories:
− the frequency, or number of measurements
− the relative frequency, or proportion = frequency / Total number of observations
− the percentage
 A pie chart is the familiar circular graph that shows how the measurements are distributed among the categories.
 A bar chart shows the same distribution of measurements in categories, with the height of the bar measuring how

Page 2 of 160
STATISTICS OVERVIEW

often a particular category was observed.

 A bar chart in which the bars are ordered from largest to smallest is called a Pareto chart.
Example: A survey of 400 individuals are made to rate the quality of a school.
The data is summarized:

Rating Frequency Relative frequency Percentage

A 35 0.0875 8.75%
B 260 0.65 65%
C 93 0.2325 23.25%
D 12 0.03 3%
Total 400 1 100%

2).1 Stem and leaf plots

This plot presents a graphical display of the data using the actual numerical values of each data point.
Each number is split into two parts
 The first digit(s) form the stem
 The last digit forms the leaf
The leaf should only ever contain a single digit.

Constructing a Stem and Leaf Plot:

1. Divide each measurement into two parts: the stem and the leaf
2. List the stems in a column, with a vertical line to their right.
3. For each measurement, record the leaf portion in the same row as its matching stem.
4. Order the leaves from lowest to highest in each stem
5. Provide a key to your stem and leaf coding so that the reader can recreate the actual

Example: The following Table lists the prices (in dollars) of 19 different brands of walking shoes.
Construct a tem and leaf plot to display the distribution of the data.

90 70 70 70 75 70
65 68 60 74 70 95
75 70 68 65 40 65
70 70 68 65 40 65

Solution

Page 3 of 160
STATISTICS OVERVIEW

Classification Stem and Leaf Plot

40s 40 4 0
50s 5
60s 65 68 60 68 65 65 6 0 5 5 5 8 8
70s 70 70 70 75 70 74 70 75 70 70 7 0 0 0 0 0 0 0 4 5 5
80s 8
90s 90 95 9 0 5

We can use stem and leaf diagrams to calculate averages like the median, the mode and the mean, and to calculate
measures of spread like the range and the interquartile range.

Dual stem and leaf diagram

Comparing data sets is simplified by using a dual stem and leaf diagram which have two sets of data represented
back to back.
Example: Two sets of data showing test scores of 20 males and 20 females.

Solution

Female Male
0 6 7 0 1 4 5
1 2 4 8 8 9 1 0 2 3 4 4 8
2 4 5 5 5 6 6 2 2 7 7 9
3 0 1 2 3 3 3 6 3 0 0 0 3 6 8
4 4 0

By combining them together to form one dual stem and leaf diagram, we can directly compare the two sets of
data.
Female Male
7 6 0 1 4 5
9 8 8 4 2 1 0 2 3 4 4 8
6 6 5 5 5 4 2 2 7 7 9
5 3 3 3 2 1 0 3 0 0 0 3 6 8
4 0

 The data must be closely related for it to be compared effectively on the diagram.

Note

The digits in the leaf for Females is still in ascending order but from right to left, rather than left to right. Also,
the key represents the data values for each side of the stem and leaf diagram.

2).2 Histogram
A histogram for a quantitative data set is a graph that describes the relative frequency (or frequency) of the variable in
which the possible values of the variable are divided into a few groups (classes, or intervals), the relative frequency (or
frequency) is represented by a rectangle with the height representing the proportion or relative frequency of occurrence
for a particular class (or group) of the variable being measured. And these rectangles should touch each other.
 On the X axis: The class, (or group) of the variable are plotted along the x axis.
 On the Y -axis: The relative frequency or frequency of observations within the class is the height on the Y axis.

Page 4 of 160
STATISTICS OVERVIEW

How to construct a histogram?

Constructing a relative frequency histogram for continuous variables:
1. Choose the number of classes, usually between 5 and 15.
2. Calculate the approximate class width by dividing the difference between the largest and smallest values

Range largest − smallest

Class width = =
number of classes number of classes

3. Round the approximate class width up to a convenient number.

4. Locate the class boundaries.
− If discrete, assign one or more integers to a class.
− If continuous, use Method of left inclusion: Include the left class boundary point but not the right boundary
point in the class.

Note

Different methods may be used in different software. Some may use right inclusion. Some may add an addi-
tional decimal place for the class boundary.

5. Construct a statistical table containing the classes, their boundaries, and their relative frequencies
6. Construct the histogram like a bar graph with the rectangle height = relative frequency of the class / class width

relative frequency of the class

Rectangle height =
class width

Example: The following Table lists the prices (in dollars) of 19 different brands of walking shoes.
Construct a relative histogram to display the distribution of the data.
90 70 70 70 75 70 65 68 60 74 70 95 75 70 68 65 40 65 70
Solution

1. Determine number of classes: for example, use k = 6 classes

55
2. Range = 95 − 40 = 55 ⇒ Class width = ≈ 9.17
6
3. Round the class width up ⇒ class width ≈ 9.17 ≈ 10
4. Use left-inclusion to determine class boundaries: [40, 50), [50, 60), [60, 70), [70, 80), [80, 90), [90, 100)
5. Construct a Relative Frequency Table: first count number of observations in each class. This is the frequency,
fi
call it fi . Relative frequency (rfi ) = where n = the total number of data points
n
6. Draw a two-dimensional graph with
 X-axis: the class boundaries of the variable
 Y -axis: the relative frequency for each class, and a rectangle with the relative frequency as the height for each
class.

Group Frequency Relative Frequency

[40, 50) 1 1/19
[50, 60) 0 0
[60, 70) 6 6/19
[70, 80) 10 10/19
[80, 90) 0 0
[90, 100) 2 2/19

Page 5 of 160
STATISTICS OVERVIEW

3) Measure of location (Mean, Median, Mode, Percentiles and Quartiles)

 A parameter is a descriptive measure of a population. In most real world cases, the population parameter is not
known. For example, the average gas price in the whole nation.
 A statistic is a descriptive measure of a sample. We use statistic to estimate the corresponding parameter. For example,
Average gas price of the nation is not known. However, we can take a random sample of 100 stations and compute the
sample average gas price, then use the sample average to estimate the unknown population average.
The population mean, is computed using all the individuals in a population, the total number of all individuals is N .
The population mean is a parameter
P
x1 + x2 + .... + xN xi
µ= = (1..1)
N N

The sample mean is computed using sample data.

The sample mean is a statistic that is an unbiased estimator of the population mean.
P
x1 + x2 + .... + xn xi
x̄ = = (1..2)
n n

Note

In real world applications, population mean µ is usually not known, and is estimated by using sample mean x̄

Example: Computing a Population Mean and a Sample Mean

The following data represent the travel times (in minutes) to work for all seven employees of a start-up web development
company.
23 36 23 18 5 26 43
(a) Compute the population mean of this data.
(b) Then take a simple random sample of n = 3 employees. Compute the sample mean. Obtain a second simple random
sample of n = 3 employees. Again compute the sample mean.

Solution

(a) Total number of individuals in population N = 7

x1 + x2 + ... + x7 23 + 36 + 23 + 18 + 26 + 43
Population mean µ = = ≈ 24.9 ≈ 25
7 7
x1 + x2 + x3 23 + 36 + 23
(b) Sample n1 = {23, 36, 23} ⇒ x1 = = ≈ 27.3 ≈ 27
3 3
x1 + x2 + x3 18 + 5 + 26
Sample n2 = {18, 5, 26} ⇒ x2 = = ≈ 16.3 ≈ 16
3 3

The median of a variable is the value that lies in the middle of the data when arranged in ascending order. That is, half
the data is below the median and half the data is above the median. We use m to represent the median.

Steps in Computing the Median of a Data Set

1. Arrange the data in ascending order.
2. Determine the number of observations (n).
3. Determine the observation in the middle of the data set.
n+1
The position is
2
n+1 n+1
 is an integer ⇒ n is ODD: locate the data value at the position.
2 2

Page 6 of 160
STATISTICS OVERVIEW

n+1
 If is NOT an integer ⇒ n is EVEN: the median is the average of the two data values on either side of the
2
n+1
observations that lies in the position.
2

The mode of a variable is the most frequent observation of the variable that occurs in the data set
If there are two values that occur with the most frequency, we say the data has is bimodal.
Example: Find the median and the mode of the following pulse rate data
80 76 65 68 72 73 65 80 100
Solution
1. Arrange the data in ascending order:
65 65 68 72 73 76 80 80 100
2. Number of observations n = 9 ⇒ n is odd
n+1 9+1
3. Position of median = = = 5 ⇒ median = 73
2 2
Modes are 65 and 80

 Qualitative data:
− Cannot add or order them … the mean and median do not exist
− The mode is the only one of these three measurements that exists

3).1 Comparison of Mean, Median, and Mode for different shapes of distributions

3).2 Percentiles
The pth percentile of a data set is a value such that at least p percent of the items take on this value or less and at least
(100 − p) percent of the items take on this value or more.

Steps to calculate percentile

1. Arrange the data in ascending order
2. Compute index i, the position of the pth percentile.

p
i=( )n (1..3)
100

3. If i is NOT an integer, round up. The pth percentile is the value in the ith position.
4. If i is an integer, the pth percentile is the average of the values in positions i and i + 1.

Page 7 of 160
STATISTICS OVERVIEW

3).3 Quartiles
Quartiles are specific percentiles.
 First Quartile = 25th Percentile
 Second Quartile = 50th Percentile = Median
 Third Quartile = 75th Percentile

4) Measure of Variability

 Five different measures of variability: Range, Variance, Standard Deviation (σ), Interquartile Range (IQR), Coeffi-
cient of Variation
 Measures of variability measure the degree that the data values spread. The larger the data values spread, the larger
the variation of the data values.

4).1 How to measure the variation?

 Range = R = Largest Data Value - Smalless Data Value
 The sample variance is:

(xi − x̄)2 (x1 − x̄)2 + (x2 − x̄)2 + ... + (xn − x̄)2

P
2
s = = (1..4)
n−1 n−1

The sample standard deviation is:

√
s= s2 (1..5)

Note
The divider (n − 1) is called the Degrees of Freedom.

The population variance is symbolically represented by lower case Greek sigma squared.

(xi − µ)2 (x1 − µ)2 + (x2 − µ)2 + (xN − µ)2

P
σ2 = = (1..6)
N N

The population standard deviation is:

√
σ= σ2 (1..7)

4).2 Meaning of variation

 Approximately 68% of the data is between ±1s of the mean
 Approximate 95% of the data is between ±2s of the mean
 Approximately 99.7% of the data is between ±3s of the mean
If the data is mound-shaped, approximately 95% of the data lies within 4s of the mean 100% within 6s of the mean
range R
S ∼ = (sample)
4 4
range R
S ∼ = (census)
6 6
R R
Therefore, if we know the approximate range of the data, one can estimate S using or (more conservative)
4 6
x−µ
Population z-score: z =
σ
x − x̄
Sample z-score: z =
σ
Empirical Rule can be expressed based on z-score: If the distribution is mounded-shaped, approximately 95% of the
data having z-score between –2 and 2, and almost 100% of data having zscores between –3 and 3.
One can use z-score to identify potential rare events similar to Empirical Rule:
 If the distribution is mounded-shaped, there is only 5% of chance to have z-score outside [−2, 2], Therefore, it is a

Page 8 of 160
STATISTICS OVERVIEW

possible outlier.
 If the distribution is mounded-shaped, there is less than 1% of chance to have z-score outside [−3, 3], Therefore, it is
probably an outlier

4).3 Interquartile Range

 The interquartile range of a data set is the difference between the third quartile and the first quartile.

IQR = Third Quantile − First Quantile = Q3 − Q1 (1..8)

 It is the range for the middle 50% of the data.

 It overcomes the sensitivity to extreme data values
The Five-Number Summary

4).4 Coefficient of Variation

The coefficient of variation indicates how large the standard deviation is in relation to the mean
The coefficient of variation is computed as follows:
 Sample coefficient of variation:
s
CV = ( )100% (1..9)
x̄
 Population coefficient of variation:
σ
CV = ( )100% (1..10)
µ̄

Page 9 of 160
STATISTICS OVERVIEW

2. PROBABILITY

5) Overview of Probability

 Probability is a measure of one’s belief in the occurrence of a future event.

 Prior to performing the experiment or activity, you do not know for certain which particular outcome will occur.

5).1 Elements of Probability: Sample Space

The set of all possible outcomes for a given random experiment is called the sample space, denoted by S (discrete vs
continuous)
 ω = the element of the sample space
 nS = the number of elements in the sample space

5).2 Elements of Probability: Event

 An event is any collection of outcomes contained in the sample space S.
 An event is a set of possible outcomes that is of interest. Using mathematical notation, suppose that S is a sample
space for a random experiment. We say that A is an event in S if the outcome ω satisfies {ω ∈ S : ω ∈ A}.
 We would like to develop a mathematical framework so that we can assign probability to an event A. This will quantify
how likely the event is. The probability that the event A occurs is denoted by P (A).

6) Set Theory
The union The intersection Mutually exclusive
Characteristics The union of two events A The set of all outcome ω in Mutually exclusive events
and B is the set of all both events. cannot occur at the same
outcomes ω in either event or time.
both.
Notation A∪B = {ω : ω ∈ A∨ω ∈ B} A∩B = {ω : ω ∈ A∧ω ∈ B} A∩B =∅
Venn diagram

7) Complementary Events

The complement of an event is the set of the outcomes not included in the event, but still part of the sample space
The complement of A is denoted by A or AC or A′ :

AC = {ω : ω ∈
/ A ∧ ω ∈ S}

The Venn diagram:

Page 10 of 160
STATISTICS OVERVIEW

7).1 DeMorgan’s Law

It is useful when the calculation involves two events and their complements
Let A and B be two events

(A ∪ B)C = AC ∩ B C
(2..1)
(A ∩ B)C = AC ∪ B C

8) Kolmogorov Axioms

For any sample space S, a probability P must satisfy:

 0 ≤ P (A) ≤ 1 for any event A
 P (S) = 1
 If A1 , A2 , ..., An are pairwise mutually exclusive events, then
n
[ n
X
P( Ai ) = P (Ai ) (2..2)
i=1 i=1

Note

“Pairwise mutually exclusive” means that Ai ∩ Aj = ∅ ∀i ̸= j, i, j = 1, 2, ..., n.

8).1 Properties
These three properties are usually referred to as the Axioms of Probability. Any function P satisfies the Axioms of
Probability is called a probability function. Other nice properties can be derived simply by using them.
 P (∅)= 0
 If A ⊆ B ⇒ P (A) ≤ P (B)
 P (A) = 1 − P (AC )
Additive law: If events A and B intersect, you have to subtract out the “double count”

P (A ∪ B) = P (A) + P (B) − P (A ∩ B) (2..3)

The Venn diagram

Page 11 of 160
STATISTICS OVERVIEW

If A and B are mutually exclusive, then A ∩ B = ∅ and P (∅) = 0, it follows that

P (A ∪ B) = P (A) + P (B) (2..4)

9) Equiprobability Model

Suppose that a sample space S contains nS < ∞ outcomes, each of which is equally likely. If the event A contains
nA outcomes, then
nA
P (A) = (2..5)
nS
WARNING: If the outcomes in S are not equally likely, then this result is not applicable.

10) Interpretation of Probability

 One particular interpretation of probability is to view probability as a limiting proportion.

 If the experiment is repeatable, then P (A) can be interpreted as the percentage of times that A will occur “over the
long run.” This is called the relative frequency interpretation.

11) Counting Methods

For equal likelihood probability model, the probability of event A depends on the size of A, i.e., the number of ele-
ments in A, which is denoted by nA .
Recall that if an experiment can result in any one of N different, but equally likely, outcomes and if exactly n of these
outcomes corresponds to event A, then the probability of event A is

nA n
P (A) = = (2..6)
nS N

Consider selecting an element from a set having m elements and selecting another from a set having n elements. Then
there are m × n possible pairs that could be selected.
Example: There are 5 processes needed to manufacture the side panel for a car: clean, press, cut, paint, polish. Our
plant has 6 cleaning stations, 3 pressing stations, 8 cutting stations, 5 painting stations, and 8 polishing stations.
(a) How many possible different “pathways” through the manufacturing exist?
(b) What is the number of “pathways” that include a particular pressing station?
(c) Assuming stations are chosen randomly, what is the probability that a panel follows any particular path?
(d) Assuming stations are chosen randomly, what is the probability that a panel goes through pressing station 1?

Solution

(a) Possible different ”pathways” through the manufacturing exist = 6 × 3 × 8 × 5 × 8 = 5760

(b) Number of ”pathways” that include a particular pressing station = 6 × 1 × 8 × 5 × 8 = 1920
1
(c) Probability that a panel follows any particular path =
5760
1
(d) The probability that a panel goes through pressing station 1 =
1920

12) Permutation and Combination

12).1 Permutation: Ordered Combination
Definition: A permutation is an ordered arrangement of a specific number of distinct objects (Order matters!). We

Page 12 of 160
STATISTICS OVERVIEW

denote the number of possible permutations of n distinct objects, taken r at a time, as

n!
n Pr = Prn = Pr,n = n × (n − 1) × (n − 2) × ... × (n − r + 1) = (2..7)
(n − r)!

where n ≥ r and n! ≡ n × (n − 1) × (n − 2) × ... × 2 × 1 read as “n factorial” and we define 0! = 1

Example: Let there be n = 10 people and suppose that we are to sit r of them into r = 6 chairs, assume the chairs are
numbered. How many ways of arranging people into chairs?

Solution
n! 10! 10!
Number of arrangements = P610 = = = = (10)(9)(8)(7)(6)(5) = 151200
(n − r)! (10 − 6)! 4!

12).2 Combinations: Unordered Samples

Definition: Different selections of objects in which the order of objects does not matter are called combinations.
Theorem: The number of unordered subsets of size r chosen (without replacement) from n available objects is:
!
n n!
n Cr = Crn = = (2..8)
r r!(n − r)!

Example: Let there be 10 people and suppose we want to obtain a sample of size 5 from this group, with the order in
which they are included not important. How many number of possible samples?

Solution

n! 10! 10! (10)(9)(8)(7)(6)

Number of possible samples = C510 = = = = = 252
r!(n − r)! 5!(10 − 5)! 5!5! (5)(4)(3)(2)(1)

12).3 Relationship
n!
n Pr =
(n − r)!
n!
n Cr =
r!(n − r)!
=⇒n Cr × r! =n Pr

Reason: Once we have chosen r objects, there are r! ways to permute those r objects
Example: Counting Methods
9 out of 100 computer chips are defective. We choose a random sample of n = 3
(a) How many different samples of 3 are possible?
(b) How many of the samples of 3 contain exactly 1 defective chip?
(c) What is the probability of choosing exactly 1 defective chip in a random sample of 3? (This is called hypergeometric
distribution later).
(d) What is the probability of choosing at least 1 defective chip in a random sample of 3? (Hint: it is easier to calculate
the probability of choosing no defective chip.)

Solution

100! 100! (100)(99)(98)

(a) Number of possible different samples of 3 N = C3100 = = = 161700
3!(100 − 3)! 3!97! (3)(2)(1)
9! 91! 9! 91!
(b) Number of samples of 3 contain exactly 1 defective chip n1 = C19 ×C291 = × = ×
1!(9 − 1)! 2!(91 − 2)! 1!8! 2!89!

Page 13 of 160
STATISTICS OVERVIEW

9 (91)(90)
↔ n1 = × = 36855
1 (2)(1)
(c) X = number of defective chip in a random sample of 3
C 9 × C 91 n1 36855 351
P (X = 1) = 1 100 2 = = = ≈ 0.23
C3 N 161700 1540
C 91 121485 1157
(d) P (X = 0) = 3 = =
N 161700 1540
1157 383
⇒ P (X ≥ 1) = 1 − P (X = 0) = 1 − = ≈ 0.25
1540 1540

Example: What is the probability of getting 4 or 5 heads when tossing a fair coin 10 times?
This problem will be solved using binomial distribution (*)

Solution
1
Probability of getting heads: p =
2
X = number of heads
! when tossing a fair coin
10 1 10 10! 1 10! 1 (10)(9)(8)(7) 1
P (X = 4) = ( ) = × ( )10 = × ( )10 = × ( )10 ≈ 0.205
4 2 4!(10 − 4)! 2 4!6! 2 (4)(3)(2)(1) 2
!
10
10 1 10! 1 10! 1 (10)(9)(8)(7)(6) 1
P (X = 5) = ( ) = × ( )10 = × ( )10 = × ( )10 ≈ 0.246
5 2 5!(10 − 5)! 2 5!5! 2 (5)(4)(3)(2)(1) 2
⇒ P (X = 4 ∪ X = 5) = P (X = 4) + P (X = 5) − P (X = 4 ∩ X = 5)
Since P (X = 4) and P (X = 5) are mutually exclusive ⇒ P (X = 4 ∩ X = 5) = 0
⇒ P (X = 4 ∪ X = 5) = P (X = 4) + P (X = 5) = 0.205 + 0.246 = 0.451

Example: A university warehouse has received a shipment of 25 printers, of which 10 are laser printers and 15 are
inkjet models. If 6 of these 25 are selected randomly to be checked by a technician, what is the probability that exactly
3 of those selected are laser printers?

Solution

10! 15!
×
C310 × C315 3!(10 − 3)! 3!(15 − 3)! 78
X = number of chosen laser printers ⇒ P (X = 3) = = = ≈ 0.31
C625 25! 253
6!(25 − 6)!

13) Conditional Probability

A conditional probability is the probability that an event will occur, when another event is known to occur or to have
occurred
Definition: Let A and B be events in a sample space S with P (B) > 0. The conditional probability of A, given that B
has occurred, is
P (A ∩ B)
P (A|B) = (2..9)
P (B)
The Venn diagram representation:

Page 14 of 160
STATISTICS OVERVIEW

Figure 2..1: Venn diagram

Multiplication rule:
P (A ∩ B) = P (A|B) × P (B) (2..10)

If B = S, then the conditional probability of A, given B has occurred, is

P (A ∩ S) P (A) P (A)
P (A|B) = P (A|S) = = = = P (A) (2..11)
P (S) P (S) 1

If B ̸= S, then we can understand the conditional probability in this informal way: we are only interested in the
probability of event A when event B has occurred. So, we manually “shrink” the sample space S to be B so that we can
calculate the probability of A in the restricted space with the scale of B.
Example: In a company, 36 percent of the employees have a degree from a SEC university, 22 percent of the employees
that have a degree from the SEC are also engineers, and 30 percent of the employees are engineers. An employee is
selected at random.
(a) Compute the probability that an employee is an engineer and is from the SEC.
(b) Compute the conditional probability that an employee is from the SEC, given that s/he is an engineer.

Solution

Let A = {The employee is an engineer} and B = {The employee is from SEC}.

P (SEC) = P (B) = 0.36, P (Engineers|SEC) = P (A|B) = 0.22, P (Engineers) = P (A) = 0.30
P (Engineer ∩ SEC) = P (A ∩ B)
P (A ∩ B)
P (A|B) = ⇒ P (A ∩ B) = P (A|B) × P (B) = 0.22 × 0.36 = 0.0792
P (B)
P (A ∩ B) 0.0792
P (SEC|Engineers) = P (B|A) = = = 0.264
P (A) 0.30

Example: Roommate Profile Revisit

Given that a randomly chosen roommate snores, what is the probability that he/she likes to party?

Solution

Page 15 of 160
STATISTICS OVERVIEW

Snore [Link] sum

[Link] 150 100 250
[Link] 200 550 750
Sum 350 650 1000

Let A = {Liking to party}, B = {Snoring}

150
P (A ∩ B) = = 0.15
1000
350
P (B) = = 0.35
1000
0.15
P (Liking to party|Snoring) = P (A|B) = ≈ 0.43
0.35

In both examples, the conditional probability P (B|A) and the unconditional probability P (B) are NOT equal. In other
words, knowledge that A “has occurred” has changed the likelihood that B occurs.
In other situations, it might be that the occurrence (or non-occurrence) of a companion event has no effect on the prob-
ability of the event of interest. This leads us to the definition of independence

14) Independence

Definition: Two events are independent if any one of the following equivalent statement is true:

P (A|B) = P (A)
P (B|A) = P (B)
P (A ∩ B) = P (A)P (B)

Example: An electronics store sells three different brands of DVD players. Of its DVD player sales, 50% are brand
1 (the least expensive), 30% are brand 2, and 20% are brand 3. Each manufacturer offers a 1-year warranty on parts
and labor. It is known that 25% of brand 1’s DVD players require warranty repair work, whereas the corresponding
percentages for brands 2 and 3 are 20% and 10%, respectively.
(a) What is the probability that a randomly selected purchaser has bought a brand 1 DVD player that will need repair
while under warranty?
(b) What is the probability that a randomly selected purchaser has a DVD player that will need repair while under
warranty?
(c) If a customer returns to the store with a DVD player that needs warranty repair work, what is the probability that it
is a brand 1 DVD player? A brand 2 DVD player? A brand 3 DVD player?

Solution

P (Brand 1) = 0.5, P (Brand 2) = 0.3, P (Brand 3) = 0.2

P (Repair|Brand 1) = 0.25, P (Repair|Brand 2) = 0.2, P (Repair|Brand 3) = 0.1
P (Brand 1 ∩ Repair)
(a) P (Repair|Brand 1) = ⇒ P (Brand 1 ∩ Repair) = P (Repair|Brand 1) × P (Brand 1)
P (Brand 1)
⇒ P (Brand 1 ∩ Repair) = 0.25 × 0.5 = 0.125
P (Brand 2 ∩ Repair)
(b) P (Repair|Brand 2) = ⇒ P (Brand 2 ∩ Repair) = P (Repair|Brand 2) × P (Brand 2)
P (Brand 2)
⇒ P (Brand 2 ∩ Repair) = 0.2 × 0.3 = 0.06
P (Brand 3 ∩ Repair)
P (Repair|Brand 3) = ⇒ P (Brand 3 ∩ Repair) = P (Repair|Brand 3) × P (Brand 3)
P (Brand 3)
⇒ P (Brand 3 ∩ Repair) = 0.1 × 0.2 = 0.02
P (Repair) = P (Brand 1 ∩ Repair) + P (Brand 2 ∩ Repair) + P (Brand 3 ∩ Repair) = 0.125 + 0.06 + 0.02 = 0.205

Page 16 of 160
STATISTICS OVERVIEW

P (Brand 1 ∩ Repair) 0.125

(c) P (Brand 1|Repair) = = ≈ 0.610
P (Repair) 0.205
P (Brand 2 ∩ Repair) 0.06
P (Brand 2|Repair) = = ≈ 0.293
P (Repair) 0.205
P (Brand 3 ∩ Repair) 0.02
P (Brand 3|Repair) = = ≈ 0.098
P (Repair) 0.205

Example: It is known that 30% of a certain company’s washing machines require service while under warranty, whereas
only 10% of its dryers need such service. If someone purchases both a washer and a dryer made by this company, what
is the probability that both machines will need warranty service?

Solution

Let A = washing machine require service while under warranty ⇒ P (A) = 0.3
B = dryer need service while under warranty ⇒ P (B) = 0.1
Since A and B are independent events ⇒ P (A ∩ B) = P (A)P (B) = 0.3 × 0.1 = 0.03

Example: Complementary Events

The probability that Tom will be alive in 20 years is 0.75 (A). The probability that Nancy will be alive in 20 years is 0.85
(B).
(a) Assuming independence, what is the probability that neither will be alive 20 years from now?
(b) Still assuming independence, what is the probability that only one of the two, Tom or Nancy, will be alive in twenty
years?

Solution

(a) P (AC ∩ B C ) = P (AC )P (B C ) = [1 − P (A)][1 − P (B)] = (1 − 0.75)(1 − 0.85) = 0.0375

(b) P [(AC ∩ B) ∪ (A ∩ B C )] = P (AC )P (B) + P (A)P (B C )[1 − P (A)]P (B) + P (A)[1 − P (B)]
⇒ P [(AC ∩ B) ∪ (A ∩ B C )] = (1 − 0.75) × 0.85 + 0.75 × (1 − 0.85) = 0.325

Note

If event A and B are mutually exclusive ⇒ A and B are NOT independent.

15) Partition

Definition: For some positive integer k, the collection of sets {B1 , B2 , ..., Bk } forms “a” partition of S if:

(i)S = B1 ∪ B2 ∪ ... ∪ Bk
(2..12)
(ii)Bi ∩ Bj = ∅, for i ̸= j

Note

For any subset A of S and any partition {B1 , B2 , ..., Bk } of S, we can decompose A as:

A = A ∪ S = A ∪ (B1 ∪ B2 ∪ ... ∪ Bk ) = (A ∩ B1 ) ∪ (A ∩ B2 ) ∪ ... ∪ (A ∩ Bk ) (2..13)

16) The Law of Total Probability

k
X
P (A) = P (A|Bi )P (Bi ) (2..14)
i=1

Page 17 of 160
STATISTICS OVERVIEW

A = A ∪ S = A ∪ (B1 ∪ B2 ∪ ... ∪ Bk )
A = A ∪ S = (A ∩ B1 ) ∪ (A ∩ B2 ) ∪ ... ∪ (A ∩ Bk )
A = A ∪ S = P (A ∩ B1 ) + P (A ∩ B2 ) + ... + P (A ∩ Bk )
A = A ∪ S = P (A|B1 )P (B1 ) + P (A|B2 )P (B2 ) + ... + P (A|Bk )P (Bk )
Pk
A=A∪S = i=1 P (A|Bi )P (Bi )

17) Bayes Theorem

P (A ∩ B) P (A|B)P (B)
P (B|A) = = (2..15)
P (A) P (A)
Example: Only 1 in 1000 adults is afflicted with a rare disease for which a diagnostic test has been developed. The
test is such that when an individual actually has the disease, a positive result will occur 99% of the time, whereas an
individual without disease will show a positive test result only 2% of the time. If a randomly selected individual is tested
and the result is positive, what is the probability that the individual has the disease?

Solution

Let A = individual has disease, B = positive test result

1
P (A) = = 0.001, P (B|A) = 0.99, P (B|AC ) = 0.02
1000
Law of Total Probability: P (B) = P (B|A)P (A) + P (B|AC )P (AC ) = (0.99)(0.001) + (0.02)(1 − 0.001) =
0.02097
P (A ∩ B) P (B|A)P (A) 0.99 × 0.001
P (A|B) = = = = 0.0472
P (B) P (B) 0.02097

Example:
(a) We purchase 30% of our parts from Vendor A. Vendor A’s defective rate is 5%. What is the probability that a randomly
chosen part is a defective part from Vendor A? (b) We are manufacturing widgets. 50% are red and 30% are white. What
is the probability that a randomly chosen widget will not be white?

Solution

(a) P (Vendor A) = 0.3, P (defective rate) = 0.05

P (Vendor A ∩ defective rate) = 0.3 × 0.05 = 0.015
(b) P (whiteC ) = 1 − 0.3 = 0.7

Example: When a computer goes down, there is a 75% chance that it is due to overload and a 15% chance that it is
due to a software problem. There is an 85% chance that it is due to an overload or a software problem. What is the
probability that both of these problems are at fault?

Solution

Let A = overload, B = software problem

P (A) = 0.75, P (B) = 0.15, P (A ∪ B) = 0.85
P (A ∪ B) = P (A) + P (B) − P (A ∩ B) ↔ 0.85 = 0.75 + 0.15 − P (A ∩ B)
⇒ P (A ∩ B) = 0.75 − 0.15 − 0.85 = 0.05

Example: It has been found that 80% of all accidents at foundries involve human error and 40% involve equipment
malfunction. 35% involve both problems. If an accident involves an equipment malfunction, what is the probability
that there was also human error?

Solution

Page 18 of 160
STATISTICS OVERVIEW

Let A = human error, B = equipment malfunction

P (A) = 0.8, P (B) = 0.4, P (A ∩ B) = 0.35
P (A ∩ B) 0.35
P (A|B) = = = 0.875
P (B) 0.4

Example: Consider the following electrical circuit: The probability on the components is their reliability (probability
that they will operate). Components are independent of each other. What is the probability that the circuit will not
operate when the switch is thrown?

Solution

P (circuit) = P (A ∩ B ∩ C)
Since A, B and C are independent ⇒ P (A ∩ B ∩ C) = P (A)P (B)P (C)
If the circuit doesn’t operate ⇒ P (circuit fails) = P (AC )P (B C )P (C C ) = (1 − 0.95)(1 − 0.95)(1 − 0.95) =
0.000125

Note
The probability of each component operating (reliability) and Components are independent.
 The probability of the circuit not operating is the complement of the probability of the circuit operating.
 In a series circuit, all components must operate for the circuit to operate.
1. Define the probability of a component failing
 The probability of a component failing is 1 minus the probability of it operating.
 Let pi be the probability of component i operating.
 The probability of component i failing is 1 − pi .
2. Calculate the probability of all components failing
 Since the components are independent, the probability of all components failing is the product of their individual
probabilities of failing.
 If there are n components, the probability of all failing is: P (circuit fails) = P (all components fail)
The probability that the circuit will not operate is equal to the probability that all components fail
n
Y
P (circuit fails) = (1 − pi ) (2..16)
i=1

Example: Consider the electrical circuit below. Probabilities on the components are reliabilities and all components
are independent. What is the probability that the circuit will work when the switch is thrown?

Solution

Page 19 of 160
 STATISTICS OVERVIEW

P (circuit operate) = P [(A ∪ B) ∩ C] = P (A ∪ B)P (C) = [P (A) + P (B) − P (A ∩ B)]P (C)

⇒ P (circuit operate) = [0.9 + 0.9 − (0.9)(0.9)](0.95) = 0.9405

Note

The circuit works if either the upper or lower branch works (or both).

Example: An insurance company classifies people as ‘accident-prone’ and ‘nonaccident-prone’. For a fixed year, the
probability that an accident-prone person has an accident is 0.4, and the probability that a non-accident-prone person
has an accident is 0.2. The population is estimated to be 30 percent accident-prone. Define the events
A = {policy holder has an accident}
B = {policy holder is accident-prone}
We are given that P (B) = 0.3, P (A|B) = 0.4, and P (A|B C ) = 0.2.
(a) What is the probability that a new policy-holder will have an accident?
(b) Suppose that the policy-holder does have an accident. What is the probability that s/he is ‘accident-prone’?

Solution

(a) Law of Total Probability: P (A) = P (A|B)P (B) + P (A|B C )P (B C ) = P (A|B)P (B) + P (A|B C )[1 − P (B)]
⇒ P (A) = (0.4)(0.3) + (0.2)(1 − 0.3) = 0.26
P (B ∩ A) P (A|B)P (B) (0.4)(0.3)
(b) P (B|A) = = = ≈ 0.462
P (A) P (A) 0.26

Example: Birthday Problem

What is the probability that at least two students in the class (n = 48) have the same birthday - month and day?
 Year has 365 days, i.e., forget leap year.
 Equal likelihood for each day.

Solution

A = number of students have the same birthday

P 365
P (A = 0) = 4848 ≈ 0.033
365
⇒ P (A ≥ 2) = 1 − P (A = 0) = 1 − 0.033 = 0.967

It seems that there are two different ways to calculate with R. One code may result in an error.
1 > choose ( 3 6 5 , 48) ∗ f a c t o r i a l ( 4 8 ) / 365ˆ48
2 > f a c t o r i a l (365) / (365ˆ48 ∗ f a c t o r i a l (365 −48) )

Page 20 of 160
STATISTICS OVERVIEW

3. DISCRETE RANDOM VARIABLE & PROBABILITY DISTRIBUTIONS

18) Random Variable

Definition: A random variable is a function from a sample space S into the real numbers. We usually denote ran-
dom variables with uppercase letters, e.g., X, Y, ...

X:S→R

Examples:
 Experiment: toss a coin 2 times, Random variable: X = number of heads in 2 tosses
 Experiment: toss 10 dice, Random variable: X = sum of the numbers
 Experiment: apply different amounts of fertilizer to corn plants, Random variable: X = yield/acre

18).1 Sample Space of Random Variable

In defining a random variable, we have also defined a new sample space. For example, in the experiment of tossing a
coin 2 times, the original sample space is
S = {HH, HT, TH, TT}
We define a random variable X = number of heads in 2 tosses. The new sample space is
X = {0, 1, 2}
The new sample space X is called the range of the random variable X

18).2 Type of Random Variables

 A discrete random variable can take one of a countable list of distinct values. Its sample space has finite or countable
outcomes
 A continuous random variable can take any value in an interval of the real number line. Its sample space has un-
countable outcomes.
Examples:
 Time until a projectile returns to earth.
 The number of times a transistor in computer memory changes state in one operation.
 The volume of gasoline that is lost to evaporation during the filling of a gas tank.
Consider toss a fair coin 10 times. The sample space S contains total 210 = 1024 elements, which is of the form
S = {T T T T T T T T T T, ..., HHHHHHHHHH}
The range of the random variable Y , which represents the number of tails in 10 coin flips, is {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}.
This is because Y can take any value from 0 (no tails) to 10 (all tails).
 A random variable (like Y ) assigns a numerical value to each possible outcome of a random event (here, the coin flips).
 Sample space: The sample space includes all possible outcomes of the coin flips. For 10 flips, this would be all
sequences of 10 heads (H) and tails (T ).
 Mapping: The random variable Y maps each sequence in the sample space to the number of tails in that sequence.
− Y (T T T T T T T T T T ) = 10 (10 tails)
− Y (HHHHHT T T T T ) = 2 (2 tails)
− Y (HHHHHHHHHH) = 0 (0 tails)
 Range: The range of Y is the set of all possible values that Y can take. Since the maximum number of tails possible is
10 (if all 10 flips are tails) and the minimum is 0 (if all 10 flips are heads), the range includes all integer values in between.
⇒ The range (all possible values) of Y is {0, 1, 2, 3, ..., 10}, which is much smaller than S.

Note

Page 21 of 160
STATISTICS OVERVIEW

Denote realized value of the random variable by corresponding lowercase letters

18).3 Probability Mass Functions of Discrete Variables

Definition: Let X be a discrete random variable defined on some sample space S. The probability mass function (pmf)
associated with X is defined to be
pX (x) = P (X = x) (3..1)

A pmf p(x) for a discrete random variable X satisfies the following:

 0 ≤ p(x) ≤ 1, for all possible values of x
 The sum of the probabilities, taken over all possible values of X, must equal 1; i.e.,
X
pX (x) = 1
all x

18).4 Cumulative Distribution Function

Definition: Let Y be a random variable, the cumulative distribution function (cdf) of Y is defined as

FY (y) = P (Y ≤ y) (3..2)

 FY (y) = P (Y ≤ y) is read, ‘the probability that the random variable Y is less than or equal to the value y.’

Properties of cumulative distribution function

 limy→−∞ FY (y) = 0
 limy→∞ FY (y) = 1
 Right continuous limy→a+ FY (y) = FY (a)
 Non-decreasing y1 ≤ y2 =⇒FY (y1 ) ≤ FY (y2 )

How to plot Cummulative Dsitribution Function?

 X-axis: Represents the values of the random variable being plotted.

 Y -axis: Represents the cumulative probability, ranging from 0 to 1 (or 0% to 100%).
 Stepped Line: The plot typically shows a stepped line, where each step represents the proportion of values in the
dataset that are less than or equal to the corresponding value on the X-axis.
Example: Mechanical Components
An assembly consists of three mechanical components. Suppose that the probabilities that the first, second, and third
components meet specifications are 0.90, 0.95 and 0.99, respectively. Assume the components are independent.
We define event Si = the ith component is within specification and Si = the ith component is not within specification,
where i = 1, 2, 3.
(a) What is pmf of Y ?
(b) What is cdf of Y ?

Solution

 P (S1 S2 S3 ) = (0.90)(0.95)(0.99) = 0.84645  P (S1 S2 S3 ) = (0.90)(0.95)(0.01) = 0.00855

 P (S1 S2 S3 ) = (0.90)(0.05)(0.01) = 0.00045  P (S1 S2 S3 ) = (0.90)(0.05)(0.99) = 0.04455
 P (S1 S2 S3 ) = (0.10)(0.95)(0.01) = 0.00095  P (S1 S2 S3 ) = (0.10)(0.95)(0.99) = 0.09405
 P (S1 S2 S3 ) = (0.10)(0.05)(0.99) = 0.00495  P (S1 S2 S3 ) = (0.10)(0.05)(0.01) = 0.00005
Possible outcomes for one assembly is:

Page 22 of 160
STATISTICS OVERVIEW

S1 S2 S3 S1 S2 S3 S1 S2 S3 S1 S2 S3 S1 S2 S3 S1 S2 S3 S1 S2 S3 S1 S2 S3
0.84645 0.00045 0.00095 0.00495 0.00855 0.04455 0.09405 0.00005
Let Y = Number of components within specification in a randomly chosen assembly
(a) Probability Mass Function (pmf):
 P (Y = 0) = P (S1 S2 S3 ) = 0.00005
 P (Y = 1) = P (S1 S2 S3 ) + P (S1 S2 S3 ) + P (S1 S2 S3 ) = 0.00045 + 0.00095 + 0.00495 = 0.00635
 P (Y = 2) = P (S1 S2 S3 ) + P (S1 S2 S3 ) + P (S1 S2 S3 ) = 0.00855 + 0.04455 + 0.09405 = 0.14715
 P (Y = 3) = P (S1 S2 S3 ) = 0.84645
Y 0 1 2 3
P (Y = y) 0.00005 0.00635 0.14715 0.84645
(b) Cummulative Distribution Function (cdf):
 FY (y) = P (Y ≤ 0) = P (Y = 0) = 0.00005, when y ≤ 0
 FY (y) = P (Y ≤ 1) = P (Y = 0) + P (Y = 1) = 0.00005 + 0.00635 = 0.0064, when 0 < y ≤ 1
 FY (y) = P (Y ≤ 2) = P (Y = 0) + P (Y = 1) + P (Y = 2) = 0.00005 + 0.00635 + 0.14715 = 0.15355, when
1<y≤2
 FY (y) = P (Y ≤ 3) = P (Y = 0) + P (Y = 1) + P (Y = 2) + P (Y = 3) = 0.00005 + 0.00635 + 0.14715 +
0.84645 = 1 when y > 3
Y 0 1 2 3
P (Y ≤ y) 0.00005 0.0064 0.15355 1

Example: Consider whether the next person buying a computer at a certain electronics store buys a laptop or a desktop
model. We denote X = 1 if the customer purchases a desktop and X = 0 if the customer purchases a laptop. Suppose
that 20% of all purchases during that week select a desktop
Specify the pmf and cdf of X

Solution

P (X = 1) = 0.2
P (X = 0) + P (X = 1) = 1 ↔ P (X = 0) = 1 − P (X = 1) = 0.8
0.8, if x = 0
 Probability mass function: pX (x) = P (X = x) =
0.2, if x = 1

0.8, if x ≥ 0
 Cummulative distribution function: FX (x) = P (X ≤ x) =
1, if 0 < x ≤ 1

Page 23 of 160
STATISTICS OVERVIEW

18).5 Expected Value of a Random Variable

Definition: Let Y be a discrete random variable with pmf pY (y), the expected value or expectation of Y is defined as
X
µ = E(Y ) = yPY (y) (3..3)
all y

The expected value for a discrete random variable Y is simply a weighted average of the possible values of Y . Each
value y is weighted by its probability pY (y).
 In statistical applications, µ = E(Y ) is commonly called the population mean

Note
X is symmetry about µ

18).6 Expectation and Mean

Expectation:
X
µ = E(Y ) = yPY (y) (3..4)
all y

Arithmetic mean when there are n observations, x1 , x2 , ..., xn

n
1X
X̄ = xi (3..5)
n i=1

Note
The expected value can be interpreted as the long-run average

Example: Mechanical Components

Solution
 The pmf of Y :
Y 0 1 2 3
P (Y = y) 0.00005 0.00635 0.14715 0.84645
 The expected value of Y is:
P
µ = E(Y ) = all y ypY (y) = 0(0.00005) + 1(0.00635) + 2(0.14715) + 3(0.84645) = 2.84
 If we randomly choose an assembly, and record the number of components within specification. Over the long
run, the average of these Y observation would be close (converge) to 2.84.

Expected Value of Functions of Y

Theorem: Let Y be a discrete random variable with pmf pY (y). Let g be a real-valued function defined on the range of
Y . The expected value or expectation of g(Y ) is defined as
X
E[g(Y )] = g(y)PY (y) (3..6)
all y

Interpretation: The expectation E[g(Y )] could be viewed as the weighted average of the function g when evaluated
at the random variable Y

Properties of Expectation

Let Y be a discrete random variable with pmf pY (y). Suppose that g1 , g2 , ..., gk are real-valued functions, and let c be a
real constant. The expectation of Y satisfies the following properties:

Page 24 of 160
STATISTICS OVERVIEW

 E(c) = c (c = constant)
 E(cY ) = cE(Y )
Pk Pk
 Linearity: E{ j=1 gj (Y )} = j=1 E{gj (Y )}

Note
The 2nd and 3rd rule together imply that

Xk k
X
E{ cj gj (Y )} = cj E{gj (Y )} for constant cj (3..7)
j=1 j=1

Example: Mechanical Components

In the Mechanical Components example, Y = number of components within specification in a randomly chosen assembly.
We found E(Y ) = 2.84. Suppose that the cost (in dollars) to repair the assembly is given by the cost function g(Y ) =
(3Y )2 . What is the expected cost to repair an assembly?

Solution
 The pmf of Y :
Y 0 1 2 3
P (Y = y) 0.00005 0.00635 0.14715 0.84645
2 2
g(Y ) = (3 − Y ) = Y − 6Y + 9
The expected cost: E[g(Y )] = E[Y 2 − 6Y + 9] = E[Y 2 ] − 6E[Y ] + 9
The expected value of Y 2 : E(Y 2 ) = all y y 2 pY (y) = (02 )(0.00005)+(12 )(0.00635)+(22 )(0.14715)+(32 )(0.84645)
P

⇒ E(Y 2 ) = 8.213
⇒ The expected cost: E[g(Y )] = E[Y 2 ] − 6E[Y ] + 9 = 8.213 − 6(2.84) + 9 = 0.173
⇒ On average, the repair cost on each assembly is $0.173

Example: Project Management

A project manager for an engineering firm submitted bids on three projects. The following table summarizes the firms
chances of having the three projects accepted.

Project A B C
Probability of accept 0.30 0.80 0.10
Probabilty of reject 0.70 0.20 0.90

Assuming the projects are independent of one another

(a) What is the probability that the firm will have all three projects accepted?
(b) What is the probability of having at least one project accepted?
(c) Let Y = number of projects accepted. Fill in the following table and calculate the expected number of projects
accepted.

Y 0 1 2 3
pY (y) 0.126

Solution
(a) Since the projects are independent of one another ⇒ P (Y = 3) = P (A ∩ B ∩ C) = P (A)P (B)P (C)
↔ P (Y = 3) = (0.30)(0.80)(0.10) = 0.024
(b) Since the projects are independent of one another ⇒ P (Y = 0) = P (Ā ∩ B̄ ∩ C̄) = P (Ā) ∩ P (B̄) ∩ P (C̄)
↔ P (Y = 0) = (0.7)(0.2)(0.9) = 0.126
P (Y ≥ 1) = 1 − P (Y = 0) = 1 − 0.126 = 0.874

Page 25 of 160
STATISTICS OVERVIEW

(c) P (Y = 1) = P (A)P (B̄)P (C̄) + P (Ā)P (B)P (C̄) + P (Ā)P (B̄)P (C)
⇒ P (Y = 1) = (0.30)(0.20)(0.90) + (0.70)(0.80)(0.90) + (0.70)(0.20)(0.10) = 0.572
P (Y = 2) = P (A)P (B)P (C̄) + P (A)P (B̄)P (C) + P (Ā)P (B)P (C)
⇒ P (Y = 2) = (0.30)(0.80)(0.90) + (0.30)(0.20)(0.10) + (0.70)(0.80)(0.10) = 0.278
Y 0 1 2 3
pY (y) 0.126 0.572 0.278 0.024

Expected number of project accepted: E(Y ) = 0(0.126) + 1(0.572) + 2(0.278) + 3(0.024) = 1.2

18).7 Variance and Standard deviation

Definition: Let Y be a discrete random variable with pmf pY (y) and expected value E(Y ) = µ. The variance of Y is
defined as
X
σ 2 ≡ var(Y ) ≡ E[(Y − µ)2 ] = (y − µ)2 pY (y) (3..8)
all y

Note

Variance is always non-negative!

Definition: The standard deviation of Y is the positive square root of the variance:
√
(3..9)
p
σ= σ2 = var(Y )

Note

Suppose Y is a random variable with mean µ and variance σ 2

 The variance is the average squared distance from the mean µ
 The variance of a random variable Y is a measure of dispersion or scatter in the possible values for Y .
 The larger (smaller) σ 2 is, the more (less) spread in the possible values of Y about the population mean E(Y )
 Computing Formula:
var(Y ) = E[(Y − µ)2 ] = E[Y 2 ] − [E(Y )]2 (3..10)

Properties

Suppose X and Y are random variable with finite variance. Let c be a constant.
 var(c) = 0
 var(cX) = c2 var(X)
 Suppose X and Y are independent, then

var(X + Y ) = var(X) + var(Y )

(3..11)
var(X − Y ) = var(X) − var(Y )

Example: The pmf of the number X of DVD’s checked out was given p(X = 1) = 0.3, p(X = 2) = 0.25, p(X = 3) =
0.15, p(X = 4) = 0.05, p(X = 5) = 0.1, and p(X = 6) = 0.15.
Compute E(X) and var(X).

Solution
P
E(X) = all x xPX (x) = 1(0.3) + 2(0.25) + 3(0.15) + 4(0.05) + 5(0.1) + 6(0.15) = 2.85
E(X 2 ) = all x x2 PX (x) = (12 )(0.3) + (22 )(0.25) + (32 )(0.15) + (42 )(0.05) + (52 )(0.1) + (62 )(0.15) = 11.35
P

Page 26 of 160
STATISTICS OVERVIEW

var(X) = E(X 2 ) − [E(X)]2 = 11.35 − (2.85)2 = 3.2275

19) Bernoulli Trial

Definition: Many experiments can be envisioned as consisting of a sequence of “trials,” a trial is called Bernoulli trial if
 a trial results in a “success” (1) or “failure” (0)
 trials are independent
 the probability of a “success” in each trial, denoted as p, 0 < p < 1, is the same on every trial, i.e., identical.
Example: Each sample of water has a 10% chance of containing a particular organic pollutant. Assume that the samples
are independent with regard to the presence of the pollutant.

Solution

 trial = Detecting water pollutant

 success = polluted water is observed
 p = P (”success”) = P (pollutted water) = 0.1

Example: Ninety-eight percent of all air traffic radar signals are correctly interpreted the first time they are transmitted.

Solution

 trial = radar signal

 success = signal is correctly interpreted
 p = P (”success”) = P (correct interpretation) = 0.98

Example: When circuit boards used in the manufacture of Blue Ray players are tested, the long-run percentage of
defective boards is 5 percent.

Solution

 trial = test circuit boards

 success = defective boards
 p = P (”success”) = P (defective board) = 0.05

19).1 Expectation and Variance

Usually, a success is recorded a success as”1” and a failure as ”0”.
The probability of success is equal to the probability to get ”1”, i.e., success.
Let p = P (X = 1) and q = 1 − p = P (X = 0)
Expectation:
E(X) = p (3..12)

Variance:
var(X) = p(1 − p) = pq (3..13)

This random variable X is called a Bernoulli random variable

Notation: X ∼ Bernoulli(p)

20) Binomial Distribution

Suppose that n Bernoulli trials are performed. Define

Y = the number of successes (out of n trials performed)
The random variable Y has a binomial distribution with number of trials n and success probability p

Page 27 of 160
STATISTICS OVERVIEW

Notation: Y ∼ B(n, p)
The pmf of Y :  
 n
 py (1 − p)n−y , for y = 0, 1, 2, ..., n



p(y) = y (3..14)


otherwise

0,
!
n
Recall that is the number of ways to choose r distinct unordered objects from n distinct objects
y
!
n n!
=
y r!(n − r)!

Example: Water Filters

A manufacturer of water filters for refrigerators monitors the process for defective filters. Historically, this process
averages 5% defective filters. Five filters are randomly selected.
(a) Find the probability that no filters are defective.
(b) Find the probability that exactly 1 filter is defective.
(c) Find the probability that exactly 2 filter is defective.

Solution

X = number of defective
! filters !
n 5 5!
(a) P (X = 0) = py (1 − p)n−y = (0.05)0 (1 − 0.05)5−0 = (0.95)5 = 0.7738
y 0 0!(5 − 0)!
! !
n 5 5!
(b) P (X = 1) = py (1 − p)n−y = (0.05)1 (1 − 0.05)5−1 = (0.05)(0.95)4 = 0.2036
y 1 1!(5 − 1)!
! !
n 5 5!
(c) P (X = 2) = py (1 − p)n−y = (0.05)2 (1 − 0.05)5−2 = (0.0025)(0.95)3 = 0.0214
y 2 2!(5 − 2)!

20).1 Expectation and Variance

If Y ∼ B(n, p), then:
Expectation:
E(Y ) = np (3..15)

Variance:
var(Y ) = np(1 − p) = npq (3..16)

20).2 Binomial R function

We can use R to calculate the pmf and cdf of binomial distribution directly by using
pY (y) = P (Y = y) = dbinom(y, n, p)
and
FY (y) = P (Y ≤ y) = pbinom(y, n, p)
Example: Radon Levels Historically, 10% of homes in Florida have radon levels higher than that recommended by the
EPA. Assume that radon level for each home is independent of one another. 20 homes are randomly selected.
(a) Find the probability that exactly 3 have radon levels higher than the EPA recommendation.
(b) What is the probability that no more than 5 homes out of the sample having higher radon level than recommended?
(c) What is the probability that at least 5 homes out of the sample having higher radon level than recommended?
(d) What is the probability that 2 to 8 homes out of the sample having higher random level than recommended?
(e) What is the probability that less than 2 or greater than 8 homes out of the sample having higher random level than

Page 28 of 160
 STATISTICS OVERVIEW

recommended?

Solution

X = number of radon levels higher than the EPA recommendation

20!
(a) P (X = 3) = C320 (0.10)3 (1 − 0.10)20−3 = (0.10)3 (0.90)17 = 0.1901
3!(20 − 3)!
(b) P (X ≤ 5) = P (X = 0) + P (X = 1) + P (X = 2) + P (X = 3) + P (X = 4) + P (X = 5)
⇒ P (X ≤ 5) = C020 (0.10)0 (0.90)20 +C120 (0.10)1 (0.90)19 +C220 (0.10)2 (0.90)18 +0.1901+C420 (0.10)4 (0.90)16 +
C520 (0.10)5 (0.90)15
⇒ P (X ≤ 5) = 0.1216 + 0.2702 + 0.2852 + 0.1901 + 0.0898 + 0.0319 = 0.9888
(c) P (X ≥ 5) = 1 − P (X < 5) = 1 − P (X = 4) − P (X = 3) − P (X = 2) − P (X = 1) − P (X = 0)
⇒ P (X ≥ 5) = 1 − (0.1216 + 0.2702 + 0.2852 + 0.1901 + 0.0898) = 0.0431
(d) P (2 ≤ X ≤ 8) = P (X = 2) + P (X = 3) + P (X = 4) + P (X = 5) + P (X = 6) + P (X = 7) + P (X = 8)
⇒ P (2 ≤ X ≤ 8) = 0.2852 + 0.1901 + 0.0898 + 0.0319 + C620 (0.10)6 (0.90)14 + C720 (0.10)6 (0.90)13 +
C820 (0.10)8 (0.90)12 = 0.2852 + 0.1901 + 0.0898 + 0.0319 + 0.0089 + 0.00197 + 0.00036
⇒ P (2 ≤ X ≤ 8) = 0.6082
(e) P (X < 2 ∪ X > 8) = 1 − P (2 ≤ X ≤ 8) = 0.3918

Doing calcultation by R is much easier

1 > dbinom ( 3 , 2 0 , 0 . 1 ) # pmf f u n c t i o n
2 [1] 0.1901199
3 > pbinom ( 5 , 2 0 , 0 . 1 ) # c d f f u n c t i o n
4 [1] 0.9887469

21) Geometric Distribution

The geometric distribution also arises in experiments involving Bernoulli trial:

 each trial results in a “success” or “failure”
 the trials are independent
 the probability of a “success” in each trial, denoted as p, 0 < p < 1, is the same on every trial.
Definition: Suppose that Bernoulli trials are continually observed.
Y = the number of trials to observe the first success.
We say that Y has a geometric distribution with success probability p.
Notation: Y ∼ geom(p)

21).1 The pmf of Geometric Distribution

If Y ∼ geom(p), then the probability mass function (pmf) of Y is given by

(1 − p)y−1 p y = 1, 2, 3, ...
pY (y) = (3..17)
0 otherwise

21).2 Mean and Variance

Suppose Y ∼ geom(p), the expected value of Y is

1
E(Y ) = (3..18)
p

Page 29 of 160
 STATISTICS OVERVIEW

The variance of Y is
1−p
var(Y ) = (3..19)
p2
Example: Wafer
The probability that a wafer contains a large particle of contamination is 0.01. It is assumed that the wafers are inde-
pendent.
(a) What is the probability that exactly 125 wafers need to be analyzed until a large particle is detected?
(b) How many wafers need to be analyzed until a large particle is detected on average?

Solution

(a) pX (x) = (1 − p)n−1 p = (1 − 0.01)125−1 (0.01) ≈ 0.0029

1 1
(b) Expected value of wafers: E(X) = = = 100
p 0.01

1 > dgeom ( 1 2 5 , 0 . 0 1 )
2 [1] 0.002847078

Example: Fruit Fly

Biology students are checking the eye color of fruit flies. For each fly, the probability of observing white eyes is p = 0.25.
We interpret
 trial: to identify the eye color of a fruit fly
 success: The fly has white eyes
 p = P (white eyes) = 0.25
If the Bernoulli trial assumptions hold (independent and identical), then
Y = the number of flies needed to find the first white-eyed ∼ geom(p = 0.25)
(a) What is the probability that the first white-eyed fly is observed on the fifth trial?
(b) What is the probability that the first white-eyed fly is observed before the fourth fly is examined?

Solution

(a) pX (x) = (1 − x)y−1 p = (1 − 0.25)5−1 (0.25) = 0.079

(c) pX (x < 4) = pX (x = 1) + pX (x = 2) + pX (x = 3)
⇒ pX (x < 4) = (1 − 0.25)1−1 (0.25) + (1 − 0.25)2−1 (0.25) + (1 − 0.25)3−1 (0.25) = 0.578

1 > dgeom ( 4 , 0 . 2 5 )
2 [1] 0.07910156
3 > pgeom ( 2 , 0 . 2 5 )
4 [1] 0.578125

21).3 Geometric Distribution in a Different View

We consider a random variable X as the number of failures until the first success. Then, the pmf of X is

p(1 − p)x , x = 0, 1, 2, ...
pX (x) = (3..20)
0 otherwise

Mean and Variance

1−p
E(X) =
p
(3..21)
1−p
var(X) =
p2

Page 30 of 160
 STATISTICS OVERVIEW

21).4 Geometric Distribution pmf & cdf R Code

We can use R to calculate the pmf and cdf of geometric distribution directly by using
pY (y) = P (Y = y) = dgeom(y − 1, p)
and
FY (y) = P (Y ≤ y) = pgeom(y − 1, p)

22) Negative Binomial Distribution

The negative binomial distribution also arises in experiments involving Bernoulli trial:
 each trial results in a “success” or “failure”;
 the trials are independent;
 the probability of a “success” in each trial, denoted as p, 0 < p < 1, is the same on every trial.
Definition: Suppose that Bernoulli trials are continually observed
Y = the number of trials to observe the r-th success.
We say that Y has a negative binomial distribution with waiting parameter r and success probability p.
Notation: Y ∼ N B(r, p). The negative binomial distribution is a generalization of the geometric distribution.
 If r = 1, then N B(1, p) = geom(p)
If Y ∼ N B(r, p), then the probability mass function of Y is given by
 

 y − 1

  pr (1 − p)y−r , y = r, r + 1, r + 2, ...
pY (y) = r−1 (3..22)


otherwise

0

22).1 Mean and Variance

r
E(Y ) =
p
(3..23)
r(1 − p)
var(Y ) =
p2

Example: Fruit Fly

Biology students are checking the eye color of fruit flies. For each fly, the probability of observing white eyes is p = 0.25.
What is the probability the third white-eyed fly is observed on the tenth fly checked?

Solution

y−1 r 10−1 9! 9!
pY (y) = Cr−1 p (1 − p)y−r = C3−1 (0.25)3 (1 − 0.25)10−3 (0.25)3 (0.75)7 = (0.25)3 (0.75)7 =
2!(9 − 2)! 2!7!
0.075

1 > dnbinom ( 7 , 3 , 0 . 2 5 )
2 [1] 0.07508469

22).2 Negative Binomial Distribution (Textbook Version)

The alternative formulation of the Negative Binomial distribution is to count the number of failures until the r-th
success.
X = the number of failures to observe the r-th success.

Page 31 of 160
 STATISTICS OVERVIEW

If X ∼ N B(r, p), then the probability mass function of X is given by

 

 x+r−1

  pr (1 − p)x , x = 0, 1, 2, ...
pX (x) = r−1 (3..24)


otherwise

0

Mean and Variance

1−p
E(X) = r
p
(3..25)
1−p
var(X) = r 2
p

22).3 Negative Binomial Distribution pmf & cdf R Code

We can use R to calculate the pmf and cdf of negative binomial distribution directly by using
pY (y) = P (Y = y) = dnbinom(y − r, r, p)
and
FY (y) = P (Y ≤ y) = pnbinom(y − r, r, p)
Example: Biology students are checking the eye color of fruit flies. For each fly, the probability of observing white eyes
is p = 0.25. What is the probability the third white-eyed fly is observed on the tenth fly checked? We denote X to be
the number of failures to observe third success in 10 trials.

Solution

x+r−1 r 7+3−1 9!
pX (x) = Cr−1 p (1 − p)x = C7−1 (0.25)3 (1 − 0.25)7 = (0.25)3 (1 − 0.25)7 = 0.075
2!(9 − 2)!

1 > dnbinom ( 7 , 3 , 0 . 2 5 )
2 [1] 0.07508469

23) Hypergeometric Distribution

Suppose we have a bowl containing N balls, r of which are red and N − r of which are black. We draw n(< N )
balls out (without replacement). Our random variable of interest Y is the number of sampled balls that are red. Then Y
has a hypergeometric distribution. Define
Y = the number of success (out of the n selected)
We say that Y has a hypergeometric distribution and write Y ∼ hyper(N, n, r).
The pmf of Y :
  


 r N − r

   
y n − y N −r
Cry × Cn−y


, 0 ≤ y ≤ r, 0 ≤ n − y ≤ N − r where y = 0, 1, 2, , ...n

=

  
p(y) = N CnN (3..26)

  

n





otherwise

0,


Page 32 of 160
 STATISTICS OVERVIEW

23).1 Mean and Variance

If Y ∼ hyper(N, n, r), then:
Mean:
r
E(Y ) = n( ) (3..27)
N
Variance:
r N −r N −n
var(Y ) = n( )( )( ) (3..28)
N N N −1
Example: Parts
A batch of parts contains 100 parts from a local supplier of tubing and 200 parts from a supplier of tubing in the next
state.
(a) If four parts are selected randomly and without replacement, what is the probability they are all from the local
supplier?
(b) What is the probability that two or more parts in the sample are from the local supplier?

Solution

100! 200!
C4100 C0200 ( )( )
(a) pY (y) = P (Y = 4) = = 4!96! 200! = 0.0119
C4300 300!
4!296!
100! 200! 100! 200!
C0100 C4200 C1100 C3200 ( )( )+( )( )
(b) pY (y) = P (Y < 2) = P (Y = 0)+P (Y = 1) = + = 0!100! 4!196! 1!99! 3!197!
C4300 C4300 300!
4!296!
⇒ pY (y) = P (Y < 2) = 0.593
⇒ pY (y) = P (Y ≥ 2) = 1 − P (X < 2) = 1 − 0.593 = 0.407

Example: Generator
If a shipment of 100 generators contains 5 faulty generators, what is the probability that we select 10 generators from
the shipment and get a faulty one?

Solution

5! 95!
C15 C995 ( )( )
pY (y) = P (Y = 1) = = 1!4! 9!86! = 0.339
100
C10 100!
10!90!

Example: Five individuals from an animal population thought to be near extinction in a certain region have been
caught, tagged, and released to mix into the population. After they have had an opportunity to mix, a random sample of
10 of these animals is selected. Let X be the number of tagged animals in the second sample. Suppose there are actually
25 animals of this type in the region.
What is the probability that exactly two of the animals in the second are tagged

Solution

5! 20!
C25 C820 ( )( )
pX (x) = P (X = 2) = = 2!3! 8!12! = 0.385
C1025 25!
10!15!

23).2 Hypergeometric Distribution in R Code

The R code to calculate P (Y = y) is of the form dhyper(y, r, N − r, n). Using R,
1 > dhyper ( 4 , 1 0 0 , 2 0 0 , 4 )

Page 33 of 160
 STATISTICS OVERVIEW

2 [1] 0.01185408

What is the probability that two or more parts in the sample are from the local supplier?
P (X ≥ 2) = 1 − P (X ≤ 1) = 1 − 0.593, where P (X ≤ 1) can be computed via R:
1 > phyper ( 1 , 1 0 0 , 2 0 0 , 4 )
2 [1] 0.5925943

24) Poisson Distribution

The Poisson distribution is commonly used to model counts, such as

 the number of customers entering a post office in a given hour
 the number of α-particles discharged from a radioactive substance in one second
 the number of machine breakdowns per month  the number of insurance claims received per day
 the number of defects on a piece of raw material

24).1 Poisson Process

Let the number of occurrences in a given continuous interval of time or space be counted. A Poisson process enjoys the
following properties:
 The number of occurrences in non-overlapping intervals are independent random variables.
 The probability of an occurrence in a sufficiently short interval is proportional to the length of the interval.
 The probability of two or more occurrences in a sufficiently short interval is zero.

24).2 Poisson Distribution

Poisson distribution can be used to model the number of events occurring in a continuous time or space.
Let λ be the average number of occurrences per a unit interval. Let Y = the number of “occurrences” over in a unit
interval of time (or space). Suppose Poisson distribution is adequate to describe Y .
The pmf of Y is given by  y −λ
λ e ,

y = 0, 1, 2, ...
pY (y) = y! (3..29)
otherwise

0

The shorthand notation is Y ∼ Poisson(λ).

Mean:
E(Y ) = λ (3..30)

Variance:
var(Y ) = λ (3..31)

Example: Insulated Wire

“Historically the process has averaged 2.6 breaks in the insulation per 1000 meters of wire” implies that the average
number of occurrences λ = 2.6, and the base units is 1000 meters. Let X = number of breaks in 1000 meters of wire.
So X ∼ Poisson(2.6).
(a) The probability that 1000 meters of wire will have 1 or fewer breaks in insulation.
(b) If we were inspecting 2000 meters of wire, what distribution does Y follow?
(c) If we were inspecting 500 meters of wire, what distribution does Y follow?

Solution

Page 34 of 160
 STATISTICS OVERVIEW

λ0 e−λ λ1 e−λ 2.60 e−2.6 2.61 e−2.6

(a) P (X ≤ 1) = P (X = 0) + P (X = 1) = + = + = 0.267
0! 1! 0! 1!
(b) 1000 meters: λ = 2.6
2000 meters: λ = (2.6)(2) = 5.2 ⇒ Y ∼ Poisson(5.2)
(c) 1000 meters: λ = 2.6
2.6
500 meters: λ = = 1.3 ⇒ Y ∼ Poisson(1.3)
2

24).3 Poisson Distribution in R Code

2.60 e−2.6 2.61 e−2.6
P (Y = 0 ∪ Y = 1) = +
0! 1!
Using R
1 > dpois ( 0 , 2 . 6 ) + dpois (1 , 2 . 6 )
2 [1] 0.2673849

Example:
(a) Suppose we average 5 radioactive particles passing a counter in 1 millisecond. What is the probability that exactly 3
particles will pass in one millisecond?
(b) Suppose we average 5 radioactive particles passing a counter in 1 millisecond. What is the probability that exactly
10 particles will pass in three milliseconds?

Solution

λ3 e−λ 53 e−5
(a) P (X = 3) = = = 0.14
3! 3!
(b) 1 millisecond: λ = 5
3 milliseconds: λ = 15
λ10 e−λ 1510 e−15
P (X = 10) = = = 0.0486
10! 10!

1 > dpois ( 3 , 5 )
2 [1] 0.1403739
3 > dpois (10 , 15)
4 [1] 0.04861075

Example: Let X denote the number of traps in a particular type of metal oxide semiconductor transistor, and suppose
it has a Poisson distribution with λ = 2.
(a) What is the probability that there are exactly three traps?
(b) What is the probability that there are at most three traps?

Solution

λ3 e−λ 23 e−2
(a) P (X = 3) = = = 0.18
3! 3!
λ0 e−λ λ1 e−λ λ2 e−λ λ3 e−λ
(b) P (X ≤ 3) = P (X = 0) + P (X = 1) + P (X = 2) + P (X = 3) = + + +
0! 1! 2! 3!
20 e−2 21 e−2 22 e−2 23 e−2
⇒ P (X ≤ 3) = + + + = 0.857
0! 1! 2! 3!

1 > dpois (3 , 2)
2 [1] 0.180447
3 > dpois (0 , 2) + dpois (1 , 2) + dpois (2 , 2) + dpois (3 , 2)
4 [1] 0.8571235

Page 35 of 160
STATISTICS OVERVIEW

24).4 Conditions for a Poisson Distribution

 Areas of inspection are independent of one another.
 The probability of the event occurring at any particular point in space or time is negligible
 The mean remains constant over all areas of inspection.
The pmf of Poisson distribution is derived based on these three assumption!
Link: [Link] cowan/stat/notes/[Link]

24).5 Relationship Between Binomial and Poisson

λ
For some fixed λ, we have p = , then for large n, B(n, p) is approximately Poisson(λ = np).
n
We can show that !
n e−λ λy
lim py (1 − p)n−y =
n→∞ y y!

Page 36 of 160
STATISTICS OVERVIEW

4. CONTINUOUS RANDOM VARIABLE & PROBABILITY DISTRIBUTION

25) Continuous Random Variable

A continuous random variable is one with an interval (either finite or infinite) of real numbers for its range
Example:
 Let X = length in meter.
 Let X = temperature in ◦ F .
 Let X = time in seconds
Because the number of possible values of X is uncountably infinite, the probability mass function (pmf) is no longer
suitable.
For a continuous random variable, P (X = x) = 0, the reason for that will become clear shortly.
For a continuous random variable, we are interested in probabilities of intervals, such as P (a ≤ X ≤ b), where a and b
are real numbers.
We will introduce the probability density function (pdf) to calculate probabilities, such as P (a ≤ X ≤ b).

26) Probability Density Function

Every continuous random variable X has a probability density function (pdf), denoted by fX (x).
Probability density function fX (x) is a function such that
 fX (x) ≥ 0 for in x ∈ R
R∞
 −∞ fX (x) dx = 1
Rb
 P (a ≤ X ≤ b) = a fX (x) dx which represents the area under fX (x) from a to b for any b > a
 If x0 is a specific value, then P (X = x0 ) = 0. We assign 0 to area under a point

27) Cumulative Distribution Function

Let x0 be a specific value of interest, the cumulative distribution function (CDF) is defined via
Z x0
FX (x0 ) = P (X ≤ x0 ) = fX (x)dx (4..1)
−∞

If x1 and x2 are specific values, then

Z x2
P (x1 ≤ X ≤ x2 ) = fX (x)dx = FX (x2 ) − FX (x1 ) (4..2)
x1

Note

Page 37 of 160
STATISTICS OVERVIEW

 P (x1 ≤ XZ ≤ x2 ) = P (x1 < X < x2 ) = P (x1 ≤ X < x2 ) = P (x1 < X ≤ x2 )

x2
 FX (x) = fX (x): cdf FX (x) is antiderivative of pdf fX (x)
x1
 fX (x) = [FX (x)]′ : pdf fX (x) is derivative of cdf FX (x)

Example: Electric Current

Let the continuous random variable X denote the current measured in a thin copper wire in milliamperes. Assume that
the range of X (measured in mA) is [0, 20], and assume that the probability density function of X is fX (x) = 0.05 for
0 ≤ x ≤ 20. What is the probability that a current measurement is less than 10 milliamperes?

Solution
10
R x2 R 10
P (0 ≤ X ≤ 10) = x1
f (x)dx = 0
0.05dx = 0.05x = (0.05)(10) − (0.05)(0) = 0.5
0
P (0 ≤ X ≤ 10) = P (0 ≤ X < 10) = 0.5

Example: Suppose that Y has the pdf 

3y 2 0 < y < 1
fY (x) =
0 otherwise
(a) Find the CDF of Y .
(b) Calculate P (Y < 0.3)
(c) Calculate P (0.3 ≤ Y ≤ 0.8)

Solution
y
Ry y3
(a) For 0 < y < 1: fY (y) = 0
3y 2 dy = 3 = (y)3 − (0)3 = y 3
3 0
 y ≤ 0 ⇒ fY (y) = 0
 y ≥ 1 ⇒ fY (y) =
1


 0 y≤0

⇒ CDF: FY (y) = y 3 0 < y < 1


1 y ≥ 1

0.3
R 0.3
(b) P (Y < 0.3) = P (0 < Y < 0.3) = 0
3y 2 dy = y 3 = (0.3)3 − 03 = 0.027
0
0.8
R 0.8
(c) P (0.3 ≤ Y ≤ 0.8) = 0.3
2
3y dy = y 3
= (0.8) − (0.3)3 = 0.485
3
0.3

Example: “Time headway” in traffic flow is the elapsed time between the time that one car finishes passing a fixed point
and the instant that the next car begins to pass that point. Let X denote the time headway for two randomly chosen
consecutive cars on a freeway during a period ofheavy flow. The pdf of X is
0.15e−0.15(x−0.5) x ≥ 0.5
fX (x) =
0 otherwise
What is the probability that headway time is at most 5 seconds?

Solution
R5
P (0.5 ≤ X ≤ 5) = 0.5
0.15e−0.15(x−0.5) dx
R5
u = −0.15(x − 0.5) ⇒ P (0.5 ≤ X ≤ 5) = − 0.5
0.15eu du = −e−0.15(x−0.5) − (−e−0.15(0.5−0.5) ) ≈ 0.491

28) Mean and Variance of a Continuous Random Variable

Page 38 of 160
STATISTICS OVERVIEW

The mean (expectation) and variance can also be defined for a continuous random variable. Integration replaces sum-
mation in the calculation of expectation for a discrete random variable.
Recall that for a discrete random variable Y . The mean of Y is defined as
X
E(Y ) = µY = ypY (y)
all y
For a continuous random variable X. The mean of X is defined as
Z +∞
E(X) = µX = xfX (x)dx (4..3)
−∞

Theorem: Let X be a continuous random variable with pdf fX (x). Suppose that g is a real-valued function. Then, g(X)
is a random variable and Z +∞
E[g(X)] = g(x)fX (x)dx (4..4)
−∞

The variance of X, denoted as var(X) or σ 2 , is

Z +∞
2
σ = var(X) = E[(X − µ) ] = 2
(x − µ)2 fX (x)dx (4..5)
−∞

The population standard deviation of X is the positive square root of the variance
√
σ= σ2 (4..6)

The computing formula for variance is the same as the discrete case, i.e.,

var(X) = E(X 2 ) − [E(X)]2 (4..7)

Example: Electric Current

Recall that the pdf of X is 
0.05 0 ≤ x ≤ 20
fX (x) =
0 otherwise
Compute E(X) and var(X).
Solution
20
R 20 x2 202 02
E(X) = 0
0.05xdx = 0.05 = 0.05( − ) = 10
2 0 2 2
3 20
20 20 x 203 03 400
E(X 2 ) = 0 0.05x2 dx = 0.05 0 x2 dx = 0.05
R R
= 0.05( − )=
3 0 3 3 3
2 2 400 2 100
var(X) = E(X ) − [E(X)] = − 10 =
3 3

29) Relationship Between fY (y) and FY (y)

Definition: A random variable Y is continuous if its cdf FY (y) is continuous for −∞ < y < ∞.
Suppose Y is a continuous random variable with cdf FY (y). Then pdf of Y is

d
fY (y) = FY (y) (4..8)
dy

Corollary: From the Fundamental Theorem of Calculus, We see that

Z y
FY (y) = f (t)dt (4..9)
−∞

Page 39 of 160
STATISTICS OVERVIEW

30) Percentiles

Definition: Let p be a number in [0, 1]. The 100p-th percentile of the distribution of a continuous random variable
Y , denoted by η(p), is defined by
Z η(p)
p = F {η(p)} = f (y)dy (4..10)
−∞

The median of a continuous distribution, denoted m, is the 50-th percentile, i.e.

Z m
1
= F (m) = f (y)dy (4..11)
2 −∞

31) Uniform Distribution

Definition: A continuous random variable X is said to have a uniform distribution on the interval [a, b] if the pdf
of X is  1
 a≤x≤b
fX (x) = b − a (4..12)
0 otherwise


Notation: X ∼ U (a, b).

Mean:
a+b
E(X) = (4..13)
2
Variance:
(b − a)2
var(X) = (4..14)
12
If [a, b] = [0, 1], it is said that X follows the standard uniform distribution.
Example: The distribution of the amount of gravel (in tons) sold by a particular construction supply company in a given
week is a continuous random variable with pdf
 3 (1 − x2 ) 0 ≤ x ≤ 1


fX (x) = 2
0 otherwise
(a) What is the cdf of X?
(b) What is the median?
Solution
x
Rx 3 3 t3 3 x3 3 03 3 x3
(a) FX (x) = 0 2
(1 − t2
)dt = (t − ) = (x − ) − (0 − ) = x −
2 3 0 2 3 2 3 2 2
3 m 3
Rm 3 1 3 x 1 3 m 3 03 1
(b) Median F (m) = 0 (1 − x2 )dx = ⇒ (x − ) = ⇔ (m − ) − (0 − ) =
2 2 2 3 0 2 2 3 2 3 2
3 m3 1 3m − m3 1
⇒ m− = ⇔ = ⇒ 3m − m3 = 1 ⇒ −m3 + 3m − 1 = 0
2
 2 2 2 2


 m = −1.879

⇒ m = 1.532



m = 0.3473
Since 0 ≤ x ≤ 1 ⇒ 0 ≤ m ≤ 1 ⇒ m = 0.3473

Example: The pdf of X is

 3 (1 − x2 )dx 0 ≤ x ≤ 1


fX (x) = 2
0 otherwise
Compute E(X) and var(X)

Page 40 of 160
STATISTICS OVERVIEW

Solution
1
3 1 3 x2 x4 3 12 14 3 02 04
Z
R1 3 2 3 3
E(X) = 0 2
x(1 − x )dx = x − x dx = ( − ) = ( − ) − ( − )=
2 0 2 2 4 0 2 2 4 2 2 4 8
Z 1 Z 1 3 5 1 3 5 3 5
3 3 3 x x 3 1 1 3 0 0 1
E(X 2 ) = x2 (1 − x2 )dx = x2 − x4 dx = ( − ) = ( − ) − ( − ) =
0 2 2 0 2 3 5 0 2 3 5 2 3 5 5
1 3 19
var(X) = E(X 2 ) − [E(X)]2 = − ( )2 = ≈ 0.0594
5 8 320

32) Normal Distribution

 Most widely used distribution

 Central Limit Theorem (Chapter 5): Whatever the distribution the random variable follows, if we repeat the random
experiment over and over, the average result over the replicates follows normal distribution almost all the time when
the number of the replicates goes to large.
 Other names: “Gaussian distribution”,“bell-shaped distribution” or “bell-shaped curve.

32).1 Density of Normal Distribution

A random variable X with the pdf

(x − µ)2
1 −
f (x) = √ e 2σ 2 , −∞ < x < ∞
2πσ

is a normal random variable with parameters µ and σ, where ∞ < µ < ∞ and σ > 0. Also

E(X) = µ
var(X) = σ 2

We use X ∼ N (µ, σ 2 ) to denote the distribution. If X ∼ N (0, 1), it is called the standard normal distribution.
Now our objective is to calculate probabilities (of intervals) for a normal random variable through R or normal proba-
bility table.
The plot of the pdfs of normal distributions with different parameter values:

CDF: The cdf of a normal random variable does not exist in closed form. Probabilities involving normal random variables
and normal quantiles can be computed numerically

32).2 Characteristics of Normal pdf

 Bell-shaped curve.
 ∞ < x < ∞, i.e., the range of X is the whole real line
 µ determines the location of a distribution and has the highest value of f (x) at µ.
 The curve is symmetric about µ
 σ determines the dispersion of a distribution.

Page 41 of 160
STATISTICS OVERVIEW

 Inflection points at µ ± σ
Example: Strip of Wire
Assume that the current measurements in a strip of wire follow a normal distribution with a mean of 10 milliamperes
and a variance of 4 (milliamperes)2 . What is the probability that a measurement exceeds 13 milliamperes?

Solution
p √
σ = var(X) = 4 = 2
X −µ 13 − 10
Z= = = 1.5
σ 2
P (X > 13) = 1 − P (Z ≤ 13) = 1 − 0.9332 = 0.0668

32).3 Empirical Rule

For any normal random variable X, we have

Example: Earthquakes in a Town

Since 1900, the magnitude of earthquakes that measures 0.1 or higher on the Richter Scale in a certain location in
California is distributed approximately normally, with µ = 6.2 and σ = 0.5, according to data obtained from the United
States Geological Survey.
(a) Approximately what percent of the earthquakes are above 5.7 on the Richter Scale?
(b) What is the approximate probability that an earthquake is above 6.7?

Solution

(a) 5.7 = 6.2−0.5 = µ−σ ⇒ P (X > 5.7) = P (X > µ−σ) = 1−(X ≤ µ−σ) = 1−0.135−0.235−0.15 = 0.84
(b) 6.7 = 6.2 + 0.5 = µ − σ ⇒ P (X > 6.7) = P (X > µ + σ) = 0.16

32).4 Standard Normal Distribution

If X is a normal random variable with E(X) = µ and var(X) = σ 2 , the random variable

X −µ
Z= (4..15)
σ

is a normal random variable with E(Z) = 0 and var(Z) = 1. That is, Z is called the standard normal random
variable
 Creating a new random variable by this transformation is referred to as standardizing
 Z is traditionally used as the symbol for a standard normal random variable
 Φ(z) is commonly used to stand for the cdf of Z.

Page 42 of 160
STATISTICS OVERVIEW

Table entry for z is the area under the standard normal curve to the left of z

STANDARD NORMAL PROBABILITY TABLE (POSITIVE z-SCORE)

z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
0.0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359
0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753
0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141
0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517
0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6806 0.6844 0.6879

0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224
0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549
0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852
0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133
0.9 0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389

1.0 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621
1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830
1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015
1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177
1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319

1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441
1.6 0.9452 0.9452 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545
1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633
1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706
1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767

2.0 0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817
2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857
2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890
2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916
2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936

2.5 0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952
2.6 0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964
2.7 0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974
2.8 0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981
2.9 0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986

3.0 0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990
3.1 0.9990 0.9991 0.9991 0.9991 0.9992 0.9992 0.9992 0.9992 0.9993 0.9993
3.2 0.9993 0.9993 0.9994 0.9994 0.9994 0.9994 0.9994 0.9995 0.9995 0.9995
3.3 0.9995 0.9995 0.9995 0.9996 0.9996 0.9996 0.9996 0.9996 0.9996 0.9997
3.4 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9998

Table 4..1: Normal Distribution Table (Positive z-score)

Page 43 of 160
STATISTICS OVERVIEW

Table entry for z is the area under the standard normal curve from the left of z

STANDARD NORMAL PROBABILITY TABLE (NEGATIVE z-SCORE)

z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
−3.4 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0002
−3.3 0.0005 0.0005 0.0005 0.0004 0.0004 0.0004 0.0004 0.0004 0.0004 0.0003
−3.2 0.0007 0.0007 0.0006 0.0006 0.0006 0.0006 0.0006 0.0005 0.0005 0.0005
−3.1 0.0010 0.0009 0.0009 0.0009 0.0008 0.0008 0.0008 0.0008 0.0007 0.0007
−3.0 0.0013 0.0013 0.0013 0.0012 0.0012 0.0011 0.0011 0.0011 0.0010 0.0010

−2.9 0.0019 0.0018 0.0018 0.0017 0.0016 0.0016 0.0015 0.0015 0.0014 0.0014
−2.8 0.0026 0.0025 0.0024 0.0023 0.0023 0.0022 0.0021 0.0021 0.0020 0.0019
−2.7 0.0035 0.0034 0.0033 0.0032 0.0031 0.0030 0.0029 0.0028 0.0027 0.0026
−2.6 0.0047 0.0045 0.0044 0.0043 0.0041 0.0040 0.0039 0.0038 0.3700 0.0036
−2.5 0.0062 0.0060 0.0059 0.0057 0.0055 0.0054 0.0052 0.0051 0.0049 0.0048

−2.4 0.0082 0.0080 0.0078 0.0075 0.0073 0.0071 0.0069 0.0068 0.0066 0.0064
−2.3 0.0107 0.0104 0.0102 0.0099 0.0096 0.0094 0.0091 0.0089 0.0087 0.0084
−2.2 0.0139 0.0136 0.0132 0.0129 0.0125 0.0122 0.0119 0.0116 0.0113 0.0110
−2.1 0.0179 0.0174 0.0170 0.0166 0.0162 0.0158 0.0154 0.0150 0.0146 0.0143
−2.0 0.0228 0.0222 0.0217 0.0212 0.0207 0.0202 0.0197 0.0192 0.0188 0.0183

−1.9 0.0287 0.0281 0.0274 0.0268 0.0262 0.0256 0.0250 0.0244 0.0239 0.0233
−1.8 0.0359 0.0351 0.0344 0.0336 0.0329 0.0322 0.0314 0.0307 0.0301 0.0294
−1.7 0.0446 0.0436 0.0427 0.0418 0.0409 0.0401 0.0392 0.0384 0.0375 0.0367
−1.6 0.0548 0.0537 0.0526 0.0516 0.0505 0.0495 0.0485 0.0475 0.0465 0.0455
−1.5 0.0668 0.0655 0.0643 0.0630 0.0618 0.0606 0.0594 0.0582 0.0571 0.0559

−1.4 0.0808 0.0793 0.0778 0.0764 0.0749 0.0735 0.0721 0.0708 0.0694 0.0681
−1.3 0.0968 0.0951 0.0934 0.0918 0.0901 0.0885 0.0869 0.0853 0.0838 0.0823
−1.2 0.1151 0.1131 0.1112 0.01093 0.1075 0.1056 0.1038 0.1020 0.1003 0.0985
−1.1 0.1357 0.1335 0.1314 0.1292 0.1271 0.1251 0.1230 0.1210 0.1190 0.1170
−1.0 0.1587 0.1562 0.1539 0.1515 0.1492 0.1469 0.1446 0.1423 0.1401 0.1379

−0.9 0.1841 0.1814 0.1788 0.1762 0.1736 0.1711 0.1685 0.1660 0.1635 0.1611
−0.8 0.2119 0.2090 0.2061 0.2033 0.2005 0.1977 0.1949 0.1922 0.1894 0.1867
−0.7 0.2420 0.2389 0.2358 0.2327 0.2296 0.2266 0.2236 0.2206 0.2177 0.2148
−0.6 0.2743 0.2709 0.2676 0.2643 0.2611 0.2578 0.2546 0.2514 0.2483 0.2451
−0.5 0.3085 0.3050 0.3015 0.2981 0.2946 0.2912 0.2877 0.2843 0.2810 0.2776

−0.4 0.3446 0.3409 0.3372 0.3336 0.3300 0.3264 0.3228 0.3192 0.3156 0.3121
−0.3 0.3821 0.3783 0.3745 0.3707 0.3669 0.3632 0.3594 0.3557 0.3520 0.3483
−0.2 0.4207 0.4168 0.4129 0.4090 0.4052 0.4013 0.3974 0.3936 0.3897 0.3859
−0.1 0.4602 0.4562 0.4522 0.4483 0.4443 0.4404 0.4364 0.4325 0.4286 0.4247
−0.0 0.5000 0.4960 0.4920 0.4880 0.4840 0.4801 0.4761 0.4721 0.4681 0.4641

Table 4..2: Normal Distribution Table (Negative z-score)

Example: With the help of normal probability table, we can calculate the probabilities for nonstandard normal distri-
bution through standardizing.
 Suppose X ∼ N (10, 4), we want to calculate P (X > 13)

Page 44 of 160
STATISTICS OVERVIEW

 P (X < 7)?
 P (X > 7)?
 P (5 < X < 7)?

Solution
p √
σ = var(X) = 4 = 2 ⇒ X ∼ N (10, 2)
13 − 10
(a) P (X > 13) = P (Z > = 1.5) = P (Z > 1.5) = 1 − P (X ≤ 1.5) = 1 − 0.9332 = 0.0668
2
7 − 10
(b) Z = = −1.5 ⇒ P (X < 7) = P (Z < −1.5)
2
P (Z > 1.5) + P (Z < 1.5) = 1 ⇒ P (Z > 1.5) = 1 − P (Z < 1.5) = 1 − 0.9332 = 0.0668
⇒ P (X < 7) = P (Z < −1.5) = P (Z > 1.5) = 0.0668
(c) P (Z > −1.5) + P (Z < −1.5) = 1 ⇒ P (Z > −1.5) = 1 − 0.0668 = 0.9332
⇒ P (X > 7) = P (Z > −1.5) = 0.9332
5 − 10 7 − 10
(d) P (5 < X < 7) = P ( <Z< ) = P (−2.5 < Z < −1.5) = P (Z < −1.5) − P (Z > −2.5)
2 2
⇒ P (5 < X < 7) = P (Z < −1.5) − P (Z < −2.5)
P (Z < 2.5) + P (Z > 2.5) = 1 ⇒ P (Z > 2.5) = 1 − P (Z < 2.5) = 1 − 0.9938 = 0.0062
P (Z < −2.5) = P (Z > 2.5) = 0.0062
⇒ P (5 < X < 7) = 0.0668 − 0.0062 = 0.0606

Note

 P (Z < −z) = P (Z > z)

 P (Z > −z) = P (Z < z)
 P (Z < z) + P (Z > z) = 1

Example: Steel Bolt

The thickness of a certain steel bolt that continuously feeds a manufacturing process is normally distributed with a
mean of 10.0 mm and standard deviation of 0.3 mm. Manufacturing becomes concerned about the process if the bolts
get thicker than 10.5 mm or thinner than 9.5 mm.
Find the probability that the thickness of a randomly selected bolt is greater than 10.5 or smaller than 9.5

Solution
9.5 − 10.0 10.5 − 10.0
P (X < 9.5 ∪ X > 10.5) = P (X < 9.5) + P (X > 10.5) = P (Z < ) + P (Z > )
0.3 0.3
⇒ P (X < 9.5 ∪ X > 10.5) = P (Z < −1.67) + P (Z > 1.67)
P (Z > 1.67) + P (Z < 1.67) = 1 ⇒ P (Z > 1.67) = 1 − 0.9525 = 0.0475
⇒ P (Z < −1.67) = P (Z > 1.67) = 0.0475
P (X < 9.5 ∪ X > 10.5) = P (Z < −1.67) + P (Z > 1.67) = 0.0475 + 0.0475 = 0.095

32).5 Inverse Normal Probabilities

Sometimes we want to answer a question which is the reverse situation. We know the probability, and want to find the
corresponding value of Y .

Page 45 of 160
 STATISTICS OVERVIEW

Example: Steel Bolt

The thickness of a certain steel bolt that continuously feeds a manufacturing process is normally distributed with a
mean of 10.0 mm and standard deviation of 0.3 mm. Manufacturing becomes concerned about the process if the bolts
get thicker than 10.5 mm or thinner than 9.5 mm.
(a) What is the cutoff value that approximately 2.5% of the bolts produced will have thicknesses less than this value?
(b) What is the cutoff value that approximately 1% of the bolts produced will have thicknesses greater than this value?

Solution
(a) For 2.5% in the lower tail, (z ≈ −1.96).
X −µ
Z= ⇒ X = Zσ + µ = (−1.96)(0.3) + 10.0 = 9.412
σ
(b) For 1% in the upper tail, (z ≈ 2.33).
X −µ
Z= ⇒ X = Zσ + µ = (2.33)(0.3) + 10.0 = 10.699
σ

1 > qnorm ( 0 . 0 2 5 , 0 , 1 , l o w e r . t a i l = TRUE ) # P ( X <= x )

2 [1] −1.959964
3 > qnorm ( 0 . 0 1 , 0 , 1 , l o w e r . t a i l = FALSE ) # P ( X > x )
4 [1] 2.326348

Example: Volume
The fill volume of an automatic filling machine used for filling cans of carbonated beverage is normally distributed with
a mean of 12.4 fluid ounces and a standard deviation of 0.1 fluid ounce.
(a) What is the probability that a randomly chosen can will contain between 12.3 and 12.5 ounces?
(b) 2.5% of the cans will contain less than ounces.

Solution

12.3 − 12.4 12.5 − 12.4

(a) P (12.3 < X < 12.5) = P ( <Z< ) = P (−1 < Z < 1) =
0.1 0.1
P (Z < 1) − P (Z < −1)
⇒ (P 12.3 < X < 12.5) = P (−1 < Z < 1) = 0.8413 − 0.1587 = 0.6826
X −µ
(b) Z = ⇒ X = µ + Zσ
σ
Z < 0 ⇒ X = µ − Zσ = 12.4 − (1.96)(0.1) = 12.204

33) Approximating the Binomial Distribution

Suppose that X ∼ B(n, p). It can be approximated by a normal distribution via

x + 0.5 − np
P (X ≤ x) ≈ Φ( √ ) (4..16)
npq

Page 46 of 160
STATISTICS OVERVIEW

where Φ is the cdf of the standard normal distribution. In practice, the approximation is adequate provided that both
np ≥ 10 and nq ≥ 10.
 What is the constant 0.5?
− The constant 0.5 is a continuity correction factor.
− It is used because the binomial distribution is discrete, while the normal distribution is continuous.
− When approximating a discrete distribution with a continuous one, we need to adjust the boundaries.
− For example, P (X ≤ k) in the binomial distribution is approximated by P (Y ≤ k + 0.5) in the normal distribution.
 Why do we need a condition like np ≥ 10 and nq ≥ 10?
− These conditions ensure that the binomial distribution is sufficiently symmetric and bell-shaped.
− If np or nq are too small, the binomial distribution will be skewed.
− The normal approximation is only accurate when the binomial distribution is close to a normal distribution.
− The conditions np ≥ 10 and nq ≥ 10 are a rule of thumb to ensure the approximation is reasonable.
Procedure for the Normal Approximation to the Binomial Distribution

1. Check to see whether the normal approximation can be used

√
2. Find the mean (µ = np) and the standard deviation (σ = npq)
3. Write the problem in probability notation, using X
4. Rewrite the problem using continuity correction factor, and show the corresponding area under the normal
curve
5. Find the corresponding z-score
6. Find the solution

33).1 Correction for Continuity

 If P (X = a) ⇒ P (a − 0.5 < X < a + 0.5)
 If P (X ≥ a) ⇒ P (X > a − 0.5)
 If P (X > a) ⇒ P (X > a + 0.5)
 If P (X ≤ a) ⇒ P (X < a + 0.5)
 If P (X < a) ⇒ P (X < a − 0.5)
Example: Suppose that 25% of all students at a large public university financial aid. Let X be the number of students
in a random sample of size 50 who receive financial aid, so that p = 0.25.
(a) What is the approximate probability that at most 10 students receive aid? How about the exact probability?
(b) What is the approximate probability that between 5 and 15 students receive aid? How about the exact probability?

Solution
√ p
µ = np = (0.25)(50) = 12.5, σ = npq = (50)(0.25)(0.75) = 3.062
(a) Approximately probability:
10.5 − 12.5
X ≤ 10 ⇒ P (X < 10 + 0.5) = P (X < 10.5) = P (Z < ) = P (Z < −0.65) = 0.2578
3.062
10
X
Exact probability: P (X ≤ 10) = Ck50 (0.25)k (0.75)50−k ≈ 0.2622
k=0
(b) Approximately probability:
4.5 − 12.5 15.5 − 12.5
5 ≤ X ≤ 15 ⇒ P (5 − 0.5 < X < 15 + 0.5) = P (4.5 < X < 15.5) = P ( <Z< )
3.062 3.062
⇒ P (4.5 < X < 15.5) = P (−2.61 < Z < 0.98) = P (Z < 0.98) − P (Z < −2.61) = 0.8365 − 0.0045 = 0.832
15
X
Exact probability: P (5 ≤ X ≤ 15) = Ck50 (0.25)k (0.75)50−k ≈ 0.8382
5

34) Skewness

Page 47 of 160
STATISTICS OVERVIEW

Skewness is a key statistical measure that shows how data is spread out in a dataset. It tells us if the data points
are skewed to the left (negative skew) or to the right (positive skew) in relation to the mean. It is important because
it helps us to understand the shape of the data distribution which is important for accurate data analysis and helps in
identifying outliers and finding the best statistical methods to use for analysis.

34).1 Types of Skewness

Skewness describes the direction and degree of asymmetry in a dataset’s distribution
1. Positive Skewness (Right Skew)
In a positively skewed distribution, the right tail is longer than the left which means most data points are on the left
with a few large values pulling the distribution to the right.
Relationship:

Mean > Median > Mode

2. Negative Skewness (Left Skew)

In a negatively skewed distribution, the left tail is longer which means most data points are on the right with a few
smaller values pulling the distribution to the left.
Relationship:

Mean < Median < Mode

3. Zero Skewness (Symmetrical Distribution)

Zero skewness shows a perfectly symmetrical distribution where the mean, median and mode are equal. In a symmetrical
distribution, the data points are evenly distributed around the central point.
Relationship:

Mean = Median = Mode

34).2 Tests of Skewness

Visual Inspection

This is the simplest and quickest method for assessing skewness by creating a histogram or a density plot of the given
data.
 If the plot has a long tail on the right, the data is positively skewed (right-skewed).
 If the plot has a long tail on the left, the data is negatively skewed (left-skewed).
 If the plot is roughly symmetric, the data has no skewness (zero skew).

Page 48 of 160
STATISTICS OVERVIEW

Skewness Coefficient (Pearson’s First Coefficient of Skewness)

This is a numerical measure of skewness based on the relationship between the mean and mode. It helps us to find if
the data is skewed when the mean and mode are not equal.

Skewness = Mean − Mode (4..17)

 Positive Skew: If the mean is greater than the mode, the skewness is positive.
 Negative Skew: If the mean is smaller than the mode, the skewness is negative.
 Zero Skew: If the mean is equal to the mode, the skewness is zero which indicates a symmetric distribution.

Skeness based on quartiles

This method checks the distances between the quartiles to find skewness. If the quartiles are not equidistant, it suggests
skewness condition:
Q3 − M = M − Q1 ⇒ zero skew (4..18)

where:
 Q3 : the third quartile (median of the second half of data)
 Q2 = M : median of the whole data
 Q1 L : the first quartile (median of the first half of data)
If this condition is not met, it shows either a positive or negative skew which depends on which side is longer.

Measurement of Skewness

Skewness is measured using different techniques to quantify the degree of asymmetry in a dataset’s distribution. Below
are three common methods to measure skewness:
1. Karl Pearson’s Measure
Karl Pearson’s Measure uses the mean, median and standard deviation of the given data to measure the asymmetry of
the distribution. It provides a dimensionless number that helps to quantify how skewed the data is.
With respect to Mean and Median
3(X̄ − M )
Sk = (4..19)
σ
With respect to Mean and Mode
X̄ − Mode
Sk = (4..20)
σ
where:
 Sk : Karl Pearson’s skewness coefficient
 X̄ : mean of the data
 M : median of the data
 σ : standard deviation of the data
Interpretation:
 Skewness = 0: The distribution is symmetric means the mean, median and mode are equal.
 Skewness > 0: The distribution is positively skewed (right-skewed) with the tail on the right side longer than the left.
 Skewness < 0: The distribution is negatively skewed (left-skewed) with the tail on the left side longer than the right.
Example: Calculate Pearson’s skewness coefficient for a dataset of exam scores: 85, 88, 92, 94, 96, 98, 100, 100, 100, 100.

Solution
85 + 88 + 92 + 94 + 96 + 98 + 100 + 100 + 100 + 100 953
X̄ = = = 95.3
10 10

Page 49 of 160
STATISTICS OVERVIEW

96 + 98 194
Because n = 10 ⇒ M = = = 97
P10 2 2
(xi − X̄) 268.1 √
σ 2 = i=1 = = 26.81 ⇒ σ = 26.81 ≈ 5.18
n 10
100 is the most occurrence value in the data ⇒ Mode = 100
3(X̄ − M ) 3(95.3 − 97)
With respect to Mean and Median Sk = = = −1.02
σ 5.18
X̄ − Mode 95.3 − 100
With respect to Mean and Mode Sk = = = −0.94
σ 5.18
Conclusion: Since the skewness coefficient Sk is negative which shows a slight negative skewness in the distri-
bution of exam scores. This means that the tail of the distribution is slightly longer on the left side and most of the
scores are concentrated on the right side of the mean.

2. Bowley’s Measure
Bowley’s Skewness Coefficient is another method for calculating skewness based on quartiles (Q1 , Q2 , Q3 ). Unlike
Karl Pearson’s measure it does not rely on the mean or standard deviation which makes it useful for data that might
not follow a normal distribution. It’s calculated using the first quartile (Q1 ), the second quartile (Q2 or median) and the
third quartile (Q3 ).
Q3 + Q1 − 2Q2
B= (4..21)
Q3 − Q1
where:
 Q1 = first quartile (25th percentile)
 Q2 = second quartile (50th percentile)
 Q3 = third quartile (75th percentile)
Interpretation:
 B = 0 : The distribution is perfectly symmetric (no skewness).
 B < 0 : The distribution is negatively skewed (left-skewed) with the tail on the left side longer.
 B > 0 : The distribution is positively skewed (right-skewed) with the tail on the right side longer.
Example: Calculate Bowley’s Measure of Skewness for the following dataset representing the ages of a group of people
in a sample: 20, 24, 28, 32, 35, 40, 42, 45, 50.

Solution

n = 9 ⇒ Q2 = M = 35
24 + 28
First half data: 20, 24, 28, 32 ⇒ Q1 = = 26
2
42 + 45
Second half data: 40, 42, 45, 50 ⇒ Q2 = = 43.5
2
Q3 + Q1 − 2Q2 43.5 + 26 − 2(35)
B= = ≈ −0.03
Q3 − Q1 43.5 − 26
Conclusion: Since B < 0, this shows a negatively skewed (left-skewed) distribution means the tail is longer on
the left side.

3. Kelly’s Measure
Kelly’s Skewness Measure calculates skewness by comparing certain percentiles in the data which typically the 10th,
50th (median) and 90th percentiles. This measure is useful when dealing with datasets that are not normally distributed
or when other skewness measures may not be as effective.

P90 + P10 − 2P50

SKL = (4..22)
P90 − P10

where:
 SKL > 0 : Positive skew means the right tail is longer or heavier.
 SKL < 0 : Negative skew means the left tail is longer or heavier.
 SKL ≈ 0 : The distribution is symmetric shows little or no skewness.

Page 50 of 160
STATISTICS OVERVIEW

Example: Calculate Kelly’s Coefficient of Skewness for the following data: 5, 7, 8, 9, 10, 12, 15, 16, 18, 20.

Solution

n = 10 ⇒ 0.1n = 1 ⇒ The 10th percentile corresponds to the value at the position 1 ⇒ P10 = 5
10 + 12
n = 10 ⇒ P50 = = 11
2
n = 10 ⇒ 0.9n = 9 ⇒ the 90th percentile corresponds to the value at the position 9 ⇒ P90 = 18
P90 + P10 − 2P50 18 + 5 − 2(11)
SKL = = ≈ 0.08
P90 + P10 18 − 5
Conclusion: Since SKL > 0, this shows a slight positive skew (right-skewed) means the distribution has a longer
tail on the right side.

34).3 Interpretation of Skewness

Interpreting skewness involves understanding both the direction (left or right) and the magnitude (degree of skew) of
the data distribution.

Direction of Skewness

1. Negative skewness (Left skewed): If the skewness is negative, it shows that the distribution is skewed to the left.
In a left-skewed distribution:
 The tail on the left side (the smaller values) is longer and contains outliers.
 The majority of data points are concentrated on the right side.
 The mean is less than the median.
2. Positive skewness (Right skewed): A positive skewness shows that the distribution is skewed to the right. In a
right-skewed distribution:
 The tail on the right side (the larger values) is longer and may contain outliers.
 Most data points are concentrated on the left side.
 The mean is greater than the median
2. Zero skewness (Symmetric): A skewness value close to zero suggests a symmetric distribution where the data is
evenly distributed on both sides of the mean. This means there is no skewness.

34).4 Magnitude of Skewness

The magnitude of skewness gives us information about how extreme the skewness is:
 Skewness close to 0 (between −0.5 and 0.5): The distribution is approximately symmetric.  Skewness below −1: Strong
left skewness (negative skew) with a long tail on the left side.  Skewness above 1: Strong right skewness (positive skew)
with a long tail on the right side.
34).5 Handling skewness in data
Skewed data can impact the accuracy of statistical analyses and predictions. There are various methods to handle skew-
ness depending on the nature of the data and the analysis we want to perform.
1. Data Transformation
 Log Transformation: It is useful for right-skewed data, compressing high values to create a more symmetric distribu-
tion.
 Square Root/Cube Root: It helps reduce positive skew, especially for count data.
 Box-Cox Transformation: A flexible method for handling both positive and negative skew.
2. Removing Outliers
Outliers can cause skewness, so removing them may improve symmetry:
 Z-score: It identify and remove data points with z-scores beyond ±3.
 IQR Method: It remove data points beyond 1.5 times the interquartile range.
3. Non-Parametric Tests

Page 51 of 160
STATISTICS OVERVIEW

When transformations aren’t effective, consider non-parametric tests like the Mann-Whitney U Test or Kruskal-
Wallis Test which do not assume normal distribution and focus on medians rather than means.
4. Machine Learning Models
Some models handle skewed data better:
 Tree-based Models: Decision trees and random forests are less sensitive to skewness.
 Generalized Linear Models (GLM): Use appropriate link functions to model skewed data effectively.

34).6 Difference between Dispersion and Skewness

While dispersion and skewness may seem similar but they measure different aspects of data distribution. Dispersion
refers to the extent to which data points are spread out from the central value (mean or median). It gives us an under-
standing of how varied the data is.

Dispersion Skewness
Measures the spread of data around the central Measures the shape of the distribution and di-
value (mean, median). rection (left or right).
Variance, standard deviation, range, interquar- Pearson’s coefficient of skewness, moment
tile range (IQR). skewness, QQ plots.
Dispersion affects the mean’s interpretation but Skewness shows the relationship between the
is not directly related. mean and median.
High dispersion means data points are spread Positive skew: Right tail longer. Negative skew:
out widely Left tail longer. Zero skew: Symmetric.
Helps understand the variability of data Helps identify the shape and asymmetry of data.
Test scores spread, stock price variability, age Income distribution (right-skewed), exam
range. scores (left/right-skewed).

35) Exponential Distribution

We have discussed Poisson distribution in the previous chapter which for example can model the number of car ac-
cidents for a given length of time t.
The waiting time between accidents is another random variable that is often of interest. We can use an exponential
distribution to model such a waiting period.
We define:
X = the waiting time between two car accidents
and
N = the number of accidents during time of length x
We know that if the mean number of accidents is λ per base unit, then the random variable N ∼ Poisson(λx).
To model the waiting time, suppose there is no accident during the time of length x. Now,

e−λx (λx)0
P (X > x) = P (N = 0) = = e−λx
0!

which means that there is no events in [0, x]

By the complement rule, it follows that

FX (x) = P (X ≤ x) = 1 − P (X > x) = 1 − e−λx (4..23)

By differentiating the CDF of X, the pdf of X is


λe−λx , x≥0
fX (x) = (4..24)
0, otherwise

Page 52 of 160
STATISTICS OVERVIEW

where λ is “mean number of events per unit interval.”

The plot of pdfs of exponential distributions with different values of λ is shown below.
The shorthand notation for X following exponential distribution is given by X ∼ expon(λ)

35).1 Mean, Median and Variance

Suppose that X ∼ expon(λ), then

1
E(X) = µ =
λ
1
var(X) = 2 (4..25)
λ
ln (2)
Median = = µ ln (2)
λ

35).2 Summary
Suppose that X ∼ expon(λ), then
 pdf: fX (x) = λe−λx for x ≥ 0
dexp(x, λ) in R
 cdf: FX (x) = P (X ≤ x) = 1 − e−λx , for x ≥ 0
P (X > x) = 1 − FX (x) = e−λx for x ≥ 0
pexp(x, λ) in R
1
 Mean: E(X) =
λ
1
 Variance: var(X) = 2
λ
Example: Computer Usage
Let X denote the time in hours from the start of the interval until the first log-on. Then, X has an exponential distribution
with 1 log-on per hour. We are interested in the probability that X exceeds 6 minutes. (Hint: Because λ is given in
log-ons per hour, we express all time units in hours. That is, 6 minutes = 0.1 hour)
Solution
P (X > 0.1) = e−λx = e(−1)(0.1) = 0.9048

Example: Accidents
The time between accidents at a factory follows an exponential distribution with a historical average of 1 accident every

Page 53 of 160
STATISTICS OVERVIEW

900 days.
(a) What is the probability that there will be more than 1200 days between the next two accidents?
(b) What is the probability that there will be less than 900 days between the next two accidents?

Solution
1
λ=
900
1
− (1200)
(a) P (X > 1200) = e −λx
= e 900 = 0.2636
1
− (900)
(b) P (X > 900) = e−λx = e 900 = 0.3679
⇒ P (X < 900) = 1 − P (X > 900) = 1 − 0.3679 = 0.6321

1
Example: Suppose that a random variable X follows an exponential distribution with λ = .
6
(a) What is the probability that X is at most 10?
(b) What is the probability that X is between 5 and 10?

Solution
1
(10)
−
(a) P (X ≤ 10) = 1 − e −λx
=1−e 6 = 0.8111
1
(5) −
(b) P (5 ≤ X ≤ 10) = P (X ≤ 10) − P (X ≤ 5) = 0.8111 − (1 − e ) = 0.8111 − 1 + e
λx 6 = 0.2457

Exponential or Poisson Distribution?

Poisson Distribution Exponential Distribution

 We model the number of industrial accidents  We model the length of time between two industrial
occurring in one year. accidents (assuming an accident occurring is a
Poisson event).
 We model the number of radioactive particles  We model the time between radioactive particles
passing by a counter in one hour passing by a counter (assuming a particle passing by
is a Poisson event).

Example: Radioactive Particles

The arrival of radioactive particles at a counter is Poisson events. The number of particles in an interval of time follows a
Poisson distribution. Suppose, on average, we have 2 particles per millisecond. What is the probability that no particles
will pass the counter in the next 3 milliseconds?

Solution

Average number of particles in 3 milliseconds = λt = (2)(3) = 6

λ0 e−λ 60 e−6
P (X = 0) = = = 0.00248
0! 0!

Example: Machine Failures

If the number of machine failures in a given interval of time follows a Poisson distribution with an average of 1 failure
per 1000 hours
(a) What is the probability that there will be no failures during the next 2000 hours?
(b) What is the probability that the time until the next failure is more than 2000 hours?

Solution

Page 54 of 160
STATISTICS OVERVIEW

1 1
λ= ⇒ average number of failures in next 2000 hours = (2000) =2
1000 1000
0 −λ 0 −2
λ e (2) e
(a) P (X = 0) = = = 0.1353
0! 0!
1
− (2000)
(b) P (X > 2000) = e −λx
=e 1000 = 0.1353

35).3 Memoryless Property

An even more interesting property of an exponential random variable is concerned with conditional probabilities
The exponential distribution is often used in reliability studies as the model for the time until failure of a device. The
memoryless property of the exponential distribution implies that the device randomly wears out, i.e., P (X > t + △t|X > t)
remains the same for any t.
However, the lifetime L of a device that suffers slow mechanical wear, such as bearing wear, is better modeled by a
distribution s.t. P (X > t + △t|X > t) increases with t, such as the Weibull distribution.
Show P (X > t + △t|X > t) = P (X > △t)

1. Define the Conditional Probability: show that P (X > s + t|X > t) = P (X > s) for all s, t ≥ 0, where X is an
exponential random variable with parameter λ
P (X > s + t ∩ X > t)
2. Use the Definition of Conditional Probability: P (X > s + t|X > t) =
P (X > t)
3. Simplify the Numerator: X > s + t=⇒X > t ⇒ (X > s + t ∩ X > t) = X > s + t
⇒ P (X > s + t ∩ X > t) = P (X > s + t)
4. Use the Exponential Distribution’s Survival Function: The survival function for an exponential distribution is
P (X > x) = e( − λx)
P (X > s + t ∩ X > t) eλ(s+t)
5. Substitute and Simplify: = −λt = e−λs = P (X > s)
P (X > t) e

Understanding Memoryless Property

To understand this property of exponential distribution, let us assume X models the life time of a light bulb.
The memoryless property tells you that given the fact that the light bulb still “survives” at time t, the probability it will
last longer than additional △t amount of time (the conditional probability) equals to the probability that it will last
longer than △t amount of time from the beginning (the unconditional probability).
The exponential distribution is the only continuous distribution with the “memoryless” property.

36) Gamma Function

Definition: The gamma function is defined as

Z ∞
Γ(α) = y α−1 e−y dy, where α > 0 (4..26)
0

Facts about the gamma function:

 Γ(1) = 1
 Γ(α) = (α − 1)Γ(α − 1) for any α > 1
 Γ(n) = (n − 1)!, n is an integer
A random variable Y has a gamma distribution if its pdf is
 y

1 −
e β,
 α−1
y y≥0

fY (y) = Γ(α)β α (4..27)

otherwise

0

Page 55 of 160
STATISTICS OVERVIEW

where the shape parameter α > 0 and the scale parameter β > 0.

Note

The kernel of a pdf fY (y) is the part that depends on y. The gamma pdf consists of a kernel and a “normalizing
constant.” This constant is free of y, but it forces the pdf to integrate to 1 over the support.

y y
Z ∞ −
Z ∞ −
1 α−1 β α−1 β
y e dy = 1=⇒ y e dy = Γ(α)β α
0 Γ(α)β α 0

36).1 Mean and Variance

If Y ∼ Gamma(α.β),

E(Y ) = αβ
(4..28)
var(Y ) = αβ 2

37) Weibull Distribution

The Weibull distribution is often used to model the time until failure of many different physical systems.
The random variable X with probability density function:

x β
β x β−1 −( )
fX (x) = ( ) e δ , for x ≥ 0
δ δ

is a Weibull random variable with scale parameter δ > 0 and shape parameter β > 0.
The shorthand notation is X ∼ Weibull(β, δ).
By changing the values of β and δ, the Weibull pdf can assume many shapes. Because of this flexibility (and for other
reasons), the Weibull distribution is very popular among engineers in reliability applications.

Page 56 of 160
STATISTICS OVERVIEW

37).1 cdf of Weibull Distribution

If X has a Weibull distribution with parameters δ and β, then the cumulative distribution function of X is

x β
−( )
FX (x) = P (X ≤ x) = 1 − e δ , for x ≥ 0 (4..29)

It follows that
x β
−( )
P (X > x) = e δ , (4..30)

The survival function of a Weibull distribution

t β
−( )
S(t) = e δ (4..31)

37).2 Mean and Variance

1
E(X) = δΓ(1 + )
β
(4..32)
2 1
var(X) = δ 2 [Γ(1 + ) − (Γ(1 + ))2 ]
β β

Example: Bearing Wear

The time to failure (in hours) of a bearing in a mechanical shaft is satisfactorily modeled as a Weibull random variable
1
with β = and δ = 5000 hours. Determine the probability that a bearing lasts at least 6000 hours.
2
Solution
1
x β 6000
−( ) −( )2
P (X ≥ 10) = e δ = e 5000 = 0.334

Example: Battery Lifetime

The lifetime of a rechargeable battery under constant usage conditions, denoted by T (measured in hours), follows a
Weibull distribution with parameters β = 2 and δ = 10.
(a) What is the probability that a battery is still functional at time t = 20?
(b) What is the probability that a battery is still functional at time t = 20 given that the battery is functional at time
t = 10?

Solution

Page 57 of 160
STATISTICS OVERVIEW

20 2
−( )
(a) P (T > 20) = e 10 = 0.0183
10 2
−( )
(b) P (T > 10) = e 10 = 0.3679
P (T > 20 ∩ T > 10) P (T > 20) 0.0183
P (T > 20|T > 10) = = = = 0.04974
P (T > 10) P (T > 10) 0.3679

37).3 Summary
Suppose that X ∼ Weibull(β, δ)
x β
β x β−1 −( )
 pdf: fX (x) = ( ) e δ for x ≥ 0
δ δ
dweibull(x, β, δ) in R
x β
−( )
 CDF: FX (x) = 1 − e δ for x ≥ 0
pweibull(x, β, δ) in R
1
 Mean: E(X) = δΓ(1 + )
β
2 1
 Variance: var(X) = δ 2 [Γ(1 + ) − (Γ(1 + ))2 ]
β β
where Γ is called gamma function and Γ(n) = (n − 1)! if n is a positive integer.

38) Chi-squared Distribution (χ2 Distribution)

For any integer ≥ 1, a random variable Y has a χ2 distribution with ν degrees of freedom if Y is a gamma random
ν
variable with α = and β = 2.
2
ν
E(Y ) = αβ = (2) = 2
2 (4..33)
ν 2
var(Y ) = αβ 2 = (2 ) = 2ν
2

Page 58 of 160
STATISTICS OVERVIEW

CHI-SQUARED (χ2 ) DISTRIBUTION CRICTICAL VALUES

α
ν 0.995 0.99 0.975 0.95 0.90 0.10 0.05 0.025 0.01 0.005
1 0.000 0.000 0.001 0.004 0.016 2.706 3.841 5.024 6.635 7.879
2 0.010 0.020 0.061 0.103 0.211 4.605 5.991 7.378 9.210 10.597
3 0.072 0.115 0.216 0.352 0.584 6.251 7.815 9.348 11.344 12.837
4. 0.207 0.291 0.484 0.711 1.064 7.779 9.488 11.143 13.277 14.860
5. 0.412 0.554 0.831 1.145 1.610 9.236 11.070 12.832 15.085 16.748

6. 0.676 0.872 1.237 1.635 2.204 10.645 12.592 14.440 16.812 18.548
7. 0.989 1.239 1.690 2.167 2.833 12.017 14.067 16.012 18.474 20.276
8. 1.344 1.646 2.180 2.733 3.490 13.362 15.507 17.534 20.090 21.954
9. 1.735 2.088 2.700 3.325 4.168 14.684 16.919 19.022 21.665 23.587
10. 2.156 2.558 3.247 3.940 4.865 15.987 18.307 20.483 23.209 25.188

11. 2.603 3.053 3.816 4.575 5.578 17.275 19.675 21.920 24.724 26.755
12. 3.074 3.571 4.404 5.226 6.304 18.549 21.026 23.337 26.217 28.300
13. 3.565 4.107 5.009 5.892 7.042 19.812 22.362 24.736 27.688 29.819
14. 4.075 4.660 5.629 6.571 7.790 21.064 23.685 26.119 29.141 31.319
15. 4.601 5.229 6.262 7.261 8.547 22.307 24.996 27.488 30.578 32.801

16. 5.142 5.812 6.908 7.962 9.312 23.542 26.296 28.845 32.000 34.267
17. 5.697 6.408 7.564 8.672 10.085 24.769 27.587 30.191 33.409 35.718
18. 6.285 7.015 8.231 9.390 10.865 25.989 28.869 31.526 34.805 37.156
19. 6.844 7.633 8.907 10.117 11.651 27.204 30.144 32.852 36.191 38.582
20. 7.434 8.260 9.591 10.851 12.443 28.412 31.410 34.170 37.566 39.997

21. 8.034 8.897 10.283 11.591 13.240 29.615 32.671 35.489 38.932 41.401
22. 8.643 9.542 10.982 12.338 14.041 30.813 33.924 36.781 40.289 42.796
23. 9.260 10.196 11.689 13.091 14.848 32.007 35.172 38.076 41.638 44.181
24. 9.886 10.856 12.401 13.848 15.659 33.196 36.415 39.364 42.980 45.559
25. 10.520 11.524 13.120 14.611 16.473 34.382 37.652 40.646 44.314 46.928

26. 11.160 12.198 13.844 15.379 17.292 35.573 38.885 41.923 45.642 48.290
27. 11.808 12.879 14.573 16.151 18.114 36.741 40.113 43.195 46.963 49.645
28. 12.461 13.565 15.308 16.928 18.939 37.916 41.337 44.461 48.278 50.993
29. 13.121 14.256 16.047 17.708 19.768 39.087 42.557 45.722 49.588 52.336
30. 13.787 14.953 16.791 18.493 20.599 40.256 43.773 46.979 50.892 53.672

40. 20.707 22.164 24.433 26.509 29.051 51.805 55.758 59.342 63.691 66.766
50. 27.991 29.707 32.357 34.764 37.689 63.167 67.505 71.420 76.154 79.490
60. 35.534 37.485 40.482 43.188 46.459 74.397 79.082 83.298 88.379 91.962
70. 43.275 45.442 48.758 51.739 55.329 85.527 90.531 95.023 100.425 104.215
80. 51.172 53.540 57.153 60.391 64.278 96.578 101.879 106.629 112.329 116.321

90. 59.196 61.754 65.647 69.126 73.291 107.565 113.145 118.136 124.116 128.299
100. 67.328 70.065 74.222 77.929 82.358 118.498 124.342 129.561 135.807 140.169

Table 4..3: Chi-squared (χ2 ) Distribution Table

Page 59 of 160
STATISTICS OVERVIEW

5. JOINT PROBABILITY DISTRIBUTIONS AND RANDOM SAMPLES

39) Joint pmf

Definition: The joint probability mass function (joint pmf) of two discrete random variables Y1 and Y2 is

f (y1 , y2 ) = P (Y1 = y1 , Y2 = y2 ) (5..1)

The joint (bivariate) pmf gives the probability that the pair (Y1 , Y2 ) takes a specific pair of values (y1 , y2 ).
If Y1 and Y2 are discrete random variables with the joint pmf f (y1 , y2 ), then:
(i) f (y1 , y2 ) ≥ 0 for all y1 , y2
X
(ii) f (y1 , y2 ) = 1 where we sum over all possible (y1 , y2 ) pairs
y1 ,y2
Example: Tornados
Tornados are natural disasters that cause millions of dollars in damage each year. An actuary determines that the annual
numbers of tornadoes in two Iowa counties (Lee and Van Buren) are jointly distributed as indicated in the table below.
Let Y1 and Y2 denote the number of tornados seen each year in Lee and Van Buren counties, respectively

f (y1 ,2 ) y2 = 0 y2 = 1 y2 = 2 y3 = 3
y1 = 0 0.12 0.06 0.05 0.02
y1 = 1 0.13 0.15 0.12 0.03
y1 = 2 0.05 0.15 0.10 0.02

(a) What is the probability that there is no more than one tornado seen in the two counties combined?
(b) What is the probability that there are two tornadoes in Lee County?

Solution

(a) P (Y1 + Y2 ≤ 1) = f (0, 0) + f (1, 0) + f (0, 1) = 0.12 + 0.13 + 0.06 = 0.31

(b) P (Y1 = 2, Y2 ) = f (2, 0) + f (2, 1) + f (2, 2) + f (2, 3) = 0.05 + 0.15 + 0.10 + 0.02 = 0.32

40) Join cdf

Definition: The (bivariate) joint cdf of the random vector (Y1 , Y2 ) is

F (y1 , y2 ) = P (Y1 ≤ y1 , Y2 ≤ y2 ) (5..2)

If (Y1 , Y2 ) are jointly discrete,

X X
F (y1 , y2 ) = f (t1 , t2 )
t1 ≤y1 t2 ≤y2

41) Join pdf

Definition: Let Y1 and Y2 be jointly continuous random variables with joint cdf F (y1 , y2 ). If there is a nonnegative
function f (y1 , y2 ) such that Z y1 Z y2
F (y1 , y2 ) = f (t1 , t2 )dt1 dt2 (5..3)
−∞ −∞

then f (y1 , y2 ) is called the joint probability density function (joint pdf) of (y1 , y2 ).
 If Y1 and Y2 are continuous random variables with the joint pdf f (y1 , y2 ), then:
(i) f (y1 , y2 ) ≥ 0 for all y1 , y2

Page 60 of 160
STATISTICS OVERVIEW
Z ∞ Z ∞
(ii) f (y1 , y2 )dy1 dy2 = 1
−∞ −∞
Example: Completing Tasks
Suppose the time (in hours) to complete task 1 and  task 2 for a random employee has the joint pdf:
e−(y1 +y2 ) , y > 0, y > 0
1 2
f (y1 , y2 ) =
0 otherwise
Find the probability that a random employee takes less than 2 hours on task 1 and between 1 and 3 hours on task 2.
Solution
Z 3 Z 2 Z 3 Z 3
P (0 < y1 < 2, 1 < y2 < 3) = e−(y1 +y2 ) dy1 dy2 = e−(y1 +y2 ) |20 dy2 = [e−(2+y2 ) − e−(0+y2 ) ]dy2
1 0 1 1
Z 3
⇒ P (0 < y1 < 2, 1 < y2 < 3) = [e−(2+y2 ) − e−y2 ]dy2 = e−(2+y2 ) − e−y2 |31 = [e−(2+3) − e−3 ] − [e−(2+1) − e−1 ]
1
⇒ P (0 < y1 < 2, 1 < y2 < 3) = 0.275

42) Marginal pmf

Definition: If Y1 , Y2 are jointly discrete random variables with joint pmf f (y1 , y2 ), then the marginal pmf of Y1 is
X
f (y1 ) = f (y1 , y2 ) (5..4)
y2

Similarly, the pmf Y2 is

X
f (y2 ) = f (y1 , y2 ) (5..5)
y1

43) Marginal pdf

Definition: If Y1 , Y2 are jointly continuous random variables with joint pdf f (y1 , y2 ), then the marginal pdf of Y + 1
is Z ∞
f (y1 ) = f (y1 , y2 )dy2 (5..6)
−∞

Similarly, the pdf Y2 is Z ∞

f (y2 ) = f (y1 , y2 )dy1 (5..7)
−∞

Example: Suppose we model two proportions  Y1 and Y2 with the joint pdf
0.4(y + 0.4y ) 0 < y < 1, 0 < y < 1
1 2 1 2
f (y1 , y2 ) =
0 elsewhere
(a) Find the marginal pdf of Y1 .
(b) Find P (Y1 > 0.3).
Solution
Z 1 Z 1
(a) f (y1 ) = 0.4(y1 + 4y2 )dy2 = 0.4 (y1 + 4y2 )dy2 = 0.4(y1 y2 + 2y22 )|10 = 0.4(y1 + 2)
0 Z 1 0
y2 12 0.32
(b) P (Y1 > 0.3) = 0.4(y1 + 2)dy1 = 0.4( 1 + 2y1 )|10.3 = 0.4[ + 2(1) − − 2(0.3)] = 0.742
0.3 2 2 2

Example: Suppose the joint pdf (X, Y ) is given

 by
 6 (x + y 2 ) 0 ≤ x ≤ 1, 0 ≤ y ≤ 1
f (x, y) = 5
0 elsewhere

Page 61 of 160
STATISTICS OVERVIEW

Find the marginal pdfs, f (x) and f (y).

Solution
1
6 1 y3
Z Z
6 6 6 1
f (x) = (x + y 2 )dy = (x + y 2 )dy = (xy + )|10 = (x + )
5 5 0 5 3 5 3
Z 01
6 1 6 x2
Z
6 6 2 1
f (y) = (x + y 2 )dx = (x + y 2 )dx = 2 1
( + xy )|0 = (y + )
0 5 5 0 5 2 5 2

44) Conditional pmf/pdf

Definition: If Y1 , Y2 are jointly discrete/continuous random variables with joint pmf/pdf f (y1 , y2 ), then the condi-
tional pmf/pdf of Y1 given Y2 = y2 is:
f (y1 , y2 )
f (y1 |y2 ) = (5..8)
f (y2 )
provided that f (y2 ) > 0. Similarly, the conditional pmf/pdf of Y2 given Y1 = y1 is:

f (y1 , y2 )
f (y2 |y1 ) = (5..9)
f (y1 )

provided that f (y1 ) > 0

44).1 Properties of Conditional pdf

The conditional pdf itself is a pdf, so the following properties are satisfied:
(i) fZ(y1 |y2 ) ≥ 0
∞
(ii) f (y1 |y2 )dy1 = 1
−∞

Note

For the conditional pmf the integral is just replaced by summation.

44).2 Conditional Mean and Variance

The conditional mean of Y2 given Y1 = y1 , denoted as E(Y2 |y1 ), is
Z ∞
E(Y2 |y1 ) = y2 f (y2 |y1 )dy2 (5..10)
−∞

and the conditional variance of Y2 given Y1 = y1 , denoted as var(Y2 |y1 ) is

var(Y2 |y1 ) = E ( Y22 |y1 ) − [E(Y2 |y1 )]2 (5..11)

45) Independence of random variables

Theorem: If Y1 , Y2 are jointly discrete/continuous random variables with pmf/pdf’s f1 (y1 ) and f2 (y2 ) and joint pdf
f (y1 , y2 ), then Y1 and Y2 are independent if and only if

f (y1 , y2 ) = f1 (y1 )f2 (y2 ) for all y1 and y2 (5..12)

Page 62 of 160
STATISTICS OVERVIEW

46) Expected value of a Function of Two random variables

Definition: If Y1 , Y2 are jointly discrete/continuous r.v.’s with pmf/pdf f (y1 , y2 ). Then for some function g(Y1 , Y2 ):
X X


 g(y1 , y2 )f (y1 , y2 ) discrete
y
E{g(Y1 , Y2 )} = Z Z
1 y2 (5..13)
g(y1 , y2 )f (y1 , y2 )dy1 dy2 continuous




Example: Suppose that the lifetime of two components are independent of one another and that the first lifetime,
1
X1 , has an exponential distribution with λ1 = , where as the second, X2 , has an exponential distribution with
1000
1
λ2 = .
1200
What is the probability that both component lifetimes are at least 1500 hours?

Solution
1 1
−λ1 x1 −λ2 x2
−( )(1500) −( )(1500)
P (X1 ≥ 1500, X2 ≥ 1500) = f1 (X1 ≥ 1500)f2 (X2 ≥ 1500) = e e =e 1000 e 1200
⇒ P (X1 ≥ 1500, X2 ≥ 1500) ≈ 0.06393

Example:
 6 (x + y 2 ) 0 ≤ x ≤ 1, 0 ≤ y ≤ 1


f (x, y) = 5
0 elsewhere
(a) Compute that conditional pdf of Y given that X = 0.8.
(b) Compute the conditional mean of Y given that X = 0.8.

Solution
1 1
y3
Z Z
6 6 6 6 1
f (x) = (x + y 2 )dy = (x + y 2 )dy = (xy + )|10 = (x + )
0 5 5 0 5 3 5 3
6
f (y, 0.8) (0.8 + y 2 ) 0.8 + y 2 2.4 + 3y 2
(a) f (Y |X = 0.8) = = 5 = =
fX (0.8) 6 1 1 3.4
(0.8 + ) 0.8 +
Z 1 5 3 Z 1 3 Z 1
2.4 + 3y 2 1 5 3
(b) E(Y |X = 0.8) = yf (Y |X = 0.8)dy = y dy = (2.4y + 3y 3 )dy = (1.2y 2 + y 4 )|10 = 0.5735
0 0 3.4 3.4 0 17 4

47) Covariance

Definition: If Y1 and Y2 are random variables with respective means µ1 and µ2 , then:

cov(Y1 , Y2 ) = E{(Y1 − µ1 )(Y2 − µ2 )} = E(Y1 Y2 ) − µ1 µ2 (5..14)

 Interpretation:
(i) cov(Y1 , Y2 ) > 0 =⇒Y1 and Y2 are positively linearly related.
(ii) cov(Y1 , Y2 ) < 0 =⇒Y1 and Y2 are negatively linearly related.
(iii) cov(Y1 , Y2 ) = 0 =⇒ NO linear relationship between Y1 and Y2 .

47).1 Properties
If X1 , X2 , X3 and X4 are real-valued random varaibles, a, b, c, and d are real-valued constants:
 cov(X1 , a) = 0

Page 63 of 160
STATISTICS OVERVIEW

 cov(X1 , X1 ) = var(X1 )
 cov(X1 , X2 ) = cov(X2 , X1 )
 cov(aX1 , bX2 ) = abcov(X1 , X2 )
 cov(X1 + a, X2 + b) = cov(X1 , X2 )
 cov(aX1 + bX2 , cX3 + dX4 ) = accov(X1 , X3 ) + adcov(X1 , X4 ) + bccov(X2 , X3 ) + bdcov(X2 , X4 )
47).2 Independence and Covariance
Theorem: If Y1 , Y2 are independent, then
cov(Y1 , Y2 ) = 0 (5..15)

Note

The converse is not true.

Example: Let Y1 ∼ U (−1, 1) and let Y2 = Y12 ⇒ cov(Y1 , Y2 ) = 0 but Y1 and Y2 are obviously dependent

48) Correlation

It is hard to judge the strength of the linear association using the covariance since the covariance depends on the
scale of measurement.
We often use a standardized version called the correlation coefficient (denoted by ρ):

cov(Y1 , Y2 )
ρ= p p (5..16)
var(Y1 ) var(Y2 )

It is always true that 1 ≤ ρ ≤ 1, so values of ρ near 1 (or near −1) indicate a strong positive (or negative) linear associ-
ation.

49) Linear functions of Random Variables

Theorem: Consider random variables Y1 , ..., Yn and X1 , ...Xm such that E(Yi2 ) < ∞ and E(Xj2 ) < ∞ for all i, j. For
n
X n
X
constant a1 , ..., an and b1 , ...bm define U1 = ai Yi and U2 = bj Xj . Then
i=1 j=1

n
X
E(U1 ) = E(Yi ) = a1 E(Y1 ) + ... + an E(Yn )
i=1
n
X XX
var(U1 ) = a2i var(Yi ) + 2 ai aj cov(Yi , Yj )
i=1 i<j
n (5..17)
X XX
var(U1 ) = a2i var(Yi ) + ai aj cov(Yi , Yj )2
i=1 i̸=j
n X
X m
cov(U1 , U2 ) = ai bj cov(Yi , Xj )
i=1 j=1

Example: Achievement Tests

Achievement tests are commonly seen in educational or employment settings. For a large population of test-takers,
let Y1 , Y2 , and Y3 represent scores for different parts of an exam. Suppose that Y 1 ∼ N (12, 4), Y2 ∼ N (16, 9), and
Y3 ∼ N (20, 16). Suppose additionally that Y1 and Y2 are independent, cov(Y1 , Y3 ) = 0.8, and cov(Y2 , Y3 ) = 6.7. Two
different summary measures are computed to assess a subject’s performance:
U1 = 0.5Y1 − 2Y2 + Y3

Page 64 of 160
 STATISTICS OVERVIEW

U2 = 3Y1 − 2Y2 − Y3
(a) Find E(U1 ) and var(U1 )
(b) Find cov(U1 , U2 )
Solution
Y1 ∼ N (12, 4) ⇒ µ1 = 12, σ12 = 4
Y2 ∼ N (16, 9) ⇒ µ2 = 16, σ22 = 9
Y3 ∼ N (20, 16) ⇒ µ3 = 20, σ32 = 16
Y1 and Y2 are independent ⇒ cov(Y1 , Y2 ) = 0
(a) E(U1 ) = 0.5E(Y1 ) − 2E(Y2 ) + E(Y3 ) = 0.5(12) − 2(16) + 20 = −6
var(U1 ) = (0.5)2 (4)+(−2)2 (9)+(1)2 (16)+2(0.5)(−2)cov(Y1 , Y2 )+2(0.5)(1)cov(Y1 , Y3 )+2(−2)(1)cov(Y2 , Y3 )
⇒ var(U1 ) = (0.5)2 (4) + (−2)2 (9) + (1)2 (16) + 2(0.5)(−2)(0) + 2(0.5)(1)(0.8) + 2(−2)(1)(6.7) = 27
(b) cov(U1 , U2 ) = (0.5)(3)cov(Y1 , Y1 ) + (0.5)(−2)cov(Y1 , Y2 ) + (0.5)(−1)cov(Y1 , Y3 ) + (−2)(3)cov(Y2 , Y1 )
+(−2)(−2)cov(Y2 , Y2 ) + (−2)(−1)cov(Y2 , Y3 ) + (1)(3)cov(Y3 , Y1 ) + (1)(−2)cov(Y3 , Y2 ) + (1)(−1)cov(Y3 , Y3 )
⇒ cov(U1 , U2 ) = (0.5)(3)(σ12 )+(0.5)(−2)(0)+(0.5)(−1)(0.8)+(−2)(3)(0)+(−2)(−2)(σ22 )+(−2)(−1)(6.7)+
(1)(3)(0.8) + (1)(−2)(6.7) + (1)(−1)(σ32 )
⇒ cov(U1 , U2 ) = (0.5)(3)(4) + (0.5)(−1)(0.8) + (−2)(−2)(9) + (−2)(−1)(6.7) + (1)(3)(0.8) + (1)(−2)(6.7) +
(1)(−1)(16) = 1.2

Example: Let X and Y be discrete random

 variables with joint pmf
0.25 (x, y) = (−4, 1), (4, −1), (2, 2), (−2, −2)
p(x, y) =
0 elsewhere
Compute E(X), E(Y ) and cov(X, Y )
Solution
E(X) = (0.25)(−4) + (0.25)(4) + (0.25)(2) + (0.25)(−2)
E(Y ) = (0.25)(1) + (0.25)(−1) + (0.25)(2) + (0.25)(−2) = 0
E(XY ) = (−4)(1)(0.25) + (4)(−1)(0.25) + (2)(2)(0.25) + (−2)(−2)(0.25) = 0
cov(X, Y ) = E(XY ) − E(X)E(Y ) = 0 − (0)(0) = 0

50) Revision

In Chapter 3, we learned the discrete probability distributions, including Bernoulli, Binomial, Geometric, Negative
Binomial, Hypergeometric, and Poisson.
In Chapter 4, we learned the continuous probability distributions, including Exponential, Weibull, and Normal.
In chapter 3 and 4, we always assume that we know the parameter of the distribution. For example, we know the mean
µ and variance σ 2 for a normal distributed random variable, so that we can calculate all kinds of probabilities with them.
For example, suppose we know the height of 18-year-old US male follows N (µ = 176.4, σ 2 = 9) in centimeters.
Let Y = the height of one 18-year-old US male.
P (Y > 180) = 1 − P (X ≤ 180)
1 > 1 − pnorm ( 1 8 0 , 1 7 6 . 4 , 3 ) # x = 1 8 0 , m = 1 7 6 . 4 , s d = 3
2 [1] 0.1150697

However, it is natural that we do NOT know the population mean µ and population variance σ 2 in reality. What should
we do?
⇒ use statistical inference
Statistical inference deals with making (probabilistic) statements about a population of individuals based on informa-
tion that is contained in a sample taken from the population.

Page 65 of 160
STATISTICS OVERVIEW

50).1 Terminology: Population/Sample

A population refers to the entire group of “individuals” (e.g., people, parts, batteries, etc.) about which we would like
to make a statement (e.g., height probability, median weight, defective proportion, mean lifetime, etc.).
 Problem: Population can not be measured (generally)
 Solution: We observe a sample of individuals from the population to make a decision – i.e., statistical inference.
 We denote a random sample of observations by
Y1 , Y2 , ..., Yn
 n is the sample size
 Denote: y1 , y2 , ..., yn to be realizations

50).2 Terminilogy: Random Sample

The random variables Y1 , ..., Yn are a random sample of size n if
 the Yi ’s are independent random variable and
 every Yi has the same probability distribution,
 which is called i.i.d

50).3 Terminology: Parameter

A parameter is a numerical summary that describes a population. In general, population parameters are unknown.
Some very common examples:
 µ : population mean
 σ 2 : population variance
 σ : population standard deviation
 p : population proportion
Connection: all of the probability distributions that we talked about in previous chapter are indexed by population pa-
rameters.

50).4 Terminology: Statistic

A statistic is a numerical summary that can be calculated from a sample.
Suppose Y1 , Y2 , ..., Yn is a random sample from a population, some very common examples are:
 Sample mean:
n
1X
Ȳ = Yi (5..18)
n i=1

 Sample variance:
n
1 X
s2 = (Yi − Ȳ )2 (5..19)
n − 1 i=1

 Sample standard deviation:

√
s= s2 (5..20)

 Sample proportion:
n
1X
p̂ = Yi if Yi ’s are binary (5..21)
n i=1

50).5 Summary
The table below succinctly summarizes differences between a population and a sample (a parameter and a statistic):
51) Propositions

Page 66 of 160
STATISTICS OVERVIEW

Comparison between paramters and statistics

Statistics Parameters
 describes a sample  describe a population
 always known  usually unknown
 random  fixed
 ex; X̄, s2 , s  ex: µ, σ 2 , σ

Proposition: Let X1 , ..., Xn be a random sample from a distribution with mean µ and standard deviation σ. Then

E(X) = µ
σ2 (5..22)
var(X̄) =
n
n
X
We define a new random variable: Y = Xi
i=1
Proposition: Let X1 , ..., Xn be a random sample from a normal distribution with mean µ and standard deviation σ.
Then
σ2
X̄ ∼ N (µ, ) (5..23)
n
n
X
We define a new random variable: Y = Xi
i=1
Example: The amount of a particular impurity in a batch of a certain chemical product is a random variable. with mean
4.0 g and standard deviation 1.5 g. If 50 batches are independently prepared, what is the approximate probability that
the sample average of amount of impurity X is between 3.5 g and 3.8 g?

Solution
√
σ 1.5 3 2
s= √ = √ = = 0.2121
n 50 20
3.5 − 4.0 3.8 − 4.0
P (3.5 ≤ X ≤ 3.8) = P ( ≤Z≤ ) = P (−2.36 ≤ Z ≤ −0.94)
0.2121 0.2121
⇒ P (3.5 ≤ X ≤ 3.8) = P (Z ≤ −0.94) − P (Z ≤ −2.36) = 0.1736 − 0.0091 = 0.1645

52) Central Limit Theorem

THE MOST IMPORTANT THEOREM IN STATISTICS!

Central Limit Theorem: Suppose that Y1 , Y2 , ..., Yn is a random sample from a population distribution with mean µ and
variance σ 2 . When the sample size n is large, we have

σ2
Ȳ ∼ AN (µ, ) (5..24)
n

 AN is read as symptotically Normal

 It holds in a limiting sense, i.e. when n → ∞
The Central Limit Theorem (CLT) states that for a sample size greater than 30, the sample mean is approximately nor-
mally distributed.
If the population is normally distributed, the sample mean is normally distributed regardless of the sample size.

52).1 Simulation

Page 67 of 160
STATISTICS OVERVIEW

xx

Page 68 of 160
STATISTICS OVERVIEW

6. POINT ESTIMATION

Statistical inference deals with making (probabilistic) statements about a population of individuals based on informa-
tion that is contained in a sample taken from the population. We do this by
 estimating unknown population parameters with sample statistics.
 quantifying the uncertainty (variability) that arises in the estimation process.

53) Point estimators and Sampling distributions

Let θ denote a population parameter

A point estimator θ̂ is a statistic that is used to estimate a population parameter θ
Common examples of point estimators are:
 θ̂ = Ȳ =⇒ a point estimator for θ = µ
 θ̂ = s2 =⇒ a point estimator for θ = σ 2
 θ̂ = s =⇒ a point estimator for θ = σ

Note

In general, θ̂ is a statistic, the value of θ̂ will vary from sample to sample. Because a statistic is a function of random
variable

53).1 Terminology: Sampling Dsitribution

The distribution of a statistic is called a sampling distribution.
 A sampling distribution describes mathematically how a statistic would vary in repeated sampling
 What is a good estimator? In what sense is that good?

53).2 Evaluate an Estimator

Accuracy: We say that θ̂ is an unbiased estimator of θ if and onlu if

E(θ̂) = θ (6..1)

Note

If the estimator is not unbiased, then difference E(θ̂) − θ is called the bias of the estimator θ̂

When Y1 , ..., Yn is a random sample,

E(Ȳ ) = µ
(6..2)
E(s2 ) = σ 2

Precision: Suppose that θˆ1 and θˆ2 are unbiased estimators of θ. We would like to pick the estimator with smaller
variance, since it is more likely to produce an estimate close to the true value θ.

Page 69 of 160
STATISTICS OVERVIEW

We desire point estimators θ̂ which are unbiased (perfectly accurate) and have small variance (highly precise).
If θ̂ has minimum variance among all unbiased estimators, it is called the minimum variance unbiased estimator (MVUE)
of θ
The standard error of a point estimator θ̂ is defined as
q
se(θ̂) = var(θ̂) (6..3)

Note

smaller se(θ̂) ⇔ θ̂ more precise

Example: Of n1 randomly selected male smokers, X1 smoked filter cigarettes, whereas of n2 randomly selected female
smokers, X2 smoked filter cigarettes. Let p1 and p2 denote the probabilities that a randomly selected male and female,
respectively, smoke filter cigarettes.
X1 X2
(a) Show that − is an unbiased estimator for p1 − p2 .
n1 n2
(b) What is the standard error of the estimator above?
(c) How would you use the observed values x1 and x2 to estimate the standard error of your estimator?
(d) If n1 = n2 = 200, x1 = 127 and x2 = 176, use the estimator of part (a) to obtain an estimate of p1 − p2 .
(e) Use the result of part (c) and the data of part (d) to estimate the standard error of the estimator.

Solution

X1 X2 1 1 1 1
E(X) = E( − )= E(X1 ) − E(X2 ) = (n1 p1 ) − (n2 p2 ) = p1 − p2
n1 n2 n1 n2 n1 n2
X1 X2
⇒ − is an unbiased estimator of p1 − p2
n1 n2
X1 X2 1 1 1 1 p1 q1 p2 q2
(b) var(X) = var( − ) = ( )2 var(X1 ) + ( )2 var(X2 ) = ( )2 (n1 p1 q1 ) + ( )2 (n2 p2 q2 ) = +
n1 n
r2 n1 n2 n1 n2 n1 n2
p p1 q1 p2 q2
⇒ se(X) = var(X) = +
n1 n2
xi
(c) The values x1 and x2 can be plugged in the se(X) by replacing pi = with i = 1, 2 and qi = 1 − pi . However
ni
the values of n1 and n2 are compulsory to have. If the evaluations of n1 and n1 are lacked, it’s impossible to get a
realistic estimate. v
u X1 X1 X2 X2
u
(1 − ) (1 − )
r s
p1 q1 p2 q2 p1 (1 − p1 ) p2 (1 − p2 ) t n1
u
n1 n n2
se(X) = + = + = + 2
n1 n2 n1 n2 n1 n2

Page 70 of 160
STATISTICS OVERVIEW
v v
u X1 n1 − X12 X2 n2 − X22
u
u X1 X1 2 X2 X2 2
u
−( ) −( )
s
n21 n22 X1 n1 − X12 X2 n2 − X22
u u
t n1 n1 n2 n2 t
⇒ se(X) = + = + = +
n1 n2 n1 n2 n1 (n21 ) n2 (n2 )2
s
X1 n1 − X12 X2 n2 − X22
⇒ se(X) = +
n31 n32
X1 X2 127 176
(d) p = p1 − p2 = − = − = −0.245
s n 1 n 2 200 200
r
X1 n1 − X12 X2 n2 − X22 (127)(200) − (1272 ) (176)(200) − (176)2
(e) se(X) = 3 + 3 = + = 0.0411
n1 n2 2003 2003

Example: Consider a random sample X1 , ..., Xn from the pdf

f (x) = 0.5(1 + θx), −1 ≤ x ≤ 1
Show that θ̂ = 3X̄ is an unbiased estimator of θ.
Solution
1 1 1
x2 x3
Z Z Z
E(X) = xf (X)dx = 0.5x(1 + θx)dx = 0.5 (x + θx2 )dx = 0.5(
+ θ)|1−1
−1 −1 −1 2 3
12 13 (−1)2 (−1)3 1 θ 1 −θ 2θ θ
⇒ E(X) = 0.5[( + θ) − ( + θ)] = 0.5[( + ) − ( + )] = 0.5( ) =
2 3 2 3 2 3 2 3 3 3
⇒ θ = 3E(X)
From an unbiased estimator of θ, θ̂ = 3X̄ ⇒ E(3X̄) = 3E(X̄) = 3µ = θ
⇒ θ̂ = 3X̄ is an unbiased estimator of θ

54) The Methods of Moment

Definition: For k = 1, 2, ..., the k-th population moment is

E(X k ) (6..4)

The k−the sample moment is

n
1X k
X (6..5)
n i= i

Let X1 , ..., Xn be a random sample from a distribution f (x; θ1 , ..., θm ). Suppose that we want to estimate m unknown
parameters θ1 , ..., θm . The moments of moment (MOM) estimators θˆ1 , ..., θˆ2 are obtained by equating the first m sam-
ple moments to the first m population moments and solving for θ1 , ..., θm . Let u′k = E(X k ) be the k−the population
n
1X k
moment and hk =
′
X the k−the sample moment. The MOM procedure uses the following systems of m equations
n i=1 i
and m unknowns:
u′1 = h′1
u′2 = h′2
..
.
u′m = h′m
Example: Suppose X1 , ..., Xn is an iid sample from a U (0, θ) population distribution,vwhere θ > 0 is unknown.
Estimate θ using the methods of moment.
Solution
The method of moments estimator for the uniform distribution U (0, θ) is 2 times the sample mean
n
1X
The sample mean: X̄ = Xi
n i=1

Page 71 of 160
STATISTICS OVERVIEW

θ
For a uniform distribution U (0, θ), the theoretical mean is E(X) =
2
θ
The method of moments sets the sample mean equal to the theoretical mean: X̄ = ⇒ θ = 2X̄
2
⇒ the method of moments estimator for θ: θMOM = 2X̄

Example: Suppose X1 , ..., Xn is an iid sample from an exponential distribution with the parameter of λ. What is the
MOM estimator for λ?

Solution

The first sample moment is the sample mean.

n
1X
The sample mean: X̄ = Xi
n i=1
The first population moment is the expected value of X
1
The expected value of exponential distribution: E(X) =
λ
1 1
The method of moments sets the sample mean equal to the population mean: X̄ = ⇒ λ =
λ X̄
1
⇒ the method of moments estimator for λ: λMOM =
X̄

Example: Suppose X1 , ..., Xn is a random sample from a Gamma(α, β) population distribution, where both α > 0
and β > 0 are unknown. In this population-level model, there are m = 2 parameters, so we will need 2 equations to
find the MOM estimators.

Solution

The first population moment is the expected value of X: E(X) = αβ

The second population moment is the variance of X: var(X) = E(X 2 )−[E(X)]2 ↔ E(X)2 = var(X)+[E(X)]2
⇒ E(X 2 ) = αβ 2 + (αβ)2
n
1X
The first sample moment is the sample mean: X̄ = Xi
n i=1
n
1X 2
The second sample moment is the sample variance: (X̄)2 = X
n i=1 i
Themethod of moments sets the sample
 moment equal to the corresponding population mean
X̄
αβ = X̄ β =

 

n α
⇒ 2 2 1 X 2 ⇔ n
1X 2
 αβ + (αβ) = Xi αβ 2
+ (αβ)2
= X
n i=1 n i=1 i

 

n n n
2 2 2 2 1X 2 X̄ 2 2 X̄ 2 1X 2 X̄ 2 2 1X 2
 αβ + (αβ) = αβ + (αβ) = X ↔ α( ) + α ( ) = X ↔ + (X̄) = X
n i=1 i α α n i=1 i α n i=1 i
n
X̄ 2 1X 2 (X̄)2 (X̄)2 X̄ s2
↔ = Xi − (X̄)2 ↔ = s2 ⇒ α = 2 ⇒ β = =
α n i=1 α s (X̄)2 X̄
s2
(X̄)2 s2
⇒ the method of moments estimator for α and β: αMOM = 2 and βMOM =
s X̄

55) Maximum Likelihood Estimation

The method of maximum likelihood is the most common method to use when finding point estimators; i.e., when
estimating a population-level model. Maximum likelihood estimators are found by maximizing the likelihood function.
Suppose Y1 , ..., Yn is an iid sample from a population distribution denoted by pY (y|θ) or fY (y|θ), where θ (a scalar) is
an unknown population parameter. The likelihood function is denoted by L(θ|y) = L(θ|y1 , ..., yn ) is found as follows:
Y
 In the discrete case: L(θ|y) = pY (yi |θ)

Page 72 of 160
 STATISTICS OVERVIEW
n
Y
 In the continuous case: L(θ|y) = fY (yi |θ)
i=1
In the discrete case, the likelihood function L(θ|y) actually provides a (joint) probability. Suppressing pY (y|θ)’s depen-
dence on θ, note that
L(θ|y) = pY (y1 ) × pY (y2 ) × ... × pY (yn ) = P (Y1 = y1 )P (Y2 = y2 )...P (Yn = yn ) = P (Y1 = y1 , Y2 = y2 , ..., Yn = yn )
In this light, a casual (but useful) interpretation of the likelihood function is that it is “the probability of the data.” There-
fore, one can think of maximum likelihood estimates as those estimates which “maximize the probability of the data.”
Suppose Y1 , ..., Yn is an iid sample from a population distribution denoted by pY (y|θ) or fY (y|θ), where θ ∈ R is an
unknown population parameter (for now, we consider the scalar case). The maximum likelihood estimator (MLE) of θ
is the value of θ that maximizes the likelihood function L(θ|y); i.e.,

θ̂ = argmaxθ L(θ|y)

Therefore, we can find the MLE of θ by writing out the likelihood function, viewing it as a function of θ (as always), and
then maximizing it as a function of θ.
How do we maximize L(θ|y)?
For many scenarios, this reduces to a calculus problem. If L(θ|y) is a differentiable function of θ, then we can take the
derivative of θ and set it equal to zero; i.e.,
∂
L(θ|y) = 0 (6..6)
∂θ
Because the natural logarithm function is increasing, the value of θ that maximizes L(θ|y) is the same as the value of θ
that maximizes ln L(θ|y); i.e.,
θ̂ = argmaxθ L(θ|y) = argmaxθ ln L(θ|y)

Therefore, we can also take the derivative of the log-likelihood function ln L(θ|y) and set it equal to zero; i.e.,

∂
ln L(θ|y) = 0 (6..7)
∂θ

We then show that

∂
ln L(θ|y) < 0 (6..8)
∂θ
which verifies θ̂ maximizes ln L(θ|y) by the Second Derivative Test.
Example: Suppose Y1 , ..., Yn is an iid sample from a P oisson(λ) population distribution, where λ > 0 is unknown.
Derive the mle for λ.
Solution
λy e−λ
The pmf: p(y|λ) =
y!
n Pn n
Y λ i=1 yi e−nλ X
L(y1 , ..., yn |λ) = p(yi |λ) = ⇒ ln L(y1 , ..., yn |λ) = −nλ + ( yi ) ln λ − ln(y1 !...yn !)
i=1
y1 !...yn ! i=1
n
∂ 1X
⇒ ln L(y1 , ..., yn |λ) = −n + yi
∂λ λ i=1
n
1X
λ̂ is a critical point for the log likelihood if and only if λ̂ = yi .
n i=1
The second derivative test confirms this critical point is a maximum
n
1X
⇒ the maximum likelihood estimator for λ: λ̂ = Yi = Ȳ
n i=1

Page 73 of 160
STATISTICS OVERVIEW

55).1 The Invariance Principle

Let θˆ1 , ..., θˆm be the mle’s of the parameters θ1 , .., θm . Then the mle of any function h(θ1 , ..., θm ) of these parameters
is the function h(θˆ1 , ..., θˆm of the mle’s.
Example: Suppose Y1 , .., Yn is an iid sample from a N (µ, σ 2 ) population distribution where parameters −∞ < µ < ∞
and σ 2 > 0 are unknown.
(a) Derive mle’s for µ and σ 2
(b) What is the mle of σ?

Solution

(y − µ)2 n (yi − µ)2

1 −− Y 1 −
(a) The pdf of Normal distribution: f (y|µ, σ ) = √
2
e 2σ 2 2
L(µ, σ ) = √ e 2σ 2
2πσ 2 i=1 2πσ 2
n n
− 1 X
⇒ L(µ, σ 2 ) = (2πσ 2 ) 2 exp(− 2 (yi − µ)2 )
2σ i=1
n n n
− 1 X n n 1 X
L(µ, σ 2 ) = ln [(2πσ 2 ) 2 exp(− 2 (yi − µ)2 )] = − ln (2π) − ln σ 2 − 2 (yi − µ)2
2σ i=1 2 2 2σ i=1
n n n n
∂L 1 X 1 X X X
=− 2 2(yi − µ)(−1) = 0 ↔ 2 (yi − µ) = 0 ↔ (yi − µ) = 0 ↔ yi − nµ = 0
∂µ 2σ i=1 σ i=1 i=1 i=1
n n
X 1X
⇒ nµ = yi ⇒ µ = yi = ȳ
i=1
n i=1
n
∂L n 1 X
= − + (yi − µ)2 = 0
∂σ 2 2σ 2 2(σ 2 )2 i=1
n n n
1X
(yi − µ)2 = 0 ↔ −nσˆ2 + (yi − ȳ)2 = 0 ⇒ σˆ2 =
X X
Mutiply by 2(σ 2 )2 ⇒ −nσ 2 + (yi − ȳ)2 = 0
i=1 i=1
n i=1
p
(b) Use the invariance property: Since ˆ2 is the MLE of σ 2 ⇒ the MLE for σ is σˆ2
σ
v
u n
u1 X
⇒ σ̂ = t (yi − ȳ)2
n i=1

Page 74 of 160
STATISTICS OVERVIEW

7. STATISTICAL INTERVALS BASED ON A SINGLE SAMPLE

56) Introduction

Statistical inference can be classified as estimation problem and testing problem.

The goal of estimation is to make a proper guess of unknown parameter, e.g. population mean µ, population proportion
p, etc, using data.
The goal of testing is to examine whether the estimated value for the unknown parameter is good, or whether some
statistical argument is correct.

57) Inference on Population Proportion

The population proportion p emerges when the characteristic we measure on each individual is categorical, or sim-
ply binary (i.e., only two possible outcomes). Here are some examples:
 p : proportion of defective water filter
 p : proportion of HIV positive in MD
We can connect these binary outcomes to the Bernoulli trials for each individual in the sample:
 each trial results in only two possible outcomes, labeled as ”success” and ”failure”
 the trials are independent
 the probability of a success in each trial, denoted as p, remains constant. It follows that the probability of a failure in
each trial is 1 − p

58) Point Estimator of Proportion p

Suppose we define Y = the number of successes out of n sampled individuals, so Y ∼ B(n, p). A natural point
estimator for p, the population proportion, is
Y
p̂ = , (7..1)
n
the sample proportion

58).1 Property of p̂
 p̂ is unbiased estimator of p. That is,
E(p̂) = p

 To quantify the precision of p̂,

p(1 − p) pq
var(p̂) = =
n n
The asymptotic distribution of an estimator of p (p̂) generally approaches a normal distribution with a mean equal to
the true value of p and a variance that is inversely proportional to the sample size. This means that as the sample size
increases, the estimator’s distribution becomes more concentrated around the true value of p

58).2 Sampling Distribution of p̂

With the help of the central limit theorem, we can derive the asymptotic distribution of p̂. Recall that Y = the number
of successes out of n sampled individuals, and Y ∼ B(n, p). We can express Y as the sum of n independent Bernoulli
trials with success probability p. That is
n
X
Y = Xi
i=1

Page 75 of 160
STATISTICS OVERVIEW

i.i.d
where X ∼ Bernoulli(p).
E(Xi ) = p and var(Xi ) = p(1 − p)
n
Y X xi
It follows that p̂ = = . By the Central Limit Theorem:
n i=1
n

Y p(1 − p)
p̂ = = X̄ ∼ AN (p, ) (7..2)
n n

59).0 t Distribution

The t-distribution is a probability distribution similar to the standard normal distribution, but this one can be
done without knowing the standard deviation of the population. When comparing the normal distribution versus t-
distribution, we see that the graph t-distribution has tails which are bigger or fatter than the standard normal distribu-
tion, which means that there is a higher probability of finding a data point in one of the tails than when working with
a normal distribution. That is one reason t-distribution graphs look shorter than the normal distribution, because more
of its area is gone to the tails.

If we standardize Ȳ , we obtain
Ȳ − µ
Z= √ ∼ N (0, 1) (7..3)
σ/ n
However, population standard deviation σ is usually unknown. Replacing it with the sample standard deviation s,
we get a new sampling distribution:
Ȳ − µ
t = √ ∼ t(n − 1) (7..4)
s/ n
which is called t distribution with degrees of freedom ν = n − 1
The t distribution has the following characteristics:
 It is continuous and symmetric about 0
 It is indexed by a value ν called the degrees of freedom
 As ν −→ ∞, t(ν) −→ N (0, 1)
 When compared to the standard normal distribution, the t distribution in general, is less peaked and has more proba-
bility (area) in the tails, i.e., heavy tails

59).1 t Tables
What is t curve tail areas?
The ”tail area” of a t-curve refers to the region under the t-distribution curve that falls outside a specified range or
interval. It represents the probability of observing a t-statistic value more extreme than the specified range.

Page 76 of 160
STATISTICS OVERVIEW

Explanation:
Tails: The tails of the t-distribution are the regions on either side of the curve that extend out into the extreme ends.
These regions represent the probability of observing values that are significantly different from the mean
Tail Area: The area under the curve in the tails represents the probability of observing a t-statistic value that is more
extreme (either larger or smaller) than a specific value. This area is often used to determine the statistical significance
of a result, such as in a t-test.
How to determine the tail area?

1. Degrees of freedom: The shape of the t-distribution is determined by its degrees of freedom (df ), which is
typically related to the sample size. Smaller df values result in thicker tails.
2. Significance level (α): The significance level (α) is a threshold used to determine statistical significance. It
represents the probability of incorrectly rejecting the null hypothesis when it is true.
3. t-value: A t-value is a measure of how far a sample statistic is from the population parameter under the null
hypothesis.
4. t-distribution table: Use a t-distribution table to find the tail area corresponding to a given t-value and degrees
of freedom. The table typically provides the areas to right of a t-value

How to read t-tables?

1. Determine the Degrees of Freedom (df ): Degrees of freedom depend on sample size and the type of t-test
being used (one-sample, independent samples, paired samples)
2. Choose significance level (α): α represents the probability of rejecting the null hypothesis when it’s actually
true (type I error). Common α levels are 0.05, 0.01 or 0.10
3. Locate the t-table: Most t-tables have df listed in the leftmost column and α levels in the top row
4. Find the critical t-value:
 Two-tailed test: use the α level directly from the top row (e.g., 0.05)
 One-tailed test: Used in only one direction (e.g., greater than or less than), find the α level had been used for a
two-tail test (e.g., 0.05/2 = 0.025, in the top row)
5. Interpret the results: Compared your calculated t-statistic to the critical t-value
 If t-statistic is more extreme than the critical t-value (either larger in magnitude if the test is two-tailed, or larger
in the appropriate direction if the test is one-tailed), then your results are statistically significant.
 If t-statistic is less extreme than the critical t-value, then your results are not statistically significant.

There are 2 types of t distribution:

t-distribution, one-tailed area t-distribution, two-tailed area

Page 77 of 160
 STATISTICS OVERVIEW

t DISTRIBUTION TABLE
cum. prob. t0.50 t0.75 t0.80 t0.85 t0.90 t0.95 t0.975 t0.98 t0.99 t0.995 t0.9975 t0.999 t0.9995
one-tailed (α/2) 0.50 0.25 0.20 0.15 0.10 0.05 0.025 0.02 0.01 0.005 0.0025 0.001 0.0005
two-tailed (α) 1.00 0.50 0.40 0.30 0.20 0.10 0.05 0.04 0.02 0.01 0.005 0.002 0.001
df = n − 1
1 0.000 1.000 1.376 1.963 3.078 6.314 12.71 15.89 31.82 63.66 127.3 318.3 636.6
2 0.000 0.816 1.061 1.386 1.886 2.920 4.303 4.849 6.965 9.925 14.09 22.33 31.60
3 0.000 0.765 0.978 1.250 1.638 2.353 3.182 3.482 4.541 5.841 7.453 10.215 12.92
4 0.000 0.741 0.941 1.190 1.533 2.132 2.776 2.99 3.747 4.604 5.598 7.173 8.610
5 0.000 0.727 0.920 1.156 1.476 2.015 2.571 2.757 3.365 4.032 4.773 5.893 6.869

6 0.000 0.718 0.906 1.134 1.440 1.943 2.447 2.612 3.143 3.707 4.317 5.208 5.959
7 0.000 0.711 0.896 1.119 1.415 1.895 2.365 2.517 2.998 3.499 4.029 4.785 5.408
8 0.000 0.706 0.889 1.108 1.397 1.860 2.306 2.449 2.896 3.355 3.833 4.501 5.041
9 0.000 0.703 0.883 1.100 1.383 1.833 2.262 2.398 2.821 3.250 3.690 4.297 4.781
10 0.000 0.700 0.879 1.093 1.372 1.812 2.228 2.359 2.764 3.169 3.581 4.144 4.587

11 0.000 0.697 0.876 1.088 1.363 1.796 2.201 2.328 2.178 3.106 3.497 4.025 4.437
12 0.000 0.695 0.873 1.083 1.356 1.782 2.179 3.303 2.681 3.055 3.428 3.930 4.318
13 0.000 0.694 0.870 1.079 1.350 1.771 2.160 2.282 2.650 3.012 3.372 3.852 4.221
14 0.000 0.692 0.868 1.076 1.345 1.761 2.145 2.264 2.624 2.977 3.326 3.787 4.140
15 0.000 0.691 0.866 1.074 1.341 1.753 2.131 2.249 2.602 2.947 3.286 3.733 4.073

16 0.000 0.690 0.865 1.071 1.337 1.746 2.120 2.235 2.583 2.921 3.252 3.686 4.015
17 0.000 0.689 0.863 1.069 1.333 1.740 2.110 2.224 2.567 2.898 3.222 3.646 3.965
18 0.000 0.688 0.862 1.067 1.330 1.734 2.101 2.214 2.552 2.878 3.197 3.611 3.922
19 0.000 0.688 0.861 1.066 1.328 1.729 2.093 2.205 2.539 2.861 3.174 3.579 3.883
20 0.000 0.687 0.860 1.064 1.325 1.725 2.086 2.197 2.528 2.845 3.153 3.552 3.850

21 0.000 0.686 0.859 1.063 1.323 1.721 2.080 2.189 2.518 2.831 3.135 3.527 3.819
22 0.000 0.686 0.858 1.061 1.321 1.717 2.074 2.183 2.508 2.819 3.119 3.505 3.792
23 0.000 0.685 0.858 1.060 1.319 1.714 2.069 2.177 2.500 2.807 3.104 3.485 3.768
24 0.000 0.685 0.857 1.059 1.318 1.711 2.064 2.172 2.492 2.797 3.091 3.467 3.745
25 0.000 0.684 0.856 1.058 1.356 1.708 2.060 2.167 2.485 2.787 3.078 3.450 3.725

26 0.000 0.684 0.856 1.058 1.315 1.706 2.056 2.162 2.479 2.779 3.067 3.435 3.707
27 0.000 0.684 0.855 1.057 1.314 1.703 2.052 2.150 2.473 2.771 3.057 3.421 3.690
28 0.000 0.683 0.855 1.056 1.313 1.701 2.048 2.154 2.467 2.763 3.047 3.408 3.674
29 0.000 0.683 0.854 1.055 1.311 1.699 2.045 2.150 2.462 2.756 3.038 3.396 3.659
30 0.000 0.683 0.854 1.055 1.310 1.697 2.042 2.147 2.457 2.750 3.030 3.385 3.646

31 0.000 0.683 0.854 1.055 1.309 1.696 2.040 2.144 2.453 2.745 3.023 3.375 3.634
32 0.000 0.683 0.854 1.054 1.309 1.694 2.037 2.141 2.449 2.739 3.016 3.366 3.622
33 0.000 0.683 0.853 1.054 1.308 1.693 2.035 2.139 2.445 2.734 3.009 3.357 3.611
34 0.000 0.682 0.853 1.053 1.307 1.691 2.033 2.136 2.442 2.729 3.002 3.348 3.601
35 0.000 0.682 0.853 1.053 1.306 1.690 2.031 2.134 2.438 2.724 2.997 3.341 3.592

36 0.000 0.682 0.852 1.052 1.306 1.689 2.029 2.131 2.435 2.720 2.991 3.333 3.583
37 0.000 0.682 0.852 1.052 1.305 1.688 2.027 2.129 2.432 2.716 2.986 3.326 3.574
38 0.000 0.682 0.852 1.051 1.304 1.686 2.025 2.127 2.429 2.712 2.981 3.320 3.566
39 0.000 0.681 0.851 1.051 1.304 1.685 2.023 2.125 2.426 2.708 2.976 3.313 3.559
40 0.000 0.681 0.851 1.051 1.303 1.684 2.021 2.123 2.423 2.704 2.971 3.307 3.551

50 0.000 0.679 0.849 1.047 1.295 1.676 2.009 2.109 2.403 2.678 2.937 3.261 3.496
60 0.000 0.679 0.848 1.045 1.296 1.671 2.000 2.099 2.390 2.660 2.915 3.232 3.460
80 0.000 0.678 0.846 1.043 1.292 1.664 1.990 2.088 2.374 2.639 2.887 3.195 3.416
100 0.000 0.677 0.845 1.042 1.290 1.660 1.984 2.081 2.364 2.626 2.871 3.174 3.390
1000 0.000 0.675 0.842 1.037 1.282 1.646 1.962 2.056 2.330 2.581 2.813 3.098 3.300
∞ 0.000 0.674 0.841 1.036 1.282 1.645 1.960 2.054 2.326 2.576 2.807 3.090 3.291
(100 − α)% 0% 50% 60% 70% 80% 90% 95% 96% 98% 99% 99.5% 99.8% 99.9%
Page 78 of 160
Table 7..1: t-Distribution Table
 STATISTICS OVERVIEW

60) Confidence Interval (CI)

Using a point estimator ignores important information: namely, how variable the estimator is
To avoid this problem (i.e., to account for the uncertainty in the sampling procedure), we therefore pursue the topic of
interval estimation (also known as confidence intervals)
The main difference between a point estimate and an interval estimate is:
 a point estimate: one-shot guess at the value of the parameter; this ignores the variability in the estimate
 an interval estimate (i.e., confidence interval): the interval of values. It is formed by taking the point estimate and
then adjusting it downwards and upwards to account for the point estimate’s variability

60).1 Confidence Interval for proportion p

p(1 − p)
Recall that p̂ ∼ AN (p, )
n
Let us define zα be the upper α percentage point of the standard normal distribution, i.e., P (Z > zα ) = α
An approximate 100(1 − α)% confidence interval for p is
r r
p̂(1 − p̂) p̂(1 − p̂)
(p̂ − zα/2 , p̂ + zα/2 ) (7..5)
n n
r
p̂(1 − p̂)
 The quantity is called the standard error
n r
p̂(1 − p̂)
 The quantity E = zα/2 is called the margin of error
n
zα/2 2
 Sample size n to find a 100(1 − α)% confidence interval for p: n = ( ) p(1 − p)
E
Rule of thumb: To apply the Central Limit Theorem, we need np̂ ≥ 15 and n(1 − p̂) ≥ 15
Form of the interval:
point estimate ± zα/2 × standard error (7..6)

60).2 Interpretation of Confidence Interval

Suppose that we are interested in parameter p for certain population. We take a sample of size n and calculate the sample
Y
proportion p̂ = . A 95% confidence interval is given by
n

Point estimate ± 1.96 Standard Error (7..7)

The 95% confidence comes from the fact that if we repeated this experiment over and over again, approximately, 95%
of all samples would produce a confidence interval that contains the true proportion, and only 5% of the time would
the interval be in error.
We call 100(1 − α)% the confidence level and α the significance level

Confidence level 50% 60% 70% 75% 80% 85% 90% 95% 97% 98% 99% 99.5% 99.9%
Two-tailed (α) 0.500 0.400 0.300 0.250 0.200 0.150 0.100 0.050 0.030 0.020 0.010 0.005 0.001
One-tailed (α/2) 0.250 0.200 0.150 0.125 0.100 0.075 0.050 0.025 0.015 0.010 0.005 0.0025 0.0005
z-score 0.675 0.250 1.036 1.150 1.282 1.440 1.645 1.960 2.170 2.326 2.576 2.810 3.291

Example: Flu Outbreak

A certain type of flu outbreaks in northern part of the USA. The historical records shows that there are 7% of the
residences in Columbia carrying flu under usual condition. Researchers want to see whether there is an outbreak in
Columbia, there are 30 out of 250 randomly chosen people in the sample carrying flu.
Can we conclude that there is an outbreak in Columbia? Answer the following question using confidence interval
approach (95%).

Page 79 of 160
STATISTICS OVERVIEW

Solution
30
Sample proportion p̂ = = 0.12
250
Confidence level = 95% ⇒ zα/2 = 1.96
r r
p̂(1 − p̂) 0.12(1 − 0.12)
Standard error = = = 0.0205
n 250
95% confidence interval: [0.12 − 1.96(0.0205), 0.12 + 1.96(0.0205)] = (0.0798, 0.1602)
The 95% confidence interval (0.0798, 0.1602) does not include the historical flu rate of 7%
⇒ there is strong evidence to suggest that the true proportion of flu carriers in Columbia is higher than the historical
rate, indicating a potential outbreak. Since the lower bound of the confidence interval is above 7%, it’s highly likely
that the outbreak is not due to chance

Example: The fraction of defective integrated circuits produced in a photolithohraphy process is being studied. A
random sample of 300 circuits is tested, revealing 13 defectives.
(a) Calculate a 95% CI on the fraction of defective circuits.
(b) Calculate a 99% CI on the fraction of defective circuits.

Solution
13
Sample proportion: p̂ = = 0.0433
300
(a) Confidence level = 95% ⇒ zα/2 = 1.96
r r
0.0433(1 − 0.0433) 0.0433(1 − 0.0433)
95% confidence interval = (0.0433 − 1.96 , 0.0433 + 1.96 )
300 300
⇒ 95% confidence interval = (0.0203, 0.0663)
(b) Confidence level = 99% ⇒ zα/2 = 2.576
r r
0.0433(1 − 0.0433) 0.0433(1 − 0.0433)
99% confidence interval = (0.0433 − 2.576 , 0.0433 + 2.576 )
300 300
⇒ 99% confidence interval = (0.0130, 0.0736)

61) Inference of Population Mean µ

For a binary random variable, we have discussed how to estimate the population proportion using point estimate and
confidence interval.
Now, it’s time to move on to the case when the random variable is numerical and learn some methods to make an in-
ference for the population mean µ.

61).1 Sampling distribution of Ȳ

Recall sample mean Ȳ is a reasonable point estimator of the population mean
Suppose Y1 , Y2 , ..., Yn is a random sample from N (µ, σ 2 ). Then the sample mean Ȳ has the following sampling distri-
bution:
σ2
Ȳ ∼ N (µ, )
n
σ
The above result reminds us that Ȳ is an unbiased estimator of µ and se(Ȳ ) = √ .
n
Example:
 Let Y = time (in seconds) to react to brake lights during in-traffic driving.
 We assume Y ∼ N (µ = 1.5, σ 2 = 0.16). We call this the population distribution.
 Suppose that we take a random sample of n = 5 drivers, i.e., Y1 , ..., Y5 . What is the distribution of the sample mean
Ȳ ?

Solution

Page 80 of 160
STATISTICS OVERVIEW

σ2 0.16
var(Ȳ ) = = = 0.032
n 5 √ √
σ σ2 0.16
⇒ Distribution of sample mean Ȳ : Ȳ ∼ N (1.5, 0.032) with se(Ȳ ) = √ = √ = √ = 0.1789
n n 5

61).2 CI for population mean µ assuming normality and known σ 2

Recall if Y1 , Y2 , ..., Yn is a random sample from a N (µ, σ 2 ) distribution and σ 2 is known, then

Ȳ − µ
Z= √ ∼ N (0, 1) (7..8)
σ/ n

Similar to CI derivation in population proportion, a 100(1 − α)% CI for µ is given by

σ σ
(ȳ − zα/2 √ , ȳ + zα/2 √ ) (7..9)
n n

61).3 CI for population mean µ assuming normality and unknown σ 2

Recall if Y1 , Y2 , ..., Yn is a random sample from a N (µ, σ 2 ) distribution and σ 2 is unknown, then

Ȳ − µ
t= √ ∼ t(n − 1) (7..10)
S/ n

A 100(1 − α)% CI for µ is given by

S S
(ȳ − tn−1,α/2 √ , ȳ + tn−1,α/2 √ ) (7..11)
n n
where S is the sample standard deviation

61).4 CI for a population variance σ 2

Suppose that X1 , X2 , ..., Xn is a random sample from a N (µ, σ 2 ) distribution. The quantity

(n − 1)S 2
Q= ∼ χ2 (n − 1) (7..12)
σ2

A χ2 distribution with ν = n − 1 degree of freedom

The χ2 distribution has the following characteristics:
 It is continuous, skewed to the right, and always positive.
 It is indexed by a value ν called the degrees of freedom. In practice, ν is often an integer (related to the sample size n).
 The χ2 pdf formula is unnecessary for our purposes. R will compute χ2 probabilities and quantiles from the χ2
distribution

Page 81 of 160
STATISTICS OVERVIEW

A 100(1 − α)% CI for σ 2 is given by:

(n − 1)S 2 (n − 1)S 2
[ , ] (7..13)
χ2n−1,α/2 χ2n−1,1−α/2

Proof: Let χ2n−1,α/2 denote the upper α/2 quantile and χ2n−1,1−α/2 denote the lower α/2 quantile of the χ2 (n − 1)
distribution, satisfy
P (Q > χ2n−1,α/2 ) = α/2
P (Q < χ2n−1,1−α/2 ) = α/2
Unlike the N (0, 1) and t distributions, the χ distribution is not symmetric.
2

Because Q ∼ χ2 (n − 1):
1 − α = P (χ2n−1,1−α/2 < Q < χ2n−1,α/2 )
(n − 1)S 2
⇔ 1 − α = P [χ2n−1,1−α/2 < < χ2n−1,α/2 ]
σ2
1 σ2 1
⇔ 1 − α = P[ 2 < < 2 ]
χn−1,1−α/2 (n − 1)S 2 χn−1,α/2
(n − 1)S 2 (n − 1)S 2
⇔ 1 − α = P[ 2 < σ2 < 2 ]
χn−1,α/2 χn−1,1−α/2
Note

A 100(1 − α)% CI for population standard deviation σ:

s s
(n − 1)S 2 (n − 1)S 2
[ , ] (7..14)
χ2n−1,α/2 χ2n−1,1−α/2

Unlike the z and t confidence intervals for a population mean µ, the 2 interval for σ 2 (and for σ) is not robust to
departures from normality
 If the underlying population distribution is non-normal (non-Guassian), then the confidence interval formulas
for σ 2 and σ are not to be used.
It is very important to “check” the normality assumption with these interval procedures (use a qq-plot)

62) Prediction interval

Noting that
Ȳ − Yn+1
t= r ∼ tn−1 (7..15)
1
S 1+
n
where Yn+1 is a single future observation

Page 82 of 160
 STATISTICS OVERVIEW

A 100(1 − α)% prediction interval for Yn+1 is

r r
1 1
(ȳ − tn−1,α/2 S 1 + , ȳ + tn−1,α/2 S 1+ ) (7..16)
n n

Example: Pipe Diameter

Hollow pipes are used in an electrical wiring project. The manufacturer claims that the outside diameter of this pipe is
1.29 inches. The data below were collected by a design engineer. The data are measurements of Y , the outside diameter
of this type of pipe (measured in inches). These n = 25 pipes were randomly selected and measured in the same location.
1.296, 1.320, 1.311, 1.298, 1.315, 1.305, 1.278, 1.294, 1.311, 1.290,
1.284, 1.287, 1.289, 1.292, 1.301, 1.298, 1.287, 1.302, 1.304, 1.301,
1.313, 1.315, 1.306, 1.289, 1.291
(a) Based on past experience, the engineers assume a normal population distribution (for the pipe diameters) with known
population standard deviation 0.02. We want to find a 95% CI for µ, the mean pipe diameter.
(b) Assume the pipe diameters are normally distributed with unknown population variance. We want to find the 95%CI
for , the mean pipe diameter.

Solution
P25
Yi 1.296 + 1.320 + 1.311 + ... + 1.315 + 1.306 + 1.289 + 1.291 32.477
Ȳ = i=1 = = = 1.299
n 25 25
(a) Confidence level = 95% ⇒ zα/2 = 1.96
σ σ 0.02 0.02
95% confidence interval = (ȳ − zα/2 √ , ȳ + zα/2 √ ) = (1.299 − 1.96 √ , 1.299 + 1.96 √ ) = (1.291, 1.307)
n n 25 25
α
(b) 100(1 − α)% = 95% ⇒ α = 0.05 ⇒ = 0.025
2
Degree of freedom df = n − 1 = 25 − 1 = 24 ⇒ tn−1,α/2 = 2.064
P25
(yi − ȳ)2 (1.296 − 1.299)2 + ... + (1.291 − 1.299)2 0.002948
Sample variance s2 = i=1 = = = 0.0001228
n−1 √ √ 25 − 1 24
⇒ sample standard deviation s = s2 = 0.0001228 = 0.0111
S S 0.0111 0.0111
95% confidence interval = (ȳ − tn−1,α/2 √ , ȳ + tn−1,α/2 √ ) = (1.299 − 2.064 √ , 1.299 + 2.064 √ )
n n 25 25
⇒ 95% confidence interval = (1.294, 1.304)

63) Normal Quantile-Quantile (qq) Plots

Recall if Y1 , ..., Yn ) is a random sample from a N (µ, σ 2 ) distribution, then

Ȳ − µ
t = √ ∼ t(n − 1)
s/ n
An obvious question arises: “What if Y1 , ..., Yn are non-normal?”
Answer: The t distribution result still approximately holds. That is, the t distribution is robust to the normality assump-
tion.
How to assess the normal distribution assumption? Normal quantile-quantile (qq) plot
The plot is constructed as follows:

1. On the vertical axis, we plot the observed data, ordered from low to high
2. On the horizontal axis, we plot the (ordered) theoretical quantiles from the distribution assumed for the observed
data (here, norma)

For the pipe diameter data, below is the qq plot.

1 > qqnorm ( p i p e s , pch = 1 6 , main = ” ” )
2 > qqline ( pipes )

Page 83 of 160
STATISTICS OVERVIEW

64) Sample Size Determination

In a planning stage of an experiment or investigation, it is often of interest to determine how many individuals are
needed to construct a confidence interval with a given level of confidence.
For example, we might want to construct a 95 percent confidence interval for a population mean, so that the interval
length is no more than 5 units (e.g., days, inches, dollars, etc.).
Sample size determination is also associated with the practical issues like cost, time spent in data collection, personnel
training, etc
Suppose that Y1 , Y2 , ..., Yn is a random sample from a N (µ, σ 2 ) population with σ 2 known. Recall that a 100(1α)% CI
for µ is given by
σ
Ȳ ± zα/2 √ (7..17)
n
| {z }
B
σ
where B = zα/2 √ is called the margin of error
n
It is possible to determine the sample size n once we specify these three pieces of information:
 the value of σ 2 (or an educated guess at its value; e.g., from past information, etc.)
 the confidence level, 100(1 − α)%
 the margin of error, B.
Then we have
σ σzα/2 2
B = zα/2 ( √ ) ⇐⇒ n = ( ) (7..18)
n B
Example: In a bio-medical experiment, we would like to estimate the population mean, remaining life of healthy rats
that are given a certain dose of a toxic substance. Suppose that we would like to write a 95 percent confidence interval
with a margin of error equal to B = 2 days. From past studies, remaining rat lifetimes have been approximated by a
normal distribution with standard deviation = 8 days. How many rats should we use for the experiment?

Solution

Confidence level = 95% ⇒ zα/2 = 1.96

σzα/2 2 (8)(1.96) 2
Sample size n = ( ) =[ ] = 61.47 ≈ 62
B 2
⇒ we should use at least 62 rats for the experiment

Example: Assume that the helium porosity of coal samples taken from any particular seam is normally distributed with
true standard deviation 0.75.
(a) Compute a 95% CI for the true average porosity of a certain seam if the average porosity for 20 specimens from the
seam was 4.85.
(b) Compute a 98% CI for the true average porosity of a certain seam if the average porosity for 16 specimens from the

Page 84 of 160
STATISTICS OVERVIEW

seam was 4.56.

(c) How large a sample size is necessary if the width of the 95% interval is to be 0.4?
(d) What sample size is necessary to estimate true average porosity to within 0.2 with 99% confidence?

Solution

(a) Confidence level = 95% ⇒ zα/2 = 1.96

σ σ 0.75 0.75
95% confidence interval = (x̄ − zα/2 √ , x̄ + zα/2 √ ) = (4.85 − 1.96 √ , 4.85 + 1.96 √ ) = (4.521, 5.179)
n n 20 20
(b) Confidence level = 98% ⇒ zα/2 = 2.326
σ σ 0.75 0.75
98% confidence interval = (x̄ − zα/2 √ , x̄ + zα/2 √ ) = (4.56 − 2.326 √ , 4.56 + 2.326 √ ) = (4.124, 4.996)
n n 16 16
1 1
(c) W = 0.4 ⇒ B = W = (0.4) = 0.2
2 2
σzα/2 2 (0.75)(1.96) 2
n=( ) =[ ] = 54.02 ≈ 55 ⇒ there are at least 55 coal samples are necessary
B 0.2
(d) Confidence level = 99% ⇒ zα/2 = 2.576
σzα/2 2 (0.75)(2.326) 2
n=( ) =[ ] = 93.32 ≈ 94 ⇒ there are at least 94 coal samples are necessary
B 0.2

Note
1
The margin of error = Width
2
1
B= W (7..19)
2

Example: Suppose that a researcher is interested in the mean number of movies that American adults see in the movie
theater per year. Using a random sample of 16 adults, he finds that the mean for the sample is 6 and the standard
deviation is 2. (a) Find the degrees of freedom and t-score to calculate a 99% confidence interval.
(b) Find a 99% confidence interval for the population parameter of interest.
(c) Write a thorough interpretation of the confidence interval.
(d) Based on your confidence interval, is it plausible that the mean number of movies that American adults see in the
movie theater per year is 9?

Solution

(a) Degrees of freedom df = n − 1 = 16 − 1 = 15

α
Confidence level = 99% ↔ 100(1 − α)% = 99% ⇒ α = 0.01 ⇒ = 0.005
2
⇒ t0.005,15 = 2.947
(b) Confidence level = 99% ⇒ zα/2 = 2.576
σ σ 2 2
99% confidence interval = (x̄ − zα/2 √ , x̄ + zα/2 √ ) = (6 − 2.576 √ , 6 + 2.576 √ ) = (4.712, 7.288)
n n 16 16
(c) The confidence interval is the range of values that you expect your estimate to fall between a certain percentage
of the time if you run your experiment again or re-sample the population in the same way.
⇒ 99% of all samples would produce values fall between (4.712, 7.288) and only 1% will be error
(d) (4.712, 7.288) is the interval which all values fall outside will be unaccepted
Because 9 > 7.288 which is the upperbound of the interval
⇒ it is not plausible that the mean number of movies that American adults see in the movie theater per year is 9

Page 85 of 160
STATISTICS OVERVIEW

8. TESTS OF HYPOTHESES BASED ON A SINGLE SAMPLES

65) Statistical Hypothesis

65).1 Defintion
A statistical hypothesis is a statement about the parameters of one or more populations.
Example:
 The proportion of underweight milk is more than 3% in a local farm.
 More than 7% of the landings for a certain airline exceed the runway.
 The defective rate of the water filter is less than 5%.
 The rate of failing STAT 355 class is 10%.
Types of statistical hypothesis
 Null hypothesis (H0 ) – It states a hypothesis of no difference/agreement with the population of interest.
 Alternative hypothesis (Ha ) – The inverse of H0 . It states a hypothesis of disagreement with the population of interest.
In most situations, it reflect the proposition in the research hypothesis.

65).2 Relationship between confidence interval (CI) and hypothesis testing

 The region covered by a confidence interval is the region or area covered by the H0 , regarded as the acceptance re-
gion. If a test statistics value falls in this region, we accept H0 .
 The region outside the confidence interval at both tails for a two-tailed test, or at either tail for a one-tailed test is
regarded as the rejection region (critical region). If a test statistics value that falls in this region, we reject H0 and
accept Ha .

65).3 Testing procedure

1. State the null (H0 ) and alternative (Ha or H1 ) hypotheses
2. Collect the data and calculate test statistics assuming H0 is true
3. Assuming the H0 is true, calculate the p-value
4. Make a decision based on the p-value. We either reject H0 or fail to reject H0
 If the p-value is small, we reject the null hypothesis and conclude the alternative hypothesis.
 If the p-value is not small, we do not reject the null hypothesis and do not conclude the alternative hypothesis.
There is one remaining question, how small should p-value be to be considered as ”small”?
⇒ level of significance (denoted as α) is required, and α is determined before seeing the data (p-value)
 p-value < α ⇒ reject H0
 p-value ≥ α ⇒ do not reject H0

There are typically three ways to test hypotheses: the confidence interval approach, the critical value approach, and the
p-value approach.

Note
Hypothesis testing using confidence interval:
Construct a 100(1α)% confidence interval for the true proportion
 If the entire confidence interval lies above p0 , then the claim that Ha : p > p0 is supported.
 If the confidence interval includes p0 or is entirely below p0 , then the claim is not supported
 If the lower bound of this interval is greater than p0 , then the claim is supported.

66) Hypothesis Testing for the population proportion p

Page 86 of 160
STATISTICS OVERVIEW

To estimate the population proportion p, sample proportion p̂ as point estimator

When np̂ ≥ 5 and n(1 − p̂) ≥ 5), the Central Limit Theorem:

p(1 − p)
p̂ ∼ AN (p, ) (8..1)
n

To control the type I error, recalling that the type I error is under the assumption H0 is in fact true
So when H0 is true ⇒ p = p0 :
p0 (1 − p0 )
p̂ ∼ AN (p0 , ) (8..2)
n
Then
p̂ − p0
z0 = r ∼ AN (0, 1) (8..3)
p0 (1 − p0 )
n

Null hypothesis:
H0 : p = p 0
Alternative hypothesis:
Ha : p < p0
Ha : p > p0
Ha : p ̸= p0
The test statistics (Critical value approach)
1. Compute the test statistics:
p̂ − p0
z0 = r ∼ AN (0, 1)
p0 (1 − p0 )
n
2. Find the corresponding rejection region:
Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis Ha : p < p0 z0 < −zα
Right-tail hypothesis Ha : p > p0 z0 > zα
Two-tail hypothesis Ha : p ̸= p0 z0 > zα/2 or z0 < −zα/2

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .
P -value approach

p̂ − p0
z0 = r ∼ AN (0, 1)
p0 (1 − p0 )
n
2. Find the corresponding rejection region:
Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis Ha : p < p0 p-value = P (z < z0 ) = Φ(z0 ) < α
Right-tail hypothesis Ha : p > p0 p-value = P (Z > z0 ) = 1 − Φ(z0 ) < α
Two-tail hypothesis Ha : p ̸= p0 p-value = 2P (Z < −|z0 |) = 2[1 − Φ(|z0 |)] < α

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .

Example: Suppose that 500 parts are tested in manufacturing and 10 are rejected. Using both the approaches (critical

Page 87 of 160
STATISTICS OVERVIEW

value approach and P -value approach) to test the hypothesis H0 : p = 0.03 against Ha : p < 0.03 at significance level
α = 0.05.
Solution
10
Sample proportion p̂ = = 0.02
500
Null hypothesis:
H0 : p = 0.03 where p0 = 0.03
Alternative hypothesis:
Ha : p < 0.03 where p0 = 0.03
Sol 1: Critical value approach
p̂ − p0 0.02 − 0.03
Test statistics: z0 = r =r = −1.31
p0 (1 − p0 ) 0.03(1 − 0.03)
n 500
zα = z0.05 = 1.645
−1.31 > −1.645 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the real popu-
lation mean is less than 3%
Sol 2: p-value
p̂ − p0 0.02 − 0.03
Test statistics: z0 = r =r = −1.31
p0 (1 − p0 ) 0.03(1 − 0.03)
n 500
p-value = P (Z < −1.31) = 1 − Φ(1.31) = 1 − 0.9049 = 0.0951
p-value > α ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the real popu-
lation mean is less than 3%

Example: Defective Water Filters

Historically, the defective rate of water filters is 7% in a certain factory. A new quality control system is introduced to
reduce the defective rate. Suppose that we randomly choose 300 water filters, and calculate p̂ = 0.041. We want to test
whether the new system reduce the defective rate or not.
Let p be the proportion of defective water filters in the factory after introducing the new system.
Solution
H0 : p0 = 0.07
Ha : p < 0.07
Ha implies that the new system reduces the defective rate
Define α = 0.05
Assuming H0 is true, the test statistics is calculated as
p̂ − p0 0.041 − 0.07
z0 = r =r = −1.97
p0 (1 − p0 ) 0.07(1 − 0.07)
n 300
Because p < p0 ⇒ p-value = P (Z < −1.97) = Φ(−1.97) = 0.0244
Since p-value < α ⇒ reject H0
⇒ There is sufficient evidence to conclude the new system reduces the defective rate

Example: Flu Outbreak

A certain type of flu outbreaks in northern part of the USA. The historical records shows that there are 7% of the
residences in Columbia carrying flu under usual condition. Researchers want to see whether there is an outbreak in
Columbia, there are 30 out of 250 randomly chosen people in the sample carrying flu.
Can we conclude that there is an outbreak in Columbia? Answer the question using hypothesis testing approach with
α = 0.05.

Page 88 of 160
STATISTICS OVERVIEW

Solution
H0 : p0 = 0.07
Ha : pa > 0.07
30
Sample proportion p̂ = = 0.12
250
p̂ − p0 0.12 − 0.07
Test statistics z0 = r =r = 3.10
p0 (1 − p0 ) 0.07(1 − 0.07)
n 250
Because p > p0 ⇒ p-value = P (Z > z0 ) = P (Z > 3.10) = 1 − P (Z ≤ 3.10) = 1 − Φ(3.10) = 1 − 0.9990 =
0.001
Since p-value < α ⇒ reject H0
⇒ There is sufficient evidence to conclude that there is an outbreak in Columbia

Example: Football Helmets

A researcher claims that at least 10% of all football helmets have manufacturing flaws that could potentially cause injury
to the wearer. A sample of 200 helmets revealed that 16 helmets contained such defects.
(a) Does this finding support the researcher’s claim? Use α = 0.01. Find the p-value.
(b) Explain how the previous question could be answered with a confidence interval.
Solution
H0 : p ≤ 0.1
Ha : p > 0.1
16
(a) Sample proportion p̂ = = 0.08
200
p̂ − p0 0.08 − 0.10
Test statistics z0 = r =r = −0.94
p0 (1 − p0 ) 0.1(1 − 0.1)
n 200
Because p > p0 ⇒ p-value = P (Z > −0.94) = 1 − P (Z ≤ −0.94) = 1 − Φ(−0.94) = 1 − 0.1736 = 0.8264
Since p-value > α ⇒ H0 is not rejected
⇒ there is not enough evidence that at least 10% of all football helmets have manufacturing flaws that could
potentially cause injury to the wearer
α
(b) = 0.01 = P (Z > zα/2 ) = 1 − P (Z ≤ zα/2 ) = 1 − Φ(zα/2 ) ⇒ Φ(zα/2 ) = 0.99 ⇒ zα/2 = 2.33
2 r r
p̂(1 − p̂) p̂(1 − p̂)
99% CI = (p̂ − zα/2 p̂ − zα/2 )
n
r n r
0.08(1 − 0.08) 0.08(1 − 0.08)
⇒ 99% CI = (0.08 − 2.33 , 0.08 + 2.33 ) = (0.0353, 0.1250)
200 200
Because 99% confidence interval includes 10% ⇒ the claim is not supported

Example: Lenses
A manufacturer of inter-ocular lenses will qualify a new grinding machine if there is evidence that the percentage of
polished lenses that contain surface defects does not exceed 2%. A random sample of 250 lenses contains 6 defective
lenses
(a) Formulate and test an appropriate set of hypotheses to determine whether the machine can be qualified. Use α = 0.01.
Find the p-value.
(b) Explain how the previous question could be answered with a confidence interval.
Solution
(a)
H0 : p ≥ 0.02
Ha : p < 0.02
6
Sample proportion p̂ = = 0.024
250

Page 89 of 160
STATISTICS OVERVIEW

p̂ − p0 0.024 − 0.02
Test statistics z0 = r =r = 0.45
p0 (1 − p0 ) 0.02(1 − 0.02)
n 250
Because p < p0 ⇒ p-value = P (Z < z0 ) = Φ(z0 ) = Φ(0.45) = 0.6736
Since p-value > α ⇒ H0 is not rejected
⇒ There is not enough evidence to conclude that the percentage of defective lenses is less than 2%.
⇒ the machine cannot be qualified
(b) Construct a 99% upper confidence bound for the proportion of defective lenses. If this upper bound is less than
2%, then the machine can be qualified.
α
= 0.01 = P (Z > zα/2 ) = 1 − P (Z ≤ zα/2 ) = 1 − Φ(zα/2 ) ⇒ Φ(zα/2 ) = 0.99 ⇒ zα/2 = 2.33
2 r r
p̂(1 − p̂) p̂(1 − p̂)
99% CI = (p̂ − zα/2 p̂ − zα/2 )
nr n r
0.024(1 − 0.024) 0.024(1 − 0.024)
⇒ 99% CI = (0.024 − 2.33 , 0.024 + 2.33 )
250 250
⇒ 99% CI = (0.00145, 0.0466)
Since the upperbound of 4.66% is greater than 2% ⇒ the machine cannot be qualified

67) Method of Evaluating a Test: Type I and Type II Errors

There are two mistakes we can make in a hypothesis test.

 Type I error: H0 is rejected but in reality H0 is true (false positive conclusion)
 Type II error: H0 is not rejected but in reality H0 is false (false negative conclusion)

Type I and Type II Error

Null hypothesis is TRUE FALSE
Type I error Correct decision
Rejected False positive True positive
Probability = α Probability = 1 − β
Correct decision Type II error
Not rejected True negative False negative
Probability = 1 − α Probability = β

67).1 Type I Error

A Type I error means rejecting the null hypothesis when it’s actually true. It means concluding that results are statisti-
cally significant when, in reality, they came about purely by chance or because of unrelated factors.
The risk of committing this error is the significance level (alpha or α) you choose. That’s a value that you set at the
beginning of your study to assess the statistical probability of obtaining your results (p-value).

α = P (reject H0 |H0 is true) (8..4)

 If the p-value of your test is lower than the significance level, it means your results are statistically significant and
consistent with the alternative hypothesis

p-value < α ⇒ H0 is rejected, Ha is accepted

 If your p-value is higher than the significance level, then your results are considered statistically non-significant.

p-value ≥ α ⇒ H0 is accepted, Ha is rejected

To reduce the Type I error probability, you can simply set a lower significance level.

Page 90 of 160
STATISTICS OVERVIEW

Type I Error rate

At the tail end, the shaded area represents α. It’s also called a critical region in statistics.
If your results fall in the critical region of this curve, they are considered statistically significant and the null hypothesis
is rejected.
 Because this is a false positive conclusion, the H0 is actually true in this case

Note

Typically, when designing a hypothesis test one controls (or specifies) the type I error probability.
It is common to specify values of 0.10, 0.05, and 0.01 for α, but these values may not be appropriate in all situations

67).2 Type II Error

A Type II error means not rejecting the null hypothesis when it’s actually false.
A Type II error means failing to conclude there was an effect when there actually was. In reality, your study may not
have had enough statistical power to detect an effect of a certain size
Power is the extent to which a test can correctly detect a real effect when there is one. A power level of 80% or higher
is usually considered acceptable.
 The power of a test is the probability of correctly rejecting a false null hypothesis.

Power = P (rejectH0 |H0 is false) (8..5)

The risk of a Type II error is inversely related to the statistical power of a study.
 The higher the statistical power, the lower the probability of making a Type II error.
 These two probabilities are complementary.
Statistical power is determined by:
 Size of the effect: Larger effects are more easily detected.
 Measurement error: Systematic and random errors in recorded data reduce power.
 Sample size: Larger samples reduce sampling error and increase power.
 Significance level: Increasing the significance level increases power.
To (indirectly) reduce the risk of a Type II error, you can increase the sample size or the significance level.

Page 91 of 160
STATISTICS OVERVIEW

Type II Error rate

The Type II error rate is β, represented by the shaded area on the left side.

β = P (fail to reject H0 |Ha is true at some value) (8..6)

The remaining area under the curve represents statistical power, which is 1 − β.
Increasing the statistical power of your test directly decreases the risk of making a Type II error.

67).3 Trade-off between Type I and Type II errors

The Type I and Type II error rates influence each other. That’s because the significance level α (the Type I error rate)
affects statistical power, which is inversely related to the Type II error rate.
 Setting a lower significance level decreases a Type I error risk, but increases a Type II error risk.
 Increasing the power of a test decreases a Type II error risk, but increases a Type I error risk.
 An increase in the sample size will generally reduce both α and 1β
 Type I and II errors are related. A decrease in the probability of a type I error always results in an increase in the
probability of a type II error, and vice versa, provided the sample size n does not change
 When the null hypothesis is false, β decreases as the true value of the parameter approaches the value hypothesized in
the null hypothesis. The value of β increases as the difference between the true parameter and the hypothesized value
increases
Example: Controlling Risk

Hypothesis
Type Probability
H0 Ha
P (Type I error) = P (Z ≥ z0 )
Right-tail test H0 : µ = µ0 Ha : µ > µ0
P (Type II error) = P (Z < z0 )
P (Type I error) = P (Z < z0 )
Left-tail test H0 : µ = µ0 Ha : µ < µ0
P (Type II error) = P (Z ≥ z0 )
P (Type I error) = α
Two-tail test H0 : µ = µ0 Ha : µ ̸= µ0
P (Type II error) = 1 − Power

You are planning to hold a party for our STAT 355 class, and you want to know at 95% confidence level, whether less
than 60% of students will attend.
H0 : p = 0.6

Page 92 of 160
STATISTICS OVERVIEW

Ha : p < 0.6
(a) Describe a Type I error for this problem and the potential consequence.
(b) Describe a Type II error for this problem and the potential consequence.

Solution

(a) Type I error: H0 is rejected but H0 is true in fact

Description: It may seems there is sufficient evidence that there is less than 60% of students will attend the party,
hower there is 60% of them may be there
⇒ the number of attendance will be wrongly counted ⇒ wrong conclusion
(b) Type II: H0 is not rejected but H0 is false in fact
Description: Contrary to Type I error, it’s likely to have 60% of students will attend the party, but the amount
will be less than 60%
⇒ the number of attendance will be wrongly counted ⇒ wrong conclusion

68) Hypothesis Testing for Population Mean µ

68).1 Hypothesis Testing for the mean µ with known variance σ 2
Suppose X1 , X2 , ..., Xn is a random sample of size n from a population.
The population is normally distributed, (if it is not, the conditions of the central limit theorem apply, and n is suf-
ficiently large.)
⇒ sample mean X̄ as a point estimator for µ
σ2
 X̄ ∼ N (µ, )
n
Null hypothesis:
H0 : µ = µ0
Alternative hypothesis:
Ha : µ < µ0
Ha : µ > µ0
Ha : µ ̸= µ0
Confidence interval approach
1. Find the right type of confidence interval.
2. Find the corresponding rejection region:

Page 93 of 160
STATISTICS OVERVIEW

Hypothesis type Alternative hypothesis Rejection criterion

σ
Left-tail hypothesis Ha : µ < µ0 x̄ + zα √ < µ0
n
σ
Right-tail hypothesis Ha : µ > µ0 x̄ − zα √ > µ0
n
σ
Two-tail hypothesis Ha : µ ̸= µ0 µ0 ∈/ (x̄ ± zα/2 √ )
n
3. Compare: find whether or not the rejection criterion is satisfied
4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .
The test statistics (Critical value approach)
1. Compute the test statistics:
x̄ − µ0
z0 = √ (8..7)
σ/ n

2. Find the corresponding rejection region:

Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis Ha : µ < µ0 z0 < −zα
Right-tail hypothesis Ha : µ > µ0 z0 > zα
Two-tail hypothesis Ha : µ ̸= µ0 z0 > zα/2 or z0 < −zα/2

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .
P -value approach
1. Compute the test statistics:
x̄ − µ0
z0 = √ (8..8)
σ/ n

2. Find the corresponding rejection region:

Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis Ha : µ < µ0 p-value = P (Z < z0 ) = Φ(z0 ) < α
Right-tail hypothesis Ha : µ > µ0 p-value = P (Z > z0 ) = 1 − Φ(z0 ) < α
Two-tail hypothesis µ ̸= µ0 p-value = 2P (Z < −|z0 |) = 2[1 − Φ(|z0 |)] < α

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .

Note
Recall that Φ(z0 ) is the cumulative distribution function of the standard normal distribution; i.e., Φ(z) = P (Z ≤
z).

Proof: for two-sided alternative Ha : µ ̸= µ0

z0 > zα/2 or z0 < −zα/2 z0 > zα/2 or z0 < −zα/2

x̄ − µ0 x̄ − µ0 ⇔ P (Z ≥ |z0 |) < P (Z ≥ zα/2
⇔ √ > zα/2 or √ < −zα/2 α
σ/ n σ/ n ⇔ 1 − P (Z ≤ |z0 | <
σ σ 2 Example:
⇔ x̄ − zα/2 √ > µ0 or x̄ + zα/2 √ < µ0 ⇔ 2[1 − P (Z ≤ |z0 |)] < α
n n
σ ⇔ 2[1 − Φ(|z0 |)] < α
⇔ µ0 ∈ / (x̄ ± zα/2 √ )
n ⇔ p-value < α
Civil engineers have found that the ability to see and read a sign at night depends in part on its “surround luminance;”

Page 94 of 160
STATISTICS OVERVIEW

i.e., the light intensity near the sign. It is believed that the mean surround luminance is 10 candela per m2 in a large
metropolitan area. The data below are n = 30 measurements of the random variable X, the surround luminance (in
candela per m2 ). The 30 measurements constitute a random sample from all the signs in the large metropolitan area in
question:

10.9 1.7 9.5 2.9 9.1 3.2 9.1 7.4 13.3 13.1
6.6 13.7 1.5 6.3 7.4 9.9 13.6 17.3 3.6 4.9
13.1 7.8 10.3 10.3 9.6 5.7 2.6 15.1 2.9 16.2

Based on past experience, the engineers assume a normal population distribution (for the population of all signs) with
known population variance σ 2 = 20. From this data what conclusions should we draw about the hypothesized mean
surround luminance, at the α = 0.05 significance level.
Solution
P
x 10.9 + 1.7 + ... + 2.9 + 16.2 258.6
x̄ = i = = = 8.62
n 10 30
Null hypothesis:
H0 : µ = µ0 where µ0 = 10
Alternative hypothesis:
Ha : µ ̸= µ0 where µ0 = 10
Sol 1: Confidence interval approach
α = 0.05 ⇒ confidence level = 95% ⇒ zα/2 = 1.960
√ √
σ σ 20 20
95% CI = (x̄ − zα/2 √ , x̄ + zα/2 √ ) = (8.62 − 1.96 √ , 8.62 + 1.96 √ ) = (7.02, 10.22)
n n 30 30
µ0 ∈ (7.02, 10.22) ⇒ fail to reject H0
Conclusion: At the 5% significance level, the data do not provide sufficient evidence to conclude that the mean
surround luminance for the specified large metropolitan area differs from 10 candela per m2 .
Sol 2: Critical value apporach
x̄ − µ0 8.62 − 10
Test statistic: z0 = √ =√ √ = −1.69
σ/ n 20/ 30
α = 0.05 ⇒ confidence level = 95% ⇒ zα/2 = 1.960
−1.69 < 1.96 and −1.69 > −1.960 ⇒ fail to reject H0
Conclusion: At the 5% significance level, the data do not provide sufficient evidence to conclude that the mean
surround luminance for the specified large metropolitan area differs from 10 candela per m2 .
Sol 3: p-value approach
x̄ − µ0 8.62 − 10
Test statistics z0 = √ =√ √ = −1.69
σ/ n 20/ 30
p-value = 2P (Z < −|z0 |) = 2[1 − Φ(z0 )] = 2[1 − Φ(1.69)] = 2[1 − 0.9545] = 0.091
p-value > α ⇒ fail to reject H0
Conclusion: At the 5% significance level, the data do not provide sufficient evidence to conclude that the mean
surround luminance for the specified large metropolitan area differs from 10 candela per m2 .

68).2 Hypothesis Testing for the mean µ with unknown variance σ 2

Suppose X1 , X2 , ..., Xn is a random sample of size n from a population.
The population is normally distributed, (if it is not, the conditions of the central limit theorem apply, and n is suf-
ficiently large.)
⇒ sample mean X̄ as a point estimator for µ
X̄ − µ
 √ ∼ t(n − 1)
S/ n
Null hypothesis:

Page 95 of 160
STATISTICS OVERVIEW

H0 : µ = µ0
Alternative hypothesis:
Ha : µ < µ0
Ha : µ > µ0
Ha : µ ̸= µ0
Confidence interval approach
1. Find the right type of confidence interval.
2. Find the corresponding rejection region:
Hypothesis type Alternative hypothesis Rejection criterion
s
Left-tail hypothesis Ha : µ < µ0 x̄ + tn−1,α √ < µ0
n
s
Right-tail hypothesis Ha : µ > µ0 x̄ − tn−1,α √
n
s
Two-tail hypothesis Ha : µ ̸= µ0 µ0 ∈/ (x̄ ± tn−1,α/2 √ )
n
3. Compare: find whether or not the rejection criterion is satisfied
4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .
The test statistics (Critical value approach)
1. Compute the test statistics:
x̄ − µ0
t0 = √ (8..9)
s/ n
2. Find the corresponding rejection region:
Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis Ha : µ < µ0 t0 < −tn−1,α
Right-tail hypothesis Ha : µ > µ0 t0 > tn−1,α
Two-tail hypothesis Ha : µ ̸= µ0 t0 > tn−1,α/2 or t0 < −tn−1,α/2

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .
P -value approach
1. Compute the test statistics:
x̄ − µ0
t0 = √ (8..10)
s/ n
2. Find the corresponding rejection region:
Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis Ha : µ < µ0 p-value < α
Right-tail hypothesis Ha : µ > µ0 p-value < α
Two-tail hypothesis Ha : µ ̸= µ0 p-value < α

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population mean is (less than, greater than, or differs from) .

Note
If you are working with a two-tailed test don’t forget to divide the alpha value by 2 to find the correct t-value.

Example: Civil engineers have found that the ability to see and read a sign at night depends in part on its “surround
luminance;” i.e., the light intensity near the sign. It is believed that the mean surround luminance is 10 candela per m2 in
a large metropolitan area. The data below are n = 30 measurements of the random variable X, the surround luminance

Page 96 of 160
STATISTICS OVERVIEW

(in candela per m2 ). The 30 measurements constitute a random sample from all the signs in the large metropolitan area
in question:

10.9 1.7 9.5 2.9 9.1 3.2 9.1 7.4 13.3 13.1
6.6 13.7 1.5 6.3 7.4 9.9 13.6 17.3 3.6 4.9
13.1 7.8 10.3 10.3 9.6 5.7 2.6 15.1 2.9 16.2

Based on past experience, the engineers assume a normal population distribution (for the population of all signs). From
this data what conclusions should we draw about the hypothesized mean surround luminance, at the α = 0.05 signifi-
cance level.

Solution
P
x 10.9 + 1.7 + ... + 2.9 + 16.2 258.6
x̄ = i = = = 8.62
n 10 30
2
(10.9 − 8.62) + ... + (16.2 − 8.62)2
2
P
(x − x̄)
s2 = i = = 20.43
n−1 30 − 1
tn−1,α/2 = t29,0.025 = 2.045
Null hypothesis:
H0 : µ = µ0 where µ0 = 10
Alternative hypothesis:
Ha : µ ̸= µ0 where µ0 = 10
Sol 1: Confidence interval approach
α = 0.05 ⇒ confidence level = 95%
s s
95% CI = (x̄ − tn−1,α/2 √ , x̄ + tn−1,α/2 √ )
n√ n √
20.43 20.43
↔ 95% CI = (8.62 − 2.045 √ , 8.62 + 2.045 √ ) = (6.932, 10.31)
30 30
µ0 ∈ (6.932, 10.31) ⇒ fail to reject H0
Conclusion: At the 5% significance level, the data do not provide sufficient evidence to conclude that the mean
surround luminance for the specified large metropolitan area differs from 10 candela per m2
Sol 2: Critical value approach
x̄ − µ0 8.62 − 10
Test statistics: t0 = √ =√ √ = −1.67
s/ n 20.43/ 30
Critical value = tn−1,α = 2.045
−1.67 < 2.045 and −1.67 > −2.045 ⇒ fail to reject H0
Conclusion: At the 5% significance level, the data do not provide sufficient evidence to conclude that the mean
surround luminance for the specified large metropolitan area differs from 10 candela per m2
Sol 3: p-value approach
x̄ − µ0 8.62 − 10
Test statistics: t0 = √ =√ √ = −1.67
s/ n 20.43/ 30
1.311 < |t0 | < 1.699 ⇒ 0.1 < p-value < 0.2
0.1 > 0.05 ⇒ p-value > α ⇒ fail to reject H0
Conclusion: At the 5% significance level, the data do not provide sufficient evidence to conclude that the mean
surround luminance for the specified large metropolitan area differs from 10 candela per m2

Example: Pipe Diameter (in Chapter 7)

Go back to pipe diameter data. Based on past experience, the engineers assume a normal population distribution (for
the pipe diameters) with known population standard deviation 0.02. Researchers want to find out whether the pipe
diameter is 1.31 with a significance level 0.05.

Solution

Page 97 of 160
 STATISTICS OVERVIEW

H0 : µ = 1.31
P25 Ha : µ ̸= 1.31
Yi 1.296 + 1.320 + 1.311 + ... + 1.315 + 1.306 + 1.289 + 1.291 32.477
Ȳ = i=1 = = = 1.299
n 25 25
1.299 − 1.31
z0 = √ = −2.75
0.02/ 25
Because µ ̸= µ0 ⇒ p-value = 2P (Z < −|z0 |) = 2P (Z < −| − 2.5|) = 2P (Z < −2.75) = 2(0.0030) = 0.006
Since p-value < α ⇒ reject H0
⇒ There is sufficient evidence to conclude the mean pipe diameter is not 1.31 inches

Example: Pipe Diameter (in Chapter 7)

Go back to pipe diameter data. Assume the pipe diameters are normality distributed with unknown population vari-
ance. Researchers what to find out whether the pipe diameter is less than 1.308. Assume a significance level 0.05.

Solution
H0 : µ ≥ 1.308
P25 Ha : µ < 1.308
Yi 1.296 + 1.320 + 1.311 + ... + 1.315 + 1.306 + 1.289 + 1.291 32.477
Ȳ = i=1 = = = 1.299
n P25 25 25
2 2 2
(yi − ȳ) (1.296 − 1.299) + ... + (1.291 − 1.299) 0.002948
Sample variance s2 = i=1 = = = 0.0001228
n−1 √ √ 25 − 1 24
⇒ sample standard deviation s = s2 = 0.0001228 = 0.0111
Ȳ − µ0 1.299 − 1.308
t= √ = √ = −4.05
s/ n 0.0111/ 25
Degree of freedom ν = n − 1 = 25 − 1 = 24
α
α = 0.05 ⇒ = 0.025 ⇒ tα/2 = t0.025,24 = 2.064
2
Because t < tα/2 ⇒ reject H0
⇒ There is sufficient evidence that the pipe diameter is less than 1.308

1 > t . t e s t ( p i p e s , a l t e r n a t i v e = ” l e s s ” , mu = 1 . 3 0 8 )
2 One Sample t − t e s t
3 data : pipes
4 t = − 4 . 0 2 4 3 , d f = 2 4 , p− v a l u e = 0 . 0 0 0 2 4 7 8
5 a l t e r n a t i v e h y p o t h e s i s : t r u e mean i s l e s s t h a n 1 . 3 0 8
6 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
7 −Inf 1.302872
8 sample e s t i m a t e s :
9 mean o f x
10 1.29908

Example: Sales
For a long time, the monthly average sales for a product have been $30, 000. A company invests a lot of money into
new advertising for the product and then conducts a hypothesis test with a level of significance 0.05 in order to see
if monthly sales of the product have increased. The hypotheses are: H0 : µ = 30, 000 and Ha : µ > 30, 000. The
conclusion of the test is that monthly sales have increased.
(a) What is the probability of type I error for this test?
(b) Suppose in reality, monthly sales did not increase and the advertisment had no effect? What type of error occurred?

Solution
(a) The significance level α represents the probability of committing a type I error. It’s the threshold for rejecting
the null hypothesis
⇒ p = α = 0.05

Page 98 of 160
STATISTICS OVERVIEW

(b) H0 : µ = 30, 000 while the conclusion is that the monthly sales µ > 30, 000 which mean H0 is rejected
While in reality, monthly sales did not increase and the advertisement had no effect ⇒ H0 : µ = 30, 000 is true
⇒ H0 is rejected but in reality H0 is true ⇒ type I error

True or False
(a) If we reject H0 : µ = 0 in a study with α = 0.01, then we also reject it with α = 0.05.
(b) A study about the change in weight on a new diet reports p-value = 0.043 for testing H0 : µ = 0 against Ha : µ ̸= 0
under α = 0.05. If the authors report a 95% confidence interval for µ, then the interval would contain 0.
(c) A 95% C.I. for µ = population mean IQ is (96, 110). In the test of H0 : µ = 100 against Ha : µ ̸= 100, the p-value
would be greater than 0.05.
(d) When testing H0 : µ = 100 against Ha : µ ̸= 100, the probability of a Type II error decreases when the true
population mean is further away from 100.

Solution

(a) If H0 is rejected at α = 0.01 ⇒ p-value < 0.01 ⇒ p-value < 0.05 ⇒ TRUE
(b) A 95% confidence interval is equivalent to a two-sided significance test at α = 0.05
Since p-value is 0.043, H0 : µ = 0 is rejected at α = 0.05 ⇒ 0 is not one of the plausible values
⇒ it is not within the 95% confidence interval ⇒ FALSE
(c) 100 is within the 95% confidence interval, µ = 100 is plausible ⇒ H0 : µ = 100 will not be rejected (p-value
> 0.05) for a two-sided test ⇒ TRUE
(d) When the true population mean µ is further away from the hypothesized mean µ0 = 100, the difference
between the true state and the null hypothesis becomes larger
A larger difference makes it easier for the statistical test to detect this difference
⇒ the power of the test, which is the probability of correctly rejecting a false null hypothesis, increases
The probability of a Type II error, β, is given by β = 1− Power
Since the power of the test increases as the true mean moves further from the hypothesized mean, the value of 1−
Power will decrease.
⇒ the probability of a Type II error decreases ⇒ TRUE

69) Hypothesis Testing for the population variance σ 2

Suppose that X1 , X2 , ..., Xn is a random sample from a N (µ, σ 2 ) distribution (normality is important in this case).
The quantity
(n − 1)S 2
Q= ∼ χ2 (n − 1) (8..11)
σ2
with χ2 (chi-squared) distribution with ν = n − 1 degrees of freedom

Null hypothesis:
H0 : σ = σ0 (or H0 : σ 2 = σ02 )
Alternative hypothesis:
Ha : σ < σ0 (or Ha : σ 2 < σ02 )
Ha : σ > σ0 (or Ha : σ 2 > σ02 )
Ha : σ ̸= σ0 (or Ha : σ 2 ̸= σ02 )
Confidence interval approach
1. Find the right type of confidence interval.
2. Find the corresponding rejection region:

Page 99 of 160
STATISTICS OVERVIEW

Hypothesis type Alternative hypothesis Rejection criterion

(n − 1)s2
Left-tail hypothesis Ha : σ 2 < σ02 < σ02
χ2n−1,1−α
(n − 1)s2
Right-tail hypothesis Ha : σ 2 > σ02 > σ02
χ2n−1,α
(n − 1)s2 (n − 1)s2
Two-tail hypothesis Ha : σ 2 ̸= σ02 σ02 ∈
/[ 2 , ]
χn−1,α/2 χ2n−1,1−α/2

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population variance is (less than, greater than, or differs from) .
The test statistics (Critical value approach)
1. Compute the test statistics:
(n − 1)s2
Q0 = (8..12)
σ02
2. Find the corresponding rejection region:
Hypothesis type Alternative hypothesis Rejection criterion
Left-tail hypothesis 2
Ha : σ < σ02 Q0 < χ2n−1,1−α
Right-tail hypothesis 2
Ha : σ > σ02 Q0 > χ2n−1,α
Two-tail hypothesis Ha : σ 2 ̸= σ02 Q0 > χ2n−1,α/2 or Q0 < χ2n−1,1−α/2

3. Compare: find whether or not the rejection criterion is satisfied

4. Conclusions: At α% significance level, the data (do or do not) provide sufficient evidence to conclude
that the real population variance is (less than, greater than, or differs from) .

Note
The value of χ2n−1,α can be found via the Chi-square Distribution Table

Example: An automated filling machine is used to fill bottles with liquid detergent. A random sample of 20 bottles
results in a sample variance of fill volume of s2 = 0.0153 (fluid ounces)2 . If the variance of fill volume exceeds 0.01
(fluid ounces)2 , an unacceptable proportion of bottles will be underfilled or overfilled. Is there evidence in the sample
data to suggest that the manufacturer has a problem with underfilled or overfilled bottles? Use α = 0.05, and assume
that fill volume has a normal distribution (use both the confidence interval and critical value approaches).
Solution
Sample variance: s2 = 0.0153 (fluid ounces)2
Threshold variance: σ 2 = 0.01 (fluid ounces)2
Null hypothesis:
H0 : σ 2 ≤ 0.01 where σ02 = 0.01
Alternative hypothesis:
Ha : σ 2 > 0.01 where σ02 = 0.01
Sol 1: Confidence interval approach
(n − 1)s2 (n − 1)s2 (n − 1)s2 (n − 1)s2
α = 0.05 ⇒ 95% CI = [ 2 , 2 ]=[ 2 , 2 ]
χn−1,α χn−1,1−α χ19,0.05 χ19,0.95
(20 − 1)(0.0153) (20 − 1)(0.0153)
⇔ 95% CI = [ , ] = (0.0096, 0.029)
30.144 10.117
0.0096 < 0.01 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the real popu-
lation variance is greater than 0.01
Sol 2: Critical value approach

Page 100 of 160

STATISTICS OVERVIEW

(n − 1)s2 (20 − 1)(0.0153)

Test statistics Q0 = = = 29.07
σ02 0.01
χ2n−1,α = χ219,0.05 = 30.144
29.07 < 30.144 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the real popu-
lation variance is greater than 0.01

Page 101 of 160

STATISTICS OVERVIEW

9. INFERENCES BASED ON TWO SAMPLES

70) Difference between two population means

In practice, it is very common to compare the same characteristic (mean, proportion, variance) from two different
distributions.
Example:
 the mean starting salaries of male and female engineers (compare µ1 and µ2 )
 the proportion of scrap produced from two manufacturing processes (compare p1 and p2 )
 the variance of sound levels from two indoor swimming pool designs (compare σ12 and σ22 ).
How do two (several) sub-populations compare? In particular, are their means the same?
Example:
 Is this drug’s effectiveness the same in children and adults?
 Does cig brand A have the same amount of nicotine as brand B?
 Is this class of students more statistically savvy than the last?
Solution: collect samples from both (all sub-populations, and perform a two-sample test (ANOVA)
Statistically speaking, for two samples, we want to test whether µ1 − µ2 = 0 that is, whether µ1 = µ2 .
Example: Difference Between Two Population Means
Forty patients were randomly assigned to a treatment group (m = 20) or a control group (n = 20).
One patient in the treatment group and four in the control group dropped out because of complications.
The data analysis is then based on two random samples, i.e. the treatment group (m = 19) and control group (n = 16).
70).1 Basic assumptions
Suppose we have two independent samples with uneven sample sizes:
 X1 , X2 , ..., Xm ∼ N (µ1 , σ12 ) is a random sample from a distribution with mean µ1 and variance σ12 (sample size n)
 Y1 , Y2 , .....Yn ∼ N (µ2 , σ22 ) is a random sample from a distribution with mean µ2 and variance σ22 (sample size m)
Point estimators:
n n
1 X 1 X
X̄ = Xi = sample mean for sample X S12 = (Xi − X̄)2 = sample variance for sample
n1 i=1 n − 1 i=1
X
m m
1 X 1 X
Ȳ = Yj = sample mean for sample Y S22 = (Yi − Ȳ )2 = sample variance for sample
n1 j=1 m − 1 j=1
Y
The expected value of X̄ − Ȳ
E(X̄ − Ȳ ) = E(X̄) − E(Ȳ ) = µ1 − µ2 (9..1)

The variance of X̄ − Ȳ
σ12 σ2
2
σX̄− Ȳ = var(X̄ − Ȳ ) = var(X̄) + var(Ȳ ) = + 2 (9..2)
m n
The standard deviation of X̄ − Ȳ r
σ12 σ2
q
σX̄−Ȳ = var(X̄ − Ȳ ) = + 2 (9..3)
m n
The sample variances must be used to estimate this when population variances are unknown.
The construction of confidence intervals and hypothesis testings depend on the values of σ12 and σ22
There are three cased need to be considered:
 known σ12 and σ22
 unknown σ12 and σ22 : σ12 = σ22
 unknown σ12 and σ22 : σ12 ̸= σ22

Page 102 of 160

STATISTICS OVERVIEW

71).0 Inference on µ1 and µ2 assuming known σ12 and σ22

A logical point estimator of µ1 − µ2 is the difference in sample mean X̄ − Ȳ .

σ12
X̄ ∼ N (µ1 , )
n
2
σ
Ȳ ∼ N (µ2 , 2 )
m
Because of X̄ and Ȳ are normal distributions ⇒ X̄ − Ȳ are also normally distributed
σ2 σ2
=⇒ X̄ − Ȳ ∼ N (µ1 − µ2 , 1 + 2 )
n m

71).1 Confidence Interval (CI) assuming known σ12 and σ22

Standardizing X̄ − Ȳ gives the standard normal variable

X̄ − Ȳ − (µ1 − µ2 )
Z= r ∼ N (0, 1) (9..4)
σ12 σ22
+
n m

A (1 − α)100% confidence interval for µ1 − µ2 is

r r
σ12 σ2 σ12 σ2
[(x̄ − ȳ) − zα/2 + 2 , (x̄ − ȳ) + zα/2 + 2] (9..5)
n m n m

For this sampling distribution to hold (exactly), we need

 the two samples to be independent
 the two population distributions to be normal (Gaussian)
The sampling distribution Z ∼ N (0, 1) should suggest to you that confidence interval quantiles will come from the
standard normal distribution; note that this distribution depends on the sample sizes from both samples.
Proof:
1 − α = P (−zα/2 < Z < zα/2 )
x̄ − ȳ − (µ1 − µ2 )
⇔ 1 − α = P [−zα/2 < r < zα/2 ]
σ12 σ22
+
n m
−(µ1 − µ2 )
⇔ 1 − α = P [(−zα/2 − (x̄ − ȳ) < r < zα/2 − (x̄ − ȳ)]
σ12 σ22
+
r n m r
σ12 σ22 σ12 σ2
⇔ 1 − α = P [−zα/2 − (x̄ − ȳ) + < −(µ1 − µ2 ) < zα/2 − (x̄ − ȳ) + 2]
r n m r n m
σ12 σ22 σ12 σ22
⇔ 1 − α = P [zα/2 − (x̄ − ȳ) + < µ1 − µ2 < zα/2 − (x̄ + ȳ) + ]
n m n m

71).2 Hypothesis Testing for µ1 and µ2 assuming known σ12 and σ22
In a hypothesis-testing problem, the null hypothesis (H0 ) states that µ1 − µ2 has a specified value.
If we are testing equality of the two means, then µ1 − µ2 = 0 under the null hypothesis.

σ12 σ2
 Recall we have X̄ − Ȳ ∼ N (µ1 − µ2 , + 2)
n m
Null hypothesis:
H0 : µ1 − µ2 = △0
Alternative hypothesis:

Page 103 of 160

STATISTICS OVERVIEW

Ha : µ1 − µ2 < △0
Ha : µ1 − µ2 > △0
Ha : µ1 − µ2 ̸= △0
Confidence interval approach
Hypothesis type Alternative hypothesis Reject criterion
r
σ12 σ2
Left-tail hypothesis Ha : µ1 − µ2 < △0 (x̄ − ȳ) + zα + 2 < △0
rn m
σ12 σ22
Right-tail hypothesis Ha : µ1 − µ2 > △0 (x̄ − ȳ) − zα + > △0
n rm
2
σ1 σ2
Two-tail hypothesis Ha : µ1 − µ2 ̸= △0 △0 ∈ / [(x̄ − ȳ) ± zα + 2]
n m
The test statistics (Critical value approach) Test statistics:

(x̄ − ȳ) − △0
z0 = r (9..6)
σ12 σ2
+ 2
n m

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µ1 − µ2 < △0 z0 < −zα
Right-tail hypothesis Ha : µ1 − µ2 > △0 z0 > zα
Two-tail hypothesis Hα : µ1 − µ2 ̸= 0 z0 > zα/2 or z0 < −zα/2

P -value approach
Test statistics:
(x̄ − ȳ) − △0
z0 = r (9..7)
σ12 σ2
+ 2
n m

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µ1 − µ2 < △0 p-value = P (Z < z0 ) = Φ(z0 ) < α
Right-tail hypothesis Ha : µ1 − µ2 > △0 p-value = P (Z > z0 ) = 1 − Φ(z0 ) < α
Two-tail hypothesis Ha : µ1 − µ2 ̸= △0 p-value = 2(P < −|z0 |) = 2[1 − Φ(|z0 |)] < α

Example: Analysis of a random sample consisting of 20 specimens of cold-rolled steel to determine yield strengths
resulted in a sample average strength of x̄ = 29.8 ksi
A second random sample of 25 two-sided galvanized steel specimens gave a sample average strength of ȳ = 34.7 ksi
Assuming that the two yield-strength distributions are normal with σ1 = 4.0 and σ2 = 5.0, does the data indicate that
the corresponding true average yield strengths µ1 and µ2 are different?
Let’s carry out a test at significance level α = 0.01
Solution
Null hypothesis:
H0 : µ1 − µ2 = 0 where △0 = 0
Alternative hypothesis:
Ha : µ1 − µ2 ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
α = 0.01 ⇒ confidence levelr = 99% r
σ12 σ22 σ12 σ2
99% CI = [(x̄ − ȳ) − zα/2 + , (x̄ − ȳ) + zα/2 + 2]
n mr n m r
4.02 5.02 4.02 5.02
⇔ 99% CI = [(29.8 − 34.7) − 2.576 + , (29.8 − 34.7) + 2.576 + ]
20 25 20 25

Page 104 of 160

STATISTICS OVERVIEW

⇒ 99% CI = (−8.356, −1.443)

/ (−8.356, −1.443) ⇒ reject H0 ⇒ µ1 and µ2 are different
0∈
Sol 2: Critical value approach
x − y − △0 29.8 − 34.7 − 0
Test statistics z0 = r = r = −3.652
σ12 σ22 4.02 5.02
+ +
n m 20 25
Critical z-value = ±zα/2 ⇒ z0.005 = ±2.576
−3.652 > 2.576 and −3.652 < −2.576 ⇒ reject H0 ⇒ µ1 and µ2 are different
Sol 3: p-value approach
x − y − △0 29.8 − 34.7 − 0
Test statistics z0 = r = r = −3.652
2 2
σ1 σ2 4.02 5.02
+ +
n m 20 25
z0 = −3.652 ⇒ p-value = 2P (Z < −|z0 |) = 2P (Z < −3.652) ≈ 0
p-value < α ⇒ reject H0 ⇒ µ1 and µ2 are different

Example: Drying Time

A product developer is interested in reducing the drying time of a primer paint. Two formulations of the paint are tested;
formulation 1 is the standard chemistry, and formulation 2 has a new drying ingredient that should reduce the drying
time. From experience, it is known that the standard deviation of drying time is 8 minutes, and this inherent variability
should be unaffected by the addition of the new ingredient. Ten specimens are painted with formulation 1, and another
10 specimens are painted with formulation 2; the 20 specimens are painted in random order. The two sample average
drying times are y1 = 121 minutes and y2 = 112 minutes, respectively.
What conclusion can the product developer draw about the effectiveness of the new ingredient, using α = 0.05?

Solution
y1 = 121 y2 = 112 σ1 = σ2 = σ = 8 n1 = n2 = 10
Null hypothesis:
H0 : µ1 − µ2 = 0 where △0 = 0
Alternative hypothesis:
Ha : µ1 − µ2 > 0 where △0 = 0
Sol 1: Confidence
s interval approach s r
σ2 σ2 σ2 σ2 82 82
(y1 − y2 ) − zα + = (y1 − y2 ) − z0.05 + = (121 − 112) − 1.645 + = 3.115
n1 n2 n1 n2 10 10
3.115 > 0 ⇒ reject H0
Conclusion: At 5% significance level, the data do provide sufficient evidence to conclude that the new drying
ingredient has drying time less than the standard chemistry
Sol 2: Critical value approach
y1 − y2 − △0 121 − 112 − 0
Test statistics z0 = s = r = 2.516
σ12 σ22 82 82
+ +
n1 n2 10 10
Critical z-value = zα ⇒ z0.05 = 1.645
2.516 > 1.645 ⇒ reject H0
Conclusion: At 5% significance level, the data do provide sufficient evidence to conclude that the new drying
ingredient has drying time less than the standard chemistry
Sol 3: p-value approach
y1 − y2 − △0 121 − 112 − 0
Test statistics z0 = s = r = 2.516 ≈ 2.52
σ12 2
σ2 82 82
+ +
n1 n2 10 10
z0 = 2.52 ⇒ p-value = P (Z > 2.52) = 1 − Φ(2.52) = 1 − 0.9941 = 0.0059

Page 105 of 160

STATISTICS OVERVIEW

p-value < α ⇒ reject H0

Conclusion: At 5% significance level, the data do provide sufficient evidence to conclude that the new drying
ingredient has drying time less than the standard chemistry

Example: Fish Weights

In the vicinity of a nuclear power plant, environmental engineers from the EPA would like to determine if there is a
difference between the mean weight in fish (of the same species) from two locations. Independent samples are taken
from each location and the following weights (in ounces) are observed:

Location 1: 21.9 18.5 12.3 16.7 21.0 15.1 18.2 23.0 36.8 26.6
Location 2: 22.0 20.6 15.4 17.9 24.4 15.6 11.4 17.5

Suppose we know both distributions are normal and σ1 = 7, σ2 = 4.

(a) Construct a 90 percent confidence interval for the mean difference µ1 − µ2 . Here, µ1 (µ2 ) denotes the population
mean weight of all fish at location 1 (2)
(b) Test whether the two distribution have the same mean at α = 0.05.

Solution
n = 10,
Pi m = 8
xi 21.9 + 18.5 + ... + 36.8 + 26.6
x̄ = n = = 21.01
Pin 10
y 22.0 + 20.6 + ... + 11.4 + 17.5
ȳ = m = = 18.1
m 8
(a) 100(1 − α)% = 90% ⇔ 1 − α = 0.9 ⇒ α = 0.1 ⇒ zα/2 = z0.05 = 1.645
r r
σ12 σ22 σ12 σ2
90% CI = [(x̄ − ȳ) − zα/2 + , (x̄ − ȳ) + zα/2 + 2]
n mr n m r
72 42 72 42
⇔ 90% CI = [(21.01 − 18.1) − 1.645 + , (21.01 − 18.1) + 1.645 + ]
10 8 10 8
⇔ 90% CI = (−1.411, 7.231)
(b) Null hypothesis:
µ1 − µ2 = 0 where △0 = 0
Alternative hypothesis:
µ1 − µ2 ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
α = 0.05 ⇒ confidence interval
r = 95% r
σ12 σ22 σ12 σ2
95% CI = [(x̄ − ȳ) − zα/2 + , (x̄ − ȳ) + zα/2 + 2]
n mr n m r
72 42 72 42
⇔ 95% CI = [(21.01 − 18.1) − 1.960 + , (21.01 − 18.1) + 1.960 + ]
10 8 10 8
⇔ 95% CI = (−2.238, 8.058)
0 ∈ (−2.238, 8.058) ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the two distri-
butions have different means
Sol 2: Critical value approach
x̄ − ȳ − △0 21.01 − 18.1
Test statistics z0 = r = r = 1.107
2 2
σ1 σ2 72 42
+ +
n n 10 8
Critical z-value = ±zα/2 ⇒ z0.025 ± 1.960
1.107 < 1.960 and 1.107 > −1.960 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the two distri-
butions have different means

Page 106 of 160

STATISTICS OVERVIEW

Sol 3: p-value approach

x̄ − ȳ − △0 21.01 − 18.1
Test statistics z0 = r = r = 1.107 ≈ 1.11
2 2
σ1 σ2 72 42
+ +
n m 10 8
z0 = 1.11 ⇒ p-value = 2P (Z < −|z0 |) = 2[1 − Φ(1.11)] = 2(1 − 0.8665) = 0.267
0.267 > 0.05 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the two distri-
butions have different means

72) Inference on µ1 and µ2 assuming unknown σ12 and σ22 : σ12 = σ22

Assuming the variances of the two distributions are the same, we want to test whether two populations (means) are the
same or not.

72).1 Confidence Interval (CI) assuming unknown σ12 and σ22 : σ12 = σ22
Just like inference for single proportion, single mean, and single variance, we need a sampling distribution involving
µ1 − µ2 . It can be shown that
(X̄ − Ȳ ) − (µ1 − µ2 )
T = r (9..8)
1 1
SP2 ( + )
n m
where
(n − 1)S12 + (m − 1)S22
SP2 = (9..9)
n+m−2
The statistics SP2 is called the pooled sample variance estimator of the common population variance σ 2 . Algebraically,
it is simply a weighted average of the two sample variances S12 and S22 (where the weights are functions of the sample
sizes n1 and n2 )
For this sampling distribution to hold (exactly), we need:
 the two samples to be independent
 the two population distributions to be normal (Gaussian)
 the two population distributions to have the same variance: σ12 = σ22
The sampling distribution T ∼ t(n1 + n2 − 2) should suggest to you that confidence interval quantiles will come from
this t distribution; note that this distribution depends on the sample sizes from both samples.

Page 107 of 160

STATISTICS OVERVIEW

In particular, because T ∼ t(n1 + n2 − 2), we can find the value tn1 +n2 −2,α/2 that satisfies

P (−tn+m−2,α/2 < T < tn+m−2,α/2 ) = 1 − α (9..10)

A 100(1 − α)% confidence interval of µ1 − µ2 is given by:

r r
11 1 1
[(x̄ − ȳ) − tn+m−2,α/2 SP2 ( + ), (x̄ − ȳ) + tn+m−2,α/2 SP2 ( + )] (9..11)
n m n m

where: r
1 1
(x̄ − ȳ) ± tn+m−2,α/2 SP2 ( + )
| {z } | {z }| n m
point estimator quantile
{z }
standard error

Interpretation: We are 100(1 − α)% confident that the population mean difference µ1 − µ2 is in this interval
72).2 Hypothesis Testing for µ1 and µ2 assuming unknown σ12 and σ22 : σ12 = σ22

Null hypotehsis:
H0 : µ1 − µ2 = △0
Alternative hypothesis:
Ha : µ1 − µ2 < △0
Ha : µ1 − µ2 > △0
Ha : µ1 − µ2 < △0
Confidence interval approach
Hypothesis type Alternative hypothesis Rejection criterion
r
1 1
Left-tail hypothesis Ha : µ1 − µ2 < △0 (x̄ − ȳ) + tn+m−2,α SP2 ( + ) < △0
r n m
1 1
Right-tail hypothesis Ha : µ1 − µ2 > △0 (x̄ − ȳ) − tn+m−2,α SP2 ( + ) > △0
nrm
1 1
Two-tail hypothesis Ha : µ1 − µ2 ̸= △0 △0 ∈ / [(x̄ − ȳ) ± tn+m−2,α/2 SP2 ( + )]
n m
The test statistics (Critical value approach)
Test statistics:
(x̄ − ȳ) − △0
t0 = r (9..12)
1 1
SP2 ( + )
n m

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µ1 − µ2 < △0 t0 < −tn+m−2,α
Right-tail hypothesis Ha : µ1 − µ2 > △0 t0 > tn+m−2,α
Two-tail hypothesis Ha : µ1 − µ2 ̸= 0 t0 > tn+m−2,α/2 or t0 < −tn+m−2,α/2

P -value approach
Test statistics:
(x̄ − ȳ) − △0
t0 = r (9..13)
1 1
SP2 ( + )
n m

Page 108 of 160

STATISTICS OVERVIEW

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µ1 − µ2 < △0 p-value < α
Right-tail hypothesis Ha : µ1 − µ2 > △0 p-value < α
Two-tail hypothesis Ha : µ1 − µ2 ̸= △0 p-value < α

Example: Fish Weights

In the vicinity of a nuclear power plant, environmental engineers from the EPA would like to determine if there is a
difference between the mean weight in fish (of the same species) from two locations. Independent samples are taken
from each location and the following weights (in ounces) are observed:

Location 1: 21.9 18.5 12.3 16.7 21.0 15.1 18.2 23.0 36.8 26.6
Location 2: 22.0 20.6 15.4 17.9 24.4 15.6 11.4 17.5

Suppose we know both distributions are normal σ12 = σ22 .

(a) Construct a 90 percent confidence interval for the mean difference µ1 − µ2 . Here, µ1 (µ2 ) denotes the population
mean weight of all fish at location 1 (2)
(b) Test whether the two distribution have the same mean at α = 0.05.

Solution
n = 10,
Pi m = 8
(xi − x̄)2
P
nx 21.9 + 18.5 + ... + 36.8 + 26.6 2
x̄ = = = 21.01 ⇒ S1 = = 47.65
Pn
i
10 Pn−1 2
y 22.0 + 20.6 + ... + 11.4 + 17.5 (yi − ȳ)
y2 = m = = 18.1 ⇒ S22 = = 17.14
m 8 m−1
2 2
(n − 1)S1 + (m − 1)S2 (10 − 1)(47.65) + (8 − 1)(17.14)
SP2 = = = 34.30
n+m−2 10 + 8 − 2
(a) 100(1 − α)% = 90% ⇔ 1 − α = 0.9 ⇒ α = 0.1
tn+m−2,α/2 = t16,0.05 = 1.746
r r
1 1 1 1
90% CI = [(x̄ − ȳ) − tn+m−2,α/2 SP2 (
+ ), (x̄ − ȳ) + tn+m−2,α/2 SP2 ( + )]
r n m nr m
1 1 1 1
⇔ 90% CI = [(21.01 − 18.1) − 1.746 34.30( + ), (21.01 − 18.1) + 1.746 34.30( + )]
10 8 10 8
⇔ 90% CI = (−1.94, 7.76)
(b) Null hypothesis:
H0 : µ1 − µ2 = 0 where △0 = 0
Alternative hypothesis:
Ha : µ1 − µ2 ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
α = 0.05 ⇒ tn+m−2,α/2 = t16,0.025 = 2.120
α = 0.05 ⇒ confidence level = 95%
r r
1 1 1 1
95% CI = [(x̄ − ȳ) − tn+m−2,α/2 SP2 (
+ ), (x̄ − ȳ) + tn+m−2,α/2 SP2 ( + )]
rn m nr m
1 1 1 1
⇔ 95% CI = [(21.01 − 18.1) − 2.120 34.30( + ), (21.01 − 18.1) + 2.120 34.30( + )]
10 8 10 8
⇔ 95% CI = (−2.98, 8.80)
0 ∈ (−2.98, 8.80) ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the two distri-
butions have the same means
Sol 2: Critical value approach

Page 109 of 160

 STATISTICS OVERVIEW

(x̄ − ȳ) − △0 (21.01 − 18.1) − 0

Test statistics t0 = r = r = 1.047
1 1 1 1
SP2 ( + ) 34.30( + )
n m 10 8
Critical t-value = ±tn+m−2,α/2 ⇒ t16,0.025 = ±2.120
1.047 < 2.120 and 1.047 > −2.120 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the two distri-
butions have the same means
Sol 3: p-value approach
(x̄ − ȳ) − △0 (21.01 − 18.1) − 0
Test statistics t0 = r = r = 1.047
1 1 1 1
SP2 ( + ) 34.30( + )
n m 10 8
0.865 < t0 < 1.071 ⇒ 0.3 < p-value < 0.4
0.3 > 0.05 ⇒ p-value > α ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the two distri-
butions have the same means

Note

t-scores and p-values are inversely related: a larger t-score generally corresponds to a smaller p-value, and vice
versa

Check assumptions

We start by checking normality assumption through Q-Q plots. The points are approximately around a straight line,
which indicates the normality assumption is reasonable for both populations.

In order to visually assess the equal variance assumption, we use boxplots to display the data in each sample.
1 > loc .1 = c (21.9 ,18.5 ,12.3 ,16.7 ,21.0 ,15.1 ,18.2 ,23.0 ,36.8 ,26.6)
2 > loc .2 = c (22.0 ,20.6 ,15.4 ,17.9 ,24.4 ,15.6 ,11.4 ,17.5)
3 # C r e a t e s i d e by s i d e b o x p l o t s
4 > b o x p l o t ( l o c . 1 , l o c . 2 , x l a b = ” ” , names= c ( ” L o c a t i o n 1 ” , ” L o c a t i o n 2 ” ) , y l a b = ” Weight ( i n o u n c e s ) ” , y l i m = c
( 0 , 4 0 ) , c o l =” grey ” )

Page 110 of 160

 STATISTICS OVERVIEW

Figure 9..1: Boxplots of fish weights by location

To be formal, we can also perform a hypothesis test of

σ12
H0 : =1
σ22
σ2
Ha : 12 ̸= 1
σ2
The R output
1 > loc .1 = c (21.9 ,18.5 ,12.3 ,16.7 ,21.0 ,15.1 ,18.2 ,23.0 ,36.8 ,26.6)
2 > loc .2 = c (22.0 ,20.6 ,15.4 ,17.9 ,24.4 ,15.6 ,11.4 ,17.5)
3 > var . t e s t ( loc . 1 , loc . 2 )
4 F t e s t t o compare two v a r i a n c e s
5 d a t a : l o c . 1 and l o c . 2
6 F = 2 . 7 8 0 3 , num d f = 9 , denom d f = 7 , p− v a l u e = 0 . 1 9 1 4
7 a l t e r n a t i v e h y p o t h e s i s : t r u e r a t i o of v a r i a n c e s i s not equal to 1
8 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
9 0.5764409 11.6690466
10 sample e s t i m a t e s :
11 r a t i o of variances
12 2.780299

Recall: A 100(1 − α)% confidence interval of µ1 − µ2 is given by

r
1 1
(x̄ − ȳ) ± tn+m−2,α/2 SP2 ( + )
| {z } | {z }| n m
point estimator quantile
{z }
standard error

The sample summary statistic is

Variable n mean x̄ variance S12

Location 1 10 21.01 47.65
Variable n mean ȳ variance S22
Location 2 8 18.1 17.14

tn+m−2,α/2 = t16,0.025 = 2.120

The R output is
1 > t . t e s t ( l o c . 1 , l o c . 2 , c o n f . l e v e l = 0 . 9 0 , v a r . e q u a l =TRUE )
2 Two Sample t − t e s t

Page 111 of 160

 STATISTICS OVERVIEW

3 d a t a : l o c . 1 and l o c . 2
4 t = 1 . 0 4 7 4 , d f = 1 6 , p− v a l u e = 0 . 3 1 0 5
5 a l t e r n a t i v e h y p o t h e s i s : t r u e d i f f e r e n c e i n means i s n o t e q u a l t o 0
6 90 p e r c e n t c o n f i d e n c e i n t e r v a l :
7 −1.940438 7.760438
8 sample e s t i m a t e s :
9 mean o f x mean o f y
10 21.01 18.10

Interpretation: We are 90% confident that the mean weight difference in fish for location 1 and 4 is between −1.94
and 7.76 oz.
Note that this interval includes ”0”. By the relationship between hypothesis test and confidence interval, we fail to reject
H0 : µ1 − µ2 = 0 against Ha : µ1 − µ2 ̸= 0. at α = 0.1 level.
Therefore, we do not have sufficient evidence that the population mean fish weights 1 and µ2 are different at 0.1 level
of significance.
Note
We should only use this interval if there is strong evidence that the population variances σ12 and σ22 are equal (or
at least close). Otherwise, we should use a different interval
The two sample t interval (and the unequal variance version) is robust to normality departures. This means that we
can feel comfortable with the interval even if the underlying population distributions are not perfectly normal.

73) Inference on µ1 and µ2 assuming unknown σ12 and σ22 : σ12 ̸= σ22
73).1 Confidence Interval (CI) assuming unknown σ12 and σ22 : σ12 ̸= σ22
When σ12 ̸= σ22 , the construction of a 100(1 − α)% is no longer based on the exact t distribution
We need to approximate the degrees of freedom of a t distribution, and write an approximate confidence interval
An approximate 100(1 − α)% confidence interval µ1 − µ2 is given by
r r
S12 S2 S12 S2
[(x̄ − ȳ) − tν,α/2 + 2 , (x̄ − ȳ) + tν,α/2 + 2] (9..14)
n m n m

where the degrees of freedom ν is calculatee as

S12 S2
+ 2 )2
(
ν= n m (9..15)
S12 2 S2
( ) ( 2 )2
n + m
n−1 m−1

This interval is always approximately valid, as long

 the two samples are independent
 the two population distributions are approximately normal (Gaussian)
Interpretation: We are 100(1 − α)% confident that the population mean difference µ1 − µ2 is in this interval
73).2 Hypothesis Testing for µ1 and µ2 assuming unknown σ12 and σ22 : σ12 ̸= σ22
Null hypothesis:
H0 : µ1 − µ2 = △0
Alternative hypothesis:
Ha : µ1 − µ2 < △0
Ha : µ1 − µ2 > △0

Page 112 of 160

STATISTICS OVERVIEW

Ha : µ1 − µ2 ̸= △0
Confidence interval approach
Hypothesis type Alternative hypothesis Rejection criterion
r
S12 S2
Left-tail hypothesis Ha : µ1 − µ2 < △0 (x̄ − ȳ) + tν,α + 2 < △0
rn m
S12 S22
Right-tail hypothesis Ha : µ1 − µ2 > △0 (x̄ − ȳ) − tν,α + > △0
n m
r
S12 S2
Two-tail hypothesis Ha : µ1 − µ2 ̸= △0 △0 ∈ / [(x̄ − ȳ) ± tν,α/2 + 2]
n m
P -value approach
Test statistics:
(x̄ − ȳ) − △0
t0 = r (9..16)
1 1
SP2 ( + )
n m

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µ1 − µ2 < △0 p-value < α
Right-tail hypothesis Ha : µ1 − µ2 > △0 p-value < α
Two-tail hypothesis Ha : µ1 − µ2 ̸= 0 p-value < α

Note
There are two confidence intervals for µ1 − µ2 assuming unknown σ12 and σ22
 σ12 = σ22 (equal variance assumption)
 σ12 ̸= σ22 (unequal variance assumption)
If you are unsure about which interval to use, go with the unequal variance interval
The penalty for using it when σ12 = σ22 is much smaller than the penalty for using the equal variance interval when
σ12 ̸= σ22

Example: Recycling Project

You are part of a recycling project that is examining how much paper is being discarded (not recycled) by employees at
two large plants. These data are obtained on the amount of white paper thrown out per year by employees (data are in
hundreds of pounds). Samples of employees at each plant were randomly selected.
Are there differences in the mean amounts of white paper discarded by employees at the two plants? (use α = 0.05).

Plant 1: 3.01 2.58 3.04 1.75 2.87 2.57 2.51 2.93 2.85 3.09
1.43 3.36 3.18 2.74 2.25 1.95 3.68 2.29 1.86 2.63
2.83 2.04 2.23 1.92 3.02
Plant 2: 3.79 2.08 3.66 1.53 4.07 4.31 2.62 4.52 3.80 5.30
3.41 0.82 3.03 1.95 6.45 1.86 1.87 3.78 2.74 3.81

Solution
Pi Pi
xi 3.01 + 2.58 + ... + 1.92 + 3.02 (xi − x̄)2
x̄ = n = = 2.584 ⇒ S12 = n = 0.307
Pin 25 Pi n − 1 2
yi 3.79 + 2.08 + ... + 2.74 + 3.81 (yi − x̄)
ȳ = m = = 3.27 ⇒ S22 = m = 1.881
m 20 m−1
(n − 1)S12 + (m − 1)S22 (25 − 1)(0.307) + (20 − 1)(1.881)
SP2 = = = 1.002
n+m−2 25 + 20 − 2
(S12 /n + S22 /m)2 (0.307/25 + 1.881/20)2
Degree of freedom ν = 2 2 2 2 = = 24.19 ≈ 24
(S1 /n) (S2 /m) (0.307/25)2 (1.881/20)2
+ +
n−1 m−1 25 − 1 20 − 1

Page 113 of 160

STATISTICS OVERVIEW

Null hypothesis:
H0 : µ1 − µ2 = 0 where △0 = 0
Alternative hypothesis:
Ha : µ1 − µ2 ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
α = 0.05 ⇒ t24,0.025 = 2.064
α = 0.05 ⇒ confidence level r= 95% r
S12 S22 S12 S2
95% CI = [(x̄ − ȳ) − tν,α/2 + , (x̄ − ȳ) + tν,α/2 + 2]
n m
r n m r
0.307 1.881 0.307 1.881
⇔ 95% CI = (2.584 − 3.27) − 2.064 + , (2.584 − 3.27) + 2.064 +
25 20 25 20
⇔ 95% CI = (−1.359, −0.013)
/ (−1.312, 0.032) ⇒ reject H0
0∈
Conclusion: At 5% significance level, the data provide sufficient evidence to conclude amount of white paper
discarded by employees at two plants have different means
Sol 2: p-value approach
(x̄ − ȳ) − △0 (2.584 − 3.27) − 0
Test statistics t0 = r = r = −2.28
2 1 1 1 1
SP ( + ) 1.002( + )
n m 25 20
2.172 < |t0 | < 2.492 ⇒ 0.02 < p-value < 0.04
0.04 < 0.05 ⇒ p-value < α ⇒ reject H0
Conclusion: At 5% significance level, the data provide sufficient evidence to conclude amount of white paper
discarded by employees at two plants have different means

Figure 9..2: Boxplots of recycling project by plant

Check assumptions

To be formal, we can also perform a hypothesis test of

σ12
H0 : =1
σ22
σ2
Ha : 12 ̸= 1
σ2
The R output is

Page 114 of 160

 STATISTICS OVERVIEW

1 plant .1 = c (3.01 ,2.58 ,3.04 ,1.75 ,2.87 ,2.57 ,2.51 ,2.93 ,2.85 ,3.09 ,
2 + 1.43 ,3.36 ,3.18 ,2.74 ,2.25 ,1.95 ,3.68 ,2.29 ,1.86 ,2.63 ,
3 + 2.83 ,2.04 ,2.23 ,1.92 ,3.02)
4 plant .2 = c (3.79 ,2.08 ,3.66 ,1.53 ,4.07 ,4.31 ,2.62 ,4.52 ,3.80 ,5.30 ,
5 + 3.41 ,0.82 ,3.03 ,1.95 ,6.45 ,1.86 ,1.87 ,3.78 ,2.74 ,3.81)
6 var . t e s t ( plant . 1 , plant . 2 )
7 F t e s t t o compare two v a r i a n c e s
8 d a t a : p l a n t . 1 and p l a n t . 2
9 F = 0 . 1 6 3 5 , num d f = 2 4 , denom d f = 1 9 , p− v a l u e = 5 . 7 4 2 e −05
10 a l t e r n a t i v e h y p o t h e s i s : t r u e r a t i o of v a r i a n c e s i s not equal to 1
11 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
12 0.06668721 0.38352280
13 sample e s t i m a t e s :
14 r a t i o of variances
15 0.1635384

A 90% approximate (unequal variance) confidence interval for µ1 − µ2 is given by

1 > # Calculate t interval directly
2 > t . t e s t ( p l a n t . 1 , p l a n t . 2 , c o n f . l e v e l = 0 . 9 5 , v a r . e q u a l =FALSE )
3 Welch Two Sample t − t e s t
4 d a t a : p l a n t . 1 and p l a n t . 2
5 t = − 2 . 1 0 3 7 , d f = 2 3 . 9 7 2 , p− v a l u e = 0 . 0 4 6 0 8
6 a l t e r n a t i v e h y p o t h e s i s : t r u e d i f f e r e n c e i n means i s n o t e q u a l t o 0
7 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
8 −1.35825799 −0.01294201
9 sample e s t i m a t e s :
10 mean o f x mean o f y
11 2.5844 3.2700

We can also test for Ha : µ1 − µ2 < 0 and Ha : µ1 − µ2 > 0 by specifying ”alternative”

1 > t . t e s t ( p l a n t . 1 , p l a n t . 2 , c o n f . l e v e l = 0 . 9 5 , v a r . e q u a l =FALSE , a l t e r n a t i v e = ” l e s s ” )
2 Welch Two Sample t − t e s t
3 d a t a : p l a n t . 1 and p l a n t . 2
4 t = − 2 . 1 0 3 7 , d f = 2 3 . 9 7 2 , p− v a l u e = 0 . 0 2 3 0 4
5 a l t e r n a t i v e h y p o t h e s i s : t r u e d i f f e r e n c e i n means i s l e s s t h a n 0
6 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
7 − I n f −0.1280042
8 sample e s t i m a t e s :
9 mean o f x mean o f y
10 2.5844 3.2700
11 > t . t e s t ( p l a n t . 1 , p l a n t . 2 , c o n f . l e v e l = 0 . 9 5 , v a r . e q u a l =FALSE , a l t e r n a t i v e = ” g r e a t e r ” )
12 Welch Two Sample t − t e s t
13 d a t a : p l a n t . 1 and p l a n t . 2
14 t = − 2 . 1 0 3 7 , d f = 2 3 . 9 7 2 , p− v a l u e = 0 . 9 7 7
15 a l t e r n a t i v e h y p o t h e s i s : t r u e d i f f e r e n c e i n means i s g r e a t e r t h a n 0
16 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
17 −1.243196 I nf
18 sample e s t i m a t e s :
19 mean o f x mean o f y
20 2.5844 3.2700

Page 115 of 160

STATISTICS OVERVIEW

74) Inference on p1 and p2 : Independent samples

We also can extend our confidence interval procedure for a single population proportion p to two populations. De-
fine
p1 = population proportion of ”successes” in Population 1
p2 = population proportion of ”successess” in Population 2
For example, we might want to compare the proportion of
 defective circuit boards for two different suppliers
 satisfied customers before and after a product design change (e.g., Facebook, etc.)
 on-time payments for two classes of customers
 HIV positives for individuals in two demographic classes.
Point estimators: We assume that there are two independent random samples of individuals (one sample from each
population to be compared). Define
Y1 = number of ”successes” in Sample 1 (out of n1 individuals) ∼ B(n1 , p1 )
Y2 = number of ”successes” in Sample 2 (out of n2 individuals) ∼ B(n2 , p2 )
The point estimators for p1 and p2 are the sample proportions, defined by:

Y1
pb1 =
n1
(9..17)
Y2
pb2 =
n2
A natural point estimator for p1 − p2 is given by

Y1 Y2
pb1 − pb2 = − (9..18)
n1 n2

74).1 Confidence Interval (CI) of p1 and p2 : Independent samples

Recall that we have approximate sampling distribution pb1 and pb2

p1 (1 − p1 )
pb1 ∼ AN (p1 , )
n1
p2 (1 − p2 )
pb2 ∼ AN (p2 , )
n2

When the sample sizes n1 and n2 are large, the sampling distribution:

(pb1 − pb2 ) − (p1 − p2 )

Z=r ∼ AN (0, 1) (9..19)
p1 (1 − p1 ) p2 (1 − p2 )
+
n1 n2

An approximate 100(1 − α)% confidence interval for p1 − p2

s s
pb1 (1 − pb1 ) pb2 (1 − pb2 ) pb1 (1 − pb1 ) pb2 (1 − pb2 )
[(pb1 − pb2 ) − zα/2 + , (pb1 − pb2 ) + zα/2 + −] (9..20)
n1 n2 n1 n2

For the Z sampling distribution to hold approximately (and therefore for the interval above to be useful), we need
 the two samples to be independent

Page 116 of 160

STATISTICS OVERVIEW

 the sample sizes n1 and n2 to be ”large”, common rules of thumb are to require

ni pbi ≥ 5
n1 (1 − pbi ) ≥ 5

for each sample i = 1, 2

Under these conditions, the Central Limit Theorem should adequately approximate the true sampling distribution of
Z, thereby making the confidence interval formula above approximate valid
Interpretation: We are 100(1 − α)% confident that the population proportion difference p1 − p2 is in this interval

Note

The value zα/2 is the upper α/2 quantile from the N (0, 1) distribution

74).2 Hypothesis Testing for p1 and p2 : Independent samples

In two-sample situations, it is often of interest to see if the proportions p1 and p2 are different
While we are designing a test, we always want to bound the Type I Error, which is defined as “reject H0 when H0 in
fact is true.” So, when H0 is in fact true, we can view the two independent samples are from the same population; i.e.,
p1 = p2 = p. Then we have Z can be written as:

(pb1 − pb2 ) − (p1 − p2 ) pb1 − pb2

Z0 = q =r ∼ AN (0, 1) (9..21)
p1 (1−p1 )
+ p2 (1−p2 ) 1 1
n1 n2 p(1 − p)( + )
n1 n2

Further a natural estimator of p

Y1 + Y2
pb0 = (9..22)
n1 + n2
Thus, we have
pb1 − pb2
Z0 = r ∼ AN (0, 1) (9..23)
1 1
p̂(1 − p̂)( + )
n1 n2

Null hypothesis:
H0 : p1 = p2
Alternative hypothesis:
Ha : p1 < p2
Ha : p1 > p2
Ha : p1 ̸= p2
Critical value approach
Test statistics:
pb1 − pb2
z0 = r (9..24)
1 1
p̂(1 − p̂)( + )
n1 n2

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : p1 < p2 z0 < −zα
Right-tail hypothesis Ha : p1 > p2 z0 > zα
Two-tail hypothesis Ha : p1 ̸= p2 z0 > zα/2 or z0 < −zα/2

P -value approach

Page 117 of 160

STATISTICS OVERVIEW

Test statistics:
pb1 − pb2
z0 = r (9..25)
1 1
p̂(1 − p̂)( + )
n1 n2

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : p1 < p2 p-value = P (Z < z0 ) = Φ(z0 ) < α
Right-tail hypothesis Ha : p1 > p2 p-value = P (Z > z0 ) = 1 − Φ(z0 ) < α
Two-tail hypothesis Ha : p1 ̸= p2 p-value = 2P (Z < −|z0 |) = 2[1 − Φ(|z0 |)] < α

Example: A programmable lighting control system is being designed. The purpose of the system is to reduce electricity
consumption costs in buildings. The system eventually will entail the use of a large number of transceivers (a device
comprised of both a transmitter and a receiver). Two types of transceivers are being considered. In life testing, 200
transceivers (randomly selected) were tested for each type.
Transceiver 1: 20 failures were observed (out of 200)
Transceiver 2: 14 failures were observed (out of 200)
Define p1 (p2 ) to be the population proportion of Transceiver 1 (Transceiver 2) failures. Write a 95% confidence interval
for p1 − p2 . Is there a significant difference between the failure rates p1 and p2 ?

Solution
Y1 20
pb1 = = = 0.1
n1 200
Y2 14
pb2 = = = 0.07
n2 200
Y1 + Y2 20 + 14 34
p̂ = = = = 0.085
n1 + n2 200 + 200 400

Null hypothesis:
H0 : p1 − p2 = 0
Alternative hypothesis:
Ha : p1 − p2 ̸= 0
Sol 1: The test statistics (Critical value approach)
pb1 − pb2 0.1 − 0.07
Test statistics z0 = r =r = 1.075
1 1 1 1
p̂(1 − p̂)( + ) 0.085(1 − 0.085)( + )
n1 n2 200 200
100(1 − α)% = 95% ⇒ 1 − α = 0.95 ⇒ α = 0.05
Critical z-value = ±zα/2 ⇒ z0.025 = ±1.960
1.075 < 1.960 and 1.075 > −1.96 ⇒ fail to reject H0
Conclusion: At 5% signficance level, the data do not provide sufficient evidence to conclude failure rates p1 and
p2 have difference
Sol 2: p-value approach
pb1 − pb2 0.1 − 0.07
Test statistics z0 = r =r = 1.075 ≈ 1.08
1 1 1 1
p̂(1 − p̂)( + ) 0.085(1 − 0.085)( + )
n1 n2 200 200
z0 = 1.08 ⇒ p-value = 2P (Z < −|z0 |) = 2P (Z < −1.08) = 2[1 − Φ(1.08)] = 2(1 − 0.8599) = 0.2802
100(1 − α)% = 95% ⇒ 1 − α = 0.95 ⇒ α = 0.05
p-value > α ⇒ fail to reject H0
Conclusion: At 5% signficance level, the data do not provide sufficient evidence to conclude failure rates p1 and
p2 have difference

Example: Proportion of Exceedence

Airplanes approaching the runway for landing are required to stay within the localizer (a certain distance left and right

Page 118 of 160

 STATISTICS OVERVIEW

of the runway). When an airplane deviates from the localizer, it is sometimes referred to as an exceedence. Consider
two airlines at a large airport. During a three-week period, airline 1 had 14 exceedences out of 156 flights and airline 2
had 11 exceedences out of 198 flights.
We are interested in whether airline 1 has a significantly higher exceedence rate than airline 2 or not.

Solution

Y1 14
pb1 = = = 0.0897
n1 156
Y2 11
pb2 = = = 0.0556
n2 198
Y1 + Y2 14 + 11
p̂ = = = 0.0706
n1 + n2 156 + 198

Assume significance level α = 0.05

Null hypothesis:
H0 : p1 = p2
Alternative hypothesis:
Ha : p1 > p2
Sol 1: Critical value approach
pb1 − pb2 0.0897 − 0.0556
Test statistics z0 = r =r = 1.247
1 1 1 1
p̂(1 − p̂)( + ) 0.0706(1 − 0.0706)( + )
n1 n2 156 198
α = 0.05 ⇒ z0.05 = 1.96
1.247 < 1.96 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude airline 1 has a sig-
nificantly higher exceedence rate than airline 2
Sol 2: p-value approach
pb1 − pb2 0.0897 − 0.0556
Test statistics z0 = r =r = 1.247 ≈ 1.25
1 1 1 1
p̂(1 − p̂)( + ) 0.0706(1 − 0.0706)( + )
n1 n2 156 198
z0 = 1.25 ⇒ p-value = P (Z > z0 ) = 1 − Φ(z0 ) = 1 − Φ(1.25) = 1 − 0.8944 = 0.1056
p-value > α ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude airline 1 has a
significantly higher exceedence rate than airline 2

1 > prop . t e s t ( c ( 1 4 , 1 1 ) , c ( 1 5 6 , 1 9 8 ) , a l t e r n a t i v e = ” g r e a t e r ” , c o r r e c t =F )
2 2− s a m p l e t e s t f o r e q u a l i t y o f p r o p o r t i o n s w i t h o u t c o n t i n u i t y
3 correction
4 data : c ( 1 4 , 11) out of c ( 1 5 6 , 198)
5 X− s q u a r e d = 1 . 5 5 3 8 , d f = 1 , p− v a l u e = 0 . 1 0 6 3
6 a lt er na ti ve hypothesis : greater
7 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
8 −0.01200417 1.00000000
9 sample e s t i m a t e s :
10 prop 1 prop 2
11 0.08974359 0.05555556

75) F Distribution

The F distribution is a univariate continuous distribution often used in hypothesis testing

Page 119 of 160

STATISTICS OVERVIEW

A random variable X has an F distribution if it can be written as

Y1 /ν1
X= (9..26)
Y2 /ν2

where
 Chi-square random variable Y1 with ν1 degree of freedom
 Chi-square random variable Y2 with ν2 degree of freedom
 Y1 and Y2 are independent
There is an important connection between an F variable and chi-squared variables

75).1 Definition
Let X be a continuous random variable. Let it supports be the set of positive real numbers:

RX = [0, ∞)

Let ν1 , ν2 ∈ N. We say that X has an F distribution with ν1 and ν2 degrees of freedom if and only if its probability
density function (pdf) is 
cxν1 /2−1 (1 + ν1
ν2 x)
−(ν1 +ν2 )/2
if x ∈ RX
fX (x) = (9..27)
0 if x ∈
/ RX

where c is a constant:
ν1 ν1 /2 1
c=( ) ν1 ν2 (9..28)
ν2 B( 2 , 2 )
and B() is a beta function

Expected value

The expected value of an F random variable X is well-defined only for ν2 > 2

ν2
E[X] = (9..29)
ν2 − 2

Variance

The variance of an F random variable X is well-defined only for n2 > 4

2ν22 (ν1 + ν2 − 2)
var[X] = (9..30)
ν1 (ν2 − 2)2 (ν2 − 4))

Page 120 of 160

STATISTICS OVERVIEW

75).2 Characteristics
The F distribution has the following characteristics:
 continuous, skewed right, not symmetric and always positive
 indexed by two degrees of freedom parameters ν1 and ν2 , these are usually integers and are often related to sample
sizes

Figure 9..3: F probability density function with different degrees of freedom

75).3 F Tables
An F distribution with numerator degree of freedom ν1 = n1 − 1 and denominator degree of freedom ν2 = n2 − 1
 F Distribution Table (1)
 F Distribution Table (2)

76) Inference on ratio of population variances σ22 /σ12 : Independent samples

You will recall that when we wrote a confidence interval for µ1 − µ2 , the difference of the population means (with
independent samples), we proposed two different intervals:
 one interval that assumed σ12 = σ22
 one interval that assumed σ12 ̸= σ22

76).1 Confidence Interval (CI) of σ22 /σ12 : Independent samples

Suppose that we have two independent samples:
Sample 1: X1 , X2 , ..., Xn ∼ N (µ1 , σ12 ) random sample
Sample 2: Y1 , Y2 , .., Ym ∼ N (µ2 , σ22 ) random sample
Sampling distribution:
S12 /σ12
R= ∼ F (n1 − 1, n2 − 1) (9..31)
S22 /σ22
with ν1 = n − 1 and ν2 = m − 1
σ22
A 100(1 − α)% confidence interval for the ratio of population variances :
σ12

S22 S2
( 2 × Fn−1,m−1,1−α/2 , 22 × Fn−1,m−1,α/2 ) (9..32)
S1 S1

Page 121 of 160

STATISTICS OVERVIEW

Proof:
1 − α = P (Fn−1,m−1,1−α/2 ≤ R ≤ Fn−1,m−1,α/2 )
S 2 /σ 2
⇔ 1 − α = P (Fn−1,m−1,1−α/2 ≤ 12 12 ≤ Fn−1,m−1,α/2 )
S2 /σ2
S22 σ2 S2
⇔ 1 − α = P ( 2 × Fn−1,m−1,1−α/2 ≤ 22 ≤ 22 × Fn−1,m−1,α/2 )
S1 σ1 S1
Based on the relationship:
1
Fν2 ,ν1 ,α =
Fν1 ,ν2 ,1−α
σ12
A 100(1 − α)% confidence interval for
σ22

S12 1 S2 1 S2 1 S2
( 2 × , 12 × ) = ( 12 × , 12 × Fm−1,n−1,α/2 ) (9..33)
S2 Fn−1,m−1,α/2 S2 Fn−1,m−1,1−α/2 S2 Fn−1,m−1,α/2 S2

Taking square root of the interval:

σ2
 A 100(1 − α)% confidence interval for
σ1

S2 q S2 q
( × Fn−1,m−1,1−α/2 , × Fn−1,m−1,α/2 ) (9..34)
S1 S1
σ1
 A 100(1 − α)% confidence interval for
σ2
s s s
S1 1 S1 1 S1 1 S1 q
( × , × )=( × , × Fm−1,n−1,α/2 ) (9..35)
S2 Fn−1,m−1,α/2 S2 Fn−1,m−1,1−α/2 S2 Fn−1,m−1,α/2 S2

76).2 Hypothesis Testing for σ22 /σ12 : Independent samples

Note

When H0 is true:
S12 /σ12 S12
r0 = = ∼ F (n − 1, m − 1)
S22 /σ22 S22

Null hypothesis:
H0 : σ12 = σ22
Alternative hypothesis:
Ha : σ12 < σ22
Ha : σ12 > σ22
Ha : σ12 ̸= σ22
Confidence interval approach
Hypothesis type Alternative hypothesis Rejection criterion
S2
Left-tail hypothesis σ12 < σ22 1 > 12 × Fm−1,n−1,α
S2
S2 1
Right-tail hypothesis Ha : σ12 > σ22 1 < 12 ×
S2 Fn−1,m−1,α
S12 1 S2
Two-tail hypothesis σ12 ̸= σ22 1∈/( 2× , 12 × Fm−1,n−1,α/2 )
S2 Fn−1,m−1,α/2 S2
Critical value approach

Page 122 of 160

 STATISTICS OVERVIEW

Test statistics:
s21
r0 = (9..36)
s22

Hypothesis type Alternative hypothesis Rejection criterion

1
Left-tail hypothesis σ12 < σ22 r0 < Fn−1,m−1,1−α =
Fm−1,n−1,α
Right-tail hypothesis σ12 > σ22 r0 > Fn−1,m−1,α
Two-tail hypothesis σ12 ̸= σ22 r0 > Fn−1,m−1,α/2 or r0 < Fn−1,m−1,1−α/2 =
1
Fm−1,n−1,α/2
P -value approach
Test statistics:
s21
r0 = (9..37)
s22

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis σ12 < σ22 p-value < α
Right-tail hypothesis σ12 > σ22 p-value < α
Two-tail hypothesis σ12 ̸= σ22 p-value < α

Some statisticians recommend to use this “equal/unequal variance test” before deciding which confidence interval or
testing procedure to use for µ1 − µ2 . Some statisticians do not
Like the χ2 interval for single population variance σ2 , the two-sample F interval for the ratio of two variances is not
robust to departures from normality.
 If the underlying population distributions are non-normal (non-Gaussian), then this interval should not be used
One easy solution, to check whether the sample is from a normal distribution or not, is the so-called ”normal qq plot”.
It plots sample quantiles against theoretical normal quantiles
 If it fits well by a line, it is normal
 Otherwise, non-normal

1 > d a t a =rnorm ( 1 0 0 , 3 , 2 ) # g e n e r a t e 1 0 0 s a m p l e s 1 > d a t a = r e x p ( 1 0 0 , 1 ) # g e n e r a t e 1 0 0 s a m p l e s from

from N ( 3 , 4 ) Exp ( 1 )
2 > qqnorm ( d a t a ) # p l o t t h e d o t s 2 > qqnorm ( d a t a ) # p l o t t h e d o t s
3 > q q l i n e ( d a t a ) # f i t t h e d o t s by a l i n e 3 > q q l i n e ( d a t a ) # f i t t h e d o t s by a l i n e

Example: We consider again the recycling project that examined the amount of white paper discarded per employee

Page 123 of 160

STATISTICS OVERVIEW

at two large plants. The data were obtained on the amount of white paper thrown out per year by employees (data are
in hundreds of pounds). Samples of employees at each plant (n = 25 and m = 20) were randomly selected
The boxplots did suggest that the population variances may be different The normal qq plots

Plant 1: 3.01 2.58 3.04 1.75 2.87 2.57 2.51 2.93 2.85 3.09
1.43 3.36 3.18 2.74 2.25 1.95 3.68 2.29 1.86 2.63
2.83 2.04 2.23 1.92 3.02
Plant 2: 3.79 2.08 3.66 1.53 4.07 4.31 2.62 4.52 3.80 5.30
3.41 0.82 3.03 1.95 6.45 1.86 1.87 3.78 2.74 3.81

Figure 9..4: Boxplots of recycling project by plant

Figure 9..5: qq plot - recycling project

Find a 95% confidence interval for σ22 /σ12 , the ratio of the population variances. Here σ12 (σ22 ) denotes the population
variance of the amount of white paper by employees at Plant 1 (Plant 2)

Solution
Pi Pi
n xi3.01 + 2.58 + ... + 1.92 + 3.02 2 (xi − x̄)2
x̄ = = = 2.584 ⇒ S1 = n = 0.307
Pin 25 Pi n − 1 2
yi 3.79 + 2.08 + ... + 2.74 + 3.81 (yi − x̄)
ȳ = m = = 3.27 ⇒ S22 = m = 1.881
m 20 m−1
100(1 − α) = 95 ⇒ 1 − α = 0.95 ⇒ α = 0.05

Page 124 of 160

STATISTICS OVERVIEW

S22 S22 S22 1 S22

95% CI = ( × Fn−1,m−1,1−α/2 , × F n−1,m−1,α/2 ) = ( × , × Fn−1,m−1,α/2 )
S12 S12 S12 Fm−1,n−1,α/2 S12
1.881 1 S2 1.881 1 1.881
⇔ 95% CI = × , 22 × F24,19,0.025 = ( × , × 2.4523)
0.307 F19,24,0.025 S1 0.307 2.3452 0.307
⇔ 95% CI = (2.613, 15.03)

77) Inference on µ1 and µ2 : Dependent samples (Matched-pairs)

Example: Creatine
Creatine is an organic acid that helps to supply energy to cells in the body, primarily muscle. Because of this, it is com-
monly used by those who are weight training to gain muscle mass. Does it really work? Suppose that we are designing
an experiment involving USC male undergraduates who exercise/lift weight regularly
Design 1 (Independent samples): Recruit 30 students who are representative of the population of USC male under-
graudates who exercise/lift weights. For a single weight training session, we will
 assign 15 students to take creatine
 assign 15 students an innocuous substance that looks like creatine (but has no positive/negative effect on performance)
For each student, we will record
Y = maximum bench press weight (MBPW)
We will then have two samples of data (with n = 15 and m = 15):
Sample 1 (Creatine): X1 , X2 , ..., Xn
Sample 2 (Control): Y1 , Y2 , ..., Ym
To compare the population means
µ1 = population mean MBPW for students taking creatine
µ2 = population mean MBPW for students not taking creatine
We could construct a two-sample t confidence interval for µ1 − µ2 using
r r
11 1 1
[(X̄ − Ȳ ) − tn+m−2,α/2 SP2 ( + ), (X̄ − Ȳ ) + tn+m−2,α/2 SP2 ( + )]
n m n m

or s s
S12 S2 S12 S2
[(X̄ − Ȳ ) − tν,α/2 + 2 , (X̄ − Ȳ ) + tν,α/2 + 2]
n1 n2 n1 n2

depending on our underlying assumptions about σ12 and σ22

Design 2 (Mathced pairs): Recruit 15 students who are representative of the population of USC male undergraduates
who exercise/lift weights
 Each student will be assigned first to take either creatine or the control substance.
 For each student, we will then record his value of Y (MBPW).
 After a period of recovery (e.g., 1 week), we will then have each student take the other “treatment” (creatine/control)
and record his value of Y again (but now on the other treatment).
 In other words, for each individual student, we will measure Y under both conditions.
Note: In Design 2, because MBPW measurements are taken on the same student, the difference between the measure-
ment (creatine/control) should be less variable than the difference between a creatine measurement on one student and
a control measurement on a different student.
 In other words, the student-to-student variation inherent in the latter difference is not present in the difference be-
tween MBPW measurements taken on the same individual student.
Matched pairs: In general, by obtaining a pair of measurements on a single individual (e.g., student, raw material,
machine, etc.), where on of measurement corresponds to ”Treatment 1” and the other measurement corresponds to
”Treatment 2”, you eliminate variation among the individuals.

Page 125 of 160

STATISTICS OVERVIEW

Design Source of Variation

Two Independent Samples among students, within students
Matched Pairs within students

This allows you to compare the two experimental conditions (e.g., creatine/control, biodegradability treatments, opera-
tors, etc.) under more homogeneous conditions where only variation within individuals is present (that is, the variation
arising from the difference in the two experimental conditions).
Advantage: When you remove extra variability, this enables you to do a better job at comparing the two experimental
conditions (treatments). By “better job,” I mean, you can more precisely estimate the difference between the treatments
(excess variability that naturally arises among individuals is not getting in the way). This gives you a better chance of
identifying a difference between the treatments if one really exists
Note: In matched pairs experiments, it is important to randomize the order in which treatments are assigned. This may
eliminate “common patterns” that may be seen when always following, say, Treatment 1 with Treatment 2. In practice,
the experimenter could flip a fair coin to determine which treatment is applied first

Student j Creatine MBPW (Xj ) Control MBPW (Yi ) Difference (Dj = Xj − Yj )

1 230 200 30
2 140 155 −15
3 215 205 10
4 190 190 0
5 200 170 30
6 230 225 5
7 220 200 20
8 255 260 −5
9 220 240 −20
10 200 195 5
11 90 110 −20
12 130 105 25
13 255 230 25
14 80 85 −5
15 265 255 10

77).1 Implementation
Note

Basically, you take the difference, then perform a T -test on the differences

Data from matched pairs experiments are analyzed by examining the difference in responses of the two treatments.
Specifically, compute
Dj = Xj − Yj (9..38)

for each individual j = 1, 2, ..., n. After doing this, we have essentially created a ”one sample problem”, where our data
are the so-called data differences:
D1 , D2 , ..., Dn
The one sample 100(1 − α))% confidence interval

SD SD
(D − tn−1,α/2 √ , D + tn−1,α/2 √ ) (9..39)
n n

where D and SD are the sample mean and sample standard deviation of the differnces, respectively, is an interval esti-
mate for

Page 126 of 160

STATISTICS OVERVIEW

µD = mean difference between 2 treatments

Interpretation: We are 100(1 − α)% confident that the mean different µD is in this interval

77).2 Hypothesis Testing for µ1 and µ2 : Dependent samples (Matched-pairs)

Note
Basically, you need to run a T -test on the differences

µD = µ1 − µ2
Null hypothesis:
H0 : µD = △0
Alternative hypothesis:
Ha : µD < △0
Ha : µD > △0
Ha : µD ̸= △0
Confidence interval approach
Hypothesis type Alternative hypothesis Rejection criterion
SD
Left-tail hypothesis Ha : µD < △0 △0 > D + tn−1,α √
n
SD
Right-tail hypothesis Ha : µD > △0 △0 < D − tn−1,α √
n
SD
Two-tail hypothesis Ha : µD ̸= △0 / (D ± tn−1,α/2 √ )
△0 ∈
n
The test statistics (Critical value approach)
Test statistics:
D − △0
t0 = √ (9..40)
SD / n

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µD < △0 t0 < −tn−1,α
Right-tail hypothesis Ha : µD > △0 t0 > tn−1,α
Two-tail hypothesis Ha : µD ̸= △0 t0 > tn−1,α/2 or t0 < −tn−1,α/2
P -value approach
Test statistics:
D − △0
t0 = √ (9..41)
SD / n

Hypothesis type Alternative hypothesis Rejection criterion

Left-tail hypothesis Ha : µD < △0 p-value < α
Right-tail hypothesis Ha : µD > △0 p-value < α
Two-tail hypothesis Ha : µD ̸= △0 p-value < α

Example: Creatine (continue)

Find a 95% confidence interval for µD and test whether there is a differnce using α = 0.05
Solution
Pi
Di 30 + (−15) + ... + (−5) + 10 95
D= n = = = 6.333
Pin 15 15
(Di − D)
q
2
SD = n = 298.1 ⇒ SD = SD 2 = 17.27
n−1
100(1 − α) = 95 ⇒ 1 − α = 0.95 ⇒ α = 0.05

Page 127 of 160

 STATISTICS OVERVIEW

SD SD SD SD
95% CI = (D − tn−1,α/2 √ , D + tn−1,α/2 √ ) = (D − t14,0.025 √ , D + t14,0.025 √ )
n n n n
17.27 17.27
⇔ 95% CI = (6.333 − 2.145 √ , 6.333 + 2.145 √ )
15 15
⇔ 95% CI = (−3.23, 15.9)
Null hypothesis:
H0 : µD = 0 where △0 = 0
Alternative hypothesis:
Ha : µD ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
0 ∈ (−3.23, 15.9) ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude there is a difference
in mean maximum bench press weight (MBPW)
Sol 2: Critical value approach
D − △0 6.333 − 0
Test statistics t0 = √ = √ = 1.422
SD / n 17.27/ 15
α = 0.05 ⇒ t14,0.025 = ±2.145
1.422 < 2.145 and 1.422 > −2.145 ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude there is a difference
in mean maximum bench press weight (MBPW)
Sol 3: p-value approach
D − △0 6.333 − 0
Test statistics t0 = √ = √ = 1.422
SD / n 17.27/ 15
1.345 < 1.422 < 1.761 ⇒ 0.1 < p-value < 0.2
0.1 > 0.05 ⇒ p-value > α ⇒ fail to reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude there is a difference
in mean maximum bench press weight (MBPW)

1 > creatine = c (230 ,140 ,215 ,190 ,200 ,230 ,220 ,255 ,220 ,200 ,90 ,130 ,255 ,80 ,265)
2 > control = c (200 ,155 ,205 ,190 ,170 ,225 ,200 ,260 ,240 ,195 ,110 ,105 ,230 ,85 ,255)
3 > t . t e s t ( c r e a t i n e , c o n t r o l , c o n f . l e v e l = 0 . 9 5 , a l t e r n a t i v e = ” two . s i d e d ” , p a i r e d =TRUE )
4 Paired t − t e s t
5 d a t a : c r e a t i n e and c o n t r o l
6 t = 1 . 4 2 0 7 , d f = 1 4 , p− v a l u e = 0 . 1 7 7 3
7 a l t e r n a t i v e h y p o t h e s i s : t r u e d i f f e r e n c e i n means i s n o t e q u a l t o 0
8 95 p e r c e n t c o n f i d e n c e i n t e r v a l :
9 −3.227946 15.894612
10 sample e s t i m a t e s :
11 mean o f t h e d i f f e r e n c e s
12 6.333333

78) One-way analysis of variance (ANOVA)

When there are two means, we have recently seen that the design of the experiment/studey completely determines
how the data are to be analyzed
 When the two samples are independent, this is called a (two) independent-sample design
 When the two samples are obtained on the same individuals (dependent samples), this is called a matched-pairs design
 Confidence interval procedures for µ1 − µ2 depend on the design of the study
The purpose of an experiment is to investigate differences between or among two or more treatments. In a statistical
framework, we do this by comparing the population means of the responses to each treatment

Page 128 of 160

 STATISTICS OVERVIEW

 In order to detect treatment mean differences, we must try to control the effects of error so that any variation we
observe can be attributed to the effects of the treatments rather than to differences among the individuals
Blocking: Designs involving meaningful grouping of individuals, that is, blocking, can help reduce the effects of ex-
perimental error by identifying systematic components of variations among individuals
 The matched pairs design for comparing two treatments is an example of such a design. In this situation, individuals
themselves are treated as ”blocks”

One-way Classification: Consider an experiment to compare t ≥ 2 treatments set up as follows:

 We obtain a random sample of individuals and randomly assign them to treatments. Samples corresponding to
the treatment groups are independent (i.e., the individuals in each treatment sample are unrelated).
 In observational studies (where no treatment is physically applied to individuals), individuals are inherently
different to begin with. We therefore simply take random samples from each treatment population
 We do not attempt to group individuals according to some other factor (e.g., location, gender, weight, variety,
etc.). This would be an example of blocking. Main point: In a one-way classification design, the only way in which
individuals are “classified” is by the treatment group assignment. Hence, such an arrangement is called a one-way
classification. When individuals are thought to be “basically alike” (other than the possible effect of treatment),
experimental error consists only of the variation among the individuals themselves, that is, there are no other
systematic sources of variation

Example: Four types of mortars: (1) ordinary cement mortar (OCM), polymer impregnated mortar (PIM), resin mortar
(RM), and (4) polymer cement mortar (PCM), were subjected to a compression test to measure strength (MPa). Here are
the strength measurements taken on different mortar specimens (36 in all)

OCM: 51.45 42.96 41.11 48.06 38.27 38.88 42.74 49.62

PIM: 64.97 64.21 57.39 52.79 64.87 53.27 51.24 55.87 61.76 67.15
RM: 48.95 62.41 52.11 60.45 58.07 52.16 61.71 61.06 57.63 56.80
PCM: 35.28 38.59 48.64 50.99 51.52 52.85 46.75 48.31

Figure 9..6: Boxplots of strength measurements (MPa) for four mortar types

1 > ocm <− c ( 5 1 . 4 5 , 42.96 , 41.11 , 48.06 , 38.27 , 38.88 , 42.74 , 49.62)
2 > pim <− c ( 6 4 . 9 7 , 64.21 , 57.39 , 52.79 , 64.87 , 53.27 , 51.24 , 55.87 , 61.76 , 67.15)
3 > rm <− c ( 4 8 . 9 5 , 62.41 , 52.11 , 60.45 , 58.07 , 52.16 , 61.71 , 61.06 , 57.63 , 56.80)
4 > pcm <− c ( 3 5 . 2 8 , 38.59 , 48.64 , 50.99 , 51.52 , 52.85 , 46.75 , 48.3)

Page 129 of 160

 STATISTICS OVERVIEW

5 > b o x p l o t ( ocm , pim , rm , pcm , y l a b = ” S t r e n g t h ( MPa ) ” , names = c ( ”OCM” , ” PIM ” , ”RM” , ”PCM” ) , y l i m = c
(0 , 100) )

In this example:
 “Treatment” = mortar type (OCM, PIM, RM, and PCM). There are t = 4 treatment groups
 Individuals = mortar specimens
 This is an example of an observational study; not an experiment. That is, we do not physically apply a treatment here;
instead, the mortar specimens are inherently different to begin with. We simply take random samples of each mortar
type.
Let:
 t = the number of populations or treatments being compared
 µ1 = the mean of population 1 or the true average response when treatment 1 is applied
.
 ..
 µt = the mean of population t or the true average response when treatment t is applied
Assumptions: We have independent random samples from t ≥ 2 normal distributions, each of which has the same
variance (but possibly different means)
Sample 1: Y11 , Y12 , ..., Y1n1 ∼ N (µ1 , σ 2 )
Sample 2: Y21 , Y22 , ..., Y2n2 ∼ N (µ2 , σ 2 )
.. ..
. .
Sample t: Yt1 , Yt2 , ..., Ytnt ∼ N (µt , σ 2 )
The t population or treatment distributions are all normal with the same variance σ 2 . That is, each Xi,j is normally
distributed with

E(Xi,j ) = µi
(9..42)
var(Xi,j ) = σ 2

Statistical Hypothesis:
Null hypothesis:
H0 : µ1 = µ2 = ... = µt
Alternative hypothesis:
Ha : the population mean µi are not all equal
 The null hypothesis H0 says that there is “no treatment difference,” that is, all treatment population means are the
same.
 The alternative hypothesis Ha says that a difference among the t population means exists somewhere (but does not
specify how the means are different).
 The goal is to decide which hypothesis is more supported by the observed data
Notation: Let t denote the number of treatments to be compared. Define:
 Xi,j = the random variable that denotes the jth measurement taken from the i population, or the measurement taken
on the jth experimental unit that receives the ith treatment (for i = 1, 2, ..., t and j = 1, 2, ..., ni )
 xi,j = the observed value of Xi,j when the experiment is performed
 ni = the number of replications for treatment i
 When n1 = n2 = ... = nt = n, we say the design is balanced, otherwise, the design is unbalanced
 Let N = n1 + n2 + ... + nt denote the total number of individuals measured. If the design is balanced, then

N = nt (9..43)

Page 130 of 160

STATISTICS OVERVIEW

The sample mean of the ith sample:

ni
1 X
Xi = Xi,j (9..44)
ni j=1

The sample variance of the ith sample:

ni
1 X
Si2 = (Xi,j − Xi )2 (9..45)
ni − 1 j=1

The sample mean of all the data (across all t treatment groups) (also called the grand mean):

t ni
1 XX
X.. = Xi,j (9..46)
N i=1 j=1

Note
Rule of Thumb: If the largest s is not much more than two times the smallest, it is reasonable to assume equal
σ 2 ’s
The individual sample sizes in ANOVA are often too small for t separate plots to be informative. A single plot can be
constructed by subtracting x1 from each observation in the first sample, x2 from each observation in the second, and
so on, and then plotting these deviations against the z percentiles

Note
The straightness of the pattern gives strong support to the normality assumption.

The procedure we develop is formulated by deriving two estimators for σ 2 . These two estimators are formed by
(1) looking at the variance of the observations within samples
(2) looking at the variance of the sample means across the t samples.
 ”WITHIN” Estimator: To estimate σ 2 within samples, we take a weighted average (weighted by the sample size) of
the t sample variance, that is, we ”pool” all variance estimates together to form one estimate.
The residual sum of squares:

ni
t X
X
SSres = (n1 − 1)S12 + (n2 − 1)S22 + ... + (nt − 1)St2 = (Xi,j − Xi )2 (9..47)
i=1 j=1
| {z }
(ni −1)Si2

Mathematics shows that

SSres
E( ) = N − t =⇒ E(M Sres ) = σ 2 (9..48)
σ2

Page 131 of 160

STATISTICS OVERVIEW

where residual mean squares:

SSres
M Sres = (9..49)
N −t

Note
M Sres is an unbiased estimator of σ 2 regardless of whether or not H0 is true.

 ”ACROSS” Estimator: To derive the ”across-sample” estimator, we assume a common sample size n1 = n2 = ... =
nt = n (to simplify notation). Recall that if a sample arises from a normal population, then the sample mean is also
normally distributed, i.e.,
σ2
Xi ∼ N (µi , )
n

Note
If all the treatment population means are equal, that H0 : µ1 = µ2 = ... = µt = µ is true, then

σ2
Xi ∼ N (µ, )
n
The sample variance of this ”random sample”

t
1 X
Si2 = (Xi − X.. )2 (9..50)
t − 1 i=1

The expected value of this ”random sample”

t
1 X σ2
E[ (Xi − X.. )2 ] = (9..51)
t − 1 i=1 n

Therefore, the treatment mean squares

t
1 X
M Strt = n(Xi − X.. )2 (9..52)
t − 1 i=1

where the treatment sums of squares

t
X
SStrt = n(Xi − X.. )2 (9..53)
i=1

M Strt is second point estimator for σ 2 . Recall that M Strt is an unbiased estimator of σ 2 only when H0 : µ1 = µ2 = ... = µt
is true. If we have different sample sizes, we simply adjust M Strt

t
1 X
M Strt = ni (Xi − Xii )2 (9..54)
t − 1 i=1
| {z }
SStrt

Motivation:
 When H0 is true (i.e., the treatment means are the same), then

E(M Strt ) = σ 2
(9..55)
E(M Sres ) = σ 2

These two facts suggest that when H0 is true

M Strt
F = ≈1 (9..56)
M Sres

Page 132 of 160

STATISTICS OVERVIEW

 When H0 is not true (i.e., the treatment means are different), then

E(M Strt ) > σ 2

(9..57)
E(M Sres ) = σ 2

These two facs suggest than when H0 is not true

M Strt
F = >1 (9..58)
M Sres

Null hypothesis:
H0 : µ1 = µ2 = ... = µt
Alternative hypothesis:
Ha : the population mean µi are not all equal
The test statistics
M Strt
F = ∼ F (t − 1, N − t) (9..59)
M Sres
The rejection criterion:
M Strt
F = > Ft−1,N −1,α (9..60)
M Sres
We ”reject H0 ” and conclude the treatment population means are different if the F statistics is far out in the right
tail of the F (t − 1, N − t) distribution. Because a large value of F is not consistent with H0 being true. Large
values of F (far out in the right tail) are more consistent with Ha

Page 133 of 160

STATISTICS OVERVIEW

10. LINEAR REGRESSION

79) Introduction

IMPORTANT: A problem that arises in engineering, economics, medicine, and other areas is that investigating the
relationship betwene two (or more) variables. In such settings, the goal is to model a continuous random variable Y as
a function of one or more independent variables, say, x1 , x2 , ..., xk . Mathematically, we can express this model as

Y = g(x1 , x2 , ..., xk ) + ϵ (10..1)

where g : Rk ⇒ R. This is called a regression model  The presence of the (random) error ϵ conveys the fact that the
relationship between the depenedent variable Y and the independent variables x1 , x2 , .., xk through g is not determin-
istic. Instead, the term ϵ ”absorbs” all variation in Y that is not explained by g(x1 , x2 , ..., xk )
The form of linear models:
Y = β0 + β1 x1 + ... + βk xk +ϵ (10..2)
| {z }
g(x1 ,x2 ,...,xk )

that is g is a linear function of β0 , β1 , ..., βk . We call this a linear regression model

 The response variable Y is asusmed to be random (but we do get to observe its value)
 The regression parameters β0 , β1 , ..., βk are asusmed to be fixed and unknown
 The independent variables x1 , x2 , ..., xk are assumed to be fixed (not random)
 The error term ϵ is assumed to be random (and not observed)
Description: More precisely, we call a regression model a linear regression model if regression parameters enter the g
function in a linear fashion. For example, each of the model is a linear regression model:
Y = β0 + β1 x +ϵ
| {z }
g(x)
Y = β0 + β1 x + β2 x2 +ϵ
| {z }
g(x)
Y = β0 + β1 x1 + β2 x2 + β3 x1 x2 +ϵ
| {z }
g(x1 ,x2 )
Main point: The term ”linear” does not refer to the shape of the regression function g. It refers to how the regression
parameters β0 , β1 , ..., βk enter the g function

80) Simple linear regression

Terminology A simple linear regression model includes only one independent variable x and is of the form:

Y = g(x) + ϵ = β0 + β1 x + ϵ

The regression function

g(x) = β0 + β1 x

is a straight line with intercept β0 and slope β1 . If E(ϵ) = 0, then

E(Y ) = E(β0 + β1 x + ϵ)
E(Y ) = β0 + β1 x + E(ϵ)
E(Y ) = β0 + β1 x+E(ϵ)
Therefore, we have these interpretations for the regression parameters β0 and β1
 β0 quantifies the mean of Y when x = 0 (y-intercept)

Page 134 of 160

 STATISTICS OVERVIEW

 β1 quantifies the change in E(Y ) brought about by a one-unit change in x

Example: Sewage Data
As part of a waste removal project, a new compression machine for processing sewage sludge is being studied. In
particular, engineers are interested in the following variables
Y = moisture control of compressed pellets (measured as a percent)
x = machine filtration rate (kg-DS/m/hr)
Engineers collect n = 20 observations of (x, Y ), the data are given below

Obs x Y Obs x Y
1 125.3 77.9 11 159.5 79.9
2 98.2 76.8 12 145.8 79.0
3 201.4 81.5 13 75.1 76.7
4 147.3 79.8 14 151.4 78.2
5 145.9 78.2 15 144.2 79.5
6 124.7 78.3 16 125.0 78.1
7 112.2 77.5 17 198.8 81.5
8 120.2 77.0 18 132.5 77.0
9 161.2 80.1 19 159.6 79.0
10 178.9 80.2 20 110.7 78.6

Figure 10..1: Scatterplot of pellet moisture Y as a function of machine filtration rate x

Scatterplot is the most common graphical display for bivariate data. From the plot, we see that
 the variable Y and x are positively related, that is, an increase in x tends to be associated with an increase in Y
 the variable Y and x are linearly related, althought there is a large amount of variable that is unexplained
This is an example where a simple linear regression model may be adequate
1 > f i l t r a t i o n . r a t e <− c ( 1 2 5 . 3 , 9 8 . 2 , 2 0 1 . 4 , 1 4 7 . 3 , 1 4 5 . 9 , 1 2 4 . 7 , 1 1 2 . 2 , 1 2 0 . 2 , 1 6 1 . 2 , 1 7 8 . 9 , 1 5 9 . 5 ,
145.8 , 75.1 , 151.4 , 144.2 , 125.0 , 198.8 , 132.5 , 159.6 , 110.7)
2 > m o i s t u r e <− c ( 7 7 . 9 , 7 6 . 8 , 8 1 . 5 , 7 9 . 8 , 7 8 . 2 , 7 8 . 3 , 7 7 . 5 , 7 7 . 0 , 8 0 . 1 , 8 0 . 2 , 7 9 . 9 , 7 9 . 0 , 7 6 . 7 , 7 8 . 2 ,
79.5 , 78.1 , 81.5 , 77.0 , 79.0 , 78.6)
3 > p l o t ( f i l t r a t i o n . r a t e , m o i s t u r e , x l a b = ” F i l t r a t i o n r a t e ( kg −DS /m/ hr ) ” , y l a b = ” M o i s t u r e (
P e r c e n t a g e ) ” , pch = 1 9 , c o l = ” b l a c k ” )

Page 135 of 160

 STATISTICS OVERVIEW

80).1 Least squares estimation

Terminology: When we say, ”fit a regression model”, we mean that we would like to estimate, the regression parameters
in the model with the observed data. Suppose that we collect (xi , Yi ), i = 1, 2, ..., n and postulate the simple linear
regression model
Yi = β0 + β1 xi + ϵi
for each i = 1, 2, ..., n.
Least squares: A widely-accepted method of estimating the model parameters β0 and β1 is at least squares. The model
of least squares says to choose the values of β0 and β1 that minimizes

n
X
Q(β0 , β1 ) = [Yi − (β0 + β1 x1 )]2 (10..3)
i=1

Denote the least squares estimators by β

c0 and β
c1 , respectively, that is, the values of β0 and β1 that minimize Q(β0 , β1 ).
A two-variable minimization argument can be used to find closed-form expressions for β c0 and β
c1 . Taking partial deriva-
tives of Q(β0 , β1 ), we obtain

n
∂Q(β0 , β1 ) X set
= −2 (Yi − β0 − β1 xi ) = 0
∂β0 i=1
n
(10..4)
∂Q(β0 , β1 ) X set
= −2 (Yi − β0 − β1 xi )xi = 0
∂β1 i=1

Solving for β0 and β1 gives the least squares estimators

c0 = Ȳ − β
β c1 x̄
Pn
(x − x̄)(Yi − Ȳ ) SSxy (10..5)
β
c1 = Pn i
i=1
2
=
i=1 (xi − x̄) SSxx

where
n
X n
X
SSxy = (xi − x̄)(Yi − Ȳ ) and SSxx = (xi − x̄)2 (10..6)
i=1 i=1

1 > f i l t r a t i o n . r a t e <− c ( 1 2 5 . 3 , 9 8 . 2 , 2 0 1 . 4 , 1 4 7 . 3 , 1 4 5 . 9 , 1 2 4 . 7 , 1 1 2 . 2 , 1 2 0 . 2 , 1 6 1 . 2 , 1 7 8 . 9 , 1 5 9 . 5 ,

Page 136 of 160

 STATISTICS OVERVIEW

145.8 , 75.1 , 151.4 , 144.2 , 125.0 , 198.8 , 132.5 , 159.6 , 110.7)

2 > m o i s t u r e <− c ( 7 7 . 9 , 7 6 . 8 , 8 1 . 5 , 7 9 . 8 , 7 8 . 2 , 7 8 . 3 , 7 7 . 5 , 7 7 . 0 , 8 0 . 1 , 8 0 . 2 , 7 9 . 9 , 7 9 . 0 , 7 6 . 7 , 7 8 . 2 ,
79.5 , 78.1 , 81.5 , 77.0 , 79.0 , 78.6)
3 > model <− lm ( m o i s t u r e ˜ f i l t r a t i o n . r a t e )
4 > model
5

6 Call :
7 lm ( f o r m u l a = m o i s t u r e ˜ f i l t r a t i o n . r a t e )
8

9 Coefficients :
10 ( Intercept ) f i l t r a t i o n . rate
11 72.95855 0.04103
12

13 > p l o t ( f i l t r a t i o n . r a t e , m o i s t u r e , x l a b = ” F i l t r a t i o n r a t e ( kg −DS /m/ hr ) ” , y l a b = ” M o i s t u r e (

P e r c e n t a g e ) ” , pch = 1 9 , c o l = ” b l a c k ” )
14 > a b l i n e ( model , c o l = ” b l a c k ” , lwd = 2 )

From the output, we see the least squares estimates (to 3 dp) for the sewage data are
β
c0 = 72.959
β
c1 = 0.041
Therefore, the equation of the least squares line that relates moisture percentage Y to the filtration rate x is
Ŷ = 72.959 + 0.041x
or, in other words,
Moisture
\ = 72.959 + 0.041 × Filtration rate
The least squares line is also known as the prediction equation. This is because we can predict the value of Y (moisture)
for any value of x (filtration rate). For example, when the filtration rate is x = 150kg-DS/m/hr, we would predict the
moisture percentage to be
Ŷ (150) = 72.959 + 0.041(150) ≈ 79.109

80).2 Model assumptions and properties of least squares estimators

We wish to investigate the properties of β
c0 and β
c1 as estimators of the true regression parameters β0 and β1 in the
simple linear regression model
Yi = β0 + β1 xi + ϵi
for i = 1, 2, ..., n. To do this, we need assumptions on the error terms ϵi . Specifically, we will assume throughout that
 E(ϵi ) = 0 for i = 1, 2, ..., n
 (ϵi ) = σ 2 for i = 1, 2, ..., n, that is, the variance is constant
 the random variables ϵi are independent
 the random variables ϵi are normally distributed
Under these assumptions
Yi ∼ N (β0 + β1 xi , σ 2 )
Fact 1: The least squares estimators β
c0 and β
c1 are unbiased estimators of β0 and β1 , respectively, that is

E(β
c0 ) = β0
(10..7)
E(β
c1 ) = β1

Fact 2: The least squares estimators β

c0 and β
c1 have the following sampling distributions:

c0 ∼ N (β0 , c00 σ 2 )
β
(10..8)
c1 ∼ N (β1 , c11 σ 2 )
β

Page 137 of 160

STATISTICS OVERVIEW

where
1 (x̄)2 1
c00 = + and c11 = (10..9)
n SSxx SSxx
Knowing these sampling distributions is critical if we want to write confidence intervals and perform hypothesis tests
for β0 and β1

80).3 Estimating the error variance

In the simple linear regression model
Yi = β0 + β1 xi + ϵi
where ϵi ∼ N (0, σ ), we estimate σ , the error variance
2 2

Terminology: In the simple linear regression mode, define the ithe fitted value by

Ybi = β
c0 + β
c1 xi (10..10)

where β
c0 and β
c1 are the least squares estimators. Each observation has its own fitted value. Geometrically, an observa-
tion’s fitted value is the (perpendicular) projection of its Y value, upward or downward, onto the least squares line
Terminology: We define the ithe residual by
ei = Yi − Ybi (10..11)

Each observation has its own residual. Geometrically, an observation’s residual is the vertical distance (i.e., length) be-
tween Y value and its fitted value
 If an observation’s Y value is above the least squares regression line, its residual is positive
 If an observation’s Y value is below the least square regression line, its residual is negative
 In the simple linear regression model (provided that the model includes an intercept term β0 ), we have the following
algebraic result
n
X n
X
ei = (Yi − Ybi ) = 0 (10..12)
i=1 i=1

that is, the sum of residual (from a least square fit) is equal to zero
Terminology: We define the residual sum of squares
n
X n
X
SSres ≡ e2i = (Yi − Ybi )2 (10..13)
i=1 i=1

Fact 3: In the simple linear regression mode

Pn
SSres − Ybi )2
i=1 (Yi
M Sres = = (10..14)
n−2 n−2

is an unbiased estimator of σ 2 , that is, E(M Sres ) = σ 2 . The quantity

r
p SSres
σ̂ = M Sres = (10..15)
n−2

estimates σ and is called the residual standard error.

Example: Sewage Data (cnt)

Page 138 of 160

 STATISTICS OVERVIEW

Obs x Y Ŷ = β
c0 + βc1 x e = Y − Ŷ Obs x Y Ŷ = β
c0 + βc1 x e = Y − Ŷ
1 125.3 77.9 78.100 −0.200 11 159.5 79.9 79.503 0.397
2 98.2 76.8 76.988 0.188 12 145.8 79.0 78.941 0.059
3 201.4 81.5 81.223 0.277 13 75.1 76.7 76.040 0.660
4 147.3 79.8 79.003 0.797 14 151.4 78.2 79.171 0.971
5 145.9 78.2 78.945 0.745 15 144.2 79.5 78.876 0.624
6 124.7 78.3 78.075 0.225 16 125.0 78.1 78.088 0.012
7 112.2 77.5 77.563 0.062 17 198.8 81.5 81.116 0.384
8 120.2 77.0 77.891 0.891 18 132.5 77.0 78.396 1.396
9 161.2 80.1 79.573 0.527 19 159.6 79.0 79.508 0.508
10 178.9 80.2 80.299 0.099 20 110.7 78.6 77.501 1.099

1 > f i l t r a t i o n . r a t e <− c ( 1 2 5 . 3 , 9 8 . 2 , 2 0 1 . 4 , 1 4 7 . 3 , 1 4 5 . 9 , 1 2 4 . 7 , 1 1 2 . 2 , 1 2 0 . 2 , 1 6 1 . 2 , 1 7 8 . 9 , 1 5 9 . 5 ,
145.8 , 75.1 , 151.4 , 144.2 , 125.0 , 198.8 , 132.5 , 159.6 , 110.7)
2 > m o i s t u r e <− c ( 7 7 . 9 , 7 6 . 8 , 8 1 . 5 , 7 9 . 8 , 7 8 . 2 , 7 8 . 3 , 7 7 . 5 , 7 7 . 0 , 8 0 . 1 , 8 0 . 2 , 7 9 . 9 , 7 9 . 0 , 7 6 . 7 , 7 8 . 2 ,
79.5 , 78.1 , 81.5 , 77.0 , 79.0 , 78.6)
3 > sewage . d a t a <− d a t a . f r a m e ( f i l t r a t i o n . r a t e , m o i s t u r e )
4 > model <− lm ( m o i s t u r e ˜ f i l t r a t i o n . r a t e , d a t a = sewage . d a t a )
5 > summary ( model )
6

7 Call :
8 lm ( f o r m u l a = m o i s t u r e ˜ f i l t r a t i o n . r a t e , d a t a = sewage . d a t a )
9

10 Residuals :
11 Min 1Q Median 3Q
12 −1.39552 −0.27694 0.03548 0.42913
13 Max
14 1.09901
15

16 Coefficients :
17 Estimate Std . Error
18 ( Intercept ) 72.958547 0.697528
19 f i l t r a t i o n . rate 0.041034 0.004837
20 t v a l u e Pr ( >| t | )
21 ( Intercept ) 1 0 4 . 5 9 6 < 2 e −16 ∗ ∗ ∗
22 f i l t r a t i o n . rate 8 . 4 8 4 1 . 0 5 e −07 ∗ ∗ ∗
23 −−−
24 S i g n i f . codes :
25 0 ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05 .
26 0.1 1
27

28 R e s i d u a l s t a n d a r d e r r o r : 0 . 6 6 5 3 on 18 d e g r e e s o f f r e e d o m
29 M u l t i p l e R− s q u a r e d : 0 . 7 9 9 9 , A d j u s t e d R− s q u a r e d : 0.7888
30 F − s t a t i s t i c : 7 1 . 9 7 on 1 and 18 DF , p− v a l u e : 1 . 0 5 2 e −07
31

32 > sewage . f i t <− round ( p r e d i c t ( model ) , d i g i t s = 3 )

33 > sewage . r e s i d u a l s <− round ( r e s i d u a l s ( model ) , d i g i t s = 3 )
34 > sewage . d a t a $ sewage . f i t <− sewage . f i t
35 > sewage . d a t a $ sewage . r e s i d u a l s <− sewage . r e s i d u a l s
36

37 > MS r e s = sum ( sewage . r e s i d u a l s ˆ 2 ) / 18

38 > MS r e s
39 [1] 0.44275
40

41 > r e s . s t d . e r r o r = s q r t ( MS r e s )
42 > res . std . error

Page 139 of 160

 STATISTICS OVERVIEW

43 [1] 0.6653946

80).4 Inference for β0 and β1

In the simple linear regression model
Yi = β0 + β1 xi + ϵi
the regression parameters β0 and β1 are unknown. It is therefore of interest to construct confidence intervals and
perform hypothesis tests for these parameters
 In practice, inference for the slop parameter β1 is of primarily interest because of its connection to the independent
variable x in the model
 Inference for β0 is less meaningful, unless one is explicitly interested in the mean of Y when x = 0

Confidence Interval (CI) of β1

Under our model assumptions, the following sampling distribution arises:

c1 − β1
β
t= r ∼ t(n − 2) (10..16)
M Sres
SSxx

A 100(1 − α)% confidence interval for β1

r r
M Sres c M Sres
c1 − tn−2,α/2
(β , β1 + tn−2,α/2 ) (10..17)
SSxx SSxx

The form of the interval:

point estimate ± quantile × standard error}
| {z } | {z } |√ {z
β tn−2,α/2 M Sres /SSxx
1
c
 The value tn−2,α/2 is the upper α/2 from the t(n − 2) distribution
Interpretation: We are 100(1 − α)% confident that the population regression slope β1 is in this interval

Note

When interpreting the interval, of particular interest to us is the value β1 = 0

 If β1 = 0 is in the confidence interval, this suggests that Y and x are not linearly related
 If β1 = 0 is not in the confidence interval, this suggests that Y and x are linearly related

Hypothesis Testing for β1

Null hypothesis:
H0 = β 1 = △ 0
Alternative hypothesis:
Ha : β1 < △0
Ha : β1 > △0
Ha : β1 ̸= △0
where △0 is a fixed value (often, △0 = 0)
Confidence interval approach

Page 140 of 160

 STATISTICS OVERVIEW

Hypothesis type Alternative hypothesis Reject criterion

r
M Sres
Left-tail hypothesis Ha : β1 < △0 △0 > β
c1 + tn−2,α
r SSxx
M Sres
Right-tail hypothesis Ha : β1 > △0 △0 < β1 − tn−2,α
c
SSxx
r
M Sres
Two-tail hypothesis Ha : β1 ̸= △0 △0 ∈
/ (β1 ± tn−2,α/2
c )
SSxx
The test statistics (Critical value approach)

β
c1
t0 = (10..18)
M Sres
SSxx

Hypothesis type Alternative hypothesis Reject criterion

Left-tail hypothesis Ha : β1 < △0 t0 < −tn−2,α
Right-tail hypothesis Ha : β1 > △0 t0 > tn−2,α
Two-tail hypothesis Ha : β1 ̸= △0 t0 > tn−2,α/2 or t0 < −tn−2,α/2

P -value approach
β
c1
t0 = (10..19)
M Sres
SSxx

Hypothesis type Alternative hypothesis Reject criterion

Left-tail hypothesis Ha : β1 < △0 p-value < α
Right-tail hypothesis Ha : β1 > △0 p-value < α
Two-tail hypothesis Ha : β1 ̸= △0 p-value < α

Example: Sewage Data (cnt)

Obs x Y Ŷ = β
c0 + βc1 x e = Y − Ŷ Obs x Y Ŷ = β
c0 + βc1 x e = Y − Ŷ
1 125.3 77.9 78.100 −0.200 11 159.5 79.9 79.503 0.397
2 98.2 76.8 76.988 0.188 12 145.8 79.0 78.941 0.059
3 201.4 81.5 81.223 0.277 13 75.1 76.7 76.040 0.660
4 147.3 79.8 79.003 0.797 14 151.4 78.2 79.171 0.971
5 145.9 78.2 78.945 0.745 15 144.2 79.5 78.876 0.624
6 124.7 78.3 78.075 0.225 16 125.0 78.1 78.088 0.012
7 112.2 77.5 77.563 0.062 17 198.8 81.5 81.116 0.384
8 120.2 77.0 77.891 0.891 18 132.5 77.0 78.396 1.396
9 161.2 80.1 79.573 0.527 19 159.6 79.0 79.508 0.508
10 178.9 80.2 80.299 0.099 20 110.7 78.6 77.501 1.099

Null hypothesis:
β1 = 0 where △0 = 0
Alternative hypothesis:
β1 ̸= 0 where △0 = 0
1 > summary ( lm ( m o i s t u r e ˜ f i l t r a t i o n . r a t e ) )
2

3 Call :
4 lm ( f o r m u l a = m o i s t u r e ˜ f i l t r a t i o n . r a t e )
5

6 Residuals :
7 Min 1Q Median 3Q

Page 141 of 160

 STATISTICS OVERVIEW

8 −1.39552 −0.27694 0.03548 0.42913

9 Max
10 1.09901
11

12 Coefficients :
13 Estimate Std . Error
14 ( Intercept ) 72.958547 0.697528
15 f i l t r a t i o n . rate 0.041034 0.004837
16 t v a l u e Pr ( >| t | )
17 ( Intercept ) 1 0 4 . 5 9 6 < 2 e −16 ∗ ∗ ∗
18 f i l t r a t i o n . rate 8 . 4 8 4 1 . 0 5 e −07 ∗ ∗ ∗
19 −−−
20 S i g n i f . codes :
21 0 ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05 .
22 0.1 1
23

24 R e s i d u a l s t a n d a r d e r r o r : 0 . 6 6 5 3 on 18 d e g r e e s o f f r e e d o m
25 M u l t i p l e R− s q u a r e d : 0 . 7 9 9 9 , A d j u s t e d R− s q u a r e d : 0.7888
26 F − s t a t i s t i c : 7 1 . 9 7 on 1 and 18 DF , p− v a l u e : 1 . 0 5 2 e −07

Solution
Null hypothesis:
H0 : β1 = 0 where △0 = 0
Alternative hypothesis:
Ha : β1 ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
The Estimate output gives β
c1 = 0.041034
The [Link] outcome gives se(b β
c1 ) = 0.004837
95% CI = β1 ± t18,0.025 se(
c b β1 ) = 0.041034 ± 2.101(0.004837) = (0.0309, 0.0512)
c
/ (0.0309, 0.0512) ⇒ reject H0
0∈
Sol 2: Critical value approach
The t value gives t values: t0 = 8.484
tn−2,α/2 = t18,0.025 = 2.101
⇒ t0 > t18,0.025 ⇒ reject H0
Sol 3: Critical value approach
The Pr(> |t|) gives p-value = 0.000000105
0.000000105 < 0.05 ⇒ p-value < α ⇒ reject H0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the population
regression slope β1 is different from 0

80).5 Confidence and Prediction intervals for a give x = x0

Consider the simple linear regression model
Yi = β0 + β1 xi + ϵi
where ϵi N (0, σ ). We are often interested in using the fitted model to learn about the response variable Y at a certain
2

setting for the independent variable x = x0 . For example, in our sewage example, we might be interested in the moisture
percentage Y when the filtration rate is x = 150kg-DS/m/hr. Two potential goal arise:
 We might be interested in estimating the mean response of Y when x = x0 . This mean response is denoted by E(Y |x0 ).
This value is the mean of the following probability distribution

Y (x0 ) ∼ N (β0 + β1 x0 , σ 2 ) (10..20)

Page 142 of 160

STATISTICS OVERVIEW

 We might be interested in predicting a new response Y when x = x0 . This predicted response is denoted by Y ∗ (x0 ).
This value is a new outcome frome
Y (x0 ) ∼ N (β0 + β1 x0 , σ 2 ) (10..21)

In the first problem, we are interested in estimating the mean of the response variable Y at a certain value of x, whereas
in the second problem, we are interested in predicting the value of a new random variable Y at a certain value of x.
Conceptually, the second problem is far more difficult than the first
 100(1 − α)% interval for the mean E(Y |x0 ) is called a confidence interval (since it is for the mean response)
 100(1 − α)% interval for the new value Y ∗ (x0 ) is called a prediction interval (since it is for a new random variable)
Point estimator/Predictor: To construct either interval, we start with the same quantity
Ŷ (x0 ) = β
c0 + β
c1 x0
where β
c0 and β
c1 are the least squares estimates from the fit of the model
 In the confidence interval for E(Y |x0 ), we call Ŷ (x0 ) a point estimator
 In the prediction interval for Y (x0 ), we call Ŷ (x0 ) a point predictor
Confidence interval: A 100(1 − α)% confidence interval for the mean E(Y |x0 ) is given by
s s
1 (x0 − x̄)2 1 (x0 − x̄)2
[Ŷ (x0 ) − tn−2,α/2 M Sres ( + ), Ŷ (x0 ) + tn−2,α/2 M Sres ( + )] (10..22)
n SSxx n SSxx

Prediction interval: A 100(1 − α)% prediction interval for the new response Y ∗ (x0 ) is given by
s s
1 (x0 − x̄)2 1 (x0 − x̄)2
[Ŷ (x0 ) − tn−2,α/2 M Sres (1 + + ), Ŷ (x0 ) + tn−2,α/2 M Sres (1 + + )] (10..23)
n SSxx n SSxx

Note

Comparison: The two intervals are identical except for the extra ”1” in the standard error part of the pre-
diction interval. This extra ”1” arises from the additional uncertaint with predicting a new response from the
N (β0 + β1 x0 , σ 2 ) distribution. Therefore, a 100(1 − α)% prediction interval for Y ∗ (x0 ) will be wider than the
corresponding 100(1 − α)% confidence interval for E(Y |x0 )
Interval length: The length of both intervals clearly depends on the value of x0 . In fact, the standard error of
Ŷ (x0 ) will be smallest when x0 = x̄ and will get larger the farther x0 is from x̄ in either direction. This implies that
the precision with which we estimate E(Y |x0 ) or predict Y ∗ (x0 ) decrease the farther we get away from x̄. This
make intuitive sense, namely, we would expect to have the most ”confidence” in our fitted modelnear the ”center”
of the observed data

It is sometimes desired to estimate E(Y |x0 ) or predict Y ∗ (x0 ) based on the fit of the model for values of x0 outside the
range of x values used in the experiment/study. This is called extrapolation and can be very dangerous. In order for our
inferences to be valid, we must believe that the straight line relationship holds for x values outside the range where we
have observed data. In some situations, this may be reasonable. In others, we may have no theoretical basis for making
such a claim without to support it.
Example: Sewage Data (cnt)
Suppose that we are interested in estimating E(Y |x0 ) and predicting a new Y ∗ (x0 ) when the filtration rate is x0 =
150kg-DS/m/hr
 E(Y |x0 ) denotes the mean moisture percentage for compressed pellets when the machine filtration rate is x0 = 150kg-
DS/m/hr. In other words, it we were to repeat the experiment over and over again, each time using a filtration rate of
x0 = 150kg-DS/m/hr, then E(Y |x0 ) denotes the mean value of Y (moisture percentage) that would be observed
 Y ∗ (x0 ) denotes a possible value of Y for a single run of the machine when the filtration rate is set at x0 = 150kg-
DS/m/hr

Page 143 of 160

 STATISTICS OVERVIEW

1 > f i l t r a t i o n . r a t e <− c ( 1 2 5 . 3 , 9 8 . 2 , 2 0 1 . 4 , 1 4 7 . 3 , 1 4 5 . 9 , 1 2 4 . 7 , 1 1 2 . 2 , 1 2 0 . 2 , 1 6 1 . 2 , 1 7 8 . 9 , 1 5 9 . 5 ,
145.8 , 75.1 , 151.4 , 144.2 , 125.0 , 198.8 , 132.5 , 159.6 , 110.7)
2 > m o i s t u r e <− c ( 7 7 . 9 , 7 6 . 8 , 8 1 . 5 , 7 9 . 8 , 7 8 . 2 , 7 8 . 3 , 7 7 . 5 , 7 7 . 0 , 8 0 . 1 , 8 0 . 2 , 7 9 . 9 , 7 9 . 0 , 7 6 . 7 , 7 8 . 2 ,
79.5 , 78.1 , 81.5 , 77.0 , 79.0 , 78.6)
3 > model <− lm ( m o i s t u r e ˜ f i l t r a t i o n . r a t e )
4 > p r e d i c t ( model , d a t a . f r a m e ( f i l t r a t i o n . r a t e = 1 5 0 ) , l e v e l = 0 . 9 5 , i n t e r v a l = ” c o n f i d e n c e ” )
5 fit lwr upr
6 1 79.11361 78.78765 79.43958
7 > p r e d i c t ( model , d a t a . f r a m e ( f i l t r a t i o n . r a t e = 1 5 0 ) , l e v e l = 0 . 9 5 , i n t e r v a l = ” p r e d i c t i o n ” )
8 fit lwr upr
9 1 79.11361 77.6783 80.54893

The point estimate (point prediction) is easily calculated: Ŷ (x0 = 150) = 72.959 + 0.041(150) ≈ 79.11361
A 95% confidence interval for E(Y |x0 ) is (78.79, 79.44). When the filtration rate is x0 = 150kg-DS/m/hr, we are 95%
confident that the mean moisture percentage is between 78.79 and 79.44 percent
A 95% prediction interval for Y ∗ (x0 = 150) is (77.68, 80.55). When the filtration rate is x0 = 150kg-DS/m/hr, we are
95% confident that the moisture percentage for a single run of the experiment will be between 77.68 and 80.55 percent

81) Multiple linear regression

81).1 Introduction
We have already considered the simple linear regression model
Yi = β0 + β1 xi + ϵi
for i = 1, 2, ..., n where ϵi ∼ N (0, σ ). We now extend this basic model to include multiple independent variables
2

x1 , x2 , ..., xk . This is much more reaslistic because, in practice, often Y depends on many different factors (i.e., not just
one). Specifically, we consider models of the form
Yi = β0 + β1 xi1 + β2 xi2 + ... + βk xik + ϵi
for i = 1, 2, ..., n. We call this a multiple linear regression model
 There are now p = k + 1 regression parameters β0 , β1 , ..., βk . These are unknown and are to be estimated with the

Page 144 of 160

STATISTICS OVERVIEW

observed data
 Schematically, we can envision the observed data as follows:  Each of the n individuals contribute a response Y and

Individual Y x1 x2 ··· xk
1 Y1 x11 x12 ··· x1k
2 Y2 x21 x22 ··· x2k
.. .. .. .. .. ..
. . . . . .
n Yn xn1 xn2 ··· xnk

a value of each of the independent variables x1 , x2 , ..., xk

 We continue to assume that ϵi ∼ N (0, σ 2 )
 We also assume that the independent variables x1 , x2 , ..., xk are fixed and measured without error
To fit the multiple linear regression model
Yi = β0 + β1 xi1 + β2 xi2 + ... + βk xik + ϵi
We again use the method of least squares. Simple computing formulae for the least squares estimators are no longer
available (as they were in simple linear regression). This is hardly a big deal because we will use computing to automate
all analyses. For instruction purposes, it is advantageous to express multiple linear regression models in terms of ma-
trices and vectors. This streamlines notation and makes the presentation easier

81).2 Least squares estimator

The notation of least squares is the same as it was in the simple linear regression model. To fit a multiple linear regression
model, we want to find the value of β0 , β1 , ..., βk that minimize
Xn
Q(β0 , β1 , ..., βk ) = [Yi − (β0 + β1 xi1 + β2 xi2 + ... + βk xik )2 ]
i=1
This could be done by solving a system of linear equations,
∂Q(β0 , β1 , ..., βk )
= 0 for j = 0, ..., k
∂βj
Example: Cheese data
The taste of matured cheese is related to the concentration of several chemicals in the final product. In a study from
the LaTrobe Valley of Victoria, Australia, samples of cheddar cheese were analyzed for their chemical composition and
were subjected to taste tests. For each specimen, the taste Y was obtained by combining the scores from several tasters.
Data were collected on the following variables:
Y = taste score (TASTE)
x1 = concentration of acetic acid (ACETIC)
x2 = concentration of hydrogen sulfide (H2S)
x3 = concentration of lactic acid (LACTIC)
Variables ACETIC and H2S were both measured on the log scale. The variable LACTIC has not been transformed.

Page 145 of 160

 STATISTICS OVERVIEW

Specimen TASTE ACETIC H2S LACTIC Specimen TASTE ACETIC H2S LACTIC
1 12.3 4.543 3.135 0.86 16 40.9 6.365 9.588 1.74
2 20.9 5.159 5.043 1.53 17 15.9 4.787 3.912 1.16
3 39.0 5.366 5.438 1.57 18 6.4 5.412 4.700 1.49
4 47.9 5.759 7.496 1.81 19 18.0 5.247 6.174 1.63
5 5.6 4.663 3.807 0.99 20 38.9 5.438 9.064 1.99
6 25.9 5.697 7.601 1.09 21 14.0 4.564 4.949 1.15
7 37.3 5.892 8.726 1.29 22 15.2 5.298 5.220 1.33
8 21.9 6.078 7.966 1.78 23 32.0 5.455 9.242 1.44
9 18.1 4.898 3.850 1.29 24 56.7 5.855 10.20 2.01
10 21.0 5.242 4.174 1.58 25 16.8 5.366 3.664 1.31
11 34.9 5.740 6.142 1.68 26 11.6 6.043 3.219 1.46
12 57.2 6.446 7.908 1.90 27 26.5 6.458 6.962 1.72
13 0.7 4.477 2.996 1.06 28 0.7 5.328 3.912 1.25
14 25.9 5.236 4.942 1.30 29 13.4 5.802 6.685 1.08
15 54.9 6.151 6.752 1.52 30 5.5 6.176 4.787 1.25

MODEL: Researchers postulate that each of the three chemical composition variables x1 , x2 and x3 is important in
describing the taste and consider the multiple linear regression model
Yi = β0 + β1 xi1 + β2 xi2 + β3 xi3 + ϵi
for i = 1, 2, ..., 30
1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)
3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > summary ( model )
7

8 Call :
9 lm ( f o r m u l a = t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
10

11 Residuals :
12 Min 1Q Median 3Q Max
13 −17.390 −6.612 −1.009 4.908 25.449
14

15 Coefficients :
16 E s t i m a t e S t d . E r r o r t v a l u e Pr ( >| t | )
17 ( Intercept ) −28.877 19.735 −1.463 0.15540
18 acetic 0.328 4.460 0.074 0.94193
19 h2s 3.912 1.248 3.133 0.00425 ∗ ∗
20 lactic 19.670 8.629 2.279 0.03109 ∗
21 −−−
22 S i g n i f . codes :
23 0 ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05 . 0.1 1
24

25 R e s i d u a l s t a n d a r d e r r o r : 1 0 . 1 3 on 26 d e g r e e s o f f r e e d o m
26 M u l t i p l e R− s q u a r e d : 0 . 6 5 1 8 , A d j u s t e d R− s q u a r e d : 0.6116
27 F − s t a t i s t i c : 1 6 . 2 2 on 3 and 26 DF , p− v a l u e : 3 . 8 1 e −06

Page 146 of 160

 STATISTICS OVERVIEW

The Estimate output gives the values of the 

least squares
 estimates 
β0
c −28.877
 c   
 β1   0.328 
 = 
 β
 c2   3.912
  

β
c3 19.670
Therefore the fitted least squares regression model is
Ŷ = −28.877 + 0.328x1 + 3.912x2 + 19.670x3
or, in other words,
TATSE
\ = −28.877 + 0.328 ACETIC + 3.912 H2S + 19.670 LACTIC

81).3 Estimating the error variance

Terminology: Define the residual sum of squares by

n
X n
X
SSres = (Yi − Ybi )2 = e2i (10..24)
i=1 i=1

In the multiple linear regression model

Yi = β0 + β1 xi1 + β2 xi2 + ... + βk xik + ϵi
where ϵi ∼ N (0, σ ) 2
SSres
M Sres = (10..25)
n−p
is an unbiased estimator of σ 2 , that is
E(M Sres ) = σ 2 (10..26)

The quantity s
p SSres
σ̂ = M Sres = (10..27)
n−p

1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)
3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > summary ( model )
7

8 Call :
9 lm ( f o r m u l a = t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
10

11 Residuals :
12 Min 1Q Median 3Q Max
13 −17.390 −6.612 −1.009 4.908 25.449
14

15 Coefficients :
16 E s t i m a t e S t d . E r r o r t v a l u e Pr ( >| t | )
17 ( Intercept ) −28.877 19.735 −1.463 0.15540
18 acetic 0.328 4.460 0.074 0.94193
19 h2s 3.912 1.248 3.133 0.00425 ∗ ∗
20 lactic 19.670 8.629 2.279 0.03109 ∗

Page 147 of 160

 STATISTICS OVERVIEW

21 −−−
22 S i g n i f . codes :
23 0 ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05 . 0.1 1
24

25 R e s i d u a l s t a n d a r d e r r o r : 1 0 . 1 3 on 26 d e g r e e s o f f r e e d o m
26 M u l t i p l e R− s q u a r e d : 0 . 6 5 1 8 , A d j u s t e d R− s q u a r e d : 0.6116
27 F − s t a t i s t i c : 1 6 . 2 2 on 3 and 26 DF , p− v a l u e : 3 . 8 1 e −06

From the Residual standard error output, σ = 10.13 ⇒ M Sres = σ 2 = (10.13)2 = 102.617

81).4 Analysis of variance for linear regresison

Identity: Algebraically, it can be shown that
n
X n
X n
X
(Yi − Ȳ )2 = (Ybi − Ȳ )2 + (Yi − Ybi )2 (10..28)
i=1 i=1 i=1
| {z } | {z } | {z }
SStotal SSreg SSres

 SStotal is the total sum of squares. SStotal is the numerator of the sample variance of Y1 , Y2 , ..., Yn . It measures the
total variation in the response data
 SSreg is the regression sum of squares. SSreg measures the variation in the response data explained by the linear
regression model.
 SSres is the residual sum of squares. SSres measures the variation in the response data not explained by the linear
regression model.
ANOVA Table: We can combine all of this information to produce an analysis of variance (ANOVA) table. Such tables
are standard in regression analysis
The degrees of freedom (df) all down:
 SStotal can be viewed as a statistic that has “lost” a degree of freedom for having to estimate the overall mean of Y
with the sample mean Ȳ . Recall that n − 1 is our divisor in the sample variances S 2
 There are k degrees of freedom associated with SSreg because there are k independent variables
 The degrees of freedom for SSres can be thought of as the divisor needed to create an unbiased estimator of σ 2 . Recall
that
SSres SSres
M Sres = = (10..29)
n−p n−k−1
is an unbiased estimator of σ 2
The sum of squares (SS) also add down. This follows from the algebraic identity noted earlier
Mean squares (MS) are the sum of squares divided by the degrees of freedom
The F statistics is formed by taking the ratio of M Sreg and M Sres

Source df (ν) SS MS F
SSreg M Sreg
Regression k SSreg M Sreg = F =
k M Sres
SSres SSres
Residual n−p SSres M Sres = =
n−p n−k−1
Total n−1 SStotal

Table 10..1: Analysis of variance table for linear regression

Page 148 of 160

 STATISTICS OVERVIEW

Coefficient of determination

Since
SStotal = SSreg + SSres (10..30)

The proportion of the total variation in the data explained by the linear regression model is called the coefficient of
determination.
SSreg SSres
R2 = =1− (10..31)
SStotal SStotal
The larger the R2 , the better the regression model explains the variability in the data

0 ≤ R2 ≤ 1 (10..32)

It is critical to understand what R2 does and does not measure. Its value is computed under the assumption that the
multiple linear regression model is correct and assesses how much of the variation in the data may be attributed to that
relationship rather than to inherent variation.
 If R2 is small, it may be that there is a lot of random inherent variation in the data, so that, although the multiple linear
regression model is reasonable, it can explain only so much of the observed overall variation
 Alternatively, R2 may be close to 1; e.g., in a simple linear regression model fit, but this may not be the best model.
In fact, R2 could be very “high,” but ultimately not relevant because it assumes the simple linear regression model is
correct. In reality, a better model may exist (e.g., a quadratic model, etc.).
 One draw back of R2 is that, it can never decrease when a new x is added. Thus, it can be difficult to judge whether
the increase is telling us anything useful about the new x. Instead, many regression users prefer to use an adjusted R2
statistic:
SSres /(n − p)
2
Radjusted =1− (10..33)
SStotal /(n − 1)

F statistics

The F statistic in the ANOVA table is used to test

Null hypothesis:
H0 : β1 = β2 = ... = βk = 0
Alternative hypothesis:
Ha : at least one of the βj is nonzero
In other words, F tests whether or not at least one of the independent variables x1 , x2 , ..., xk is important in describing
the response Y . If H0 is rejected, we do not know which one or how many of the βj ’s are nonzero, only that at least
one is
Sampling distribution: When H0 : β1 = β2 = ... = βk is true

M Sreg
F = ∼ F (k, n − p) (10..34)
M Sres

Therefore, we can gauge the evidence against H0 by comparing F to this distribution. Values of F far out in the (right)
upper tail are evidence against H0 . R automatically produces the value of F and produces the corresponding p-value.
Recall the small p-values are evidence against H0 (the smaller p-value, the more evidence)
Example: Cheese data(cnt)
1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)

Page 149 of 160

 STATISTICS OVERVIEW

3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > summary ( model )
7

8 Call :
9 lm ( f o r m u l a = t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
10

11 Residuals :
12 Min 1Q Median 3Q Max
13 −17.390 −6.612 −1.009 4.908 25.449
14

15 Coefficients :
16 E s t i m a t e S t d . E r r o r t v a l u e Pr ( >| t | )
17 ( Intercept ) −28.877 19.735 −1.463 0.15540
18 acetic 0.328 4.460 0.074 0.94193
19 h2s 3.912 1.248 3.133 0.00425 ∗ ∗
20 lactic 19.670 8.629 2.279 0.03109 ∗
21 −−−
22 S i g n i f . codes :
23 0 ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05 . 0.1 1
24

25 R e s i d u a l s t a n d a r d e r r o r : 1 0 . 1 3 on 26 d e g r e e s o f f r e e d o m
26 M u l t i p l e R− s q u a r e d : 0 . 6 5 1 8 , A d j u s t e d R− s q u a r e d : 0.6116
27 F − s t a t i s t i c : 1 6 . 2 2 on 3 and 26 DF , p− v a l u e : 3 . 8 1 e −06

Solution

Null hypothesis:
H0 : β1 = β2 = β3 = 0
Alternative hypothesis:
Ha : at least one of the βj is nonzero
Based on the F − statistic (F = 16.22) and the corresponding p-value < 0.0001 (p − value = 0.00000381),
we have strong evidence to reject H0 .
Interpretation: We conclude that at least one of the independent variables (ACETIC, H2S, LACTIC) is important
in describing taste
The Multiple R − squared gives coefficient of determination R2 : R2 = 0.6518
⇒ about 65.2% of the variability in the taste data is explained by the linear regression model that includes
ACETIC, H2S and LACTIC. The remaining 34.8% of the variability in the taste data is explained by other sources

81).5 Inference for individual regression parameters

Consider multiple linear regression model
Yi = β0 + β1 xi1 + β2 xi2 + ... + βk xik + ϵi
for i = 1, 2, ..., n where ϵi ∼ N (0, σ ). Confidence intervals and hypothesis tests for βj can help us assess the impor-
2

tance of using the independent variable xj in a model with the other independent variables. That is, inference regarding
βj is always conditional on the other variables being included in the model

Page 150 of 160

STATISTICS OVERVIEW

Confidence interval

A 100(1 − α)% confidence interval for βj for j = 0, 1, 2, ..., k

[βbj − tn−p,α/2 se(

b β ck ), βbj + tn−p,α/2 se(
b β ck )] (10..35)

Hypothesis Testing

Null hypothesis:
H0 : βj = 0
Alternative hypothesis:
Ha : βj ̸= 0
can be performed by examining the p-value output
 If H0 : βj = 0 is not rejected, then xj is not important in describing Y in the presence of the other independent
variables
 If H0 : βj = 0 is rejected, this means that xj is important in describing Y even after including the effects of the other
independent variables
Hypothesis test for β1 : If our interest was to test, where j = 0, 1, ..., k

Null hypothesis:
H0 : βj = △0
Alternative hypotheis:
Ha : βj < △0
Ha : βj > △0
Ha : βj ̸= △0
where △0 is a fixed value (often △0 = 0)
Confidence interval approach
Hypothesis type Alternative hypothesis Rejection Criterion
Left-tail hypothesis Ha : βj < △0 △0 > βbj + tn−p,α se(
b βbj )
Right-tail hypothesis Ha : βj > △0 △0 > βbj − tn−p,α se(
b βbj )
Two-tail hypothesis Ha : βj ̸= △0 / [βbj ± tn−p,α/2 se(
△0 ∈ b βbj )]

The test statistics (Critical value approach)

βbj
t0 = (10..36)
se(
b βbj )

Hypothesis type Alternative hypothesis Rejection Criterion

Left-tail hypothesis Ha : βj < △0 t0 < −tn−p,α
Right-tail hypothesis Ha : βj > △0 t0 > tn−p,α
Two-tail hypothesis Ha : βj ̸= △0 t0 > tn−p,α/2 or t0 < −tn−p,α/2

P -value approach
βbj
t0 = (10..37)
se(
b βbj )

Hypothesis type Alternative hypothesis Rejection Criterion

Left-tail hypothesis Ha : βj < △0 p-value < α
Right-tail hypothesis Ha : βj > △0 p-value < α
Two-tail hypothesis Ha : βj ̸= △0 p-value < α

Page 151 of 160

 STATISTICS OVERVIEW

Example: Cheese data(cnt)

1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)
3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > summary ( model )
7

8 Call :
9 lm ( f o r m u l a = t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
10

11 Residuals :
12 Min 1Q Median 3Q Max
13 −17.390 −6.612 −1.009 4.908 25.449
14

15 Coefficients :
16 E s t i m a t e S t d . E r r o r t v a l u e Pr ( >| t | )
17 ( Intercept ) −28.877 19.735 −1.463 0.15540
18 acetic 0.328 4.460 0.074 0.94193
19 h2s 3.912 1.248 3.133 0.00425 ∗ ∗
20 lactic 19.670 8.629 2.279 0.03109 ∗
21 −−−
22 S i g n i f . codes :
23 0 ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05 . 0.1 1
24

25 R e s i d u a l s t a n d a r d e r r o r : 1 0 . 1 3 on 26 d e g r e e s o f f r e e d o m
26 M u l t i p l e R− s q u a r e d : 0 . 6 5 1 8 , A d j u s t e d R− s q u a r e d : 0.6116
27 F − s t a t i s t i c : 1 6 . 2 2 on 3 and 26 DF , p− v a l u e : 3 . 8 1 e −06

Solution

Null hypothesis:
H0 : βj = 0 where △0 = 0
Alternative hypothesis:
Ha : βj ̸= 0 where △0 = 0
Sol 1: Confidence interval approach
The Estimate output gives the values of the
 least squares
 estimates:
β
c0 −28.877
 c   
 β1   0.328 
 = 
 β
 c2   3.912 
  
β
c3 19.670
The [Link] output give standard error:   
se(
b β c0 ) 19.735
 se(
 c   
 b β1 )  =  4.460 
  
 se(
 b β c2 )   1.248 
 

se(
b βc3 ) 8.629
95% CI for regression parameters β0 , β1 , β2 and β3

Page 152 of 160

 STATISTICS OVERVIEW

c0 ± t26,0.025 se(
β b β c0 ) = −28.877 ± 2.056(19.735) = (−69.45, 11.70) ⇒ 0 ∈ (−69.45, 11.70) ⇒ not reject
H0 : β 0 = 0
c1 ± t26,0.025 se(
β b β c1 ) = 0.328 ± 2.056(4.460) = (−8.84, 9.50) ⇒ 0 ∈ (−8.84, 9.50) ⇒ not reject H0 : β1 = 0
c2 ± t26,0.025 se(
β b β / (1.35, 6.48) ⇒ reject H0 : β2 = 0
c2 ) = 3.912 ± 2.056(1.248) = (1.35, 6.48) ⇒ 0 ∈
c3 ± t26,0.025 se(
β b β / (1.93, 37.41 ⇒) reject H0 : β3 = 0
c3 ) = 19.670 ± 2.056(8.629) = (1.93, 37.41) ⇒ 0 ∈
Sol 2: Critical value approach
The t value gives the t statistics    
tβc0 −1.463
   
 tβc   0.074 
 1 = 
 t   3.133 
 βc2   
tβc3 2.279
t26,0.025 = 2.056
 tβc0 = −1.463 > −2.056 ⇒ not reject H0 : β0 = 0
 tβc1 = 0.074 < 2.056 ⇒ not reject H0 : β1 = 0
 tβc2 = 3.133 > 2.056 ⇒ reject H0 : β2 = 0
 tβc3 = 2.279 > 2.056 ⇒ reject H0 : β3 = 0
Sol 3: p-value approach
The Pr(> |t|) outputs give p-value    
pβc0 0.15540
   
 pβc   0.94193 
 1 = 
 p   0.00425 
 βc2   
pβc3 0.03109
 pβc0 = 0.15540 > 0.05 ⇒ p-value > α ⇒ not reject H0 : β0 = 0
 pβc1 = 0.94193 > 0.05 ⇒ p-value > α ⇒ not reject H0 : β1 = 0
 pβc0 = 0.00425 < 0.05 ⇒ p-value < α ⇒ reject H0 : β2 = 0
 pβc0 = 0.03109 < 0.05 ⇒ p-value < α ⇒ reject H0 : β3 = 0
Conclusion: At 5% significance level, the data do not provide sufficient evidence to conclude that the intercept
term β0 is not statistically different from zero, and ACETIC does not significantly add to a model that include H2S
and LACTIC, whereas H2S and LACTIC does considerably add to a model that include ACETIC, H2S and LACTIC

81).6 Confidence and prediction intervals for a given x = x0

We would like to create 100(1 − α)% intervals for the mean E(Y |x0 ) and for the new value Y ∗ (x0 ).
 100(1 − α)% interval for the mean E(Y |x0 ) is called a confidence interval (since it is for the mean response)
 100(1 − α)% interval for the new value Y ∗ (x0 ) is called a prediction interval (since it is for a new random variable)
Example: Cheese data(cnt)
Suppose that we are interested estimating E(Y |x0 ) and predicting a new Y ∗ (x0 ) when ACETIC = 5.5, H2S = 6.0 and
LACTIC = 1.4, so that x0 = (5.5, 6.0, 1.4)
1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)
3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > p r e d i c t ( model , d a t a . f r a m e ( a c e t i c = 5 . 5 , h 2 s = 6 . 0 , l a c t i c = 1 . 4 ) , l e v e l = 0 . 9 5 , i n t e r v a l = ”

Page 153 of 160

 STATISTICS OVERVIEW

confidence ” )
7 fit lwr upr
8 1 23.93552 20.04506 27.82597
9 > p r e d i c t ( model , d a t a . f r a m e ( a c e t i c = 5 . 5 , h 2 s = 6 . 0 , l a c t i c = 1 . 4 ) , l e v e l = 0 . 9 5 , i n t e r v a l = ”
prediction ” )
10 fit lwr upr
11 1 23.93552 2.751379 45.11966

The Estimate output gives the values of the 

least squares
 estimates: 
β
c0 −28.877
 c   
 β1   0.328 
 = 
 β
 c2   3.912
  

β
c3 19.670
⇒ The point estimate/prediction is:
Ŷ (x0 ) = β
c0 + β
c1 x10 + β c3 x30 = −28.877 + 0.328(5.5) + 3.912(6.0) + 19.670(1.4) ≈ 23.937
c2 x20 + β
 A 95% confidence interval for E(Y |x0 ) is (20.05, 27.83).
Interpretation: When ACETIC = 5.5, H2S = 6.0 and LACTIC = 1.4, we are 95% confident that the mean taste rating
is between 20.05 and 27.83
 A 95% prediction interval for Y ∗ (x0 ) is (2.75, 45.12).
Interpretation: When ACETIC = 5.5, H2S = 6.0 and LACTIC = 1.4, we are 95% confident that the taste rating for a
new cheese specimen will be between 2.75 and 45.12

82) Model diagnostics (residual analysis)

The term ”diagnostics” refers to the process of ”checking the model assumptions”. This is an important exercise be-
cause if the model assumptions are violated, then our analysis (and all subsequent interpretations) could be compromised

82).1 Model assumptions

We first recall the model assumptions on the error terms in the linear regression model
Yi = β0 + β1 xi1 + β2 xi2 + ... + βk xik + ϵi
for i = 1, 2, ..., n. Specifically, we have made the following assumptions
 E(ϵi ) = 0 for i = 1, 2, ..., n
 (ϵi ) = σ 2 for i = 1, 2, ..., n, that is, the variance is constant
 the random variables ϵi are independent
 the random variables ϵi are normally distributed
Under these assumptions

82).2 Residuals
In checking our model assumptions, we first have to deal with the obvious problem; namely, the error term ϵi in the
model are never observed. However, from the fit of the model, we can calculate the residuals:

ϵi = Yi − Ybi (10..38)

when the ith fitted value

Ybi = β
c0 + β
c1 xi1 + β
c2 xi2 + ... + β
ck xik
We can think of the residuals e1 , e2 , ..., en as ”proxies’ for the error term ϵ1 , ϵ2 , ..., ϵn , and, therefore, we can use the
residuals to check our model assumptions instead

Page 154 of 160

 STATISTICS OVERVIEW

82).3 QQ-plot for Normality

To check the normality assumption (for the error) in linear regression, it is common to display the qq-plot of the residuals
 Recall that is the plotted points follow a straight line (approximately), this supports the normality assumption
 Substantial deviation from linearity is not consistent with the normality assumption
Example: Cheese data(cnt)
The qq-plot supports the normality assumption for the errors in the multiple linear regression model
1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)
3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > qqnorm ( r e s i d u a l s ( model ) , pch = 1 9 , c o l = ” b l a c k ” , x l a b = ” T h e o r e t i c a l Q u a n t i l e s ” , y l a b = ” Sample
Quantities ” )
7 > q q l i n e ( r e s i d u a l s ( model ) , lwd = 1 . 5 , c o l = ” b l a c k ” )

Figure 10..2: Normal qq-plot of the least squares residuals

82).4 Residual plot

By the phrase ”residual plot” which is the plot of the residuals (on the vertical y-axis) versus the predicted values (on
the horizontal x-axis). This plot is simply the scatterplot of the residuals and the predicted values

Page 155 of 160

 STATISTICS OVERVIEW

 Advanced linear model arguments show that if the model does a good job at describing the data, then the residuals
and fitted values are independent
 This means that a plot of the residuals versus the fitted values should reveal no noticeable patterns; that is, the plot
should appear to be random in nature (e.g., “a random scatter of points”).
 On the other hand, if there are definite (non-random) patterns in the residual plot, this suggests that the model is
inadequate in some way or it could point to a violation in the model assumptions
Example: Cheese data(cnt)
1 > t a s t e <− c ( 1 2 . 3 , 2 0 . 9 , 3 9 . 0 , 4 7 . 9 , 5 . 6 , 2 5 . 9 , 3 7 . 3 , 2 1 . 9 , 1 8 . 1 , 2 1 . 0 , 3 4 . 9 , 5 7 . 2 , 0 . 7 , 2 5 . 9 , 5 4 . 9 ,
40.9 , 15.9 , 6.4 , 18.0 , 38.9 , 14.0 , 15.2 , 32.0 , 56.7 , 16.8 , 11.6 , 26.5 , 0.7 , 13.4 , 5.5)
2 > a c e t i c <− c ( 4 . 5 4 3 , 5 . 1 5 9 , 5 . 3 6 6 , 5 . 7 5 9 , 4 . 6 6 3 , 5 . 6 9 7 , 5 . 8 9 2 , 6 . 0 7 8 , 4 . 8 9 8 , 5 . 2 4 2 , 5 . 7 4 0 , 6 . 4 4 6 ,
4.477 , 5.236 , 6.151 , 6.365 , 4.787 , 5.412 , 5.247 , 5.438 , 4.564 , 5.298 , 5.455 , 5.855 , 5.366 ,
6.043 , 6.458 , 5.328 , 5.802 , 6.176)
3 > h 2 s <− c ( 3 . 1 3 5 , 5 . 0 4 3 , 5 . 4 3 8 , 7 . 4 9 6 , 3 . 8 0 7 , 7 . 6 0 1 , 8 . 7 2 6 , 7 . 9 6 6 , 3 . 8 5 0 , 4 . 1 7 4 , 6 . 1 4 2 , 7 . 9 0 8 ,
2.996 , 4.942 , 6.752 , 9.588 , 3.912 , 4.700 , 6.174 , 9.064 , 4.949 , 5.220 , 9.242 , 10.20 , 3.664 ,
3.219 , 6.962 , 3.912 , 6.685 , 4.787)
4 > l a c t i c <− c ( 0 . 8 6 , 1 . 5 3 , 1 . 5 7 , 1 . 8 1 , 0 . 9 9 , 1 . 0 9 , 1 . 2 9 , 1 . 7 8 , 1 . 2 9 , 1 . 5 8 , 1 . 6 8 , 1 . 9 0 , 1 . 0 6 , 1 . 3 0 ,
1.52 , 1.74 , 1.16 , 1.49 , 1.63 , 1.99 , 1.15 , 1.33 , 1.44 , 2.01 , 1.31 , 1.46 , 1.72 , 1.25 , 1.08 , 1.25)
5 > model <− lm ( t a s t e ˜ a c e t i c + h 2 s + l a c t i c )
6 > p l o t ( f i t t e d ( model ) , r e s i d u a l s ( model ) , x l a b = ” F i t t e d v a l u e s ” , y l a b = ” R e s i d u a l s ” , pch = 1 9 , c o l =
” black ” )
7 > a b l i n e ( h = 0 , c o l = ” b l a c k ” , lwd = 1 . 5 )

Figure 10..3: Residual plot for the multiple linear regression model fit

82).5 Common violations

Although there are many ways to violate the statistical assumptions associated with linear regression, the most common
violations are

Page 156 of 160

 STATISTICS OVERVIEW

 non-constant variance (heteroscedasticity)

 misspecifying the true regression function
 correlated observations over time
Example: Electricity data
An electric company is interested in modeling peak hour electricity demand Y (measured in kWh) as a function of total
monthly energy usage x (measured in kWh). This is important for planning purposes because the generating system
must be large enough to meet the maximum demand imposed by customers.
There are n = 53 residential customers
Problem: There is a clear problem with non-constant variance here. Note how the residual plot “fans out” like the bell

Scatterplot of peak demand Y versus monthly usage x Residual plot for the simple linear regression model fit

of a trumpet. This violation may have been missed by looking at the scatterplot alone, but the residual plot highlights it.
Remedy: A common course of action to handle non-constant variance is to apply a transformation to the response
√ 1
variable Y . Common transformations are logarithmic (ln Y ), square-root ( Y ), and inverse .
Y
A square root transformation is commonly applied to address non-constant variance. Consider the simple linear regres-
sion model
W i = β0 + β1 x i + ϵi
√
for i = 1, 2, ..., 53 where wi = Yi . It is straightforward to fit a quadratic model in R. We simply regress W on x
(instead of regression Y on x)

The ”fanning out” shape that we saw previously (in the untransformed model) is now largely absent. The fitted trans-
formed model is
Ŷ = 0.580831 + 0.0000953x
or, in the other words,
√
Peak demand = 0.580831 + 0.0000953 × Monthly usage
1 > f i t . 2 = lm ( s q r t ( peak . demand ) ˜ monthly . u s a g e )
2 > f i t .2
3 Coefficients :
4 ( I n t e r c e p t ) monthly . u s a g e
5 0.580831 0.000953

Page 157 of 160

STATISTICS OVERVIEW

√
Scatterplot of square root peak demand Y versus Residual plot for the simple linear regression model fit
monthly usage x with transformed response

Example: Windmill data

A research engineer is investigating the use of a windmill to generate electricity. He has collected data on the direct
current (DC) output Y from his windmill and the corresponding wind velocity (x, measured in mph) with n = 25
observation pairs

Scatterplot of DC output Y versus wind velocity x Residual plot for the simple linear regression model fit

Problem: There is a clear quadratic relationship between DC output and wind velocity, so a simple linear regression
model fit (as shown above) is inappropriate. The residual plot shows a pronounced quadratic pattern; this pattern is not
accounted for in fitting a straight line model.
Remedy: Fit a multiple linear regression model with two independent variables: wind velocity x and its square x2 , that
is, consider the quadratic regression model
Yi = β0 + β1 xi + β2 x2i + ϵi
for i = 1, 2, ..., 25. It is straightforward to fit a quadratic model in R. We simply regress Y on x and x2 .
The fitted quadratic regression model is
Ŷ = −1.15590 + 0.72294x − 0.03812x2
or, in other words,

Page 158 of 160

 STATISTICS OVERVIEW

output = −1.15590 + 0.72294 × [Link] − 0.03812([Link])2

DC\
Note that the residual plot from the quadratic model fit now looks quite good. The quadratic trend has disappeared
(because the model now incorporates it)
1 > wind . v e l o c i t y . s q = wind . v e l o c i t y ˆ 2
2 > f i t . 2 = lm ( DC . o u t p u t ˜ wind . v e l o c i t y + wind . v e l o c i t y . s q )
3 > f i t .2
4 Coefficients :
5 ( Intercept ) wind . v e l o c i t y wind . v e l o c i t y . s q
6 −1.15590 0.72294 −0.03812

Scatterplot of DC output Y versus wind velocity x with

Residual plot for the simple linear regression model fit
least squares quadratic regression curve superimposed

Example: Global temperature data

The data below are temperature readings (in deg C) on land-air average temperature anomalies, collected one per year
from 1900 − 1997. To emphasize that the data are collected over time, I have used straight lines to connect the obser-
vations, this is called a time series plot
Temperature Y measured one time per year, and the independent variable x is year, measured as 1900, 1901, ..., 1997.
A simple linear regression model fit has been superimposed
 Unfortunately, it is all too common that people fit linear regression models to time series data and then blindly use
them for prediction purposes
 It takes neither a meteorologist nor an engineering degree to know that temperature observations collected over time
are probably correlated. Not surprisingly, residuals from a simple linear regression display clear correlation over time
 Regression techniques are generally not appropriate when analyzing time series data for this reason. More advanced
modeling techniques are needed

Page 159 of 160

STATISTICS OVERVIEW

Time series plot of Y versus year x Residual plot for the simple linear regression model fit

xx

Page 160 of 160