Module IV
Statistics
1
– Random variables (Discrete and Continuous).
– Probability mass functions and density functions.
– Cumulative distribution functions.
– Expectations and variances of random variables.
STAT 101 2
Random variables
Random variable
Let S be a sample space corresponding to the random
experiment E. A function X defined on the sample space S
to the real number system R is called a random variable.
R
X(s)=x
• Note that random variables are written by uppercase letters.
• The values taken by the random variables are written by the
corresponding lowercase letters..
STAT 101 3
Random variables
Example 5.1
• E – Tossing a fair coin twice and let X be the number of heads
obtained.
• The sample space S = {TT, TH, HT, HH}
• The Set R = {0,1, 2} is called the range space of X.
Discrete Random Variable
• Assumes a finite or countably infinite number of values.
• The values can be enumerated and listed as
STAT 101 4
Random variables
Continuous Random variables
• Assumes uncountable number of values from a region.
Examples
• X= Height of an individual in our class (Continuous random
variable).
• X = Number of Female students in our class (Discrete random
variable)
• X = Number of tosses required of a coin to obtain Head for the
fifth time. (Discrete random variable)
STAT 101 5
Random variables
Probability mass function
• Let X be a discrete random variable with range space
• With each value of X we assign a nonnegative real number
satisfying the following conditions
x1 x2 . . . xn
a)
b)
p(x1) p(x2) . . . p(xn )
• The assignment is called the probability mass
function of X.
STAT 101 6
Lecture 4 - Random variables
Example 5.2
• Let X be the number of heads obtained. The range space of X is
•We assign a nonnegative mass with each value as shown
• The assignment is a valid probability assignment and hence is
known as probability mass function of X.
P(x)
1/2
1/4
1
2
7
Random variables
• To find the probability of an event A, we add the probabilities
attached to the elements of the event.
P ( A)
x A
P (x)
• For example,
P ( X 1) P ( X 1) P ( X 2 )
1 1 3
2 4 4
STAT 101 8
Random variables
Probability density function (pdf)
A function f ( x ) can be a probability density function of a
continuous random variable X if it satisfies the following
conditions
( a ) f ( x ) 0 for all x
(b )
f ( x ) dx 1
Note that at any specific point x 0 has no meaning.
• To find the probability of an event, we integrate the density
function over the event. That is,
b
P (a X b)
a
f ( x ) dx
STAT 101 11
Random variables
Example 5.3
2 x, 0 x 1
Consider the function f ( x)
0 , elsewhere
Since, 0 x 1 , f ( x ) 0 for all x .
1 1 1
x2
0
f ( x ) dx 2 x dx 2
0
2 0
1 0 1
The function f (x ) is a valid pdf.
1 2 1
x 1 3
To find P ( 0 .5 X 1) 2 x dx 2 1
0 .5
2 0 .5
4 4
STAT 101 12
Random variables
Cumulative Distribution Function (CDF)
The cumulative distribution function of a random variable X is
defined as
F ( x) P ( X x)
• For a discrete random variable
F ( x) P (x)
X x
x
• For a continuous random variable X x
x
F ( x)
f ( x ) dx
STAT 101 13
Random variables
Example 5.4
• Consider the prob. distribution in example 4.2.
1 2 1
P(X 0) P(X 1) P(X 2)
4 4 4
P(x)
F(x)
1/2
p (2)
1/4
3/4
1/4
p (1 )
x
p (0 )
0
1 2
0 1 2
• The CDF is Properties of F(x) ?
STAT 101 14
Random variables
Example 5.5
Consider the density function 2 x, 0 x 1
f (x)
f (x) 0 , elsewhere
0 1
The cumulative distribution function
F(x)
0 if x 0
x 1
F(x) 2x dx x2 if 0 x 1 x2
0
1
1
if x 1
0
STAT 101 15
Random variables
Properties
(a) The CDF of a continuous random variable is a continuous
function.
(b) Non decreasing function
(c) F ( ) 0 and F ( ) 1
d
(d) The pdf f ( x ) F ( x )
dx
STAT 101 16
Random variables
Characteristics of a random variables
Expected value of a random variable
Let X be a discrete random variable with known probability
distribution p(x). The expected value of X is defined as
E(X ) x
1
i p ( xi )
• E(X) is a weighted average of all possible values of X.
• E(X) is a population mean and we denote it by X
• For a continuous random variable with pdf f(x),
E(X ) x f ( x ) dx
STAT 101 17
Random variables
Properties
(a) E(c) = c, where c is a constant.
(b) E(a + b X) = a + b E(X), where a and b are constants.
(c) E (X + Y ) = E (X) + E ( Y)
(d) E ( XY ) = E(X) E(Y) only if X and Y are independent
(e) Let X be a random variable and let Y = g(X), then
STAT 101 18
Random variables
Example 5.6
The probability distribution of daily demand for a product is
d 1 2 3 4 5
p(d) 0.1 0.1 0.3 0.3 0.2
Evaluate E(D).
Solution
BY definition
5
E (D) d
1
i p(d i )
1( 0 .1) 2 ( 0 .1) 3( 0 .3) 4 ( 0 .3) 5 ( 0 .2 )
0 .1 0 .2 0 .9 1 .2 1 .0 3 .4
STAT 101 19
Random variables
Variance
• measures the spread of values of X around its mean value.
• measured in the squared units of measurement for X.
• defined as
V ( X ) X2 E{ X E ( X )}2
• The positive square root of V(X) is called the standard
deviation of X and is denoted by X
• Variance X2 E { X E ( X )} 2
E{ X 2
E X
2 2 XE ( X )
E X 2 E X 2 E X
2 2
V ( X ) E X 2 E X
2
20
Random variables
Properties
(a) V (c) = 0, where c is a constant.
(b) V(X + c) = V(X)
2
(c) V(c X) = V ( X )
c
(d) If X and Y are independent, then V(X + Y) = V(X) + V(Y)
STAT 101 21
Random variables
Example 5.7
24
Suppose that X has pdf f ( x ) 4 , x 2. Evaluate the variance of X.
x
Solution
By definition
E(X ) x
2
f ( x ) dx
24 1
x 4 dx 24 x dx 24
3
2
3
2
x 2
2 x 2
STAT 101 22
Random variables
Example 5.7
E(X 2)
2
x 2 f ( x ) dx
24 1
x dx 24 x dx 24
2
2
4
12
2
x 2
x 2
V ( X ) E X E X
2 2
12 3 2 3
STAT 101 23
Binomial distribution
Take the example of 5 coin tosses. What’s the probability that you
flip exactly 3 heads in 5 coin tosses?
Binomial distribution
Solution:
One way to get exactly 3 heads: HHHTT
What’s the probability of this exact arrangement?
P(heads)xP(heads) xP(heads)xP(tails)xP(tails)
=(1/2)3 x (1/2)2
Another way to get exactly 3 heads: THHHT
Probability of this exact outcome = (1/2)1 x (1/2)3 x
(1/2)1 = (1/2)3 x (1/2)2
Binomial distribution
In fact, (1/2)3 x (1/2)2 is the probability of each
unique outcome that has exactly 3 heads and 2
tails.
So, the overall probability of 3 heads and 2 tails is:
(1/2)3 x (1/2)2 + (1/2)3 x (1/2)2 + (1/2)3 x (1/2)2 + …..
for as many unique arrangements as there are—
but how many are there??
Outcome Probability
THHHT (1/2)3 x (1/2)2
HHHTT (1/2)3 x (1/2)2
TTHHH (1/2)3 x (1/2)2
HTTHH (1/2)3 x (1/2)2 The probability
ways to HHTTH (1/2)3 x (1/2)2
5
arrange 3
of each unique
heads in
HTHHT
THTHH
(1/2)3 x (1/2)2
(1/2)3 x (1/2)2
outcome (note:
3 5 trials HTHTH (1/2)3 x (1/2)2
they are all
equal)
HHTHT (1/2)3 x (1/2)2
THHTH (1/2)3 x (1/2)2
10 arrangements x (1/2)3 x (1/2)2
5C3 = 5!/3!2! = 10
Factorial review: n! = n(n-1)(n-2)…
5
P(3 heads and 2 tails) = x P(heads)3 x P(tails)2 =
3
10 x (½)5=31.25%
Binomial distribution function:
X= the number of heads tossed in 5 coin tosses
p(x)
x
0 1 2 3 4 5
number of heads
Binomial distribution, generally
Note the general pattern emerging if you have only two possible
outcomes (call them 1/0 or yes/no or success/failure) in n independent
trials, then the probability of exactly X “successes”=
n = number of trials
X
n
nX
p (1 p)
X 1-p = probability
of failure
X=#
p=
successes probability of
out of n success
trials
Binomial distribution: example
• If I toss a coin 20 times, what’s the probability of getting exactly
10 heads?
20
10 10
(. 5 ) (. 5 ) . 176
10
Binomial distribution: example
• If I toss a coin 20 times, what’s the probability of getting of
getting 2 or fewer heads?
20 20 !
(. 5 ) 0
(. 5 ) 20
(. 5 ) 20 9 . 5 x10 7
0 20 !0!
20 20 !
(. 5 ) 1
(. 5 ) 19
(. 5 ) 20 20 x 9 . 5 x10 7 1 . 9 x10 5
1 19 !1!
20 20 !
(. 5 ) 2
(. 5 ) 18
(. 5 ) 20 190 x 9 . 5 x10 7 1 . 8 x10 4
2 18 !2!
1 . 8 x10 4
**All probability distributions are characterized by
an expected value and a variance:
If X follows a binomial distribution with parameters n and p: X ~ Bin
(n, p)
Then:
E(X) = np
Var (X) = np(1-p)
SD (X)= Note: the variance will
always lie between
0*N-.25 *N
p(1-p) reaches maximum
at p=.5
np (1 p )
P(1-p)=.25
Practice Problem
• 1. You are performing a cohort study. If the probability of
developing disease in the exposed group is .05 for the
study duration, then if you (randomly) sample 500
exposed people, how many do you expect to develop the
disease? Give a margin of error (+/- 1 standard deviation)
for your estimate.
• 2. What’s the probability that at most 10 exposed people
develop the disease?
Answer
1. How many do you expect to develop the disease? Give a margin of error (+/-
1 standard deviation) for your estimate.
X ~ binomial (500, .05)
E(X) = 500 (.05) = 25
Var(X) = 500 (.05) (.95) = 23.75
StdDev(X) = square root (23.75) = 4.87
25 4.87
Answer
2. What’s the probability that at most 10 exposed
subjects develop the disease?
This is asking for a CUMULATIVE PROBABILITY: the probability of 0 getting the
disease or 1 or 2 or 3 or 4 or up to 10.
P(X≤10) = P(X=0) + P(X=1) + P(X=2) + P(X=3) + P(X=4)+….+ P(X=10)=
500 0 500 500 1 499 500 2 498 500 10 490
(.05) (.95) (.05) (.95) (.05) (.95) ... (.05) (.95) .01
0 1 2 10
Practice Problem:
You are conducting a case-control study of
smoking and lung cancer. If the probability of
being a smoker among lung cancer cases is .6,
what’s the probability that in a group of 8 cases
you have:
a. Less than 2 smokers?
b. More than 5?
c. What are the expected value and variance of the number
of smokers?
Answer
X P(X)
8
0 1(.4) =.00065
1 7
1 8(.6) (.4) =.008
2 6
2 28(.6) (.4) =.04
3 5
3 56(.6) (.4) =.12
4 4
4 70(.6) (.4) =.23
5 3
5 56(.6) (.4) =.28
6 2
6 28(.6) (.4) =.21
7 1
7 8(.6) (.4) =.090
8
8 1(.6) =.0168
0 1 2 3 4 5 6 7 8
Answer, continued
P(>5)=.21+.09+.0168 = .3168
P(<2)=.00065 + .008 = .00865
0 1 2 3 4 5 6 7 8
E(X) = 8 (.6) = 4.8
Var(X) = 8 (.6) (.4) =1.92
StdDev(X) = 1.38
Review Question 4
In your case-control study of smoking and lung-
cancer, 60% of cases are smokers versus only 10%
of controls. What is the odds ratio between smoking
and lung cancer?
a. 2.5
b. 13.5
c. 15.0
d. 6.0
e. .05
Review Question 4
In your case-control study of smoking and lung-
cancer, 60% of cases are smokers versus only 10%
of controls. What is the odds ratio between smoking
and lung cancer?
a. 2.5
b. 13.5
c. 15.0 .6
d. 6.0 .4 3 x 9 27 13.5
e. .05 .1 2 1 2
.9
Review Question 5
What’s the probability of getting exactly 5 heads
in 10 coin tosses?
a.
10
5
(.50) (.50)
5
0
b.
10
5 5
(.50) (.50)
c. 5
10
d.
10 5
(.50) (.50)
5
10 10 0
(.50) (.50)
10
Review Question 5
What’s the probability of getting exactly 5 heads
in 10 coin tosses?
a.
10
5
(.50) (.50)
5
0
b.
10
5 5
(.50) (.50)
c. 5
10
d.
10 5
(.50) (.50)
5
10 10 0
(.50) (.50)
10
Review Question 6
A coin toss can be thought of as an example of a
binomial distribution with N=1 and p=.5. What are the
expected value and variance of a coin toss?
a. .5, .25
b. 1.0, 1.0
c. 1.5, .5
d. .25, .5
e. .5, .5
Review Question 6
A coin toss can be thought of as an example of a
binomial distribution with N=1 and p=.5. What are the
expected value and variance of a coin toss?
a. .5, .25
b. 1.0, 1.0
c. 1.5, .5
d. .25, .5
e. .5, .5
Review Question 7
If I toss a coin 10 times, what is the expected value
and variance of the number of heads?
a. 5, 5
b. 10, 5
c. 2.5, 5
d. 5, 2.5
e. 2.5, 10
Review Question 7
If I toss a coin 10 times, what is the expected value
and variance of the number of heads?
a. 5, 5
b. 10, 5
c. 2.5, 5
d. 5, 2.5
e. 2.5, 10
Review Question 8
In a randomized trial with n=150, the goal is to randomize
half to treatment and half to control. The number of
people randomized to treatment is a random variable X.
What is the probability distribution of X?
a. X~Normal(=75,=10)
b. X~Exponential(=75)
c. X~Uniform
d. X~Binomial(N=150, p=.5)
e. X~Binomial(N=75, p=.5)
Review Question 8
In a randomized trial with n=150, every subject has a 50%
chance of being randomized to treatment. The number of
people randomized to treatment is a random variable X.
What is the probability distribution of X?
a. X~Normal(=75,=10)
b. X~Exponential(=75)
c. X~Uniform
d. X~Binomial(N=150, p=.5)
e. X~Binomial(N=75, p=.5)
Review Question 9
In the same RCT with n=150, if 69 end up in the treatment
group and 81 in the control group, how far off is that from
expected?
a. Less than 1 standard deviation
b. 1 standard deviation
c. Between 1 and 2 standard deviations
d. More than 2 standard deviations
Review Question 9
In the same RCT with n=150, if 69 end up in the treatment
group and 81 in the control group, how far off is that from
expected?
a. Less than 1 standard deviation
Expected = 75
b. 1 standard deviation 81 and 69 are both 6 away from
the expected.
c. Between 1 and 2 standard deviations
Variance = 150(.25) = 37.5
d. More than 2 standard deviations Std Dev 6
Therefore, about 1 SD away
from expected.
Proportions…
• The binomial distribution forms the basis of
statistics for proportions.
• A proportion is just a binomial count divided by n.
• For example, if we sample 200 cases and find 60
smokers, X=60 but the observed proportion=.30.
• Statistics for proportions are similar to binomial
counts, but differ by a factor of n.
Stats for proportions
For binomial: x np Differs by a
factor of n.
x np(1 p )
2
x np(1 p) Differs by a
factor of n.
pˆ p
For proportion:
np(1 p ) p (1 p )
pˆ
2
2
n n
P-hat stands for “sample proportion.” p (1 p )
pˆ
n
It all comes back to normal…
• Statistics for proportions are based on a normal distribution,
because the binomial can be approximated as normal if np>5