STAT7055
Topic 3
Discrete Random Variables
STAT7055 - Topic 3 1 / 63
Random Variables
Random Variable
I Suppose we flip a fair coin three times.
I The sample space is:
S = {HHH, HHT, HT H, HT T,
T HH, T HT, T T H, T T T }
I Each outcome is equally likely to occur.
I Define a new quantity (call it X) which is equal to
the number of heads that occur in the three coin
flips.
STAT7055 - Topic 3 2 / 63
Random Variables
Random Variable
I X can take the value 0, 1, 2 or 3.
I The actual value that X takes is random and
depends on the outcome of the experiment.
I X is what we call a random variable.
I Formally, a random variable is a function that
assigns a numeric value to each simple event in a
sample space.
STAT7055 - Topic 3 3 / 63
Random Variables
Notation
I Denote random variables using uppercase letters,
e.g., X, Y , Z.
I Denote the actual observed or realised value of the
random variable by lowercase letters, e.g., x, y, z.
I Back to coin flipping example:
I X is the random variable that can take values 0, 1, 2 or
3.
I If we actually perform the experiment and observe the
outcome HHT , then the realised value of X is x = 2.
STAT7055 - Topic 3 4 / 63
Random Variables
Discrete Random Variable
I A discrete random variable is one that can take
on a countable number of possible values.
I For example:
I Flip a coin five times and let X be the number of heads
that occurs. The possible values are
X = 0, 1, 2, 3, 4 or 5.
I Flip a coin until it comes up tails and let X be the total
number of flips needed. The possible values are
X = 1, 2, 3, 4, 5, 6, 7, . . .
STAT7055 - Topic 3 5 / 63
Random Variables
Continuous Random Variable
I A continuous random variable is one that can
take on an uncountable number of possible values -
the number of possible values is infinite as a result
of continuous variation.
I For example:
I Let X be the time taken to finish a three hour exam.
I Let X be the weight of a boxer.
STAT7055 - Topic 3 6 / 63
Discrete Random Variables Discrete Probability Distributions
Discrete Probability Distribution
I For a discrete random variable X, how can we
determine P (X = x) for any given value of x?
I The probability that a discrete random variable X
takes the value x is denoted by p(x) and is equal to
the sum of all the probabilities of the simple events
for which X = x.
STAT7055 - Topic 3 7 / 63
Discrete Random Variables Discrete Probability Distributions
Discrete Probability Distribution
I A discrete probability distribution is a table or
formula listing all possible values that a discrete
random variable can take, together with the
corresponding probability for each value.
I A discrete probability distribution must satisfy two
requirements:
0 ≤ p(x) ≤ 1 for all x.
1. P
2. all x p(x) = 1
STAT7055 - Topic 3 8 / 63
Discrete Random Variables Discrete Probability Distributions
Example
I Flip a coin three times, let X be the number of
heads.
1
p(0) = P (X = 0) = P ({T T T }) =
8
3
p(1) = P (X = 1) = P ({HT T, T HT, T T H}) =
8
3
p(2) = P (X = 2) = P ({HHT, HT H, T HH}) =
8
1
p(3) = P (X = 3) = P ({HHH}) =
8
STAT7055 - Topic 3 9 / 63
Discrete Random Variables Discrete Probability Distributions
Example
x 0 1 2 3
1 3 3 1
p(x) 8 8 8 8
I What is the probability of at most one head?
1 3 1
P (X ≤ 1) = p(0) + p(1) = + =
8 8 2
I What is the probability of at least one head?
3 3 1 7
P (X ≥ 1) = p(1) + p(2) + p(3) = + + =
8 8 8 8
1 7
= 1 − p(0) = 1 − =
8 8
STAT7055 - Topic 3 10 / 63
Discrete Random Variables Discrete Probability Distributions
Probability Distributions and Populations
I Probability distributions represent populations.
I Rather than recording every observation in the
population, a probability distribution summarises the
population by listing only the possible values that
appear in the population, together with their
corresponding probabilities.
I We can calculate population parameters such as the
population mean and population variance from a
probability distribution.
STAT7055 - Topic 3 11 / 63
Discrete Random Variables Expected Value
Expected Value
I Let X be a discrete random variable with probability
distribution p(x). The expected value (or
population mean) of X is defined to be:
X
µ = E(X) = (x × p(x))
all x
I Compare this to the formula for the population
mean given in topic 1:
N N
1 X X 1
µ= Xi = xi ×
N i=1 i=1
N
STAT7055 - Topic 3 12 / 63
Discrete Random Variables Expected Value
Expected Value
I It is straightforward to calculate the expected value
of any function of a discrete random variable X.
I Let g(X) be some function of X. Then the
expected value of g(X) is defined to be:
X
E(g(X)) = (g(x) × p(x))
all x
STAT7055 - Topic 3 13 / 63
Discrete Random Variables Expected Value
Example
x 0 1 2 3
1 3 3 1
p(x) 8 8 8 8
X
E(X) = (x × p(x))
all x
1 3 3 1
=0× +1× +2× +3×
8 8 8 8
12
=
8
= 1.5
STAT7055 - Topic 3 14 / 63
Discrete Random Variables Expected Value
Example
x 0 1 2 3
1 3 3 1
p(x) 8 8 8 8
X
E(X 2 ) = x2 × p(x)
all x
1 3 3 1
= 02 × + 12 × + 22 × + 32 ×
8 8 8 8
24
=
8
=3
STAT7055 - Topic 3 15 / 63
Discrete Random Variables Expected Value
Laws of Expected Value
I If X and Y are random variables (discrete or
continuous) and c is any constant, then:
1. E(c) = c
2. E(cX) = cE(X)
3. E(X + Y ) = E(X) + E(Y )
4. E(X − Y ) = E(X) − E(Y )
I And if X and Y are independent, then:
5. E(XY ) = E(X) × E(Y )
STAT7055 - Topic 3 16 / 63
Discrete Random Variables Expected Value
Laws of Expected Value - Example
I Let Z = 3X + 2Y − 2XY + 3 with E(X) = 3,
E(Y ) = 5 and X and Y independent. Then:
E(Z) = E(3X + 2Y − 2XY + 3)
= E(3X) + E(2Y ) − E(2XY ) + E(3)
= 3E(X) + 2E(Y ) − 2E(X)E(Y ) + 3
=3×3+2×5−2×3×5+3
= −8
STAT7055 - Topic 3 17 / 63
Discrete Random Variables Variance
Variance
I Let X be a discrete random variable with probability
distribution p(x) and µ = E(X).
I The (population) variance of X is defined as:
X
σ 2 = V (X) = E (X − µ)2 = (x − µ)2 × p(x)
all x
I A shortcut formula for the variance is given below:
V (X) = E X 2 − (E(X))2
!
X
x2 × p(x) − µ2
=
all x
STAT7055 - Topic 3 18 / 63
Discrete Random Variables Variance
Example
x 0 1 2 3
1 3 3 1
p(x) 8 8 8 8
!
X
x2 × p(x) − µ2
V (X) =
all x
1 3 3 1
= 02 × + 12 × + 22 × + 32 × − 1.52
8 8 8 8
= 0.75
p √
SD(X) = V (X) = 0.75 = 0.866 = σ
STAT7055 - Topic 3 19 / 63
Discrete Random Variables Variance
Laws of Variance
I If X and Y are random variables (discrete or
continuous) and c is any constant, then:
1. V (c) = 0
2. V (cX) = c2 V (X)
3. V (X + c) = V (X)
I And if X and Y are independent, then:
4. V (X + Y ) = V (X) + V (Y )
5. V (X − Y ) = V (X) + V (Y )
STAT7055 - Topic 3 20 / 63
Discrete Random Variables Variance
Laws of Variance - Example
I Let Z = 3X − 2Y − 7 with V (X) = 2, V (Y ) = 1
and X and Y independent. Then:
V (Z) = V (3X − 2Y − 7)
= V (3X − 2Y )
= V (3X) + V (2Y )
= 9V (X) + 4V (Y )
=9×2+4×1
= 22
STAT7055 - Topic 3 21 / 63
Discrete Random Variables Bivariate Distributions
Bivariate Distribution
I If X and Y are discrete random variables, then the
bivariate distribution of X and Y is a table or
formula that lists the joint probabilities
P ({X = x} ∩ {Y = y}), denoted p(x, y), for all
pairs of x and y.
I A bivariate distribution must satisfy two
requirements:
0 ≤ p(x,
1. P P y) ≤ 1 for all x and y
2. all x all y p(x, y) = 1
STAT7055 - Topic 3 22 / 63
Discrete Random Variables Bivariate Distributions
Example
I Flip a coin three times.
I Let X be the number of heads.
I Let Y be the number of sequence changes within
the three flips, i.e., the number of times we change
from H ⇒ T or T ⇒ H.
I For example:
I HHH: x = 3 (3 heads) and y = 0 (0 sequence changes
since H ⇒ H ⇒ H).
I HHT : x = 2 (2 heads) and y = 1 (1 sequence change
since H ⇒ H ⇒ T ).
I HT H: x = 2 (2 heads) and y = 2 (2 sequence changes
since H ⇒ T ⇒ H).
STAT7055 - Topic 3 23 / 63
Discrete Random Variables Bivariate Distributions
Example
Outcome x y
HHH 3 0
HHT 2 1
HT H 2 2
T HH 2 1
TTH 1 1
T HT 1 2
HT T 1 1
TTT 0 0
STAT7055 - Topic 3 24 / 63
Discrete Random Variables Bivariate Distributions
Example
y
0 1 2
0 1/8 0 0 1/8
1 0 2/8 1/8 3/8
x 2/8 1/8 3/8
2 0
3 1/8 0 0 1/8
2/8 4/8 2/8 1
STAT7055 - Topic 3 25 / 63
Discrete Random Variables Marginal Probability Distributions
Marginal Probability Distribution
I Just like we did last topic, we can calculate marginal
probabilities for X and Y by adding across the rows
and down the columns, respectively.
I Specifically, given p(x, y) (the bivariate distribution
of X and Y ), the marginal probability
distribution of X is:
X
pX (x) = P (X = x) = p(x, y)
all y
STAT7055 - Topic 3 26 / 63
Discrete Random Variables Marginal Probability Distributions
Marginal Probability Distribution
I So for our example,
3
pX (1) = P (X = 1) = p(1, 0) + p(1, 1) + p(1, 2) =
8
I Considering Y for the moment, notice that the
events {Y = 0}, {Y = 1} and {Y = 2} are a
partition of the sample space!
I So, calculating a marginal probability distribution is
just a direct consequence of the Law of Total
Probability.
STAT7055 - Topic 3 27 / 63
Discrete Random Variables Marginal Probability Distributions
Marginal Probability Distribution
I The marginal distribution of X is:
x 0 1 2 3
1 3 3 1
pX (x) 8 8 8 8
I The marginal distribution of Y is:
y 0 1 2
2 4 2
pY (y) 8 8 8
STAT7055 - Topic 3 28 / 63
Discrete Random Variables Independence
Independence of Random Variables
I Two discrete random variables, X and Y , are
independent if and only if
p(x, y) = pX (x) × pY (y)
for all x and y.
I Note that this has to be true for all x and y. If
there is just one pair of x and y for which the above
is not true, then X and Y are not independent.
STAT7055 - Topic 3 29 / 63
Discrete Random Variables Independence
Example
I In our previous coin flipping example, X and Y are
clearly not independent since if we consider the pair
x = 0 and y = 0:
1
p(0, 0) =
8
but
1 2 1
pX (0) × pY (0) = × =
8 8 32
I That is, we have found one pair for which
p(x, y) 6= pX (x) × pY (y)
STAT7055 - Topic 3 30 / 63
Discrete Random Variables Functions of Random Variables
Sum of Two Random Variables
I Consider two real estate agents, Albert and Bob.
I Let X be the number of houses sold by Albert in a week.
I Let Y be the number of houses sold by Bob in a week.
x
pY (y)
0 1 2
0 0.12 0.42 0.06 0.6
y 1 0.21 0.06 0.03 0.3
2 0.07 0.02 0.01 0.1
pX (x) 0.4 0.5 0.1 1
STAT7055 - Topic 3 31 / 63
Discrete Random Variables Functions of Random Variables
Sum of Two Random Variables
I From the marginal probability distributions of X and
Y , it is straightforward to calculate the following:
I E(X) = 0.7
I V (X) = 0.41
I E(Y ) = 0.5
I V (Y ) = 0.45
I Suppose we are interested in the total number of
houses Albert and Bob sell in a week.
STAT7055 - Topic 3 32 / 63
Discrete Random Variables Functions of Random Variables
Sum of Two Random Variables
I That is, we are interested in the quantity X + Y ,
which itself is a random variable.
I From the bivariate distribution table, we know the
possible values of X + Y are 0, 1, 2, 3 or 4.
I Suppose we want to find the probability that a total
of two houses were sold in a week, i.e.,
P (X + Y = 2).
STAT7055 - Topic 3 33 / 63
Discrete Random Variables Functions of Random Variables
Sum of Two Random Variables
I From the table, we can find P (X + Y = 2) by
summing up all the joint probabilities for the values
of x and y which give x + y = 2.
I That is,
P (X + Y = 2) = p(0, 2) + p(1, 1) + p(2, 0)
= 0.07 + 0.06 + 0.06
= 0.19
STAT7055 - Topic 3 34 / 63
Discrete Random Variables Functions of Random Variables
Sum of Two Random Variables
I We can repeat this for X + Y = 0, 1, 3 and 4 to
obtain the probability distribution for X + Y :
x+y 0 1 2 3 4
p(x + y) 0.12 0.63 0.19 0.05 0.01
I From this we can calculate the mean and variance
of X + Y :
I E(X + Y ) = 1.2
I V (X + Y ) = 0.56
STAT7055 - Topic 3 35 / 63
Discrete Random Variables Functions of Random Variables
Functions of Two Random Variables
I Note that we could use the same approach to
calculate the probability distribution of any function
of two discrete random variables.
I For example:
I g(X, Y ) = XY
√
I g(X, Y ) = XY 3
I g(X, Y ) = YX+1
I etc.
STAT7055 - Topic 3 36 / 63
Discrete Random Variables Functions of Random Variables
Expected Value
I If X and Y are two discrete random variables with
bivariate distribution p(x, y) and g(X, Y ) is some
function of X and Y , the expected value of
g(X, Y ) is given by:
XX
E(g(X, Y )) = (g(x, y) × p(x, y))
all x all y
STAT7055 - Topic 3 37 / 63
Discrete Random Variables Covariance and Correlation
Covariance
I Let X and Y be discrete random variables with
joint probability distribution p(x, y).
I If we denote E(X) = µX and E(Y ) = µY , then the
(population) covariance between X and Y is:
σXY = Cov(X, Y )
= E ((X − µX )(Y − µY ))
XX
= ((x − µX )(y − µY ) × p(x, y))
all x all y
STAT7055 - Topic 3 38 / 63
Discrete Random Variables Covariance and Correlation
Covariance
I Just like with the variance, there is a shortcut
formula for calculating the covariance:
Cov(X, Y ) = E (XY ) − E(X)E(Y )
XX
= (xy × p(x, y)) − µX µY
all x all y
STAT7055 - Topic 3 39 / 63
Discrete Random Variables Covariance and Correlation
Correlation Coefficient
I The (population) correlation coefficient is
defined in exactly the same way as before:
σXY
ρXY =
σX σY
I Remember that the correlation always lies between
−1 and 1, i.e., −1 ≤ ρXY ≤ 1.
STAT7055 - Topic 3 40 / 63
Discrete Random Variables Covariance and Correlation
Example
I Flip a coin three times.
I X is the number of heads, Y is the number of
sequence changes.
I We know that (check for yourself):
I µX = 32
I σX2
= 34
I µY = 1
I σY2 = 12
STAT7055 - Topic 3 41 / 63
Discrete Random Variables Covariance and Correlation
Example
XX
Cov(X, Y ) = (xy × p(x, y)) − µX µY
all x all y
1 2 1
= 0×0× +1×1× +1×2×
8 8 8
2 1 1
+2 × 1 × + 2 × 2 × + 3 × 0 ×
8 8 8
3
− ×1
2
=0
STAT7055 - Topic 3 42 / 63
Discrete Random Variables Covariance and Correlation
Independence and Being Uncorrelated
I This implies ρXY = 0 so X and Y are uncorrelated.
I But remember we showed previously that X and Y
were not independent!
I Independence and being uncorrelated are not the
same thing.
I In fact, independence is a stronger condition than
being uncorrelated.
I Specifically, independence always implies a
correlation of zero, whereas being uncorrelated does
not always imply independence.
STAT7055 - Topic 3 43 / 63
Discrete Random Variables Linear Combinations
Linear Combination of Random Variables
I The quantity Z = aX + bY , where a and b are
constants, is called a linear combination of the
random variables X and Y .
I It can be shown that:
E(aX + bY ) = aE(X) + bE(Y )
V (aX + bY ) = a2 V (X) + b2 V (Y ) + 2ab × Cov(X, Y )
= a2 σ X
2
+ b2 σY2 + 2abρXY σX σY
STAT7055 - Topic 3 44 / 63
Discrete Random Variables Linear Combinations
Portfolio Diversification
I In finance, variance or standard deviation is often
used to assess the risk of an investment.
I Analysts reduce risk by diversifying their
investments - that is, combining investments where
the correlation is small.
STAT7055 - Topic 3 45 / 63
Discrete Random Variables Linear Combinations
Portfolio Diversification
I An investor forms a portfolio by putting 25% of his
money in stock A and 75% in stock B, with
population parameters given below.
Expected Standard
Value of Deviation of
Return Return
Stock A 8% 12%
Stock B 15% 22%
STAT7055 - Topic 3 46 / 63
Discrete Random Variables Linear Combinations
Expected Portfolio Return
I Let RA and RB denote the returns of stocks A and
B, respectively.
I If we let RP denote the return of the portfolio, then
we can write:
RP = 0.25RA + 0.75RB
I We are given that E(RA ) = 8 and E(RB ) = 15.
STAT7055 - Topic 3 47 / 63
Discrete Random Variables Linear Combinations
Expected Portfolio Return
I Therefore, the expected value of RP is:
E(RP ) = E(0.25RA + 0.75RB )
= 0.25 × E(RA ) + 0.75 × E(RB )
= 0.25 × 8 + 0.75 × 15
= 13.25
I That is, the expected portfolio return is 13.25%.
STAT7055 - Topic 3 48 / 63
Discrete Random Variables Linear Combinations
Variance of Portfolio Return
I Calculate the variance when the two stock returns
are perfectly positively correlated, i.e., ρAB = 1.
V (RP ) = 0.252 σA2 + 0.752 σB2
+ 2 × 0.25 × 0.75 × ρAB σA σB
= 0.252 × 122 + 0.752 × 222
+ 2 × 0.25 × 0.75 × ρAB × 12 × 22
= 281.25 + 99 × ρAB
= 281.25 + 99 × 1
= 380.25%2
STAT7055 - Topic 3 49 / 63
Discrete Random Variables Linear Combinations
Variance of Portfolio Return
I Calculate the variance when the two stock returns
are perfectly uncorrelated, i.e., ρAB = 0.
V (RP ) = 0.252 σA2 + 0.752 σB2
+ 2 × 0.25 × 0.75 × ρAB σA σB
= 0.252 × 122 + 0.752 × 222
+ 2 × 0.25 × 0.75 × ρAB × 12 × 22
= 281.25 + 99 × ρAB
= 281.25 + 99 × 0
= 281.25%2
STAT7055 - Topic 3 50 / 63
Discrete Random Variables Binomial Distribution
Bernoulli Trial
I A Bernoulli trial is a random experiment that has
the following special properties:
I On each trial there are only two possible outcomes,
which we call success and failure.
I On any given trial, the probability of a success is p and
the probability of a failure is 1 − p.
I The trials are independent - that is, the result of one
trial does not affect the result of any other trial.
STAT7055 - Topic 3 51 / 63
Discrete Random Variables Binomial Distribution
Binomial Distribution
I If a fixed number, n, of Bernoulli trials are
performed, the random variable representing the
number of successes in the n trials is called a
binomial random variable and its probability
distribution is called the binomial distribution.
I If X denotes a binomial random variable, then we
use the notation X ∼ Bin(n, p), where p is the
probability of success on any given trial.
STAT7055 - Topic 3 52 / 63
Discrete Random Variables Binomial Distribution
Some Examples
I Flip a coin ten times and let X be the number of
heads.
I X ∼ Bin(n = 10, p = 0.5).
I Pull a card from a deck, with replacement, eight
times and let X be the number of clubs.
I X ∼ Bin(n = 8, p = 0.25).
I Survey 1000 people and let X be the number of
people who think the current prime minister is doing
a good job.
I X ∼ Bin(n = 1000, p =?).
STAT7055 - Topic 3 53 / 63
Discrete Random Variables Binomial Distribution
Binomial Probability Distribution
I If X ∼ Bin(n, p) then the possible values that X
can take are 0, 1, 2, 3, . . . , n.
I The binomial probability distribution is given by
the following formula:
n!
P (X = x) = px (1 − p)n−x
x!(n − x)!
Note that n! = n × (n − 1) × (n − 2) × . . . × 2 × 1.
STAT7055 - Topic 3 54 / 63
Discrete Random Variables Binomial Distribution
Expected Value and Variance
I We could use the usual formula to calculate the
expected value:
X
E(X) = (x × p(x))
all x
n
X n!
= x× px (1 − p)n−x
x=0
x!(n − x)!
= ...
I And similarly for the variance.
I But we don’t really want to.
STAT7055 - Topic 3 55 / 63
Discrete Random Variables Binomial Distribution
Expected Value and Variance
I Instead, let’s define a new random variable for each
Bernoulli trial as follows:
(
1 if trial i is a success
Xi =
0 if trial i is a failure
I Each Xi is called a Bernoulli or indicator variable.
I We know that the Xi are independent and we also
know that n
X
X= Xi
i=1
STAT7055 - Topic 3 56 / 63
Discrete Random Variables Binomial Distribution
Expected Value and Variance
I Using the laws of expected value and variance:
n
! n n
X X X
E(X) = E Xi = E(Xi ) = p = np
i=1 i=1 i=1
n
! n n
X X X
V (X) = V Xi = V (Xi ) = p(1 − p) = np(1 − p)
i=1 i=1 i=1
STAT7055 - Topic 3 57 / 63
Discrete Random Variables Binomial Distribution
Example
I A student sitting a statistics quiz decides to answer
each of the ten multiple choice questions entirely by
chance.
I Each question has five options, only one of which is
correct.
I Let X be the number of questions the student
answers correctly.
I Then X ∼ Bin(n = 10, p = 0.2).
STAT7055 - Topic 3 58 / 63
Discrete Random Variables Binomial Distribution
Example
I What is the probability the student gets half the
answers correct?
10!
P (X = 5) = × 0.25 × (1 − 0.2)5 = 0.0264
5!(10 − 5)!
I What is the probability that the student passes, i.e.,
gets five or more correct?
P (X ≥ 5) = P (X = 5) + P (X = 6) + P (X = 7)
+ P (X = 8) + P (X = 9) + P (X = 10)
= a lot of calculations!
STAT7055 - Topic 3 59 / 63
Discrete Random Variables Binomial Distribution
Binomial Tables
I There are tables available that list P (X ≤ k) for
different values of k, n and p.
I See Appendix B of the textbook.
I From tables, look up n = 10 and p = 0.2.
P (X ≥ 5) = 1 − P (X ≤ 4)
= 1 − 0.9672 (from tables)
= 0.0328
STAT7055 - Topic 3 60 / 63
Discrete Random Variables Binomial Distribution
Binomial Tables
I What is the probability the student gets half the
answers correct?
P (X = 5) = P (X ≤ 5) − P (X ≤ 4)
= 0.9936 − 0.9672 (from tables)
= 0.0264
STAT7055 - Topic 3 61 / 63
Discrete Random Variables Binomial Distribution
Binomial Tables
I The binomial tables are a tool to make life easier by
helping us calculate binomial probabilities for
frequently used values of n and p.
I However, they are not a substitute for knowing and
being able to use the binomial probability
distribution formula - not all values of n or p will be
tabulated!
STAT7055 - Topic 3 62 / 63
Reference
Reference
I Keller 10e or 11e chapter 7.
STAT7055 - Topic 3 63 / 63