General Random Variables

Basic concepts
• Random variables with a continuous range of
possible values
– the velocity of a vehicle traveling along the
highway
 Continuous Random Variables and
PDFs
• A random variable X is called continuous if its probability law can
be described in terms of a nonnegative function 𝑓𝑋 , called the
probability density function (PDF) of X, which satisfies
𝑃 𝑋 ∈ 𝐵 = න 𝑓𝑋 𝑥 𝑑𝑥
𝐵
• The probability that the value of X falls within an interval is
𝑏
𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = න 𝑓𝑋 𝑥 𝑑𝑥
𝑎
– It is the area under the graph of the PDF

• For any single value a, we have

𝑎
𝑃 𝑋 = 𝑎 = න 𝑓𝑋 𝑥 𝑑𝑥 = 0
𝑎
– So, 𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = 𝑃 𝑎 < 𝑋 < 𝑏 = 𝑃 𝑎 ≤ 𝑋 < 𝑏 =
𝑃 𝑎<𝑋≤𝑏
• Properties of PDF 𝑓𝑋 :
– 𝑓𝑋 (𝑥) ≥ 0 for every x
∞
– 𝑃 −∞ < 𝑋 < ∞ = ‫׬‬−∞ 𝑓𝑋 𝑥 𝑑𝑥 = 1 (normalization)
• For an interval [x, x + δ] with very small length δ,
𝑥+𝛿
𝑃 [𝑥, 𝑥 + 𝛿] = න 𝑓𝑋 𝑡 𝑑𝑡 ≈ 𝑓𝑋 𝑥 . 𝛿
𝑥
• 𝑓𝑋 𝑥 can be viewed as the “probability mass per unit
length” near 𝑥.

• If 𝛿 is very small, the

probability that 𝑋 takes value
in the interval [𝑥, 𝑥 + 𝛿] is
the shaded area in the figure,
which is approximately equal
to 𝑓𝑋 𝑥 . 𝛿.
 Uniform Random Variable
• Example
– We can consider a random variable X that takes
values in an interval [a, b], and also assume that all
subintervals of the same length are equally likely.
– The PDF is
𝑐 𝑖𝑓 𝑎 ≤ 𝑥 ≤ 𝑏,
𝑓𝑋 𝑥 = ቊ
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
Here, c is constant
• For 𝑓𝑋 to satisfy the normalization property, we
must have
𝑏
න 𝑓𝑋 𝑥 𝑑𝑥 = 1
𝑎
⇒𝑐 𝑏−𝑎 =1
1
⇒𝑐=
𝑏−𝑎
 Mean and Variance of the Uniform
Random Variable
• Uniform PDF over an interval [a, b]
1
𝑓𝑋 𝑥 = ቐ𝑏 − 𝑎 𝑖𝑓 𝑎 ≤ 𝑥 ≤ 𝑏,
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
+∞
– Mean 𝐸 𝑋 = ‫׬‬−∞ 𝑥𝑓𝑋 𝑥 𝑑𝑥
𝑏
1 𝑎+𝑏
= න 𝑥 𝑑𝑥 =
𝑏−𝑎 2
𝑎
+∞ 1 𝑏 2
– 𝐸 𝑋2 = ‫׬‬−∞ 𝑥 2 𝑓𝑋 𝑥 𝑑𝑥 = ‫׬‬
𝑏−𝑎 𝑎
𝑥 𝑑𝑥
𝑎 + 𝑎𝑏 + 𝑏2
2
=
3
– 𝑣𝑎𝑟 𝑋 = 𝐸 𝑋 2 − 𝐸 𝑋 2
2
𝑎2 + 𝑎𝑏 + 𝑏2 𝑎+𝑏
= −
3 2
2
𝑏−𝑎
=
12
 Exponential Random Variable
• An exponential random variable has a PDF of
the form
−𝜆𝑥
𝑓𝑋 𝑥 = ቊ 𝜆𝑒 𝑖𝑓 𝑥 ≥ 0,
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
where λ is a positive parameter characterizing the PDF
• For any a ≥ 0, we have
∞
𝑃 𝑋 ≥ 𝑎 = න 𝜆𝑒 −𝜆𝑥 𝑑𝑥 = 𝑒 −𝜆𝑎
𝑎
∞ ∞
• ‫׬‬−∞ 𝑓𝑋 𝑥 𝑑𝑥 = ‫׬‬0 𝜆𝑒 −𝜆𝑥 𝑑𝑥 = 1 (normalization)
• An exponential random
variable can be a very
good model
– for the amount of time
until a piece of equipment
breaks down
– for the amount of time
until a light bulb burns out
– for the amount of time
until an accident occurs
• The mean
1
𝐸𝑋 =
𝜆
• The variance
1
𝑣𝑎𝑟 𝑋 = 2
𝜆
 Cumulative distribution function
• We have been dealing with discrete and continuous random variables in
a somewhat different manner, using PMFs and PDFs, respectively.
• Is it possible to describe all kinds of random variables with a single
mathematical concept?
– It can be done by cumulative distribution function (CDF).
• The CDF of a random variable 𝑋 is denoted by 𝐹𝑋 and provides the
probability 𝑃(𝑋 ≤ 𝑥)
• For every x we have
෍ 𝑝𝑋 𝑘 𝑋: 𝑑𝑖𝑠𝑐𝑟𝑒𝑡𝑒
𝑘≤𝑥
𝐹𝑋 𝑥 = 𝑃 𝑋 ≤ 𝑥 = 𝑥

න 𝑓𝑋 𝑡 𝑑𝑡 𝑋: 𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑜𝑢𝑠
−∞
• So, the CDF 𝐹𝑋 (𝑥) “accumulates” probability “up to” the value 𝑥.
CDFs of some discrete random variables.
CDFs of some continuous random variables.
 Properties of a CDF
• The CDF 𝐹𝑋 of a random variable 𝑋 is defined by
𝐹𝑋 𝑥 = 𝑃(𝑋 ≤ 𝑥), for all x, and has the
following properties.
– 𝐹𝑋 is monotonically nondecreasing:
If 𝑥 ≤ 𝑦, then 𝐹𝑋 𝑥 ≤ 𝐹𝑋 𝑦
– 𝐹𝑋 𝑥 tends to 0 as 𝑥 → −∞, and to 1 as 𝑥 → +∞.
– If X is discrete, then 𝐹𝑋 has a piecewise constant and
staircase-like form.
– If X is continuous, then 𝐹𝑋 has a continuously varying
form.
– If X is discrete and takes integer values, the PMF and the
CDF can be obtained from each other by summing or
differencing:
𝑘

𝐹𝑋 𝑘 = ෍ 𝑝𝑋 (𝑖)
𝑖=−∞
𝑝𝑋 𝑘 = 𝑃 𝑋 ≤ 𝑘 − 𝑃 𝑋 ≤ 𝑘 − 1
= 𝐹𝑋 𝑘 − 𝐹𝑋 𝑘 − 1 for all integers k
– If X is continuous, the PDF and the CDF can be obtained
from each other by integration or differentiation:
𝑥
• 𝐹𝑋 𝑥 = ‫׬‬−∞ 𝑓𝑋 𝑡 𝑑𝑡
𝑑𝐹𝑋 𝑥
• 𝑓𝑋 𝑥 =
𝑑𝑥
 Normal Random Variable
• A continuous random variable X is said to be normal or Gaussian if it has
a PDF of the form
1 𝑥−𝜇 2
−
𝑓𝑋 𝑥 = 𝑒 2𝜎2
2𝜋 𝜎
where μ and σ are two scalar parameters characterizing the PDF, with σ
assumed nonnegative.
• Here also, the normalization property holds
∞ ∞
1 𝑥−𝜇 2 1 𝑦2
− −2
න 𝑒 2𝜎2 𝑑𝑥 = න 𝑒 𝑑𝑦 = 1
2𝜋 𝜎 2𝜋
−∞ −∞

• The mean
𝐸 𝑋 =𝜇
• The variance
𝑣𝑎𝑟 𝑋 = 𝜎 2
 Normality is Preserved by Linear
Transformations
• If 𝑋 is a normal random variable with mean
𝜇 and variance 𝜎 2 , and if 𝑎, 𝑏 are scalars,
then the random variable
𝑌 = 𝑎𝑋 + 𝑏
is also normal, with mean and variance
𝐸 𝑌 = 𝑎𝜇 + 𝑏, 𝑣𝑎𝑟 𝑌 = 𝑎2 𝜎 2
 The Standard Normal Random
Variable
• A normal random variable 𝑌 with zero mean
and unit variance is said to be a standard
normal. Its PDF is defined as
1 𝑥2
−2
𝑓𝑋 𝑥 = 𝑒
2𝜋
• CDF of standard normal is denoted by Φ,
𝑦
1 𝑥2
Φ y =P Y≤y = න 𝑒− 2 𝑑𝑥
2𝜋
−∞
• Example:
Φ −0.5 = 𝑃 𝑌 ≤ −0.5
= 𝑃 𝑌 ≥ 0.5 = 1 − 𝑃 𝑌 < 0.5
= 1 − Φ 0.5 = 1 − 0.6915 = 0.3085
• Let 𝑋 be a normal random variable with mean 𝜇 and variance 𝜎 2 .
• We “standardize” 𝑋 by defining a new random variable 𝑌 given
by
𝑋−𝜇
𝑌=
𝜎
– Y is also normal, as it is linear transformation of X.

• Also, 𝐸 𝑌 = 0, and 𝑣𝑎𝑟 𝑌 = 1.

• 𝑌 is standard normal random variable.

• Determine the following standard normal probabilities

– 𝑃 𝑌 ≤ 1.25 , 𝑃 𝑌 ≤ −1.25 , 𝑎𝑛𝑑 𝑃(0.38 ≤ 𝑌 ≤ 1.25)
• Example:
The annual snowfall at a particular geographic
location is modeled as a normal random variable
with a mean of μ = 60 inches, and a standard
deviation of σ = 20. What is the probability that this
year’s snowfall will be at least 80 inches?

• Ans: 0.1587
• Example:
The time that it takes a driver to react to the brake lights on
a decelerating vehicle is critical in helping to avoid rear-end
collisions. The article “Fast-Rise Brake Lamp as a Collision-
Prevention Device” (Ergonomics, 1993: 391–395) suggests
that reaction time for an in-traffic response to a brake signal
from standard brake lights can be modeled with a normal
distribution having mean value 1.25 sec and standard
deviation of .46 sec. What is the probability that reaction
time is between 1.00 sec and 1.75 sec?

• Ans: 0.5675
• Example:
The distribution of resistance for resistors of a
certain type is known to be normal, with 10% of all
resistors having a resistance exceeding 10.256
ohms and 5% having a resistance smaller than
9.671 ohms. What are the mean value and
standard deviation of the resistance distribution?
 Conditioning of an event
• The conditional PDF of a continuous random variable 𝑋, conditioned on
a particular event 𝐴 with 𝑃(𝐴) > 0, is a function 𝑓𝑋|𝐴 that satisfies
𝑃 𝑋 ∈ 𝐵 𝐴 = න 𝑓𝑋|𝐴 𝑥 𝑑𝑥
𝐵
for any subset B of the real line.
• If we condition on X belonging to a subset A of the real line, with 𝑃(𝑋 ∈
𝐴) > 0, we have
𝑃(𝑋 ∈ 𝐵 𝑎𝑛𝑑 𝑋 ∈ 𝐴) ‫𝑥𝑑 𝑥 𝑋𝑓 𝐵∩𝐴׬‬
𝑃 𝑋∈𝐵𝑋∈𝐴 = =
𝑃(𝑋 ∈ 𝐴) 𝑃(𝑋 ∈ 𝐴)
• So, If A be a subset of the real line with P(X ∈ A) > 0, the PDF
𝑓𝑋 (𝑥)
, 𝑖𝑓 𝑥 ∈ 𝐴,
𝑓𝑋|𝐴 𝑥 𝐴 = 𝑓𝑋|𝐴 (𝑥) = ൞𝑃(𝑋 ∈ 𝐴)
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
• The conditional PDF is zero
outside the conditioning set.

• Within the conditioning set,

the conditional PDF has
exactly the same shape as the
unconditional one, except that
it is scaled by the constant
factor 1/𝑃(𝑋 ∈ 𝐴).
• Conditional expectation
∞

𝐸 𝑋 𝐴 = න 𝑥𝑓𝑋|𝐴 𝑥 𝑑𝑥
−∞
• If 𝐴1 , 𝐴2 , ⋯ , 𝐴𝑛 are disjoint events with 𝑃 𝐴𝑖 > 0 for each 𝑖,
that form a partition of the sample space, then
𝑛

𝑓𝑋 𝑥 = ෍ 𝑃 𝐴𝑖 𝑓𝑋|𝐴𝑖 (𝑥)
𝑖=1
𝐸𝑋 = σ𝑛𝑖=1 𝑃 𝐴𝑖 𝐸 𝑋 𝐴𝑖 (Total expectation theorem)

• Note: The total expectation theorem can often facilitate the

calculation of the mean, variance, and other moments of a
random variable, using a divide-and-conquer approach.
 Mean and Variance of a Piecewise
Constant PDF
• Example
Suppose that the random variable X has the piecewise constant PDF
1
, 𝑖𝑓 0 ≤ 𝑥 ≤ 1
3
𝑓𝑋 𝑥 = 2
, 𝑖𝑓 1 ≤ 𝑥 ≤ 2
3
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Consider the events
𝐴1 = {𝑋 𝑙𝑖𝑒𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 [0, 1]},
𝐴2 = {𝑋 𝑙𝑖𝑒𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑠𝑒𝑐𝑜𝑛𝑑 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 [1, 2]},

Calculate E[X] and var(X).

Answer:
E[X]=7/6
Var(X)=11/36
 Multiple continuous random variable
• Two continuous random variables associated with a common
experiment are jointly continuous and can be described in terms of a
joint PDF 𝑓𝑋,𝑌 , if 𝑓𝑋,𝑌 is a nonnegative function that satisfies
𝑃 𝑋, 𝑌 ∈ 𝐵 = ඵ 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥𝑑𝑦
(𝑥,𝑦)∈𝐵
for every subset B of the two-dimensional plane.
• In the particular case where B is a rectangle of the form B = [a, b] × [c,
d], we have
𝑑 𝑏

𝑃 𝑎 ≤ 𝑥 ≤ 𝑏, 𝑐 ≤ 𝑌 ≤ 𝑑 = න න 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥𝑑𝑦
𝑐 𝑎
• Normalization property:
∞ ∞

න න 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥𝑑𝑦 = 1
−∞ −∞
• To interpret the PDF, we let δ be very small and consider the probability of a
small rectangle.
𝑐+𝛿 𝑎+𝛿

𝑃 𝑎 ≤ 𝑥 ≤ 𝑎 + 𝛿, 𝑐 ≤ 𝑌 ≤ 𝑐 + 𝛿 = න න 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥𝑑𝑦 ≈ 𝑓𝑋,𝑌 𝑎, 𝑐 . 𝛿 2

𝑐 𝑎

• We can view 𝑓𝑋,𝑌 𝑎, 𝑐 as the “probability per unit area” in the vicinity of (a, c).

• Marginal PDF 𝑓𝑋 of 𝑋 is given by

∞

𝑓𝑋 𝑥 = න 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑦
−∞

• Marginal PDF 𝑓𝑌 of 𝑌 is given by

∞

𝑓𝑌 𝑦 = න 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥
−∞
• Consider a subset S in a two-dimensional plane.
• The corresponding uniform joint PDF on S is defined by
1
, 𝑖𝑓 (𝑥, 𝑦) ∈ 𝑆
𝑓𝑋,𝑌 𝑥, 𝑦 = ቐ𝑎𝑟𝑒𝑎 𝑜𝑓 𝑆
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
• For any set A ⊂ S, the probability that the experimental
value of (X, Y ) lies in A is
𝑃 𝑋, 𝑌 ∈ 𝐴 = ඵ 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥𝑑𝑦
(𝑥,𝑦)∈𝐴
1 𝑎𝑟𝑒𝑎 𝑜𝑓 𝐴 ∩ 𝑆
= ඵ 𝑑𝑥𝑑𝑦 =
𝑎𝑟𝑒𝑎 𝑜𝑓 𝑆 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑆
(𝑥,𝑦)∈𝐴∩𝑆
• Example
the joint PDF of the random
variables X and Y is a
constant c on the set S
shown in the figure and is
zero outside.

Find the value of c and the

marginal PDFs of X and Y .
Conditioning One Random Variable on
Another
• Let X and Y be continuous random variables
with joint PDF 𝑓𝑋,𝑌 .
• For any fixed y with 𝑓𝑌 𝑦 > 0, the conditional
PDF of X given that Y = y, is defined by
𝑓𝑋,𝑌 𝑥, 𝑦
𝑓𝑋|𝑌 𝑥|𝑦 =
𝑓𝑌 𝑦
• When thinking about the conditional PDF, it is
best to view y as a fixed number and consider
𝑓𝑋|𝑌 𝑥|𝑦 as a function of the single variable x.
• Example
Joint PDF of X and Y is given by
𝑓𝑋,𝑌 𝑥, 𝑦
6𝑥𝑦 2 − 𝑥 − 𝑦 , 0 < 𝑥 < 1, 0 < 𝑦 < 1
=ቊ
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Find marginal density of Y, 𝑓𝑌 𝑦 and
conditional PDF 𝑓𝑋|𝑌 𝑥|𝑦 .
 Expectation
• If X and Y are jointly continuous random variables, and g is some
function, then Z = g(X, Y ) is also a random variable.
∞ ∞

𝐸 𝑔 𝑋, 𝑌 = න න 𝑔 𝑥, 𝑦 𝑓𝑋,𝑌 𝑥, 𝑦 𝑑𝑥 𝑑𝑦
−∞ −∞
• For any scalars a and b,
𝐸 𝑎𝑋 + 𝑏𝑌 = 𝑎𝐸 𝑋 + 𝑏𝐸[𝑌]
• Having defined a conditional probability law, we can also define a
corresponding conditional expectation as
∞

𝐸 𝑋|𝑌 = 𝑦 = න 𝑥 𝑓𝑋|𝑌 𝑥|𝑦 𝑑𝑥

−∞
• Also,
∞

𝐸 𝑔(𝑋)|𝑌 = 𝑦 = න 𝑔(𝑥) 𝑓𝑋|𝑌 𝑥|𝑦 𝑑𝑥

−∞
• Example
Joint PDF of X and Y is given by
6𝑥𝑦 2 − 𝑥 − 𝑦 , 0 < 𝑥 < 1, 0 < 𝑦 < 1
𝑓𝑋,𝑌 𝑥, 𝑦 = ቊ
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
1 1
Find 𝑃(0 < 𝑥 < 2 , 0 < 𝑦 < 4) and 𝐸 𝑋|𝑌 = 𝑦 .

Answer:
6𝑥(2 − 𝑥 − 𝑦)
𝑓𝑋|𝑌 𝑥|𝑦 =
4 − 3𝑦
1 1
𝑃 0 < 𝑥 < ,0 < 𝑦 < =?
2 4

1 3
𝐸 𝑋𝑌=2 =
5
• A bank operates both a drive-up facility and a walk-up window.
On a randomly selected day, let X = the proportion of time that
the drive-up facility is in use (at least one customer is being
served or waiting to be served) and Y = the proportion of time
that the walk-up window is in use. Then the set of possible values
for (X, Y ) is the rectangle
𝐷 = 𝑥, 𝑦 : 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 1 .
Suppose the joint pdf of (X, Y) is given by
6 2 ,
𝑓𝑋,𝑌 𝑥, 𝑦 = ቐ5 𝑥 + 𝑦 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 1
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Verify that this is legitimate pdf.
Find the marginal pdf of X and Y.
 Independence
• Two continuous random variables X and Y
are independent if their joint PDF is the
product of the marginal PDFs:
𝑓𝑋,𝑌 𝑥, 𝑦 = 𝑓𝑋 𝑥 𝑓𝑌 𝑦 , 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑥, 𝑦.
 Joint CDFs
• If X and Y are two random variables associated
with the same experiment, we define their joint
CDF by
𝑥 𝑦

𝐹𝑋,𝑌 𝑥, 𝑦 = 𝑃 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 = න න 𝑓𝑋,𝑌 𝑠, 𝑡 𝑑𝑠 𝑑𝑡
−∞ −∞
• The joint PDF can be recovered from the joint
CDF by differentiating:
𝜕 2 𝐹𝑋,𝑌 𝑥, 𝑦
𝑓𝑋,𝑌 𝑥, 𝑦 =
𝜕𝑥𝜕𝑦
• Example
Let X and Y be described by a uniform PDF on the
unit square for 0 ≤ x, y ≤ 1.
The joint CDF is given by
𝐹𝑋,𝑌 𝑥, 𝑦 = 𝑃 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦
𝑥 𝑦 𝑥 𝑦

= න න 𝑓𝑋,𝑌 𝑠, 𝑡 𝑑𝑠 𝑑𝑡 = න න 1 𝑑𝑠 𝑑𝑡 = 𝑥𝑦
−∞ −∞ 0 0
𝜕2 𝐹𝑋,𝑌 𝑥,𝑦 𝜕2 (𝑥𝑦)
Now, 𝑓𝑋,𝑌 𝑥, 𝑦 = = =1
𝜕𝑥𝜕𝑦 𝜕𝑥𝜕𝑦