Tutorial - 3: Distribution of General Functions of
Random Variables
STATS 3J04
October 21, 2024
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 1 / 18
Outline
1 Introduction
2 CDF Method
3 PDF Method (Change of Variables)
4 Non-Invertible Transformations
5 Example: Y = X 2
6 PDF Calculation for Y = X 2
7 Discrete Random Variable Case
8 Example: Y = X 2 (Discrete Case)
9 PMF Calculation for Y = X 2 (Discrete Case)
10 Summary of Non-Invertible Functions
11 Example: Exponential to Uniform
12 Multivariate Case
13 Summary
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 2 / 18
Introduction
Given a random variable X with known distribution, we often need
the distribution of a function of X , say Y = g (X ).
This is a common problem in probability and statistics, often needed
for transformations, models, and simulations.
The key techniques for deriving the distribution of Y involve the
cumulative distribution function (CDF) and the probability density
function (PDF).
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 3 / 18
CDF Method
The CDF method is one of the primary ways to find the distribution
of Y = g (X ).
The CDF of Y is given by:
FY (y ) = P(Y ≤ y ) = P(g (X ) ≤ y ).
Once we have FY (y ), the PDF of Y is simply the derivative:
d
fY (y ) = FY (y ).
dy
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 4 / 18
PDF Method (Change of Variables)
If g (X ) is a monotonic function, the PDF of Y = g (X ) can be
obtained using the change of variables formula:
dx
fY (y ) = fX (x) ,
dy
where x = g −1 (y ).
This method works well for both discrete and continuous random
variables, as long as the transformation is invertible/one-to-one.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 5 / 18
Non-Invertible Transformations
For non-invertible functions like Y = X 2 , multiple values of X can
map to the same value of Y .
√ √
Example: For Y = X 2 , both X = y and X = − y map to the
same Y = y .
We need to account for all possible values of X that satisfy
Y = g (X ).
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 6 / 18
Finding the PDF of Y = X 2 , where X ∼ U(0, 1)
Step 1: Define the transformation
2
Y =X , where X ∼ U(0, 1).
Step 2: Invert the transformation (Notice: we will consider only positive contribution as 0 < x < 1)
√
X = Y.
Step 3: Find the derivative
dx d √ 1
= ( y) = √ .
dy dy 2 y
Step 4: Use the change of variables formula
dx
fY (y ) = fX (x)
dy
1 1
=1· √ = √ , 0 < y < 1.
2 y 2 y
Final PDF of Y :
1
fY (y ) = √ , 0 < y < 1.
2 y
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 7 / 18
Example: Y = X 2
Suppose X ∼ N (0, 1), i.e., X follows a standard normal distribution.
Let Y = X 2 , which is a non-invertible transformation.
√ √
We must consider both branches: X = y and X = − y .
Thus, the PDF of Y is:
√ d √ √ d √
fY (y ) = fX ( y ) y + fX (− y ) (− y ) .
dy dy
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 8 / 18
PDF Calculation for Y = X 2
Since X ∼ N (0, 1), the PDF of X is:
1 2
fX (x) = √ e −x /2 .
2π
For Y = X 2 , the PDF is:
1
fY (y ) = √ e −y /2 for y ≥ 0.
2πy
This is the PDF of a chi-squared distribution with 1 degree of
freedom.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 9 / 18
Discrete Random Variable Case
Now, let’s consider a discrete random variable X and the
non-invertible transformation Y = X 2 .
The process for finding the distribution of Y is similar but involves
summing the probabilities for each X that satisfies Y = X 2 .
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 10 / 18
Example: Y = X 2 (Discrete Case)
Suppose X is a discrete random variable with the following PMF:
P(X = −2) = 0.2, P(X = −1) = 0.3, P(X = 1) = 0.3, P(X = 2) = 0.2.
Let Y = X 2 . Then the possible values of Y are 0, 1, 4.
We compute the PMF of Y by summing the probabilities for all X values
that satisfy Y = X 2 .
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 11 / 18
PMF Calculation for Y = X 2 (Discrete Case)
For Y = 1, both X = 1 and X = −1 satisfy Y = X 2 .
The PMF of Y is given by:
P(Y = 1) = P(X = 1) + P(X = −1) = 0.3 + 0.3 = 0.6.
For Y = 4, both X = 2 and X = −2 satisfy Y = X 2 .
Thus,
P(Y = 4) = P(X = 2) + P(X = −2) = 0.2 + 0.2 = 0.4.
Therefore, the PMF of Y is:
P(Y = 1) = 0.6, P(Y = 4) = 0.4.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 12 / 18
Finding the pmf of Y = (X − 2)2
Given: X ∼ Binomial(5, 1/4)
X takes values in {0, 1, 2, 3, 4, 5}
Transformation: Y = (X − 2)2
The transformation Y = (X − 2)2 maps the values of X to the
corresponding values of Y :
X =0 ⇒ Y = (0 − 2)2 = 4,
X =1 ⇒ Y = (1 − 2)2 = 1,
X =2 ⇒ Y = (2 − 2)2 = 0,
X =3 ⇒ Y = (3 − 2)2 = 1,
X =4 ⇒ Y = (4 − 2)2 = 4,
X =5 ⇒ Y = (5 − 2)2 = 9.
Possible values of Y : {0, 1, 4, 9}
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 13 / 18
Frame Title
Pmfs:
P(Y = 0) = P(X = 2) = 0.2637,
P(Y = 1) = P(X = 1) + P(X = 3) = 0.4834,
P(Y = 4) = P(X = 0) + P(X = 4) = 0.2519,
P(Y = 9) = P(X = 5) = 0.00098.
Final pmf for Y :
0.2637 if y = 0,
0.4834 if y = 1,
P(Y = y ) =
0.2519 if y = 4,
0.00098 if y = 9.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 14 / 18
Summary of Non-Invertible Functions
For non-invertible transformations, we must consider all possible
pre-images of Y under g (X ).
The PDF is computed by summing the contributions from each
branch of the transformation.
√ √
In the example Y = X 2 , we considered both X = y and X = − y
when X takes values in the real line. We should only consider
√
X = y if X > 0.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 15 / 18
Example: Exponential to Uniform
Suppose X ∼ Exp(λ), with PDF fX (x) = λe −λx for x ≥ 0.
Let Y = e −λX , which is a function of X .
Find the distribution of Y :
FY (y ) = P(Y ≤ y ) = P(e −λX ≤ y )
1
= P(X ≥ − log y )
λ
1
= 1 − FX − log y = y .
λ
Thus, Y ∼ Uniform(0, 1).
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 16 / 18
Multivariate Case
For multivariate random variables, the Jacobian determinant is used
when transforming from one set of variables to another.
If Y = g (X1 , X2 , . . . , Xn ), the joint PDF of Y1 , Y2 , . . . , Yn is given by:
fY (y1 , y2 , . . . , yn ) = fX (x1 , x2 , . . . , xn ) |det(J)| ,
where J is the Jacobian matrix of the transformation.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 17 / 18
Summary
We can find the distribution of a function of a random variable using
the CDF method or the PDF method (change of variables).
For multivariate transformations, the Jacobian determinant plays a
key role.
Understanding these methods is essential in many areas of statistics
and probability, particularly in the fields of simulation, econometrics,
and machine learning.
STATS 3J04 Tutorial - 3: Distribution of General Functions of Random
October
Variables
21, 2024 18 / 18