The Probability Integral Transformation (PIT) is a fundamental concept in
probability theory and simulation. It is based on the following principle:
If X is a continuous random variable with cumulative distribution function
(CDF) FX (x), then the transformed variable U = FX (X) follows a uniform
distribution on the interval [0, 1], that is, U ∼ Uniform(0, 1).
Conversely, the inverse transform method allows us to generate samples from any
continuous distribution using uniform random numbers:
If U ∼ Uniform(0, 1), and F −1 is the inverse of the CDF of a continuous
distribution, then the transformed variable X = F −1 (U ) follows the original
distribution with CDF F , i.e., X ∼ F .
This property provides a general method for generating random numbers from any
continuous distribution with a known inverse CDF.
Example: Exponential Distribution
Let X follow an exponential distribution with rate parameter λ > 0.
The CDF of the exponential distribution is:
F (x) = 1 − e−λx , x≥0
Solving for x in terms of u gives the inverse CDF:
1
F −1 (u) = − ln(1 − u)
λ
Since U ∼ Uniform(0, 1), and 1 − U ∼ Uniform(0, 1), this simplifies to:
1
X = − ln(U )
λ
Using this transformation, we can generate samples from the exponential distribution
by:
1. Generating uniform random numbers U1 , U2 , . . . , Un ∼ Uniform(0, 1)
2. Computing Xi = − λ1 ln(Ui ) for each i
This process can be applied to other continuous distributions, as long as the inverse
CDF F −1 (u) is known or can be approximated.
1
Theorem: Sampling Distribution of the Sample Mean
Let X1 , X2 , . . . , Xn be a random sample of size n drawn from a population with
Pnmean
2 1
µ and variance σ . Then the sampling distribution of the sample mean X̄ = n i=1 Xi
has:
Mean: E[X̄] = µ
σ2
Variance: Var(X̄) = n
As n increases, the sampling distribution of X̄ tends to a normal distribution due to
the Central Limit Theorem (CLT), regardless of the original population distribution.
Special Case: Sampling from a Normal Distribution
If the population is normally distributed, i.e., Xi ∼ N (µ, σ 2 ), then for any sample size n,
the sampling distribution of the sample mean X̄ is also normally distributed:
σ2
X̄ ∼ N µ,
n
Procedure: Simulating the Sampling Distribution of
the Sample Mean
To empirically study the sampling distribution of the sample mean, follow the steps below:
1. Choose a distribution: Select a population distribution (e.g., Normal, Binomial,
Poisson, Exponential, Gamma) with known parameters (mean µ and standard de-
viation σ).
2. Fix parameters:
Sample size: n
Number of repetitions (samples): k
3. Repeat the following process k times:
(a) Draw a random sample of size n from the chosen distribution.
(b) Compute the sample mean X̄i for the i-th sample.
4. Collect all sample means: This results in a set of k sample means: X̄1 , X̄2 , . . . , X̄k .
5. Compute the mean and variance of the sample means:
k
1X
Empirical Mean of Sample Means = X̄i
k i=1
k
1 X 2
Empirical Variance of Sample Means = X̄i − X̄
k − 1 i=1
2
6. Compare the empirical values with theoretical expectations:
Theoretical Mean: µ
Theoretical Variance: σ 2 /n
7. Visualize the results: Plot a histogram or density plot of the sample means to
observe the shape of the sampling distribution.
Illustrative Example (in R)
Consider a population with a normal distribution, µ = 100, and σ = 15. We take repeated
samples of size n = 30 and compute the sample means.
R Code
[Link](123)
n <- 30
num_samples <- 1000
mu <- 100
sigma <- 15
sample_means <- replicate(num_samples, {
sample <- rnorm(n, mean = mu, sd = sigma)
mean(sample)
})
mean(sample_means) # Empirical mean
var(sample_means) # Empirical variance
Expected Results
Theoretical mean of sampling distribution: µ = 100
Theoretical variance: σ 2 /n = 152 /30 = 7.5
Empirical mean and variance from R simulation should approximate these values.
Practice Questions
Binomial Distribution
1. What is the expected value and variance of the sampling distribution of the mean
for a Binomial(n = 10, p = 0.4) population?
2. Simulate the sampling distribution of the mean for Binomial(n = 10, p = 0.8) and
observe how skewness changes with p.
3
Poisson Distribution
1. If X ∼ Poisson(λ = 4), simulate the sampling distribution of the mean for sample
size n = 50. Compare theoretical and empirical mean and variance.
2. How does the distribution of sample means change as λ increases?
Exponential Distribution
1. For X ∼ Exponential(λ = 1.5), simulate sample means and check normality for
n = 10 and n = 50.
2. Does the Central Limit Theorem apply here? How does sample size affect the
result?
Normal Distribution
1. Use X ∼ N (70, 102 ). Simulate the sampling distribution of the mean for different
values of n and confirm that it remains normal.
Gamma Distribution
1. Take X ∼ Gamma(shape = 2, rate = 1). Simulate and compare the sampling
distribution for n = 10 and n = 100.
2. How does the shape of the original distribution influence the sample mean distri-
bution?