Multivariate distribution
Introduction to Multivariate Distributions
A Multivariate distribution is a probability distribution that describes the behavior
of two or more random variables simultaneously. In contrast to univariate
distributions, which deal with a single variable, multivariate distributions allow us to
model the relationships and dependencies between multiple variables.
Multivariate distributions are essential for many real-world applications, as they allow
us to understand the joint behavior of multiple variables and capture the correlations
between them.
Key Concepts in Multivariate Distributions
Random Vectors: A multivariate distribution involves a random vector, which is a
vector of random variables. For instance, if you are modeling two variables, say X1
and X2, your random vector would be:
If a random vector X = (X₁, X₂, …, Xₖ) has a joint distribution describing all
variables together, then X follows a k-variate (multivariate) distribution.
A multivariate distribution includes:
1. Joint distribution (distribution of all variables together)
2. Marginal distributions (distribution of each variable)
3. Conditional distributions (distribution of one variable given another)
4. Covariance and correlation (measure of dependence)
1. Joint Probability Distribution: The joint probability distribution describes
the likelihood of different combinations of values for the variables. In a multivariate
case, it tells us the probability of observing a specific set of values across the multiple
variables.
For two variables, the joint distribution is expressed as P(X1,X2), while for more than
two variables, it becomes P(X1,X2,…,Xn)
Joint cumulative distribution function
Many sample observations (black) are shown from a joint probability distribution.
The marginal densities are shown as well (in blue and in red).
The generalization of the preceding two-variable case is the joint probability
distribution of n discrete random variables X 1 , X 2 , … , X n which is:
This identity is known as the chain rule of probability.
for n discrete random variables X 1 , X 2 , … , X n to
The joint probability density function to more than two continuous random
variables is defined as the derivative of the joint cumulative distribution function
Again, since these are probability distributions, one has
2. Marginal Distribution:
The marginal distribution of a subset of variables is obtained by integrating (or
summing, in the discrete case) the joint distribution over the other variables. For
example, the marginal distribution of X1 is:
Continous
Discrete
3. Conditional Distribution:
The conditional distribution describes the distribution of one subset of variables given
specific values of the other variables. For example, the conditional distribution
of X1 given X2=x2 is:
4. Covariance and Correlation:
In multivariate distributions, it's important to consider how the variables are related to
each other. Covariance measures the linear relationship between two random
variables: whether they tend to increase or decrease together. Correlation is a
standardized version of covariance that measures the strength of this relationship.
Covariance between Xi and Xj:
where μi and μj are the means of Xi and Xj, respectively.
Correlation is given by:
where σ1 and σ2 are the standard deviations of X1 and X2.
Correlation measures the strength and direction of the linear relationship between two
variables, normalized to the range [−1,1].
5. Independence:
Variables X1,X2,…,Xp are independent if their joint distribution factors into the
product of their marginal distributions:
6. Mean Vector and Covariance Matrix:
For a multivariate distribution, the mean vector represents the expected values of all
the variables, and the covariance matrix captures the covariances between the
variables. For two variables X1 and X2, this can be represented as:
Multivariate Normal Distribution:
The most widely used multivariate distribution, characterized by a mean vector μ and
a covariance matrix Σ. Its PDF is:
Properties (Statements Only)
1. Marginal distributions are Normal
X
Each variable i individually is normally distributed.
2. Conditional distributions are also Normal
3. Mean vector
E( X )=μ
4. Covariance matrix Σ is
o Symmetric
o Positive definite
o Diagonal entries = variances
o Off-diagonal entries = covariances
5. Linear combinations of variables are Normal
If Y =a X
1 1 + a X
2 2 +⋯+a k X k , then Y is normally distributed.
6. Contours of equal probability are ellipses/ellipsoids
(For 2 variables → ellipses, for 3 → ellipsoids)
Real-Life Applications of Multivariate Normal Distribution
1. Finance & Investment
● Modeling returns of multiple stocks.
● Used in portfolio optimization, risk analysis, and covariance estimation.
● Joint movement of stock prices often approximated as MVN.
2. Machine Learning & AI
● Used in:
o Gaussian Mixture Models (GMM)
o Naive Bayes (Gaussian version)
o Linear/Quadratic Discriminant Analysis (LDA/QDA)
● Feature vectors such as height–weight, pixel intensities, embeddings, etc.
3. Medical & Health Studies
● Studying correlated health indicators:
o Blood pressure (systolic & diastolic)
o Cholesterol levels
o Body mass index (BMI)
● Multivariate normal helps in diagnosis and risk prediction.
4. Meteorology & Environmental Science
● Modeling correlated weather variables:
o Temperature
o Humidity
o Air pressure
o Wind speed
● Useful for climate forecasting and environmental simulations.
5. Quality Control (Industrial Engineering)
● Hotelling’s T² multivariate control chart
uses MVN for monitoring multiple quality characteristics simultaneously.