To simulate a multivariate normal vector, we are essentially "coloring" white noise.
We start
with independent random numbers and use linear algebra to give them the specific mean and
correlation structure we want.
Here is the step-by-step breakdown of the logic:
1. Setup the Target Parameters
Before simulating, you must define the "target" shape of your data.
● Mean Vector (mu): This defines the center of your data distribution for each
dimension.
● Covariance Matrix (Sigma): This defines the spread (variance) of each variable and
how they move together (covariance).
2. Decomposition (The "Square Root" of the Matrix)
Since we are working with variances (which are squared units), we need the "square root" of
the covariance matrix to apply it to our random samples. We use Cholesky Decomposition:
Sigma = LLT
This produces L, a lower-triangular matrix. Think of L as a "recipe" that tells us how much of
each independent random input should be mixed into each final variable.
3. Generate Independent "White Noise"
We generate a vector Z consisting of independent standard normal variables:
Z ~ N(0, 1)
At this stage, the variables have zero correlation and unit variance. If you plotted them,
they would look like a perfect, circular cloud of points.
4. Induce Correlation (The Linear Transformation)
We multiply the Cholesky factor L by our independent vector Z:
Xuncentered= LZ
Because L contains the structure of our covariance matrix, this multiplication "stretches" and
"rotates" the circular cloud of points into an elliptical shape that matches our desired
correlations.
5. Shift to the Mean
Finally, we add the mean vector mu to the result:
X = mu + LZ
This shifts the entire "cloud" from being centered at the origin (0,0) to the desired location in
space.
PYTHON CODE
How many observations in samples?
The variable samples will contain 1,000 observations.
● Final Shape: The matrix will be 2 * 1000.
● Structure:
o 2 Rows: Row 1 contains the 1,000 simulated values for Variable X_1; Row 2
contains the 1,000 simulated values for Variable X_2.
o 1,000 Columns: Each column represents one "pair" of data points (x_1, x_2)
that make up a single observation.
Visualization of the dimensions:
Test of Mean Vectors: Hotelling’s 𝑇 2 Example
The following measurements were recorded for four subjects:
Observation Variable 1 (X1) Variable 2 (X2)
1 5.0 3.5
2 6.0 3.0
3 4.5 3.0
4 4.5 2.5
Pre-tests in multivariate settings
Test for Covariance Pattern (Equality of Variances)
Box’s M Test is a critical "gatekeeper" test. If this test is significant (p < 0.05),
it means the covariance matrices of your two groups are too different, and using
a standard Hotelling’s T2 might lead to incorrect results.
Test for Correlation Pattern (Bartlett's Sphericity)
Imagine we measure Heart Rate (y1) and Breathing Rate (y2) for 3 people:
Subject y1 y2
1 2 4
2 4 8
3 6 12
Multivariate Linear Regression
MANOVA (Multivariate Analysis of Variance)
Let's assume the mean scores for each diet group are:
Weight Loss Blood Sugar Group Means
Diet Group Person
(y1) (y2) (y̅1 ,y
̅̅̅)
2
Low Carb (G1) 1, 2, 3 9, 8, 7 6, 5, 4 (8, 5)
Low Fat (G2) 4, 5, 6 5, 4, 3 3, 2, 1 (4, 2)
Mediterranean (G3) 7, 8, 9 7, 6, 5 9, 8, 7 (6, 8)
Calculate the Hypothesis Matrix (H)
Calculate the Error Matrix (E)
Compute Wilks' Lambda