Joint Normal Random Variables

H. Krieger, Mathematics 156, Harvey Mudd College Fall, 2008

Properties: Characteristic functions have similar properties to other transforms, such as generating functions, although some of them are more dicult to prove than for the case of generating functions. For example: 1. The characteristic function of the sum of independent random variables is the product of the characteristic functions of each of the random variables. 2. The characteristic function uniquely determines the distribution. 3. Convergence in distribution is equivalent to pointwise convergence of the corresponding characteristic functions. 4. Change of location and scale: Suppose > 0 and R. Then ( ) FX+ (t) = E eit(X+) = eit FX (t). Examples: Here are a few characteristic functions that can be very useful. 1. Suppose X is Bernoulli with probability of success p. Then FX (t) = eit p + (1 p) = 1 p(1 eit ). 2. Consequently, if Y is binomial(n, p) then ]n [ ]n [ FY (t) = eit p + (1 p) = 1 p(1 eit ) . 3. Suppose X is Poisson with parameter > 0. Then
it it FX (t) = e ee = e(1e ) .

4. Suppose Z is standard normal, i.e. N (0, 1), so that Z has density

for x R. Then

2 FZ (t) = et /2 =

2(t).

5. Consequently, if X is normal(, 2 ), i.e, X Z + where Z N (0, 1), then 2 2 FX (t) = eit FZ (t) = eit e t /2 . Joint Characteristic Functions: If X = [X1 , X2 , . . . , Xn ]T is a random n-vector, then the characteristic function of X, or of the distribution of X or of the joint distribution of X1 , X2 , . . . , Xn , is dened for uT = [u1 , u2 , . . . , un ] Rn by ( T ) ( ) FX (u) = E eiu X = E ei(u1 X1 +u2 X2 ++un Xn ) . Properties: Many of the properties of joint characteristic functions are the same. For example: 1. The joint characteristic function of the sum of independent random nvectors is the product of the joint characteristic functions of each of the random n-vectors. 2. The joint characteristic function uniquely determines the joint distribution. 3. Convergence in distribution is equivalent to pointwise convergence of the corresponding joint characteristic functions. However, there is one new property, similar to that of joint generating functions, namely: the random variables X1 , X2 , . . . , Xn are independent if and only if the characteristic function of their joint distribution is the product of their individual characteristic functions. In other words, for all [u1 , u2 , . . . , un ] Rn ,
n n ( ) ( ) E eiuj Xj = FXj (uj ). FX (u) = E ei(u1 X1 +u2 X2 ++un Xn ) = j=1 j=1

Denition: The random n-vector X T = [X1 , X2 , . . . , Xn ] is (or the random variables X1 , X2 , . . . , Xn are) said to have a joint normal distribution if and only if the random variable u T X = u 1 X1 + u 2 X2 + + u n Xn is normally distributed for every uT = [u1 , u2 , . . . , un ] Rn . (Note that we consider constant random variables to be normally distributed with variance 0.) 2

Theorem:X1 , X2 , . . . , Xn have a joint normal distribution if and only if the characteristic function of their joint distribution is:
T T FX (u) = eiu eu u/2 ,

where is a vector (which is called the mean vector), with components j = E(Xj ), and is a non-negative denite symmetric matrix (which is called the 2 covariance matrix), with entries j,k = cov(Xj , Xk ). Proof: First suppose that X1 , X2 , . . . , Xn have a joint normal distribution. Then if uT = [u1 , u2 , . . . , un ] Rn , we know that the random variable V = u T X = u 1 X1 + u 2 X2 + + u n Xn has a normal distribution. Furthermore, E(uT X) = uT and V ar(uT X) = uT u. Consequently, ( ) T T T FX (u) = E ei(u X) = FV (1) = eiu eu u/2 . On the other hand, if the characteristic function of their joint distribution is as given and V = u T X = u 1 X1 + u 2 X2 + + u n Xn , then for t R the characteristic function of V must be ( ) T T T 2 FV (t) = E(eitV ) = E eit(u X) = FX (tu) = eit(u ) e(u u)t /2 . Thus, by the uniqueness of characteristic functions, V must be normally distributed (with mean uT and variance uT u). Theorem: X1 , X2 , . . . , Xn have a joint normal distribution with mean vector and non-singular (i.e. positive denite) covariance matrix if and only if X1 , X2 , . . . , Xn have a joint density function given for x = [x1 , x2 , . . . , xn ]T Rn by T 1 1 fX (x) = e(x) (x)/2 . (2)n/2 (det )1/2 Proof: Because of the uniqueness property of joint characteristic functions, it suces to show that for every uT = [u1 , u2 , ..., un ] Rn , we have T T T eiu x fX (x) dx = eiu eu u/2 . Note that if this is true, then by letting u = 0 Rn we get Rn fX (x) dx = 1. Since fX (x) 0 for all x Rn , we see that fX must be a joint density T function. In the integral, Rn eiu x fX (x) dx, we make a change of variable y = 1/2 (x ) or, equivalently, x = + 1/2 y. This is possible since the 3
Rn

positive denite symmetric matrix has a positive denite symmetric square root 1/2 with inverse which we denote by 1/2 . Note that the Jacobian of this transformation ( ) dx 1/2 = det 1/2 = (det ) . dy Consequently, T eiu x fX (x) dx
Rn

= =

1 (2)n/2 eiu (2)n/2

eiu
Rn

(+1/2 y) y T y/2

dy

eiu
Rn

1/2 y y T y/2

dy.

Letting v = 1/2 u, we see that uT 1/2 y = v T y so that eiu (2)n/2

eiu
Rn

1/2 y y T y/2

dy

= eiu

1 2 j=1
n j=1
2

eivj yj eyj /2 dyj

= eiu However, n
j=1

evj /2 .

2 vj = v T v = uT u, which gives the result.