Multivariate Normal Distribution

A
•

Def : n -
dimension random vector ☒ =
( Xnxz , "

:* ,

is said to have a multivariate normal distribution with

and ( invertible ) covariance
"
matrix
mean it EIR ,

quadratic form in ☒
I HRn×h if it
pdf is
-

1- aH= y¥zpñ exp { -

ICI -

A
ATE 'c¥µ→ }
A row vector ¥n collect .

determinant of matrix
"

I IEIR
Denoted as I ~
Nnl 11 , E)
It
2
e-
g. n =L .
I =
02 ,
III =P .
=
g-

fix )=µ¥ ,
exp (-1-21×-1%5) ✗ HR .
 (
Iii
? ?:?)
: "
z
-

-
earn ri
onxiixis

On , Thz
'

Ohn
.
In
particular ,

Cps d) .

.
rii =
Var Ni ) .

I is symmetric , positive definite

semi -

matrix .

C all eigenvalues are non -

negative ) ,

in:) 1%2
ma .
a- -

E -

-
'
.

. .

recall for by matrix

( ea bd )
An
a 2- - a -

,
-

its inverse is Ai
'
-

[Link] ( Ic ba ) -

' '
( inverse An Ai Aj An Is 2- d
identity )
-

= =
.
:
.

( Ear ego:)
'
so , E- =
-

.
 T -1
The quadratic form 11 -

A) I ( I -

A) becomes
ri

( ) ( )
Por Xi Mi
¥Epy
-
-

Ch M K Ma )
.
- -

Xzyuz
-
papa rid

oÉpy [ rich Mi zpoioicx , pinch µ )

'
- -
.
-
= -

ftp.?-)--.p--HFY-i-ael*-tY)FHY-.)i-
+ rich -

1%1-51 .

The determinant II / = riri -

piriri 4- pyriri -

funk )=z-µ¥ie×P / -

¥p, -1 - - .

] } .

which is the form of a bivariate normal pdf

with 5 ( Mv /Un ri rat e)
parameters .
•
Review of linear algebra :

""
Matrix square root :
If FIR is a
square symmetric
matrix ,
[Link] .
Then there exists a
unique matrix
B- ER
" 'M
, symmetric and psd , such that

Bi =
E
-

E =
A~
call B~ the ( matrix ) of # writers
We as
square root
,

"
as Mt or
Ai .

Properties :
① ( Standardization transform )
Let Z ( 2,27 2h ) Nn / In In )
"
=
i ~

called the dim ☒ dim

This is " n -

n -

" Zero ""tr

identity
standard normal
" '

random vector .
matrix

A- In , I -

In .
If :?)
[Link]/--iz-yn#zexpl-1-z(Z-EJzillz--In) ))
¥ Ézi ¥
"

µ¥ exp f- E ETI )
TER
"

-VI=( Zi Zai 2-b)

-

, ,

÷! ¥ exp /
XT
-

E zig

pdf NIOID .

21,22, Zn are
mutually independent
-

i
-

and have a common dist 'm as N / 41) .

For a
general I ~
Nnlit E) ,

"
Let the of I
'

§ as
-
'
square root .

(E ) In)
' "
Then ( I -

it ) ~
Nnl In , .

¥
 '"
Conversely , if I =
I Z +
µ ,
then

I ~ Nnc it E) ,

② Theorem 3.5.1 : ( Affine transform of normal )

suppose W~~Nncµ
"
,
E) .
Let V~=
EEK
where ERM IRM Then
, b- c-
,
Affine transformation
I ~
Nm ( A- A -11 , I E. IT )
matrix transpose .

"
e.
g. ① .
Take W~= Z ~
Nn Cdn , Ty ) .
=
I !
1=11 I
"
.
=
I' Z t
H .

I "I
" ' '" "'
E- II ] =
In -11=11 .
Couch ) =
I II I =

=
I .
 I ~
Nnl A , E) .

transformation
② .
Take W~ =
In Nn 41 E) , .
I

541 A)
"
" '
k=
EET
'
-

=
5 .

bi -
.

µ .

'
"
E- Evil =
# 11+1
'
=
5111 + 1- ⑦ 4- µ )
=
In .

'

(E)
"
Cover ) =
A~[Link] =
I j =
2g

( (EHLE ) 1K¥
E)
'
I =
I =

left multiply right multiply

I ~ Nnl In TA ) ,
 row vector

IRN
"

③ .
Take m -4 ,
C-
,
=
a~T = Canarian ) .

b~ =D
( Yon )
.

V= A~W~ =
give = Cali : an )

=
-¥
,
ai Wi linear combination
Wi 's .
of

E- TV7= GTA =
Éaipli
in
.

quadratic form
Corfu ) =
A- I T= g. Tz g p in a~ .

¥ ,
aiwi ~
NI É,aiµi ,
ai IG )

marginal distributions :

} dim

( Yg ) }
m -

If we write ☒ = HR ?
4- m ) dim -

block form
?⃝
 ( Y;! )
" m

ERM
-

where Xin =
.
I -

- HR .

and covariance matrix

we
partition the mean in
the same
way .
m n M
-

:t÷:
E Ee
Kiki
" m

a- "
nm
. n

M n m -

Here , Mj is the mean Hector ) of Xd ,

ply XI
-
. -
- -

. . -

. .
.

is the covariance matrix of Xi

Iv
Izz is Xy
- -
-
- -
-
- -

. . -
.

-212 collects all covariances between Xi and Xy .

 m Am

Now we define A~=m[ Imi I ]

1pm
✗ in
- uptake b~=m
FIRM 'M c- ' FIRM
then ☒ + k A
projection matrix

(¥)
.

=
timid ]
=
Ivy I
-

+
I -

I
=
¥

TETE ) =
A~ -

ETI ] -1k =

( ¥ ¥ )( ¥ )
.

Cover ) =
A. EAT =
Item :& )
. .

=
In ☒ ~
Nm ( Hr , In ) .

similarly , ¥ ~
Nam 1Mt , .
④ Conditional distribution :
☒ =/ ¥ ) ~
Nnl # E) i
.

'
☒ I # =
I ~Nm( it +
É II µ;) -

E)
'

En -

En Ei .

In the special case when a- 2, m =/ .

'
E- [ Xi I Xi =
Hi =
µ, -1 Ez Ezi Ha
-

M)
po
"
.
riz
=
Mi t p
-

¥
,
( Xz -

Ma ) .

'
Var ( ✗ 11×2=214 = In - In Zzi -221 =
4- f) ri .

Ii pr Fi Fara
.
'

Xi Huk ~
NC Mite E. Him . 4- f) ri)
 Implication :
☒ I ¥ if and only if Iz =D .

e-
g. I -
N} ( it -27, ,
it =/ §) .
E- -

ff ?
ii. Find P ( 4×1 -3×2+5×3 <
8)
( ¥;)
,

Y =
4×1 3×2-15×3
-
=
GT ☒ = 14 -3 5)

/ %)
.

TETY )= ɵ~ =
(4-35) =
4.5-3-3+5.7
=
46

-3511¥ I %) / %)
.

vary )= at EA =
14

14
( Yg ) 289
-3 5)
= =

Y - N ( 46,289 )
ply c 8) =
Ipf 2 <
=p / 2<23--0.9772 .
 (2) .
Find IP ( Xi > 8
/ Xz =L , ✗ 3--10) .

Izzi ( to -1µs )
'

E- Exit Xi -1,113=10 ] =
µ +
-

÷f¥¥iÉ) 0
! 2 9 =
5.75
+ e-

.
HI :})-1¥ )
Var 1×11×2=11×3--10) = Ei , -

Ii Egil Ey

[=¥HÉ- (1)
'
1,0119 } )
-

=
4- c-

3.72 .

3.72 )
✗ 11×2--1 Xi lo N ( 5.75
-
,
-
.

PIZ 1.17 )
8¥)
>
PIX 781×2=1
=

, ✗ i. to) =p 12 >
0.121
,
=
.