2.

Conditional Expectation

2. Conditional Expectation (9/10/04; cf. Ross)

Intro / Definition

Examples

Conditional Expectation

Computing Probabilities by Conditioning

1
 2. Conditional Expectation

Intro / Definition

Recall conditional probability: Pr(A|B) = Pr(A∩B)/Pr(B)

if Pr(B) > 0.

Suppose that X and Y are jointly discrete RV’s. Then

if Pr(Y = y) > 0,
Pr(X = x ∩ Y = y) f (x, y)
Pr(X = x|Y = y) = =
Pr(Y = y) fY (y)
Pr(X = x|Y = 2) defines the probabilities on X given
that Y = 2.
2
 2. Conditional Expectation

Definition: If fY (y) > 0, then fX|Y (x|y) ≡ ff(x,y)

(y)
is the
Y
conditional pmf/pdf of X given Y = y.

Remark: Usually just write f (x|y) instead of fX|Y (x|y).

Remark: Of course, fY |X (y|x) = f (y|x) = ff (x,y)

(x)
.
X

3
 2. Conditional Expectation

Old Discrete Example: f (x, y) = Pr(X = x, Y = y).

X = 1 X = 2 X = 3 X = 4 fY (y)
Y =1 .01 .07 .09 .03 .2
Y =2 .20 .00 .05 .25 .5
Y =3 .09 .03 .06 .12 .3
fX (x) .3 .1 .2 .4 1

Find f (x|2).

4
 2. Conditional Expectation

Then





0.4 if x = 1
0 if x = 2


f (x, 2) f (x, 2)



f (x|2) = = = 0.1 if x = 3
fY (2) 0.5 


 0.5 if x = 4


0 otherwise




5
 2. Conditional Expectation

Old Cts Example:

21 2
f (x, y) = x y, if x2 ≤ y ≤ 1
4

21 2
fX (x) = x (1 − x4), if −1 ≤ x ≤ 1
8

7 5/2
fY (y) = y , if 0 ≤ y ≤ 1
2
Find f (y|X = 1/2).

6
 2. Conditional Expectation

1 f (1
2 , y)
f (y| ) =
2 fX ( 12)
21 · 1 y
= 21 4 4
1 · (1 − 1 ) , if 1
4 ≤y≤1
8 · 4 16

32
= y, if 1
4 ≤y≤1
15

7
 2. Conditional Expectation

More generally,
f (x, y)
f (y|x) =
fX (x)
21 x2y
= 4
21 x2(1 − x4) , if x2 ≤ y ≤ 1
8
2y 2 ≤ y ≤ 1.
= if x
1 − x4
Note: 2/(1 − x4) is a constant with respect to y, and
we can check to see that f (y|x) is a legit condl pdf:
Z 12y
2 4
dy = 1.
x 1−x

8
 2. Conditional Expectation

Typical Problem: Given fX (x) and f (y|x), find fY (y).

Steps: (1) f (x, y) = fX (x)f (y|x)

(2) fY (y) = < f (x, y) dx.
R

Example: fX (x) = 2x, 0 < x < 1.

Given X = x, suppose that Y |x ∼ U(0, x). Now find

fY (y).

9
 2. Conditional Expectation

Solution: Y |x ∼ U(0, x) ⇒ f (y|x) = 1/x, 0 < y < x.

So

f (x, y) = fX (x)f (y|x)

1
= 2x · , if 0 < x < 1 and 0 < y < x
x
= 2, if 0 < y < x < 1.

Thus,
Z Z 1
fY (y) = f (x, y) dx = 2 dx = 2(1 − y), 0 < y < 1.
< y

10
 2. Conditional Expectation

Conditional Expectation

Usual definition of expectation:

 P

y yf (y) discrete
E[Y ] =
yf (y) dy continuous
R
<


f (y|x) is the conditional pdf/pmf of Y given X = x.

Definition: The conditional expectation of Y given

X = x is
 P
y yf (y|x) discrete

E[Y |X = x] ≡
yf (y|x) dy continuous
R
<

11
 2. Conditional Expectation

Note that E[Y |X = x] is a function of x.

Example: Suppose that






0.2 if y = 1
0.3 if y = 2


f (y|X = 2) = 



0.5 if y = 3
0 otherwise



Then
X
E[Y |X = 2] = yf (y|2) = 1(.2)+2(.3)+3(.5) = 2.3.
y

12
 2. Conditional Expectation

Old Cts Example:

21 2
f (x, y) = x y, if x2 ≤ y ≤ 1.
4
Recall that
2y 2 ≤ y ≤ 1.
f (y|x) = if x
1 − x4
Thus,
Z 2 Z1 2 2 1 − x6
E[Y |x] = yf (y|x) dy = 4 2
y dy = · 4
.
< 1−x x 3 1−x

13
 2. Conditional Expectation

Theorem (double expectations): E[E(Y |X)] = E[Y ].

Remarks: Yikes, what the heck is this!? The exp

value (averaged over all X’s) of the conditional exp
value (of Y |X) is the plain old exp value (of Y ).

Think of the outside exp value as the exp value of

h(X) = E(Y |X). Then the Law of the Unconscious
Statistician miraculously gives us E[Y ].

14
 2. Conditional Expectation

Proof (cts case): By the Unconscious Statistician,

Z
E[E(Y |X)] = E(Y |x)fX (x) dx
<

Z ÃZ !
= yf (y|x) dy fX (x) dx
< <

Z Z
= yf (y|x)fX (x) dx dy
< <

Z Z
= y f (x, y) dx dy
< <

Z
= yfY (y) dy = E[Y ].
<
15
 2. Conditional Expectation

Old Example: Suppose f (x, y) = 21

4 x 2y, if x2 ≤ y ≤ 1.

Find E[Y ] two ways.

By previous examples, we know that

21 2
fX (x) = x (1 − x4), if −1 ≤ x ≤ 1
8

7 5/2
fY (y) = y , if 0 ≤ y ≤ 1
2

2 1 − x6
E[Y |x] = · 4
.
3 1−x
16
 2. Conditional Expectation

Solution #1 (old, boring way):

Z 7 7/2 7
Z 1
E[Y ] = yfY (y) dy = y dy = .
< 0 2 9

Solution #2 (new, exciting way):

E[Y ] = E[E(Y |X)]

Z
= E(Y |x)fX (x) dx
<

Z 1 Ã2 1−x 6 !Ã
21
!
7
= · x2(1 − x4) dx = .
−1 3 1 − x4 8 9
17
 2. Conditional Expectation

Notice that both answers are the same (good)!

Believe it or not, sometimes it’s easier to calculate

E[Y ] indirectly by using our double expectation trick.

18
 2. Conditional Expectation

Example: An alternative way to calculate the mean

of the Geom(p).

Let N ∼ Geom(p), e.g., N could be the number of

coin flips before H appears.

Let 
 1 if first flip is H
Y = .
 0 otherwise

We’ll apply a “standard conditioning argument” (in

the discrete case) to compute E[N ].
19
 2. Conditional Expectation

E[N ] = E[E(N |Y )]
X
= E(N |y)fY (y)
y
= E(N |Y = 0)Pr(Y = 0) + E(N |Y = 1)Pr(Y = 1)
= (1 + E[N ])(1 − p) + 1(p).

Solving, we get E[N ] = 1/p.

20
 2. Conditional Expectation

Theorem (expectation of a random number of RV’s):

Suppose that X1, X2, . . . are independent RV’s, all with

the same mean. Also suppose that N is a nonneg-
ative, integer-valued RV, that’s independent of the
Xi’s. Then
N
 
X
E Xi = E[N ]E[X1].
i=1

21
 2. Conditional Expectation

Proof: By double expectation, we have

N N
    ¯ 
X X ¯
E Xi = E E  Xi¯¯¯N 
i=1 i=1
∞ N
 ¯ 
X X ¯
= E Xi¯¯¯N = n Pr(N = n)
n=1 i=1
∞ n ¯ 
X X ¯
= E X i¯¯N
¯
= n Pr(N = n)
n=1 i=1
∞ n

X X
= E X  Pr(N
i = n)
n=1 i=1
∞
X
= nE[X1]Pr(N = n)
n=1
∞
X
= E[X1] nPr(N = n). ♦
n=1
22
 2. Conditional Expectation

Example: Suppose the number of times we roll a die

is N ∼ Pois(10). If Xi denotes the value of the ith
toss, then the expected number of rolls is
N
 
X
E Xi = E[N ]E[X1] = 10(3.5) = 35. ♦
i=1

Theorem: Under the same conditions as before,

N
 

Xi = E[N ]Var(X1) + (E[X1])2Var(N ).

X
Var 
i=1

Proof: See, for instance, Ross. ♦

23
 2. Conditional Expectation

Computing Probabilities by Conditioning

Let A be some event, and define the RV Y as:


 1 if A occurs
Y = .
 0 otherwise

Then
X
E[Y ] = yfY (y) = Pr(Y = 1) = Pr(A).
y

24
 2. Conditional Expectation

Similarly, for any RV X, we have

X
E[Y |X = x] = yfY (y|x)
y
= Pr(Y = 1|X = x)
= Pr(A|X = x).

25
 2. Conditional Expectation

Further, since E[Y ] = E[E(Y |X)], we have

Pr(A) = E[Y ]
= E[E(Y |X)]
Z
= E[Y |x]dFX (x)
Z<
= Pr(A|X = x)dFX (x).
<

26
 2. Conditional Expectation

Example/Theorem: If X and Y are independent con-

tinuous RV’s, then
Z
Pr(Y < X) = FY (x)fX (x) dx,
<

where FY (·) is the c.d.f. of Y and fX (·) is the p.d.f.

of X.

27
 2. Conditional Expectation

Proof: (Actually, there are many proofs.) Let the

event A = {Y < X}. Then
Z
Pr(Y < X) = Pr(Y < X|X = x)fX (x) dx
Z<
= Pr(Y < x|X = x)fX (x) dx
Z<
= Pr(Y < x)fX (x) dx
<
(since X, Y are indep). ♦

28
 2. Conditional Expectation

Example: If X ∼ Exp(µ) and Y ∼ Exp(λ) are indepen-

dent RV’s. Then
Z
Pr(Y < X) = FY (x)fX (x) dx
Z<∞
= (1 − e−λx)µe−µx dx
0
λ
= . ♦
λ+µ

29
 2. Conditional Expectation

Example/Theorem: If X and Y are independent con-

tinuous RV’s, then
Z
Pr(X + Y < a) = FY (a − x)fX (x) dx,
<

where FY (·) is the c.d.f. of Y and fX (·) is the p.d.f.

of X. The quantity X + Y is called a convolution.

30
 2. Conditional Expectation

Proof:
Z
Pr(X + Y < a) = Pr(X + Y < a|X = x)fX (x) dx
Z<
= Pr(Y < a − x|X = x)fX (x) dx
Z<
= Pr(Y < a − x)fX (x) dx
<
(since X, Y are indep). ♦

31
 2. Conditional Expectation

iid
Example: Suppose X, Y ∼ Exp(λ). Note that




 1 − e−λ(a−x) if a − x ≥ 0 and x ≥ 0
FY (a − x) =  (i.e., 0 ≤ x ≤ a)
0 if otherwise




Z
Pr(X + Y < a) = FY (a − x)fX (x) dx
Z<a
= (1 − e−λ(a−x))λe−λx dx
0
= 1 − e−λa − λae−λa, if a ≥ 0.

d
Pr(X + Y < a) = λ2ae−λa, a ≥ 0.
da
This implies that X + Y ∼ Gamma(2, λ). ♦
32