IST 299 Probability
Dr. Ayten Yiğiter and Dr. Esra Polat Lecture Notes
SECTION 6: CONDITIONAL PROBABILITY
[Link] Probability and Cumulative Distribution Functions
Let f ( x ) be the probability density function (pdf) of a continuous random variable X.
Consider an event A. If P ( A) ≠ 0 then the conditional pdf f ( x A ) is defined as shown in
below:
P ( x ≤ X ≤ x + ∆x ) A
f ( x A ) = lim
∆x →0 ∆x
(1)
f ( x)
=
P ( A)
The cumulative distribution function (cdf) of random variable X,
P ( X ≤ x ) ∩ A
F ( x A) = . (2)
P ( A)
Here, ( X ≤ x ) ∩ A = {s : X ( s ) ≤ x and s ∈ A}
From Eq. (1) and Eq. (2) the following could be written:
f ( x A) = F ( x A)
d
(3)
dx
F ( x A ) satisfies the following properties:
1) lim F ( x A ) = 0 and lim F ( x A ) = 1
x →−∞ x →+∞
Here, −∞ and +∞ represent the lower and upper bounds, respectively, for the domain of the
event A.
2) The following equation could be written:
F ( x2 A) − F ( x1 A ) = P ( x1 < X ≤ x2 ) A
P ( x1 < X ≤ x2 ) ∩ A
=
P ( A)
1
The conditional pdf f ( x A ) also shows the all properties of pdf:
1) f ( x A ) = 0, x ∉ A
2) f ( x A ) ≥ 0, x ∈ A
+∞
3) f ( x A)dx = F ( +∞ A) − F ( −∞ A) = 1
−∞
Here, −∞ and +∞ represent the lower and upper bounds, respectively, for the A.
For discrete random variables, conditional probability and conditional cumulative distribution
functions are defined in a similar style. Conditional probability function shows the properties
of probability function and conditional cumulative distribution function shows the properties
of cumulative distribution function.
A Special Example:
Let f ( x ) be the pdf of a continuous random variable X. Find the conditional cdf F ( x A )
and conditional pdf f ( x A ) .
Solution: Let A be X ≤ a , thus,
P ( X ≤ x ) ∩ ( X ≤ a ) P ( X ≤ x) F ( x)
F ( x X ≤ a) = = = is obtained.
P ( X ≤ a) P ( X ≤ a) F (a)
Here, ( X ≤ x ) ∩ ( X ≤ a ) = ( X ≤ x ) .
If x > a , as ( X ≤ x ) ∩ ( X ≤ a ) = ( X ≤ a )
P ( X ≤ x ) ∩ ( X ≤ a ) F ( a )
then F ( x X ≤ a ) = = = 1 could be written.
P ( X ≤ a) F (a)
If the derivative of F ( x X ≤ a ) is taken according to Eq. (3), the conditional pdf is found as
in below:
f ( x) f ( x)
f ( x X ≤ a) = = , x≤a
F (a) P ( X ≤ a)
= 0, x>a
Example 1: The random variable X of the life lengths of batteries is associated with a
probability density function of the form
1 − x /200
e , for x > 0
f ( x ) = 200
0, otherwise
2
Find the probability that a battery of this type lasts more than 300 hours, given that it already
has been in use for more than 200 hours.
P ( X > 300 )
Solution 1: We are interested in P ( X > 300 X > 200 ) = because the
P ( X > 200 )
intersection of the events (X > 300) and (X > 200) is the event (X > 300).
∞
1 − x /200
P ( X > 300 ) 200
e dx
e−300/200
= 300
= = e−1/2 = 0.606
P ( X > 200 ) ∞
1 − x / 200 e −1
200
200 e dx
Solution 2: Sometimes it is convenient to look at cumulative probabilities of the form
F ( x) = P ( X ≤ x)
x
1 − t / 200
= e dt
0
200
x
= −e− t / 200
0
− x /200
1 − e , x>0
=
0, x≤0
P ( X > 300 ) 1 − P ( X ≤ 300 ) 1 − F ( 300 ) 1 − (1 − e−3/ 2 ) e −3/2
P ( X > 300 X > 200 ) = = = = = −1 = e−1/2 = 0.606
P ( X > 200 ) 1 − P ( X ≤ 200 ) 1 − F ( 200 ) 1 − (1 − e−2/ 2 ) e
Example 2: The pdf of continuous random variable X is given in below:
1
f ( x) = x, 1< x < 5
12
= 0, otherwise
a) Find the conditional cdf F ( x X > 3) .
b) Find the conditional pdf f ( x X > 3) .
Solution:
x
t dt = ( x 2 − 1) .
1 1
a) The cdf of X, F ( x ) =
1
12 24
3
x2 −1
, 1< x < 5
24
F ( x ) = 0, x <1
1, x≥5
For x > 3
P ( X ≤ x ) ∩ ( X > 3) P ( 3 < X ≤ x ) F ( x ) − F ( 3)
F ( x X > 3) = = =
P ( X > 3) P ( X > 3) 1 − F ( 3)
1 2
( x − 1) −
8
24 = x − 9
2
is obtained.
= 24
8 16
1−
24
x2 − 9
, 3< x <5
16
F ( x X > 3) = 0, x<3
1, x≥5
52 − 9 16
For x=5 F ( 5 X > 3) = = =1
16 16
For x ≤ 3
P ( X ≤ x ) ∩ ( X > 3)
F ( x X > 3) = = 0 is obtained.
P ( X > 3)
b) The conditional pdf,
f ( x X > 3) = F ( x X > 3) = x
d 1
dx 8
f ( x X > 3 ) = x, 3 < x < 5
1
8
= 0, otherwise
6.1.1. The Law of Total Probability
Let A1 , A2 ,⋯ , An events satisfy the conditions given in below:
a) Ai ∩ A j = ∅, i ≠ j
b) A1 ∪ A2 ∪ ⋯ ∪ An = S
It could be showed that the equality of F ( x ) = P ( A1 ) F ( x A1 ) + ⋯ + P ( An ) F ( x An ) exists.
4
[Link] Expected Value
Consider the event A. We will show the conditional expected value of random variable X for
any event A as E ( X A ) .
If X is a continuous random variable,
E ( X A ) = x f ( x A) dx (4)
A
If X is a discrete random variable,
E ( X A) = x p ( x A) (5)
A
Example 3: The pdf of continuous random variable X is given in below:
1
f ( x) = x, 1 < x < 5
12
= 0, otherwise
Find conditional expectation E ( X X ≥ 2 ) .
Solution:
The conditional pdf
1
f ( x) x
f ( x X ≥ 2) = 12 x /12 x 24 2
= = = × = x
P ( X ≥ 2) 5
1 x 2 5 12 21 21
2 12 xdx 24
2
f ( x X ≥ 2) =
2
x, 2 < x < 5
21
= 0, otherwise
The conditional expected value,
5
117 × 2 26
5 5
2 x3
E ( X X ≥ 2 ) = x f ( x X ≥ 2 ) dx = x x dx =
2 2
= (125 − 8 ) = =
2 2
21 21 3 2 63 63 7
If A1 , A2 ,⋯ , An are disjoint events and sums of them gives the sample space S, then,
considering Eq. (1) and Eq. (4),
E ( X ) = P ( A1 ) E ( X A1 ) + ⋯ + P ( An ) E ( X An ) could be written.
5
[Link] Variance
The conditional variance of random variable X (continuous or discrete) V ( X A ) could be
found by the help of conditional probability functions.
If X is a continuous random variable,
E ( X 2 A) = x 2 f ( x A) dx (6)
A
V ( X A ) = E ( X 2 A ) − E ( X A )
2
(7)
E ( g ( X ) A ) = g ( x) f ( x A ) dx (8)
A
V ( g ( X ) A ) = E ( g ( X ) 2 A ) − E ( g ( X ) A )
2
(9)
If X is a discrete random variable,
E ( X 2 A) = x 2 p ( x A ) (10)
A
V ( X A ) = E ( X 2 A ) − E ( X A )
2
(11)
E ( g ( X ) A ) = g ( x) p ( x A ) (12)
A
V ( g ( X ) A ) = E ( g ( X ) 2 A ) − E ( g ( X ) A)
2
(13)
Example 4: Find conditional variance V ( X X ≥ 2 ) for Example 3.
5 5 5
2x4
E ( X X ≥ 2 ) = x f ( x X ≥ 2 ) dx = x . xdx = ( 5 − 24 ) =
2 1 4 203
2 2 2
=
2 2
21 84 2
42 14
203 26 1421 − 1352 69
2
V ( X X ≥ 2) = − = =
14 7 98 98