72 Chapter 2 Probability

2.7 Bayes’ Rule

Bayesian statistics is a collection of tools that is used in a special form of statistical
inference which applies in the analysis of experimental data in many practical
situations in science and engineering. Bayes’ rule is one of the most important
rules in probability theory. It is the foundation of Bayesian inference, which will
be discussed in Chapter 18.

Total Probability
Let us now return to the illustration of Section 2.6, where an individual is being
selected at random from the adults of a small town to tour the country and publicize
the advantages of establishing new industries in the town. Suppose that we are
now given the additional information that 36 of those employed and 12 of those
unemployed are members of the Rotary Club. We wish to find the probability of
the event A that the individual selected is a member of the Rotary Club. Referring
to Figure 2.12, we can write A as the union of the two mutually exclusive events
E ∩ A and E  ∩ A. Hence, A = (E ∩ A) ∪ (E  ∩ A), and by Corollary 2.1 of Theorem
2.7, and then Theorem 2.10, we can write

P (A) = P [(E ∩ A) ∪ (E  ∩ A)] = P (E ∩ A) + P (E  ∩ A)

= P (E)P (A|E) + P (E  )P (A|E  ).

E E
A

EA

E  A

Figure 2.12: Venn diagram for the events A, E, and E  .

The data of Section 2.6, together with the additional data given above for the set
A, enable us to compute
600 2 36 3
P (E) = = , P (A|E) = = ,
900 3 600 50
and
1 12 1
P (E  ) = , P (A|E  ) = = .
3 300 25
If we display these probabilities by means of the tree diagram of Figure 2.13, where
the first branch yields the probability P (E)P (A|E) and the second branch yields
2.7 Bayes’ Rule 73

E P(A|E) = 3/50 A
P(E)P(A|E)

3
2/
=
E)
P(
P(
E'
)=
1/
3
P(E')P(A|E')
E' P(A|E)  1/25 A'

Figure 2.13: Tree diagram for the data on page 63, using additional information
on page 72.

the probability P (E  )P (A|E  ), it follows that

     
2 3 1 1 4
P (A) = + = .
3 50 3 25 75

A generalization of the foregoing illustration to the case where the sample space
is partitioned into k subsets is covered by the following theorem, sometimes called
the theorem of total probability or the rule of elimination.

Theorem 2.13: If the events B1 , B2 , . . . , Bk constitute a partition of the sample space S such that
P (Bi ) = 0 for i = 1, 2, . . . , k, then for any event A of S,


k 
k
P (A) = P (Bi ∩ A) = P (Bi )P (A|Bi ).
i=1 i=1

B3
B1

B4 B5
A

B2
…

Figure 2.14: Partitioning the sample space S.

74 Chapter 2 Probability

Proof : Consider the Venn diagram of Figure 2.14. The event A is seen to be the union of
the mutually exclusive events

B1 ∩ A, B2 ∩ A, . . . , Bk ∩ A;

that is,

A = (B1 ∩ A) ∪ (B2 ∩ A) ∪ · · · ∪ (Bk ∩ A).

Using Corollary 2.2 of Theorem 2.7 and Theorem 2.10, we have

P (A) = P [(B1 ∩ A) ∪ (B2 ∩ A) ∪ · · · ∪ (Bk ∩ A)]

= P (B1 ∩ A) + P (B2 ∩ A) + · · · + P (Bk ∩ A)

k
= P (Bi ∩ A)
i=1

k
= P (Bi )P (A|Bi ).
i=1

Example 2.41: In a certain assembly plant, three machines, B1 , B2 , and B3 , make 30%, 45%, and
25%, respectively, of the products. It is known from past experience that 2%, 3%,
and 2% of the products made by each machine, respectively, are defective. Now,
suppose that a finished product is randomly selected. What is the probability that
it is defective?
Solution : Consider the following events:
A: the product is defective,
B1 : the product is made by machine B1 ,
B2 : the product is made by machine B2 ,
B3 : the product is made by machine B3 .
Applying the rule of elimination, we can write

P (A) = P (B1 )P (A|B1 ) + P (B2 )P (A|B2 ) + P (B3 )P (A|B3 ).

Referring to the tree diagram of Figure 2.15, we find that the three branches give
the probabilities

P (B1 )P (A|B1 ) = (0.3)(0.02) = 0.006,

P (B2 )P (A|B2 ) = (0.45)(0.03) = 0.0135,
P (B3 )P (A|B3 ) = (0.25)(0.02) = 0.005,

and hence

P (A) = 0.006 + 0.0135 + 0.005 = 0.0245.

2.7 Bayes’ Rule 75

B1 P(A | B 1 ) = 0.02 A

3
0.
)=
1
B
P(
P(B 2 ) = 0.45 P(A | B 2 ) = 0.03 A
B2

P(
B
3
)=
A

0.
25
B3 P(A | B 3 ) = 0.02

Figure 2.15: Tree diagram for Example 2.41.

Bayes’ Rule
Instead of asking for P (A) in Example 2.41, by the rule of elimination, suppose
that we now consider the problem of finding the conditional probability P (Bi |A).
In other words, suppose that a product was randomly selected and it is defective.
What is the probability that this product was made by machine Bi ? Questions of
this type can be answered by using the following theorem, called Bayes’ rule:

Theorem 2.14: (Bayes’ Rule) If the events B1 , B2 , . . . , Bk constitute a partition of the sample
space S such that P (Bi ) = 0 for i = 1, 2, . . . , k, then for any event A in S such
that P (A) = 0,

P (Br ∩ A) P (Br )P (A|Br )

P (Br |A) = = for r = 1, 2, . . . , k.

k 
k
P (Bi ∩ A) P (Bi )P (A|Bi )
i=1 i=1

Proof : By the definition of conditional probability,

P (Br ∩ A)
P (Br |A) = ,
P (A)
and then using Theorem 2.13 in the denominator, we have
P (Br ∩ A) P (Br )P (A|Br )
P (Br |A) = = ,

k 
k
P (Bi ∩ A) P (Bi )P (A|Bi )
i=1 i=1

which completes the proof.

Example 2.42: With reference to Example 2.41, if a product was chosen randomly and found to
be defective, what is the probability that it was made by machine B3 ?
Solution : Using Bayes’ rule to write
P (B3 )P (A|B3 )
P (B3 |A) = ,
P (B1 )P (A|B1 ) + P (B2 )P (A|B2 ) + P (B3 )P (A|B3 )
 / /

76 Chapter 2 Probability

and then substituting the probabilities calculated in Example 2.41, we have

0.005 0.005 10
P (B3 |A) = = = .
0.006 + 0.0135 + 0.005 0.0245 49
In view of the fact that a defective product was selected, this result suggests that
it probably was not made by machine B3 .

Example 2.43: A manufacturing firm employs three analytical plans for the design and devel-
opment of a particular product. For cost reasons, all three are used at varying
times. In fact, plans 1, 2, and 3 are used for 30%, 20%, and 50% of the products,
respectively. The defect rate is different for the three procedures as follows:

P (D|P1 ) = 0.01, P (D|P2 ) = 0.03, P (D|P3 ) = 0.02,

where P (D|Pj ) is the probability of a defective product, given plan j. If a random

product was observed and found to be defective, which plan was most likely used
and thus responsible?
Solution : From the statement of the problem

P (P1 ) = 0.30, P (P2 ) = 0.20, and P (P3 ) = 0.50,

we must find P (Pj |D) for j = 1, 2, 3. Bayes’ rule (Theorem 2.14) shows

P (P1 )P (D|P1 )
P (P1 |D) =
P (P1 )P (D|P1 ) + P (P2 )P (D|P2 ) + P (P3 )P (D|P3 )
(0.30)(0.01) 0.003
= = = 0.158.
(0.3)(0.01) + (0.20)(0.03) + (0.50)(0.02) 0.019

Similarly,

(0.03)(0.20) (0.02)(0.50)
P (P2 |D) = = 0.316 and P (P3 |D) = = 0.526.
0.019 0.019
The conditional probability of a defect given plan 3 is the largest of the three; thus
a defective for a random product is most likely the result of the use of plan 3.
Using Bayes’ rule, a statistical methodology called the Bayesian approach has
attracted a lot of attention in applications. An introduction to the Bayesian method
will be discussed in Chapter 18.

Exercises

2.95 In a certain region of the country it is known ability that an adult over 40 years of age is diagnosed
from past experience that the probability of selecting as having cancer?
an adult over 40 years of age with cancer is 0.05. If
the probability of a doctor correctly diagnosing a per- 2.96 Police plan to enforce speed limits by using radar
son with cancer as having the disease is 0.78 and the traps at four different locations within the city limits.
probability of incorrectly diagnosing a person without The radar traps at each of the locations L1 , L2 , L3 ,
cancer as having the disease is 0.06, what is the prob- and L4 will be operated 40%, 30%, 20%, and 30% of
 / /

Review Exercises 77

the time. If a person who is speeding on her way to one-year period, the number of malfunctions reported
work has probabilities of 0.2, 0.1, 0.5, and 0.2, respec- by each station and the causes are shown below.
tively, of passing through these locations, what is the Station A B C
probability that she will receive a speeding ticket? Problems with electricity supplied 2 1 1
Computer malfunction 4 3 2
2.97 Referring to Exercise 2.95, what is the probabil- Malfunctioning electrical equipment 5 4 2
ity that a person diagnosed as having cancer actually Caused by other human errors 7 7 5
has the disease?
Suppose that a malfunction was reported and it was
2.98 If the person in Exercise 2.96 received a speed- found to be caused by other human errors. What is
ing ticket on her way to work, what is the probability the probability that it came from station C?
that she passed through the radar trap located at L2 ?
2.101 A paint-store chain produces and sells latex
2.99 Suppose that the four inspectors at a film fac- and semigloss paint. Based on long-range sales, the
tory are supposed to stamp the expiration date on each probability that a customer will purchase latex paint is
package of film at the end of the assembly line. John, 0.75. Of those that purchase latex paint, 60% also pur-
who stamps 20% of the packages, fails to stamp the chase rollers. But only 30% of semigloss paint buyers
expiration date once in every 200 packages; Tom, who purchase rollers. A randomly selected buyer purchases
stamps 60% of the packages, fails to stamp the expira- a roller and a can of paint. What is the probability
tion date once in every 100 packages; Jeff, who stamps that the paint is latex?
15% of the packages, fails to stamp the expiration date
once in every 90 packages; and Pat, who stamps 5% of 2.102 Denote by A, B, and C the events that a grand
the packages, fails to stamp the expiration date once prize is behind doors A, B, and C, respectively. Sup-
in every 200 packages. If a customer complains that pose you randomly picked a door, say A. The game
her package of film does not show the expiration date, host opened a door, say B, and showed there was no
what is the probability that it was inspected by John? prize behind it. Now the host offers you the option
of either staying at the door that you picked (A) or
2.100 A regional telephone company operates three switching to the remaining unopened door (C). Use
identical relay stations at different locations. During a probability to explain whether you should switch or
not.

Review Exercises

2.103 A truth serum has the property that 90% of 2.106 The probabilities that a service station will
the guilty suspects are properly judged while, of course, pump gas into 0, 1, 2, 3, 4, or 5 or more cars during
10% of the guilty suspects are improperly found inno- a certain 30-minute period are 0.03, 0.18, 0.24, 0.28,
cent. On the other hand, innocent suspects are mis- 0.10, and 0.17, respectively. Find the probability that
judged 1% of the time. If the suspect was selected in this 30-minute period
from a group of suspects of which only 5% have ever (a) more than 2 cars receive gas;
committed a crime, and the serum indicates that he is
(b) at most 4 cars receive gas;
guilty, what is the probability that he is innocent?
(c) 4 or more cars receive gas.
2.104 An allergist claims that 50% of the patients
she tests are allergic to some type of weed. What is 2.107 How many bridge hands are possible contain-
the probability that ing 4 spades, 6 diamonds, 1 club, and 2 hearts?
(a) exactly 3 of her next 4 patients are allergic to
weeds? 2.108 If the probability is 0.1 that a person will make
a mistake on his or her state income tax return, find
(b) none of her next 4 patients is allergic to weeds?
the probability that
(a) four totally unrelated persons each make a mistake;
2.105 By comparing appropriate regions of Venn di-
agrams, verify that (b) Mr. Jones and Ms. Clark both make mistakes,
and Mr. Roberts and Ms. Williams do not make a
(a) (A ∩ B) ∪ (A ∩ B  ) = A; mistake.
(b) A ∩ (B  ∪ C) = (A ∩ B  ) ∪ (A ∩ C).