Chapter: Probability

1. CONDITIONAL PROBABILITY

Conditional probability asks: once we know that an event F has already happened, how does that
change the probability of another event E from the same sample space?
The extra information that F has occurred effectively shrinks the sample space from S down to F —
only outcomes favourable to F remain relevant. The probability of E, recomputed on this smaller
sample space, is called the conditional probability of E given F, written P(E|F).

Formula
• P(E|F) = n(E ∩ F) / n(F), for equally likely outcomes
• Equivalently, P(E|F) = P(E ∩ F) / P(F), provided P(F) ≠ 0
Note: P(E|F) is defined only when P(F) ≠ 0, i.e. F is not an impossible event.

Example:
Three fair coins are tossed, so S has 8 equally likely outcomes. Let E = 'at least two heads' =
{HHH,HHT,HTH,THH} and F = 'first coin shows tail' = {THH,THT,TTH,TTT}. Then E∩F = {THH},
P(F) = 4/8 = 1/2 and P(E∩F) = 1/8, so P(E|F) = (1/8)/(1/2) = 1/4.

Worked Example — Cards

Example:
Ten cards numbered 1–10 are mixed and one is drawn. Given the number is more than 3, find the
probability it is even. Let A = 'even' = {2,4,6,8,10} and B = 'greater than 3' = {4,5,6,7,8,9,10}. Then
A∩B = {4,6,8,10}, so P(A|B) = P(A∩B)/P(B) = (4/10)/(7/10) = 4/7.

Worked Example — School Data

Example:
A school has 1000 students, 430 of whom are girls; 10% of the girls are in Class XII. Let E =
'studies in Class XII' and F = 'is a girl'. Then P(F) = 0.43 and P(E∩F) = 0.043, so P(E|F) =
0.043/0.43 = 0.1 — i.e. given the student is a girl, there's a 10% chance she's in Class XII (as
expected, since exactly 10% of girls are).

Worked Example — Unequal Outcomes (Tree Diagram)

Conditional probability works just as well when outcomes are not equally likely — a coin is tossed; on
heads it is tossed again, on tails a die is thrown instead. This gives 8 outcomes with unequal
probabilities (1/4 each for HH, HT; 1/12 each for the six tail-then-die outcomes).
Example:
Find P('die shows more than 4' | 'at least one tail'). Here F = {HT,T1,T2,T3,T4,T5,T6} has P(F) = 1/4
+ 6×(1/12) = 3/4, and E∩F = {T5,T6} has probability 2×(1/12) = 1/6. So P(E|F) = (1/6)/(3/4) = 2/9.

2. PROPERTIES OF CONDITIONAL PROBABILITY

• Property 1: P(S|F) = P(F|F) = 1 — the whole space and F itself are certain given F has occurred.
• Property 2: For any events A, B and F with P(F) ≠ 0, P((A∪B)|F) = P(A|F) + P(B|F) − P((A∩B)|F).
If A and B are disjoint, this simplifies to P((A∪B)|F) = P(A|F) + P(B|F).
 • Property 3: P(E′|F) = 1 − P(E|F) — the conditional probability of 'not E' given F is the
complement, exactly like ordinary probability.
Note: These three properties show conditional probability behaves just like an ordinary probability measure,
just restricted to the reduced sample space F.

Example:
If P(A) = 7/13, P(B) = 9/13 and P(A∩B) = 4/13, then P(A|B) = (4/13)/(9/13) = 4/9.

Example:
A family has two children (sample space {(b,b),(g,b),(b,g),(g,g)}, each equally likely). Given at least
one child is a boy, find the probability both are boys. Here E = 'both boys' = {(b,b)}, F = 'at least one
boy' = {(b,b),(g,b),(b,g)}, so P(F)=3/4, P(E∩F)=1/4, giving P(E|F) = 1/3.

3. MULTIPLICATION THEOREM ON PROBABILITY

Rearranging the conditional-probability formula gives a way to compute the probability that two
events both occur, using P(E|F) or P(F|E):
• P(E ∩ F) = P(F) · P(E|F) = P(E) · P(F|E), provided P(E) ≠ 0 and P(F) ≠ 0
This is called the multiplication rule of probability, and it extends to three or more events:
• P(E ∩ F ∩ G) = P(E) · P(F|E) · P(G|E∩F)
Example:
An urn has 10 black and 5 white balls; two are drawn one after another without replacement. Let E
= 'first ball black', F = 'second ball black'. Then P(E) = 10/15, and once one black ball is removed,
P(F|E) = 9/14. So P(both black) = P(E)·P(F|E) = (10/15)×(9/14) = 3/7.

Example:
Three cards are drawn without replacement from a 52-card deck. Find the probability the first two
are kings and the third is an ace. P(K) = 4/52, then P(K|K) = 3/51 (three kings left in 51 cards), then
P(A|KK) = 4/50 (four aces left in 50 cards). So P(KKA) = (4/52)×(3/51)×(4/50) = 2/5525.

4. INDEPENDENT EVENTS

Two events E and F are independent if knowing that one has occurred gives no new information
about the other — the conditional probability simply equals the unconditional one.

Definition
• E and F are independent if P(F|E) = P(F) (P(E)≠0) and P(E|F) = P(E) (P(F)≠0).
• Equivalently — and more commonly used as the working definition — E and F are independent
exactly when P(E ∩ F) = P(E) · P(F).
• If this product rule fails, E and F are called dependent.

Independent vs Mutually Exclusive

These are easy to confuse but are quite different ideas: mutual exclusivity is about the events
themselves (they share no outcomes), while independence is about their probabilities. Two events
with nonzero probabilities can never be both mutually exclusive and independent at the same time —
if they're mutually exclusive, knowing one occurred tells you the other definitely did NOT, which is
about as far from 'no new information' as possible.

Mutual Independence of Three Events

Events A, B, C are mutually independent if all of the following hold simultaneously:
• P(A∩B) = P(A)P(B), P(A∩C) = P(A)P(C), P(B∩C) = P(B)P(C)
• P(A∩B∩C) = P(A)P(B)P(C)
Note: All four conditions are needed — pairwise independence alone does not guarantee mutual
independence.

Example:
A die is thrown once. Let E = 'multiple of 3' = {3,6} and F = 'even' = {2,4,6}. P(E) = 1/3, P(F) = 1/2,
P(E∩F) = P({6}) = 1/6 = P(E)·P(F), so E and F are independent.

Example:
Three coins are tossed. E = 'three heads or three tails', F = 'at least two heads', G = 'at most two
heads'. Checking the product rule for each pair shows (E,F) are independent, but (E,G) and (F,G)
are dependent — a good reminder that independence must be checked pair by pair, not assumed
from 'related-sounding' definitions.
Example:
If E and F are independent, then so are E and F′ (and E′ and F, and E′ and F′). This follows
because E = (E∩F) ∪ (E∩F′) are disjoint pieces, so P(E∩F′) = P(E) − P(E∩F) = P(E) − P(E)P(F) =
P(E)(1−P(F)) = P(E)P(F′).

Example:
For two independent events A, B, the probability that at least one occurs is P(A∪B) =
P(A)+P(B)−P(A)P(B), which simplifies neatly to 1 − P(A′)P(B′) — i.e. one minus the probability that
neither occurs.

5. BAYES' THEOREM

Partition of a Sample Space

Events E1, E2, ..., En partition a sample space S if they are pairwise disjoint (Ei∩Ej = ∅ for i≠j), their
union is all of S, and each has nonzero probability. Any event E and its complement E′ form the
simplest possible partition.

Theorem of Total Probability

If {E1,...,En} partitions S, then any event A can be broken into pieces lying in each Ei, giving:
• P(A) = P(E1)P(A|E1) + P(E2)P(A|E2) + ... + P(En)P(A|En)
Example:
A construction job has a 0.65 chance of a strike. If there's no strike, P(on time) = 0.80; if there is a
strike, P(on time) = 0.32. Using total probability with the partition {strike, no strike}: P(on time) =
0.65×0.32 + 0.35×0.80 = 0.488.

Bayes' Theorem
Bayes' theorem answers the reverse question: given that A has occurred, what's the probability it was
'caused' by a particular Ei?
• P(Ei|A) = [P(Ei)·P(A|Ei)] / [Σj P(Ej)·P(A|Ej)]
Note: Terminology: the P(Ei) are called prior probabilities, and P(Ei|A) is the posterior probability — the
updated belief about Ei after seeing evidence A.
 Example:
Bag I has 3 red, 4 black balls; Bag II has 5 red, 6 black. A bag is picked at random and a red ball
drawn — find P(it was Bag II). P(E1)=P(E2)=1/2, P(A|E1)=3/7, P(A|E2)=5/11. By Bayes' theorem,
P(E2|A) = (½×5/11) / (½×3/7 + ½×5/11) = 35/68.

Example:
HIV test example: only 0.1% of a population actually has HIV. The test is 90% accurate for people
who have it, and 99% accurate for people who don't (so a 1% false-positive rate). Given a positive
result, the probability the person truly has HIV is only about 0.083 — despite the test sounding
accurate, a positive result is still more likely to be a false alarm than a true case, simply because
HIV is so rare in the population to begin with. This is a classic illustration of why base rates matter.

Example:
Machines A, B, C make 25%, 35%, 40% of a factory's bolts, with defect rates 5%, 4%, 2%
respectively. Given a randomly chosen bolt is defective, P(it came from machine B) = (0.35×0.04) /
(0.25×0.05 + 0.35×0.04 + 0.40×0.02) = 28/69.

6. SAMPLE QUESTIONS FOR PRACTICE

A few questions in the style typically asked from this chapter, to get a feel for how each idea above is
actually tested.

Conditional Probability
Q1. If P(A) = 0.6, P(B) = 0.3 and P(A∩B) = 0.2, find P(A|B) and P(B|A).
Q2. A die is thrown twice and the sum of the numbers appearing is 6. Find the conditional probability
that the number 4 has appeared at least once.
Q3. A fair die is rolled. Consider E = {1,3,5}, F = {2,3} and G = {2,3,4,5}. Find P(E|F) and P((E∪F)|G).

Multiplication Theorem
Q4. Two cards are drawn without replacement from a well-shuffled deck of 52 cards. Find the
probability that both cards are black.
Q5. A box has 15 oranges, 12 good and 3 bad. Three are drawn without replacement. Find the
probability all three are good.

Independent Events
Q6. Given two independent events A and B with P(A) = 0.3, P(B) = 0.6, find P(A and B), P(A or B),
and P(neither A nor B).
Q7. A and B independently try to solve a problem, with individual probabilities of success 1/2 and 1/3.
Find the probability the problem gets solved.
Q8. One card is drawn from a deck of 52. Check whether E: 'card is black' and F: 'card is a king' are
independent.

Bayes' Theorem
Q9. Bag I contains 4 red and 4 black balls; Bag II contains 2 red and 6 black balls. A bag is chosen at
random and a ball drawn is found to be red. Find the probability it was drawn from Bag I.
Q10. A factory has two machines, A producing 60% of output (2% defective) and B producing 40%
(1% defective). An item picked at random is found defective — find the probability it came from
machine B.
Q11. In a group, 60% read a Hindi newspaper, 40% read an English one, and 20% read both. If a
Hindi-newspaper reader is picked at random, find the probability she also reads the English one.
Q12 (Long Answer). Three identical boxes contain, respectively, two gold coins, two silver coins, and
one of each. A box is picked at random and one coin drawn is gold. Find the probability the other coin
in that box is also gold.