Module 1, Probability, Chapter 1

MAST90105 Methods of
Mathematical Statistics

School of Mathematics & Statistics

University of Melbourne
Two basic rules for counting

• Multiplication or Tree principle If one choice has n1 possibilities

and another has n2 possibilities, then a choice of both has n1 × n2
possibilities
• Addition principle If one event has n1 outcomes and a disjoint one
has n2 outcomes, then the union has n1 + n2 outcomes

2 of 20
Permutations

The number of permutations or orderings of a subsetof size r from a

set of size n is

n Pr = n(n − 1) . . . (n − r + 1)
n(n − 1) . . . 1
=
(n − r)(n − r − 1) . . . 1
n!
=
(n − r)!

3 of 20
Combinations

The number of combinations or samples without replacement or

subsets of size r from a set of size n is
(n − 1) . . . (n − r + 1)
n Cr =
r(r − 1) . . . 1
n(n − 1) . . . 1
=
r(r − 1) . . . 1 × (n − r)(n − r − 1) . . . 1
n!
=
r!(n − r)!
 
n
=
r

The last equality is the definition of the bracket and this number n Cr
is often called a Binomial Coefficient.
4 of 20
Example - Sampling without Replacement

Suppose that 10000 lines of code have 30 lines which could be

improved (for example by clarification of purpose, simplification or
documentation) . If a random sample of 10 lines of code is taken,
what is the chance that:
(a) 1 or
(b) 0
lines of the code sampled could be improved?

5 of 20
Example Solution
• The sample space has all the samples of size 10 taken without
replacement is 10,000


10 .
• The 1 line of code in the sample that could be improved must be
chosen from the 30 improvable lines. There are 30


1 = 30 such
choices.
• The other 9 lines of code in the sample must be chosen from the
10, 000 − 30 = 9,
970 lines of code that cannot be improved.
There are 9,970
9 ways to do that.
• The multiplication principle says
30

that the
9,970

total number of samples
with one improvable line is 1 × 9 .
• Random sampling without replacement means that all the samples
are equally likely so the answer is
30 9,970



1 × 9
10,000


10
6 of 20
Example Solution Ctd 1

There are very big numbers in this calculation but modern symbolic
packages compute them and the ratio exactly. The next two slides
show the calculations for (a) and (b) using the package Mathematica.
They show that the probability of one line in the sample needing
improvement is about 0.03.

7 of 20
Example Solution Ctd 2
This is the Mathematica command to find the number of sample
points:
Binomial[10000, 10]
This is the output of it:
2, 743, 355, 077, 591, 282, 538, 231, 819, 720, 749, 000
This calculates the probability that one line of code in the sample
could be improved:
Binomial[30, 1] ∗ Binomial[10000 − 30, 9]/Binomial[10000, 10]
Here is the output:
2143751464028247883152007617
73351740042547661450048655635
This asks to evaluate the fraction
 2143751464028247883152007617  as a decimal:
N 73351740042547661450048655635
Here is the result:
0.0292256
8 of 20
Example Solution Ctd 3

Now for the probability that there are no lines of code that could be
improved, Input and Output:
Binomial[30, 0] ∗ Binomial[10000 − 30, 10]/Binomial[10000, 10]

1016852777770732245908435612997
1047882000607823735000695080500

 1016852777770732245908435612997 
N 1047882000607823735000695080500

0.970389

9 of 20
Ex. Sampling W’out Replacement - Comments,
Questions
• There is a chance of 0.9995 that there is at most one line of code
that could be improved
• Do we really have to work out those big numbers?
Approximations?
• What happens with sampling with replacement? Comparison?
• In practice, we’d observe the number of lines of code that could be
improved in a sample, and want to infer from that how good the
code is in total
• To do this we could use the probabilities calculated to make an
inference about the plausibility of 30 in total
• For example, if we observed 5 lines of code in our sample of 10
that could be improved, then 30 in total is implausible
10 of 20
Conditional Probability

• Continuing the last example, given an observed number of 5 lines

of code in our sample, the probability that the total number of
lines of code among the 10,000 is 30 or 50 or 100 ... would help us
decide the quality of the code?
• This is the approach of Bayesian statistics

11 of 20
Conditional Probability

• The frequentist approach, on the other hand, would be to start

with the Assumption that there was 30 or 50 or 100 lines of code
that could be improved, to compute the probability of the observed
5 lines of improvable code in the sample
• This probablity could then be plotted against the total number of
lines, and the maximum picked out - this called the maximum
likelihood estimate
• Conditional probabilities are the heart of the Bayesian approach,
but are also very important for frequentists

12 of 20
Conditional Probability

Definition. The conditional probability of an event A, given that event

B has occurred, is defined by

P (A ∩ B)
P (A|B) = ,
P (B)
provided that P (B) > 0

13 of 20
Example - Conditional Prob Definition

Two cards are drawn without replacement from a deck of cards. (a)
What is the conditional probability of getting an ace on the second
draw given that the first draw is an ace? (b) What is the conditional
probability of getting an ace on the first draw given that the second
draw is an ace?

14 of 20
Example Solution
Let A1 be the event that the first draw is an ace and A2 be the event
that the second draw is an ace. Then

n(A1 )
P (A1 ) = , so
n(S)
4 × 51
P (A1 ) =
52 × 51
n(A2 )
P (A2 ) = , so
n(S)
4 × 3 + 48 × 4
P (A2 ) =
52 × 51
n(A1 ∩ A2 ) 4×3
P (A1 ∩ A2 ) = , so P (A1 ∩ A2 ) =
n(S) 52 × 51
15 of 20
Example Solution Ctd

Answering (a)
P (A1 ∩ A2 )
P (A2 |A1 ) =
P (A1 )
which gives
4×3 3
P (A2 |A1 ) = =
4 × 51 51
and
P (A1 ∩ A2 )
P (A1 |A2 ) = ,
P (A2 )
so
4×3 3
P (A1 |A2 ) = =
4 × 51 51

16 of 20
Conditional Probability Example - Comment

• The answer to (a) is not perhaps surprising since there are 51 cards
left after the first choice of which 3 are aces given that the first
drawer was an ace
• But why should (b) have the same answer?
• A thought experiment leads to the idea of symmetry of events, in
this case called exchangeability

17 of 20
Rules of Conditional Probability

Conditional probability satisfies the same rules as ordinary probability.

For unions, the conditioning event needs to be the same. For
example, if A1 and A2 are disjoint events and P (B) > 0,

P (A1 ∪ A2 |B) = P (A1 |B) + P (A2 |B)

18 of 20
Multiplication Rule

The definition of conditional probability can be re-expressed as,

P (A ∩ B) = P (A|B) × P (B)
or equally

P (A ∩ B) = P (B|A) × P (A)

19 of 20
Example - System Failure

A device has two components. It will operate if at least one of the

two components is operating. The probability that one component
will fail when both are working in a one-year period is 0.01. However,
when one fails, the probability of the other failing is 0.03 in that
one-year period due to added strain. The two components cannot fail
at the same [Link] is the probability that the device fails during a
one-year period

20 of 20