Pearson

Higher Nationals in Computing

Unit 14: Mathematics

Chapter 04 : Probability
Contents
[Link] ........................................................................................................................ 3
4.1 Introduction to Probability ................................................................................................... 3
4.2 Sample Space and Events ..................................................................................................... 4
4.3 Some Basic Theorems on Probability Space ........................................................................ 4
4.4 Mutually Exclusive (or disjoint) Events ................................................................................ 6
4.5 Conditional Probability and Tree Diagrams ......................................................................... 6
4.5.1 Conditional Probability ................................................................................................. 6
4.5.2 Properties of the conditional probability function ...................................................... 6
4.6 Multiplication Rule ............................................................................................................... 7
4.7 Tree Diagrams ...................................................................................................................... 7
4.8 Independent Events ............................................................................................................. 9
4.9 Random Variables .............................................................................................................. 10
4.10 Probability Distributions of a random variable. ................................................................. 11
4.11 Discrete Probability Distributions ...................................................................................... 11
4.11.1 Probability (Mass) Function ....................................................................................... 11
4.11.2 Mean and Variance of a Discrete Random Variable .................................................. 11
4.11.3 The Binomial Distribution........................................................................................... 13
4.12 Continuous Random Variables ........................................................................................... 15
4.12.1 Normal Probability Distribution ................................................................................. 15
4.12.2 Empirical Rule ............................................................................................................. 17
4.12.3 Standard Normal Distribution .................................................................................... 17
4.12.4 Approximation of Normal Distribution to Binomial Distribution ............................... 22
References to the Recommended Resources ................................................................... 23

Unit 14 - Mathematics for Computing

2
[Link]
4.1 Introduction to Probability

In our day-to-day life, the term probability is a measure of one’s belief in the occurrence
of a future event. This measure can range from 0 to 1, where 0 implying “impossible” and
1 implying “certain” or “sure”. This is also expressed as the “chance” of occurrence of the
event. In this case the percentage is from 0% to 100%.

Consider the following events:

• {Heavy rain in Colombo tomorrow}
• {A tsunami occurring tomorrow in some part of Sri Lanka}
• {sun rise from the west}
• {A death occurring somewhere in Sri Lanka tomorrow}
• {an accident reported in Colombo tomorrow}

Can you assign reasonable probabilities to these events?

Some reasonable answers for the probabilities of those events:

• 20 (possible, but less probable)

• 00001 ( we now know that it is not impossible, but is highly improbable)
• 0 (it is an impossible event)
• 9999 (very highly probable)
• 55 ( probable event)

Unit 14 - Mathematics for Computing

3
4.2 Sample Space and Events
The sample space associated with all possible outcomes of an experiment. Individual
outcomes in a sample space are called sample points. These are also called simple
events or elementary outcomes.
Sample space for tossing a coin 3 times is,
S = {HHH,HHT,HTH,THH,HTT,THT,TTH,TTT}

A discrete sample space contains either a finite or countable finite number of sample
points.
A continuous sample space contains infinite numbers of sample points.

An event A is any subset of the sample space S. A compound event consists of a

collection of sample points whereas a simple event consists of only a single sample
point. If S is discrete, then every subset of S is an e vent.

We can combine events to form new events using various set operations:
• A ∪ B is, if A occurs or B occurs (or both).
• A ∩ B is, if A occurs and B occurs.
• Ac (or A/ ) is the complement of A, that occurs if and only if A does not occur.

4.3 Some Basic Theorems on Probability Space

Axioms of Probability

Let S be the sample space associated with a random experiment. For every event A in
S (i.e. for every subset of S) we assign a number, P(A), called the “probability of A” so
that the following 3 axioms hold:

Theorem 1. If Φ is the empty set, then P( Φ ) = 0.

Unit 14 - Mathematics for Computing

4
 Proof: Let A be any set. Then,
A ∩ Φ = Φ which implies that A and Φ are disjoint, and Also
A ∪ Φ = A . Now by axiom 3,
P( A) = P( A ∪ Φ) = P( A) + P(Φ) which implies that P( Φ ) = 0.

Theorem 2. If A/ is the complement of an event A, then P( A /) = 1 – P(A).

Proof: Note that the sample space S = Then, by axioms 2 and 3,

Theorem 3. Let A be any event. Then, P( A) ≤ 1 .

Theorem 4. If B ⊂ A , then P(B) ≤ P( A)

Proof:

Theorem 5. If A and B are any two events, then P( A ∪ B) = P( A) + P(B) − P( A ∩ B)

Unit 14 - Mathematics for Computing

5
 4.4 Mutually Exclusive (or disjoint) Events
Definition:
Two sets A and B are said to be mutually exclusive (or disjoint) if A ∩ B = Φ . That is,
mutually exclusive sets have no sample points in common.

Example: Consider an experiment of observing the blood type (with Rh factor) of a

randomly picked person.

A = {person has blood type A} and

B = {person has blood type B}.
Then, A = { A+, A-} and B = { B+, B-}. Clearly A and B are disjoint events.

4.5 Conditional Probability and Tree Diagrams

4.5.1 Conditional Probability

Let A and B be two events in a sample space S. Conditional probability of A, given that
B has occurred, is denoted by P(A | B) and is defined as

P(A | B) in a certain sense measures the relative probability of A with respect to the
reduced space B.

4.5.2 Properties of the conditional probability function

Let A, B, C, and D be arbitrary events in a sample space S with P(D) > 0. Then,

Unit 14 - Mathematics for Computing

6
Example:
The following table shows the percentage passing or failing a job competency exam
listed according to sex, for a certain population of employees.

Find the probability of an employee has passed the exam given that the employee is a
Male.
P(A I M) = ?
𝑃(𝐴 ∩ 𝑀)
𝑃(𝐴|𝑀) =
𝑃(𝑀)
P(𝐴 ∩ 𝑀) = 24/100
P(M) = 40/100
Therefore,
𝑃(𝐴∩𝑀) 24/100 24 3
𝑃(𝐴|𝑀) = = 40/100 = 40 = 5 = 0.6 = 60%
𝑃(𝑀)

4.6 Multiplication Rule

For any two events A and B in S with P(A) > 0 and P(B) > 0,
P(A ∩ B) = P(A). P(B | A) = P(B). P(A | B) – follows directly from definition.
Extension to many:
P( A1 ∩ A2 ∩ .... ∩ Ak ) = P(A1 ). P( A2 | A1 ). P( A3 | A1 ∩ A2 ).....P( Ak | A1 ∩ A2 ∩ ....Ak
−1 )

4.7 Tree Diagrams

A tree diagram is a convenient tool for describing a sequence of (random)

experiments in which each experiment has a finite number of outcomes with given
probabilities.

Unit 14 - Mathematics for Computing

7
 Example:
Suppose we have 3 boxes as follows:
Box 1 has 5 white balls and 3 black balls;
Box 2 has 10 white balls and 5 black balls; and
Box3 has 6 white balls and 2 black balls.
If we pick a box at random, and draw a ball at random, what is the
probability, that the ball is white? that the ball is black?
[Here, if you know the outcome of the first experiment, i.e. selecting a box, then
the answer is straight forward. But since it’s random, the outcome is not known
in advance! A tree diagram showing all possible outcomes can help us find the
answer].

In the first experiment, there are 3 outcomes: {box 1, box 2, box 3}.
In the first experiment, there are 3 outcomes: {box 1, box 2, box 3
In the second, box 1 has 2 outcomes: {W, B}; box 2 has 2 outcomes: {W, B}; and
box 3 has

2 outcomes: {W, B}.

This process looks like branches of a tree! The following is a tree diagram to
describe the sequence. The probability of each outcome is shown along the
corresponding branch.

Now, to calculate the probability of any event in the last column, simply multiply
the probabilities along its path.
For example, probability of selecting Box 1 and then a White ball = P(1 ∩W ) =

Unit 14 - Mathematics for Computing

8
 (1/3). (5/8) = 5/24.
Similarly, P( 2 ∩W ) = (1/3). (2/3) = 2/9.
P( 3 ∩W ) = (1/3). (3/4) = ¼.

Now since there are 3 mutually exclusive paths which lead to a White ball, the
probability that the ball is White is the sum of the probabilities of these 3 paths.
In notation, P( W ) = P( (1 ∩W ) ∪ ( 2 ∩W ) ∪ ( 3 ∩W )) = 5/24 + 2/9 + 1/4 = 49/72.
Similarly, P(B) = 23/72. (This is obtained by using the complement rule, or by the
formula.)

With Replacement Experiment and Without Replacement Experiment

With Replacement Experiment :
• Assume there are n elements and we are going to choose k elements one after the
other.
• Here we are choosing an element by replacing the previously chosen one.
• Therefore, choosing an element is independent from choosing the previous one.
Without Replacement Experiment :
• Assume there are n elements and we are going to choose k elements one after the
other.
• Here we are choosing an element by not replacing the previously chosen one.
• Therefore, choosing an element is dependent on choosing the previous one.

4.8 Independent Events

An event B is said to be independent of an event A if the probability that B occurs is

not influenced by whether A has occurred or not. In other words, B is independent
of A if P(B | A) = P(B). Similarly, A is independent of B if P(A | B)= P(A).
Now,
P( A ∩ B) = P(B | A). P(A) = P(A | B). P(B) (by the multiplication rule)
Consider
(i) P(B | A) = P(B). This implies P( A ∩ B) = P(A) . P(B).
P(A | B)= P(A). This again implies P( A ∩ B) = P(A) . P(B).
This leads to a formal definition of independence.
Definition

Unit 14 - Mathematics for Computing

9
Events A and B are independent if and only if P( A ∩ B) = P(A) . P(B).
(By saying A and B are independent, we mean that A is independent of B, and B is
independent of A).
Extension to Three Events
Events A, B, and C, are (mutually) independent if and only if they are pair-wise
independent, and in addition, if P( A ∩ B ∩ C) = P( A). P(B). P(C ) .
Note: The definition of pair-wise independence is the one for two events given above.
Note: If A and B are independent, it can be shown that A and B , A and C, C and B , are
all independent.

4.9 Random Variables

• Random Variable (RV) is a function applicable to all elements of a sample space

• Discrete Random Variable: An RV that can take on only a finite or countable

infinite set of outcomes

Unit 14 - Mathematics for Computing

10
 • Continuous Random Variable: An RV that can take infinite set of outcomes (but
may be reported “discretely”)

• Random Variables are denoted by upper case letters (Y)

• Individual outcomes for an RV are denoted by lower case letters (y)

4.10 Probability Distributions of a random variable.

• Probabilities that a variable can take is denoted by a Probability distribution of

a random variable.
• Discrete Probability Distribution: Probabilities of a discrete random variable.
• Continuous Probability Distribution: Probabilities of a continuous random
variable
• Discrete Probabilities denoted by: PY (y) / P(y) / P(Y=y)
• Continuous Densities denoted by: fY (y) / f(y)
• Cumulative Distribution Function: FY (y) / F(y) / P(Y≤y)

4.11 Discrete Probability Distributions

4.11.1 Probability (Mass) Function

4.11.2 Mean and Variance of a Discrete Random Variable

Having identified the distribution of a discrete random variable, it may now be
important to introduce the center and spread of a random variable. We usually use the
mean to describe the center of a random variable. The mean of a random variable is
often called the expected value of the random variable.
The variance and standard deviation are used to describe the spread of a random
variable. Variance is in fact a measure of how spread-out the values are around their
mean.

Unit 14 - Mathematics for Computing

11
Definition:

The mean and variance of a discrete random variable are given by the following formulas.
Note that the mean (or expected value) is denoted by µ (or E(X)), and the variance is
denoted by σ2 or V(X)

E( X ) = ∑[x.P( X = x)]

where x is the value of the random variable and P(X = x) is the probability that X takes the
value x. Note that µ is the population mean because the sum ∑ is taken over all values of the
r.v.
σ 2 = V ( X ) = E ( X − E ( X ))2 .
Here σ is the population standard deviation.

Note:
The expected value of X always lies between the smallest and largest values of X.
The variance of X is never negative.

To find the standard deviation of the random variable, take the square root of the variance.
By definition,
V ( X ) = E (X − E (X ))2
= E(X2 − 2X .E(X ) + [E(X )]2 )

= E(X2 ) − 2E (X ).E (X ) + [E (X )]2

= E(X2 ) −[E (X )]2

Example :

Consider the previous example 3, and consider X as the number of heads. What are the
expected value and variance of X?
Solution:
We have seen that the probability distribution of X was:

X=x 0 1 2 3
P(X=x) 1/8 3/8 3/8 1/8

Unit 14 - Mathematics for Computing

12
So, E(X) = 0.(1/8) + 1.(3/8) + 2.(3/8) + 3.(1/8) = 3/2.

V(X) = 02.(1/8) + 12.(3/8) + 22.(3/8) + 32.(1/8) - [3/2]2 = 3/4,

since V(X) = E(X2) – [E(X)]2.

4.11.3 The Binomial Distribution

Consider repeated and independent trials of a random experiment with two possible
outcomes; we call one of the outcomes success and the other outcome failure, for
convenience.
[This is purely subjective; any one outcome can be called a success, and the other, a failure].
Let p be the probability of success, so that q = 1 − p is the probability of failure.
If we are interested in the number of successes in a fixed sequence of repeated trials, the
following theorem applies:

Unit 14 - Mathematics for Computing

13
Properties of the Binomial Distribution
Let the random variable X denote the number of “successes” in a sequence of n
independent trials with constant p. Then, we say X is a binomial random variable having the
binomial distribution as given above.

Unit 14 - Mathematics for Computing

14
4.12 Continuous Random Variables
A random variable is Continuous, if the possible outcomes of a random variable can be
described using an interval of real numbers.
Example :
When boiling a water pot, the water may contain any value between room temperature to
100 C◦. 40.2 C◦ , 40.22C◦ and even 89.999999 C◦ is possible. So the water temperature is a
continuous random variable.

4.12.1 Normal Probability Distribution

Normal distribution is one of the most important examples of a continuous probability

distribution. It is probably the most important distribution in statistics since many
measurements have (approximate) normal distributions, and hence many statistical
methods have been developed for normally distributed data.
The Normal (also known as Gaussian) distribution has two parameters, the mean, µ, and the
variance, σ2. Note that µ and σ2 satisfy - ∞ < µ < ∞, σ2 > 0.

If the continuous random variable is distributed as Normal with mean µ and variance σ2, we
write it as X ~ N (µ, σ2).

Unit 14 - Mathematics for Computing

15
The shape of the normal distribution takes the familiar bell-shaped curve which is symmetrical
about the vertical line over the mean µ. The parameter σ controls the spread of the curve. If
the standard deviation σ is large, the curve is flat and more spread out, and if it is small, the
curve is more peaked (see below).

2)-
2
X~N X~N Peake
2) >
(µ, σ1 - Flat (µ, σ2 d
σ
Here, σ1 2 2

The total area under the normal curve is always 1. The graph of the normal distribution is
important because the portion of the area under the curve above a given interval represents
the probability that a measurement will lie in that interval.

The formula of the shape of the normal distribution is the normal probability density
function. If X ~ N (µ, σ2), then the normal probability density function is:

The following empirical rule gives us information about the percentage of data that lies within
one, two, and three deviations of the mean

Unit 14 - Mathematics for Computing

16
4.12.2 Empirical Rule
For a normal distribution,
• Approximately 68% of the data values will lie within one standard deviation on each
side of the mean.
• Approximately 95% of the data values will lie within two standard deviations on each
side of the mean.
• Approximately 99.7% (or almost all) of the data values will lie within 15 three standard
deviations on each side of the mean.

Example :
The lifetime of a computer component is normally distributed with mean µ = 6000
hours and standard deviation σ = 500 hours. What is the probability that a computer
component selected at random will last from 6000 to 6500 hours?
Solution:
The probability that the lifetime will be between 6000 and 6500 hours can be
considered as the corresponding percentage of the area under the curve within that
interval. We can identify that this interval is in fact µ and µ + σ. As we know from the
empirical rule that the area under the normal curve between µ - σ and µ + σ is 68%, the
area between µ and µ + σ is half of 68% (or 34%) because the normal distribution is
symmetric over µ. This tells us that the probability a computer component will last from
6000 to 6500 hours is 0.34.

4.12.3 Standard Normal Distribution

Normal distributions vary from one another as they depend on two parameters, the mean
µ and the standard deviation σ. The computation of the area under the curve in a specified
interval of x values (i.e. probability) is not easy due to the complexity of the normal
probability density function. It would be a futile task to obtain a table of areas (probabilities)
under the normal curve for each different combination of µ and σ.
Mathematicians have found a way to standardize the distributions so that we can use one
table of areas for all normal distributions. For this, any normal distribution is converted to
the standard normal distribution.

The standard normal distribution is a normal distribution with mean µ = 0 and

Unit 14 - Mathematics for Computing

17
 standard deviation σ = 1. We write it as X ~ N (0, 1).

The proof of the theorem is not given here.

The above theorem says that any normal random variable X (with mean = µ and variance =
σ2) can be converted to a standard normal random variable Z (with mean = 0 and variance
= 1). The advantage here is that we can use one table which shows the areas (probabilities)
under the standard normal distribution for any interval of z values. A table is given at the
end to find the probabilities under the standard normal distribution.

Example :
Use the table of standard normal distribution to find:
the area (probability) under the standard normal distribution to the left of z = -
1.00.
the area (probability) between z = 1.00 and z = 2.70

Solution:
To find the area (probability) to the left of z = -1.00, we use the row headed by -1.0 under
the column Z of the table, and then move to the corresponding position on the right under
the column .00 . We can see that this value is 0.1587. (Table 04-a)

Area (probability) between 1.00 and 2.70 (Table 04-b)

(area left of 2.70) – (area left of 1.00)

= 0.9965 – 0.8413
= 0.1552

Area (probability) to the right of 0.95 (Table 04-b)

(area under entire curve) – (area to the left of 0.95) because P(a ≤ X ≤ b) = FX(b) - FX(a)
= 1.0000 – 0.8289= 0.1711

Alternatively, Area to the right of 0.95 = Area to the left of -0.95 = 0.1711
This is due to the fact that the standard normal r.v. Z is symmetric about zero, and hence,
for any positive number c, P(Z ≤ - c) = P(Z ≥ c) = 1 - P(Z ≤ c).

Unit 14 - Mathematics for Computing

18
 Note:
Any table is limited in the number of entries it contains. Interpolation can be used to
extend the range of values tabulated. For example, suppose you need to find the
probability (area) under the standard normal distribution to the left of z = -1.02. The
standard normal table does not give the P value at z = -1.02 (see the table), and it only
gives the probability values corresponding to z = -1.00 and z = -1.05. Here we assume
that the normal density function is changing at a roughly constant rate between, say, -
1.00 and -1.05. So the z = -1.02 will be about two fifth of the way between the
corresponding P values for z = -1.00 (i.e. 0.1587) and z = -1.05 (i.e.
0.1469). So, the corresponding P value for z = -1.02 is:
(2/5)*(0.1587 – 0.1469) + 0.1469 = 0.15162
Sometimes, we may need to calculate the value of a normal random variable required
for the variable to be at a certain proportion or probability rather than calculating the
proportion or probability for a given value of a normal random variable. This is illustrated
by an example given below.
Example :
Consider the average marks (say X) obtained by 200 students in a particular class. It is
assumed that the average marks of students are normally distributed with mean = 58.7
and standard deviation = 15.2. Find the average mark of a student at the 40th percentile.
In other words, we need to know the average mark of a student that separates the
bottom 20% of students from the top 80% of students.

Solution:
The following figure shows the normal curve with the unknown value of X separating the
bottom 20% of the distribution from the top 80% of the distribution.

The area closest to 0.20 in the standard normal table under column P is 0.1977. The
corresponding z value is -0.85. As the average marks (X) are distributed as normal with
mean 58.7 and standard deviation 15.2, the x value can be found using z = (x – µ)/ σ.
So, x = µ + z.σ = 58.7 +(-0.85)(15.2) = 45.78

So, the average mark of a student that separates the bottom of the data from the top 80%
is 45.78.

Unit 14 - Mathematics for Computing

19
Unit 14 - Mathematics for Computing
20
Unit 14 - Mathematics for Computing
21
4.12.4 Approximation of Normal Distribution to Binomial Distribution

Example :
Records show that 60% of the customers of a service station pay with a credit card. If 100
customers will come to the service station in next day , find the probability that at most 65
will pay with a credit card.
In order to find the solutions to the above scenario, Binomial distribution can be
considered.
• X = Number of customers will pay by a credit card
• p = 60% = 0.6
• n = 100
X ~ Binomial (n, p)
X ~ Binomial (100, 0.6)
100 𝑥
𝑃(𝑋 = 𝑥) = 𝑥𝐶 (0.6) (1 − 0.6)𝑛−𝑥 ; 𝑥 = 0,1,2, … ,120

P(X ≤ 65) = P(X = 0) + P(X = 1) + … P(X = 65)

100
= 0C (0.6)0 (1 − 0.6)100−0 + 1001C (0.6)1 (1 − 0.6)100−1 + …
+ 100 65
65C (0.6) (1 − 0.6)
100−65

Calculations are little difficult.

• Normal Distribution Approximation can be applied to a Binomial Distribution,
when n ≥ 30
np ≥ 5
np(1-p) ≥ 5
• Calculation are done by taking the parameters for the normal distribution as
µ = np
σ2 = np(1-p)
Solution:

Unit 14 - Mathematics for Computing

22
 References to the Recommended Resources

Textbooks
Stroud, K. A. (2009) Foundation Mathematics. Basingstoke: Palgrave Macmillan.

Journals
Journal of Computational Mathematics. Global Science Press.

Links
This unit links to the following related units:
Unit 18: Discrete Maths
Unit 33: Applied Analytical Models

Unit 14 - Mathematics for Computing

23