Expectation and Variance Algebra

-2-
Expectation and Variance Algebra
2.1 Mathematical expectation
As previously explained, the mathematical expectation of a random variable x can be defined
by:
n
   xi . f ( xi )
i 1
If x is a discrete variable, and:
b
   x. f ( x).dx
a
If x is a continuous variable (eqt.4.3)

In this chapter we shall use both notations μ and E(x) for expectation.
The following discussion will be based on discrete random variables. However, the results
obtained hold for continuous variables as well.
Now, there is a need to generalize the concept of expectation to the following:
Let g(x) be a function of the random variable x, then E(g(x)) is defined as follows:
N
E(g(x))=  g ( xi ). f ( xi ) (2.1)
i 1
If x is a discrete variable, and:
b

E(g(x)) =  g ( x). f ( x).dx

a
(2.2)

If x is a continuous variable
So far we have only dealt with one single random variable at a time. In practice, however,
there are many instances where more than one variable is encountered.
Let us assume that we are in presence of two random variables x and y.
Two cases are encountered:
First, x and y may be totally independent variables. An example is the number of
malfunctioning pumps in a production unit (A) and that of another production unit (B).
On the other hand, the two variables may be dependent in a more or less strong way. For
example, let x denote the mark of students in a course A and y their marks in another course B.
Normally these two variables will
be related somehow since an A – student is more likely to get good marks in both courses.
If the variables are dependent then they possess a joint probability function (or density
function in case of continuous variables): f(xi, yi).
If, on the other hand, they are independent, then this function can be written as a product as
follows:
f(xi, yi) = f(xi).g(yi)

9
 Expectation and Variance Algebra

The following example shows how the expectations of two independent variables are related.
Let x be a random variable of probability function f(x) and y be another independent random
variable of probability function g(y).
For example, the following table gives the probability function related to the random variable
x: “Number of pumps out of order” in pump house A, while
The next table gives that in a pump house B (y).

xi 0 1 2 3 4 5
f(xi) 0.15 0.35 0.2 0.15 0.1 0.05

yi 0 1 2 3 4 5
g(yi) 0.4 0.3 0.15 0.05 0.05 0.05

The variable x + y will represent the number of pumps out of order in both houses.
Now, E(x) = 1.85 and E(y) = 1.2
To get the distribution of the random variable x + y, we reason as follows;
x + y = 0 means that there are no pumps out of order in both houses. This means, according to
the law of independent events, that:
f(0) = 0.15×0.4 = 0.06
Similarly, f(1) = 0.15×0.3 + 0.35×0.4) = 0.185
Also, f(2) = 0.15×0.15 + 0.35×0.30 + 0.2×0.4 = 0.2075.
Similarly:
f(3) = 0.18, f(4) = 0.14, f(5) = 0.1075, f(6) = 0.065, f(7) = 0.03, f(8) = 0.015, f(9) = 0.0075,
f(10) = 0.0025
The following table summarizes these results:

x+y 0 1 2 3 4 5 6 7 8 9 10
f(x,y) 0.06 0.185 .2075 0.18 0.14 .1075 0.065 0.03 0.015 .0075 .0025

The value of E(x + y) as calculated = 3.05

This verifies the rule: E(x + y) = E(x) + E(y)
The following rules can be applied for mathematical expectation. They will be proved for
discrete variables although they apply for both types of variables.

Rule 1:
If C is a constant, then: E(C.x) = C.E(x) (2.3)
Proof:
E(C.x) = Σ [Link].f(xi) = C. Σ xi.f(xi) = C.E(x)

Rule 2:
If X and Y are two random variables, then:
E(x + y) = E(x) + E(y) (2.4)

10
 Expectation and Variance Algebra

Proof:
For simplicity, let x and y have the distributions:

xi x1 x2
f(xi) f(x1) f(x2)
And,
yi y1 y2
f(yi) f(y1) f(y2)

E(x) = x2.f(x1) + x2.f(x2) and E(y) = y2.g(y1) + y2.g(y2)

Now, x + y will have the following distribution:

x+y x1 + y 1 x1 + y 2 x2 + y 1 x2 + y 2
f(x, y) f(x1).g(y1) f(x1).g(y2) f(x2).g(y1) f(x2).g(y2)

Hence, E(x+y) = (x1 + y1). f(x1).g(y1) + (x1 + y2). f(x1).g(y2) + (x2 + y1). f(x2).g(y1) + (x2 + y2).
f(x2).g(y2)
= x2.f(x1).g(y1) + x2.f(x1).g(y2) + x2.f(x2).g(y1) + x2.f(x2).g(y2) + y2.f(x1).g(y1) + y2.f(x2).g(y1) +
y2.f(x1).g(y2) + y2.f(x2).g(y2)
Noting that f(x1) + f(x2) = g(y1) + g(y2) = 1, we get:
E(x+y) = x2.f(x1) + x2.f(x2) + y2.g(y1) + y2.g(y2) = E(x) + E(y)

Rule 3:
If X and Y are two independent random variables, then,
E(x.y) = E(x).E(y) (2.5)
Proof:
We follow the same logic as in the previous section.

x.y x2.y1 x2.y2 x2.y1 x2.y2

f(x, y) f(x1).g(y1) f(x1).g(y2) f(x2).g(y1) f(x2).g(y2)

E(x.y) = x2.y2.f(x1).g(y1) + x2.y2.f(x1).g(y2) + x2.y2.f(x2).g(y1) + x2.y2.f(x2).g(y2)

= x2.f(x1).[y1g(y1) +y2.g(y2)] + x2.f(x2).[y1g(y1) +y2.g(y2)]
= [x2.f(x1) + x2.f(x2)]. [y2.f(y1) + y2.f(y2)] = E(x).E(y)

2.2 Rules related to variance

As previously seen in a previous course, the variance of a discrete random variable is defined
by:
N
 2   xi2 . f ( xi )   2 (2.6)
i 1

The variance of a continuous variable was defined as:

11
 Expectation and Variance Algebra

b
   x 2 . f ( x).dx   2
2
(2.7)
a

The following notation can be adopted in both cases:

Var(x) = E(x2) – [E(x)]2 (2.8)
It is then obvious that if c is a constant, then Var(c) = 0
The following rules relate to variance:

Rule 4:
If C is a constant, then: Var(C.x) = [Link](x) (2.7)
Proof:
Var(C.x) = E(Cx)2 – [E(Cx)]2 = E(C2x2) – [C.E(x)]2 From rule 1:
Var(C.x) = C2.E(x2) – C2.E(x2) = [Link](x)

Rule 5:
If X and Y are two independent random variables, then,
Var(x ± y) = Var(x) + Var(y) (2.9)
This is sometimes written in the form:
σ2x+y = σ2x + σ2y

Proof:
Var(x + y) = E(x+y) 2 – [E(x+y)]2 = E(x2+y2+2xy) – [E(x+y)]2
From rules 1 and 2, we get:
Var(x + y) = E(x2) + E(y2) + 2.E(x.y) – [E(x)+E(y)]2
= E(x2) + E(y2) + 2.E(x.y) – [E(x)]2 – [E(y)]2 – 2.E(x).E(y)
From rule 3, E(x.y) = E(x).E(y), so that the last line simplifies to:
Var(x + y) = E(x2) – [E(x)]2 + E(y2) – [E(y)]2 = Var(x) + Var(y)
In the same way, we can prove that:
Var(x – y) = Var(x) + Var(y) (2.10)

6.3 The covariance

This is a measure of the degree of dependence of the two variables. It is defined as follows:
Cov (x, y) = E[(x – μx).(y – μy)] (2.11)

Rule 6:
Cov (x, y) = E(x.y) – E(x).E(y) (2.12)
Proof:
Cov (x, y) = E [(x – E(x)).(y – E(y))] = E (x.y) – E(x).E(y) – E(y).E(x) + E(x).E(y)
= E(x.y) – E(x).E(y)

Rule 7:
If X and Y are two dependent random variables, then,
Var(x + y) = Var(x) + Var(y) + [Link](x, y) (2.13)

12
 Expectation and Variance Algebra

The proof can be readily obtained from that of rule 5.

Similarly, we have:
Var(x – y) = Var(x) + Var(y) – [Link](x, y) (2.14)

Rule 8:
If x and y are independent, then Cov(x, y) = 0 (2.15)
This is readily deduced from (2.5) and (2.12)

Rule 9:
If x and y are linearly dependent then Cov(x, y) = σx.σy (2.16)
Proof:
Let y = a.x + b, hence: Cov(x, y) = E[x.(a.x + b)] – E(x).E(a.x + b)
= a.E(x2) + b.E(x) – a.[E(x)]2 – b.E(x) = a.(E(x2) – [E(x)]2) = [Link](x)
On the other hand,
σx.σy = [Var(x).Var(a.x + b)]1/2 = [Var(x).a2Var(x) + Var(b)]1/2 = [Link](x),
which concludes the proof.
The two latter rules (8 and 9) set a method for deciding about the degree of dependence
between two variables, where the coefficient known as the correlation coefficient is defined as:
𝐶𝑜𝑣(𝑥,𝑦)
R= (2.17)
𝜎𝑥 𝜎𝑦

Using EXCEL
The calculations related to multivariable expectations and variances are highly simplified by
the use of the functions: Average, standard deviation (Stdevp.), population variance (Varp.)
and covariance (Cov.).

Example 2.1
The silica content in drilling cuttings from a site (A) follows a normal distribution with mean
value = 36% and standard deviation = 2.4%, while corresponding results for site (B) show that
the mean silica content is 34.5% with a standard deviation of 2.2%.
Calculate the probability that a random sample taken from site B would have silica content
higher than that of a sample drawn from site A.

Solution:
Let x be the silica content of samples (A) and y that of (B).
Then: E(x) = 36, σx = 2.4 and E(y) = 34.5, σx = 2.2
The required probability is: P(y > x)
If we define a new random variable v = x – y, then the required probability is P(v < 0).
Now, E(z) = E(x) – E(y) = 1.5
The two variables being independent, then:
Var(z) = Var(x) + Var(y) = 2.42 + 2.22 = 10.6. Hence, σz = 3.26
Since normal distribution is assumed, hence using the function (Normdist.) with value = 0,
Mean = 1.5 and standard deviation = 3.26, we get: P(x < 0) = 0.323

13
 Expectation and Variance Algebra

Example 2.2
A company produces pigment batches by essentially mixing 3 ingredients (A), (B) and (C) in
the mass ration 1:4:5. It is known that all three materials contain impurities. Tests have shown
that the purity of each is normally distributed with the following mean values and standard
deviations:
μ σ
A 0.98 0.005
B 0.95 0.010
C 0.92 0.015

Find the probability that a randomly chosen batch will be at least 95% pure.

Solution:
Let x, y, z represent the fraction of pure materials in A, B and C respectively. The purity of the
batch can be expressed as:
v = 0.1x + 0.4y + 0.5z (The three variables being independent)
Since, E(v) = 0.1E(x) + 0.4E(y) + 0.5E(z), hence: E(v) = 0.938
Also, Var(v) = Var(0.1x) + Var(0.4y) + Var(0.5z) = 0.12×Var(x) + 0.42×Var(y) + 0.52×Var(z)
= 0.01×0.0052 + 0.16×0.012 + 0.25×0.0152 = 7.25×10-5 and σv = 0.008514
So, the purity of the batch (v) is a random variable with mean value = 0.938 and standard
deviation = 0.008514
The required probability is therefore: P(v > 0.95).
Since normal distribution is assumed, hence using the function (Normdist.) with value = 0.95,
Mean = 0.938 and standard deviation = 0.008514, we get: P(v < 0.95) = 0.9207, hence: P(v >
0.95) = 0.0793

Example 2.3
The year’s work marks of a class of 12 students were graded out of 30. Their final exam was
graded from 70. Their year’s work scores are shown below.

Y.W. 21 23 17 25 18 14 29 26 20 22 12 25

On the other hand, their final marks were randomly submitted as follows

Final 52 48 33 65 28 27 42 38 27 58 55 42

If it is known from previous practice that the correlation coefficient between the two scores
(year’s work and final) is about 0.7, give an estimate as for the probable percentage of students
who will get a total score exceeding 60.(Assume normal distribution prevailing)

Solution:
Let x = year’s work mark and y = final mark. For obvious reasons we cannot consider the two
variables to be independent.
E(x) = 21 and E(y) = 42.92 From rule (2): E(x + y) = 21 + 42.92 = 63.92
The population standard deviations are obtained as: σx = 4.848 and σy = 12.325

14
 Expectation and Variance Algebra

Squaring up, we get Varp(x) = 23.5 and Varp(y) = 152.91

From equation (2.17), Cov(x,y) = R.σx.σy = 0.7×4.848×12.325 = 42.826
From rule 7: Var(x + y) = 23.5 + 152.91 + 2×42.826 = 259.06 Hence sx + y ≈ 16.1
The required probability is: P(x + y > 60)
Assuming normal distribution of population with mean value = 63.92 and standard deviation =
16.1, we get: P(x + y < 60) ≈ 0.40
Hence the probable percentage of students getting a total score > 60 will be about 60%

2.4 Exercise problems

In all forthcoming problems the different variables will be assumed to be normally
distributed throughout the population.

(1) The following tables show the percent of P2O5 in two samples collected from two different
phosphate ores
A 32.5 30.9 33.4 32.7 32.3 33.1
B 29.6 30.4 32.1 32.6 32.0 32.1

Calculate the probability that a random specimen of B would have higher P2O5 content than
that of a random specimen from A.

(2) A company produces one nominal ton of powder material by mixing M1 kg of ingredient A
with M2 kg of B. The cost of A is 3.5 E.P. / kg and that of B is 10 E.P. / kg. If the nominal
values of M1 and M2 are 300 kg and 700 kg respectively, and their standard deviations 2.4
and 2 kg respectively, calculate the probable limits of the cost of one ton of powder.

(3) We consider the dust concentration at one km from a chimney (x) and its concentration at
one km of another chimney (y). Actual field data have shown that the mean value of x is
50 mg/Nm3 while that of y is 20 mg/Nm3. Their standard deviations are 4 and 3
respectively. Calculate the probability that the value of y be less than one third that of x, if
the two sets of data are independent.

(4) The following tables show data obtained for of the daily consumption of three raw
materials in 8 consecutive days (in ton):

x 23.5 22.7 22.8 26.2 25 22 20.5 25

y 16.5 19 14.6 15.2 18 13.7 12.8 14.5
z 43 45.5 40.6 38.8 42.6 44.5 45.6 48

If the costs per ton of all three materials (in E.P.) are:
x y z
1200 1800 800
Estimate the probability that the total daily cost of raw materials would lie within the range:
(80000, 90000).

15
 Expectation and Variance Algebra

(5) The total score of a student is calculated as 40% on quizzes and assignments, 20% on
midterm exam and 40% on the final exam. The following table summarizes the statistics
related to a certain course:

Task Quizzes Midterm Final

Mean score 32 14 32
Standard deviation 2.75 2.90 3.55
Determine the mean and standard deviation of the total score and estimate the number of
students obtaining A in that course (>90%).

16