E(X°) exists, V 1$s$r
Remark. The above result states that if the moments of a specified order exist, then all the
lower order moments automatically exist. However, the converse is not true, i.e, we may have
distributions for which all the moments of a specified order exist but no higher order moment
exist. For example, for the r.v. with pdf:
[2/8 5 x21
a) {2 xed
we have :
E(x) -| xpoyde=2 x? dx =|
1 1
(2-2
E(x) -{ = pcydr=2] Lice
, MATHEMATICAL EXPECTATION 6-9
Thus for the above distribution, Ist order moment (mean) exists but 2nd order moment
(variance) does not exist
As another illustration, consider a r.v. X with pdf.
p(x) = ner x20, a>0
Hy” Eorecrsna |” wayne
a
Put x = ay and using Beta integral : f oar B (m, n), we shall get, on simplification
0
wy’ =(r+1)a".B(r+1,1) =a"
However, nafszarnetrenes | irate on
as the integral is not convergent. Hence in this case only the moments up to rth order exist and
higher order moments do not exist.
Property 10. If X and Y are independent random variables, then
E(h(X). kK =E fh (XI ELK) ~» (6-28)
where h(.) is a function of X alone and k(.) is a function of Y alone, provided expectations on
both sides exist.
Proof. Let fx(x) and gy(y) be the marginal p.d,f’s of X and Y respectively. Since X
and Y are independent, their joint [Link]. fury (x,y) is given by :
fay) = fx fr) =)
By def., for continuous r.v.’s
E10). O) = J |” Hey Ky) sls waa
= [J nearer sw dry tFom C9
Since E [h (X) k (Y)] exists, the integral on the right-hand side is absolutely
convergent and hence by Fuibini’s theorem for integrable functions, we can change
the order of integration to get
emoorond| _reyendel[ { “a gcndd] = 20001-8800
as desired.
_ Remark. The result can be proved for discrete random variables X and Y on replacing
integration by summation over the given range of X and Y.
6-5. PROPERTIES OF VARIANCE
If X isa random variable, then V (aX + b) = a2 V(X), ++(6:29)
where a and b are constants.
Proof. Let Y =aX +b. Then E(Y) =a (X) +b
Y-E(Y)=a[X-E(X)]
Squaring and taking expectation of both sides, we get a
6-10 FUNDAMENTALS OF MATHEMATICAL STATISTig,
E[Y-E()P=@E[X-E(X)P
> V(Y)=@ V(X) or V(aX +b) =a? V(X),
where V (X) is written for variance of X.
Cor. (i) fb =0, then V (aX) =a? V (x) +++ (6-25)
=> Variance is not independent of change of scale.
(ii)If a =0,then V(b)=0 => Variance of a constant is zero. + (625)
(iii) If a =1,then V(X+b)=V(X) + (6:28,
=> Variance is independent of change of origin.
6-6. COVARIANCE
If X and Y are two random variables, then covariance between them is defined as
Cov (XY) = E[[X-E(X)}{Y- EM] + (6-3)
=E[XY-XE(Y)-YE(X) +E (X)E(Y))
=E(XY)-E(Y)E(X)-E(X)E( +E (XE)
=E(XY)-E(X)E() «= (6304)
If X and Y are independent then E (XY) = E (X) E (Y) and hence in this case
Cov (X,Y) = E(X)E(Y) -E(X)E(Y)=0 +» (6-306)
, Remarks 1. Cov (aX,bY) = E[laX—£ (aX)} {bY-E YH]
Ela (X-E(X)}b(Y-E(Y))]
ab E((X-E(X)(Y-E()
ab Cov (X,Y) (631)
2 Cov(X +a, ¥ +8) = Cov(XY) .. (6313)
3. cov ( a 4, = teow (x,y .. (6318)
ox" oy | oxoy
4. Similarly, we shall get :
Cov (aX +b, cY +) = acCov (X,Y) (6319
Cov (X4Y,Z) = Cov (X, Z) +Cov (Y, Z) (6314
Cov (aX + bY, eX +dY) = a00,2+ boy? + (ad + be) Cov (X, Y) (6310
5. If X and Y are independent, Cov (X, Y) =0.. {c-f. (630b)]. However, the converse is not
true. For illustrations see Chapter 10 on Correlation.
6-6-1. Variance of a Linear Combination of Random Variables
Let X;, Xp, wy Xq be n random variables, then
non
; Vv (2 4 x) == ar V(X) +2 x 4; a; Cov (X;, X)) wn (632)
lisp
Proof. Let U = a, X, +0) Xo +... ay Xy,
so that E(U) = ay E(X1) + a2 E (Xp) + 1. + dy E (X,)
U-E(U) = ay (X1~E(X)} +a (Xp E(Xq)) +. tay (Xp —E (Xl)
Squaring and taking expectation of both sides, we get
E(U~E (U)P = ay? E[X; ~E (Xi)P + ap? E [Xp ~ E (Xa) +... + 42 E [Xp - E (XW?
+2 a aa) E [(X,~E(X)} (X,- E(X))] ie
MATHEMATICAL, EXPECTATION 6-11
=p VU) =a? V(X) + a9? V(X) +. ay? V(X) +2 LL aaj Cov (X;, X})
iat jat
tej
nr n n n
= v( ZX a X= L a2 V(X)+2 ZL ajajCov(X, X))
i=l ist isl j=l
ie} |
Remarks 1. Ifo,=1;i=1,2,..,n, then
VOX Xt AX) =VORYHVOG) + FVOKG)EZ BE CoveXy X) (632)
hi
igj
2, IEXy, Xp, Xp are independent (pairwise), then Cov (X,, X)) = 0, (i#).
Thus from (632) and (6-322), we get
V(X, + 0gX_ + +. X,) =)? V(X) +03? V(X) +. #.4,? V (X,)
and V(X + Xp X= V(X) + VK) +. V(X),
provided X;, X,,.... X, are independent.
3.1f a= 1 =a, and a,=4, =... =a, =0, then from (6:32), we get
V(X, +X,) = V(X) + V(X) + 2 Cov (Xy, X2)
Again ifa, = 1, a.=-1 anda, =a,=...=a, =0, then
V(X -X,) = V(X) + V(X) -2 Cov (X,, X;)
Thus we have
V(X, £X,) = V(X,) + V(X.) £2Cov (X,, X;) «- (632c)
IfX, and X; are independent, then Cov (X;, X;) = 0 and we get
7 V(X,£X,) = V(X) + V(X) (6324)
“Example 6-1. Let X be a random variable with the following probability distribution :
x : 3 6 9
P(X=x) : 1/6 1/2 18
Find E (X) and E (X2) and using the laws of expectation, evaluate E (2X + 1)2.
Solution. (x) =Lxple)=(-3)xF 46x 549% he
.. (632b)
EOC) =Satptx)=9x1436x L481 x
EQX+1p = EX? + AX +1) = 46 (X2) +46 (xX) +1 = 4x 44x 41 = 209
‘Example 6:2. (a) Find the expectation of the number on a die when thrown.
(b) Ti i ;
poy z wo unbiased dice are thrown. Find the expected values of the sum of numbers of
Solution. (a) Let X be th i
when thrown. Thee X co a ae variable representing the number on a die
any one of the values 1, 2, 3, ..., 6 each wi a
probability}. Hence y e values each with equal
C 1 1 1
E(X)=l xretygyl .
(x) 6x1 x24 6x34 4h oa Made de +6) 16x77 at) 6-12 FUNDAMENTALS OF MATHEMATICAL STATISTICS
Remark. This does not mean that in a random throw of a dice, the player will get the
number 5. In fact, one can never get this (fractional) number in a throw of a dice. Rather,
this implies that if the player tosses the dice for a “long” period, then on the average toss he will
(b) The probabili
function of X (the sum of numbers obtained on two dice), is
egxx 12 [3 | +t Te{7{~[utn
Probability | 1/36 | 2/36 | 3/36 | 4/36 576 | 6/36 [oe | 2/36 | 1736
E(X) = Lx
6
nada Zod R 5x a r6x 5 7% Hp 48 3 + 9S
+10xZ4 Ux H+ 1x y
= xe +6+12+20 +30 +42 + 40 + 36 +30 + 22+ 12)= 4x252=7
Aliter. Let X; be the number obtained on the ith dice (i = 1, 2) when thrown. Then
the sum of the number of points on two dice is given by :
7,7
S =X,+X_ => E(S)=E (Xi) +E (X)= 7+ 9 =7 [On using (*)]
Remark. This result can be generalised to the sum of points obtained in a random throw of
n dice. Then
. 5 Z_ In
E(S) = LDE(K)= L a=
i=l isl
Example 6-3. In four tosses of a coin, let X be the number of heads. Tabulate the 16
possible outcomes with the corresponding values of X. By simple counting, derive the
probability distribution of X and hence calculate the expected value of X.
Solution. Let H represent a head, T a tail and X, the random variable denoting the
number of heads.
{[Link] Outcomes — No. of Heads (X) [Link]. Outcomes No. of Heads (X))
; HHHH 4 9 HTHT 2 |
| 2 HHHT 3 10 THTH 2 |
| 3 HHTH 3 n THHT 2 |
| 4 HTHH 3 2 HTTT 1 |
| 5 THHH 3 13 THTT 1 |
| 6 HHTT 2 “4 TTHT 1
| 7 HTTH 2 15 TTTH 1
| 8 ITHH 26 TITT 0 |
“The random variable X takes the values 0, 1, 2, 3 and 4. Since, from the above
table, we find that the number of cases favourable to the coming of 0, 1, 2, 3 and 4
heads are 1,4,6, 4nd 1 respectively, we have
1 4 1
P(X=0)= 4g P(X= = ie= 4g, P(X == 6
4
P(X =3)= 76 = 1, P(X=4)= i.
The probability distribution of X can be summarized as follows :
x ; 1 2 3 4
1 3
px) : 1 i
a 4 8 4 16 MATHEMATICAL EXPECTATION 6-13
‘
3 1 3.3
E(x) = L xpix)=t-p42 gts hed igeptgtat y=?
x=0
Example 64. An urn contains 7 white and 3 red balls, Two balls are drawn together, at
random from this urn. Compute the probability that neither of them
is white. Find also the probability of getting one white and one red ball. Hence compute the
expected number of white balls drawn
Solution, Let X denote the number of white balls drawn. The probability
distribution of X is obtained as follows:
x: 0 1 2
: 5 1 70, °C, 1G 7
p(x) WC, = 15 mC, = WC, = 15
Then expected number of white balls drawn is :
1 7 7 _ 2
E(X)=0x75 +1x 7g+2 75 = 55
Example 6:5. A gamester has a disc with a freely revolving needle. The disc is divided
into 20 equal sectors by thin lines and the sectors are marked 0, 1, 2, ., 19. The gamester
treats 5 or any multiple of 5 as lucky numbers and zero as a special lucky number. He allows a
player to whirl the needle on a charge of 10 paise. When the needdle stops at the lucky number
the gamester pays back the player twice the sum charged and at the special lucky number the
gamester pays to the player 5 times of the sum charged. Is the game fair ? What is the
expectation of the player ?
Solution.
[ Event Favourable p(x) Player's Gain (x)
[Lucky number 5,10, 15 3/20 — 20-10=10p
| Special lucky No. 0 1/20 50-10=40p
‘Other numbers | 1,2,3,4,6,7,8,9, 11,12, 16/20 -10p
13,14, 16,17, 18,19 |
E(X) = Bx 10+ dy 40~ 38x 10 =— 50, ie, the game isnot fir
Example 6:6.A box contains 2"tickels among which "C; tickets bear the number
j;i=0, 1,2, .-.m. A group of m tickets is drawn. What is the expectation of the sum of their
numbers ?
Solution. Let X;; i=1, 2, ..., m be the variable representing the number on the ith
ticket drawn. Then the sum ‘S’ of the numbers on the tickets drawn is given by :
7 ®
S =Xy+Xpt..4X,= L X;, sothat £(S)= LY £(X))
in ist
; X;is a random variable which can take any one of the possible values 0), 1, 2, ..., 11
with respective probabilities :"Cy/2", "C,/2","C)/2", ..., "C,/2",
1
E (Xi) = oy (1"Cy +2. "Cy +3."Cy +2 +H",
1
= pe (Lne 2 SED yg mn a)
n
= jell + (n-1) + mands +a} 6.14
1
= on
». ES) = L E(X)= L n="
1 int
ti
an (MICA IC HMI,
FUNDAMENTALS OF MATHEMATICAL STATISTICS
Example 6-7. A coin is tossed until a head appears. What is the expectation of the
number of tosses required ?
Solution. Let X denote the number of tosses required to get the first head. Then X
can materialise in the follawing ways:
[ Event | — Probability, p(x)
H 1
| 2
TH 111
| 2%274
TTH dyad
2%2%275
E(X) =D xp(x)eix beaxbeaxteaxde..
x)= 2 Po) 2 4 8 16
0
This is an arithmetic-geometric series with ratio of GP being r=
epee2-te3 tag b
Let S =1ly+2 qt3 gt iet
Then
a-
[Since the sum of an infinite G. P. with first term a and common ratio (<1) is 7,
Hence, substituting in (*), we have E(X)=2.
8. What is the expectation of the number of failures preceding the first
with constant probability p of success in each
Example 6-
success in an infinite series of independent trials
14,4
H2-g43. 76+
or $=2.
J)
trial ?
Solution. Let the random variable X denote the number of failures preceding the
first success. Then X can take the values 0, 1,2, «», 2. We have
P(X =x) =p (2) = P(x failures precede the first success) = 9" P,
is the probability of failure in a trial. Then by def,
where q = 1-p,
E()= L expe L x gtpepq L xq t= pq +24 3p + 4p +) w()
x=0 x=0 rel
Now 1 +29 +3q? + 4g? +... is an infinite arithmetic-geometric series.
Let S =142q +37 + 4p +
qs = q+ UP + 3p +.
(1-gS =l+q+@rgt- 7
+ [From (*)]
1+ 2q +39? +493... =
Hence E (X) NER EE EEEEEEEEEEETTDEN
MATHEMATICAL EXPECTATION 615
Example 6-9. A box contains ‘a’ white and ‘b’ black balls. ‘c’ balls are drawn at random
Find the expected value of the number of white balls drawn.
Solution. Let a variable X,, associated with ith draw, be defined as follows
1, if ith ball drawn is white
i= { 0, if ith ball drawn is black
Then the number ‘S’ of the white balls among ‘c’ balls drawn is given by :
s ehitkrntke Ex = E=E E(X) Lt)
Now P (X;=1) =P (of drawing a white ball) = “at
and P (X,=0) = P (of drawing a black ball) =
E(X) =1.P(Xj=1) 40. P(X)=0)=22¢ |
Hence E(S)= x (=) = a [From (*)]
=I
Example 6-10. Let the rv. X have the distribution :
P(X =0)=P(X=2)=p;P(X=1)=1-2p,for0 ps}.
For what p is the Var (X) a maximum ?
Solution. Here the r.v. X takes the values 0, 1, and 2 with respective probabilities
p.1~2pand p, 0p}. Thus
E(X)=Oxp+1x(1-2p)+2xp =LE(X*)=0xp+I?x(1-2p)+2xp=1+2p
Var (X) = E(X2)~{E (X)}=2p; Op < }
Obviously, for 0 E(8)= % E(x)
int
Now E(X) = LP (K=1)+0.P(%=0)=P(X,=1)=4
Hence E() = 2 (j)=n
V(S) = V(XY+X+..+X) =D V(X)+2E ¥ Cov (X),X)) (1)
ist inn jai
vey
Now
V(X) =E(KA-(E OP = 12. P(X, =1) +02. P(X =0)~(1)°= ---Q)
Cov (Xi, X)) = E (XX) - E(X) E (X) -@)
EQGX)=1.P (GX, =1)40.P(Xxi=0) = MHA _ wo"
since X, X;= 1 if and only if both card numbers i and j are in their respective matching
Places and there are (n - 2) ! arrangements of the remaining cards that correspond to
this event. Substituting in (3), we get
1 11 1
COV (Xi X) Tay ~ n= OY ~&
Substituting from (2) and (4) in (1), we have
y (n=1 y 1
= = en (t
VOS)= 2 (=) 2d 4 {r=} =n (2
iLVt>0 = etel
Also
or l-et>0
ete} 50, vis0
Hence P(x) = et(1-e)-1 > 0, Vt>0,x=1,2,3,... 6-1
8 FUNDAMENTALS OF MATHEMATICAL STATISTICS
Also = -L p(x) set L (1-ety- =e Lo 1, (@=1-¢'
rel rel rel
set(ltatatea4..)ee
Jectiaty
set[1-(l-e)} set =
Hence p (x) defined in (*) represents the probability function of a r.v. X.
E (X) =Ex-pQjeet L xQ-ey tect Z [Link] (a=1-e")
xel xel
set (1 +204 3024 4+...) =e! (1-a)? (See Remark 1)
set(etyt=e!
E(X2) =Exep(xyset L [Link]-t set (1+ 4a +90? + 160° +...)
ral
=e*(1 +a) (1-ay3=et (2-e*) e* (See Remark 2)
Hence Var (X) = E (X2)-[E (x) =e! (2-et) e¥-e
= [(2-e4)-i] =e (1-e) =e (¢-1).
Remarks 1. Consider
S$ =1+20 +30? +4a° +... (Arithmetic-geometric series)
= aS = at+2a+30+.
1
= (l-a)S =l+a+a@+a+.. ay 7 S=(1-a)?
YD xa) = 1420+ 3a? + da? +... = (1-0)? °
rel
2. Consider
S$ =14+ 2.043224 h.+5? at...
= S =1+ 40+ 9a? + 160° + 25a* +...
—3a$ = -3a- 12a?- 270 - 48a‘-...
+305 = + 3a? + 120° + 2704 +...
-@S = - @- 4a-
‘Adding the above equations, we get
(-apS =1+a = S=(1ta)(1-ay? a (")
Z xtat-) =1 +40 49a? + 160° + 1. =(1 4a) (1-ay?
yl
‘The results (*) and (*4) are quite useful for numerical problems and should be committed to
memory:
Example 6-16. A man with n keys wants to open his door and tries the keys
independently and at random. Find the mean and variance of the number of trials required to
open the door, (i) if unsuccessful keys are not eliminated from further selection, and (ii) if they
are.
Solution. (i) Suppose the man gets the first success at #th trial, ie., he is unable to
the door in the first (x ~ 1) trials. If unsuccessful keys are not eliminated then X is
a random variable which can take the values 1, 2,3,... Oe eE—-tt—te
6-19
MATHEMATICAL EXPECTATION
1
Probability of success at the first trial = hn
1
"
If unsuccessful keys are not eliminated then the probability of success and
consequently of failure is constant for each trial.
Probability of failure at the first trial = 1 ~
Hence p(x) = Probability of 1st success at the x th trial = (1 =
Thus
= x-1 1
E(x)= L xpQ) y x(1- 3) tel EY var where A=1- 1.
a1 5
E(X) = 142A +342 4449+.) =L LA)? [See (1), Example (615)]
2
1 1
=afr-(0- ‘} :
_ . x-1
E(X%)=L x.p(xye L 2 (1- 2) 7
fap Pen
eh OD ar tehae 2. Age. alee Ade.)
pea
La+aya-ay [See (**), Example (6.15)]
a{t+(- )}p-G- = enn
Hence V(X) = E(X%)—(E (X)}?=(2n-1) n-n=n-n=n (1-1)
(ii) If unsuccessful keys are eliminated from further selection, then the random
variable X will take the values from 1 to n. In this case, we have
Probability of success at the first trial =1
"
Probability of success at the 2nd trial
Probability of success at the 3rd trial
and 0 on.
Hence probability of 1st success at the 2nd tral = (1 -
Probability of first success at the third trial = (1-
and so on. In general, we have
P (3) = Probability of first success at the xth trial =!
E(X)= Zo xp@yeh DY yontt
xel a
EQ) = XY tpqel Feo ev enn
oMs Oo ss
6-20 FUNDAMENTALS OF MATHEMATICAL STATISTICS
wager) (net)
Hence V(X) = E(X?)-(E(X)P = t
1 r= 1
= "Eh 2 Qn41)-3 (n+) ="
Example 6-17. Ina lottery m tickets are drawn at a time out of n tickets numbered 1 to
n. Find the expectation and the variance of the sum $ of the numbers in the tickets drawn.
Solution. Let X; denote the score on the ith ticket drawn.
Then $= X1+Xz+...+Xm = X;jis the total score on the m tickets drawn.
fal
E(S) = & E(X)
i=l
Now each X; is a random variable which assumes the values 1, 2, 3, ... ," each
with equal probability 1/n.
5 E(X) = ba+2+3+..¢m)=0).
roy 3 (2)
Hence,
isl
V(S) =V (Xy + X24 0+ +Xm) = 2X V(X) +2 LY Cov (X;, X))
i=l ij
ig
EKA = bate eegte. may. Maen nel) GeV enet
_ _(n+1)Qn+1)_(n+1)_ 1
V(x) = EKA -HE OKgR =e AnD (mgt) = ST
Also Cov (X;, Xj) = E (XX) - E(X) EX)
To find E (X; X)), we note that the variables X; and X; can take the values as
shown below :
x x,
1 23,00
2 13,0
1 12... (0-1)
In other words, the variable X;Xj can take n(n-1) possible values and
P(X; = 1X, =) = gqgy kth Hence
1:2 413 + cesses tee Lat
F 21D +23 $c He
EQX)=aqcay | *
# ML + MQ FH ooosceene HM. (N-1) MATHEMATICAL EXPECTATION
6-21
1(1+2434....4n)-22]
+2(14+2434 +n)-22
1 tee
n(n)
tn(l+2+. +n)=n?
1
= n(n)
1 [{* nen)" nines erst] ato Bn? ~ n - 2;
(rs 1) mn + D@n +1) (+1) Gr —n=2)
{(1+243+...m?-(12 4224. 4nd)}
=n(n=) 2 1207-1)
. voy yy 2 et WGrt=n=2)_(n+1)?__ (nt)
© Cov (XX) = EMG sna 2_ (251). HD [ant _2-3 (nt —1)] =- OD
_¥ (ma [+1] _m(?-1) -1f-
Hence V(S) = x ( TZ )+2 BE {- 12 J. 2 +[Link] {=n},
[Since there are "C; covariance terms in Cov (X; X))]
Vis) = MED [—1)-(m-n] = Mer e=m,
Example 6-18. A die is thrown (n + 2) times. After each throw a ‘+ is recorded for 4, 5,
or 6 and ‘~’ for 1, 2 or 3, the signs forming an ordered sequence. To each, except the first and
the last sign, is attached a characteristic random variable which takes the value 1 if both the
neighbouring signs differ from the one between them and 0 otherwise. If Xr, Xz, .... Xy are
characteristic random variables, find the mean and variance of X = © X,
iat
f f
Solution. x= lx = EwH= LEX) .Ai)
ia a
Now —_ E(X)) = LP (X;=1) +0.P (X= 0) = P(X)=1) i)
For X,=1, there are the following two mutually exclusive possibilities :
@- + 4 j@) + - +
and since the probability of each sign is, we have by addition probability theorem:
2 ays -
p(x=1) =P +P 4) +(3) = 5 ii)
E(x) = 1 [From(iy), — Hence E(X)= X(3) oh [From ()]
fal
V(X) =V(X1+ X24... + Xn) = Lv(Xp+2 zy x Cov (X, XC)
ml jet je
Now = E(X?)_ = 12.P (X;=1)+ 0%. P(X)=0)= [From (iti)]
1 3
V(X) =E(XA)-(E (X)P=4 ~ 16 = 16
E(X)X) = LP (X)=10Xj=1) + 0.P (X= 00%)=9)
OP (X= 10X)=0)40-P (X= 00% = 1)
=P(Xj=10 X=)
(j=) =
6