1

STATISTICAL INFERENCE

We know that statistical data is nothing but a random sample of observations drawn from a
population described by a random variable whose probability distribution is unknown or partly
unknown and we try to know about the properties of the population on the basis of knowledge of the
properties of the sample. This inductive process of going from known sample to the unknown
population is called ‘Statistical Inference ‘

Formally, let x be a random variable describing the population under investigation. Suppose X has
þ.m.ƒ 𝑓𝑜(𝑥) = 𝑃(𝑥 = 𝑥) or þ d ƒ 𝑓𝑜(𝑥) which depend on some unknown parameter 𝜃 (single or vector
valued) that may have any value in a set Ω (called the parameters space). We assume that the
functional form of 𝑓𝑜(𝑥) is known but not the parameter 𝜃(except that 𝜃 ∈ Ω). For example, the family
of distributions {𝑓𝜃(𝑥), 𝜃 ∈ Ω} may be the family of Poisson distribution {𝑃(𝜆), 𝜆 ≥ 0} or normal
distribution {𝑁(𝜇, 𝜎2),−∞ <𝜇 < ∞, 𝜎 ≥ 0}

Two problem of statistical inference are-

1. To estimate the value of 𝜃 − problem of estimation

2. To test a hypothesis about 𝜃 - problem of testing of the hypothesis

POINT ESTIMATION

Definition: A random sample of size ‘n’ from the distribution of X is a set of independent and
identically distributed random variables {𝑥1, 𝑥2, … , 𝑥𝑛} each of which has the same distribution as that
of X. The probability of the sample is given by

𝑓𝑜(𝑥1, 𝑥2, … , 𝑥𝑛) = 𝑓𝑜(𝑥1)𝑓𝑜(𝑥2) … 𝑓𝑜(𝑥𝑛)

Definition: A statistic T = T (x1,x2,…, xn) is any function of the sample values, which does not depend
on the unknown parameter 𝜃. Evidently, T is a random variable which has its own probability
distribution (called the ‘ Sampling distribution’ of T)

For example, are

some statistics.

If we use the statistic T to estimate the unknown parameter 𝜃, it is called the estimator (or point
estimators) of 𝜃 and the value of T obtained from a given sample is its ‘estimate’

Remark: Obviously, for T to be a good estimator of 𝜃 , the difference [𝑇 − 𝜃] should be as small as

possible. However, since T is itself a random variable all that we can hope for is that it is close to 𝜃
with high probability.
Theorem : Let (X1, X2,…,Xn) be a random sample of ‘n’ observations on X with mean 𝐸(𝑋) = 𝜇 and

variance 𝑉𝑎𝑟(𝑥) = 𝜎2 Let the sample mean and sample variance be

 2

Then,

(I)E(̅𝑋)̅ =𝜇

Prof: We have

= 𝑛𝜎2 − 𝑛𝜎2/𝑛

= (𝑛 − 1)𝜎2

PROPERTIES OF ESTIMATORS

UNBIASEDNESS:

An estimator T of an unknown parameter 𝜃 is called unbiased if

𝐸(𝑇) = 𝜃 for all 𝜃 ∈ Ω

𝐸𝑥𝑎𝑚𝑝𝑙𝑒. If (𝑥1, 𝑥2, … , 𝑥𝑛) is a random sample from any population with mean 𝜇 and variance 𝜎2, the
sample mean 𝑥̅ is an unbiased estimator of 𝜇 but the sample variance 𝑆2 is not an unbiased estimator
of 𝜎2.

However, is an unbiased estimator of 𝜎2.

 3

𝐸𝑥. if (𝑥1, 𝑥2, … 𝑥𝑛) is a random sample from 𝑎 normal distribution 𝑁(𝜇, 𝐼) show that 𝑇 =

is an unbiased estimator of 𝜇2 ,

Soln.

E(xi2)= V(x)+E(xi)= (𝜇2 + 1)

𝑬𝒙𝒂𝒎𝒑𝒍𝒆: Let (𝑥1, 𝑥2, … 𝑥𝑛) be a random sample of observation from a Bernoulli distribution ƒ𝜃(𝑥) =

𝜃𝑥(1 − 𝜃)1−𝑥(𝑥 = 0,1) show that is an unbiased estimator of 𝜃 where 𝑦 = ∑𝑖𝑛 𝑥𝑖

Soln: We know that 𝐸(𝑥𝑖) = 𝜃 and 𝑉(𝑥𝑖) = 𝜃(1 − 𝜃) so that 𝐸(𝑌) = 𝑛 𝜃 and 𝑉(𝑌) = 𝑛𝜃(1 − 𝜃)

Now

𝐸(𝑌(𝑌 − 1) = 𝐸(𝑌2) − 𝐸(𝑌)

= 𝑉(𝑌) + [𝐸(𝑌)]2 − 𝐸(𝑌)

= 𝑛𝜃(1 − 𝜃) + 𝑛2𝜃2 − 𝑛𝜃

= 𝑛(𝑛 − 1)𝜃2

E(T)=

Showing it to be an unbiased estimator of 𝜃2

Example: Show that the mean 𝑥̅ of a random sample of size 𝑛 from the exponential distribution

ƒ is an unbiased estimator of 𝜃 and has variance 𝜃2/𝑛

Soln: We know that

𝐸(𝑥𝑖) = 𝜃 and 𝑉(𝑥𝑖) = 𝜃2 (𝑖 = 1, . . , 𝑛)

𝐸(̅𝑋̅)= 𝜃 and 𝑉(̅𝑋)̅ = 𝜃2/𝑛

Example: Let (𝑥1, 𝑥2, … 𝑥𝑛) to a random sample from a normal distribution with mean 0 and variance
𝜃 (0< 𝜃 < ∞) so that is an unbiased estimator of 𝜃 and has variance 2𝜃2/n Sohm we know
that
 4

Also

Example Let (𝑥1, 𝑥2, … 𝑥𝑛) be a random sample from the rectangular distribution 𝑅(0, 𝜃) having

þ, 𝑑, 𝑓

Show that and 𝑇3 = (𝑛 + 1)𝛾𝑖 are all unbiased for 𝜃 , where 𝑌1 =

min(𝑥1, 𝑥2, … 𝑥𝑛) and 𝑌𝑛 = max( 𝑥1, 𝑥2, … 𝑥𝑛)

Soln: We know that

𝐸(𝑥) = 𝜃/2 and 𝑉(𝑥) = 𝜃2/12

To obtain the expectation of T2 and T3 we need to obtain their distribution. The

𝑑. 𝑓. of Yn is-

𝐹𝓎(𝓎) = 𝑃(𝑌𝑛 ≤ 𝓎)

= 𝑃(max (𝑥1, 𝑥2, … 𝑥𝑛) ≤ 𝓎)

= 𝑃(𝑥𝑖 ⪕ 𝓎, 𝑥𝑛 ⪕ 𝓎)

= [𝑃(𝑥 ⪕ 𝓎)]𝑛

þ, 𝑑, 𝑓 𝑜𝑓 𝑌𝑛 is-
 5

Hence,

Or

So that T2 is unbiased for 𝜃

[We can check that V (T2) =

Again, the 𝑑. 𝑓. of Yi is-

𝐹𝑌𝑖(𝓎) = 𝑃{𝑌𝑖 ≤ 𝓎}

= P{min (𝑥1,𝑥2, … 𝑥𝑛) ≤ 𝓎}

= 𝐼 − 𝑃{𝑥1 > 𝑦, 𝑥2 > 𝑦, … 𝑥𝑛 > 𝑦}

= 𝐼 − [𝐼 − 𝑃(𝑋 < 𝓎)]𝑛

þ, 𝑑, 𝑓 of 𝑌𝑖is

Hence,

So that 𝐸(𝑇3) = 𝐸[(𝑛 + 1)𝑌1] = 𝜃

Example: Let ((𝑥1, 𝑥2, … 𝑥𝑛) be a random variable from the Rectangular distribution 𝑅(𝜃, 2𝜃) having
þ, 𝑑, 𝑓
 6

Show that

And are all unbiased

Soln: We can show that the distribution ƪ𝑥(𝑛)𝑑𝑥(𝑖) have þ, 𝑑, 𝑓 given by

Example: Let 𝑦1, 𝑦2,𝑦3 be the order statistics of a random sample of size 3 from 𝑎 uniform

distribution having þ, show that 4 are all unbiased estimator

of 𝜃. Also obtain their variance.

Soln: We can show that 𝑌1, 𝑌2, 𝑌3 have þ, 𝑑, 𝑓

𝐸(𝑦1) = 𝜃/4, 𝐸(𝑦2) = 𝜃/2 ,𝐸(𝑋3) = 3⁄4𝜃

𝑉(𝑦1) = 3𝜃2/80, 𝑉(𝑦2) = 𝜃2/20, 𝑉(𝑦3) = 3𝜃3/80

*If 𝑦1,𝑦2,… , 𝑦𝑛 are two unbiased estimator with variance and correlation coeff. P between than
the linear combination which is unbiased and has minimum variance is.

*If 𝑦1,𝑦2,… , 𝑦𝑛 are ind ept unbiased estimators if 𝜃 with variance the linear
combination with minimum variance is

𝑌 = 𝓀1𝑦1 + 𝓀2𝓀2 + +𝓀𝑛𝓀𝑛

Where

Example Let ‘T’ be an unbiased estimator of . Does it imply that and , are unbiased for
respectively?

Soln :
 7

If , then so that which is impossible since T has to be of

independent of .

Also,

If , then so that which is impossible.

Example let , be independent unbiased estimator of , having finite variance .

Obtain a linear combination of which is unbiased and has the smallest variance.

Sohn Let

Evidently, or

Then

Minimising we get

Or

The linear combination with minimum variance is

Note : then =1/3

Remarks: An unbiased estimator may not exist. Let x be a random variable with Bernoulli
distribution.

It can be shown that no unbiased estimator exists for 𝜃2.

(𝑖𝑖) Unbiased estimator may be assured.

Let X be a random variable having Poisson distribution 𝑃(𝑥) and suppose we want estimator 𝓰(𝜆)
=ℯ3𝜆. Consider a sample of one observation and the estimator T= . Then E(T)= ℯ−3𝜆 so that T is an
unbiased estimator of ℯ−3𝜆 but T(x)= (-2) X for x even and T(x) < 0 for 𝑥 odd, which is absurd since
ℯ−3𝜆 is always positive.

(𝑖𝑖𝑖) Instead of the parameter 𝜃 we may be interested in estimating a function 𝓰(𝜃). 𝓰(𝜃) is said to
be ‘estimable’ if there exists an estimator T Such that E(T)= 𝓰(𝜃), 𝜃 ∈ 𝛺.

Minimum Variance Unbiased (MVU) estimators : The class of unbiased estimators may, in general,
be quite large and we would like to choose the best estimator from this class. Among two
 8

estimators of 𝜃 which are both unbiased , we would choose the one with smaller variance. The
reason for doing this rests on the interpretation of variance as a measure of concentration about the
mean. Thus, if T is unbiased for 𝜃, then by Chebyshev’s inequality-

Therefore, the smaller 𝑉𝑎𝑟(𝑇) is, the larger the lower bound of the probability of concentration of T
about 𝜃 becomes. Consequently, within the restricted class of unbiased estimators we would choose
the estimator with the smallest variance.

Definition: An estimator T =T (X1,…, Xn) is said to be a uniformly minimum variance unbiased

(UMVU) estimator of 𝜃 (or an estimator for 𝓰(𝜃) if it is unbiased and has the smallest variance
within the class of unbiased estimators of 𝜃 (or 𝓰(𝜃),) of all 𝜃 ∈ 𝛺. That is if T is any other unbiased
estimator of 𝜃, then-

𝑉𝑎𝑟(𝑇) ≤ 𝑉𝑎𝑟(𝑇′)𝑓𝑜𝑟 all 𝜃 ∈ 𝛺

Suppose we decide to restrict ourselves to the class of all unbiased estimators with finite variance.
The problem arises as to how we find an UMVU estimator, if such an estimator exists. For this we
would first determine a lower bound for the variances of all estimators (in the class of unbiased
estimators under consideration) and then would try to determine an unbiased estimator whose
variance is equal to this lower bound. The lower bound for the variances will be given by the Cramer-
Rao inequality for which we assume the following regularity conditions:

Let X be a random variable with þ.𝑑. 𝑓 𝑓(𝑥; 𝜃) 𝜃 ∈ 𝛺

(𝑖) 𝛺 is an open interval (finite or not )

(𝑖𝑖)𝑓(𝑥; 𝜃) is positive on a set S independent of 𝜃.

exists for all 𝜃 ∈ 𝛺

May be differentiated under the integral sign.

May be differentiated under the integral sign where T(X1, Xn) is any unbiased estimator of 𝜃

Cramer-Rao inequality: Let (X1,…, Xn) be a random sample of n observations on X with þ. 𝑑. 𝑓 𝑓(𝑥;
𝜃) and suppose the above regularity conditions hold. If T is any unbiased estimator of 𝜃, then-

Proof: We have
 9

Which gives, on differentiating.𝑤. 𝑟. 𝑡 𝜃

Or

Or

Also, since T is unbiased estimator of 𝜃, 𝑤𝑒 ℎ𝑎𝑣𝑒

Which given on differentiation 𝑤. 𝑟. 𝑡. 𝜃

But

So that (2) becomes

Or E (TZ) =I ............... (3)

Where

From (1) we immediately get

And
 10

Now, 𝐶𝑜𝑣(𝑇𝑍) = 𝐸(𝑇𝑍) − 𝐸(𝑇)𝐸(𝑍)

=1

(i)An unbiased estimator T whose variance equals the lower bound

If and only if T is if the from where

Proof:

I𝓯𝓯
𝑅(𝑇, 𝑍) = 1

𝑖. 𝑒 , 𝑖𝒻 T is a linear 𝒻unction of Z, say

𝑇 = 𝑎𝜃 + 𝑏𝜃𝑧

But 𝐸(𝑇) = 𝑎𝜃 = 𝜃

𝑖. 𝑒 T= 𝜃 + 𝑏𝜃𝑧

Let (𝑥1, . . 𝑥𝑛) be a random sample from R (0, 𝜃)

CRB

We know that is UMVUE whose variance is-

Therefore, we have

Since 𝑃(𝑇, 𝑍) ≤ 1 we get

 11

Remark: (𝑖) the left page

(𝑖𝑖) If 𝓰(𝜃) is an estimable function for which an unbiased estimator is T (𝑖. 𝑒. 𝐸(𝑇) = ℊ(𝜃)) then
C.R Inequality becomes-

(𝑖𝑖𝑖) It can be show that

(𝑖𝑣) If an unbiased estimator exists which is such that its variance is equal to the lower bound
CRB= then it will be UMVUE.
.

(𝑣) If there is no unbiased estimator whose variance equals the C R B it does not mean that
UMVUE will not exist. Such estimators can be found (if these exists ) by other methods.
(𝑣𝑖) In case of distributions not satisfying the regularity conditions (e.g.: Rectangular distribution)
UMVU estimators, if these exists can be found by other methods. For such cases UMVU estimator may
have variance less than CRB.

Example: Let (𝑥1, . . . 𝑥𝑛) be a random sample from a Bernoulli distribution 𝑓(𝑥; 𝜃) = 𝜃𝑥(1 −

𝜃)1−𝑥(𝑥 = 0,1), 0 < 𝜃 < 1

Show that is a UMVU of 𝜃

Sohn : log 𝑓(𝑥; 𝜃) = 𝑥𝑙𝑜𝑔𝜃 + (1 − 𝑥) log(1 − 𝜃)

So that

By CR inequality we have C R B
 12

Now, that is equal to C R B. Hence 𝑥̅ is UMVUE of 𝜃

Example: Let x be a random sample having Binomial distribution

Show that 𝑥̅⁄𝑚 is UMVUE of 𝜃.

Soln:

So that

For sample of one observation X let T=T(X) be an unbiased

estimator. The C.R.B is . Now
𝑋̅ so that is UMVUE of 𝜃
(see left page) 𝑚

Example: Let (x1,..., xn) be a random sample from a Poisson distribution

Show that 𝑥̅ is UMVUE of 𝜃.

Soln: 𝑙𝑜𝑔𝑓(𝑥, 𝜃) = −𝜃 + 𝑥𝑙𝑜𝑔𝜃 − 𝑙𝑜𝑔𝑥;

The C.R.B = 𝜃⁄𝑛

Now so that 𝑥̅ is UMVUE of 𝜃

Example: Let (𝑥1, . . , 𝑥𝑛) be a random sample from a normal distribution 𝑁( 𝜃 , 𝜎2) where variance 𝜎
is known show that 𝑥̅ is UMVUE of 𝜃.

Soln:
 13

Or

The C.R.B= 𝜎2⁄𝑛

Now 𝐸(𝑥̅) = 𝜃 and 𝑉(𝑥̅) = 𝜎2⁄𝑛 so that 𝑥̅ is UMVUE of 𝜃

Example Let 𝑥1, . . , 𝑥𝑛 be a random sample from a normal distribution 𝑁(𝜇, 𝜃) where 𝜇 is known and
𝜃 is that variance to be estimated. Show that is UMVUE of 𝜃

(𝑥−𝜇)2

Soln:

Or

The C.R.B= 2𝜃2⁄𝑛

Consider the estimator for which E(S2)= 𝜃 and V(Sso that S2 is UMVUE of
𝜃
𝑛

Example An UMVU estimator is unique , in the sense that if T O and TI are both UMVU estimator
then TO = TI almost surely (𝑖. 𝑒 𝑃(𝑇𝑂 ≠ 𝑇𝐼) = 0)

Soln: Since both 𝑇𝑂 and 𝑇𝐼 are unbiased

𝐸(𝑇𝑂) = 𝐸(𝑇𝐼) = 𝜃 for all 𝜃 𝜖 𝛺

And since both are UMVUE,

𝑉(𝑇𝑂) = 𝑉(𝑇𝐼)for all 𝜃 𝜖 𝛺

Consider the new estimator

 14

Which is also unbiased. Moreover,

Where 𝜌 is the corr. Coefficient between 𝑇𝑂 𝑎𝑛𝑑 𝑇𝐼

By definition, 𝑉(𝑇) ≥ 𝑉(𝑇𝑂). It follows that 𝜌 ≥ 𝐼. Therefore 𝜌 =I so that, for every 𝜃, 𝑇𝑂 and 𝑇𝐼 are
linearly related, 𝑖. 𝑒.

𝑇𝑂 = 𝑎 + 𝑏𝑇𝐼

Where 𝑎, 𝑏 are amstants (may depend on 𝜃) and b≥ 0 . 𝑇aking expectation and variance we get

Which imply that b=1 and 𝑎 = 0. Therefore

𝑇0 = 𝑇

CONSISTENCY
Definition: A sequence of estimator {𝑇𝑛}.𝑛 = 1,2, … of a parameter 𝜃 is said to be consistent if, as n→∞
𝑇𝑛 → 𝑝 𝜃 for each fixed 𝜃 𝜖 𝛺 that is , for any 𝜖(> 0)

𝑇𝑛 𝑐𝑜𝑛𝑣𝑒𝑟𝑔𝑒𝑠 𝑡𝑜 𝜃 𝑖𝑛 𝑝𝑟𝑜𝑏𝑎𝑏𝑙𝑖𝑡𝑦

Or 𝑃{|𝑇𝑛 − 𝜃| > 𝜖} → 0

Or 𝑃{|𝑇𝑛 − 𝜃| ≤ 𝜖} → 1

𝑎𝑠 𝑛 → ∞

Remarks:

(𝒊) For increase in sample size a consistent estimator will become more and more close to 𝜃

(𝑖𝑖)Consistency is essentially a large sample property. We speak of the consistency of a sequence of

estimators rather than that of one estimator.
(iii) If {𝑇𝑛} is a sequence of estimator which is consistent for 𝜃 and {𝐶𝑛}, {𝑔𝑛} are sequence of
constants such that 𝐶𝑛 → 0 𝑔 → 1 as 𝑛 → ∞ then {𝑇𝑛 + 𝐶𝑛} 𝑎𝑛𝑑 {ℊ𝑛𝑇𝑛} are sequences of consistent
estimators also.

(iv) We will show later that if {𝑇𝑛} is a sequence of estimators such that 𝐸(𝑇𝑛) → 𝜃 and 𝑉(𝑇𝑛) → 0
and 𝑛 → ∞ then {𝑇𝑛} is consistent.

Examples:

1. Let (𝑥1, … 𝑥𝑛) be a random sample from any distribution with finite mean 𝜃.
Then it follows from LLN that 𝑥̅ so that ̅𝑥̅→̅ is consistent for 𝜃. If the
distribution has finite variance
 15

(𝜎2, 𝑠𝑎𝑦) 𝑉(𝑥̅) = 𝜎2⁄𝑛 → 0 so that it follows from Remark (IV) that 𝑥̅ is consistent .it can be shown

that the sample median is also consistent for 𝜃

2. Suppose (𝑥1, … , 𝑥𝑛) is a random sample from 𝑁(𝜇, 𝜎2). Let

4 The following is an example of an estimator which is unbiased but not consistent

Let (𝑥1, … 𝑥𝑛) be a random sample from rectangular distribution. 𝑅(𝑂, 𝜃) and let 𝑌𝑖 = 𝑚𝑖𝑛(𝑥1, … 𝑥𝑛)
consider the estimator 𝑇 = (𝑛 + 1)𝑌1. This is unbiased . Now for a any 𝐸(> 0),

𝜖 𝜖

(𝑒𝜃 − 𝑒−𝜃)

𝑛→∞

Which is some fixed number

𝑃{[𝑇 − 𝜃]𝜖} + 1

Thus, T is not constant

We can show that

By remark (iv) above 𝑠2 + 𝑠′2 are both constant for is biased and 𝑠′2 is unbiased.

3. Let (𝑥1, … . . 𝑥𝑛) be for a random sample for gamma distribution

 16

1 𝑥𝑥þ−1(𝑥 ≥ 𝜃, 𝜃 > 0)
þ 𝑘𝑛𝑜𝑤𝑛 f(x, 𝜃) = 𝜃þΓ(þ) 𝑒𝜃

Show that 𝑋̅⁄þ is unbiased and consistent for 𝜃

2

Soln:
𝑛þ

𝑋̅⁄þ is unbiased and consistent

Theorem: If {𝑇𝑛} is a sequence of estimators (of 𝜃)such that

𝐸(𝑇𝑛) = 𝜃𝑛 → 𝜃

And 𝑉(𝑇𝑛) → 0

As n→∞ then {𝑇𝑛} is consistent estimator of 𝜃.

Proof: By Chebyshev’s inequality, for any 𝜖(> 0)we have

As n→∞ by given condition of the theorem so that 𝑇𝑛 is consistent for 𝜃.

Theorem: If{𝑇𝑛} is a sequence of consistent estimators of 𝜃 and 𝓰(𝜃) is a continuous function of 𝜃,

then {ℊ(Tn)} is consistent for 𝓰(𝜃)

Proof: Since Tn is consistent for 𝜃, for any 𝜖1(> 𝑜)

𝑃{|𝑇𝑛 − 𝜃| ≤ 𝜖1} → 1

As n→∞

Also , since 𝓰(𝜃) is a continuous function , given 𝜖(> 0)we can choose 𝜖1(> 𝑜)such that

|𝑇𝑛 − 𝜃| ≤ 𝜖1 → |ℊ(Tn) − ℊ(𝜃)| ≤ 𝜖

Therefore ,

𝑃{|𝑇𝑛 − 𝜃| ≤ 𝜖1} ≤ 𝑃{|ℊ(Tn) − ℊ(𝜃)| ≤ 𝜖}

But as n→∞, L.H.S → 1 and, consequently, R.H.S →1, 𝑖, 𝑒.

𝑃{|ℊ(Tn) − ℊ(𝜃)| ≤ 𝜖} → 1

As n→∞. Hence ℊ(Tn) is consistent forℊ(𝜃).

 17

We can prove the following results:

(i) If {Tn} is consistent for, then is consistent for 𝜃2.

(ii) If {Tn} is consistent for 𝜃(R and non-negative) then √𝑇𝑛 is consistent for √𝜃. Proof For

any 𝜖(> 0) we have

Since L. H. S→0, R. H.S →0 as n→∞

(iii) If {Tn} is consistent for , then {𝑇𝑛 ± 𝑇′𝑛} is consistent for 𝜃 + 𝜃′.

Proof: for any 𝜖(> 0),we have

≤ 𝑃{|𝑇𝑛 − 𝜃| + |𝑇′𝑛 − 𝜃′| ≥ 𝜖}

As n→∞.

There fore {𝑇𝑛 + 𝑇′𝑛} is consistent for (𝜃 + 𝜃′)

(iv)if 𝑇𝑛 and are consistent for 𝜃 and 𝜃’ respectively , is consistent for 𝜃𝜃′. Proof:

we can write

EFFICIENCY:

If 𝑇1 and 𝑇2 are two unbiased estimators of a parameter 𝜃 , each having finite variance 𝑇1 is said to be
more efficient then 𝑇2 if 𝑉(𝑇1) >𝑉(𝑇2). The (relative) efficient of 𝑇1 relative to 𝑇2 is defined by

It is used to judge the efficiency of an unbiased estimator by comparing its variance with the
Cramer- Rao lower bound (C R B) .
 18

Definition: Assume that the regularity condition of CR inequality hold (we call it a regular situation)
for family{𝑓(𝑥, 𝜃), 𝜃 ∈ 𝛺}. An unbiased estimator T* of 𝜃 is called most efficient if 𝑉(𝑇∗) equals the
CRB. In this situation, the ‘efficiency’ of any other unbiased estimator T of 𝜃 is defined by

Where T* is the most efficient estimator defined above Remarks:

(i)The above definition not proper in−

(𝑎) regular situation when there is no unbiased estimator whose variance equals the CRB but
an UMVUE exists and maybe found by other methods.

(b)Non-regular situations when an UMVUE exists and may be found by other methods

(ii)The UMVUE is ‘most efficient‘ estimator in the examples considered earlier all UMVUE, whose
variances equalled CRB are most efficient
Example Consider 𝑎, 𝑟, 𝑠(𝑥1, … 𝑥𝑛) from a normal distribution𝑁(𝜇, 𝜃) where mean 𝜇 is known and
variance 𝜃(0 < 𝜃 < ∞ ) is to be estimated

We has seen that is UMVUE of 𝜃 for which the variance is equal to CRB and

consequently, 𝑠2 is most efficient . Let

Then 𝐸(𝑆′2) = 𝜃 and so that the efficiency of 𝑠′2 is given by

Asymptotic efficiency: As different from the above definition of efficiency we may define efficiency

in another way as follows, which may be called asymptotic efficiency.

Let us confine ourselves to consistent estimators which are asymptotically normally distributed.

Among this class, the estimator with the minimum asymptotic variance is called the ‘most efficient

estimator’. It is also called best asymptotically normal (BAN) or consistent asymptotically normal

efficient (CANE) estimator it we denote by avar(T*) the asymptotic variance of a BAN estimator T*

then the efficiency of any other estimator T (within the class of asymptotically normal estimators) is

defined by

Where avar (T) is the asymptotic variance of T.

Example: Let (𝑥1, … , 𝑥𝑛) be a random sample from a normal distribution 𝑁(𝜇, 𝜎), Consider the ‘most
efficient estimator 𝑥̅ and another estimator 𝑥̅me. It can be show that both are CAN estimator. We have
 19

And

So that the efficiency of 𝑥̅𝑚𝑒 is given by

Example: let T1, T2 be two unbiased estimators of 𝜃, having the same variance. Show that the
correlation coefficient ρ between T1, T2 cannot be smaller than 2e-1, where e is the efficiency of each
estimator,

Proof. Let To be the most efficient estimator then

Consider the unbiased estimator

Its variance is

Since 𝑇𝑜is UMVUE, V (T)≥ 𝑉(𝑇𝑜) which gives

Example: let 𝑇𝑜 be an UMVME (or most efficient estimator) where 𝑇1 an unbiased with efficiency ‘e’.
If 𝜌 is the correction coefficient between 𝑇𝑜 and𝑇1, then show that . Soln: we have

𝑒 = 𝑉(𝑇𝑜)/𝑉(𝑇1)

Or 𝑉(𝑇1) = 𝑉(𝑇𝑜)/𝑒

Consider the estimator

(Which the linear combination of 𝑇𝑜, 𝑇1 with minimum variance) then T is also unbiased, having
variance

Or
 20

Since are both non-negative 𝑉(𝑇) ≤ 𝑉(𝑇𝑜) but since 𝑇𝑜 is UMVUE, 𝑉(𝑇) 𝑉(𝑇𝑜).
therefore 𝑉(𝑇) = 𝑉(𝑇𝑜) , and 𝜌 = √𝑒

SUFFICIENCY CRITERION:
A preliminary choice among statistics for estimating 𝜃 , before having for a UMVUE as BAN estimator,
can be made on the basic of another enter on suggested by R.A fisher. This is called ‘sufficiency’
criterion.

Definition: let (𝒙𝟏,… , 𝒙𝒏) be a random sample from the distribution of X having þ, 𝑑, 𝑓 𝑓(𝑥, 𝜃) 𝜃 𝜖
𝛺.A statistic 𝑇 = 𝑇(𝒙𝟏,… , 𝒙𝒏) is defined to be sufficient statistic if and only if the conditional
distribution of (𝒙𝟏,… , 𝒙𝒏) given T=t does not depend on 𝜃, for any value t.

[Note: In such a case if we know the value of the sufficient statistic T, then the sample values are not
needed to tell us anything more about 𝜃].

Also the conditional distribution of any other statistic T (which is not for 𝛺 tray) given T is
independent of 𝜃.
A necessary and sufficient condition for T to be sufficient for 𝜃 is that the joint þ. 𝑑, 𝑓 of (𝒙𝟏,… , 𝒙𝒏)
should be of the form

𝑓(𝑥1, … , 𝑥𝑛; 𝜃) = 𝓰(T, 𝜃)ℎ(𝑥1, … , 𝑥𝑛)

Where the first term on 𝑟, ℎ, 𝑠., depends on T and 𝜃 and the second them is independent of 𝜃. 𝑇his is
known as Nyman’s Factorisation Theorem which provides a simple method of judging whether a
statistic T is sufficient

Remark: Any one to one function of a sufficient statistic is also a sufficient statistic

Example: Consider n Bernoulli trials with probability of success P. The associated Bernoulli random
variables (𝑥1, … , 𝑥𝑛) have common distribution given by

The joint probability function of (𝒙𝟏,… , 𝒙𝒏) is

𝑓(𝑥1, . . , 𝑥𝑛, 𝑝) = p∑in xi (1 − 𝑝)𝑛−∑𝑖𝑛 𝑥𝑖

= ℊ (∑ xi, p) (xi, xn)

Where

And 𝒽(𝑥1, … , 𝑥𝑛) = 1

Therefore is sufficient for p, and, so is .

Example (𝑥1, … , 𝑥𝑛) be a random sample from a position distribution P(𝜆)𝑖. 𝑒

The joint probability function of (𝑥1, … , 𝑥𝑛) 𝑖𝑠

 21

Where

Hence.

Are sufficient for 𝜆

Example: let (𝑥1, … , 𝑥𝑛)be a random sample from a Normal population 𝑁(𝜇, 𝜎). Case

I: 𝜇 unknown, 𝜎 known (=𝜎𝑜)

Where 𝓰(x̅ , μ) = 𝑒 − ∑𝑛𝑖 𝑥𝑖/ 2𝜎02 As

Which show that x̅ is sufficient for 𝜇 .

Case II: 𝜇 is know(= 𝜇𝑜), 𝜎 unknown

Where

Which show that is sufficient for 𝜎

Case III: Both μ and 𝜎 are unknown

Which shows that an jointly sufficient for [𝜇, 𝜎] Similarly,[ 𝑥̅, ∑(𝑥𝑖, 𝑥)2 / n-1]are also
sufficient for [𝜇, 𝜎],

Example let (𝑥1, … , 𝑥𝑛) be a random sample from a gamer distribution having þ, 𝑑, 𝑓
 22

We have

Case I 𝜃 unknown but þ is known

We can write

So that is sufficient for 𝜃.

Case II: 𝜃 Known but þ unknown

We can write

So that is sufficient for þ

Case III : Both 𝜃 and þ are unknown it is seen that are jointly sufficient for (𝜃, þ)

Example: let (𝑥𝑖, 𝑥𝑛, ) be a random sample from the experiential distribution

It follows from above that is sufficient for 𝜃.

Example let (𝑥1, … , 𝑥𝑛)be a random sample from the distribution with þ, 𝑑, 𝑓

𝑓(𝑥, 𝜃) = 𝜃𝑥𝜃−1, 𝜃 ≤ 𝑥 ≤ 1 We

have

So that is sufficient for 𝜃

Example let(𝑥1, … , 𝑥𝑛)be a 𝑎. 𝑟. 𝑠 from the Laplace distribution having þ, 𝑑, 𝑓

We have

For no single statistics T it is possible to express the above in the form ℊ[T, θ]𝒽(𝑥𝑖, 𝑥𝑛, ) . Hence there
exists no statistic T which taken alone is sufficient for θ. However the whole set (𝑥1, … , 𝑥𝑛) or the set
of order statistics (𝑥(1), … , 𝑥(𝑛))is jointly sufficient for θ
 23

Example let (𝑥1, … , 𝑥𝑛) be a random sample from the Rectangular distribution 𝑅(0, 𝜃) having þ, 𝑑, 𝑓.

We have

Where 𝐼𝐴(𝑥) is the indicator function for which

But

Where 𝑋(1) and 𝑥(𝑛) are the minimum and maximum of sample values(𝑥1, … , 𝑥𝑛) Therefore,

we can write

𝑓((𝑥1,… , 𝑥𝑛), 𝜃) = ℊ[x(n),𝜃]𝒽((𝑥𝑖, 𝑥𝑛) Where

𝒽(𝑥𝑖, 𝑥𝑛) = 𝐼[𝑂,𝑥(𝑛)](𝑥𝑖)

Where shows that 𝑥(𝑛) is sufficient for 𝜃

Example : If x has þ, 𝑑, 𝑓

We can check that

So that 𝑥(1) is sufficient for 𝜃

Example Let (𝑥1, … , 𝑥𝑛) be a random sample from the rectangular distribution 𝑅(𝜃1, 𝜃2) having
þ, 𝑑, 𝑓

The

þ, 𝑑, 𝑓((𝑥1,… , 𝑥𝑛)) is given by

Where We

can write
 24

= 𝓰[𝑥(𝑖),𝑥(𝑛),𝜃1𝜃2]𝓱(𝑥𝑖, 𝑥𝑛)

Where

𝓰[𝑥(𝑖), 𝑥(𝑛), 𝜃1𝜃2] = 𝑰[𝜽𝟏.𝑿(𝒏)](𝒙(𝒊))I{X(1),θ2](𝑥(𝑖))

And 𝓱((𝑥1, … , 𝑥𝑛)) = 1

Hence [𝑥(1), … , 𝑥(𝑛)] are jointly sufficient for 𝜃1, 𝜃2

Corollary : If 𝜃1 is known 𝑥(𝑛) is sufficient for 𝜃2

If 𝜃1 is known 𝑥(𝑖) is sufficient for 𝜃1

Example: let ((𝑥1, … , 𝑥𝑛)) be 𝑎, 𝑟, 𝑠 from the rectangular distribution R (-𝜃, 𝜃).

Then

So that [𝑥(1), … , 𝑥(𝑛)] are jointly sufficient for 𝜃

Example: [𝑥(1), … , 𝑥(𝑛)] are jointly sufficient for and 𝑅(𝜃, 𝜃 + 1) Example:

Let (𝑥1, … , 𝑥𝑛)be a random from an exponential distribution

𝑓(𝑥) = 𝜆𝑒−𝜆(𝑥−𝜃), 𝜃 ≤ 𝑥 < ∞

Case I : 𝜆 Unknown 𝜃 known (= 𝜃𝑜)

Which show that is sufficient for 𝜆or 𝑥̅ is sufficient for 𝜆

Case II: 𝜆 know (= 𝜆𝑜) , 𝜃 Unknown

Which shows that 𝑥(𝑖) is sufficient for 𝜃

 25

Case III: Both 𝜆, 𝜃 unknown

It is easy to check that [∑ 𝑥, 𝑥(𝑖)] are jointly sufficient for [𝜆, 𝜃]

METHHODS OF ESTIMATION:

For important methods of obtaining estimators are (I) methods of moments,(II) methods of
maximum likelihood (III)method of minimum χ2 and (IV) method of least squares.

(I)Method of moments

Suppose the distribution of a random variable X has K parameters which have to be

estimated. let denote the moment of about in general is a known function of
so that Let be a random sample from the distribution of X and let
be the . Sample moment from the equation

Whose solution is say , where is the estimate of Those are the method of
moments estimators of the parameters.

Example let

The equation

Have the solution

Let

∑𝑛𝑖 𝑥 𝑖2 ∑𝑛𝑖 (𝑥 𝑖 −𝑥 )2
𝜎=√ 𝑛
− 𝑥 2 =√ 𝑛

Example let ~P (λ) and let be random sample from P(λ).

The equation

Provides the estimator

Example let be a random sample from the exponential distribution

 26

The moment equation

Provides the estimator

Remark: (I) the method of moments estimators are not uniquely defined. We may equate the central

moments instead of the raw moments and obtain solutions.

(II) These estimator are not, in general, consistent and efficient but will be so only if the parent

distributions is of particular form.

(III) When population moments do not exist auchy population) this method of estimation is

inapplicable.

METHOD OF MAXIMUM LIKELIHOOD

Consider , the joint of sample of observations of having the

whose parameters is to be estimated. When the values are given,

may be looked upon as a function of which is called the likelihood function

of and is denoted by it gives the likelihood that the assumes the

value when is the parameter.

We want to know from which distribution . for what value of is the likelihood largest for this set

of observations. In other words we want to find the value of , denoted by which maximizes

. The value maximizes the likelihood function is in general, a function of

say

Such that Then

is called the maximum likelihood estimator or MLE.

In many cases it would be more convenient to deal with log , rather then , since log is

maximized for the some value of as . For obtaining we find the value of for which
 27

We must however, check that this provides the absolute maximum. It the derivate dose not exists at 𝜃

= 𝜃 or equation (1) is not solvable this method of solving (1) will fail.

Example: Let (𝑥1, . . , 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the Bernoulli distribution.

𝑓(𝑥, 𝜃) = 𝜃𝑥(1 − 𝜃)𝑖−𝑥,𝑥 = 𝜃, 1

Then the likelihood

And

Differentiating and equating to zero, we have

Or

Or 𝑒 = ∑𝑛𝑖 𝑥𝑖/𝑛=𝑥̅

𝑚. ℓ. 𝑒 of 𝜃 is 𝜃 = 𝑥̅

Example: Let (𝑥1, . . , 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the Poisson’s distribution

𝑛𝑥
𝑛𝑥

Then

And log !

Or

Equating to zero we get 𝜆 = 𝑥̅

𝑚. ℓ. 𝑒 of 𝜆 is 𝜆 = 𝑥̅

Example: Let 𝑥1, . . , 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the truncated Binomial distribution having þ, 𝑑, 𝑓
 28

Then

And

Equating to zero we get

∑ 𝑥𝑖[(1 − 𝜃){1 − (1 − 𝜃)2}] + (∑𝑥𝑖 − 2𝑛)[𝜃{1 − (1 − 𝜃)2}]

−2𝑛𝜃(1 − 𝜃)2] = 𝜃

Or ∑ 𝑥𝑖[1 − (1 − 𝜃)2] = 2𝑛𝜃

Or ∑ 𝑥𝑖[𝜃(2 − 𝜃)] = 2𝑛𝜃

Or Or

Example: Let (𝑥1, . . , 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the normal distribution 𝑁(𝜇, 𝜎)

Case I: 𝜇 unknown but 𝜎 = 𝜎0(known)

𝑛 2

Then

And

Or

Equating to zero we get 𝜇 = 𝑥̅

𝑚. ℓ. 𝑒 Of 𝜇 = 𝑥̅
 29

Case II: μ = μ0(known)but σ unknown

Then

And

Or Equating

to zero we get

2 ∑𝑛𝑖 (𝑥 𝑖 −𝜇 𝜃 )
𝑚. ℓ. 𝑒 Of 𝜎 is 𝜎=√ 𝑛

Case III: Both 𝜇 and 𝜎 are unknown

𝑛 2

Then

And

Differentiating partially 𝑤. 𝑟. 𝑡 𝜇, 𝜎 we get

And

∑𝑛𝑖 (𝑥 𝑖 −𝑥 )2
√
Equating to zero both the derivatives and solving the equations we get 𝜇 = 𝑥̅ and 𝜎 = 𝑛

𝑚. ℓ. 𝑒 are 𝜇 = 𝑥̅ and

Example: Let be from the exponential distribution

Then And
 30

Quoting to zero, we get so that the of is

Example: Let be from the exponential distribution

Then

If we differentiate and equate to zero we get which does not yield any

result. Now is maximized by choosing the maximum value of subject to the condition

Which shows that so that the of

Example: X has

Example: Let be from the distribution

Then
And log 𝐿(𝜃) = 𝑛𝑙𝑜𝑔 𝜃 + (𝜃 − 1)∑𝑛𝑖 𝑙𝑜𝑔𝑥𝑖

Or

Equating to zero we get

Example: Let(𝑥1, . . , 𝑥𝑛) have rectangular distribution 𝑅 (0, 𝜃) having þ, 𝑑, 𝑓

 31

Then

Which is maximized when 𝜃 is maximum subject to the condition

0 ≤ 𝑥(1) ≤ ⋯ ≤ 𝑥(𝑛) ≤ 𝜃

The minimum value of 𝜃 is 𝑥(𝑛) so that

𝑚. ℓ. 𝑒 of 𝜃 is𝜃 = 𝑥(𝑖)

Example: Let(𝑥1, . . , 𝑥𝑛) be 𝑎, 𝑟, 𝑠 of the regular distribution 𝑅(−𝜃, 𝜃) having þ, 𝑑, 𝑓

Then

When is maximized when 𝜃 is minimum subject to the condition −𝜃 ≤ 𝑥(1) ≤ ⋯ ≤ 𝑥(𝑛) ≤ 𝜃

So that since −𝜃 ≤ 𝑥(1) or 𝜃 ⩾ −𝑥(1)

𝑚. ℓ. 𝑒 of 𝜃 is 𝜃 = −𝑥(𝑖)

Example: Let(𝑥1, . . . 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the regular distribution 𝑅(𝜃1, 𝜃2) having þ, 𝑑, 𝑓

Then

In maximized when (𝜃2 − 𝜃1) is minimum 𝑖, 𝑒𝜃1 is maximum and 𝜃2 is minimum subject to the

condition

𝜃1 ⪕ 𝑥(𝑖) ⪕. ⪕ 𝑥(𝑛) ⪕ 𝜃2

We have to take 𝜃2 = 𝑥(𝑛) and 𝜃1 = 𝑥(𝑖) so that 𝑚. ℓ. 𝑒 𝑜𝑓𝜃1and 𝜃2 are 𝜃1 = 𝑥(𝑖) and𝜃2 = 𝑥(𝑛) Example:

Let(𝑥1, . . . 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the regular distribution 𝑅(𝜃 − 𝑐, 𝜃 + 𝑐) having þ, 𝑑, 𝑓

 32

Then is maxi zed for any 𝜃such that

𝜃 − 𝑐 ⪁ 𝑥(𝑖) ⪁ ⋯ ⪕ 𝑥(𝑛) ⪕ 𝜃 + 𝑐

𝑖. 𝑒 𝜃 − 𝑐 ⪁ 𝑥(𝑖) or 𝜃 ⪕ 𝑥(𝑖)𝑐 and 𝜃 + 𝑐 ⩾ 𝑥(𝑛) − 𝑐

And 𝜃 + 𝑐⩾𝑥(𝑛) is 𝜃⩾𝑥(𝑛) − 𝑐

This shows that any statistics which lies in between 𝑥(𝑛) − 𝑐 and the

𝑚. 𝑙. 𝑒 is not unique in this case

Example 12 It x has 𝑅(𝜃, 𝜃 + 𝐼),any statistics which lies between 𝑥(𝑛) − 1 and 𝑥(𝑖) is a 𝑚. 𝑙. 𝑒 if 𝜃

Example 13 Let(𝑥1, . . . 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the regular distribution 𝑅(𝜃, 2𝜃) having þ, 𝑑, 𝑓

Then

Is maxi zed when 𝜃 is minimum subject to the condition 𝜃 ⪁ 𝑥(𝑖) ⪁ ⋯ ⪁ 𝑥(𝑛) ⪁ 2𝜃

𝑖. 𝑒 𝜃 ⪁ 𝑥(1) … … (𝑖)

And 𝜃 ⩾ 𝑥(𝑛) … … . (𝑖𝑖)

Since

The minimum value of 𝜃 satisfying so that the 𝑚. 𝑙. 𝑒of 𝜃 is

Example: Let(𝑥𝑖, . . 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the regular distribution 𝑅(−𝜃, 𝜃) having þ, 𝑑, 𝑓

 33

Then

This is maximized when 𝜃 is minimum subject to the condition

𝑥(𝑛) ⪁ 𝑜𝑟𝜃 ⩾ 𝑥(𝑛)

And −𝜃 ⪁ 𝑥(𝑖)𝑜𝑟𝜃 ⩾ −𝑥(𝑖)

This happens when 𝜃 = max (−𝑥(𝑖), 𝑥(𝑛))

𝑚, 𝑙, 𝑒 of 𝜃 = max (−𝑋(1), 𝑋(𝑛))

Example: Let(𝑥1, . . . 𝑥𝑛) be 𝑎, 𝑟, 𝑠 from the Laplace distribution with þ, 𝑑, 𝑓

Then

And
Which is maximized when is the sample median.

of is

Example: Let be n independent such that has normal distribution

We have to estimate and then

And
 34

Or

Equating to zero , we get

Or

Of is

We have

Optimum properties of MLE: (i) If is of and Ψ is a simple valued function of with

unique inverse, then Ψ is the of Ψ .

(ii) If a sufficient statistics exists for is a function of this sufficient statistics.

(iii) Suppose statistics certain regularity conditions and is the of

a random sample of size n from

Then- is consistent sequence of estimators of

(b) is asymptotically normally distributed with mean variance

(c)The sequence of estimators has the smallest asymptotic variance among all consistent,

asymptotically normally distributed estimate of , is BAN or CANE or most efficient.

(iii) METHOD OF MINIMUM χ2: Let X be with where parameter to be estimated

Suppose S1, S2....S

 35

Let be from a Bernoulli distribution having

36
37
38
39
40
41
 42

(b)

(b)
43
44
 45

W1 W2 W3

Let

We have using (i),

 46

0.01024 1.00000

0.23040 0.98976

0.34560 0.68256

0.25420 0.33696
0.07776 0.07776
47
48
 49

: MP test is given by W=
50
51
 52

will here the critical region

Since is a non-decreasing function of , the critical region is { k} where k=

(8) Suppose x has the following distribution

H0 : f

Let us take a single observation. The MP test of H0 Vs H1 has the critical region

C}

²
Or C’

Since L.H.S. is a non-increasing function of , the critical region is { k} where k =

(9) Suppose X has the following distribution

H
H1: f1(x) = 1 ; 0<x<1
 53

Let us take a single observation. The MP test of H0 VS H1has the critical region given by

Where

We see that C

If either x <k1 or x>k2

Hence MP or region is

{ x <k1 }U{ x>k2 }

The size of the test is 𝑃H0 { x <k1}U{ x>k2} + 𝑃H0{x>k2} = α

For simplicity we can take k₂ = 1-k1

(10) Let X have the rectangular distribution R(0,ϴ) having p.d.f.

f ϴ

We want to test

H0: ϴ = ϴ0 Vs

H1:ϴ = ϴ1(>ϴ0)

We have

L(ϴ) =

Therefore the MP test has the critical region W ={ C}

Now,

𝐿(ϴ0)
This shows that 𝐿(ϴ1) is an increasing function of 𝑥(𝑛) and, therefore

Hence the MP test has the critical region

{ 𝑥(𝑛) ≥k}

The value of k is determined by the size condition

P { 𝑥(𝑛) ≥k/𝛳0} = α
𝑛 1

Since 𝑥(𝑛) has p.d.f. fϴ

We have

Remark: the above test is UMP for H0: ϴ=ϴ0 against H1:ϴ>ϴ0
As we have remarked, UMP test may not always exist. Therefore we for their restrict the class of
tests by considering unbiased tests (defined below) and then try to obtain UMP test in the class of
unbiased tests. If such a test exists we call it uniformly not powerful unbiased test (UMPU test)
 54

Definition Suppose we are testing a sample hypothesis Hϴ: 𝜃 = 𝜃0 against a conqurite alternative

𝐻𝑖(𝑚𝑎𝑦 𝑏𝑒 𝜃 ≠ 𝜃0𝑜𝑟 𝜃 > 𝜃0 𝑜𝑟 𝜃 < 𝜃0) A test T is called unbiased if

𝑃𝑜(𝑇) ⩾∝ for all 𝜃 𝜖𝐻𝑖

Where ∝ is the size of T 𝑖, 𝑒. 𝑃𝑜(𝑇) =∝

Remark: Suppose 𝜃 = 𝜃1 is one of the alternative value of 𝜃. If the test is not unbiased it may happen
that 𝑃𝑜(𝑇) <∝= 𝑃00(𝑇) which means that the probability of rejecting 𝐻𝑜 when it is false is less then
the probability if rejecting 𝐻𝑜 when it is true if the test is unbiased it will not happen. Theorem
A MP test or UMP test is unbiased.

Prof Let T be a MP (or UMP) test of size ∝. Consider another test T which rejects the null hypothesis
HO: 𝜃 = 𝜃0 with probability ∝ irrespective of the sample outcome. We may just toss a coin for which
the probability of is ∝ and decide to reject the null hypothesis Hϴ if we get ∝ , irrespective if the
sample values obtained. Then

𝑃𝑇{𝑅𝑒𝑗𝑒𝑐𝑡𝐻𝑜/HO is 𝑡𝑟𝑢𝑒} =∝

So that the size of the test T=∝. Also the power of test T is also∝, since

𝑃𝑇{𝑅𝑒𝑗𝑒𝑐𝑡𝐻𝑜/HO is 𝑓𝑎𝑙𝑠𝑒 } =∝

But T being MP (or UMP) is such that

𝑃𝑇 (𝜃) ⩾ 𝑃𝑇(𝜃) for 𝜃 𝜖 Hi

Or 𝑃𝑇 (𝜃) ⩾∝ for 𝜃 ≠ 𝜃0

Remark: It may be shown that the following tests are UMPU for two sided alternative 𝐻𝑖 ∶ 𝜃 ≠ 𝜃0 in
example 1,2 and 3

For example 1, UMPU test is {𝑥̅ ⩾ 𝓀1𝑜𝑟𝑥̅ ⪕ 𝓀2}

For example 2, UMPU test is{[𝑥] ⩾ 𝓀}

For example 3, UMPU test is{∑(𝑥𝑖 − 𝜇)2 ⩾ 𝓀1 𝑜𝑟 ∑(𝑥𝑖 − 𝜇)2 ⪕ 𝓀2}

The constant 𝓀, 𝓀1, 𝓀2 are determined from size condition

Now we consider a produce for constructing tests that has some intuitive appeal and that .
Frequently, though not always, leads to UMP or UMPU test. Also the produce leads to test that have
decided large sample properties

Suppose we are given a sample (𝑥1, … , 𝑥𝑛) from a distribution with þ, 𝑑, 𝑓 𝑓(𝑥, 𝜃 ) (where 𝜃 may be
a vector) and we deice to test the null hypothesis 𝐻𝑜 ∶ 𝜃 𝜖 𝑤(⊂ 𝛺) against the alternative hypothesis
𝐻𝑖 ∶ 𝜃 𝜖 𝑤(⊂ 𝛺) where 𝛺 is the parameter space,

The likelihood function of the sample is given by

 55

Define the likelihood ratio

max 𝐿(𝜃)
Where denotes the maximum of the likelihood function when 𝜃 is restricted to values in 𝜃 𝜔
w and max 𝐿(𝜃) denotes the maximum of the likelihood for when 𝜃 takes all possible values in𝛺

Obviously, 0 ≤ 𝜆 ≤ 1 and λ is also to 1 of the sample shows that 𝜃 lies actually in 𝛚. Definition

The likelihood ratio test of 𝐻𝑜 against 𝐻𝑖 has the critical region

𝑤 = {𝜆 ⪕ 𝜆𝑜}

When 𝜆𝑜 is determined by the size condition

Remark (1) For testing a simple hypothesis against a simple alternative likelihood ratio test is
equivalent to the test given by the Neyman –Pearson lemma.

(ii) if a sufficient statistics exists the L.R test is a function of the sufficient statistics.

(iii) Under some regularity condition -2 loge λ is asymptotically distributed as a χ2 𝑟. 𝑣. with

degrees of freedom equal to the difference between the number in 𝛚.

Example: (1) Let X be a r.v. having a normal distribution 𝑁(𝜇, 𝜎) where 𝜎 (=𝜎𝑜) is known

We want to test 𝐻𝑂: 𝜇 = 𝜇𝑜

Against 𝐻1: 𝜇 ≠ 𝜇𝑜

We have the likelihood function

Then

Since MLE of μ is 𝜇 = 𝑥̅ , therefore

The LR test critical region is given by λ ≤ λ0

− ∑𝑛(𝑥𝑖−𝜇0)2/2𝜎20

Or

𝑒2𝜎20[∑(𝑥𝑖−𝑥̅) −∑(𝑥𝑖−𝜇0)² ≤ 𝜆0

Or
 56

or

or k’ Remark (i) the above test is not UMP test

since there exists other UMP tests for 𝐻1:𝜇 > 𝜇0and

under 𝐻𝑂 so that 𝑘 can 𝑘 found easily by using size condition (2)

Let x ~𝑁(𝑂, 𝐼) where both 𝜇 and 𝜎 are unknown we want to test

𝐻𝑂:𝜇 = 𝜇𝑜

Against 𝐻𝑖: 𝜇 ≠ 𝜇0

We have the likelihood for

Under 𝐻𝑂: 𝜇 = 𝜇𝑜, (given) so the MLE of 𝜎 is

In general, 𝑚, 𝑙, 𝑒 of 𝜇 is 𝜇 = 𝑥̅ and MLEof 𝜎 is

Therefore, we have

And

The L.R test critical region is given by

Or
2

Or

Or

Since the above cr region becomes

 57

Or ’’

Where ’

It is know that has t distribution on under There fore the values of can be
found from the size condition

Where Y~

(3) Let X ~N when both and are unknown we want to test

Against

We have the likelihood function

Under , the of is

In general, of is and of is

Then we have

And

L.R test cr region is given by

Or

Or

We note that has a maximum at

Therefore if and only if or that is the critical region is

But it is know that has distribution on (n-i) using the tables and size
condition we can get the values of and
 58

(3a) suppose in example 3 the value of is know. Then the L.R cr region because

Where /n

In than case has

(4)Let x have an exponential distribution

We want to test

Against

We have the likelihood function

Then we get

Also

Because of is

The LR test cr region is given by

Where

Since at lains maximum at taking we see that λ=i if and λ⪕ for

LR test critical region because

Remark (i) if one take we shall get the L.R critical region as in both case of one –
sided alternation the L.R test are UMP test.

(2) Since has gamma distribution we can find the value of by using size condition

(5) Let ( be from and be from another where two

samples (distribution) are independent.

We want to test

Where it is assumed that 𝜎1 = 𝜎2(= 𝜎𝑢𝑛𝑘𝑜𝑤𝑛 ) we that the like hood function
 59

In general the 𝑚, 𝑙, 𝑒 of 𝜇1, 𝜇2 and 𝜎 are

And

Also and

Therefore

Against the 𝑚, 𝑙, 𝑒 under 𝐻𝑂are

And

Therefore

So that the LR cr region is given by

Or

Or Or

2 2

Where
The cr region can be within as

Since under find 𝓀 such that 𝑃{𝛾 ⩾ 𝓀} =∝

 60

Where 𝛾~𝑡𝑛1+𝑛2−2

(6)Let (𝑋𝐼,. 𝑋𝑛𝐼) be 𝑎, 𝑟, 𝑠 from N (𝜇, 𝜎𝑖) and(𝛾1, . 𝛾𝑛2 )𝑛 N (𝜇2, 𝜎2) where two samples (and two
distributions) are indecent

We want to test

Against

We have the likelihood function

In general, be 𝑚, 𝑙, 𝑒 of 𝜇1, 𝜇2, 𝜎1, 𝜎2 are

So that

Against, the m, l, e under 𝐻𝑂 are

𝑛 +𝑛2
1 − 12
So that max 𝐿(𝜇1 , 𝜇2 , 𝜎1 , 𝜎2 ) = 𝑛 1 +𝑛 2 𝑒
𝑛 +𝑛 2
(2𝜋) 1 2(𝑠 ) 2

Therefore, the LR cr region is given by

𝑛1 𝑛2

Or

Or

Where

Setting 𝑔(𝑓)𝑓𝑜𝑟the 𝐿. 𝐻. 𝑆 of (i) we have 𝓰(o)=o and 𝓰(f)→o∞. Furthermore 𝓰(f) attains its

maximum for , it is impressing between o and f may and derision in (f may, ∞).
Therefore 𝓰(f)⪕ 𝜆𝑜 if and only if 𝑓 < 𝓀1 or 𝑓 > the LR cr region can be within as {𝐹 < 𝓀1 𝑜𝑟 𝐹 > 𝓀2}

Where

But under𝐻𝑜, 𝐹~𝐹𝑛1−1,𝑛2−1 ,

Hence 𝓀1, 𝓀2 can be obtained from the size condition 𝑃{𝑓 > 𝓀1 𝑜𝑟 𝐹 < 𝓀2} =∝whese F~𝐹𝑛1−1,𝑛2−1
 61

Definition: A 𝑟, 𝑣. 𝑥 is said to have a Gamma distribution 𝐺(∝, 𝛽) of its þ. 𝑑. 𝑓. is given by

;x≥0

=0 ;𝑥<0

(∝> 0, 𝛽 > 0) We

have , 𝑡<𝛽

𝐸(𝑋) =∝/𝛽

𝑉(𝑋) =∝/𝛽2

If ∝= 1 we get the exponential distribution

𝑓(𝑥) = 𝛽𝑒−𝛽𝑥 , 𝑥 ⩾ 0(𝛽 > 0)

𝐸(𝑋) = 1/𝛽

𝑉(𝑋) = 1/𝛽2

If ∝= 𝑛/2(𝑛 a positive integer) 𝛽 = 1/2 we get the 𝑥2 distribution on 𝑛, 𝑑, 𝑓 where þ, 𝑑, 𝑓 is

We have 𝑚, 𝑔, 𝑓 𝑀𝑥(𝑡) = (1 − 2𝑡)−𝑛/2

Definition: A 𝑟, 𝑣 X is said to have a 𝑡 −distribution on 𝑛, 𝑑, 𝑓 if its þ, 𝑑, 𝑓 is given by

If X~𝑛(𝑜 − 𝑖), 𝛾~x2(n) and x and 𝛾 are inept then

Define: A 𝑟, 𝑣X is said to have a 𝐹 − distribution on(𝑚, 𝑛)𝑑, 𝑓 if its þ, 𝑑, 𝑓 is given by

=0 , x<0

Of and γ where and γ are independent has

Percentage points the upper percent point of the distribution is where

The upper percent point of the distribution where

Since t-distribution is symmetrical

 62

The upper percent point of the F(m ,n, distribution is Fm ,n, where

Note that
Use of and distribution in testing problem
Use of distribution (i) Testing the variance of a of a distribution: Given a sample of
size n from a normal distribution where is unknown, we would like to test
against alternative > or < or the tests are summarised n follows know

Alternative reject at level if

HO:

Ho:

Ho:

know

Alternative reject at level if

HO: ,

Ho:

Ho:

Where

(2) Testing proportions in (>2) classes Suppose takes values in one of

(>2)mutually exclusive classes AI

Against for all

For a random (x, … 𝑥𝑛) of n observation let the observed frequencies in the 𝓀 classes be 𝑜1,𝑜2,

and the expected frequencies under the 𝐻𝑜 be 𝑒1,𝑒2, … … . 𝑒𝓀 (∑𝑛𝑖 𝑒𝑖 = 𝑛) where 𝑒𝑖 = 𝑛þ𝑖
calculate

Them, for large sample, 𝑥2has 𝑥2(𝓀 − 𝑖) the test of 𝐻𝑜 has the cr. region

Note: it we want to test 𝐻𝑜þ1 = þ2, … … . = þ𝑛 we take to any

 63

(3) Testing goodness of fit: given a sample (𝑥1, . . 𝑥𝑛) of Observation on 𝑎. 𝑟. 𝑣 X arranged in the
form of a frequencies distribution having 𝓀 classes AI, … . . A𝓀 we would like to test the
hypothesis that distribution of X has a specified from with þ, 𝑑, 𝑓(𝑜𝑟 þ, 𝑚, 𝑓)𝑓𝑜(𝑥, 𝜃) the
parameter 𝜃 be a simple one or a vector (𝑜𝑖,… . . 𝜃ℯ)

Let the observed frequencies in the 𝓀 classes be and the expected

frequencies under

Such that 𝑒𝑖 = 𝑃𝐻𝑜(𝑥 𝜖 𝐴𝑖) Calculate

Then, for large sample, 𝑥2has 𝑥2(𝓀 − 𝑖) the test of 𝐻𝑜 has the cr. Region

Note if 𝑟(𝑜𝑓 ℓ) parameters in 𝜃 are estimated from the sample then χ2 has χ2(𝓀 − 𝑟 − 𝑖)if any
expected frequency is lass then 5 we pool this class with the adjoining class and denote by𝓀 the
effective number ƪ classes after paroling

(4) Testing independence of two attributes in a 𝓀xℓ contingency table

In a (𝓀xℓ) contingency table for two attributes, we want to test

𝐻𝑜: Two attributes are independent

Against 𝐻𝑜: Two attributes are not independent

Let 𝑂𝑖𝑗 = observed frequency in the (𝑖, 𝒿) the cell

And 𝑒𝑖𝑗 = expected a=(𝑖𝑡ℎ 𝑟𝑜𝑤 𝑡𝑜𝑡𝑎𝑙 𝑥𝑗𝑡ℎ𝑐𝑜𝑙𝑢𝑚 𝑡𝑜𝑡𝑎𝑙)n “ “ “under 𝐻𝑜 Calculate

Where n=total frequency. Then has on the test of has the cr. Region

(5) Testing the homogeneity of correlation coefficients.

Suppose are sample correlation coefficients corresponding to normal Distribution

with population correlation coefficients we want to test

Us : all correlation coefficients are not equal we use the friskers z-trans function of correlation

coefficients given by so that

 64

Where n is the sample size.

We calculate z1, z2,......z

And

Then has on and the test of has cr. Region

Remark: if is accepted we may obtain an estimate of the common corresponding coefficients

by solving

Uses if t-distribution:

be a sample of size n from a normal

population and, as usual, and are the sample mean and sample variance. We would like
to let the null hypothesis against alterative or or the tests are
summarised as follows:

be two
samples from in dept normal populations and respectively let be as
usual and let

Be the pooled variance.

We would like to test 𝐻𝑂: 𝜇1 = 𝜇2 against alternative 𝜇1 < 𝜇2 or 𝜇1 ≠ 𝜇2 the test are summarised as
follows:

Case I

Alternative Reject 𝐻𝑂 at level ∝ 𝑖𝑓

𝐻𝑖: 𝜇1 > 𝜇2

𝐻𝑖: 𝜇1 < 𝜇2 “ ⪕ −𝑧∝

𝐻𝑖: 𝜇1 ≠ 𝜇2
 65

Case II 𝜎1, 𝜎2 unknown (𝜎1 = 𝜎2) essential corruption

Alternative Reject 𝐻𝑂 at level ∝ 𝑖𝑓

Remark: if we want to test 𝐻𝑂 = 𝜇1 − 𝜇2 = (≠ 𝑜)𝑤𝑒 use the statistics

Uses of F-distribution:

(1)Testing equality of two population variances:

Let two samples of sizes 𝑛1 and 𝑛2 be given from two independent normal population 𝑁(𝜇1, 𝜎1) and
𝑁(𝜇2, 𝜎2), respectively .Let be the two sample variance. We would like to test the null
hypothesis 𝐻𝑜:𝜎1 = 𝜎2 against𝐻𝑖: 𝜎1 ≠ 𝜎2 The test are cr follows:

Case I 𝜇1, 𝜇2 known

Reject 𝐻𝑜 if either

Or

Case II I 𝜇1, 𝜇2 known

Reject 𝐻𝑜 if either

Or

(2) Testing the multiple correlation coefficient: Given a sample of size or from a bivariate
normal population (𝑥1, 𝑥2, 𝑥3) with multiple correlation coefficient 𝑅1(23) of 𝑥1𝑜𝑟(𝑥2, 𝑥3) we
would like to test the null hypotheses 𝐻𝑂𝑅1(23) = 0 let the sample multiple correlation coefficient
be 𝑅1(23). The test is to reject 𝐻𝑂 at level ∝ if

(3) Testing the equality of means of 𝓀 normal distribution (𝓴 > 𝟐)[see left page]

Farceur’s z-transformation of correlation coefficient: Suppose a sample of size n is drawn from a

bivariate population with correlation coefficient the variables Fisher intruded the transformation
 66

Where r is a sample correlation coefficient Though the population correlation coefficient P may be
widely different from zero, the new statistics z may be amounted to be normally distributed even
when n is as small as 10 it has hen show that z has approximate mean

And approximate mean1⁄(𝑛 − 3), 𝑖. 𝑒

(I)For testing 𝐻𝑜 ∶ 𝑃 = 𝑃𝑜 against 𝐻𝑖 ∶ 𝑃 ≠ 𝑃𝑜we reject 𝐻𝑜 if

Where and 𝑁∝ is the appear ∝ % point of normal distribution 𝑁(𝑂, 1)

(ii)For testing 𝐻𝑜 ∶ 𝑝1 = 𝑝2 against 𝐻𝑖 ∶ 𝑝1 ≠ 𝑝2 involving two populations, let 𝑟1, 𝑟2 be the sample
correlation coefficient for two independent sample of size 𝑛1, 𝑛2 from the two populations and let
𝑧1,𝑧2be there transformed values,𝑖, 𝑒

The test is to reject 𝐻𝑜 at level ∝ if

(iii)Let 𝑟1, 𝑟2 … . . 𝑟𝓀 be sample correlation coefficient for 𝓀 sample of sizes 𝑛1, 𝑛2 … 𝑛𝓀 drown from
𝓀 independent vicariate normal population with correlation coefficients 𝑝1𝑝2 … . . 𝑝𝓀. Let 𝑧1, 𝑧𝓀 be
the transformed values and let

The test is to reject 𝐻𝑜 at level ∝ if

If 𝐻𝑜 is accepted an estimate of common correlation coefficient p is 𝑝𝑟 where𝑧̅ is the transformed

values of 𝑝∗(x) For large sample

Large sample tests so for we have considered tests of hypothesis which contain assumptions
regarding the population are satisfied .Now we consider some approximate test which are valid only
for sufficiently large samples, but they have wide applicability and hold for all populations satisfying
certain general conditions rather than being valid for some particular populations only (e.g. normal )
 67

(i)Testing a proportion: Suppose in a population is the proportion of members with a qualitative

character A. Let p be the proportion of members with A in a random sample of size n. we would like to
test the hypothesis H0: P=P0 .The test is to reject H0 at level α if

(ii)Testing the equality of two population proportions: Let 𝑝1, 𝑝2 be two population proportions and þ1,
þ2 be the two sample proportions dream from there indecent population the test of 𝐻𝑜: , 𝑃1,𝑃2 is to
reject 𝐻𝑜at level ∝if

Where

(iii)Testing for a st. deviation: let s be the st. Deviation of a sample of observation of size drown from a
population with st. Deviation 𝜎(x) the test of 𝐻𝑂: 𝜎 = 𝜎𝑜 is to reject 𝐻𝑂 at level ∝if

(iv)Testing for equality of two population st. Deviation Let 𝓈1, 𝓈2 be the st. Deviation of two sample of
sprees 𝑛1, 𝑛2 from two independent population with st. Deviation 𝜎1, 𝜎2 Let

The test of𝐻𝑜: 𝜎1, 𝜎2 is to reject the at level ∝if

Definition:- For a random sample (𝑥1, … , 𝑥𝑛) from the distribution of a 𝑟. 𝑣. 𝑥 having þ, 𝑑, 𝑓 𝑓(𝑥, 𝜃) Let
𝐿1𝐿1(𝑥1, … , 𝑥𝑛)and 𝐿2(𝑥1, … , 𝑥𝑛) be two statistics such that 𝐿1 ≤ 𝐿2. The interval [𝐿1, 𝐿2] is a confidence
interval for 𝜃 with. Confidence coefficient 1−∝ (0 <∝< 1) if 𝑃𝜃[𝐿1 ≤ 𝜃 ≤ 𝐿2] = 1−∝ for all 𝜃 𝜖 𝛺 𝐿1 and
𝐿2 are called the lower and upper confidence limits, respectively at least one of them should not be a
constant.

Interval Estimation
Estimation of a parameter by a sample value is known as point estimation. An alternation produce is to
give an interval within which the parameter may be supposed to lie with high probability. This is called
interval estimation and the interval is called the confidence for the parameter

Suppose 𝑎, 𝑟, 𝑣 x has Normal distribution 𝑁(𝜇, 𝜎) with unknown mean 𝜇 and known st. Deviation𝜎. Let
(𝑥𝑖, … , 𝑥𝑛) be the values of a random sample of size or from then distribution .We know that the sample

mean and, hence . It follows that

Or, equivalently,
 68

This shows that, in respected sampling the probability is 0.95 that the interval

Will include 𝜇, We say that above is a confidence interval for 𝜇 with confidence coefficient ,95. The two
end points are known as 95% confidence limits for𝜇.

Let us now consider the general problem Let 𝑎, 𝑟, 𝑣 x has distribution depending on an unknown
parameter 𝜃 which is to be estimated. Suppose Z is a statistics (usually it is a function of a sufficient
statistics if it exists) which is a function of 𝜃 but whose distribution does not depend on𝜃. Such a
statistics z is called a ploetal function Let 𝜆1 and 𝜆2 be two numbers such that

𝑃{𝜆1 ≤ 𝑍 ≤ 𝜆2} = 1−∝ − (1)

For a specified ∝ (𝑜 <∝< 1)

The above inequality can be solved such that it assumes the from

𝑃{𝜃1((𝑥1, … , 𝑥𝑛)) ≤ 𝜃 ≤ 𝜃2(𝜆1, . . 𝜆2)} = 1−∝

For all 𝜃 where 𝜃1and 𝜃2are random variables which do not depend on𝜃.Finally, if we astute the sample
value [𝜃1((𝑥1, … , 𝑥𝑛)), 𝜃2((𝑥1, … , 𝑥𝑛))] becomes a confidence interval for 𝜃 with desired confidence
coefficient 1−∝.

Remark: the numbers 𝜆1, 𝜆2 may be chosen in several ways, giving rise to several confidence intervals.
We usually choose confidence intervals of shortest length.

Example (i) 𝑋~𝑁(𝜇, 𝜎) where 𝜎 is Known and 𝜇 is to be estimated

Which has 𝑁(𝑂, 𝐼) distribution For a specified∝ let 𝑁∝/2 be the % critical value of 𝑁(𝑜, 1)then

Or

So that

Isa confidence interval of 𝜇 with confidence coefficient (𝑖−∝)

(2) . 𝑥~𝑁(𝜇, 𝜎), 𝜎 unknown and 𝜇 to be estimated

where

Then z has t(n-i) distribution , so that for a specified ∝,

Or

So that
 69

Is a confidence interval of 𝜇 with confidence coefficient (1−∝)

(3) 𝑥~𝑁(𝜇, 𝜎), 𝜇 known and 𝜎 is to be estimated

Then z has 𝑥2(𝑛) distribution, so that for a specified ∝

Or

There, 𝑎(1−∝)% confidence interval of 𝜎2

(4) 𝑥~𝑁(𝜇, 𝜎), 𝜇 Unknown and 𝜎 is to be estimated

Where

Then z has 𝑥2(𝑛) distribution, such that

Or

Therefore, a confidence interval of is

(5) Let x have an exponential distribution with parameter λ which is to be estimated

Then Z has ditribution, so that for a specified

Or

Therefore, a confidence interval of λ is

(6) Let X ~N and γ ~N where . We want a confidence for

Where are usually defined

Then Z has distribution, such that

 70

Or

So that a confidence interval for is

With confidence coefficient 1

(7)Let X ~N and γ ~N where are unknown and it is requested to obtain a

confidence interval of

Let Z

So that Z has F distribution on

Then

Or

Or

So that

Is a confidence interval of with confidence coefficient

(8) Simultaneous confidence region for for a normal distribution.

Let with unknown

One many chose a confidence region for using the two relations

Diagrammatically shown as the shaded region below

Where

But it is difficult to find the probability of the sample to full in the shaded region (confidence region)

Alternatively, using the independence of and we chose the cofidence region by the help of relation
 71

A,d Since

are indept

Chosing such that we can

Obtain the boundaris of the confidence .region without difficully this is shown by the shaded region
below

Where

Approximate confidence intervals(for large samples)

Let x be bernoulli with

we want to find confidence interval for P.

For lage sample size ,n, we have

Or

Where þ is the sample propostion

Them , approxi mately ,

Or

So that

Is a confidence interval for P

(II) For two sample we can similerly find a confidence interval for as follows:

Where

So that

Is a confidence interval for

 72

(iii) Let x be having mean , variance and we want a confidence interval for For that

approximately .

Or

Then

Is a confidence for

(iv) For two sample we an similerly find a cofidence interval for as follows:

Where

So that

Is a confidence interval for

(v) Let have a bivanate normal distribution with coefficient P and me want to find a confidence
region for P.

By using Fisher,s Z transformation

and

whose is the corr crofficient in a sample of

size n

Then So

that

Or

So that

Gives a confidence interval for ξ.From this we can earily obtain the corrponding confidence
interval for P.
 73

NON-PARAMETRIC INFERENCE

In all problems of statictics inference considered so fan we assumed that the distribution of the random
variable breing sampled is know n except for some parameters . in pratice however the functional
from in the distribution is seldom if ever , known if is therefore desivable to devise some produres
that are free from this assumption concering distribution such produres are commonly refered to as
distribution free or non-parametric methods the term distribution free refers to the fact that no
assumptions are made about the underlying distribution execpt that the distribution function being
sampled is absolutely continuous or purely discrete. The term non-parametric refers to the factors that
there are no parameters involved in the traditional sense of the parameter used so for.

We will consider only the inferential problem of testing of hypothesis and dercribe a few
non-parametrictests

Single- sample problems : (a)The problem of fit : the problem of fit is to test the hypothesis that a
sample of obsevations (𝑥𝑖, 𝑥𝑛) is from some specified distribution against the alternative that it is from
some other [Link] we have to test

𝐻𝑜: 𝑥~𝐹𝑜(𝑥) = 𝐹𝑜(𝑥)

Against 𝐻𝑜: 𝑥~𝐹(𝑋) ≠ 𝐹𝑜(𝑥)for some 𝑥

(i)Chi- square test: Let there be 𝓀 categories and let þ𝑖 be the probality of a random obsevation from
𝐹𝑜(𝑥) to fall in the 𝑖𝑡ℎ category (𝑖 = 1,2, … . 𝑛).For a sample of size n, Let 𝑜𝑖 be the obsevarved freqnecy
in the 𝑖𝑡ℎ category and let 𝑒𝑖 = 𝑛þ𝑖 be the expected frequency in the 𝑖𝑡ℎ category under 𝐻𝑜. To test 𝐻𝑜
we use the chi-square statics

The larger the value of 𝑥2 the more likely it is that the 𝑜𝑖,𝑠 did not come from 𝐹𝑜(𝑥). The 𝑥2 −statistic
for large samples has a 𝑥2 distribution on (𝓀 − 1)d.f .Thus an approximate level ∝ test is provided by
rejecting 𝐻𝑜 if

𝑥2 > 𝑥𝓀2−1∝,

(ii)Kolmogoror – Smironv one sample test : For the sample (𝑥𝑖,… 𝑥𝑛)let the empirical distribution
function 𝐹𝑛(x) be given by

𝑜 𝑥(𝑖)
𝑖𝑓 𝑥 <
𝐹𝑛(𝑥) {𝓀⁄𝑛 𝑖𝑓 𝑥(𝓀) ⪕ 𝑥 < 𝑖𝑓 𝑥(𝓀−𝑖)
𝑖 𝑥⩾ 𝑥(𝑛)
(𝓀 = 1,2, … 𝑛, −1) whese 𝑥(1),𝑥(2), … . 𝑥(𝑛) are the order statistic , Evidently ,

For testing 𝐻𝑂: 𝐹(𝑥) = 𝐹𝑜(𝑥) against the two sided alternative 𝐻𝑖: 𝐹(𝑥) ≠ 𝐹𝑜(𝑥) we use the Kolmogoror –
Smironv statictic

It can be shown that the K-S statistic 𝐷𝑛 is completely distribution free for any continouns distribution
𝐹𝑜(𝑥)

At level ∝, Kolmogoror – Smironv test rejects 𝐻𝑂 if

 74

𝐷𝑛 > 𝐷𝑛,∝

Whese 𝑃(𝐷𝑛 > 𝐷𝑛,∝) ⪕∝

Tables of 𝐷𝑛,∝ for given ∝ and n are available

Remark1:For testing𝐻𝑂: 𝐹(𝑥) = 𝐹𝑜(𝑥) against one-sided alternatives 𝐻1: 𝐹(𝑥) > 𝐹𝑜(𝑥) or 𝐻2:𝐹(𝑥) <
𝐹𝑜(𝑥) based on one-sided K.S statistics and 𝐷𝑛− are also available

Remark 2: For small sample 𝑥2 −test is not available but K.S test can be applied. For discrete distibution
[Link] is not availible but 𝑥2 −test can be appled K.S test is more powerful then 𝑥2 −test.

(B) The problem of Location: Let (𝑥𝑖, … . 𝑥𝑛) be a radom sample from a distribution 𝐹(𝑥) with unknown
median ξ ,where 𝐹(𝑥) is assumed to be continus in the neigbourhood of ξ. By definition of median

.We would like to test the hypothesis (x) If n>25, normal appronimution may be used

We take

𝐻𝑜: 𝜉 = 𝜉𝑜 against one sided or two sided alternatioes

Sign Test: We from the n differences (𝑥𝑖 − 𝜉𝑂)𝐼 = 1,2 … … … 𝑛 and find out the number, R,of position
differences (differences having postive signs ) 𝑖, 𝑒 when (𝑥𝑖 − 𝜉𝑂) > 𝑜.
1
If 𝐻𝑂 is true, and R has a Biomial distribution with paramer2 .
We

may use an exect test of𝐻𝑂 based on the Biomial Distribution. In the case of one-sided alternative

𝐻𝑖: 𝜉 > 𝜉𝑜

The sample will have an excess of positive signs and in the case of

𝐻𝑖: 𝜉 > 𝜉𝑜

The sample will have a small number of postive signs

The signs test based on R, for testing 𝐻𝑂 can be summarised as follows :

The critical values 𝑅1∝, 𝑅2∝, 𝑅∝/2, 𝑅∝/2 are calculate from tables of Biomaial distribution

Rajred –sample signs test: Here we assume that we have a random sample of n pains (𝑥𝑛, 𝑥𝑛) giving the
the differences

𝐷𝑖 = 𝑥𝑖 − 𝑦𝑖 ,𝑖 = 1, … 𝑛

It is assumed that the distribution of D=X-Y is absolutely continous with median ξ

We have , now a single sample 𝐷𝐼, … . . 𝐷𝑛 and we can test 𝐻𝑜: ξ = 𝜉𝑜 which can be taken to be oby the
sign test descrited above.

Remark the above two sign test s are , repectively aralogoun to single sample 𝑡 − 𝑡𝑒𝑠𝑡 and paired ttest
for testing location of a normal distribution ,

Two sample problems : let (𝑥𝑖,… … 𝑥𝑛) and (𝛾𝑖, … … 𝛾𝑛) be independent random sample s from two
absolutely continous distribution 𝐹𝑥(𝑥) and 𝐹𝛾(𝓎) , respectively

Suppose we want to test

 75

𝐻𝑜: 𝐹𝑥(𝑥) = 𝐹𝛾(𝓎) for all 𝑥

Against 𝐻𝑖: 𝐹𝑥(𝑥) ≠ 𝐹𝛾(𝓎) for same 𝑥

Run test(Wald –Wolfowitz): we assarge the m, x’s and n 𝛾′𝑠 in increasing order of size
and count the numbers of runs .if 𝐻𝑜 is true the (m+n) values will be well mixed up and
we expect that R, the total number of runs , will be relatively large. But R will be small if the samples come
from differernt popaltions 𝑖, 𝑒 𝐻𝑜 is false in the extreme case , if all the value of y are greater than all the
value of x, or vice – vera , there will be only two runs

The run test of 𝐻𝑜 against 𝐻𝑖 at level ∝ is to reject 𝐻𝑜 if

𝑅 ⪕ 𝑅∝

Where 𝑅∝ is the largest interteger such that

𝑃(𝑅 ⪕ 𝑅∝/𝐻𝑜) ⪕∝

It can be show that distribution of R, under 𝐻𝑜 is given by

And

Tables of critical values of R based on above have been given by swed and Eisenhant For

large m,n(both greater then 10), Ris asymptohcally Normally distributed with

And

Median it test: We arrange the x’s and y’s in asscending order of size and find the median M of the
contied sample let

𝑉 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑥′𝑠𝑤ℎ𝑖𝑐ℎ 𝑎𝑟𝑒 ⪕ 𝑚𝑒𝑑𝑖𝑎𝑛 𝑀

If V is large it is reasomable to conclude that the actual median of x is smaller than the median of Y
𝑖, 𝑒 𝐻𝑜: 𝐹𝑥(𝑥) = 𝐹𝑌(𝑥) is respected

Hown of 𝐻𝑖: 𝐹𝑥(𝑥) > 𝐹𝑌(𝑥) −

On the other hand , if V is too small it is reamable to condude that the actual median of X is greater
than the median of y 𝑖. 𝑒 𝐻𝑜: 𝐹𝑥(𝑥) = 𝐹𝑦(𝑥)is respected in fovoues of 𝐻𝑖:𝐹𝑥(𝑥) < 𝐹𝑌(𝑥)

For the two sided alternative , we use the two sided test .

The median test can be summarised as follows:

It can be shown that the distribution of V, under 𝐻𝑜 is given by

𝑚 𝑛

Where 𝑚 + 𝑛 = 2þ, þ positive integer

And
 76

𝑚 𝑛

Where 𝑚 + 𝑛 = 2þ + 1, þ𝑖𝑠 𝑎 positive integer

Wilcoxon- Mann –Whitney U test: This is the most widely used two- sample non-parametric test and is
a useful alternative to the t-test assumotions.

The test is like the run test based on the pattern of 𝑚, 𝑥′𝑠 and 𝑛, 𝑦′𝑠 arranged in ascending order of size .
The Main- Whitney U statistic is defined as the number of times as X preades 𝑎 𝑌 In the combined sample
of size 𝑚 + 𝑛. We define

And write

Note that is the number of 𝑦𝑗′𝑠 that are larger than 𝑥𝑖 and hence U is the number of values of
𝑥𝑖, … … … 𝑥𝑛 that are smaller than each of 𝑦𝑖,… … … , 𝑦𝑛. For example , suppose the contined sample
when ordered is as follows :

𝑋2 < 𝑋1 < 𝑌3 < 𝑌2 < 𝑋4 < 𝑌1 < 𝑋3

Then U=7, becouse there are three values of X<𝑌1, two values of X<𝑌2 and two values of X<𝑌3

It is obseved that U=0 if all the are larger than all and U=mn of all the are smaller than all
the 𝑦𝑖′𝑠. Thus 𝑜 ⪕ 𝑈 ⪕ 𝑚𝑛. If U is large the values of y tend to be larger than X (Y is stochastically larger
than X) and this supposts the alternative 𝐹𝑥(𝑥) > 𝐹𝛾(𝑥). Similarly, if U is small, the values of Y tend to be
smaller than X and this supposts the alternative 𝐹𝑥(𝑥) > 𝐹𝛾(𝑥).

Thereforer , U-test can be summarised as follows:

𝐻𝑜 𝐻𝑖 𝑅𝑒𝑗𝑒𝑐𝑡𝐻𝑜 𝑖𝑓

𝐹𝑥(𝑥) = 𝐹𝛾(𝑥) 𝐹𝑥(𝑥) > 𝐹𝛾(𝑥). 𝑈 ⩾ 𝐶1

𝐹𝑥(𝑥) = 𝐹𝛾(𝑥) 𝐹𝑥(𝑥) < 𝐹𝛾(𝑥) 𝑈 ⪕ 𝐶2

𝐹𝑥(𝑥) = 𝐹𝛾(𝑥) 𝐹𝑥(𝑥) ≠ 𝐹𝛾(𝑥) 𝑈 ⩾ 𝐶3𝑜𝑟 𝑈 ⪕ 𝐶4

It can be shown that Under 𝐻𝑂

And

The tables of distribution of U for small samples are given by table and Mann-Whitney. For large samples
U has asymptotic normal distribution,𝑖, 𝑒

APPENDIX

Distribution of function of random variables (transformations method)

 77

Therom: suppose Xis a continuous 𝑟, 𝑢 with þ, 𝑑, 𝑓 𝑓𝑥(𝑥). Set 𝑥 = {𝑥,𝑓𝑥(𝑥) > 𝑜}.Let

(i) 𝓎 = ℊ(𝑥) difine a d.f transformation of 𝑥 anto 𝑥

(ii)the derivative of 𝑥 = ℊ−1(𝑥) 𝜔. 𝑟. 𝑡 𝓎 is continous and non-zero for 𝓎 𝜖 𝑥, where ℊ−1(𝓎) is the

inverse for of 𝓎(𝑥) 𝑖, 𝑒 ℊ−1(𝓎) isthat 𝑥 for which ℊ(𝑥) = 𝓎 Then 𝛾 = ℊ(𝑥) is a cont. 𝑟, 𝑢 with þ, 𝑑,

𝑓.

Therom : let 𝑥1 and 𝑥2 be jointly continous 𝑟. 𝑢. 𝑠 with þ, 𝑑, 𝑓 𝑓𝑥1,𝑥2(𝑥1, 𝑥2). Set 𝑥 =

{(𝑥1, 𝑥2): 𝑓((𝑥1, 𝑥2) > 𝑜}Assumu that

(i)𝓎1, = ℊ1(𝑥1, 𝑥2) and 𝓎2, = ℊ2(𝑥1, 𝑥2) defines i:i transformation of x onto x.

(ii)The first partical of derivatives of are continous over x.

(iii) The jacebian of transformation is non-zero for (𝓎1𝓎1)𝜖𝑥.Then the joint þ, 𝑑, 𝑓 of 𝛾1 = ℊ, (𝑥1,
𝑥2)and 𝛾2 = ℊ, (𝑥1, 𝑥2)is given by

Where

X2- distribution

Definition : A continous 𝑟, 𝑢, 𝑥 is said to have the X2- distribution on n degrees of freedom if its þ, 𝑑, 𝑓 is
given by

=𝑜 𝑥<𝑜

The 𝑚, ℊ, 𝑓 of x is given by

From this we can earily show that

𝐸(𝑋) = 𝑛 𝑎𝑛𝑑 𝑣(𝑥) = 2𝑛

For 𝑛 ⪕ 2the þ, 𝑑, 𝑓 of 𝑥2(𝑛) steadily dencress as 𝑥 iscrese while for 𝑛 > 2 there is a uniqne maximum
at 𝑥 = 𝑛 − 2

Theorom : Let 𝑥1, 𝑥2 … … . . 𝑥𝑛 be n independent standand normal r,v,s 𝑖. 𝑒 𝑥𝑖~𝑁(𝑜, 1), 𝑖 = 1, … 𝑛 Then
has a X2- distribution on 𝑛, 𝑑, 𝑓.
 78

Proof: Let X be 𝑁(𝑜, 1) the 𝑚, ℊ, 𝑓 of 𝑥2 is given by

Which show that 𝑥2~𝑥2(1) Then , the is given by

𝑀𝑋2(𝑡) = [𝑀𝑋2(𝑡)]𝑛 = (1 − 2𝑡)−𝑛/2

Which shows that 𝛾~𝑥2(𝑛)

Therom : Let 𝛾1, 𝛾2 … . 𝛾𝑛 be indepent 𝑟, 𝑢, 𝑠 with X2- distribution on 𝑛𝑖, … … . 𝑛𝓀 degrees of freedom
resp .

Then

Proof :the 𝑚, ℊ, 𝑓 Z

𝑀𝑍(1) = 𝐸𝑒𝑡𝑧

= (1 − 2𝑡)−(𝑛𝑖+..+𝑛𝓀)/2

Which about that y~𝑥2(𝑛𝑖 + ⋯ + 𝑛𝓀)

Crollanj : Let (𝑥𝑖, … . . 𝑥𝑛)be a random simple from a Normal distributuion 𝑁(𝜇, 𝜎).Then has
𝑥2 distribution on 𝑛, 𝑑, 𝑓.

Therom: Let (𝑥𝑖, … . . 𝑥𝑛)be a random simple from a Normal distributuion

And be the sample mean and sample variance. Then has 𝑥2 distribution on
(𝑛 − 𝑖)𝑑, 𝑓.

Therom: For large 𝑛, √2𝑥2 can be shown to be approximately normally distributred with mean
and st-dearation unity.

Therom: Assume that y has distribution function 𝐹𝑌 which satifies some regularity conditions ad which
has r-unknown parameters 𝜃1, 𝜃2 … . 𝜃𝑟 and that (𝑦𝑖,. . 𝑦𝑛) is a random sample of [Link] 𝜃𝑖, 𝜃𝑟 be the 𝑚.
ℓ, 𝑒 of 𝜃′𝑠 .Suppose the sample is distribution in 𝓀 non-orerlapping intervals {𝐼𝐽} where
𝐼𝐽 = {𝓎: 𝑎𝑗−𝑖 < 𝑦 < 𝑎𝑗−𝑖},𝑗 = 1, … 𝓀(𝑎𝑜 = −∞𝑎𝓀 = ∞and . Let 𝑥𝑖,… . . 𝑥𝓀 be the number of sample values
falling in these inervals, respectively if me define
 79

𝑝𝑗 = 𝑃{𝑌𝑓𝑎𝑙𝑙𝑠 𝑖𝑛𝐼𝐽}, 𝑗 = 1, … 𝓀

Where 𝜃𝑖, 𝜃𝓀 replace 𝜃𝑖, 𝜃𝓀 in 𝐹𝑦 ,then the distribution of the statistics Lerger is
appoximately distributed as 𝑥2on 𝓀 − 𝑟 − 𝑖 𝑑, 𝑓 as n gets

Students t-distribution

Definintion : A Continous 𝑟, 𝑢, 𝑥 is said to have the t-distribution on 𝑛, 𝑑, 𝑓 if its þ, 𝑑, 𝑓 is given by

Remark : For 𝑛 = 𝑖the þ, 𝑑, 𝑓

Which shows that it is a couchy distribution We will therefore, assume that 𝑛 > 𝑖

Remark:the þ, 𝑑, 𝑓 of t-distribution is symmctric about again. For large n the t-distribution tends to
Normal distribution. For small n hawever t-distribution deviates considerally from the normal in fact if
𝑇~𝑡(𝑛)and 𝑧~𝑁(𝑜, 𝑖)

𝑃{[𝑇] ⩾ 𝑡𝑜} ⩾ 𝑃{[𝑍] > 𝑡𝑜}

Moments : Since the distribution a symmetrial about origin 𝜇2𝑟 + 1 = 0 For

2r<n

Therom : Let 𝑥~𝑁(𝑜, 1) and 𝑦~𝑥2(𝑛) and Let 𝑥and 𝑦 be independent .Then
80