A.

8 Kronecker Product
Let A be an m n matrix and B be a p q matrix. Then
0 1
a11 B a12 B a1n B
B .. C
A B=@ . A mp nq
am1 B am2 B amn B

Property: Provided the matrices are conformable

(A B)(C D) = (AC BD)

(A B) 1 = A 1 B 1
(A B)0 = A0 B 0
A (B + C) = (A B) + (A C)

B Statistical Review
B.1 Random Variables and Probability Distribution
Objective of Empirical Economist

The goal is to learn from a subset of observations (sample) about the whole pop-
ulation.

– Example: our interest is to learn about the mean food expenditures by UK

families.

We assume there is an unknown process that generates the data (sample) we have

– We can described this data generating process by a distribution function or

probability density function.
Frequentist interpretation: A sample is one particular outcome of a –
statistical experiment where we draw repeatedly from this distribution.

B.1.1 Cumulative Distribution Function and Probability Density Function

Cumulative Distribution Function (univariate)

Pr(X x) = F (x) Cumulative Distribution Function (CDF),

96
 F (1) = Pr(X < 1) = 1
F ( 1) = 0
If x > y then F (x) F (y)
0 F (x) 1

F (x)

– The probability associated with the continuous random variable X taking

any particular value x is zero (Pr(X = x) = 0)
– Continuous Random Variable has a smooth, non-decreasing CDF.

Probability Density Function (univariate)

@F (x)
@x
f (x); is the Probability Density Function (PDF):

– The p.d.f. is de…ned so that f (x) 0 and

Rb
RPr(a
1
X b) = a
f (x)dx = F (b) F (a) 0
1
f (x)dx =1

Graphically this implies that the probability that the random variable
takes values between a and b is the highlighted area under the p.d.f.

y = f (x)

Multivariate distribution theory

Vector of rv’s X = (X1 ; ::; Xn ) : multivariate distribution theory.

Joint Distribution Function

Pr(X1 x1 ; X2 x2 ; : : : ; X n xn ) = F (x); where x is a vector

97
 Joint Density Function

@ n F (x1 ; x2 ; : : : ; xn )
f (x1 ; x2 ; : : : ; xn ) =
@x1 @x2 : : : @xn

The Marginal Density Function of the random variable X1 from the joint pdf
f (x1 ; :::; xn ; ) is given by
Z 1 Z 1
f1 (x1 ; ) = ::: f (x1 ; :::; xn ; )dx2 :::dxn
1 1

– Integrate out all the other random variables.

B.1.2 Conditional Density Function

Sometimes we may wish to describe the probability of events given that something else
has happened.

Let f (x; y) bivariate distribution of random variables X and Y; the Conditional

Density Function:

f (x; y) f (x; y)
f (yjx) = and f (xjy) = (Bayes’Theorem)
fX (x) fY (y)

– f (yjx) : distribution of y for each value of (conditional on) x

– If X and Y are independent: f (yjx) = fY (y) and f (xjy) = fX (x)

B.1.3 Independence
f (x; y) = f (yjx)fX (x) and/or f (x; y) = f (xjy)fY (y)

– ) If X and Y are independent: f (x; y) = fX (x)fY (y)

B.2 Random Sampling

Objective of Empirical Economist

A sample we use to learn something about the whole population should be repre-
sentative of the population.

98
 One way to do so is to use random sampling, whereby each member of the popu-
lation has an equal chance of being included in the sample

Let X1 ; :::; Xn be a sample of n observations on one or more variables

De…nition: X1 ; :::; Xn is a random sample if the n observations are drawn inde-

pendently from the same population

– The sample of observations is said to be independent, identically distributed

(i.i.d.).

The joint p.d.f. of the random sample fX1 ; X2 ; :::Xn g is

independence
– f (x1 ; :::; xn ; ) = f1 (x1 ; )f2 (x2 ; ) : : : fn (xn ; )
identical Q
– = f (x1 ; )f (x2 ; ):::f (xn ; ) = ni=1 f (xi ; )

B.3 Estimators
Objective of Empirical Economist

Call a statistic that is intended to serve as a basis for learning about an unknown
quantity (parameter) an estimator . Denote by b:

– Example: the average of a random sample on food expenditures in the UK is

an estimator of the mean of food expenditures in the UK

De…nition: Statistic is a random variable Q which is a function of the sample

X = fX1 ; :::; Xn g.

– It can be computed from the data in a sample, and does not involve any
unknown quantities

Estimators are random variables

Estimate: A realized value of an estimator is said to be an estimate.

2
– Example: A particular estimate of X is x, and particular estimate of SX is
P
n
s2x = n 1 1 (xi x)2 :
i=1

99
 Table 1: Parameters of Interest and Estimators
Population moments: Possible Estimator:
(sample moments)
P
N
Mean: X = N1 Xi
i=1
X = E(Xi ) = Sample average

2 1
P
N
Variance: SX = N 1
(Xi X)2
i=1
2 2
X = E((Xi X) ) = SampleV ar(Xi )

1
P
N
Covariance SXY = N 1
(Xi X)(Yi Y)
i=1
XY = E((Xi X )(Yi Y )) = SampleCov(Xi ; Yi )

– For a given UK sample compute the average food expenditures

Sampling Distribution

– Random sample: expect a close resemblance between sample moments and

population moments
If another sample was drawn under identical conditions, di¤erent values
would be obtained.
– The Sampling Distribution (p.d.f. of estimator) reveals the way in which the
sample moments re‡ect the population moments (realizations of estimator
under repeated sampling)
– We use the sampling distribution for Hypothesis testing purposes.

B.4 Moments of Random Variables

Moments provide useful information about random variables
The mean of a random variable:
Z 1
E(X) = xf (x)dx : "average tendence"
1

The variance of a random variable:

Z 1
2
Var(X) = E((X E(X)) ) = (x )2 f (x)dx 2
1
2 2
= E(X ) E(X) "variability"

100
 – mean=1st moment about zero; variance=2nd moment about the mean

rth (population) moment of a random variable about the point a is

E((X a)r )

– With a = E(X); r = 3 addresses symmetry of distribution (skewness) and

r = 4, thickness of tails (kurtosis)

For any constants a and b:

E(a + b:X) = a + b:E(X)

It is a linear operator.

BUT Jensen’s inequality:

Z 1 Z 1
E(g(X)) = g(x)f (x)dx 6= g(E(X)) = g xf (x)dx
1 1

– Example: E(X 2 ) 6= [E(X)]2

Law of Iterated Expectations:

E [E(Y jX)] = E(Y )

Covariance tells us something about co-varying of random variables:

The Covariance Matrix is a multivariate 2nd order moment about the mean.

– For the random variables X1 ; :::; Xn it is an n n matrix with elements

Cov(Xi ; Xj ) = ij = E((Xi EXi )(Xj EXj )); i; j = 1; :::; n
ij measures the tendency of Xi and Xj to be “associated”
ij > 0 ( <0) tend to observe Xi and Xj moving in the same (opposite)
direction

2
– Diagonal of covariance matrix contains the variances ii = i

– O¤-diagonal of covariance matrix contains the covariances ij

– Covariance matrix is symmetric: ij = ji

– Covariance matrix is positive semi-de…nite.

101
 The Correlation Matrix is a matrix closely related to the covariance matrix:
Cov(Xi ; Xj ) ij ij
ij = =p =
StdDev(Xi )StdDev(Xj ) ii jj i j

– Diagonal of correlation matrix contains 1’s. A variable is always perfectly

correlated with itself

– Correlation matrix is symmetric

– Correlation is scale-free: Corr(ai + bi Xi ; aj + bj Xj ) = Corr(Xi ; Xj )
– Correlation is a measure of linear relatedness only

B.4.1 Independence and Uncorrelatedness

Independence ) Zero covariance
?
– Cov(X; Y ) = E((X X )(Y Y )) = E(XY ) X Y = 0
RR indep R R
E(XY ) = xyf (x; y)dxdy = xyfX (x)fY (y)dxdy
R R
= xfX (x)dx yfY (y)dy = X Y = E(X)E(Y )

So Cov(X; Y ) = E(X)E(Y ) X Y =0

BUT

Zero covariance 6) independence (exception under the normal distribution).

B.5 Sampling distribution of Estimators (Examples)

B.5.1 Sampling Distribution of the Sample Mean
2
If Y1 ; :::; Yn is a random sample from a population: mean ; variance ; i.e.,
Y1 ; :::; Yn i.i.d. ( ; 2 )

– ) Y is a rv with mean and variance 2 =n (cannot say anything more than

this!)
Pn Pn Pn
E(Y ) = E( n1 Yi ) = n1 E(Yi ) = n1 =
i=1 i=1 i=1
" #
h 2
i Pn 2
Var(Y ) = E( Y EY = n12 E (Yi )
i=1

102
 " #
1
P
n P
n
= n2
E [(Yi )2 ] + E [(Yi )(Yj )]
i=1 i6=j

1
P
n
= n2
Var(Yi ) + 0 by Independence
i=1

1
Pn
2
2
= n2
=
i=1 n
2
Y1 ; :::; Yn i.i.d. N ( ; )

–) Y N ( ; 2 =n) (here we can give the full sampling distribution, because

a linear combination of normal random variables is normal)

B.5.2 Sampling Distribution of the Sample Variance

2
If Y1 ; :::; Yn i.i.d. N ( ; ); then
2
(n 1) SY2 2
n 1

– (no proof given here).

B.6 Distributional Results

Some important distributions are discussed next. These distributions are used
heavily when we want to test hypotheses about our population parameters (see
below).

B.6.1 Normal Distribution

2
Scalar: W s N ( ; )
1 1
f (w) = p exp( 2
(w )2 )
2 2 2

– 2 parameters and 2
continuous, symmetric, unimodal
positive everywhere (bell-shaped)

103
 2
– Width: :

Multivariate: W s N ( ; );
W; : n dimensional vector, : n dimensional covariance matrix
n 1 1
f (w) = (2 ) 2 (det ) 2 exp( (w )0 1
(w )):
2
2
– When W s N (0; I) this simpli…es to (oft used joint density)

1 X
n
2 n=2
f (w) = (2 ) exp( 2
wi2 )
2 i=1
2
= f (w1 )::f (wn ), where Wi i.i.d. N (0; )

Important properties of normal random variables:

If W has a multivariate normal distribution, then all marginals are also normal:
2
W s N ( ; ) ! Wi s N ( i ; i ):

All linear transformations of W are normal

– V = c + BW; with c a constant vector and B a constant matrix

V s N (c + B ; B B 0 )

If X and Y are (jointly) normal, then

X and Y are uncorrelated ) X and Y independent.

2 X
If X s N ( ; ) then Z = s N (0; 1)
x x
Pr(X x) = Pr(Z )= ( )

is the CDF of the standard normal which is tabulated (see table A.2).

– Pr(X x) does not have a simple formula (integral)

104
B.6.2 Chi-Squared Distribution
If Zi , i = 1; ::p; are independent N (0; 1) random variables, then
p
X
W = Zi2 2
p
i=1

– The 2p density function has a single parameter, p, the number of degrees of

freedom.
2
–A p random variable only takes positive values and is skewed to the right.

2 2
– df( )red/solid < df( )blue/dashed

2
– If W p; E(W ) = p; Var(W ) = 2p

B.6.3 Student-t Distribution

If Z is a N (0; 1) random variable, and X is a 2p random variable, then provided
Z and X are independent:
Z
t= p s tp
X=p

– The tp density function has a single parameter p; the degrees of freedom (df).
Symmetric about zero; unimodal; positive everywhere; leptokurtic (i.e. there
is more area in the tails relative to a normal).

– 0.4 df(t)red/solid >df(t)blue/dashed

0.2 E(t) = 0 for p > 1 (1 otherwise)
Var(t) = p=(p 2) for p > 2 (1 otherwise)
-2 0 2

– We can use the tables for the normal distribution if p is large: t N (0; 1)
when p ! 1:

105
B.6.4 F-Distribution
If X is a 2m random variable and Y is a 2k random variable, then provided X and
Y are independent
X=m
F = s Fm;k
Y =k
– The Fm;k distribution has 2 parameters m and k. It is positive and skewed to
the right.

– E(Fm;k ) = k k 2 for k > 2

= (1 otherwise)

– If F1 Fm;k , then F2 = 1=F1 Fk;m :

Pr(F1 < a) = Pr(1=F1 > 1=a) = Pr(F2 > 1=a) = 1 Pr(F2 < 1=a)
– Suppose T t(m): Then W T 2 F1;m :
p
Proof, T = N (0; 1)= 2 =m and T 2 = ( 2 2
m 1 =1) = ( m =m) :

B.6.5 Examples
2
Let X1 ; X2 ; :::Xn be a random sample of size n from N ( ; ) then
p
n(X )
s tn 1
SX
– Note p
p n(X )
n(X )
=p
SX 2 (n 2
1)SX =(n 1)
p
n(X )
N (0; 1);
2 2 2
(n 1)SX n 1; and
p
n(X ) 2 2
and (n 1)SX two independent random variables.
2
Let SX and SY2 be the sample variances from mutually independent samples of
sizes m and n respectively drawn from normal distributions. Let X and Y have
population variance 2X and 2Y respectively:
2 2
Y SX
2 2
s Fm 1 ;n 1
X SY

106
 – Note
2 2 2 2
Y SX X (m 1)SX =(m 1)
2 2
= 2
X SY Y (n 1)SY2 =(n 1)
2 2 2
X (m 1)SX m 1,
2 2 2
Y (n 1)SY n 1 , and
2
X (m 1)SX and Y 2 (n 1)SY2 are two independent random variables.
2

B.7 Properties Estimators

Desirable Properties of Estimators

Classical Frequentist interpretation

– The relevant issue is NOT the closeness of our estimates to the unknown
parameters for a particular realized sample
– BUT the closeness of our estimates on average over repeated samples.

– Example: Consider X as our estimator X : We have obtained a realized

value x using our sample.
Note: We will not ask “How close is x to X in a particular sample”?
BUT
“How close is x on average over repeated samples?” I.e., what is the
expected value of X? Is our estimator unbiased, E X = X ? This is a
…nite sample property!
“How variable is x under repeated drawings?”What is Var(X)? Indicates
the precision of our estimator! Again a …nite sample property!
We like to ask “What is the sampling distribution of X?” This allows
us to make precise statements about the reasonable values X can take.
Hypothesis testing!
“How close is x to X when our sample is arbitrarily large?”
?
lim Pr( X X > ") = 0; ">0
N !1

B.7.1 Finite Sampling Properties

Unbiasedness
b denotes the sampling error, which may be arbitrary large for a given sample

Unbiased: Eb =

107
 – p.d.f. of b

Bias(b) = Eb
Let X1 ; :::; Xn be a random sample from a population with mean
– Example: X is an unbiased estimator of : E(X) =
2 2
– Example: X is a biased estimator of 2 : E(X ) 6= 2

2 Pn P
n
E(X ) = E ( n1 Xi )2 = n 2 E ( Xi )2
i=1 i=1
2
P
n PP
=n E(Xi2 ) + n 2 E(Xi Xj )
i=1 i6=j
Making use of
2
Var(Xi ) = E(Xi2 ) 2
2
E(Xi Xj ) = E(Xi )E(Xj ) = by independence
2
= n 1( 2
+ 2 ) + n 2 n(n 1) 2 = 2 + n 6= 2
2 2
Bias(X ) = n
(asymptotically, though, bias is zero)

E¢ ciency
Unbiasedness is a desirable property but is rarely used by itself as an estimation
criterion.
– Reason: 9 many unbiased estimators that are obviously poor uses of the data.
In judging two unbiased estimators choose estimator with lower variance.

p.d.f.’s of two unbiased estimators

108
 Var(^red/dashed ) <Var(^blue/solid )
n 1P r
Example: Let r = ; and take X r = Xi as an estimator of :
2 r i=1

– Both X and X r are unbiased estimators of

– However, the variance of these two estimators are di¤erent:

2 2 2
Var(X) = E(X ) = =n, and similarly
2 2 2
Var(X r ) = E(X r ) = =r:

So Var(X r ) = 2Var(X):

X r is ine¢ cient relative to X

– Prefer X as an estimator of over X r :

Disgarding observations is NOT an e¢ cient use of your data!

There may be biased estimators that have smaller variances than the unbiased
ones. A criterion that recognizes a possible trade-o¤ between bias and variance is
the Mean Squared Error.

Mean Squared Error:

M SE(b) = E[(b )2 ] = V ar(b) + Bias2 (b)

E¢ ciency: Consider two estimators b1 and b2 of the same unknown parameter .

Then

– b1 is more e¢ cient than b2 i¤

M SE(b1 ) < M SE(b2 )

If both estimators are unbiased: “i¤ V ar(b1 ) < V ar(b2 )

Sampling Distribution

As discussed above:
2
– Say Y1 ; :::; Yn i.i.d. N ( ; )

109
 Estimator Y for (population mean) has the sampling distribution:
2
Y N( ; =n):

The suitably rescaled estimator SY2 for 2

(population variance) has the
sampling distribution:
2
(n 1) SY2 2
n 1

B.7.2 Large Sampling Properties

Maybe in …nite samples we do NOT know
I whether an estimator is unbiased; or
I what the …nite sampling distibution of our estimator is.
(need to compute moments / requires existence of moments).

In this case, the idea is to work with results that come about as n ! 1 and treat
the n ! 1 results as approximations.

– Consistency. An estimator b of is consistent if, when the sample size in-

creases, b gets “closer”to
p
Denoted by: plim(b) = (or b ! )
Example: if our random sample on UK food exp. is in…nitely large, we
would expect the sample average to be arbitrarily close to the mean UK
food exp.
Property of ANY useful estimator!!!
– Limiting distribution- enables us to conduct hypothesis testing (asymptoti-
cally)

Consistency
Su¢ cient condition for consistency (not necessary)

If lim E(b) = and lim V ar(b) = 0

n!1 n!1

110
 – Example: X is a consistent estimator of the population mean,
2
E(X) = and Var(X) = =n ! 0 as n ! 1
2 2
– Example: X is a consistent estimator of
By Slutsky Theorem a direct consequence of the consistency of X (later).
2
Nevertheless, X is a biased estimator of 2 :

plim(x) = g (plim x)

B.8 Point versus interval estimate

Let b be an estimator of a parameter

– Point Estimate: The single realization for b in a particular sample.

Usually we want to know more. How con…dent can we be about our
estimate?
E.g., we estimate that the expected temperature next Saturday at 4pm
is 22C. Does it mean “something between 21C and 23C with very high
probability”or “something between 15C and 29C with high probability”?
– Standard error: SE(^) provides a indicator of the precision of our estimator
The squared root of the estimated variance
Tells us in the above example which statement is more appropriate.
– Interval estimate: Range of values that will contain the true parameter with
a preassigned probability.
Generally, if b is the point estimate, then the Con…dence interval will be
b a measure of sampling error.

B.9 Interval estimation

9 probability that the estimate will be quite far from the truth, regardless of the
properties of an estimator.

– Use the sample data to construct an interval, such that we can expect this
interval to
contain the true parameter in some speci…ed proportion of samples, or
equivalently,
contain the true parameter with some desired level of con…dence.

111
 – The interval boundaries (like the estimates themselves) di¤er depending on
the sample we use.

Let b be an estimator of the parameter .

– A 100(1 )% con…dence interval for is

an interval [lower(b);upper(b)] such that

Pr(lower(b) upper(b)) = 1

Example:
2
XP1 ; X2 :::Xn random sample from N ( ; ); assume 2 is known. ConsiderpX =
1 n
n i=1pXi as an estimator of : A 95% con…dence interval for is [X 1:96 = n; X+
1:96 = n].

2 p X
– X s N( ; n
) sampling distribution ) n s N (0; 1)
p
n(X )
– Pr( 1:96 1:96) = 0:95 (from table A.2)

1:96 1:96
– Pr(X p
n
X+ p
n
) = 0:95

We attach a 100(1 )% con…dence (probability), to the interval. In repeated

sampling, an interval constructed in this fashion will contain the true parameter
100(1 )% of the time.

– In the above example was set at 5%; yielding a 95% con…dence interval.

Example:
2
XP1 ; X2 :::Xn random sample from N ( ; ); assume 2 is known. ConsiderpX =
1 n
n i=1pXi as an estimator of : A 95% con…dence interval for is [X 1:96 = n; X+
1:96 = n].
2
XP1 ; X2 :::Xn random sample from N ( ; ); 2 unknown, n = [Link] X =
1
n
Xi again as a point estimate of : A 95% con…dence interval for is [X
p p
2:086SX = n; X + 2:086SX = n]

– Need to use SX as an estimator for !

112
 p p
– To de…ne the con…dence interval use n XSX instead of n X ! Its distrib-
ution is tn 1 , which has fatter tails than N (0; 1) (Accounts for the fact that
we have to estimate ).
p
n(X )
Pr( 2:086 SX
2:086) = 0:95 (from table A.4, n = 21)

2:086S 2:086S
Pr(X p X
n
X+ p X)
n
= 0:95
– Typically slightly wider!
2
Let X1 ; X2 ; :::Xn be a random sample of size n from N ( ; ) then
p
n(X )
s tn 1
SX

Note p
p n(X )
n(X )
=p
SX 2 (n 2
1)SX =(n 1)
p
n(X )
– N (0; 1);
2 2 2
– (n 1)SX n 1; and
p
n(X ) 2 2
– and (n 1)SX two independent random variables.

B.10 Hypothesis Testing

Devise learning rules about the unknown parameter(s) of the true population
that is generating the data.

Hypothesis test: a rule that determines whether a particular value 0 2 is

consistent with the evidence of the sample.

– is the parameter space –the set of reasonable parameter values.

The Null Hypothesis H0 : = 0

The Alternative Hypothesis can be

– one sided: HA : > 0 (or HA : < 0)

– two sided: HA : 6= 0

113
 The test procedure is a rule, stated in terms of the data, that dictates whether the
null hypothesis should be rejected or not.

The classical, or Neyman-Pearson, methodology involves partitioning the sample

space into two regions.

– If the observed data (i.e., the test statistic) fall in the rejection region then
we reject H0 ;
– If the test statistic falls in the acceptance region,
then do not reject H0 .
– The acceptance/rejection region is de…ned by our willingness to commit a
type I error (signi…cance level)!

Hypothesis Testing: Type of errors

Since the sample is random, the test statistic is also random. The same test can
lead to di¤erent conclusions in di¤erent samples. As such, there are two ways a
procedure can be in error:

– Type I error: The procedure can lead to the rejection of the null when it is
true.
– Type II error: The procedure can fail to reject the null when it is false.

Do not Reject H0 Reject H0

H0 true X Type I error,
H0 false Type II error, X

Type I error: The procedure can lead to the rejection of the null when it is true.

– Probability of Type I error = level of signi…cance ( )

– The level of signi…cance is under the control of the analyst.

Type II error: The procedure can fail to reject the null when it is false.

– Probability of Type II error =

– For a given signi…cance level , we would like to be as small as possible.
Since is de…ned in terms of the alternative hypothesis, it depends on the
value of the parameter.
– = (1 power of the test)
Power of a test is the ability of the test to reject when the null is false!

114
B.10.1 One sided test
2 2
X1 ; X2 ; :::; Xn random sample from N ( ; ); known
Test: H0 : = 0 against HA : > 0

– Consider X as an estimator of
X
– Test statistic Z = p 0
2 =n
s N (0; 1) under H0

– Acceptance/Rejection Rule: Reject H0 if px 0

2 =n
is greater than the critical
value c . The rejection region is (c ; 1), while the acceptance region is
[ 1; c ].

– Choose critical value c s.t

Pr(Z > c ) 1 (c ) =

– If = 0:05, c = 1:645 (see Table A.1 of Maddala)

The p-value
Given n; and x, we could have asked the question: Which levels of signi…cance
( ) would lead us to reject H0 ?

– p-value provides the lowest level of signi…cance ( ) at which we would reject

the null.
– After calculating the Z-statistic
x
z= p0
= n
the p-value is de…ned by:
P (Z > z) = 1 (z)
Example H0 : = 25; HA : > 25 (assume 2 = (2:4)2 and n = 30).
x p 25
Given a sample average x = 26:3; z = 2:4= 30
= 2:97:
From the standard normal table (2:97) = 0:9985 :
p-value = 1 (2:97) = 0:15%
i.e.,we would start rejecting H0 at the 0.15% of signifance (clearly than
would reject at 5% level of signifance as well)

115
B.10.2 Two sided test
2 2
X1 ; X2 ; :::; Xn random sample from X N( ; ); unknown
Test: H0 : = 0 against H1 : 6= 0

– Consider again X as an estimator of .

X
–Z = p 0
2 =n
is no longer a valid test-statistic

Because it contains the unknown quantity: 2

2
Need to use an estimator thereof: SX !
This does change the sampling distribution of our new test-statistic

X 0
p
2
SX =n

– Test statistic T = pX 0
2 =n
s tN 1 under H0
SX

x
– Acceptance/Rejection Rule: We reject H0 if p 0
>c =2 .
s2x =n

–
– Choose critical value c =2 such that Pr(jT j > c =2 ) =

Pr(jT j > c =2 ) = Pr(T > c =2 ) + Pr(T < c =2 )

symmetry
= 2 Pr(T > c =2 ) = 2(1 Pr(T < c =2 ))

– If = 0:05, c =2 = 2:086; N = 21. (see Table A.4).

The associated p-value (the lowest level of signi…cance at which we would reject
the null is given by

x x
Pr(jT j > p 0
) = 2 Pr T > p 0
s2x =n s2x =n

116
B.10.3 Power of a Test
For a given signi…cance level (size) we would like to be as small as possible.

– Equivalently, for a given sign…cance level we want the power of our test (our
ability to reject the null when it is false) to be large!
– If, for a given signi…cance level, the type II error is large: we say that our
test has little power.
Making use of ine¢ cient estimators when performing an hypothesis test
will result in a loss of power of the test.
With ine¢ cient estimators it is more di¢ cult to reject the null, even when
the null is false!
– To compute the power: specify the alternative!
– Power will be stronger for alternatives that are far away from the null (hope-
fully, even getting close to 1!)

Example

X1 ; X2 ; :::; Xn random sample N ( ; 1)

H0 : = 1
H1 : = 2 (one-sided)

– Consider X as an estimator for

X 1
– Test statistic Z = p N (0; 1) under H0
1=N

= Pr(Z > c j = 1)
–
= Pr(Z < c j = 2)

under H0 : Z N (0; 1) and under H1 : Z N ( p 1 ; 1):

1=N

Power function
Power function for a test, expresses the probability of rejection of H0 given a
particular alternative
Pr(Reject H0 jH1 )

117
 Let X1 ; X2 :::Xn s N ( ; 1)
H0 : = 0
H1 : 6 = 0

X
– The decision rule is to reject H0 is to reject if p 0
>c =2 :
1=n

– The power of the test for a given alternative equals

!
X 0
Pr p >c =2 j
1=n

– In order to compute this probability need to realize that given the alternative
X X
p 0
is NOT N (0; 1); but instead p is.
1=n 1=n

Graphically

– (i) The power of this test is equal to the signi…cance level at = 0

– (ii) The test is more powerful the further the true mean is from 0

Example

Power function for a test: X1 ; X2 :::Xn s N ( ; 1)

H0 : =1
H1 : >1

X 1
– H0 : p N (0; 1);
1=n
X 1
Acc/Rej rule: reject H0 if p > 1:645; = 5%
1=n

– Type II error (Power) depends on the alternative we consider:

Say H1 : =2
= Pr(Not reject H0 jH0 is false as = 2)

118
 X 1 X 1
= Pr( p < 1:645j = 2); note p N (0; 1)
1=n 1=n
X 1 p1 X
p1 j = 2); note p 2
Pr( p < 1:645 N (0; 1)
1=n 1=n 1=n 1=n
X 2 p1 j = 2)
= Pr( p < 1:645
1=n 1=n

) = 1:645 p1 use tables (power=1- )

1=n

B.10.4 Hypothesis testing and con…dence intervals

Consider the test
H0 : = 0; H1 : 6= 0

Let b be an estimator of the parameter .

– We reject H0 : = 0 if
b 0
SE(b)
>C =2

where C =2 is such that Pr (Reject H0 jH0 true) = ; or

reject H0 if t 62 [ C =2 ; C =2 ]

A 100(1 )% con…dence interval for is

[b C =2 SE(b); b + C =2 SE(b)]

– If a hypothesized value of the parameter does not fall in this range of plausible
values, then the data are not consistent with the hypothesis, and it should be
rejected.
– We reject H0 if 0 62 [b C =2 SE(b); b + C =2 SE(b)]

B.10.5 Important Examples

T-test
2 2
X1 ; :::; Xn random sample from N ( ; ); unknown
H0 : = 0
H1 : 6= 0

2 2
– Use (X; SX ) as estimators of ( ; ).
X
– Construct statistic T = p 2 0 tn 1 under H0 (see above).
SX =n

119
 – Choose signi…cance level and construct acceptance region [ c =2 ; c =2 ], where
Pr(jT j > c =2 ) = (use table A.4).

– Compute the statistic for your sample (t), and reject H0 if t 2

=[ c =2 ; c =2 ]:

Identical
p to reject Hp 0 if 0 2= [x c =2 SE(X); x +c =2 SE(X)] [x
2 2
c =2 sX =n; x + c =2 sX =n]

Chi-Squared test
2
X1 ; :::; Xn random sample from N ( ; )
H0 : 2 = 2
0
H1 : 2 6= 2
0

2 2
– Use SX as estimators of .
2 2 2
– Construct statistic C = 0 (n 1)SX n 1 under H0 (see above).

– Choose signi…cance level and construct acceptance region [c1 ; c2 ], where

Pr(C < c1 ) = =2 and Pr(C > c2 ) = =2 (use table A.3).

– Compute the statistic for your sample (c), and reject H0 if c 2

= [c1 ; c2 ]:
2
Exercise: Can you construct a 95% con…dence interval for 0?

F-test
Use two independent samples of sizes m and n from normal populations X1 ; :::; Xm ; Y1 ; :::; Yn .
Construct a test for
H0 : 2X = 2Y
H1 : 2X 6= 2Y
2
– Use (SX ; SY2 ) as estimators of ( 2X ; 2Y ).
S2
– Construct statistic F = X2 Fm 1 ;n 1 under H0 (see above).
SY
– Choose signi…cance level and construct acceptance region [c1 ; c2 ], where
Pr(F < c1 ) = =2 and Pr(F > c2 ) = =2. Compute the statistic for your
sample (f ), and reject H0 if f 2 = [c1 ; c2 ]:

– In order to obtain c1 you need to realize that Pr(F < c1 ) = Pr(1=F > 1=c1 )
where the distribution of 1=F under the null if Fn 1 ;m 1 (reverse the degrees
of freedom) (use Table A.6)

120