Solution to Homework 1: Review of Probability and Statistics

ECON20110/30370 Econometrics

Judith Guo
October 1, 2024

Q1 (a) Marginal and Conditional Distributions:

Let’s first calculate the marginal probability mass functions (pmfs) of X and Y :
Marginal pmf of X:
To find the marginal pmf of X, we sum the joint pmf fX,Y (x, y) over all possible values of Y . For
x = 0 and x = 1:

f (0, y) = 0.1 + 0.0 + 0.2 = 0.3 if x = 0
∑y∈{0,1,2} X,Y



fX (x) = ∑ fX,Y (x, y) = ∑y∈{0,1,2} fX,Y (1, y) = 0.3 + 0.1 + 0.3 = 0.7 if x = 1
y∈{0,1,2}



0 otherwise

Marginal pmf of Y :
Similarly, the marginal pmf of Y is found by summing over all possible values of X:




∑x∈{0,1} fX,Y (x, 0) = 0.1 + 0.3 = 0.4 if y = 0


x∈{0,1} f X,Y (x, 1) = 0.0 + 0.1 = 0.1 if y = 1
∑
fY (y) = ∑ fX,Y (x, y) =
x∈{0,1} f X,Y (x, 2) = 0.2 + 0.3 = 0.5 if y = 2
x∈{0,1}
∑





0 otherwise

Conditional pmf of Y given X = x:

The conditional pmf of Y given X = x is defined as:

fX,Y (x, y)
fY |X (y | x) =
fX (x)

Using the joint pmf and the marginal pmf of X, we can compute the conditional pmf of Y for each
value of x:
• For x = 0:

0.1 1 0 0.2 2
fY |X (0|0) = = , fY |X (1|0) = = 0, fY |X (2|0) = =
0.3 3 0.3 0.3 3

• For x = 1:

0.3 3 0.1 1 0.3 3

fY |X (0|1) = = , fY |X (1|1) = = , fY |X (2|1) = =
0.7 7 0.7 7 0.7 7

Thus, the conditional pmf of Y given X = x can be summarised in the following table:

1
Judith Guo ECON 20110/30370

y
fY |X (y | x) 0 1 2
1 2
x 0 3 0 3
3 1 3
1 7 7 7

(b) Independence of X and Y

To check whether X and Y are independent, we need to see if the joint pmf fX,Y (x, y) can be
factored into the product of the marginal pmfs of X and Y , i.e., fX,Y (x, y) = fX (x) fY (y).
This implies that for X and Y to be independent, the conditional pmf of Y given X = x should
equal the marginal pmf of Y , represented as fY |X (y | x) = fY (y).
However, as we can see from the calculations above, this is not the case. For example, fY |X (0|0) =
1
3 while fY (0) = 0.4. Since the conditional and marginal pmfs differ, X and Y are not independent.
(c) Law of Iterated Expectations (LIE)
We will first compute the expectations and then verify the LIE.

• Calculate E(Y )
The expected value of Y is calculated by summing over all possible values of Y , weighted by
their respective probabilities:

E(Y ) = ∑ y · fY (y) = 0 × 0.4 + 1 × 0.1 + 2 × 0.5 = 1.1

y∈{0,1,2}

• Calculate E[Y | X = x]
Next, we compute the conditional expectation of Y given each value of x:
– For x = 0:

1 2 4
E[Y | X = 0] = ∑ y · fY |X (y | 0) = 0 × +1×0+2× =
y∈{0,1,2}
3 3 3

– For x = 1:

3 1 3
E[Y | X = 1] = ∑ y · fY |X (y | 1) = 0 × +1× +2× = 1
y∈{0,1,2}
7 7 7

• Verify the LIE

Recall that P{X = x} represents the marginal pmf of X, so P{X = x} = fX (x).
Now, let’s verify that:
E(Y ) = ∑ E[Y | X = x] · fX (x)
x∈{0,1}

2
Judith Guo ECON 20110/30370

Substituting the values we’ve already calculated:

E(Y ) = ∑ E[Y | X = x] · fX (x) = E[Y | X = 0] · fX (0) + E[Y | X = 1] · fX (1)

x∈{0,1}
4
= × 0.3 + 1 × 0.7
3
= 0.4 + 0.7
= 1.1

Thus, we have verified that:

E(Y ) = 1.1,

which is consistent with our earlier calculation of E(Y ) via the definition of expectation.
(d) Conditional Expectation
Now, let’s derive the expression for E[Y |X].
From our earlier calculations, we have:

4 if x = 0
3
µ(x) := E[Y | X = x] =
1 if x = 1

We can express this using an indicator function:

4
µ(x) = · 1{x = 0} + 1 · 1{x = 1}.
3

Here 1{·} is the indicator function, which equals 1 if the condition inside the curly brackets is true
and 0 otherwise. For example, when x = 0, the indicator function 1{x = 0} = 1 and 1{x = 1} = 0,
so we get µ(0) = 34 × 1 + 1 × 0 = 43 .
Since x can only take the values 0 or 1, we can simplify the expression for µ(x) as:

4
µ(x) = · [1 − 1{x = 1}] + 1 · 1{x = 1}
3

which further simplifies to:

4 1
µ(x) = − · 1{x = 1}
3 3
Now, replacing x (a real number) with X (the random variable), we get:

4 1
µ(X) = E[Y | X] = − · 1{X = 1}
3 3

Remarks:
Let’s clarify the difference between functions of real numbers and functions of random variables:
• µ(x) is a function of a real number, x:

4 1
µ(x) = E[Y | X = x] = − · 1{x = 1}
3 3

3
Judith Guo ECON 20110/30370

Here, µ(x) represents the expected value of Y given that X takes a particular value, x. In this
case, x can only be 0 or 1. Since x is a known, fixed number, there’s no uncertainty involved.
You simply plug in the value of x to get a specific result:
– When x = 0, 1x = 1 = 0, so µ(0) = 43 .
– When x = 1, 1x = 1 = 1, so µ(1) = 1.
Since we’re working with fixed values of x, µ(x) is also a fixed number. There’s no ran-
domness here—it’s just like evaluating any mathematical function. For example, when you
compute f (x) = x2 , you get a concrete value for each x. In the same way, µ(x) is a deter-
ministic function, giving a specific outcome depending on the value of x.
• µ(X) is a function of a random variable, X:

4 1
µ(X) = E[Y | X] = − · 1{X = 1}
3 3

Now, when we write µ(X), X is no longer a fixed number; it’s a random variable. This means
X can take different values based on some underlying random process. In this example, X
can be either 0 or 1, but unlike before, we don’t know for sure which value it will take.
Here, X = 1 with probability 0.3, and X = 0 with probability 0.7. Because X is random,
µ(X) is also random—it depends on the outcome of X:
4
– If X = 0, then µ(X) = 3 with probability 0.7.
– If X = 1, then µ(X) = 1 with probability 0.3.
So, µ(X) is a random variable, since its value depends on the random outcome of X. The
randomness in µ(X) comes entirely from X—as X changes due to the random process, so
does the value of µ(X).
• Coin Flip Analogy:
Think of flipping a coin. Before it lands, you don’t know whether it will be heads (0) or
tails (1), and this uncertainty reflects the randomness in µ(X). Once the coin lands, and you
observe the outcome (say heads, X = 0), the uncertainty disappears. At that point, µ(0) is
just 34 —a fixed value. There’s no longer any randomness, just like after the coin flip, you
know the result.

Q2 (a) Covariance and Linearity of Expectation

We aim to show that:

cov(Z, aX + bY + c) = a cov(Z, X) + b cov(Z,Y ).

Proof:
By the definition of covariance,

cov(Z, aX + bY + c) = E{[Z − E(Z)][aX + bY + c − E(aX + bY + c)]}

4
Judith Guo ECON 20110/30370

Using the linearity of expectation, we simplify:

E(aX + bY + c) = aE(X) + bE(Y ) + c

Substituting back and rearranging, we get:

cov(Z, aX + bY + c) = E{[Z − E(Z)][aX + bY + c − (aE(X) + bE(Y ) + c)]}

 
= E{[Z − E(Z)] a[X − E(X)] + b[Y − E(Y )] + c − c }
 
= E{[Z − E(Z)] a[X − E(X)] + b[Y − E(Y )] }

Expanding the expectation:

cov(Z, aX + bY + c) = aE{[Z − E(Z)][X − E(X)]} + bE{[Z − E(Z)][Y − E(Y )]}

= a cov(Z, X) + b cov(Y, Z)

(b) Variance Formula

We need to show that:

var(aX + bY + c) = a2 var(X) + 2ab cov(X,Y ) + b2 var(Y )

Proof:
Note that the variance of a random variable is the covariance of the variable with itself:

var(W ) = cov(W,W ).

Let Z = aX + bY + c. Then,

var(aX + bY + c) = var(Z) = cov(Z, Z).

Using the result from part (a), we have:

cov(Z, Z) = cov (Z, aX + bY + c) = a cov(Z, X) + b cov(Y, Z).

Now, compute cov(Z, X) and cov(Z,Y ) again using the result from part (a)

cov(Z, X) = cov(aX + bY + c, X) = a cov(X, X) + b cov(Y, X) = a var(X) + b cov(Y, X),

and similarly
cov(Z,Y ) = cov(aX + bY + c,Y ) = a cov(X,Y ) + b var(Y ).

5
Judith Guo ECON 20110/30370

Substituting back into var(Z):

var(aX + bY + c) = a[a var(X) + b cov(Y, X)] + b[a cov(X,Y ) + b var(Y )]

= a2 var(X) + 2ab cov(X,Y ) + b2 var(Y ).

Remark:
These two calculation rules

cov(Z, aX + bY + c) = a cov(Z, X) + b cov(Z,Y )

and
var(aX + bY + c) = a2 var(X) + 2ab cov(X,Y ) + b2 var(Y )

are frequently used to simplify computations and deepen our understanding of statistical relationships
in Econometrics.

Q3 Best Constant Predictor

To show that E(X) is the best constant predictor for X, we need to find the value of c that minimises
the mean squared error:
L(c) := E[(X − c)2 ]

Let’s start by expanding the squared term:

L(c) = E[(X − c)2 ] = E X 2 − 2cX + c2 = E(X 2 ) − 2cE(X) + c2

where we’ve used the linearity of expectation to separate the terms.

To find the value of c that minimises L(c), we take the derivative of L(c) with respect to c and set it to
zero (the first-order condition):

dL
= −2E(X) + 2c = 0 =⇒ c∗ = E(X)
dc

So, the optimal constant predictor is c∗ = E(X).

To confirm that this is indeed a minimum, we examine the second derivative:

d2L
= 2,
dc2

which is positive. This means the function L(c) is convex, and therefore c∗ = E(X) is the global
minimum.

Q4 (a) Conducting a Hypothesis Test

We have two independent groups:
• Treatment Group:

6
Judith Guo ECON 20110/30370

– Number of participants: nT = 100

– Number improved: 75
– Let Ti be a binary random variable indicating whether the i-th participant improved:

1 if improved
Ti =
0 if not improved

Note that Ti is essentially a Bernoulli random variable with success probability pT (here,
improvement). Using the properties of a Bernoulli random variable, we have E(Ti ) = pT
and var(Ti ) = pT (1 − pT ).
– Sample proportion of improvement:
nT
1 75
pbT = T = ∑ Ti = 100 = 0.75.
nT i=1

• Control Group:
– Number of participants: nC = 100
– Number improved: 65
– Let Ci be a binary random variable indicating whether the i-th participant improved:

1 if improved
Ci =
0 if not improved

Similarly, this is also a Bernoulli random variable with success probability pC . There-
fore, we have E(Ci ) = pC and var(Ci ) = pC (1 − pC ).
– Sample proportion of improvement:

1 nC 65
pbC = C = ∑ Ci = = 0.65.
nC i=1 100

Formulating the Hypotheses:

We want to test whether the new dietary supplement is effective. The hypotheses are:

H : p = p ⇐⇒ p − p = 0 (the supplement has no effect)
0 T C T C
H : p > p ⇐⇒ p − p > 0 (the supplement is beneficial)
1 T C T C

This is a one-sided test to determine if the treatment group has a higher improvement rate than
the control group.
Calculating the Test Statistic:
We use the test statistic:
pbT − pbC − 0
t= ,
se ( pbT − pbC )

7
Judith Guo ECON 20110/30370

where se ( pbT − pbC ) is the standard error of the difference between the sample proportions. Since
we have independent samples, the standard error is calculated as:
s
pbT (1 − pbT ) pbC (1 − pbC )
se ( pbT − pbC ) = +
nT nC
 1/2
0.75(1 − 0.75) 0.65(1 − 0.65)
= +
100 100
≈ 0.0644.

(See the appendix on page 11 for a brief illustration of this formula.)

Now, we calculate the test statistic:

0.75 − 0.65
t= ≈ 1.55.
0.0644

Determining Critical Values:

Under the null hypothesis, and with large sample sizes, the test statistic approximately follows
a standard normal distribution by the Central Limit Theorem (CLT). So we can use the standard
normal distribution to find the critical values corresponding to our significance levels.
Let Z ∼ N(0, 1) denote a standard normal random variable. Then, for a one-sided test:
• At the 10% significance level (α = 0.10), the critical value c0.10 satisfies P(Z > c0.10 ) = 0.10.
Therefore, c0.10 = 1.28.
• At the 1% significance level (α = 0.01), the critical value c0.01 satisfies P(Z > c0.01 ) = 0.01.
Therefore, c0.01 = 2.33.
Making the Decision:
Our calculated test statistic is t ≈ 1.55.
• At the 10% level: t = 1.55 > 1.28, so we reject the null hypothesis.
• At the 1% level: t = 1.55 < 2.33, so we fail to reject the null hypothesis.
Conclusion:
There is sufficient evidence at the 10% significance level to conclude that the new dietary supple-
ment is beneficial. However, the evidence is not strong enough to be considered significant at the
1% level.
(b) Calculating the p-value
The p-value represents the probability of observing a test statistic at least as extreme as the one
calculated, assuming the null hypothesis is true.
In our case, the calculated test statistic is t ≈ 1.55. Under the null hypothesis, and with large
a
sample sizes, t approximately follows a standard normal distribution, denoted as t ∼ N(0, 1).
Therefore, the p-value is:

p = P {Z ≥ t} = 1 − Φ(t) = 1 − Φ(1.55) = 1 − 0.9394 ≈ 0.06.

8
Judith Guo ECON 20110/30370

where Φ(1.55) is the cumulative distribution function of the standard normal distribution evalu-
ated at 1.55.
Interpretation and Conclusion
The p-value of approximately 0.06 indicates that there is a 6% chance of observing a test statistic
as large as, or larger than, 1.55, assuming the null hypothesis is true.
In hypothesis testing, we compare the p-value to the chosen significance level α:
• If the p-value is less than α, it means the observed result is sufficiently unlikely under the
null hypothesis. Therefore, we have enough evidence to reject the null hypothesis.
• If the p-value is greater than α, the observed result is not unlikely enough to warrant rejec-
tion. Thus, we fail to reject the null hypothesis.
Therefore, the p-value supports our earlier conclusion: there is evidence to suggest that the sup-
plement is beneficial at the 10% level, but this evidence is not strong enough to be considered
significant at the 1% level.
(c) Type I and Type II Errors
In hypothesis testing, two types of errors can occur:
• Type I error:
– This occurs when we incorrectly reject a true null hypothesis.
– In the context of our clinical trial, a Type I error would mean concluding that the dietary
supplement is beneficial when, in reality, it has no effect.
– The consequences of a Type I error might include:
* Allocating resources to produce and distribute an ineffective supplement.
* Exposing participants to potential side effects without real benefits.
* Neglecting other treatments that might be more effective.
• Type II error:
– This happens when we fail to reject a false null hypothesis.
– In our study, a Type II error would mean concluding that the supplement is ineffective
when it actually has a beneficial effect.
– The consequences of a Type II error might include:
* Missing the opportunity to provide an effective treatment to patients.
* Patients continuing to suffer without access to a helpful supplement.
* Delaying further research and development of a promising intervention.
Which Error is More Critical to Avoid?
Deciding which type of error is more critical depends on the context and potential consequences:

• If the supplement is safe, inexpensive, and has the potential to significantly improve pa-
tients’ health, then avoiding a Type II error becomes more important. Failing to recognise an
effective treatment could deny patients the benefits and lead to continued suffering.

9
Judith Guo ECON 20110/30370

• Conversely, if the supplement carries risks, is costly, or could divert resources from more
effective treatments, then avoiding a Type I error is crucial. Incorrectly concluding that
the supplement is beneficial when it is not could result in unnecessary harm and wasted
resources.
There is no definitive answer, as the priority depends on the specific circumstances. However, if
patient safety is considered the top priority, there tends to be a greater emphasis on avoiding Type
I errors. This cautious approach ensures that only treatments with strong evidence of effectiveness
and safety are approved.
(d) Effect of Increased Sample Size
With the larger sample sizes, we recalculate the standard error (with improvement rates remaining
the same):
 1/2
0.75(1 − 0.75) 0.65(1 − 0.65)
se ( pbT − pbC ) = + ≈ 0.0372
300 300
and hence
0.75 − 0.65
t= ≈ 2.688.
0.0372
At the 1% significance level, the critical value for a one-sided test is 2.33. Since t ≈ 2.688 > 2.33,
we reject the null hypothesis even at the 1% significance level. So, with the larger sample size,
we have stronger statistical evidence to conclude that the dietary supplement is beneficial.
This is not surprising:
• Increasing the sample size reduces the standard error, making our estimate of the difference
in proportions more precise.
• A smaller standard error leads to a larger test statistic if the observed difference remains the
same.
• We gain more power to discriminate between pT − pC being zero, and small but nonzero.
Therefore, with more participants, even modest differences between groups become statistically
significant. This highlights the importance of sample size in hypothesis testing: larger samples
provide more reliable and conclusive results.

10
Judith Guo ECON 20110/30370

Appendix: Derivation of the Standard Error Formula in Question 4(a)

Recall that the standard error is an estimate of the standard deviation of a statistic. For example, for the
sample mean of a random variable W , W = n1 ∑ni=1 Wi , its standard error is given by:
q
se(W ) = sd(W
b )= c )
var(W

where sd
b denotes the estimated standard deviation, and var
c denotes the estimated variance.
In the context of Question 4(a), we need to compute the standard error of the difference between the
sample proportions pbT and pbC . This standard error is:
q
se ( pbT − pbC ) = c ( pbT − pbC )
var

c ( pbT − pbC ), we proceed step by step:

To find var

1. Variance of the Difference:

Since the treatment and control groups are independent, the covariance between pbT and pbC is zero.
Therefore, the variance of the difference is the sum of the variances:

var ( pbT − pbC ) = var ( pbT ) + var ( pbC )

This follows from the property that if two random variables Z1 and Z2 are independent, then var(Z1 −
Z2 ) = var(Z1 ) + var(Z2 ). Recall from Q2-(b) above, for any two random variables Z1 and Z2 , we have

if Z1 and Z2 independent 2
var(aZ1 ± bZ2 ) = a2 var(Z1 ) + b2 (Z2 ) ± 2abcov(Z1 , Z2 ) = a var(Z1 ) + b2 (Z2 ).

2. Sample Proportions as Sample Means:

The sample proportions pbT and pbC are sample means of Bernoulli random variables:

1 nT
1 nC
pbT = T = ∑ Ti , pbC = C = ∑ Ci
nT i=1 nC i=1

where each Ti and Ci is a binary indicator (1 for improvement, 0 for no improvement).

Then,


var(Ti ) var(Ci )
var ( pbT − pbC ) = var ( pbT ) + var ( pbC ) = var T + var C =(1) +
nT nC
where =(1) follows from that, given i.i.d. data and the rules in Q2-(b), we have, for instance

nT nT nT

1 1 1
var T = var( ∑ Ti ) = 2
var( ∑ Ti ) =by independence 2 ∑ var(Ti )
nT i=1 nT i=1 nT i=1
1
=by identical distribution nT var(Ti )
n2T
1
= var(Ti ).
nT

11
Judith Guo ECON 20110/30370

3. Variance of Bernoulli Variables:

For a Bernoulli random variable Ti , the variance is var(Ti ) = pT (1 − pT ), and similarly for Ci . Then

var(Ti ) var(Ci ) pT (1 − pT ) pC (1 − pC )
var ( pbT − pbC ) = + = + .
nT nC nT nC

4. Estimating the Variance:

Since we don’t know pT and pC , we estimate them using the sample proportions:

pbT (1 − pbT ) pbC (1 − pbC )

c ( pbT − pbC ) =
var + ,
nT nC

giving s
pbT (1 − pbT ) pbC (1 − pbC )
q
se ( pbT − pbC ) = c ( pbT − pbC ) =
var + .
nT nC

Summary:

• The standard error of the difference between two independent sample proportions is calculated by
summing the variances of each sample proportion and taking the square root.

• Each sample proportion’s variance is estimated using the formula for the variance of a Bernoulli distri-
bution divided by the sample size.

12