Random Variable — Notes

(English)

1. Random Variable

If the values taken by a variable are associated with

probabilities, then that variable is called a Random
Variable.

Example: Suppose a coin is tossed twice. The sample

space is (H,H), (H,T), (T,H), (T,T) — 4 outcomes, each
with probability 1/4. Let X = number of heads
obtained. X can take values 0, 1, 2.

P(X=0) = P(T,T) = 1/4

P(X=1) = P(H,T) or (T,H) = 1/4 + 1/4 = 1/2

P(X=2) = P(H,H) = 1/4

A Random Variable is a real-valued function defined

on a sample space.

Discrete Random Variable

If a Random Variable is defined on a discrete sample

space, it is called a Discrete Random Variable (D.R.V.).
If the variable X takes values x₁, x₂, ... xₙ with
corresponding probabilities p₁, p₂, ... pₙ, then the
variable is called a Discrete R.V.

Example: X = sum of the faces of two dice when two

dice are thrown. X can take values 2, 3, ..., 12 with
respective probabilities 1/36, 2/36, ..., 1/36. This is a
D.R.V.

Continuous Random Variable

If a variable X takes all possible values within an

interval [a, b] along with their respective probabilities,
then it becomes a Continuous Random Variable
(C.R.V.).

Properties of Random Variables

1. A R.V. is denoted by a capital letter.

2. If X is a R.V., then Xʳ is also a R.V.

3. If X is a R.V., then 1/X is also a R.V., provided X ≠ 0.

4. If X is a R.V. and a, b are any two constants, then

aX + b is also a R.V.

5. aX ± bY is also a R.V.

6. If X₁, X₂, ... Xₙ are n R.V.'s, then Uₙ = max(X₁, X₂, ...

Xₙ) is also a R.V., and Vₙ = min(X₁, X₂, ... Xₙ) is also
a R.V.
2. Probability Distribution

1. Something that shows the values taken by a R.V.

and their associated probabilities is called a
Probability Distribution.

2. The total probability of a probability distribution is

equal to 1.

3. With the help of a probability distribution, it is easy

to find the probability of any event of an
experiment associated with it.

4. Every R.V. has a probability distribution. Since R.V.'s

are of two types, probability distributions are also
of two types: Discrete P.D. (related to discrete
variables) and Continuous P.D. (related to
continuous variables).

Probability Mass Function (PMF)

Used exclusively for a Discrete R.V. A D.R.V. takes on

distinct, isolated values.

If X is a D.R.V., its PMF P(x) is defined as P(x) = P(X =

x), with:

1. P(xᵢ) ≥ 0 — probability of any outcome cannot be

negative

2. Σ P(xᵢ) = 1 — the sum of probabilities of all

possible outcomes must equal 1

Probability Density Function (Continuous case)

For a continuous R.V. X, if a and b are two numbers (a
≤ X ≤ b), the function f(x) that gives the probability of X
lying between these numbers is called the probability
density function.

P(a ≤ X ≤ b) = ∫ₐᵇ f(x) dx

Conditions:

1. f(x) ≥ 0

2. ∫ f(x) dx = 1 (over the entire range)

Note: The probability of any single particular

point/value in a C.R.V. is 0. A C.R.V. cannot have
distinct/isolated values (unlike a discrete R.V.) — it
takes a continuous range of values.

3. Cumulative Distribution Function

(CDF)

For any R.V. X, the distribution function is denoted by

F(x):
F(x) = P(X ≤ x)

F(x) is defined for x ranging from −∞ to +∞. The value

of F(x) always lies between 0 and 1. That is, for x ∈
(−∞, ∞), F(x) ∈ [0, 1].

Properties of the Distribution Function

For any a < b:

1. P(a < X ≤ b) = F(b) − F(a)

2. P(a ≤ X ≤ b) = F(b) − F(a) + P(X=a)

3. P(a < X < b) = F(b) − F(a) − P(X=b)

4. P(a ≤ X < b) = F(b) − F(a) − P(X=b) + P(X=a)

5. 0 ≤ F(x) ≤ 1

6. F(x) ≤ F(y) if x < y

7. F(−∞) = 0, F(∞) = 1

Discrete CDF

If X is a discrete R.V. taking values x₁, x₂, x₃..., with

respective probabilities pᵢ ≥ 0, Σpᵢ = 1, then the
distribution function is:

F(x) = Σ pᵢ for all i such that xᵢ ≤ x

At any point x other than the values taken by the

variable, the value of the CDF is 0 (no jump/no
additional probability contributed).

Properties of Discrete CDF

(i) The discrete distribution function increases in

steps.
(ii) At each point there is a jump.
(iii) It stays flat (constant) between two points.
(iv) P(x) = P(X = xⱼ) = F(x) − F(xⱼ₋₁), i.e., the probability
of any value equals the size of the jump in the CDF at
that value.
(v) The distribution function is non-decreasing.
(vi) For a discrete distribution, F(x) is right-continuous.
(vii) F(−∞) = 0, F(∞) = 1
(Note: the discrete distribution/CDF graph forms an
"increasing staircase" model.)

Continuous Distribution Function

For a continuous R.V. X, the distribution function is:

F(x) = P(X ≤ x) = ∫₋∞ˣ f(x) dx, −∞ < x < ∞

Its derivative gives the probability density function.

Properties of Continuous Distribution Function

1. 0 ≤ F(x) ≤ 1, −∞ < x < ∞

2. The continuous distribution function is a non-

decreasing function.

3. F(−∞) = 0, F(∞) = 1

4. F'(x) = f(x) ≥ 0 — i.e., the derivative of the CDF

equals the PDF

5. F(x) is continuous everywhere.

6. F(x) has no discontinuities.

7. For a continuous distribution, the probability at any

single specific point equals 0; it doesn't matter
whether the boundary is included or excluded.

8. P(a < X < b) = P(a < X ≤ b) = P(a ≤ X < b) = ∫ₐᵇ f(x) dx

4. Measures Related to a Continuous
Distribution

If f(x) is the probability density function of variable X,

defined between limits a and b:

(i) Mean = ∫ₐᵇ x f(x) dx

(ii) Harmonic Mean = ∫ₐᵇ (1/x) f(x) dx
(iii) Geometric Mean = ∫ₐᵇ log(x) f(x) dx
(iv) μᵣ' (moment about origin) = ∫ₐᵇ xʳ f(x) dx
(v) μᵣ (moment about x = A) = ∫ₐᵇ (x−A)ʳ f(x) dx
(vi) μᵣ (moment about mean) = ∫ₐᵇ (x−mean)ʳ f(x) dx

Moments about origin:

μ₁' = mean = ∫ₐᵇ x f(x) dx

μ₂' = ∫ₐᵇ xʳ f(x) dx

Variance:
μ₂ = μ₂' − (μ₁')² = ∫ₐᵇ x² f(x) dx − [∫ₐᵇ x f(x) dx]²

μ₃ = μ₃' − 3μ₂'μ₁' + 2μ₁'³

μ₄ = μ₄' − 4μ₃'μ₁' + 6μ₂'μ₁'² − 3μ₁'⁴

(vii) Median: ∫₋∞ᵐ f(x) dx = ∫ₘ^∞ f(x) dx = 1/2

(viii) Mean Deviation:

M.D. = ∫ₐᵇ |x − mean| f(x) dx
M.D. about A = ∫ₐᵇ |x − A| f(x) dx

(ix) Quartiles: ∫ₐ^Q1 f(x) dx = 1/4, ∫ₐ^Q3 f(x) dx = 3/4,

∫ₐ^Di f(x) dx = i/10 (i = 1 to 9)
(x) Mode: the value of X where f(x) has a maximum,
i.e., f'(x) = 0 and f''(x) < 0, within the given range [a, b].

Note: The distribution function should always be

denoted by F, while the density function should be
denoted with lowercase f.

5. Mathematical Expectation

For a Discrete Random Variable

If X is a discrete R.V. with values x₁, x₂, ... xₙ having

probabilities P(x₁), P(x₂), ... P(xₙ), then the
mean/expectation is:

E(X) = Σ xᵢ P(xᵢ) = x₁P(x₁) + x₂P(x₂) + ... + xₙP(xₙ)

For a Continuous Random Variable

If X is a continuous R.V. with probability density

function f(x), then the mean is defined as:

E(X) = ∫₋∞^∞ x f(x) dx

Properties of Mathematical Expectation

1. E(constant) = constant

2. E(aX + b) = aE(X) + b

3. E(X + Y) = E(X) + E(Y)

4. If X, Y are independent, E(XY) = E(X)·E(Y)

Also:

E(X) = mean

E[(X−μ)²] = Variance

V(X) = E(X²) − [E(X)]²

6. Bernoulli Distribution

A trial or experiment with only two possible

outcomes is called a Bernoulli trial.

Examples: coin toss → Head/Tail; product testing

→ defective/non-defective; exam → pass/fail.

If a discrete R.V. X taking values 0 or 1 has probability

of "success" p, then the probability mass function is:

P(X=x) = pˣ(1−p)^(1−x) for x = 0, 1; 0 otherwise

where 0 ≤ p ≤ 1.

At x=0: P(X=0) = 1−p = q

At x=1: P(X=1) = p

Also, p + q = 1.

Mean = E(X) = Σx P(x) = 0(q) + 1(p) = p

E(X²) = 0(q) + 1(p) = p
Variance V(X) = E(X²) − [E(X)]² = p − p² = p(1−p) = pq
SD(X) = √(pq)

MGF: Mₓ(t) = q + peᵗ

Mean > Variance, since (p > pq when q < 1).

If X ~ B(p), then ΣXᵢ ~ Binomial(n, p).

7. Binomial Distribution

Binomial distribution is also called the "distribution of

trials." It is used when a fixed number of independent
trials, each with two possible outcomes, is repeated.

Assumptions / Conditions for a Binomial

Experiment

1. The total number of trials (n) should be fixed in

advance.

2. Each trial results in only one of two possible

outcomes — termed "success" and "failure."

3. All trials should be independent — the outcome of

one trial should not influence another. The
probability of success should remain constant
throughout.

4. The probability of "success" in each trial remains

fixed. The probability of failure (q) also does not
change.

For 'n' independent trials, the number of successes 'x'

follows nCₓ combinations, with the probability of any
particular arrangement being pˣ q^(n−x), so the total
probability for x successes is: nCₓ pˣ q^(n−x)
If X is a random variable following a Binomial
distribution, then its PMF is:

P(X=x) = nCₓ pˣ q^(n−x), x = 0,1,2,...,n; q = 1−p

n = number of trials

p = probability of success

q = probability of failure

n, p are the two parameters of the Binomial

distribution.

Measures of Binomial Distribution

Mean = np

Variance = npq

SD = √(npq)

μ₁' = np, μ₂ = npq, μ₃ = npq(q−p), μ₄ = 3n²p²q² +

npq(1−6pq)

β₁ = (q−p)²/(npq), β₂ = 3 + (1−6pq)/(npq)
γ₁ = √β₁ = (q−p)/√(npq), γ₂ = β₂ − 3 = (1−6pq)/(npq)
β₁ = (1−2p)²/(npq)

The Binomial distribution can have either one or two

modes.

Mode = [(n+1)p] when (n+1)p is not an integer.

If (n+1)p is an integer, say m, then there are two

modes: m, m−1.
 If p = q = 1/2: mode = n/2 (single mode if n is even;
two modes (n±1)/2 if n is odd).

p = q = 1/2 → distribution is symmetric.

p > 1/2 → distribution is negatively skewed

(skewed towards left).

p < 1/2 → distribution is positively skewed.

β₂ > 3 → the Binomial distribution is leptokurtic.

As n → ∞, β₂ → 3, the Binomial distribution

approaches mesokurtic (normal-like) shape.

Properties of Binomial Distribution

1. When n=1, the Binomial distribution reduces to

Bernoulli distribution.

2. The sum of two independent Binomial variables

(with same p) is also Binomial.

3. Mean > Variance.

4. V(X) is maximum when p = q = 0.5, giving max

variance = n/4.

5. If X₁ ~ B(n₁,p), X₂ ~ B(n₂,p), then X₁+X₂ ~ B(n₁+n₂,

p).

6. If n → ∞, p → 0, np = finite (constant), then

Binomial distribution tends to Poisson distribution.

7. If n → ∞, Binomial → Normal distribution.

Examples: Number of defective bulbs in a box of n
bulbs; number of postgraduates in a group of n men.

8. Poisson Distribution

Poisson distribution arises as a limiting case of

Binomial distribution when the number of trials
becomes infinitely large, p becomes very small, and np
remains a finite constant. It is also called a "rare
events" distribution.

Poisson distribution has only one parameter, denoted

λ (lambda), also written as m.

This distribution was first introduced by the French

mathematician Simeon Denis Poisson in 1837.

When n is very large, p is very small, and np = λ

(constant/finite).

If X is a discrete R.V. following a Poisson distribution,

its PMF is:

P(X=x) = (e^(−λ) λˣ)/x!, x = 0,1,2,...,∞; parameter λ

Mean = λ

Variance = λ

SD = √λ

μ₃ = λ, μ₄ = 3λ² + λ

β₁ = 1/λ, γ₁ = √(1/λ), β₂ = 3 + 1/λ, γ₂ = 1/λ

Poisson distribution is always positively skewed and
leptokurtic.

Poisson distribution's mode: if 'λ' is not an integer,

mode = [λ] (integer part). If 'λ' is an integer, there are
two modes: λ, λ−1.

Properties of Poisson Distribution

1. Poisson distribution has only one parameter.

2. Mean = Variance = λ

3. If X ~ P(λ₁), Y ~ P(λ₂), then X+Y ~ P(λ₁+λ₂).

4. Poisson distribution is always a positively skewed

distribution.

5. As λ → ∞, Poisson distribution → Normal

distribution.

e = 2.7183 = 1 + 1/1! + 1/2! + ...

Examples:

Number of accidents on a highway in one day

Number of telephone calls received in an hour

Number of printing mistakes per page

Number of misprints

9. Normal Distribution
 The Normal distribution was first discovered in
1733 by the English mathematician De Moivre;
later this Normal distribution was expanded in the
context of the Binomial distribution.

Independently, before 1774, unaware of the

Continental (De Moivre) work, English scholars
named it "Gaussian distribution." In the early 19th
century (1809), Gauss applied it to the errors of
astronomical observations. In studying errors
made in astronomical observations, Gauss
developed the theory of the normal distribution of
errors, and this became the origin of the term
"Gaussian distribution."

In the 18th and 19th centuries, mathematicians

studying various random variables found that
most naturally-occurring samples followed a
common, similar distribution pattern. Hence it
came to be called "Normal." So, to determine the
reliability of these observations, statisticians took
the normal distribution as the model reference for
comparing sets of observations.

Definition

If X is a continuous R.V. following a normal

distribution, its probability density function is:

f(x) = (1/(σ√(2π))) e^(−1/2 · ((x−μ)/σ)²), −∞ < x < ∞

The parameters of this distribution are μ and σ. Here μ,
σ represent the mean of the distribution and standard
deviation, respectively.

The mean value μ, standard deviation σ and having a

normal random variable — the collective notation is X
~ N(μ,σ²). When mean = 0, standard deviation = 1, this
normal variable 'x' is called the standard normal
variable. Its distribution is called Standard Normal
Distribution. This is generally denoted by Z.

Properties of Normal Distribution

1. Normal distribution is a bell-shaped distribution; its

mean, median, and mode are all equal.

2. Its first quartile Q₁ and third quartile Q₃ are

equidistant from the median.

3. This is a unimodal distribution.

4. This distribution is symmetric about a vertical line

(the mean).

5. For this distribution, mean deviation ≈ (4/5) ×

standard deviation, and quartile deviation ≈ (2/3) ×
standard deviation, approximately.

6. Any linear combination of normal random

variables is again a normal random variable.

Area/Probability Properties

The curve depicting the frequency of normal

probabilities is called the normal probability curve.
This has certain key characteristics:

1. The normal probability curve is bell-shaped. It is

symmetric about the mean.

2. The normal probability curve extends from −∞ to

+∞ along the X-axis, being asymptotic to it.

3. Points of inflection of the curve occur at μ ± σ.

4. Total area under this curve is 1.

5. Total area under the curve from −∞ to μ is 0.5, and

from μ to +∞ is also 0.5.

6. Area within μ − σ to μ + σ = 0.6826

7. Area within μ − 2σ to μ + 2σ = 0.9544

8. Area within μ − 3σ to μ + 3σ = 0.9973

9. Quartiles of the normal distribution: μ ± 0.6745σ

(i.e., Q₁ = μ − 0.6745σ, Q₃ = μ + 0.6745σ)

10. Statistical Inference

Statistical Inference is the process of drawing

conclusions about the parameters and other
characteristics of a population, using data from a
sample, based on probability theory.

Statistical methods are broadly divided into two

categories:

Descriptive Statistics — organizing, summarizing, and

presenting data (collecting data, tabulating, drawing
charts, etc.)

Inferential Statistics — using sample data to draw

conclusions about population parameters.

Why Sampling Is Needed for Inference

Studying the entire population directly requires

more money, time, and effort.

Sometimes the population is infinite or

inaccessible (e.g., destructive testing of bulbs).

Hence, a sample of manageable size is drawn

from the population, and the sample data is used
to draw inferences accurately about the
population.

Population: The complete set of items under study.

Sample: A finite subset selected from the population.
Parameter: The fixed, known/unknown numerical
value that characterizes the population (e.g., μ, σ², p).

Statistic: A quantity computed from sample values

(e.g., x̄, s², p̂ — sample mean, sample variance, sample
proportion).

Sampling Distribution: The distribution formed by the

values of a statistic computed from all possible
samples.

Standard Error: The standard deviation of the

sampling distribution.
Two Main Types of Problems in Statistical
Inference

1. Estimation: Estimating population parameter values

using appropriate methods based on sample data.

Point Estimation: Estimating using a single

specific value (e.g., x̄ = 70)

Interval Estimation: Giving a range of values at a

specified confidence level.

2. Hypothesis Testing: Testing whether a

claim/statement made about the population is
consistent with the sample data.

Null Hypothesis (H₀)

Alternative Hypothesis (H₁)

Steps in Statistical Inference

1. Define the population and clearly state the

objective of the study.

2. Take a random sample from the population.

3. Collect the necessary data from the sample.

4. Draw conclusions/estimate the parameters based

on the collected data.

5. Report the findings and interpret the results.

11. Point Estimation

Point estimation is the process of using sample data

to compute a single specific numerical value as the
estimate for an unknown value of a population
parameter.

Example: To know the average income (μ) of people in

a city, it is not feasible to collect data from everyone,
so a suitable sample is taken; using its data, a single
specific value is given as the estimate for the
population income mean. This is called point
estimation.

1. Estimator

The rule or formula used to estimate the value of a

population parameter using sample data is called an
Estimator.

Example: To estimate the population mean (μ), we use

x̄ = Σx/n (the sample mean formula) — this formula
itself is called the Estimator.

2. Estimate

The specific numerical value obtained by applying the

estimator to a particular given sample is called the
Estimate.
Example: For a sample with values 40, 50, 60:
x̄ = (40+50+60)/3 = 50

Estimator = x̄ (sample mean formula)

Estimate = 50 (the actual computed value)

12. Criteria for a Good Estimator

(1) Unbiasedness

If the expected value of an estimator exactly equals

the value of the population parameter, that estimator is
called an Unbiased Estimator.

In symbols: E(θ̂) = θ

where θ̂ = estimator, θ = population parameter, E(θ̂) =

expected value of the estimator.

Definition: If the expected value of an estimator equals

the true value of the population parameter, that
estimator is called an unbiased estimator.
→ In general, if the mean of the sampling distribution
of an estimator equals the parameter, it is called
unbiased.

E(statistic) = parameter

Bias = E(θ̂) − θ
 If bias is positive, i.e., E(θ̂) > θ, then the estimator
tends to overestimate on average.

If bias is negative, i.e., E(θ̂) < θ, then the estimator

tends to underestimate on average.

If bias = 0, the estimator is unbiased: E(θ̂) = θ.

An estimator built using the sample mean is

always unbiased for the population mean.

Examples:

1. If population X ~ N(μ,σ²), then to estimate

population mean μ, the sample mean (x̄) is used.
E(x̄) = μ. Hence, sample mean is an unbiased
estimator of population mean.

2. X ~ B(n,p), parameter — p: p̂ = X/n; E(p̂) = p → p̂ =

X/n is an unbiased estimator of p.

3. X ~ P(λ), parameter — λ: E(x̄) = λ → x̄ is an

unbiased estimator of λ.

(2) Consistency

Given sample size increasing without bound, if an

estimator's values converge (approach) increasingly
close to the true population parameter value, that
estimator is called a consistent estimator.

In symbols: θ̂ →ᵖ θ as n → ∞

where θ̂ = estimator, θ = population parameter. As n →

∞, the estimator converges to the true parameter
value.

Sufficient conditions for consistency:

E(Tₙ) → θ
V(Tₙ) → 0

Examples:

1. X ~ N(μ,σ²): Sample mean x̄ is a consistent

estimator of population mean μ.

2. X ~ B(n,p): p̂ = X/n is a consistent estimator of p.

3. X ~ P(λ): λ̂ = x̄ is a consistent estimator of λ.

(3) Efficiency

For a given parameter, there can be several unbiased,

consistent estimators. For example, from N(μ,σ²), if a
sample of size n is drawn, both the sample mean x̄
and sample median x̃ are unbiased, consistent
estimators of μ.

Among all such estimators, we need a criterion to

determine which is the best; the sample size and cost
involved should be minimized while achieving the best
inference using these estimators. This criterion is
called efficiency.

Definition: Given two unbiased estimators θ̂₁, θ̂₂, if

Var(θ̂₁) < Var(θ̂₂), then θ̂₁ is called the more efficient
estimator.
Relative Efficiency

Relative Efficiency = Variance of second estimator /

Variance of first estimator

If this efficiency value is greater than 1, the first-

mentioned estimator is called more efficient.

Example: X ~ N(μ,σ²); population mean estimation:

two unbiased estimators are x̄ and x̃ (median):

Var(x̄) = σ²/n
Var(x̃) = πσ²/2n

Relative Efficiency = Var(x̄)/Var(x̃) = (σ²/n)/(πσ²/2n) =

2/π = 0.637

So, the relative efficiency of median = 63.7%

Relative efficiency of sample mean = 100%

Since the mean has smaller variance than the median

for consistent estimators of the population mean, the
mean is said to be more efficient than the median.

Important:
Var(x̄) = σ²/n, Var(median) = πσ²/2n

→ Relative efficiency of the median compared to the

mean = 63.7% (π/2 = 1.57)
→ Relative efficiency of mean compared to median =
value: 100% is not correct here — the relative efficiency
figure is 63.7%.
(4) Sufficiency

If a statistic computed from the sample contains all

the information about the population parameter (uses
complete information), that statistic is called a
sufficient statistic. An estimator that provides
sufficient information about the parameter is called a
sufficient estimator.

Examples:

1. X ~ N(μ,σ²), σ² known: x̄ can be said to be

sufficient for μ, because it uses all the sample
information relevant for estimating μ, i.e., the
sample mean is efficient in this sense.

2. X ~ P(λ): Here, Σxᵢ is the sufficient statistic.

To determine whether an estimator uses all the

necessary information for estimating a given
parameter, we use certain theorems called
factorization theorems.

Factorization Theorem

Consider a random sample x₁, x₂, ... xₙ. A statistic T is

said to be sufficient for parameter θ if the joint
probability distribution can be factored into two parts:
one part depending only on the sample statistic T and
the parameter θ, and the other part not depending on
the parameter at all. This is generally called the
Neyman-Fisher factorization theorem.
13. Methods of Estimation

Estimation Methods generally used:

1. Method of Moments

2. Method of Minimum Variance

3. Method of Least Squares

4. Method of Least Chi-Square

5. Method of Minimum Probabilities (?)

(1) Method of Moments

To estimate the parameters of a distribution, the

moments of the distribution are equated to the
corresponding sample moments; this is the
fundamental idea of this method.

This method was introduced by Karl Pearson in

1894.

This method aims to estimate the population

parameters by equating the population moments
to the corresponding sample moments.

Example: X ~ N(μ,σ²)
E(X) = μ, E(X²) = μ² + σ²

Equating with corresponding sample moments:

μ̂ = x̄ = (1/n)Σxᵢ
μ̂² + σ̂² = (1/n)Σxᵢ²
σ̂² = (1/n)Σxᵢ² − x̄²

Advantages: (1) Easy to compute. (2) Applicable to a

wide range of distributions.

Disadvantages: (1) For some distributions, this

method may yield estimates outside the valid
parameter range.

(2) Method of Maximum Likelihood (MLE)

The method that chooses parameter values which

maximize the probability (likelihood) of obtaining the
observed sample is called MLE. In simple terms:
whatever parameter value 'θ' makes the observed
sample most probable, that value is taken as the
estimate.

Likelihood Function
Let x₁, x₂, ..., xₙ be a random sample drawn from a
population with density function f(x;θ). The likelihood
function is denoted L(θ):

L(θ) = f(x₁;θ)·f(x₂;θ)···f(xₙ;θ) = ∏ᵢ f(xᵢ;θ)

To simplify computation involving products, take the

log:
l(θ) = ln L(θ) = Σ ln f(xᵢ;θ)

To find the value of θ that maximizes l(θ), differentiate

with respect to θ and set to zero:
dl(θ)/dθ = 0

The solution of this equation gives the MLE.

To confirm this is indeed a maximum, check the

second derivative condition:
d²l(θ)/dθ² < 0

14. Interval Estimation

In many situations, a single point estimate for a

parameter is not sufficient. Because although the point
estimate might be close to the true parameter value in
certain samples, in some cases the estimate could
differ significantly from the true value due to sample
fluctuation, so instead we give a range/interval
estimate.

In interval estimation, the parameter estimate has two

limits/bounds. These are called confidence limits. In
these limits, the reliability level associated is called the
confidence level.

One-Sided Confidence Interval

Sometimes, we need only one bound of the confidence

interval (either upper limit U or lower limit L). Such
confidence intervals with only one bound are called
one-sided confidence intervals.
Examples:

1. To know the upper bound of a component's failure

rate: P(θ ≤ U) = 1−α

2. To know the lower bound of some quality-related

probability parameter: P(L ≤ θ) = 1−α

Two-Sided Confidence Interval

A two-sided confidence interval for parameter θ has

two bounds (lower limit L and upper limit U). And the
true parameter value lies between these two bounds:

P(L ≤ θ ≤ U) = 1 − α

Here, α is divided into two equal parts:

P(θ < L) = α/2, P(θ > U) = α/2

→ Length of the confidence interval = U − L

→ Confidence intervals are not unique.

Example:
X₁,X₂,...Xₙ ~ N(μ,σ²)

When σ² is known, the 100(1−α)% confidence interval

for μ:
[x̄ − z_(α/2)·σ/√n, x̄ + z_(α/2)·σ/√n]

When σ² is unknown, the 100(1−α)% confidence

interval for μ:
[x̄ − t_(n−1,α/2)·s/√n, x̄ + t_(n−1,α/2)·s/√n]
15. Hypothesis Testing — Basic
Concepts

Hypothesis: A statement/assumption made about the

population is called a hypothesis.

Statistical Hypothesis: A statement made about

population parameters is called a statistical
hypothesis. This is denoted by H.

Compound Hypothesis (Hypothesis Pair): A

hypothesis testing problem generally has two
contrasting statements. Based on sample values, the
process of deciding which of these two statements is
true/acceptable is called hypothesis testing. These
two statements together are called a "pair of
hypotheses."

Example scenarios:
(i) Introducing a new product to the market
(ii) Deciding whether to launch the new product

Suppose historically, in the market, only 10% of

products were previously successful, but the new
product's success rate is claimed to be 20%. Let 'p'
denote the proportion representing success. Based on
customer response to the new product, the statistical
hypotheses regarding the above claims can be stated
as:

(i) H₁: p = 0.2 (ii) H₂: p = 0.1

Null Hypothesis: The hypothesis that is initially
assumed to be true (about the population), used
mainly for testing purposes. This is denoted H₀.

Alternative Hypothesis: The hypothesis that we adopt

(accept) if the null hypothesis is rejected, is called the
alternative hypothesis (or its statistical hypothesis).
Denoted H₁.

Simple Hypothesis: A hypothesis that completely

specifies the population distribution is called a simple
hypothesis.
Example: H₁: X ~ B(100, 1/2), where both n, p are fully
specified.

Composite Hypothesis: A hypothesis where the

population distribution is not completely specified is
called a composite hypothesis.
Example: X ~ B(100,p), p ≥ 0.5

Test Statistic

A random sample x₁, x₂, ... xₙ is used to compute a

function of these values called the test statistic.
Depending on its value, we decide whether to reject or
accept the given hypothesis. The purpose of choosing
a good test statistic is to obtain a decision rule with
maximum efficiency for the given hypothesis testing
problem, based on the appropriate sampling
distribution. The set of all possible values a test
statistic can take is called the sample space of the
test statistic.

Critical Values: After computing the test statistic value

from a sample, comparing this against the hypothesis,
the boundary values used to decide rejection or
acceptance of the hypothesis are called critical values.
The set of critical (test statistic) values is called the
critical region, and other remaining values form the
acceptance region.

Critical Region and Acceptance Region

A given hypothesis is tested using a test statistic, and

to make a decision, the sample space of the test
statistic is divided into two mutually exclusive regions.
If the calculated statistic falls into a certain region, the
null hypothesis is rejected — this region is called the
Critical Region (Rejection Region), generally denoted
C. The other region is called the Acceptance Region
(AR), generally denoted C'. If the test statistic falls in
region C, we reject H₀; otherwise we accept (do not
reject) H₀.

Type I and Type II Errors

Whether we accept or reject H₀ based on the sample,

four possible outcomes/scenarios can occur
regarding the true state of the population:
 Population True Decision: Decision:
State Reject H₀ Accept H₀

H₀ is True Error (Type I) Correct

H₀ is False Correct Error (Type II)

Type I Error: Rejecting the null hypothesis (H₀) when it

is actually true is called a Type I error.

Type II Error: Accepting (failing to reject) the null

hypothesis when it is actually false is called a Type II
error.

Level of Significance

The probability of committing a Type I error is called

the level of significance, denoted α. This is also called
the size of the critical region.

Power of the Test: The probability of correctly

rejecting a false null hypothesis is called the power of
the test, denoted (1−β).

β = P(T ∈ C' | H₀ is false)