Statistics and Probability

Data Representation and Summarization

Data representation and summarization refers to the process of

organizing, presenting, and describing data in a concise and meaningful
form. It helps in understanding patterns, trends, and characteristics of
data through tables, graphs, and statistical measures.
Explanation
Raw data is often difficult to interpret directly. Statistical methods are
used to summarize data and present it in a form that is easier to analyze
and understand.
Data can be classified based on its nature and measurement scale, and it
can be represented using frequency distributions and graphical techniques.
Types of Data
1. Qualitative Data
Qualitative data (categorical data) describes characteristics or attributes
that cannot be measured numerically.
Examples:
• Gender
• Blood group
• Religion
• Eye color
Characteristics:
• Non-numerical
• Classified into categories
• Used for comparison and classification
2. Quantitative Data
Quantitative data consists of numerical values that can be measured or
counted.
Examples:
• Height
• Weight
• Age
• Income
Quantitative data is further classified into:
(a) Discrete Data
Data that can take only specific countable values.
Examples:
• Number of students in a class
• Number of cars in a parking lot
(b) Continuous Data
Data that can take any value within a given range.
Examples:
• Height
• Temperature
• Time
 Measurement Scales
Measurement scales determine the nature of data and the statistical
techniques that can be applied.
1. Nominal Scale
The nominal scale classifies data into categories without any order.
Examples:
• Gender
• Blood group
• Nationality
Characteristics:
• Labels only
• No ranking
• Arithmetic operations are not meaningful
2. Ordinal Scale
The ordinal scale classifies data into categories with a meaningful order.
Examples:
• Class ranks
• Satisfaction levels (Poor, Fair, Good, Excellent)
Characteristics:
• Ranking exists
• Differences between ranks are not measurable
3. Interval Scale
The interval scale has ordered categories with equal intervals between
values, but no true zero.
Examples:
• Temperature in Celsius
• Temperature in Fahrenheit
Characteristics:
• Equal intervals
• Addition and subtraction are meaningful
• Zero does not indicate absence of quantity
4. Ratio Scale
The ratio scale possesses all properties of the interval scale and includes a
true zero.
Examples:
• Height
• Weight
• Age
• Income
Characteristics:
• Equal intervals
• True zero exists
• All arithmetic operations are meaningful

Frequency Distribution

A frequency distribution is a table that shows how often each value or

class of values occurs in a dataset.
Example

Marks Frequency

0–10 3

10–20 5

20–30 7
Marks Frequency

30–40 4

40–50 1

Advantages
1. Organizes large datasets.
2. Reveals patterns and trends.
3. Facilitates graphical representation.
4. Simplifies statistical analysis.

Graphical Representation of Data

Graphs provide a visual representation of data and make interpretation

easier.
1. Histogram
A histogram is a graph consisting of adjacent rectangles representing
frequencies of continuous data.
Characteristics:
• Horizontal axis represents class intervals.
• Vertical axis represents frequencies.
• Bars touch each other because data is continuous.
Uses:
• Displays frequency distribution.
• Identifies shape of distribution.
• Detects skewness and outliers.
2. Frequency Polygon
A frequency polygon is a line graph obtained by joining the midpoints of
the tops of histogram bars.
Characteristics:
• Shows trends in data.
• Easier to compare multiple distributions.
• Can be drawn without a histogram.
Uses:
• Comparison of datasets.
• Visualizing distribution shape.

Stem and Leaf Plot

A stem and leaf plot is a graphical method of displaying quantitative data

while retaining the original data values.
The data values are split into:
• Stem: Leading digit(s)
• Leaf: Final digit
Example
Data:
12, 14, 15, 18, 21, 23, 25, 27

Stem-and-Leaf Plot:

Stem Leaf

1 2458

2 1357
Advantages
1. Preserves original data values.
2. Easy to construct.
3. Displays distribution shape.
4. Useful for small datasets.
Properties
1. Qualitative data is categorical, while quantitative data is numerical.
2. Nominal and ordinal scales are categorical scales.
3. Interval and ratio scales are numerical scales.
4. Frequency distributions summarize large datasets.
5. Histograms are used for continuous data.
6. Frequency polygons help compare distributions.
7. Stem-and-leaf plots preserve original observations.
Applications
1. Statistical data analysis
2. Business reporting
3. Market research
4. Scientific experiments
5. Educational assessment
6. Survey analysis
 Measures of Central Tendency

Measures of Central Tendency are statistical measures that describe the

central or typical value of a dataset. They provide a single value that
represents the entire dataset.
The three most commonly used measures of central tendency are:
1. Mean
2. Median
3. Mode
Explanation
A measure of central tendency identifies the center of a distribution and
provides a summary of the data.
These measures help answer the question:
"What is the typical or representative value of the dataset?"
Different measures may be appropriate depending on the nature of the
data.
1. Mean
The Arithmetic Mean is the sum of all observations divided by the total
number of observations.
Formula
∑𝑥
𝑥ˉ =
𝑛
where:
• 𝑥ˉ= Mean
• ∑𝑥= Sum of all observations
• 𝑛= Number of observations
Example
Data:
5, 8, 10, 12, 15
Mean:
5 + 8 + 10 + 12 + 15 50
𝑥ˉ = = = 10
5 5
Properties of Mean
1. Uses all observations in the dataset.
2. Easy to calculate.
3. Algebraically tractable.
4. Sensitive to extreme values (outliers).
5. Widely used in statistical analysis.
Advantages
1. Simple and easy to understand.
2. Uses all data values.
3. Suitable for mathematical calculations.
Limitations
1. Affected by very large or very small values.
2. May not represent highly skewed data accurately.
2. Median
The Median is the middle value of a dataset when the observations are
arranged in ascending or descending order.
Formula
For odd number of observations:
𝑛 + 1 𝑡ℎ
Median = ( ) observation
2
For even number of observations:
Middle Value1 + Middle Value2
Median =
2
Example 1 (Odd Number of Observations)
Data: 2, 4, 6, 8, 10
Middle value: 6
Therefore, Median = 6
Example 2 (Even Number of Observations)
Data:
2, 4, 6, 8
Median:
4+6
=5
2
Properties of Median
1. Depends on the position of observations.
2. Not affected by extreme values.
3. Suitable for skewed distributions.
4. Divides the dataset into two equal parts.
Advantages
1. Easy to understand.
2. Resistant to outliers.
3. Useful for ordinal data.
Limitations
1. Does not use all observations.
2. Less suitable for algebraic calculations.
3. Mode
The Mode is the observation that occurs most frequently in a dataset.
Formula
There is no mathematical formula.
The mode is identified by determining the value with the highest
frequency.
Example
Data:
2, 3, 4, 4, 4, 5, 6
The value 4 occurs most frequently.
Therefore,
Mode = 4
Types of Mode
Unimodal
One mode.
Example:
2, 4, 4, 6
Mode = 4
Bimodal
Two modes.
Example:
2, 2, 4, 4, 6
Modes = 2 and 4
Multimodal
More than two modes.
Example:
1, 1, 2, 2, 3, 3
Modes = 1, 2, and 3
Properties of Mode
1. Represents the most common observation.
2. Can be used for both numerical and categorical data.
3. Not affected by extreme values.
4. A dataset may have one, more than one, or no mode.
Advantages
1. Simple to determine.
2. Useful for categorical data.
3. Not affected by outliers.
Limitations
1. May not be unique.
2. Does not utilize all observations.
Comparison of Mean, Median, and Mode
Measure Uses All Data Affected by Outliers
Mean Arithmetic average Yes Yes
Median Middle value No No
Mode Most frequent value No No
Applications
Mean
• Academic performance analysis
• Economic studies
• Scientific research
Median
• Income distribution studies
• Property price analysis
• Skewed datasets
Mode
• Market research
• Consumer preference studies
• Categorical data analysis

Measures of Dispersion

Measures of Dispersion are statistical measures that describe the spread or

variability of data around a central value. They indicate how closely or
widely the observations are distributed.
The most common measures of dispersion are:
1. Range
2. Variance
3. Standard Deviation
Explanation
While measures of central tendency provide a representative value of a
dataset, measures of dispersion indicate how much the observations differ
from one another.
• Small dispersion indicates that data values are close to each other.
• Large dispersion indicates that data values are widely spread.
Measures of dispersion help assess the consistency and reliability of data.
1. Range
The Range is the difference between the largest and smallest observations
in a dataset.
Formula
Range = Maximum Value − Minimum Value
Example
Data:
5, 8, 10, 12, 15
Maximum Value = 15
Minimum Value = 5
Range:
15 − 5 = 10
Therefore,
Range = 10
Properties
1. Simplest measure of dispersion.
2. Depends only on the largest and smallest values.
3. Easy to calculate.
4. Highly affected by extreme values.
Advantages
1. Simple and quick to compute.
2. Provides a rough measure of spread.
Limitations
1. Uses only two observations.
2. Not reliable for large datasets.
3. Sensitive to outliers.
2. Variance
Variance measures the average squared deviation of observations from
the mean.
It indicates how far the observations are spread from the average value.
Formula (Population Variance)

2
∑(𝑥 − 𝑥ˉ)2
𝜎 =
𝑁
where:
• 𝜎 2 = Variance
• 𝑥= Observation
• 𝑥ˉ= Mean
• 𝑁= Total number of observations
Formula (Sample Variance)

2
∑(𝑥 − 𝑥ˉ)2
𝑠 =
𝑛−1
𝜎
𝑛
𝑥ˉ ≈ 0, ; 𝑠 2 ≈ 1.751
xbar=0s^2 ~ 1.751
where:
• 𝑠 2 = Sample variance , 𝑛= Sample size
Example
Data:
2, 4, 6
Mean:
2+4+6
𝑥ˉ = =4
3
Squared deviations:
(2 − 4)2 = 4
(4 − 4)2 = 0
(6 − 4)2 = 4
Variance:
4+0+4 8
𝜎2 = =
3 3
Properties
1. Uses all observations.
2. Always non-negative.
3. Larger variance indicates greater dispersion.
4. Variance equals zero only when all observations are identical.
Advantages
1. Uses all data values.
2. Widely used in statistical analysis.
3. Forms the basis for many advanced statistical techniques.
Limitations
1. Expressed in squared units.
2. Less intuitive than standard deviation.
3. Standard Deviation
The Standard Deviation is the positive square root of variance.
It measures the average spread of observations around the mean in the
same units as the original data.
Formula

𝜎 = √𝜎 2
Sample Standard Deviation

𝑠 = √𝑠 2
𝜎
𝑛
𝑥ˉ ≈ −0.01, ; 𝑠 2 ≈ 1.934
xbar=-0.01s^2 ~ 1.934
Example
Using the previous variance:
8
𝜎2 =
3

Standard deviation:

8
𝜎=√
3
𝜎 ≈ 1.63

Properties
1. Always non-negative.
2. Uses all observations.
3. Measured in the same units as the data.
4. A larger standard deviation indicates greater variability.
5. Equal to zero when all observations are identical.
Advantages
1. Most widely used measure of dispersion.
2. Easy to interpret.
3. Useful for comparing datasets.
Limitations
1. Sensitive to extreme values.
2. Computation is more complex than range.
Comparison of Measures of Dispersion
Uses All Sensitive to
Measure Formula Basis
Data Outliers
Range Maximum − Minimum No Yes
Average squared
Variance Yes Yes
deviation
Standard
Square root of variance Yes Yes
Deviation

Applications
Range
• Preliminary data analysis
• Quality control
Variance
• Statistical inference
• Probability distributions
• Risk analysis
Standard Deviation
• Data analysis
• Research studies
• Finance and investment
• Machine learning
 Moments, Skewness, and Kurtosis:
Characteristics of Data Distribution

Moments, skewness, and kurtosis are statistical measures used to describe

the shape and characteristics of a data distribution.
They provide information about:
• Central tendency
• Variability
• Symmetry
• Peakedness of a distribution
These measures help in understanding the nature of data beyond simple
averages and dispersion.

Moments

A moment is a quantitative measure that describes specific characteristics

of a distribution.
Moments are calculated with respect to a reference point, usually the
origin or the mean.
Types of Moments
1. Raw Moments (Moments about the Origin)
The 𝑟 𝑡ℎ raw moment is defined as
1
𝜇𝑟′ = ∑𝑥 𝑟
𝑁
where:
• 𝜇𝑟′ = 𝑟 𝑡ℎ raw moment
• 𝑥= observation
 • 𝑁= number of observations
2. Central Moments (Moments about the Mean)
The 𝑟 𝑡ℎ central moment is defined as
1
𝜇𝑟 = ∑(𝑥 − 𝑥ˉ)𝑟
𝑁
where:
• 𝑥ˉ= mean of the data
Important Moments
First Central Moment
𝜇1 = 0

The first central moment is always zero.

Second Central Moment
𝜇2 = 𝜎 2

Represents the variance of the distribution.

Third Central Moment
Measures asymmetry of the distribution and is related to skewness.
Fourth Central Moment
Measures peakedness of the distribution and is related to kurtosis.

Skewness

Skewness is a measure of the asymmetry of a distribution about its mean.

It indicates whether data is distributed more towards one side of the
distribution.
Coefficient of Skewness
𝜇3
𝛾1 =
𝜎3
where:
• 𝜇3 = third central moment
• 𝜎= standard deviation
Types of Skewness
1. Symmetrical Distribution
Mean = Median = Mode
Characteristics:
• Equal spread on both sides.
• Skewness = 0.
2. Positively Skewed Distribution
Mean > Median > Mode
Characteristics:
• Long tail on the right side.
• Skewness is positive.
Examples:
• Income distributions
• Property prices
3. Negatively Skewed Distribution
Mean < Median < Mode
Characteristics:
• Long tail on the left side.
• Skewness is negative.
Examples:
• Scores in very easy examinations
Properties of Skewness
1. Indicates symmetry of data.
2. Positive value indicates right skew.
3. Negative value indicates left skew.
4. Zero indicates perfect symmetry.

Kurtosis

Kurtosis measures the degree of peakedness or flatness of a distribution

relative to a normal distribution.
It describes how concentrated the observations are around the mean.
Coefficient of Kurtosis
𝜇4
𝛽2 =
𝜇22
where:
• 𝜇4 = fourth central moment
• 𝜇2 = second central moment (variance)
Types of Kurtosis
1. Mesokurtic Distribution
Characteristics:
• Same peakedness as a normal distribution.
• Kurtosis = 3.
Example:
• Normal distribution
2. Leptokurtic Distribution
Characteristics:
• Sharper peak.
• More observations near the mean.
• Heavy tails.
𝛽2 > 3

3. Platykurtic Distribution
Characteristics:
• Flatter peak.
• Observations more evenly spread.
𝛽2 < 3

Properties of Kurtosis
1. Measures peakedness of data.
2. Indicates concentration around the mean.
3. Helps identify heavy-tailed distributions.
4. Useful in risk analysis and statistical modeling.

Relationship Between Moments, Skewness, and Kurtosis

Measure Based On Describes

First Moment Mean Central location
Second Moment Variance Dispersion
Third Moment Skewness Asymmetry
Fourth Moment Kurtosis Peakedness
Example
Consider two datasets having the same mean and variance.
• Dataset A is perfectly symmetric.
• Dataset B has a long right tail.
Dataset B will have positive skewness.
Similarly:
• A sharply peaked distribution has high kurtosis.
• A flatter distribution has low kurtosis.
Thus, skewness and kurtosis provide additional information that mean
and variance alone cannot describe.
Applications
Moments
1. Statistical modeling
2. Probability theory
3. Data analysis
Skewness
1. Financial risk analysis
2. Quality control
3. Income distribution studies
Kurtosis
1. Risk management
2. Machine learning
3. Financial forecasting
4. Reliability analysis
 Moment-Generating Functions
(MGFs) of Random Variables

A Moment-Generating Function (MGF) is a function that uniquely

characterizes the probability distribution of a random variable and can be
used to generate its moments.
For a random variable 𝑋, the moment-generating function is defined as
𝑀𝑋 (𝑡) = 𝐸(𝑒 𝑡𝑋 )
where:
• 𝑀𝑋 (𝑡)= Moment-generating function of 𝑋
• 𝐸= Expected value (expectation)
• 𝑡= Real-valued parameter
The MGF exists if 𝐸(𝑒 𝑡𝑋 )is finite for values of 𝑡in some neighborhood of
0.
Explanation
The MGF is called a moment-generating function because the moments of
a random variable can be obtained by differentiating the function and
evaluating at 𝑡 = 0.
It provides a convenient way to:
• Find moments
• Identify probability distributions
• Analyze sums of independent random variables
MGF for Discrete Random Variables
If 𝑋is a discrete random variable with probability mass function 𝑃(𝑋 = 𝑥),
then
𝑀𝑋 (𝑡) = ∑𝑒 𝑡𝑥 𝑃(𝑋 = 𝑥)
where the summation extends over all possible values of 𝑋.
MGF for Continuous Random Variables
If 𝑋is a continuous random variable with probability density function
𝑓(𝑥), then
∞
𝑀𝑋 (𝑡) = ∫ 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥
−∞

Generating Moments from MGF

The moments of a random variable can be obtained by differentiating the
MGF.
First Moment (Mean)
𝐸(𝑋) = 𝑀𝑋′ (0)
Second Moment
𝐸(𝑋 2 ) = 𝑀𝑋′′ (0)
nth Moment
(𝑛)
𝐸(𝑋 𝑛 ) = 𝑀𝑋 (0)
(𝑛)
where 𝑀𝑋 (0)denotes the 𝑛𝑡ℎ derivative of the MGF evaluated at 𝑡 = 0.
Example
Consider a random variable 𝑋taking values:
0 with probability 0.5
𝑋={
1 with probability 0.5

Then
𝑀𝑋 (𝑡) = 𝐸(𝑒 𝑡𝑋 )
= 𝑒 0𝑡 (0.5) + 𝑒 𝑡 (0.5)
1 + 𝑒𝑡
=
2

Therefore,
1 + 𝑒𝑡
𝑀𝑋 (𝑡) =
2
Finding the Mean
Differentiate:
𝑒𝑡
𝑀𝑋′ (𝑡) =
2

At 𝑡 = 0,
1
𝑀𝑋′ (0) =
2

Hence,
𝐸(𝑋) = 0.5

Important MGFs
1. Bernoulli Distribution
𝑀𝑋 (𝑡) = 𝑞 + 𝑝𝑒 𝑡

where
𝑞 =1−𝑝

2. Binomial Distribution
𝑀𝑋 (𝑡) = (𝑞 + 𝑝𝑒 𝑡 )𝑛

3. Poisson Distribution
𝑡 −1)
𝑀𝑋 (𝑡) = 𝑒 𝜆(𝑒

4. Normal Distribution
1 2 2
𝑀𝑋 (𝑡) = 𝑒 𝜇𝑡+2𝜎 𝑡

where
• 𝜇= Mean
 • 𝜎 2 = Variance
Properties of MGF
1. Uniqueness Property
If two random variables have the same MGF, then they have the same
probability distribution.
2. Generates Moments
All moments can be obtained by successive differentiation.
3. MGF at Zero
𝑀𝑋 (0) = 1

4. Sum of Independent Random Variables

If 𝑋and 𝑌are independent,
𝑀𝑋+𝑌 (𝑡) = 𝑀𝑋 (𝑡)𝑀𝑌 (𝑡)
This property is very useful in probability theory.
Advantages of MGF
1. Provides a compact representation of a distribution.
2. Simplifies calculation of moments.
3. Helps identify probability distributions.
4. Useful in studying sums of independent random variables.
5. Widely used in statistical inference and probability theory.
Limitations of MGF
1. May not exist for all distributions.
2. Some distributions require alternative functions such as characteristic
functions.
3. Computation may become complex for complicated distributions.
Applications
1. Probability Theory
Used to derive moments and distribution properties.
2. Statistical Inference
Helps in deriving sampling distributions.
3. Queueing Theory
Used in modeling arrival and service processes.
4. Reliability Engineering
Used to study lifetimes of systems and components.
5. Machine Learning and Data Analysis
Used in theoretical analysis of probability models.
Probability distributions

Common Distributions

A probability distribution describes how probabilities are assigned to the

possible values of a random variable.
Some of the most important distributions in statistics and probability are:
1. Binomial Distribution
2. Poisson Distribution
3. Normal Distribution
4. Uniform Distribution
5. Exponential Distribution
These distributions are widely used in science, engineering, economics,
machine learning, and data analysis.

1. Binomial Distribution
The Binomial Distribution describes the probability of obtaining a fixed
number of successes in a fixed number of independent trials, where each
trial has only two possible outcomes: success or failure.
Conditions
1. Fixed number of trials (𝑛)
2. Independent trials
3. Two possible outcomes
4. Constant probability of success (𝑝)
Probability Mass Function
𝑛
𝑃(𝑋 = 𝑥) = ( ) 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥
𝑥
where:
• 𝑛= number of trials
• 𝑥= number of successes
• 𝑝= probability of success
Mean
𝐸(𝑋) = 𝑛𝑝

Variance
𝑉𝑎𝑟(𝑋) = 𝑛𝑝(1 − 𝑝)

Example
• Number of heads obtained in 10 coin tosses.
• Number of defective items in a sample.

2. Poisson Distribution
The Poisson Distribution describes the probability of a given number of
events occurring in a fixed interval of time, area, or space when events
occur independently at a constant average rate.
Probability Mass Function
𝑒 −𝜆 𝜆𝑥
𝑃(𝑋 = 𝑥) =
𝑥!
where:
• 𝜆= average number of occurrences
• 𝑥= number of occurrences
Mean
𝐸(𝑋) = 𝜆
Variance
𝑉𝑎𝑟(𝑋) = 𝜆
Example
• Number of phone calls received per hour.
• Number of accidents at a road intersection.
• Number of website visits per minute.

3. Normal Distribution
The Normal Distribution is a continuous probability distribution that is
symmetric about its mean and has a bell-shaped curve.
It is one of the most important distributions in statistics.
Probability Density Function
1 (𝑥−𝜇)2
−
𝑓(𝑥) = 𝑒 2𝜎2
𝜎√2𝜋
where:
• 𝜇= mean
• 𝜎= standard deviation
Characteristics
1. Bell-shaped curve.
2. Symmetric about the mean.
3. Mean = Median = Mode.
4. Total area under the curve equals 1.
Mean
𝐸(𝑋) = 𝜇
Variance
𝑉𝑎𝑟(𝑋) = 𝜎 2
Example
• Heights of people
• Examination scores
• Measurement errors

4. Uniform Distribution
The Uniform Distribution is a continuous distribution in which all values
within a specified interval are equally likely.
Probability Density Function
For 𝑎 ≤ 𝑥 ≤ 𝑏,
1
𝑓(𝑥) =
𝑏−𝑎
Mean
𝑎+𝑏
𝐸(𝑋) =
2
Variance
(𝑏 − 𝑎)2
𝑉𝑎𝑟(𝑋) =
12
Characteristics
1. Constant probability density.
2. Every value in the interval has equal likelihood.
3. Rectangular-shaped distribution.
Example
• Random number generation.
• Selecting a point uniformly on a line segment.

5. Exponential Distribution
The Exponential Distribution is a continuous probability distribution that
models the time between successive events in a Poisson process.
Probability Density Function
For 𝑥 ≥ 0,
𝑓(𝑥) = 𝜆𝑒 −𝜆𝑥
𝐴
𝑘
𝑦 = 𝐴𝑒 −𝑘𝑡 = 6𝑒 −1𝑡
where:
• 𝜆= rate parameter
Mean
1
𝐸(𝑋) =
𝜆
Variance
1
𝑉𝑎𝑟(𝑋) =
𝜆2
Characteristics
1. Continuous distribution.
2. Right-skewed.
3. Models waiting times.
4. Has the memoryless property.
Example
• Lifetime of electronic components.
• Waiting time between customer arrivals.
• Time between machine failures.
Comparison of Common Distributions

Distribution Type Parameters Mean Variance

Binomial Discrete 𝑛, 𝑝 𝑛𝑝 𝑛𝑝(1 − 𝑝)
Poisson Discrete 𝜆 𝜆 𝜆
Normal Continuous 𝜇, 𝜎 𝜇 𝜎2
𝑎+𝑏 (𝑏 − 𝑎)2
Uniform Continuous 𝑎, 𝑏
2 12
1 1
Exponential Continuous 𝜆
𝜆 𝜆2

Applications
Binomial Distribution
• Quality control
• Survey analysis
• Success-failure experiments
Poisson Distribution
• Traffic analysis
• Call center management
• Reliability studies
Normal Distribution
• Statistical inference
• Natural phenomena modeling
• Machine learning
Uniform Distribution
• Simulation
• Random number generation
• Computer graphics
Exponential Distribution
• Queueing systems
• Reliability engineering
• Survival analysis

Sampling Distributions
of Mean and Variance

A sampling distribution is the probability distribution of a statistic

obtained from all possible samples of a fixed size drawn from a
population.
Two important sampling distributions are:
1. Sampling Distribution of the Sample Mean
2. Sampling Distribution of the Sample Variance
These distributions form the basis of statistical inference, estimation, and
hypothesis testing.
Explanation
Different samples drawn from the same population generally produce
different values of a statistic.
For example:
• Different samples give different sample means.
• Different samples give different sample variances.
The distribution of these sample statistics is called a sampling distribution.
Sampling Distribution of the Sample Mean
Let
𝑋1 , 𝑋2 , … , 𝑋𝑛
be a random sample of size 𝑛drawn from a population with mean 𝜇and
variance 𝜎 2 .
The sample mean is defined as
1 𝑛
𝑋ˉ = ∑ 𝑋𝑖
𝑛 𝑖=1

The probability distribution of 𝑋ˉis called the sampling distribution of the

sample mean.
Mean of the Sampling Distribution
The expected value of the sample mean is
𝐸(𝑋ˉ) = 𝜇
Thus, the sample mean is an unbiased estimator of the population mean.
Variance of the Sampling Distribution
The variance of the sample mean is
𝜎2
𝑉𝑎𝑟(𝑋ˉ) =
𝑛
𝜎
𝑉𝑎𝑟(𝑋) = 𝜎 2 ≈ 3.24
μ-σ+σVar(X) ≈ 3.24
Standard Error of the Mean
The standard deviation of the sampling distribution is called the Standard
Error (SE).
𝜎
𝑆𝐸(𝑋ˉ) =
√𝑛
Interpretation
• Larger sample sizes produce smaller standard errors.
• Sample means become more concentrated around the population
mean.
Central Limit Theorem (CLT)
Statement
For a sufficiently large sample size 𝑛, the sampling distribution of the
sample mean approaches a normal distribution regardless of the
population distribution.
2
𝜎
𝑋ˉ ∼ 𝑁 (𝜇, )
𝑛
approximately for large 𝑛.
Importance
The Central Limit Theorem is one of the most important results in
statistics because it allows normal probability methods to be used for
many practical problems.
Sampling Distribution of the Sample Variance
The sample variance is
1 𝑛
2
𝑆 = ∑ ( 𝑋𝑖 − 𝑋ˉ)2
𝑛 − 1 𝑖=1
𝑥ˉ ≈ −0.01, ; 𝑠 2 ≈ 1.934
xbar=-0.01s^2 ~ 1.934
The probability distribution of 𝑆 2 is called the sampling distribution of the
sample variance.
Mean of Sample Variance
The sample variance is an unbiased estimator of population variance.
𝐸(𝑆 2 ) = 𝜎 2
Chi-Square Distribution
If the population is normally distributed, then
(𝑛 − 1)𝑆 2 2
∼ 𝜒𝑛−1
𝜎2
where:
•
2
𝜒𝑛−1 denotes a chi-square distribution
• Degrees of freedom = 𝑛 − 1
This result is fundamental in variance estimation and hypothesis testing.
Properties of Sampling Distribution of Variance
1. Depends on sample size.
2. Depends on the population variance.
3. Follows a chi-square distribution when the population is normal.
4. Becomes more concentrated as sample size increases.
Difference Between Population and Sampling Distribution

Population Distribution Sampling Distribution

Distribution of individual observations Distribution of sample statistics
Describes population values Describes sample estimates
Mean = 𝜇 Mean of 𝑋ˉ= 𝜇
Variance = 𝜎 2 Variance of 𝑋ˉ= 𝜎 2 /𝑛
Example
Suppose a population has:
𝜇 = 50
𝜎 = 10

and samples of size

𝑛 = 25
are repeatedly selected.
Mean of Sampling Distribution
𝐸(𝑋ˉ) = 50
Variance
102
𝑉𝑎𝑟(𝑋ˉ) = =4
25
Standard Error
10
𝑆𝐸(𝑋ˉ) = =2
5
Thus, sample means are centered at 50 with standard deviation 2.
Applications
Sampling Distribution of Mean
1. Confidence intervals
2. Hypothesis testing
3. Quality control
4. Survey analysis
Sampling Distribution of Variance
1. Variance estimation
2. Process control
3. Reliability studies
4. Statistical testing
Statistical inference

Point and Interval Estimation

Estimation is the process of using sample data to infer the value of an

unknown population parameter.
There are two main types of estimation:
1. Point Estimation – Provides a single value as an estimate of a
population parameter.
2. Interval Estimation – Provides a range of values within which the
population parameter is likely to lie.
Point Estimation
A point estimate is a single numerical value used to estimate an unknown
population parameter.
The statistic used to obtain the estimate is called a point estimator.
Examples

Population Parameter Point Estimator

Population Mean (𝜇) Sample Mean (𝑥ˉ)

Population Variance (𝜎 2 ) Sample Variance (𝑠 2 )

Population Proportion (𝑝) Sample Proportion (𝑝̂ )

Explanation
Suppose a population contains thousands of individuals.
Instead of studying the entire population, a sample is selected and a
statistic is calculated.
That statistic serves as an estimate of the corresponding population
parameter.
For example:
If the average mark of 100 sampled students is 72,
𝑥ˉ = 72
then 72 is the point estimate of the population mean.
Properties of a Good Estimator
1. Unbiasedness
An estimator is unbiased if its expected value equals the parameter being
estimated.
𝐸(𝜃̂) = 𝜃
where:
• 𝜃̂= estimator
• 𝜃= population parameter
2. Consistency
As sample size increases, the estimator approaches the true parameter
value.
3. Efficiency
Among unbiased estimators, the estimator with the smallest variance is
preferred.
4. Sufficiency
An estimator is sufficient if it contains all available information about the
parameter.
Interval Estimation
An interval estimate gives a range of values within which the population
parameter is expected to lie with a specified level of confidence.
This range is called a confidence interval.
Confidence Interval
A confidence interval consists of:
• Lower confidence limit
• Upper confidence limit
and is associated with a confidence level such as:
• 90%
• 95%
• 99%
General Form
Confidence Interval = Point Estimate ± Margin of Error
Confidence Interval for Population Mean
When population standard deviation 𝜎is known:
𝜎
𝑥ˉ ± 𝑧𝛼/2
√𝑛
where:
• 𝑥ˉ= sample mean
• 𝑧𝛼/2 = standard normal critical value
• 𝜎= population standard deviation
• 𝑛= sample size
Common Confidence Levels
Confidence Level Critical Value

90% 1.645
95% 1.96

99% 2.576
Example
Suppose:
𝑥ˉ = 50
𝜎 = 10
𝑛 = 100
95% confidence level:
𝑧 = 1.96
Margin of error:
10
1.96 ×
√100
= 1.96
Confidence interval:
50 ± 1.96
(48.04, 51.96)
Interpretation:
We are 95% confident that the population mean lies between 48.04 and
51.96.
Interpretation of Confidence Level
A 95% confidence level means that if many samples are taken and
confidence intervals are constructed, approximately 95% of those
intervals will contain the true population parameter.
Difference Between Point and Interval Estimation
Point Estimation Interval Estimation
Single value estimate Range of values
Simple to compute More informative
No measure of uncertainty Includes uncertainty
Less reliable More reliable
Advantages of Point Estimation
1. Easy to calculate.
2. Easy to interpret.
3. Useful for quick estimates.
Limitations
1. Does not indicate estimation error.
2. Provides no measure of reliability.
Advantages of Interval Estimation
1. Indicates uncertainty.
2. More informative than a point estimate.
3. Provides a confidence level.
4. Widely used in statistical inference.
Limitations
1. More computationally intensive.
2. Requires confidence level selection.
Applications
Point Estimation
1. Survey analysis
2. Business forecasting
3. Quality control
Interval Estimation
1. Scientific research
2. Medical studies
3. Opinion polls
4. Statistical inference
 Estimating Population Parameters
from Sample Data

Estimation of population parameters from sample data is the process of

using statistics calculated from a sample to infer the values of unknown
characteristics (parameters) of an entire population.
Since studying every member of a population is often impractical, a
representative sample is used to estimate population parameters.
Explanation
A population consists of all individuals or observations of interest, while a
sample is a subset selected from the population.
Population characteristics are called parameters, and sample characteristics
are called statistics.
The objective of estimation is to use sample statistics to make informed
conclusions about population parameters.
Population Parameters and Sample Statistics
Population Parameter Symbol Sample Statistic Symbol
Population Mean 𝜇 Sample Mean 𝑥ˉ
Population Variance 𝜎2 Sample Variance 𝑠2
Population Standard 𝜎 Sample Standard 𝑠
Deviation Deviation

Population Proportion 𝑝 Sample Proportion 𝑝̂

Why Estimation is Necessary

1. Populations may be very large.
2. Collecting data from every member is expensive.
3. Complete enumeration may take too much time.
4. Some populations are continuously changing.
Therefore, sample data is used to estimate population characteristics.
Estimation of Population Mean
Sample Mean as an Estimator
The population mean 𝜇is estimated using the sample mean.
∑𝑥
𝑥ˉ =
𝑛
where:
• 𝑥ˉ= sample mean
• 𝑥= observations
• 𝑛= sample size
Example
Sample data:
10, 12, 15, 18, 20
Sample mean:
10 + 12 + 15 + 18 + 20
𝑥ˉ = = 15
5
Therefore,
𝜇 ≈ 15

Estimation of Population Variance

Sample Variance as an Estimator
Population variance is estimated using sample variance.

2
∑(𝑥 − 𝑥ˉ)2
𝑠 =
𝑛−1
𝑥ˉ ≈ −0.01, ; 𝑠 2 ≈ 1.934
xbar=-0.01s^2 ~ 1.934
The denominator 𝑛 − 1ensures that the estimator is unbiased.
Example
Data:
2, 4, 6
Mean:
𝑥ˉ = 4
Sample variance:

2
(2 − 4)2 + (4 − 4)2 + (6 − 4)2
𝑠 =
2

4+0+4
= =4
2

Thus,
𝜎2 ≈ 4

Estimation of Population Standard Deviation

Population standard deviation is estimated using sample standard
deviation.

𝑠 = √𝑠 2
𝜎
𝑛
𝑥ˉ ≈ −0.01, ; 𝑠 2 ≈ 1.934
xbar=-0.01s^2 ~ 1.934
where:
• 𝑠= sample standard deviation
Estimation of Population Proportion
Sample Proportion
Population proportion is estimated using
𝑥
𝑝̂ =
𝑛
where:
• 𝑥= number of successes
• 𝑛= sample size
Example
Suppose 60 students out of a sample of 100 students pass an examination.
60
𝑝̂ = = 0.60
100

Therefore, the estimated population proportion is

𝑝 ≈ 0.60
Types of Estimation
1. Point Estimation
Provides a single numerical value as an estimate.
Examples:
𝑥ˉ, 𝑠 2 , 𝑝̂
2. Interval Estimation
Provides a range of values likely to contain the true parameter.
Example:
48 ≤ 𝜇 ≤ 52
with 95% confidence.
Characteristics of Good Estimators
1. Unbiasedness
An estimator should estimate the parameter correctly on average.
𝐸(𝜃̂) = 𝜃
2. Consistency
The estimator should become more accurate as sample size increases.
3. Efficiency
Among unbiased estimators, the one with the smallest variance is
preferred.
4. Sufficiency
The estimator should utilize all relevant sample information.
Sampling Error
The difference between a sample statistic and the corresponding
population parameter is called sampling error.
Sampling Error = Statistic − Parameter
Sampling error decreases as sample size increases.
Applications
1. Opinion Polls
Estimating voter preferences.
2. Quality Control
Estimating defect rates in manufacturing.
3. Medical Research
Estimating disease prevalence.
4. Business Analysis
Estimating customer satisfaction and market demand.
5. Scientific Research
Estimating characteristics of large populations.
 Hypothesis Testing

Hypothesis Testing is a statistical procedure used to determine whether

there is sufficient evidence from a sample to support or reject a claim
about a population parameter.
It helps researchers make decisions regarding population characteristics
based on sample data.
Explanation
In hypothesis testing, a statement about a population parameter is
proposed and then tested using sample data.
The objective is to determine whether the observed sample results are
consistent with the proposed statement.
Since conclusions are based on samples rather than the entire population,
hypothesis testing provides a systematic method for making decisions
under uncertainty.
Statistical Hypothesis
A statistical hypothesis is an assumption or claim about a population
parameter.
Examples:
• The average height of students is 170 cm.
• The proportion of defective products is 5%.
• The average lifetime of a bulb is 1000 hours.
Types of Hypotheses
1. Null Hypothesis (𝑯𝟎 )
The null hypothesis is the statement that there is no significant difference
or no effect.
It is the hypothesis initially assumed to be true.
Example:
𝐻0 : 𝜇 = 50
This states that the population mean is 50.
2. Alternative Hypothesis (𝑯𝟏 or 𝑯𝒂 )
The alternative hypothesis is the statement that contradicts the null
hypothesis.
Example:
𝐻1 : 𝜇 ≠ 50
This states that the population mean is not 50.
Types of Alternative Hypotheses
Two-Tailed Test
𝐻1 : 𝜇 ≠ 𝜇0
Used when deviations in both directions are important.
Right-Tailed Test
𝐻1 : 𝜇 > 𝜇0
Used when testing for an increase.
Left-Tailed Test
𝐻1 : 𝜇 < 𝜇0
Used when testing for a decrease.
Steps in Hypothesis Testing
Step 1: State the Hypotheses
Formulate:
• Null Hypothesis (𝐻0 )
• Alternative Hypothesis (𝐻1 )
Step 2: Select Significance Level
Choose the significance level 𝛼.
Common choices:
• 0.05
• 0.01
Step 3: Select the Appropriate Test Statistic
Examples:
• Z-test
• t-test
• Chi-square test
• F-test
Step 4: Compute the Test Statistic
Calculate the test statistic using sample data.
Step 5: Determine the Critical Region
Find the critical value corresponding to the chosen significance level.
Step 6: Make a Decision
• Reject 𝐻0 if the test statistic falls in the critical region.
• Otherwise, fail to reject 𝐻0 .
Step 7: State the Conclusion
Interpret the result in the context of the problem.
Significance Level
The significance level is the probability of rejecting the null hypothesis
when it is actually true.
It is denoted by “𝛼”
Common values:
𝛼 = 0.05
𝛼 = 0.01
Errors in Hypothesis Testing
Type I Error
Rejecting a true null hypothesis.
Probability of Type I Error:
𝑃(Type I Error) = 𝛼
Type II Error
Failing to reject a false null hypothesis.
Probability of Type II Error:
𝑃(Type II Error) = 𝛽
Test Statistic for Population Mean
When population standard deviation is known:
𝑥ˉ − 𝜇
𝑍=
𝜎/√𝑛
𝑥
𝜇
𝜎
𝑥−𝜇
𝑧= ≈ 1.2
𝜎
Φ(𝑧) ≈ 88.5%
where:
• 𝑥ˉ= sample mean
• 𝜇= hypothesized population mean
• 𝜎= population standard deviation
• 𝑛= sample size
Decision Rule
Two-Tailed Test (𝜶 = 𝟎. 𝟎𝟓)
Critical values:
±1.96
Decision:
• Reject 𝐻0 if
∣ 𝑍 ∣> 1.96
• Otherwise, fail to reject 𝐻0 .
Example
A manufacturer claims that the average lifetime of bulbs is 1000 hours.
A sample of 64 bulbs has:
𝑥ˉ = 960
𝜎 = 160
Test at 5% significance level.
Step 1
𝐻0 : 𝜇 = 1000
𝐻1 : 𝜇 ≠ 1000
Step 2
𝛼 = 0.05
Step 3
Calculate the test statistic:
960 − 1000
𝑍=
160/√64

−40
=
20

= −2
Step 4
Critical values:
±1.96
Since
∣ −2 ∣> 1.96
Reject 𝐻0 .
Conclusion
There is sufficient evidence to conclude that the average lifetime differs
from 1000 hours.
Applications
1. Medical Research
Testing effectiveness of treatments.
2. Manufacturing
Quality control and process improvement.
3. Business
Evaluating marketing strategies.
4. Education
Comparing teaching methods.
5. Scientific Research
Testing scientific theories and assumptions.
Advantages
1. Provides an objective decision-making process.
2. Quantifies uncertainty.
3. Widely applicable in research and industry.
4. Supports evidence-based conclusions.
Limitations
1. Results depend on sample quality.
2. Possibility of Type I and Type II errors.
3. Incorrect assumptions may affect conclusions.

Formulating Hypotheses

Formulating hypotheses is the process of stating claims or assumptions

about a population parameter before conducting a statistical test.
A hypothesis provides the basis for statistical inference and guides the
hypothesis testing procedure.
Explanation
In hypothesis testing, two competing statements are formulated:
1. Null Hypothesis (𝐻0 )
2. Alternative Hypothesis (𝐻1 or 𝐻𝑎 )
These hypotheses are expressed in terms of population parameters such
as:
• Population mean (𝜇)
• Population proportion (𝑝)
• Population variance (𝜎 2 )
The sample data is then used to decide whether there is sufficient
evidence against the null hypothesis.
Null Hypothesis (𝑯𝟎 )
The Null Hypothesis is a statement that assumes no effect, no difference,
or no change in the population parameter.
It represents the status quo and is assumed to be true unless evidence
suggests otherwise.
Examples
1. Population mean is 50:
𝐻0 : 𝜇 = 50
2. Population proportion is 0.4:
𝐻0 : 𝑝 = 0.4
3. Population variance is 100:
𝐻0 : 𝜎 2 = 100
Characteristics
1. Contains equality (=).
2. Represents the existing claim.
3. Tested directly using sample data.
Alternative Hypothesis (𝑯𝟏 or 𝑯𝒂 )
The Alternative Hypothesis is the statement that contradicts the null
hypothesis.
It represents the claim that the researcher wants to investigate.
Examples
1. Population mean is not 50:
𝐻1 : 𝜇 ≠ 50
2. Population proportion is greater than 0.4:
𝐻1 : 𝑝 > 0.4
3. Population variance is less than 100:
𝐻1 : 𝜎 2 < 100
Characteristics
1. Represents a change, effect, or difference.
2. Does not contain equality.
3. Accepted only when sufficient evidence exists against 𝐻0 .
Types of Alternative Hypotheses
1. Two-Tailed Hypothesis
Used when deviations in both directions are important.
𝐻1 : 𝜇 ≠ 𝜇0
Example
Testing whether the average weight differs from 70 kg.
𝐻0 : 𝜇 = 70
𝐻1 : 𝜇 ≠ 70
Characteristics
• Rejection region in both tails.
• Tests for any significant difference.
2. Right-Tailed Hypothesis
Used when testing whether a parameter is greater than a specified value.
𝐻1 : 𝜇 > 𝜇0
Example
Testing whether the average production exceeds 500 units.
𝐻0 : 𝜇 = 500
𝐻1 : 𝜇 > 500
Characteristics
• Rejection region in the right tail.
• Tests for an increase.
3. Left-Tailed Hypothesis
Used when testing whether a parameter is less than a specified value.
𝐻1 : 𝜇 < 𝜇0
Example
Testing whether the average battery life is less than 10 hours.
𝐻0 : 𝜇 = 10
𝐻1 : 𝜇 < 10
Characteristics
• Rejection region in the left tail.
• Tests for a decrease.
Steps in Formulating Hypotheses
Step 1: Identify the Population Parameter
Determine the parameter to be tested.
Examples:
• Mean (𝜇)
• Proportion (𝑝)
• Variance (𝜎 2 )
Step 2: State the Null Hypothesis
Formulate the statement containing equality.
Example:
𝐻0 : 𝜇 = 100

Step 3: State the Alternative Hypothesis

Formulate the statement opposite to the null hypothesis.
Example:
𝐻1 : 𝜇 ≠ 100

Step 4: Determine the Type of Test

• Two-tailed
• Right-tailed
• Left-tailed
based on the research question.
Examples
Example 1: Average Marks
A school claims that the average mark of students is 75.
Hypotheses:
𝐻0 : 𝜇 = 75
𝐻1 : 𝜇 ≠ 75
(Two-tailed test)
Example 2: Product Lifetime
A manufacturer claims that the average lifetime of a bulb exceeds 1000
hours.
Hypotheses:
𝐻0 : 𝜇 = 1000
𝐻1 : 𝜇 > 1000
(Right-tailed test)
Example 3: Defect Rate
A company wants to determine whether the defect rate is below 5%.
Hypotheses:
𝐻0 : 𝑝 = 0.05
𝐻1 : 𝑝 < 0.05
(Left-tailed test)
Properties
1. Hypotheses are statements about population parameters.
2. The null hypothesis contains equality.
3. The alternative hypothesis reflects the research objective.
4. Only one of the hypotheses can be true.
5. Statistical decisions are based on evidence against 𝐻0 .
Applications
1. Medical research
2. Quality control
3. Market research
4. Educational studies
5. Scientific experimentation

Type I and Type II Errors

In hypothesis testing, decisions are made based on sample data. Since
these decisions are made under uncertainty, errors may occur.
The two possible errors are:
1. Type I Error – Rejecting a true null hypothesis.
2. Type II Error – Failing to reject a false null hypothesis.
These errors are fundamental concepts in statistical inference.
Explanation
When performing a hypothesis test, there are two possible realities:
• The null hypothesis 𝐻0 is true.
• The null hypothesis 𝐻0 is false.
Similarly, there are two possible decisions:
• Reject 𝐻0
• Fail to reject 𝐻0
Combining these possibilities gives four outcomes.
Actual Situation Decision Result
𝐻0 is true Fail to reject 𝐻0 Correct Decision
𝐻0 is true Reject 𝐻0 Type I Error
𝐻0 is false Reject 𝐻0 Correct Decision
𝐻0 is false Fail to reject 𝐻0 Type II Error
Type I Error
A Type I Error occurs when a true null hypothesis is rejected.
In simple terms:
The test concludes that a significant effect exists when, in reality, no effect
exists.
Probability of Type I Error
𝑃(Type I Error) = 𝛼
where:
• 𝛼= Significance level
Common values:
𝛼 = 0.05
𝛼 = 0.01
Example
Suppose a medicine is actually safe.
𝐻0 :Medicine is safe
A statistical test incorrectly rejects 𝐻0 .
Conclusion:
"Medicine is unsafe."
Since the medicine is actually safe, a Type I Error has occurred.
Characteristics
1. Rejecting a true null hypothesis.
2. Controlled by the significance level 𝛼.
3. Also called a False Positive Error.
Type II Error
A Type II Error occurs when a false null hypothesis is not rejected.
In simple terms:
The test fails to detect a real effect that actually exists.
Probability of Type II Error
𝑃(Type II Error) = 𝛽
where:
• 𝛽= Probability of Type II Error
Example
Suppose a medicine is actually unsafe.
𝐻0 :Medicine is safe
The statistical test fails to reject 𝐻0 .
Conclusion:
"Medicine is safe."
Since the medicine is actually unsafe, a Type II Error has occurred.
Characteristics
1. Failing to reject a false null hypothesis.
2. Depends on sample size and test sensitivity.
3. Also called a False Negative Error.
Power of a Test
The Power of a Test is the probability of correctly rejecting a false null
hypothesis.
Formula
Power = 1 − 𝛽
where:
• 𝛽= Probability of Type II Error
Interpretation
• High power is desirable.
• Higher power means a greater chance of detecting a true effect.
• Increasing sample size generally increases power.
Comparison of Type I and Type II Errors

Feature Type I Error Type II Error

Rejecting a true 𝐻0 Failing to reject a false
𝐻0

Probability 𝛼 𝛽
Also Called False Positive False Negative
Consequence Detecting an effect that does Missing an effect that
not exist exists

Relationship Between 𝜶 and 𝜷

For a fixed sample size:
• Decreasing 𝛼generally increases 𝛽.
• Decreasing 𝛽generally increases 𝛼.
Therefore, a balance must be maintained.
Increasing the sample size can help reduce both errors simultaneously.
Real-Life Examples
Quality Control
𝐻0 :Product is acceptable
Type I Error : Rejecting a good product.
Type II Error : Accepting a defective product.
Medical Testing
𝐻0 :Patient is healthy
Type I Error
Diagnosing a healthy person as sick.
Type II Error
Declaring a sick person healthy.
Court Trial
𝐻0 :Defendant is innocent
Type I Error
Convicting an innocent person.
Type II Error
Acquitting a guilty person.
Factors Affecting Errors
Type I Error
Affected by:
• Significance level (𝛼)
Type II Error
Affected by:
• Sample size
• Population variability
• Effect size
• Significance level
Applications
1. Medical research
2. Quality assurance
3. Scientific experiments
4. Market research
5. Machine learning model evaluation
 Significance Levels

The Significance Level is the probability of rejecting the null hypothesis

when it is actually true.
It represents the maximum acceptable risk of committing a Type I Error in
a hypothesis test.
The significance level is denoted by: 𝛼
where:
• 𝛼= Significance Level
Explanation
In hypothesis testing, decisions are made based on sample data.
Since sample data may not perfectly represent the population, there is
always a possibility of making an incorrect decision.
The significance level specifies how much risk of a Type I Error the
researcher is willing to accept.
For example:
𝛼 = 0.05
means there is a 5% risk of rejecting a true null hypothesis.
Relationship with Type I Error
The probability of a Type I Error is equal to the significance level.
𝑃(Type I Error) = 𝛼
Thus:
• Larger 𝛼increases the chance of rejecting 𝐻0 .
• Smaller 𝛼makes rejection of 𝐻0 more difficult.
Common Significance Levels
Significance Level (𝛼) Confidence Level
0.10 90%
0.05 95%
0.01 99%
Relationship:
Confidence Level = 1 − 𝛼
Example:
𝛼 = 0.05
Confidence Level:
1 − 0.05 = 0.95
or 95%.
Choosing a Significance Level
The choice of significance level depends on the consequences of making a
Type I Error.
𝜶 = 𝟎. 𝟏𝟎
Used when:
• Higher tolerance for error exists.
• Exploratory studies are conducted.
𝜶 = 𝟎. 𝟎𝟓
Most commonly used.
Provides a reasonable balance between strictness and flexibility.
𝜶 = 𝟎. 𝟎𝟏
Used when:
• Very high accuracy is required.
• Type I Errors are costly.
Examples:
• Medical research
• Drug testing
• Safety studies
Significance Level and Critical Region
The significance level determines the size of the rejection region (critical
region).
Two-Tailed Test
For
𝛼 = 0.05
the significance level is divided equally between the two tails:
𝛼
= 0.025
2

in each tail.
Critical values:
±1.96
for the standard normal distribution.
Right-Tailed Test
Entire significance level is placed in the right tail.
For
𝛼 = 0.05
critical value:
1.645
Left-Tailed Test
Entire significance level is placed in the left tail.
For
𝛼 = 0.05

critical value:
−1.645
Significance Level and p-Value
of p-Value
The p-value is the probability of obtaining a result at least as extreme as
the observed result, assuming the null hypothesis is true.
Decision Rule
• If 𝑝≤𝛼
Reject 𝐻0 .
• If 𝑝>𝛼
Fail to reject 𝐻0 .
Example
Suppose a test produces:
𝑝 = 0.03

and
𝛼 = 0.05

Since
0.03 < 0.05

Reject the null hypothesis.

There is sufficient evidence against 𝐻0 .
Effects of Changing Significance Level
Increasing 𝜶
• Larger rejection region.
• Easier to reject 𝐻0 .
• Higher probability of Type I Error.
Decreasing 𝜶
• Smaller rejection region.
• Harder to reject 𝐻0 .
• Lower probability of Type I Error.
Advantages
1. Provides a clear decision criterion.
2. Controls the risk of Type I Error.
3. Standardizes hypothesis testing procedures.
4. Widely accepted in research and industry.
Limitations
1. Choice of 𝛼can be subjective.
2. Does not indicate the magnitude of an effect.
3. Very small significance levels may increase Type II Error.
Applications
1. Medical Research
Testing effectiveness of treatments.
2. Manufacturing
Quality control and process monitoring.
3. Business Analytics
Evaluating marketing campaigns.
4. Scientific Research
Testing theories and hypotheses.
5. Machine Learning
Evaluating model performance statistically.
 Various Tests: t-Tests,
Chi-Square Tests, and ANOVA

Statistical tests are procedures used to make decisions about population

parameters based on sample data.
Some of the most widely used hypothesis tests are:
1. t-Test – Used for testing means.
2. Chi-Square Test – Used for categorical data and variances.
3. ANOVA (Analysis of Variance) – Used for comparing multiple
means.
These tests help determine whether observed differences are statistically
significant.
1. t-Test
A t-Test is a statistical test used to determine whether there is a significant
difference between means when the population standard deviation is
unknown.
It is based on the t-distribution.
Conditions for Using a t-Test
1. Sample is randomly selected.
2. Population is approximately normal.
3. Population standard deviation is unknown.
4. Sample size is usually small.
Types of t-Tests
1. One-Sample t-Test
Used to compare a sample mean with a known population mean.
Test Statistic
𝑥ˉ − 𝜇
𝑡=
𝑠/√𝑛
where:
• 𝑥ˉ= sample mean
• 𝜇= hypothesized population mean
• 𝑠= sample standard deviation
• 𝑛= sample size
2. Independent Samples t-Test
Used to compare means of two independent groups.
Examples:
• Marks of students from two schools
• Performance of two machines
3. Paired t-Test
Used when observations occur in pairs.
Examples:
• Before and after treatment
• Pre-test and post-test scores
Applications of t-Test
1. Medical studies
2. Educational research
3. Quality control
4. Business analysis
2. Chi-Square Test
The Chi-Square Test (𝜒 2 ) is a non-parametric statistical test used to
analyze categorical data and determine whether observed frequencies
differ significantly from expected frequencies.
Chi-Square Test Statistic

2
(𝑂 − 𝐸)2
𝜒 =∑
𝐸
where:
• 𝑂= Observed frequency
• 𝐸= Expected frequency
Types of Chi-Square Tests
1. Goodness-of-Fit Test
Determines whether observed data fits a specified distribution.
Example:
• Testing whether a die is fair.
2. Test of Independence
Determines whether two categorical variables are related.
Example:
• Relationship between gender and product preference.
3. Test of Homogeneity
Determines whether different populations have the same distribution.
Example:
• Comparing voting preferences across regions.
Properties
1. Uses categorical data.
2. Based on frequencies.
3. Always non-negative.
4. Larger values indicate greater disagreement between observed and
expected frequencies.
Applications
1. Market research
2. Social sciences
3. Biology
4. Quality assurance
3. ANOVA (Analysis of Variance)
Analysis of Variance (ANOVA) is a statistical method used to test whether
three or more population means are equal.
Instead of performing multiple t-tests, ANOVA compares all group means
simultaneously.
Hypotheses in ANOVA
Null Hypothesis
All population means are equal.
𝐻0 : 𝜇1 = 𝜇2 = 𝜇3 = ⋯ = 𝜇𝑘
Alternative Hypothesis
At least one population mean differs.
𝐻1 :At least one mean is different
Basic Idea of ANOVA
ANOVA compares:
1. Variation between groups
2. Variation within groups
If between-group variation is significantly larger than within-group
variation, the null hypothesis is rejected.
F-Test Statistic
Mean Square Between Groups
𝐹=
Mean Square Within Groups
Large values of 𝐹indicate significant differences among group means.
Types of ANOVA
1. One-Way ANOVA
Uses one factor or independent variable.
Example:
• Comparing average marks of students from three schools.
2. Two-Way ANOVA
Uses two factors simultaneously.
Example:
• Studying effects of teaching method and gender on performance.
Assumptions of ANOVA
1. Independent observations.
2. Normally distributed populations.
3. Equal population variances.
4. Random sampling.
Applications
1. Agriculture
2. Education
3. Medical research
4. Industrial experiments
5. Machine learning evaluation
Comparison of t-Test, Chi-Square Test, and ANOVA

Feature t-Test Chi-Square Test ANOVA

Purpose Compare Analyze Compare multiple
means frequencies means

Data Type Quantitative Categorical Quantitative

Number of One or Two Any Three or More
Groups

Test Statistic 𝑡 𝜒2 𝐹
Distribution t-distribution Chi-square F-distribution
Used distribution
Advantages
t-Test
1. Simple and widely used.
2. Effective for small samples.
Chi-Square Test
1. Suitable for categorical data.
2. Does not require normality of raw data.
ANOVA
1. Compares several groups simultaneously.
2. Reduces multiple testing errors.
3. Efficient for experimental studies.
Limitations
t-Test
1. Limited to one or two groups.
2. Assumes normality.
Chi-Square Test
1. Requires adequate expected frequencies.
2. Sensitive to sample size.
ANOVA
1. Indicates only that differences exist.
2. Additional tests are needed to identify which groups differ.
Regression and Correlation

Correlation is a statistical measure that describes the degree and direction

of relationship between two or more variables.
It indicates how changes in one variable are associated with changes in
another variable.
For example:
• As study time increases, examination scores may increase.
• As product price increases, demand may decrease.
Correlation helps quantify such relationships.
Explanation
Correlation measures the strength and direction of association between
variables.
If two variables tend to change together, they are said to be correlated.
The correlation does not necessarily imply that one variable causes the
other; it only indicates the existence of a relationship.
Types of Correlation
1. Positive Correlation
When both variables move in the same direction.
Characteristics:
• Increase in one variable leads to an increase in the other.
• Decrease in one variable leads to a decrease in the other.
Examples:
• Height and weight
• Income and expenditure
Graphical Representation
The data points generally form an upward trend from left to right.
2. Negative Correlation
When variables move in opposite directions.
Characteristics:
• Increase in one variable leads to a decrease in the other.
• Decrease in one variable leads to an increase in the other.
Examples:
• Price and demand
• Speed and travel time
Graphical Representation
The data points generally form a downward trend from left to right.
3. Zero Correlation
When no relationship exists between the variables.
Characteristics:
• Changes in one variable do not affect the other.
• Data points appear randomly scattered.
Example:
• Shoe size and intelligence
Degree of Correlation
Perfect Positive Correlation
𝑟 = +1

Characteristics:
• All observations lie exactly on an upward-sloping straight line.
• Strongest possible positive relationship.
Perfect Negative Correlation
𝑟 = −1
Characteristics:
• All observations lie exactly on a downward-sloping straight line.
• Strongest possible negative relationship.
No Correlation
𝑟=0

Characteristics:
• No linear relationship between variables.
Correlation Coefficient
The Correlation Coefficient is a numerical measure of correlation.
The most commonly used measure is Pearson's Correlation Coefficient.
Formula
∑(𝑥 − 𝑥ˉ)(𝑦 − 𝑦ˉ)
𝑟=
√∑(𝑥 − 𝑥ˉ)2 ∑(𝑦 − 𝑦ˉ)2
where:
• 𝑟= Correlation coefficient
• 𝑥, 𝑦= observations
• 𝑥ˉ, 𝑦ˉ= means of the variables
Range of Correlation Coefficient
−1 ≤ 𝑟 ≤ +1
Interpretation
Value of 𝑟 Interpretation

+1 Perfect positive correlation

0.7to 0.99 Strong positive correlation

0.3to 0.69 Moderate positive correlation

0.01to 0.29 Weak positive correlation

0 No correlation

−0.01to −0.29 Weak negative correlation

−0.3to −0.69 Moderate negative correlation

−0.7to −0.99 Strong negative correlation

−1 Perfect negative correlation

Properties of Correlation
1. Correlation coefficient lies between −1and +1.
2. It measures the strength and direction of a linear relationship.
3. It is unit-free.
4. It is unaffected by change of origin and scale.
5. Correlation does not imply causation.
Scatter Diagram
A scatter diagram is a graphical method used to study correlation
between two variables.
Each observation is represented by a point.
Interpretation
• Upward trend → Positive correlation
• Downward trend → Negative correlation
• Random pattern → No correlation
Applications of Correlation
1. Business and Economics
• Price and demand analysis
• Sales forecasting
2. Education
• Relationship between study time and marks
3. Medicine
• Relationship between dosage and recovery
4. Engineering
• Relationship between temperature and performance
5. Machine Learning and Data Analysis
• Feature selection
• Pattern recognition
• Predictive modeling
Advantages
1. Measures strength of relationship.
2. Easy to interpret.
3. Useful for prediction and analysis.
4. Forms the basis of regression analysis.
Limitations
1. Correlation does not establish cause-and-effect relationships.
2. Sensitive to outliers.
3. Measures only linear relationships.
4. High correlation may sometimes be misleading.
 Karl Pearson Coefficient of Correlation

The Karl Pearson Coefficient of Correlation (also called Pearson's

Correlation Coefficient) is a statistical measure that quantifies the strength
and direction of the linear relationship between two variables.
It is denoted by: 𝑟
and takes values between:
−1 ≤ 𝑟 ≤ +1
Explanation
Karl Pearson's coefficient measures how closely two variables are linearly
related.
• Positive values indicate a positive relationship.
• Negative values indicate a negative relationship.
• Zero indicates no linear relationship.
The magnitude of 𝑟indicates the strength of the relationship, while the
sign indicates its direction.
Formula
The Pearson correlation coefficient is given by
∑(𝑥 − 𝑥ˉ)(𝑦 − 𝑦ˉ)
𝑟=
√∑(𝑥 − 𝑥ˉ)2 ∑(𝑦 − 𝑦ˉ)2
where:
• 𝑟= Pearson correlation coefficient
• 𝑥= values of variable 𝑋
• 𝑦= values of variable 𝑌
• 𝑥ˉ= mean of 𝑋
• 𝑦ˉ= mean of 𝑌
Computational Formula
For practical calculations,
𝑛∑𝑥𝑦 − (∑𝑥)(∑𝑦)
𝑟=
√[𝑛∑𝑥 2 − (∑𝑥)2 ][𝑛∑𝑦 2 − (∑𝑦)2 ]
where:
• 𝑛= number of observations
Interpretation of Correlation Coefficient
Value of 𝑟 Interpretation
+1 Perfect positive correlation
0.75to 0.99 Very strong positive correlation
0.50to 0.74 Moderate positive correlation
0.25to 0.49 Weak positive correlation
0 No correlation
−0.25to −0.49 Weak negative correlation
−0.50to −0.74 Moderate negative correlation
−0.75to −0.99 Very strong negative correlation
−1 Perfect negative correlation

Example
Consider the following data:
X Y
1 2
2 4
3 6
4 8
5 10
Step 1: Calculate Means
𝑥ˉ = 3
𝑦ˉ = 6

Step 2: Compute Required Quantities

X Y 𝑋 − 𝑥ˉ 𝑌 − 𝑦ˉ Product

1 2 -2 -4 8

2 4 -1 -2 2

3 6 0 0 0

4 8 1 2 2

5 10 2 4 8

∑(𝑋 − 𝑥ˉ)(𝑌 − 𝑦ˉ) = 20

∑(𝑋 − 𝑥ˉ)2 = 10
∑(𝑌 − 𝑦ˉ)2 = 40

Step 3: Apply Formula

20
𝑟=
√10 × 40
20
=
20
=1
Therefore,
𝑟=1
This indicates a perfect positive correlation.
Properties of Karl Pearson's Coefficient
1. Range
−1 ≤ 𝑟 ≤ +1

2. Unit-Free Measure
The coefficient is independent of measurement units.
3. Symmetry
𝑟𝑥𝑦 = 𝑟𝑦𝑥

Interchanging variables does not change the value.

4. Measures Linear Relationship
It measures only linear association between variables.
5. Independent of Change of Origin and Scale
The value remains unchanged under linear transformations.
Advantages
1. Simple and widely used.
2. Provides both direction and strength of relationship.
3. Uses all observations.
4. Forms the basis of regression analysis.
Limitations
1. Measures only linear relationships.
2. Sensitive to outliers.
3. Correlation does not imply causation.
4. May be misleading when the relationship is nonlinear.
Applications
1. Economics
• Income and expenditure analysis.
• Price and demand studies.
2. Education
• Relationship between study hours and examination scores.
3. Business
• Advertising expenditure and sales.
4. Engineering
• Performance analysis of systems.
5. Machine Learning and Data Science
• Feature selection.
• Exploratory data analysis.
• Predictive modeling.

Spearman's Rank Correlation Coefficient

The Spearman's Rank Correlation Coefficient is a non-parametric measure

of correlation that assesses the strength and direction of the relationship
between two variables based on their ranks rather than their actual
values.
It is denoted by: 𝜌 Or 𝑟𝑠
and its value lies between:
−1 ≤ 𝑟𝑠 ≤ +1
Explanation
In many situations, exact numerical values may not be available, but
observations can be ranked.
Spearman's rank correlation measures how closely the rankings of two
variables correspond.
It is particularly useful when:
• Data is ordinal.
• The relationship is not necessarily linear.
• Actual measurements are difficult to obtain.
Formula
When there are no tied ranks, Spearman's rank correlation coefficient is
given by
6∑𝑑 2
𝑟𝑠 = 1 −
𝑛(𝑛2 − 1)
where:
• 𝑟𝑠 = Spearman's rank correlation coefficient
• 𝑑= Difference between paired ranks
• 𝑛= Number of observations
• ∑𝑑 2 = Sum of squares of rank differences
Procedure for Calculation
Step 1
Assign ranks to the observations of both variables.
Step 2
Calculate the difference between corresponding ranks.
𝑑 = 𝑅𝑋 − 𝑅𝑌
Step 3
Find 𝑑 2 for each observation.
Step 4
Calculate : ∑𝑑 2
Step 5
Substitute into Spearman's formula.
Example
Consider the following rankings of five students:
Student Rank in Mathematics (𝑅𝑋 ) Rank in Science (𝑅𝑌 )
A 1 2
B 2 1
C 3 4
D 4 3
E 5 5

Step 1: Calculate Rank Differences

Student 𝑅𝑋 𝑅𝑌 𝑑 𝑑2
A 1 2 -1 1
B 2 1 1 1
C 3 4 -1 1
D 4 3 1 1
E 5 5 0 0

∑𝑑 2 = 4
Step 2: Apply Formula
6(4)
𝑟𝑠 = 1 −
5(52 − 1)

24
=1−
120

= 1 − 0.2

= 0.8

Therefore,
𝑟𝑠 = 0.8
This indicates a strong positive correlation.
Interpretation of Spearman's Coefficient
Value of 𝑟𝑠 Interpretation
+1 Perfect positive rank correlation
0.75to 0.99 Very strong positive correlation
0.50to 0.74 Moderate positive correlation
0.25to 0.49 Weak positive correlation
0 No correlation
−0.25to −0.49 Weak negative correlation
−0.50to −0.74 Moderate negative correlation
−0.75to −0.99 Very strong negative correlation
−1 Perfect negative rank correlation
Special Cases
Perfect Positive Correlation
When the ranks are identical:
𝑑=0
for all observations.
Hence,
𝑟𝑠 = +1
Perfect Negative Correlation
When one ranking is the exact reverse of the other.
𝑟𝑠 = −1
No Correlation
When rankings show no systematic relationship.
𝑟𝑠 ≈ 0
Advantages
1. Easy to calculate.
2. Suitable for ordinal data.
3. Does not require normal distribution.
4. Less affected by extreme values.
5. Can measure monotonic relationships.
Limitations
1. Uses ranks instead of actual values.
2. Less precise than Pearson's correlation for linear relationships.
3. Becomes more complicated when many tied ranks exist.
Difference Between Pearson and Spearman Correlation
Feature Pearson Correlation Spearman Correlation

Data Type Numerical Data Ranked Data

Relationship Measured Linear Monotonic

Based On Actual Values Ranks

Distribution Usually Normal No Normality

Requirement Data Required

Sensitivity to Outliers More Sensitive Less Sensitive

Applications
1. Education
Comparing student rankings in different subjects.
2. Psychology
Ranking behavioral characteristics.
3. Market Research
Comparing consumer preference rankings.
4. Sports
Comparing player rankings from different tournaments.
5. Social Sciences
Analyzing survey and opinion data.
 Linear Regression

Linear Regression is a statistical technique used to study the relationship

between a dependent variable and one or more independent variables by
fitting a linear equation to the observed data.
It is used to predict the value of one variable based on the value of
another variable.
Explanation
Regression analysis helps determine how a dependent variable changes
when one or more independent variables change.
For example:
• Predicting examination marks from study hours.
• Predicting sales from advertising expenditure.
• Predicting house prices from area.
In linear regression, the relationship between variables is represented by a
straight line.
Variables in Linear Regression
Independent Variable
The variable used to predict another variable.
Usually denoted by: 𝑋
Examples:
• Study hours
• Advertising expenditure
• Temperature
Dependent Variable
The variable being predicted.
Usually denoted by: 𝑌
Examples:
• Examination marks
• Sales
• Energy consumption
Types of Regression
1. Simple Linear Regression
One independent variable.
𝑌 = 𝑎 + 𝑏𝑋
2. Multiple Linear Regression
More than one independent variable.
𝑌 = 𝑎 + 𝑏1 𝑋1 + 𝑏2 𝑋2 + ⋯ + 𝑏𝑛 𝑋𝑛

Least Squares Method

The Least Squares Method is a mathematical technique used to determine

the best-fitting line or curve for a set of observed data by minimizing the
sum of the squares of the errors (residuals) between observed and
predicted values.
It is the most commonly used method for estimating the parameters of a
regression model.
Explanation
When a regression line is fitted to data, the predicted values may not
exactly match the observed values.
The difference between an observed value and its corresponding
predicted value is called the residual or error.
𝑒 = 𝑌 − 𝑌̂
where:
• 𝑌= Observed value
• 𝑌̂= Predicted value
• 𝑒= Error (Residual)
The Least Squares Method finds the regression line that minimizes the
total squared error.
Why Squared Errors are Used
If errors were added directly, positive and negative values could cancel
each other.
Squaring the errors:
1. Eliminates negative signs.
2. Gives greater importance to large errors.
3. Produces a unique best-fitting line.
Principle of Least Squares
The best regression line is obtained by minimizing
∑(𝑌 − 𝑌̂)2
This quantity is called the Sum of Squared Errors (SSE).
Regression Line
The equation of the simple linear regression line is
𝑌̂ = 𝑎 + 𝑏𝑋
where:
• 𝑌̂= Predicted value
• 𝑋= Independent variable
• 𝑎= Intercept
• 𝑏= Slope (Regression coefficient)
Determination of Regression Coefficients
Slope
The slope is calculated as
∑(𝑥 − 𝑥ˉ)(𝑦 − 𝑦ˉ)
𝑏=
∑(𝑥 − 𝑥ˉ)2
where:
• 𝑥ˉ= Mean of 𝑋
• 𝑦ˉ= Mean of 𝑌
Intercept
The intercept is obtained using
𝑎 = 𝑦ˉ − 𝑏𝑥ˉ
Normal Equations
The Least Squares Method leads to two normal equations:
∑𝑌 = 𝑛𝑎 + 𝑏∑𝑋
and
∑𝑋𝑌 = 𝑎∑𝑋 + 𝑏∑𝑋 2
Solving these equations gives the values of 𝑎and 𝑏.
Example
Consider the data:
X Y
1 2
2 4
3 5
4 4
5 5

Step 1: Calculate Means

𝑥ˉ = 3
𝑦ˉ = 4
Step 2: Calculate Slope
∑(𝑥 − 𝑥ˉ)(𝑦 − 𝑦ˉ) 6
𝑏= 2
= = 0.6
∑(𝑥 − 𝑥ˉ) 10
Step 3: Calculate Intercept
𝑎 = 4 − (0.6)(3) = 2.2
Step 4: Regression Equation
𝑌̂ = 2.2 + 0.6𝑋
This is the least-squares regression line.
Residuals
A residual is the difference between the observed value and the predicted
value.
𝑒 = 𝑌 − 𝑌̂
Interpretation
• Positive residual: observed value is above the regression line.
• Negative residual: observed value is below the regression line.
Properties of Least Squares Regression Line
1. Minimum Error Property
It minimizes the sum of squared residuals.
2. Passes Through the Mean Point
The regression line always passes through
(𝑥ˉ , 𝑦ˉ)
3. Unique Solution
For a given dataset, a unique least-squares line exists.
4. Residual Sum Property
The sum of residuals is zero.
∑𝑒 = 0
Applications
1. Business Forecasting
Predicting sales and profits.
2. Economics
Studying relationships between economic variables.
3. Engineering
Modeling system performance.
4. Scientific Research
Analyzing experimental data.
5. Machine Learning
Training predictive models and fitting linear regressions.
Advantages
1. Simple and widely used.
2. Produces the best linear fit.
3. Easy to compute.
4. Provides reliable parameter estimates.
Limitations
1. Sensitive to outliers.
2. Assumes a linear relationship.
3. Requires certain statistical assumptions for accurate inference.
4. May not fit nonlinear data well.
Relationship with Correlation
For simple linear regression:
𝑅2 = 𝑟 2
where:
• 𝑅2 = Coefficient of determination
• 𝑟= Pearson correlation coefficient
This measures the proportion of variation explained by the regression
model.
 Simple Linear Regression

Simple Linear Regression is a statistical method used to study the

relationship between one dependent variable and one independent
variable by fitting a straight-line equation to the observed data.
It is used to predict the value of the dependent variable based on the
value of the independent variable.
Explanation
Many real-life situations involve relationships between two variables.
Examples:
• Study hours and examination marks
• Advertising expenditure and sales
• Temperature and electricity consumption
Simple linear regression helps determine the nature of this relationship
and provides a mathematical model for prediction.
Variables in Simple Linear Regression
Independent Variable
The variable used to explain or predict another variable.
Usually denoted by: 𝑋
Examples:
• Study hours
• Advertising expenditure
• Temperature
Dependent Variable
The variable whose value is predicted.
Usually denoted by: 𝑌
Examples:
• Marks obtained
• Sales revenue
• Energy consumption
Regression Equation
The simple linear regression model is
𝑌 = 𝑎 + 𝑏𝑋
where:
• 𝑌= Dependent variable
• 𝑋= Independent variable
• 𝑎= Intercept
• 𝑏= Regression coefficient (slope)
Meaning of Parameters
Intercept (𝒂)
The value of 𝑌 when 𝑋 = 0
It represents the point where the regression line intersects the Y-axis.
Slope (𝒃)
The amount by which 𝑌changes for every one-unit increase in 𝑋.
Interpretation
• 𝑏 > 0: Positive relationship
• 𝑏 < 0: Negative relationship
• 𝑏 = 0: No linear relationship
Estimation of Regression Coefficients
Slope
The regression coefficient is
∑(𝑥 − 𝑥ˉ)(𝑦 − 𝑦ˉ)
𝑏=
∑(𝑥 − 𝑥ˉ)2
where:
• 𝑥ˉ= Mean of 𝑋
• 𝑦ˉ= Mean of 𝑌
Intercept
After obtaining 𝑏,
𝑎 = 𝑦ˉ − 𝑏𝑥ˉ
Regression Line
The fitted line is
𝑌̂ = 𝑎 + 𝑏𝑋
where:
• 𝑌̂= Predicted value of 𝑌
This equation is called the regression line of Y on X.
Example
Suppose the regression equation is
𝑌̂ = 15 + 4𝑋
where:
• 𝑋= Study hours
• 𝑌= Examination marks
If a student studies for
𝑋=8
hours,
then
𝑌̂ = 15 + 4(8)
𝑌̂ = 47
Therefore, the predicted score is 47 marks.
Assumptions of Simple Linear Regression
1. Linearity
A linear relationship exists between 𝑋and 𝑌.
2. Independence
Observations are independent.
3. Homoscedasticity
Variance of errors remains constant.
4. Normality of Errors
Residuals are approximately normally distributed.
5. Random Sampling
Data is collected through random sampling.
Residuals
The difference between observed and predicted values is called a residual.
𝑒 = 𝑌 − 𝑌̂
where:
• 𝑒= Residual
• 𝑌= Actual value
• 𝑌̂= Predicted value
Least Squares Criterion
The regression line is determined using the Least Squares Method.
The objective is to minimize
∑(𝑌 − 𝑌̂)2
This ensures the best-fitting line.
Coefficient of Determination
The coefficient of determination measures how much of the variation in
the dependent variable is explained by the regression model.
For simple linear regression,
𝑅2 = 𝑟 2
where:
• 𝑅2 = Coefficient of determination
• 𝑟= Pearson correlation coefficient
Interpretation of 𝑹𝟐
𝑅2 Value Interpretation
0 No explanatory power
0.25 Weak explanation
0.50 Moderate explanation
0.75 Strong explanation
1 Perfect explanation
Example:
𝑅2 = 0.81
means 81% of the variation in 𝑌is explained by 𝑋.
Applications
1. Business
• Sales forecasting
• Demand estimation
2. Economics
• Income and expenditure analysis
3. Education
• Relationship between study hours and marks
4. Engineering
• Performance prediction
5. Machine Learning
• Predictive modeling
• Data analysis
• Trend forecasting
Advantages
1. Simple and easy to understand.
2. Useful for prediction.
3. Quantifies relationships between variables.
4. Provides a foundation for advanced regression models.
Limitations
1. Assumes a linear relationship.
2. Sensitive to outliers.
3. Correlation does not imply causation.
4. May not perform well with nonlinear data.
Difference Between Correlation and Regression
Correlation Regression

Measures association Measures dependence

No prediction Used for prediction

Symmetric relationship Directional relationship

 Fitting of Polynomials and
Exponential Curves

Curve Fitting is the process of finding a mathematical equation that best

represents the relationship between variables in a dataset.
When the relationship is not adequately represented by a straight line,
polynomial or exponential curves can be fitted using the Least Squares
Method.
Polynomial Curve Fitting
A Polynomial Curve is a curve represented by a polynomial equation of
degree greater than one.
The general polynomial of degree 𝑛is:
𝑌 = 𝑎0 + 𝑎1 𝑋 + 𝑎2 𝑋 2 + ⋯ + 𝑎𝑛 𝑋 𝑛

where:
• 𝑌= Dependent variable
• 𝑋= Independent variable
• 𝑎0 , 𝑎1 , … , 𝑎𝑛 = Constants
Types of Polynomial Curves
1. Linear Polynomial (Degree 1)
𝑌 = 𝑎 + 𝑏𝑋
𝑎
𝑏
Produces a straight line.
2. Quadratic Polynomial (Degree 2)
𝑌 = 𝑎 + 𝑏𝑋 + 𝑐𝑋 2
Produces a parabolic curve.
3. Cubic Polynomial (Degree 3)
𝑌 = 𝑎 + 𝑏𝑋 + 𝑐𝑋 2 + 𝑑𝑋 3
Produces an S-shaped curve.
Principle of Polynomial Fitting
The coefficients are determined so that the sum of squared residuals is
minimized.
Residual:
𝑒 = 𝑌 − 𝑌̂
Objective:
∑(𝑌 − 𝑌̂)2
This is the Least Squares Criterion.
Quadratic Curve Fitting
For a second-degree polynomial:
𝑌 = 𝑎 + 𝑏𝑋 + 𝑐𝑋 2
the coefficients 𝑎, 𝑏, and 𝑐are obtained by solving the following normal
equations:
∑𝑌 = 𝑛𝑎 + 𝑏∑𝑋 + 𝑐∑𝑋 2

∑𝑋𝑌 = 𝑎∑𝑋 + 𝑏∑𝑋 2 + 𝑐∑𝑋 3

∑𝑋 2 𝑌 = 𝑎∑𝑋 2 + 𝑏∑𝑋 3 + 𝑐∑𝑋 4

Applications of Polynomial Fitting

1. Engineering design.
2. Growth trend analysis.
3. Economic forecasting.
4. Scientific experiments.
5. Machine learning and data modeling.
Exponential Curve Fitting
An Exponential Curve represents situations where the rate of change is
proportional to the current value.
Its general form is
𝑌 = 𝑎𝑒 𝑏𝑋
where:
• 𝑎= Initial value
• 𝑏= Growth or decay rate
• 𝑒= Euler's constant (2.71828)
Characteristics of Exponential Curves
Exponential Growth
When
𝑏>0
Examples:
• Population growth
• Compound interest
• Spread of information
Exponential Decay
When 𝑏 < 0
Examples:
• Radioactive decay
• Cooling of objects
• Depreciation of assets
Linearization of Exponential Curves
The exponential model is nonlinear.
Taking logarithms on both sides:
𝑌 = 𝑎𝑒 𝑏𝑋

ln 𝑌 = ln 𝑎 + 𝑏𝑋
Let
𝑍 = ln 𝑌
and
𝐴 = ln 𝑎
Then
𝑍 = 𝐴 + 𝑏𝑋
which is a linear equation.
Thus, ordinary linear regression techniques can be applied.

Steps for Exponential Curve Fitting

Step 1
Assume the model: 𝑌 = 𝑎𝑒 𝑏𝑋
Step 2
Take natural logarithms: ln 𝑌 = ln 𝑎 + 𝑏𝑋
Step 3
Transform: 𝑍 = ln 𝑌
Step 4
Fit the linear equation: 𝑍 = 𝐴 + 𝑏𝑋 ,using the Least Squares Method.
Step 5
Calculate: 𝑎 = 𝑒 𝐴
Step 6
Obtain the exponential equation: 𝑌 = 𝑎𝑒 𝑏𝑋
Example
Suppose the fitted linearized equation is:
𝑍 = 1.5 + 0.3𝑋
Since
𝐴 = 1.5
𝑎 = 𝑒 1.5
𝑎 ≈ 4.48
Therefore,
𝑌 = 4.48𝑒 0.3𝑋
This is the fitted exponential curve.
Comparison of Polynomial and Exponential Fitting

Feature Polynomial Fitting Exponential Fitting

Equation Form 𝑌 = 𝑎 + 𝑏𝑋 + 𝑐𝑋 2 + ⋯ 𝑌 = 𝑎𝑒 𝑏𝑋

Growth Varies with degree Constant percentage

Pattern growth/decay

Shape Flexible Rapid growth or decay

Linearization No Yes
Required

Applications General trend modeling Growth and decay processes

Advantages
Polynomial Fitting
1. Flexible for complex relationships.
2. Can model curved trends.
3. Widely used in data analysis.
Exponential Fitting
1. Suitable for growth and decay phenomena.
2. Useful in population and financial modeling.
3. Represents many natural processes accurately.
Limitations
Polynomial Fitting
1. High-degree polynomials may overfit data.
2. Predictions outside the data range may be unreliable.
Exponential Fitting
1. Assumes exponential behavior.
2. Sensitive to extreme observations.
Applications
Polynomial Curve Fitting
• Engineering analysis
• Economic forecasting
• Experimental sciences
• Machine learning
Exponential Curve Fitting
• Population growth
• Radioactive decay
• Biological growth
• Finance and compound interest
• Reliability studies