Statistical Concepts: Detailed Notes
Statistics is the science of collecting, organizing, analyzing, interpreting, and presenting data. In data analytics and
machine learning, statistical concepts help us understand patterns, make predictions, and take decisions from data.
Important concepts include sampling distributions, re-sampling methods, statistical inference, prediction error, and
regression modelling.
1. Sampling Distributions
Definition:
A sampling distribution is the probability distribution of a statistic obtained from many different samples drawn from the
same population. The statistic may be the sample mean, sample variance, or sample proportion.
Population and Sample:
• Population: Entire group of data or observations.
• Sample: Subset selected from the population.
Importance of Sampling Distribution:
• Helps estimate population parameters.
• Forms the basis of statistical inference.
• Helps calculate confidence intervals and hypothesis tests.
Central Limit Theorem (CLT):
The Central Limit Theorem states that when the sample size becomes large, the sampling distribution of the sample
mean approaches a normal distribution, regardless of the original population distribution.
Characteristics:
• Mean of sampling distribution = population mean.
• Standard deviation of sampling distribution is called standard error.
• Larger sample size reduces variability.
Standard Error Formula:
Standard Error = Population Standard Deviation / √n
Applications:
• Estimating average income of citizens.
• Election surveys and opinion polls.
• Medical research and quality testing.
2. Re-Sampling
Definition:
Re-sampling methods repeatedly draw samples from observed data and assess variability in a statistical model.
Main Types:
1. Bootstrap Method
2. Cross-Validation
Bootstrap Method:
Bootstrap sampling draws repeated samples with replacement from the original dataset.
Advantages:
• Useful when population distribution is unknown.
• Helps estimate confidence intervals.
• Easy to implement using computers.
Cross-Validation:
Cross-validation evaluates model performance by dividing the dataset into training and testing sets.
K-Fold Cross Validation:
• Data divided into K equal parts.
• One part used for testing and remaining for training.
• Process repeated K times.
Importance:
• Prevents overfitting.
• Improves model evaluation accuracy.
• Helps compare machine learning models.
3. Statistical Inference
Definition:
Statistical inference is the process of drawing conclusions about a population using sample data.
Main Components:
• Estimation
• Hypothesis Testing
Estimation:
Estimation predicts population parameters from sample statistics.
Types of Estimation:
• Point Estimation: Single value estimate.
• Interval Estimation: Range of values estimate.
Confidence Interval:
A confidence interval gives a range within which the population parameter is expected to lie.
Hypothesis Testing:
A method used to test assumptions regarding population parameters.
Important Terms:
• Null Hypothesis (H0)
• Alternative Hypothesis (H1)
• Significance Level (α)
• P-value
Steps in Hypothesis Testing:
1. State hypotheses.
2. Select significance level.
3. Compute test statistic.
4. Find p-value.
5. Accept or reject H0.
Applications:
• Medical drug testing.
• Industrial quality control.
• Social science research.
4. Prediction Error
Definition:
Prediction error measures how far predicted values are from actual values.
Importance:
• Evaluates machine learning models.
• Measures model accuracy.
• Helps improve predictive performance.
Types of Errors:
• Training Error
• Test Error
Bias-Variance Tradeoff:
• High Bias: Underfitting problem.
• High Variance: Overfitting problem.
Common Error Metrics:
1. Mean Absolute Error (MAE)
2. Mean Squared Error (MSE)
3. Root Mean Squared Error (RMSE)
4. R-squared value
Mean Squared Error:
Average squared difference between predicted and actual values.
Applications:
• Sales forecasting.
• Weather prediction.
• Stock market prediction.
5. Regression Modelling
Definition:
Regression modelling is a statistical technique used to study the relationship between dependent and independent
variables.
Objectives:
• Predict future values.
• Understand variable relationships.
• Identify important predictors.
Types of Regression:
1. Simple Linear Regression
2. Multiple Linear Regression
3. Logistic Regression
Simple Linear Regression:
Studies relationship between one independent variable and one dependent variable.
Equation:
Y = a + bX
Where:
• Y = dependent variable
• X = independent variable
• a = intercept
• b = slope coefficient
Multiple Linear Regression:
Uses more than one independent variable.
Assumptions of Regression:
• Linear relationship
• Independence of errors
• Homoscedasticity
• Normality of residuals
Advantages:
• Easy interpretation.
• Useful for forecasting.
• Identifies trends and patterns.
Applications:
• House price prediction.
• Demand forecasting.
• Risk analysis.
• Business analytics.
Conclusion:
Statistical concepts form the foundation of modern data analysis and machine learning. Sampling distributions help
understand variability, re-sampling improves model reliability, statistical inference supports decision-making,
prediction error evaluates performance, and regression modelling predicts relationships among variables. Together,
these techniques are essential in research, business analytics, economics, healthcare, and artificial intelligence.