Advanced Statistics
Probability:
It is the measure of uncertainity
Uncertainity -> things may happen or may not happen
It always ranges from 0 to 1
It is a numerical value that expresses the likelihood of an event occurring.
Probability = no of favourable outcomes
total no of outcomes
Representation of Probability:
1. Decimal ( 0 , 0.25 , 0.5 , 0.75 , 1)
2. Ratio (0 , ¼ , ½ , ¾ ,1)
3. Percentages (0 , 25, 50 , 75 , 100)
Random Experiment:
An action whose output is not predictable.
Ex:
o Tossing coin
o Rolling Dice
Characteristics of R.E:
1. The outcome is unpredictable
2. There should be atleast 2 outcomes
3. The experiment can be performed “n” no of times
Sample Space:
It is the set of all Possible outcomes
Ex:
Tossing a coin -> R.E
{ H,T} -> S.S
Event:
Desired outcome which is part of S.S
Ex: Prob of getting Head when a coin is tossed
o Tossing a coin -> R.E
o { H,T} -> S.S
o {H} -> Event
Probability = no of favourable outcomes = Event
total no of outcomes S.S
Trail:
No of times a R.E is Performed
Outcome:
Output of a R.E
Types of Events:
1. Single Outcome
i) Impossible event
Probability of the event is 0
P(A) = 0
Event never happens
ii) Unlikely event
Probability of the event is in b/w 0 to 0.5
0 < P(A) < 0.5
Very less chances
iii) Even Chance event
Probability of the event is 0.5
P(A) = 0.5
Equal Chances
iv) Likely event
Probability of the event is in b/w 0.5 to 1
0.5 < P(A) < 1
More Chances
v) Sure event
Probability of the event is 1
P(A) = 1
Definitely occurs
2. Multiple Outcome
i) Independent event
Probability of an even is not affected by another event
Sampling with Replacement
A and B -> P(A ∩ B) = P(A) + P(B) – P(A U B)
A or B -> P(A U B) = P(A) + P(B) – P(A ∩ B)
ii) Dependent event
Probability of an even is dependent on another event
Sampling without Replacement
P(A / B) = P(B ∩ A) / P(B)
P(B / A) = P(A ∩ B) / P(A)
iii) Mutually Exclusive event
Two events will not occur at the same time
P(A ∩ B) = 0
P(A U B) = P(A) + P(B)
Types of Probabilities:
1. Marginal
Individual Probabilities
P(A) , P(B)
2. Joint
Combination of events
P(A U B) , P(A ∩ B)
3. Conditional
Probability based on another event
P(A / B) , P(B / A)
Bayes Theorum
Rules / Axioms of Probability:
1. Probability of an event ranges from 0 to 1
0 < P(E) < 1
2. Probability of Sample space is 1
3. If events e1, e2, e3 ……., en are mutually exclusive then
4. If A and B are compliment events
P(A) = 1 – P(B)
5. Sum of individual probabilities is always 1
Total outcomes:
Ex: Tossing a coin 3 times -> (2)^3 = 8
Random Variable:
It is a function which takes Sample space as input and results a real
number.
X : SS -> R
Ex:
R.E -> Tossing 3 coins
S.S -> {HHH, HHT, HTH, HTT
TTT, TTH, THT, THH}
R.V -> count of heads
X -> {3, 2, 2, 1, 0, 1, 1, 2}
Probability distribution table:
x 0 1 2 3
P(X = x) 1/8 3/8 3/8 1/8
Types of Random Variable:
1. Discrete R.V
2. Continous R.V
CDF (Cummulative Distribution Function):
It is used to find the prob upto a particular value
P(X < x)
PMF (Probability Mass Function):
It is used to find the prob of a specific value / exact value
P(X = x)
Expected Value:
It is similar to mean
Variance:
Distribution
It is defined as how total frequency values are distributed among the data
points
Why?
o To understand nature of data
o To estimate population data based on sample distribution
o Hypothesis Testing
o To conclude about the data
o Few alg expect the data should follow normal distribution
Types of Distribution:
1. Discrete Probability Distribution
a) Bernoulli
b) Binomial
c) Poison’s
d) Multinomial
e) Geometric
2. Continuous Probability Distribution
a) Uniform
b) Exponential
c) Normal (Gaussian)
a. Standard Normal
b. Log Normal
d) Pareto
Discrete Probability Distribution
The outcome of a R.E is Discrete (Countable)
Ex : Tossing a coin, Rolling a dice
Bernoulli Distribution
It models the R.V with 2 possible outcomes (Success, Failure) in one trail
If a R.V (X) follows Bernoulli distribution
X ~ B (1 , P)
PMF of Bernoulli Distribution is given by
CDF of Bernoulli Distribution is given by
0 if k < 0
P(x < k) = 0.5 if 0 < k < 1
1 if k > 1
Binomial Distribution
It models the R.V with 2 possible outcomes (Success, Failure) in ‘n’ trails
If a R.V (X) follows Binomial distribution
X ~ B (n , P)
The PMF of Binomial Distribution is given by
2. Binomial Distribution
It models the Random Variable with 2 possible outcomes in “n” trail.
Example: Counting how many heads you get when flipping a coin 10 times.
If a Random Variable follows Bernoulli Distribution it is given by
Where
X-> Random Variable
B-> Binomial distribution
P-> Probability of success
n-> No of trails
The PMF of Bernoulli Distribution is given by
The Mean and Variance of Binomial Distribution are expressed as
3. Multinomial Distribution
It models the Random Variable with more than 2 possible outcomes in “n” trails.
Example: Rolling a dice 10 times and recording how often each face appears.
If a Random Variable follows Bernoulli Distribution it is given by
Where
X-> Random Variable
M-> Multinomial distribution
p1,p2,p3,….,pn-> Probability of each outcome
n-> No of trails
The PMF of Bernoulli Distribution is given by
The Mean and Variance of Multinomial Distribution are expressed as
4. Poisson’s Distribution
It models the Random Variable in no of successes in average time rate.
Example: Number of emails received in an hour or calls arriving at a call centre per minute.
If a Random Variable follows Bernoulli Distribution it is given by
Where
X-> Random Variable
λ-> Average time rate
The PMF of Poisson’s Distribution is given by
The Mean and Variance of Multinomial Distribution are expressed as
5. Geometric Probability Distribution
It models the Random Variable to find the probability of the xth trail to get the first success.
Example: The number of attempts needed before getting the first head when flipping a coin.
If a Random Variable follows Bernoulli Distribution it is given by
Where
X-> Random Variable
p-> Probability of each outcome
The Mean and Variance of Multinomial Distribution are expressed as
6. Negative Binomial Distribution
It models the Random Variable to find the probability of nth trial to get k successes.
Example: How many basketball shots a player must take to make 5 successful baskets.
If a Random Variable follows Bernoulli Distribution it is given by
Where
X-> Random Variable
NB-> Ne distribution
p-> Probability of success in each trail
r-> No of required success
The PMF of Negative Binomial Distribution is given by
The Mean and Variance of Multinomial Distribution are expressed as
Continuous Probability Distribution
The outcome of a R.E is Ranges in a certain Interval (Countable)
Ex : Height or Weight of a person
Instead of finding the probability of one exact value, we look for
the probability that a value lies between two points.
(1) Uniform Distribution :
In a Uniform Distribution, every value within a specific range has an equal
probability of occurring.
It is the simplest continuous distribution, where all intervals of the same length
have the same likelihood.
Example :
If you pick a random number between 0 and 10, every number — 2.5, 5, or 9.8 —
is equally likely to appear.
If a random variable ‘X’ is uniformly distributed between ‘a’ and ‘b’:
Where :
X → Random variable
U → Uniform distribution
a → Minimum value (lower limit)
b → Maximum value (upper limit)
The PDF of Uniform Distribution is
The CDF of Uniform Distribution is
(2) Exponential Distribution :
The Exponential Distribution is used to model the time between independent
events that occur at a constant average rate.
It is a continuous probability distribution where shorter waiting times are more
likely than longer ones.
Example :
Imagine customers arriving at a coffee shop.
On average, one customer arrives every 5 minutes, but the actual times between
arrivals are random.
Sometimes it’s 2 minutes, sometimes 8 — yet on average, it’s 5 minutes.
This randomness in waiting time follows an Exponential Distribution — where
short waiting times are more frequent, and long waits are rare.
If a random variable ‘X’ follows an Exponential Distribution with rate
parameter λ :
Where :
X → Time between events (random variable)
λ → Average Rate(events per unit time)
1/λ → Mean or average waiting time
The PDF of Exponential Distribution is
The CDF of Exponential Distribution is
(3) Normal Distribution :
The Normal Distribution, also called the ‘Gaussian Distribution’, is one of the
most important continuous probability distributions in statistics.
It describes data that tends to cluster around a mean (average), with fewer
observations as you move away from the center.
Example :
Consider the heights of adults in a population.
Most people have a height near the average (mean) — say 170 cm.
Fewer people are much taller or shorter, and extremely tall or short individuals
are rare.
This pattern forms a bell-shaped curve, which is the hallmark of the Normal
Distribution.
If a random variable ‘X’ follows the Normal Distribution :
Press enter or click to view image in full size
Where :
μ → Mean
σ → Standard deviation
N → Normal Distribution
The PDF of Normal Distribution is
Significance :
Most real-world data follows a Normal Distribution.
Many naturally occurring datasets exhibit a bell-shaped pattern.
Examples:
Heights of students
pH values of liquids
Marks or test scores of students
Used to derive statistical inferences and assumptions.
The Central Limit Theorem (CLT) states that, regardless of the population
distribution, the sampling distribution of the mean tends to be normal as
sample size increases.
Many statistical tests assume normality, such as the Z-test and T-test.
Several Machine Learning algorithms perform best with normally
distributed data.
Algorithms that assume or benefit from normality include:
Linear Regression (assumes normally distributed residuals)
Gaussian Naïve Bayes (explicitly based on the normal distribution)
Characteristics :
The shape of the curve is bell-shaped and perfectly symmetric around the
mean (μ).
The mean, median, and mode are all equal.
The total area under the curve equals 1, representing total probability.
The distribution is fully defined by its mean (μ) and standard deviation (σ).
The skewness = 0 and kurtosis = 0
About 68% of values fall within μ±1σ, 95% within μ±2σ,
and 99.7% within μ±3σ — the Empirical Rule(68–95–99.7 Rule).