BAYESIAN MODELING

EMPIRICAL VS THEORETICAL PROBABILITY

 Empirical Probability - Empirical probability, also known as experimental probability, is a
probability measure that is based on observed data, rather than theoretical assumptions.
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑖𝑚𝑒𝑠 𝑎 𝑝𝑎𝑟𝑡𝑖𝑐𝑢𝑙𝑎𝑟 𝑒𝑣𝑒𝑛𝑡 𝑜𝑐𝑐𝑢𝑟𝑠
𝐸𝑚𝑝𝑖𝑟𝑖𝑐𝑎𝑙 𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝐸𝑣𝑒𝑛𝑡 𝐴 =
𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑟𝑖𝑎𝑙𝑠.
 Suppose that, in our 100 tosses, we get heads 55 times and tails 45 times. What is the empirical probability of
getting a head
 Let's say we have a bag with 50 marbles. Out of these 50 marbles, 20 are red, 15 are blue, and 15 are green. We
start to draw marbles one at a time, replacing the marble back into the bag after each draw. After 200 draws, we
find that we've drawn a red marble 80 times, a blue marble 70 times, and a green marble 50 times. What is the
empirical probability of getting a red marble?
 Theoretical Probability- Theoretical (or classical) probability is used when each outcome in a
sample space is equally likely to occur. If we denote an event of interest as Event A, we
calculate the theoretical probability of that event as:
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝐹𝑎𝑣𝑜𝑢𝑟𝑎𝑏𝑙𝑒 𝑂𝑢𝑡𝑐𝑜𝑚𝑒𝑠 (𝑖. 𝑒, 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝐸𝑣𝑒𝑛𝑡 𝐴)
𝑇ℎ𝑒𝑜𝑟𝑒𝑡𝑖𝑐𝑎𝑙 𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝐸𝑣𝑒𝑛𝑡 𝐴 =
𝑇𝑜𝑡𝑎𝑙 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑂𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑆𝑎𝑚𝑝𝑙𝑒 𝑆𝑝𝑎𝑐𝑒
 Consider a scenario of tossing a fair coin 3 times. Find the probability of getting exactly 2 heads.
 Consider a scenario of rolling 2 dice. What is the probability of getting a sum = 7
RANDOM VARIABLE
 A random variable is a function that maps the outcomes of a random process
(known as the sample space) to a set of real numbers.
 Input: The input to the function is an outcome from the sample space of a random process.
 Output: The output of the function is a real number that we assign to each possible outcome.
 The transformation from input to output in the function of a random variable is
determined by how we choose to define the random variable.
 The choice of how to define a random variable often depends on the specific
aspects of the random process (or event) that we're interested in studying.
 Let 𝑆 be the sample space (all possible outcomes of an experiment).
A random variable 𝑋 is a function:
𝑋: 𝑆 → ℝ
that maps each outcome 𝑠 ∈ 𝑆 to a real number.
RANDOM VARIABLE
 Types of Random Variables
(a) Discrete Random Variable
 Takes countable values (finite or infinite but countable).
 Example:
 Toss 2 coins → outcomes 𝑆 = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}.
 Define 𝑋 = Number of heads.
 Then possible values: 𝑋 ∈ {0,1,2}.
(b) Continuous Random Variable
 Takes values from an interval (uncountably infinite).
 Example:
 Let 𝑌 = time taken by a person to run 100m.
 Possible values: 𝑌 ∈ [9, 20]seconds.
PROBABILITY DISTRIBUTION OF A RANDOM VARIABLE
 A probability distribution is a list of all of the possible outcomes of a random variable along with their
corresponding probability values. In other words, it tells us the likelihood of different outcomes of a random
variable.
 It provides a mathematical relationship between the all possible outcomes of random variable and probability (or
likelihood) of getting that outcome.
 Example
X 1 0
Toss a coin- S: {H,T}→ RV: X = {1, 0} P(x)
▪
½ ½
X 1 2 3 4 5 6
▪ Rolling 1 dice- S: {1,2,3,4,5,6} → RV: X = {1, 2, 3, 4, 5, 6} P(x) 1 1 1 1 1 1
6 6 6 6 6 6
 Rolling 2 dice- S: {36 items} → RV: X = {2, 3, 4, 5, 6, 7, 8, 9,10, 11, 12}
X 2 3 4 5 6 7 8 9 10 11 12
P(x) 1 1 1 1 5 1 5 1 1 1 1
36 18 12 9 36 6 36 9 12 18 36
PROBABILITY DISTRIBUTION FUNCTION
 Probability distribution function provides a mathematical relationship between the all possible outcomes of
random variable and probability (or likelihood) of getting that outcome.

 Discrete Probability Distribution

 Applies to discrete random variables (countable outcomes).
 Defined using a Probability Mass Function (PMF). The PMF of a discrete random variable 𝑋 gives the probability
that 𝑋 takes a specific value 𝑥 .
𝑃 𝑋=𝑥 =𝑓 𝑥
 Properties:
 0≤𝑃 𝑋=𝑥 ≤1
 σ𝑥 𝑃 𝑋 = 𝑥 = 1
 Example: Binomial Distribution (number of successes in 𝑛trials)
 Random variable 𝑋 ∈ {1,2,3 … 𝑛}:
𝑛 𝑘
𝑃 𝑋=𝑘 = 𝑝 ቀ1 − 𝑝)𝑛−𝑘
𝑘
 𝑛 =number of trials
 𝑘= number of successes
 𝑝= probability of success in each trial
PROBABILITY DISTRIBUTION OF A RANDOM VARIABLE
 Continuous Probability Distribution
 Applies to continuous random variables (uncountably many outcomes).
 Defined using a Probability Density Function (PDF).

 Properties:
 𝑓 𝑥 ≥ 0 for all 𝑥.
∞
 ‫׬‬−∞ 𝑓 𝑥 𝑑𝑥 = 1.
 The probability of 𝑋 lying between two points is area under curve:
𝑏
𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = න 𝑓 𝑥 𝑑𝑥
𝑎

(Note: 𝑃 𝑋 = 𝑥 = 0, since a point has no "area")

 Example: Normal Distribution (Bell Curve)
 Random variable 𝑋 = height of students.
(𝑥−𝜇)2
1 −
 PDF:𝑓 𝑥 = 𝑒 2𝜎2
2𝜋𝜎 2
 Parameters:
 𝜇 = mean (center of curve)
 𝜎 2 = variance(spread of curve)
MEAN OF A RANDOM VARIABLE
 The mean of a random variable, often called the expected value 𝐸 𝑋 𝑜𝑟 𝜇, is the average
outcome of a random event that is repeated many times.
 More technically, it's a weighted average of the possible outcomes of the random variable,
where each outcome is weighted by its probability of occurrence.
 For Discrete Random Variable: Multiply each possible value by its probability, then sum.
𝐸 𝑋 = ෍ 𝑥𝑖 ⋅ 𝑃 𝑋 = 𝑥𝑖
𝑖

 For Continuous Random Variable: Multiply value of 𝑥 by probability density, then integrate.
∞
𝐸 𝑋 = න 𝑥 ⋅ 𝑓 𝑥 𝑑𝑥
−∞
VARIANCE OF A RANDOM VARIABLE
 The variance of a random variable is a statistical measurement that describes how much
individual observations in a group differ from the mean (expected value).
 Variance measures the spread or variability of a random variable around its mean.
𝑉𝑎𝑟 𝑋 = 𝐸 ቂ 𝑋 − 𝜇)2 = 𝐸[ 𝑋 − 𝐸 𝑋 )2 = 𝐸 𝑋 2 − (𝐸 𝑋 )2

 For Discrete Random Variable

𝑉𝑎𝑟 𝑋 = ෍(𝑋 − 𝜇)2 . 𝑃(𝑋 = 𝑥𝑖 )
𝑖

 For Continuous Random Variable

∞
𝑉𝑎𝑟(𝑋) = න (𝑥 − 𝜇)2 𝑓(𝑥)𝑑𝑥
−∞
QUESTION
1. A fair die is rolled. Let random variable 𝑋= outcome of the die. Find the mean (expected
value) and variance of 𝑋.
 Solution steps
 Values of 𝑋: 1,2,3,4,5,6
1
 Probabilities: 𝑃 𝑋 = 𝑥 = 6 each
1+2+3+4+5+6
𝐸 𝑋 = ෍𝑥 ⋅ 𝑃 𝑋 = 𝑥 = = 3.5
6
𝑉𝑎𝑟 𝑋 = 𝐸 𝑋 2 − ቀ𝐸 𝑋 )2
12 + 22 + 32 + 42 + 52 + 62 91
2
𝐸𝑋 = = ≈ 15.17
6 6
𝑉𝑎𝑟 𝑋 = 15.17 − ൫3.5)2 = 15.17 − 12.25 = 2.92

2. A random variable 𝑋 represents the number of heads obtained when tossing two fair coins.
 Write the probability distribution of 𝑋.
 Calculate the mean 𝐸 𝑋 .
 Calculate the variance Var 𝑋 .
QUESTION
1. A continuous random variable 𝑌has the probability density function (PDF):
2𝑦, 0 ≤ 𝑦 ≤ 1
𝑓 𝑦 =ቊ
0, otherwise
 Find the mean 𝐸 𝑌 .
 Find the variance Var 𝑌 .
 Solution:
1 1 1 31
2
2𝑦 2
𝐸 𝑌 = න 𝑦 ⋅ 𝑓 𝑦 𝑑𝑦 = න 𝑦 ⋅ 2𝑦 𝑑𝑦 = න 2 𝑦 𝑑𝑦 = =
0 0 0 3 0
3
1 1 1
2𝑦 4 1 1
𝐸 𝑌 2 = න 𝑦 2 ⋅ 𝑓 𝑦 𝑑𝑦 = න 𝑦 2 ⋅ 2𝑦 𝑑𝑦 = න 2 𝑦 3 𝑑𝑦 = =
0 0 0 4 0 2
2
2 2 1 2 1 4 9−8 1
𝑉𝑎𝑟 𝑌 = 𝐸 𝑌 − (𝐸 𝑌 ) = − = − = = ≈ 0.0556
2 3 2 9 18 18
TYPES OF PROBABILITY
 Probability is divided in three parts based on how multiple events are related.
 Marginal Probability
 Probability of a single event without considering others.
 Example: Probability a student passed = 𝑃 𝑃𝑎𝑠𝑠 .
 Joint Probability
 Probability that two (or more) events occur together.
 Example: Probability a student is male AND passed = 𝑃 𝑀𝑎𝑙𝑒, 𝑃𝑎𝑠𝑠 = 𝑃 𝑀𝑎𝑙𝑒
∗ 𝑃(𝑃𝑎𝑠𝑠).
 Conditional Probability
 Probability of an event given another event has already occurred.
 Example: Probability a student passed given the student is male =
𝑃(𝑃𝑎𝑠𝑠, 𝑀𝑎𝑙𝑒)
𝑃 𝑃𝑎𝑠𝑠 ∣ 𝑀𝑎𝑙𝑒 =
𝑃 𝑀𝑎𝑙𝑒
TYPES OF PROBABILITY
Joint Probability
 The probability that two (or more) events happen at the same time.
 For two events 𝐴and 𝐵: 𝑃 𝐴 ∩ 𝐵 or 𝑃(𝐴, 𝐵) means probability that both 𝐴and 𝐵occur.
 Continuous Case: If 𝑋, 𝑌are continuous random variables with joint density 𝑓 𝑥 𝑦 , then:
𝑃 𝑋 ∈ 𝐴𝑌 ∈ 𝐵 = ඵ 𝑓 (𝑥, 𝑦) 𝑑𝑥 𝑑𝑦
𝐴,𝐵
 Discrete Case: For two discrete random variables 𝑋and 𝑌, the joint probability is the probability that
both events occur at the same time.
𝑃 𝑋 = 𝑥𝑖 𝑌 = 𝑦𝑗 = Probability that 𝑋 takes value 𝑥𝑖 AND 𝑌 takes value 𝑦𝑗
Number of outcomes where 𝑋 = 𝑥𝑖 and 𝑌 = 𝑦𝑗
𝑃(X = 𝑥𝑖 , 𝑌 = 𝑦𝑗 ) =
Total number of outcomes
 Example (Dice):
Let event 𝐴: “die shows an even number” = 2 4 6 .
Let event 𝐵: “die shows number greater than 3” = 4 5 6 .
 The intersection 𝐴 ∩ 𝐵 = {4,6}. So:
2 1
𝑃 𝐴𝐵 = =
6 3
TYPES OF PROBABILITY
 Marginal Probability
 The probability of a single event regardless of the occurrence of other events.
Obtained by summing (discrete) or integrating (continuous) the joint probability distribution.
 For events 𝐴 and 𝐵:
𝑃 𝐴 = ෍ 𝑃 (𝐴, 𝐵 = b) (discrete)
𝑏
or
𝑃 𝑋 = 𝑥 = න𝑓 (𝑥, 𝑦) 𝑑𝑦 (continuous)

 Example From the two-coin experiment:

 Joint probabilities: 𝑃 HH = 1/4, 𝑃 HT = 1/4, 𝑃 TH = 1/4, 𝑃 TT = 1/4
 Marginal probability of first coin being Head: 𝑃 first = H = 𝑃 HH + 𝑃 HT = 1/4 + 1/4
= 1/2
 So, marginal probability = "standalone" probability from the joint distribution.
TYPES OF PROBABILITY
Conditional Probability
 The probability of one event given that another event has already occurred. It is derived from joint
and marginal.
 For 𝐴and 𝐵:
𝑃 𝐴∩𝐵
𝑃 𝐴∣𝐵 = ,𝑃 𝐵 > 0
𝑃 𝐵
 We restrict the sample space to event 𝐵, then measure the chance of 𝐴.
 Example (Dice):
Event 𝐴: "die shows an even number" = {2,4,6}.
Event 𝐵: "die shows number greater than 3" = {4,5,6}.
 Joint probability: 𝑃(𝐴, 𝐵) = 2/6.
Marginal: 𝑃 𝐵 = 3/6 = 1/2.
So conditional probability: (given the die roll is greater than 3, the chance it is also even )
𝑃(𝐴, 𝐵) 2/6 2
𝑃 𝐴∣𝐵 = = =
𝑃 𝐵 3/6 3
 This means: given the die roll is greater than 3, the chance it is also even is 2/3.
TYPES OF PROBABILITY
Example: Two-Coin Toss
Outcome First Coin Second Coin Probability
HH H H 1/4
HT H T 1/4
TH T H 1/4
TT T T 1/4
 Joint: 𝑃(H,H)=1/4
 Marginal: 𝑃(first=H)=1/2
 Conditional: 𝑃(second=H ∣ first=H)=1/2
 Example: A card is drawn from a standard 52-card deck.
 Let 𝐴: the card is a heart.
 Let 𝐵: the card is a face card (Jack, Queen, King).
 Find:
 𝑃 𝐴
 𝑃 𝐵
 P 𝐴∩𝐵
 𝑃 𝐵∣𝐴
BAYES THEOREM
 Bayes’ Theorem is a fundamental result in probability theory that describes the probability of an
event, based on prior knowledge of conditions related to the event.
 Bayes’ theorem is just a rearrangement of conditional probability.
 It updates the probability of a hypothesis (H) after obtaining new evidence (E).
𝑃 𝐸∣𝐻 𝑃 𝐻
𝑃 𝐻∣𝐸 =
𝑃 𝐸
 𝑃 𝐻 ∣ 𝐸 : Posterior probability → probability of hypothesis 𝐻 given evidence 𝐸.
 𝑃 𝐸 ∣ 𝐻 : Likelihood → probability of evidence 𝐸 if hypothesis 𝐻 is true.
 𝑃 𝐻 : Prior probability → probability of hypothesis before seeing the evidence.
 𝑃 𝐸 : Marginal probability of evidence → overall probability that evidence 𝐸 occurs.
 Bayes’ theorem is a way to update beliefs:
 Start with an initial belief (prior probability).
 Gather new data/evidence.
 Update the belief based on how consistent the evidence is with the hypothesis.
BAYES THEOREM EXAMPLE
 A certain disease affects 1 in 1,000 people. A diagnostic test for the disease is:
 98% sensitive (if a person has the disease, the test gives a positive result 98% of the time).
 95% specific (if a person does not have the disease, the test gives a negative result 95% of the time).
 If a randomly chosen person tests positive, what is the probability that they actually have the disease?
 Solution:
 Prior probability 𝑃 𝐷𝑖𝑠𝑒𝑎𝑠𝑒 = disease prevalence = 1/1000.
 No disease: 𝑃 No Disease = 1 − 0.001 = 0.999
 Sensitivity = 𝑃 Positive ∣ 𝐷𝑖𝑠𝑒𝑎𝑠𝑒 = 0.98
 Specificity = 𝑃 Negative ∣ No Disease → = 0.95 so 𝑃 Positive ∣ No Disease = 1 − 0.95 = 0.05.
 compute:
𝑃 Positive ∣ 𝐷 𝑃 𝐷
𝑃 𝐷 ∣ Positive =
𝑃 Positive
 Where,
𝑃 Positive = 𝑃 Positive ∣ 𝐷𝑖𝑠𝑒𝑎𝑠𝑒 𝑃 𝐷𝑖𝑠𝑒𝑎𝑠𝑒 + 𝑃 Positive ∣ No Disease 𝑃 No Disease = (0.98)(0.001)+(0.05)(0.999)
0.98×0.001 0.00098
𝑃 𝐷 ∣ Positive = =
(0.98)(0.001)+(0.05)(0.999) 0.05093
BAYESIAN INFERENCE
 Bayesian inference is a method of statistical inference where we update our beliefs about a
parameter or hypothesis based on new evidence or data.
 Statistical inference is the process of drawing conclusions or making predictions about a population
based on data from a sample.
 It is a way of updating our beliefs about something when we get new information.
 Belief before evidence → called Prior
 New evidence → what we observe
 Belief after evidence → called Posterior
 It relies on Bayes’ Theorem:
𝑃 E∣𝐻 𝑃 𝐻
𝑃 𝐻∣E =
𝑃 E
 𝐻 =hypothesis (what we think may be true)
 E = Evidence (what we observed)
 𝑃 𝐻 = Prior probability → (our belief before evidence)
 𝑃 E ∣ 𝐻 = Likelihood → probability of seeing the evidence if hypothesis is true
 𝑃 E = Marginal probability of Evidence → (Total probability of evidence)
 𝑃 𝐻 ∣ E = Posterior probability → (updated belief after evidence)
COMPONENTS OF BAYESIAN INFERENCE
 Bayesian inference is based on Bayes’ theorem for parameters:
𝑃 𝐷 ∣ 𝜃 ⋅𝑃 𝜃
𝑃 𝜃∣𝐷 =
𝑃 𝐷
 Prior Distribution (𝑃 𝜃
 Represents our belief about the parameter before seeing any data.
 Example: Believing a coin is fair (𝜃 = 0.5) before tossing.
 Can be informative (based on expert knowledge) or non-informative (uniform, meaning no strong
assumption).
 Likelihood (𝑃 𝐷 ∣ 𝜃 )
 Probability of observing the data given a particular parameter value.
 Captures how well the parameter explains the observed data.
 Example: If 𝜃 = 0.6, likelihood of getting 7 heads in 10 tosses comes from Binomial distribution.
COMPONENTS OF BAYESIAN INFERENCE
 Evidence / Marginal Likelihood (𝑃 𝐷
𝑃 𝐷 = න𝑃 𝐷 ∣ 𝜃 𝑃 𝜃 𝑑𝜃

 Normalizing constant that ensures the posterior is a valid probability distribution.

 It represents the overall probability of the observed data under all possible parameter values.
 Sometimes difficult to compute, which is why advanced methods (MCMC, variational inference) are
used.
 Posterior Distribution (𝑃 𝜃 ∣ 𝐷
 The updated belief about the parameter after considering the data.
 Combines prior and likelihood:
Posterior ∝ Likelihood × Prior
 Example: After 7 heads in 10 tosses, the posterior shifts toward 𝜃 > 0.5.
 Parameter(s) of Interest (𝜃)
 The unknown(s) we are estimating.
 Could be a single parameter (e.g., probability of heads) or multiple parameters (e.g., mean and
variance of a normal distribution).
STEPS IN BAYESIAN INFERENCE
 Start with Prior (𝑃 𝐻 → (initial belief.
 Collect Evidence (𝐸).
 Compute Likelihood (𝑃 𝐸 ∣ 𝐻 → (how strongly evidence supports hypothesis.
 Normalize with 𝑃 𝐸 →make probabilities valid.
 Update to Posterior (𝑃 𝐻 ∣ 𝐸 → (new belief.
EXAMPLE (COIN TOSS)
 Imagine we don’t know whether a coin is fair or biased towards heads.
 Hypothesis 1 (𝐻1 :)Coin is fair → 𝑃 𝐻1 = 0.5
 Hypothesis 2 (𝐻2 :)Coin is biased (always heads) → 𝑃 𝐻2 = 0.5

 Our Prior belief: we’re equally unsure.

 Now we toss the coin once, and it comes up Heads. This is our Evidence (E).
 Step 1: Likelihood
 If coin is fair )𝐻1 ): 𝐸 ∣ 𝐻1 = 0.5
 If coin is biased (𝐻2 ): 𝑃 𝐸 ∣ 𝐻2 = 1

 Step 2: Bayes’ Theorem

𝑃 𝐸∣𝐻1 ⋅𝑃 𝐻1
 For 𝐻1 : 𝑃 𝐻1 ∣ 𝐸 = 𝑃 𝐸
𝑃 𝐸∣𝐻2 ⋅𝑃 𝐻2
 For 𝐻2 :𝑃 𝐻2 ∣ 𝐸 = 𝑃 𝐸

 Step 3: Find 𝑃 𝐸 (total probability of seeing Heads)

 𝑃 𝐸 = 𝑃 𝐸 ∣ 𝐻1 𝑃 𝐻1 + 𝑃 𝐸 ∣ 𝐻2 𝑃 𝐻2 = 0.5 0.5 + 1 0.5 = 0.25 + 0.5 = 0.75
 So the evidence shifted our belief toward the coin being biased.
EXAMPLE
 Step 4: Compute Posterior
0.5⋅0.5 0.25
 For 𝐻1 : 𝑃 𝐻1 ∣ 𝐸 = = = 0.33
0.75 0.75
1⋅0.5 0.5
 For 𝐻2 : 𝑃 𝐻2 ∣ 𝐸 = = = 0.67
0.75 0.75

Interpretation
 Before tossing the coin: both hypotheses had equal probability (50%-50%).
 After seeing 1 Head:
 Belief in "coin is fair" → 33%
 Belief in "coin is biased" → 67%
BAYESIAN INFERENCE FOR PARAMETER ESTIMATION
 Bayes theorem is also used for estimating unknown parameters of a model based on data.
 Formula (General Form)
𝑃 𝐷 ∣ 𝜃 ⋅𝑃 𝜃
𝑃 𝜃∣𝐷 =
𝑃 𝐷
 Components
 𝜃: Parameter(s) of interest (parameter we want to estimate): Example: Mean of a normal distribution.
 𝐷: Data (observations from data): Example: Results of 10 coin tosses, measurements of weights, test scores, etc.
 𝑃 𝜃 : Prior Distribution (Our belief about the parameter before seeing the data) Example: Before tossing the coin, we
might assume it’s fair → Prior: 𝜃 = 0.5.
 𝑃 𝐷 ∣ 𝜃 : Likelihood (The probability of the data given parameter value 𝜃). Example: If coin bias is 0.5, likelihood of 7
heads in 10 tosses is computed from the binomial formula.
 𝑃 𝐷 :Evidence (Normalization Constant) Ensures the posterior is a valid probability distribution (Marginal Probability)
 Computed as:
𝑃 𝐷 = න𝑃 𝐷 ∣ 𝜃 𝑃 𝜃 𝑑𝜃
 𝑃 𝜃 ∣ 𝐷 :Posterior Distribution (Updated belief about the parameter after seeing data)
 This is the main output of Bayesian inference.
EXAMPLE (COIN TOSS)
 Suppose we want to estimate 𝜃, the probability of getting heads.
 Prior: Assume coin is fair → 𝑃 𝜃 = Uniform 0 1 .
 Data: Toss the coin 10 times, get 7 heads.
 Likelihood: From Binomial distribution:
10 7
𝑃 𝐷∣𝜃 = 𝜃 (1 − 𝜃)3
7
 Posterior:
𝑃 𝜃 ∣ 𝐷 ∝ 𝜃 7 (1 − 𝜃)3
 This is a Beta distribution: Beta(8,4)
 Interpretation:
 Before data → believed coin was equally likely biased anywhere.
 After data → posterior suggests coin is more likely biased towards heads (~0.67).