BAYESIAN
INFERENCE
- MOHAMED KHALIDH M
BAYESIAN INFERENCE IN
PROBABILITY REASONING
• Bayesian inference is a powerful method of reasoning under uncertainty,
rooted in Bayes' Theorem, which provides a way to update the probability
estimate for a hypothesis as more evidence or information becomes available.
BAYES' THEOREM IS STATED AS:
Where:
• P(H∣E)P(H|E)P(H∣E) is the posterior probability, which is the probability of the hypothesis HHH being true
after observing evidence EEE.
• P(E∣H)P(E|H)P(E∣H) is the likelihood, the probability of observing the evidence EEE given the hypothesis
HHH.
• P(H)P(H)P(H) is the prior probability, which is the initial belief about the hypothesis before observing the
evidence.
• P(E)P(E)P(E) is the marginal likelihood or the probability of observing the evidence, which acts as a
normalizing constant.
IN PRACTICE:
• Prior: You start with an initial belief or assumption about the world. For instance, if you're trying to
predict whether it will rain, your prior might be based on past weather patterns.
• Likelihood: You update this belief with new data or evidence. For example, you might see that the sky is
cloudy, which increases the likelihood of rain.
• Posterior: After incorporating the new evidence, the updated belief is the posterior probability, which
combines the prior with the likelihood.
SIMPLE DEFINITION
• Bayesian Inference is a way to update our beliefs (or predictions) based on new evidence or
data. The key idea is that as we get more information, we adjust our beliefs.
• It's like having an initial guess about something, and then adjusting that guess as you get new
facts or observations.
REAL-TIME EXAMPLE: DECIDING IF IT WILL
RAIN
• Imagine you want to decide whether you should take an
umbrella today based on whether it will rain. Here's how
Bayesian Inference would work in this situation.
STEP 1: THE PRIOR (YOUR INITIAL
GUESS)
• Before looking out the window or checking the weather forecast, you might have a general
belief about whether it rains often. Let's say based on past experience, you believe it rains
about 30% of the time. This is your prior belief.
• So, you start with:
• Prior probability of rain = 30% (it rains 30% of the time).
STEP 2: NEW EVIDENCE (CHECK THE
CLOUDS)
• Now, you look outside and notice the sky is cloudy. This is new evidence.
From past experience, you know that when it’s cloudy, there is a 70% chance
that it will rain. This is your likelihood.
• Likelihood of rain given clouds = 70%.
STEP 3: CALCULATE THE POSTERIOR
(UPDATED BELIEF)
• Now, you want to combine your prior belief (it rains 30% of the time) with this
new evidence (cloudy skies make it 70% likely to rain).
UPDATED PROBABILITY (CALLED THE
POSTERIOR)
• P(Clouds∣Rain) is the likelihood of clouds given that it rains (which is 70% in
this case).
• P(Rain) is your prior belief about rain (30%).
• P(Clouds) is the overall chance of clouds, but we don’t need to calculate that
because we just want the updated probability.
STEP 4: UPDATE THE BELIEF
• Prior (Rain) = 30% or 0.30
• Likelihood (Clouds | Rain) = 70% or 0.70
• Now, we calculate the posterior:
• If we assume clouds happen 50% of the time in general (without considering rain), then:
• P(Clouds) = 50% or 0.50.
• So, the updated probability is:
CONCLUSION
• After seeing that the sky is cloudy, you now believe there is a 42% chance it
will rain today, which is higher than the original 30% chance you had before
you looked outside.
• This updated probability (42%) is your posterior belief about rain, based on
both your prior experience and the new evidence (cloudy skies).
KEY ADVANTAGES OF
BAYESIAN INFERENCE:
1. Incorporates prior knowledge: You can start with prior information or beliefs and
refine them as more data becomes available.
2. Handles uncertainty: It provides a framework for quantifying uncertainty, both in
terms of model parameters and predictions.
3. Adaptability: As new data comes in, the system adapts and updates its
understanding.
Bayesian inference is a foundational tool for reasoning under uncertainty, allowing AI
systems to make informed decisions even when the available information is incomplete
or uncertain. It is widely applied in machine learning, decision theory, robotics, natural
language processing, and more.