Bachelor of Science (Honours) in Data Science and Artificial Intelligence
DA109: AI Basics
Module 9
Uncertain Knowledge and Reasoning: Probabilistic Reasoning
Learning objective of Module 9
➢ Understand and apply Bayesian networks to represent and infer
probabilistic relationships, capturing uncertain knowledge efficiently
through defined syntax and semantics, and performing probabilistic
inference in practical situations.
2
Parts
➢ Representing Knowledge in an Uncertain Domain
➢ The Semantics of Bayesian Networks
➢ Efficient Representation of Conditional Distributions
➢ Exact Inference in Bayesian Networks
➢ Approximate Inference in Bayesian Networks
➢ Relational and First-Order Probability Models
➢ Other Approaches to Uncertain Reasoning
3
4
Part - I
Representing Knowledge in an Uncertain Domain
5
Importance of Representing Knowledge in an Uncertain Domain
➢ Real-World Complexity
➢ Improved Decision Making
➢ Risk Management
➢ Handling Incomplete Information
➢ Robustness to Variability
➢ Enhanced Learning and Adaptation
➢ Flexibility in Modeling
➢ Interdisciplinary Applications
➢ Facilitating Communication and Understanding
6
Conditionally Independent Variables
➢ Simple world consisting of the variables toothache, Cavity, Catch, and Weather.
➢ Weather is independent of the other variables.
➢ Toothache and Catch are conditionally independent, given Cavity.
A simple Bayesian network in which Weather is independent of the other three variables and
Toothache and Catch are conditionally independent, given Cavity
7
Conditionally Independent Variables
➢ Variables: Burglar Alarm (A), Earthquake (E), John calls (J), Mary calls (M), Burglary (B).
➢ Dependencies and Probabilities:
❑ Burglar Alarm (A): It reliably detects burglaries but sometimes triggers for earthquakes.
❑ Earthquake (E): Sometimes triggers the alarm.
❑ John calls (J): Nearly always calls when he hears the alarm, but sometimes confuses other sounds.
❑ Mary calls (M): Often misses the alarm due to loud music.
❑ Burglary (B): The event we want to estimate the probability of, given the evidence.
8
Conditionally Independent Variables
➢ Bayesian Network Representation: A typical Bayesian network for this scenario might have nodes for A,
E, J, M, and B, with edges representing dependencies:
❑ A -> B (Alarm being triggered influences the probability of a burglary).
❑ E -> A (Earthquake triggering the alarm).
❑ A -> J, A -> M (Alarm triggering influencing whether John or Mary calls).
9
10
Part - II
The Semantics of Bayesian Networks
11
The Semantics of Bayesian Networks
➢ What a network is, but not what it means?
➢ There are two ways in which one can understand the semantics of Bayesian networks:
❑ Representation of the Joint Probability Distribution
❑ Encoding of Conditional Independence Statements
➢ Equivalence of the Two Views:
❑ Constructing Network
❑ Designing Inference Procedures
12
Bayes' Theorem and Bayesian Networks
13
Representing the full joint distribution
➢ Nodes and Variables
➢ Edges and Dependencies
➢ Conditional Probability Tables (CPTs)
➢ Constructing the Full Joint Distribution
P(B,E,A,J,M)=P(B)*P(E)*P(A∣B,E)*P(J∣A)*P(M∣A)
14
Conditional Independence Relations in Bayesian Networks
➢ Definition of Conditional Independence
➢ Example of Conditional Independence
➢ Conditional Independence Relations in the Network
❑ B and E are conditionally independent
❑ J and M are conditionally independent given A
❑ J and M are not independent without conditioning on A
❑ B and E are conditionally independent given A
15
16
Part - III
Efficient Representation of Conditional Distributions
17
Efficient Representation of Conditional Distributions
➢ Key concepts
❑ Conditional Probability Table (CPT)
❑ Factorization of the Joint Distribution
❑ Reducing Complexity
18
Efficient Representation of Conditional Distributions
➢ Example
❑ Simple
19
Efficient Representation of Conditional Distributions
➢ Example
❑ Complex
20
Efficient Representation of Conditional Distributions
➢ Advantages of Efficient Representation
❑ Scalability
❑ Modularity
❑ Inference Efficiency
21
22
Part - IV
Exact Inference in Bayesian Networks
23
Why Exact Inference in Bayesian Networks
➢ Accurate Probabilistic Reasoning
➢ Decision Making
➢ Model Validation
➢ Understanding Complex Systems
➢ Guaranteed Accuracy
➢ Basis for Approximate Methods
➢ Legal and Ethical Considerations
24
Inference by enumeration
➢ Steps in Inference by Enumeration
❑ Identify the Query and Evidence Variables
❑ Sum Over Hidden Variables
❑ Normalize the Result
➢ Example
25
Inference by enumeration
➢ Step-by-Step Process
❑ Identify the Query and Evidence Variables
❑ Enumerate All Possible Worlds
❑ Compute the Joint Probabilities for Each World
❑ Calculate for Each Value of Sprinkler
❑ Sum the Probabilities
❑ Normalize the Result
26
The variable elimination algorithm
➢ Step-by-Step Process
❑ Identify the Query and Evidence Variables
❑ List All Factors
❑ Eliminate Hidden Variables
❑ Multiply the Remaining Factors
❑ Normalize the Result
27
The variable elimination algorithm
➢ Example
28
Analysis
29
Analysis
➢ Conceptual Simplicity
➢ Computational Efficiency
➢ Implementation
➢ Scalability
➢ Usage
30
31
Part - V
Approximate Inference in Bayesian Networks
32
Why Approximate Inference in Bayesian Networks
➢ Complexity of Exact Inference
➢ Scalability
➢ Real-time Decision Making
➢ Intractability of Marginalization
➢ Handling Uncertainty
➢ Integration with Machine Learning
33
Types of Approximate Inference
➢ Direct sampling methods are a category of approximate inference techniques used in Bayesian networks
Scalability
➢ Some key points:
❑ Basic Idea
❑ Gibbs Sampling
❑ Metropolis-Hastings Sampling
❑ Advantages
❑ Challenges
❑ Applications
34
Types of Approximate Inference
➢ Direct sampling method with Gibbs Sampling
❑ Diagnosis of a Medical Condition
35
Types of Approximate Inference
➢ Direct sampling method with Metropolis-Hastings Sampling
❑ Financial Risk Assessment
36
Types of Approximate Inference
➢ Markov chain simulation
❑ Markov Chain Basics
❑ Gibbs Sampling
❑ Metropolis-Hastings Sampling
❑ Iterative Process
❑ Advantages and Considerations
❑ Applications
37
Types of Approximate Inference
➢ Markov chain simulation with Gibbs Sampling
❑ Weather Forecasting
38
Types of Approximate Inference
➢ Markov chain simulation with Metropolis-Hastings Sampling
❑ Fraud Detection in Financial Transactions
39
40
Part - VI
Relational and First-Order Probability Models
41
Why Relational and First-Order Probability Models
➢ Modeling Complex Relationships
➢ Handling Uncertainty in Complex Systems
➢ Scalability and Flexibility
➢ Real-World Applications
42
Relational Probability Models
➢ Graphical Representation
➢ Probabilistic Dependencies
➢ Inference Process
➢ Learning and Parameter Estimation
43
First-Order Probability Models
➢ Logical Representation
➢ Probabilistic Integration
➢ Inference Mechanisms
➢ Applications
44
Integration and Advantages
➢ Expressiveness
➢ Scalability
➢ Real-World Applications
45
46
Part - VII
Other Approaches to Uncertain Reasoning
47
Rule-based methods for uncertain reasoning
➢ Rule-based systems emerged from early work on practical and intuitive systems for logical inference.
❑ Locality
❑ Detachment
❑ Truth-functionality
48
Representing ignorance: Dempster–Shafer theory
➢ Dempster–Shafer theory, also known as the theory of belief functions or evidence theory
➢ Key Concepts of Dempster–Shafer Theory:
❑ Frame of Discernment
❑ Basic Probability Assignment (BPA)
❑ Belief and Plausibility
❑ Dempster’s Rule of Combination
49
Representing vagueness: Fuzzy sets and fuzzy logic
➢ Key Concepts of Fuzzy Sets:
❑ Fuzzy Sets
❑ Membership Function
➢ Key Concepts of Fuzzy logic:
❑ Fuzzy logic
❑ Fuzzy operators
❑ Fuzzy Inference
➢ Applications of Fuzzy Sets and Fuzzy logic
❑ Control Systems
❑ Pattern Recognition
❑ Natural Language Processing
50
51
Summary
➢ The major points to recall are as follows:
❑ Representing Knowledge in an Uncertain Domain
❑ The Semantics of Bayesian Networks
❑ Efficient Representation of Conditional Distributions
❑ Exact Inference in Bayesian Networks
❑ Approximate Inference in Bayesian Networks
❑ Relational and First-Order Probability Models
❑ Other Approaches to Uncertain Reasoning
52
53