Module 4

Probabilistic reasoning
3.1Representing knowledge in an uncertain domain:

What is knowledge representation?

Humans are best at understanding, reasoning, and interpreting knowledge. Human
knows things, which is knowledge and as per their knowledge they perform various
actions in the real world. But how machines do all these things comes under
knowledge representation and reasoning. Hence we can describe Knowledge
representation as following:

Knowledge representation and reasoning (KR, KRR) is the part of Artificial intelligence
which concerned with AI agents thinking and how thinking contributes to intelligent
behavior of agents.

o It is responsible for representing information about the real world so that a

computer can understand and can utilize this knowledge to solve the complex
real world problems such as diagnosis a medical condition or communicating
with humans in natural language.
o It is also a way which describes how we can represent knowledge in artificial
intelligence. Knowledge representation is not just storing data into some
database, but it also enables an intelligent machine to learn from that knowledge
and experiences so that it can behave intelligently like a human.

What to Represent:
Following are the kind of knowledge which needs to be represented in AI systems:

o Object: All the facts about objects in our world domain. E.g., Guitars contains
strings, trumpets are brass instruments.
o Events: Events are the actions which occur in our world.
o Performance: It describe behavior which involves knowledge about how to do
things.
o Meta-knowledge: It is knowledge about what we know.
 o Facts: Facts are the truths about the real world and what we represent.
o Knowledge-Base: The central component of the knowledge-based agents is the
knowledge base. It is represented as KB. The Knowledgebase is a group of the
Sentences (Here, sentences are used as a technical term and not identical with
the English language).

Knowledge: Knowledge is awareness or familiarity gained by experiences of facts, data,

and situations. Following are the types of knowledge in artificial intelligence:

Types of knowledge
Following are the various types of knowledge:

1. Declarative Knowledge:

o Declarative knowledge is to know about something.

o It includes concepts, facts, and objects.
o It is also called descriptive knowledge and expressed in declarativesentences.
o It is simpler than procedural language.
2. Procedural Knowledge

o It is also known as imperative knowledge.

o Procedural knowledge is a type of knowledge which is responsible for knowing
how to do something.
o It can be directly applied to any task.
o It includes rules, strategies, procedures, agendas, etc.
o Procedural knowledge depends on the task on which it can be applied.

3. Meta-knowledge:

o Knowledge about the other types of knowledge is called Meta-knowledge.

4. Heuristic knowledge:

o Heuristic knowledge is representing knowledge of some experts in a filed or

subject.
o Heuristic knowledge is rules of thumb based on previous experiences, awareness
of approaches, and which are good to work but not guaranteed.

5. Structural knowledge:

o Structural knowledge is basic knowledge to problem-solving.

o It describes relationships between various concepts such as kind of, part of, and
grouping of something.
o It describes the relationship that exists between concepts or objects.

The relation between knowledge and intelligence:

Knowledge of real-worlds plays a vital role in intelligence and same for creating artificial
intelligence. Knowledge plays an important role in demonstrating intelligent behavior in
AI agents. An agent is only able to accurately act on some input when he has some
knowledge or experience about that input.

Let's suppose if you met some person who is speaking in a language which you don't
know, then how you will able to act on that. The same thing applies to the intelligent
behavior of the agents.
As we can see in below diagram, there is one decision maker which act by sensing the
environment and using knowledge. But if the knowledge part will not present then, it
cannot display intelligent behavior.

AI knowledge cycle:
An Artificial intelligence system has the following components for displaying intelligent
behavior:

o Perception
o Learning
o Knowledge Representation and Reasoning
o Planning
o Execution
The above diagram is showing how an AI system can interact with the real world and
what components help it to show intelligence. AI system has Perception component by
which it retrieves information from its environment. It can be visual, audio or another
form of sensory input. The learning component is responsible for learning from data
captured by Perception comportment. In the complete cycle, the main components are
knowledge representation and Reasoning. These two components are involved in
showing the intelligence in machine-like humans. These two components are
independent with each other but also coupled together. The planning and execution
depend on analysis of Knowledge representation and reasoning.

Approaches to knowledge representation:

There are mainly four approaches to knowledge representation, which are givenbelow:

1. Simple relational knowledge:

o It is the simplest way of storing facts which uses the relational method, and each
fact about a set of the object is set out systematically in columns.
o This approach of knowledge representation is famous in database systems where
the relationship between different entities is represented.
o This approach has little opportunity for inference.

Example: The following is the simple relational knowledge representation.

Player Weight Age

Player1 65 23

Player2 58 18

Player3 75 24

2. Inheritable knowledge:

o In the inheritable knowledge approach, all data must be stored into a hierarchy of
classes.
o All classes should be arranged in a generalized form or a hierarchal manner.
o In this approach, we apply inheritance property.
o Elements inherit values from other members of a class.
o This approach contains inheritable knowledge which shows a relation between
instance and class, and it is called instance relation.
o Every individual frame can represent the collection of attributes and its value.
o In this approach, objects and values are represented in Boxed nodes.
o We use Arrows which point from objects to their values.
o Example:
3. Inferential knowledge:

o Inferential knowledge approach represents knowledge in the form of formal

logics.
o This approach can be used to derive more facts.
o It guaranteed correctness.
o Example: Let's suppose there are two statements:
a. Marcus is a man
b. All men are mortal
Then it can represent as;

man(Marcus)
∀x = man (x) ----------> mortal (x)s

4. Procedural knowledge:

o Procedural knowledge approach uses small programs and codes which describes
how to do specific things, and how to proceed.
 o In this approach, one important rule is used which is If-Then rule.
o In this knowledge, we can use various coding languages such as LISP
language and Prolog language.
o We can easily represent heuristic or domain-specific knowledge using this
approach.
o But it is not necessary that we can represent all cases in this approach.

Requirements for knowledge Representation system:

A good knowledge representation system must possess the following properties.

1. Representational Accuracy:
KR system should have the ability to represent all kind of required knowledge.

2. Inferential Adequacy:
KR system should have ability to manipulate the representational structures to
produce new knowledge corresponding to existing structure.

3. Inferential Efficiency:
The ability to direct the inferential knowledge mechanism into the most
productive directions by storing appropriate guides.

4. Acquisitional efficiency- The ability to acquire the new knowledge easily

using automatic methods.

Probabilistic reasoning in Artificial intelligence

Uncertainty:
Till now, we have learned knowledge representation using first-order logic and
propositional logic with certainty, which means we were sure about the predicates. With
this knowledge representation, we might write A→B, which means if A is true then B is
true, but consider a situation where we are not sure about whether A is true or not then
we cannot express this statement, this situation is called uncertainty.

So to represent uncertain knowledge, where we are not sure about the predicates, we
need uncertain reasoning or probabilistic reasoning.
Causes of uncertainty:
Following are some leading causes of uncertainty to occur in the real world.

1. Information occurred from unreliable sources.

2. Experimental Errors
3. Equipment fault
4. Temperature variation
5. Climate change.

Probabilistic reasoning:
Probabilistic reasoning is a way of knowledge representation where we apply the
concept of probability to indicate the uncertainty in knowledge. In probabilistic
reasoning, we combine probability theory with logic to handle the uncertainty.

We use probability in probabilistic reasoning because it provides a way to handle the

uncertainty that is the result of someone's laziness and ignorance.

In the real world, there are lots of scenarios, where the certainty of something is not
confirmed, such as "It will rain today," "behavior of someone for some situations," "A
match between two teams or two players." These are probable sentences for which we
can assume that it will happen but not sure about it, so here we use probabilistic
reasoning.

Need of probabilistic reasoning in AI:

o When there are unpredictable outcomes.

o When specifications or possibilities of predicates becomes too large to handle.
o When an unknown error occurs during an experiment.

In probabilistic reasoning, there are two ways to solve problems with uncertain
knowledge:

o Bayes' rule
o Bayesian Statistics
 Note: We will learn the above two rules in later chapters.

As probabilistic reasoning uses probability and related terms, so before understanding

probabilistic reasoning, let's understand some common terms:

Probability: Probability can be defined as a chance that an uncertain event will occur. It
is the numerical measure of the likelihood that an event will occur. The value of
probability always remains between 0 and 1 that represent ideal uncertainties.

ADVERTISEMENT

1. 0 ≤ P(A) ≤ 1, where P(A) is the probability of an event A.

1. P(A) = 0, indicates total uncertainty in an event A.
1. P(A) =1, indicates total certainty in an event A.

We can find the probability of an uncertain event by using the below formula.

o P(¬A) = probability of a not happening event.

o P(¬A) + P(A) = 1.

Event: Each possible outcome of a variable is called an event.

Sample space: The collection of all possible events is called sample space.

Random variables: Random variables are used to represent the events and objects in
the real world.

Prior probability: The prior probability of an event is probability computed before

observing new information.

Posterior Probability: The probability that is calculated after all evidence or

information has taken into account. It is a combination of prior probability and new
information.

Conditional probability:
Conditional probability is a probability of occurring an event when another event has
already happened.
Let's suppose, we want to calculate the event A when event B has already occurred, "the
probability of A under the conditions of B", it can be written as:

Where P(A⋀B)= Joint probability of a and B

P(B)= Marginal probability of B.

ADVERTISEMENT

If the probability of A is given and we need to find the probability of B, then it will be
given as:

It can be explained by using the below Venn diagram, where B is occurred event, so
sample space will be reduced to set B, and now we can only calculate event A when
event B is already occurred by dividing the probability of P(A⋀B) by P( B ).

ADVERTISEMENT

ADVERTISEMENT

Example:
In a class, there are 70% of the students who like English and 40% of the students who
likes English and mathematics, and then what is the percent of students those who like
English also like mathematics?

Solution:

Let, A is an event that a student likes Mathematics

B is an event that a student likes English.

Hence, 57% are the students who like English also like Mathematics.

3.2semantics of Bayesian networks:

Bayes' theorem in Artificial intelligence

Bayes' theorem:
Bayes' theorem is also known as Bayes' rule, Bayes' law, or Bayesian reasoning, which
determines the probability of an event with uncertain knowledge.

In probability theory, it relates the conditional probability and marginal probabilities of

two random events.

Bayes' theorem was named after the British mathematician Thomas Bayes.
The Bayesian inference is an application of Bayes' theorem, which is fundamental to
Bayesian statistics.

It is a way to calculate the value of P(B|A) with the knowledge of P(A|B).

Bayes' theorem allows updating the probability prediction of an event by observing new
information of the real world.

Example: If cancer corresponds to one's age then by using Bayes' theorem, we can
determine the probability of cancer more accurately with the help of age.

Bayes' theorem can be derived using product rule and conditional probability of event A
with known event B:
 As from product rule we can write:

1. P(A ⋀ B)= P(A|B) P(B) or

Similarly, the probability of event B with known event A:

1. P(A ⋀ B)= P(B|A) P(A)

Equating right hand side of both the equations, we will get:

The above equation (a) is called as Bayes' rule or Bayes' theorem. This equation is
basic of most modern AI systems for probabilistic inference.

It shows the simple relationship between joint and conditional probabilities. Here,

P(A|B) is known as posterior, which we need to calculate, and it will be read as

Probability of hypothesis A when we have occurred an evidence B.

P(B|A) is called the likelihood, in which we consider that hypothesis is true, then we
calculate the probability of evidence.

P(A) is called the prior probability, probability of hypothesis before considering the
evidence

P(B) is called marginal probability, pure probability of an evidence.

In the equation (a), in general, we can write P (B) = P(A)*P(B|Ai), hence the Bayes' rule
can be written as:

Where A1, A2, A3,........, An is a set of mutually exclusive and exhaustive events.

Applying Bayes' rule:

Bayes' rule allows us to compute the single term P(B|A) in terms of P(A|B), P(B), and P(A).
This is very useful in cases where we have a good probability of these three terms and
want to determine the fourth one. Suppose we want to perceive the effect of some
unknown cause, and want to compute that cause, then the Bayes' rule becomes:

Example-1:

Question: what is the probability that a patient has diseases meningitis with a stiff
neck?

Given Data:

A doctor is aware that disease meningitis causes a patient to have a stiff neck, and it
occurs 80% of the time. He is also aware of some more facts, which are given as follows:

o The Known probability that a patient has meningitis disease is 1/30,000.

o The Known probability that a patient has a stiff neck is 2%.

Let a be the proposition that patient has stiff neck and b be the proposition that patient
has meningitis. , so we can calculate the following as:

P(a|b) = 0.8

P(b) = 1/30000

P(a)= .02

Hence, we can assume that 1 patient out of 750 patients has meningitis disease with a
stiff neck.

Example-2:
Question: From a standard deck of playing cards, a single card is drawn. The
probability that the card is king is 4/52, then calculate posterior probability
P(King|Face), which means the drawn face card is a king card.

Solution:

P(king): probability that the card is King= 4/52= 1/13

P(face): probability that a card is a face card= 3/13

P(Face|King): probability of face card when we assume it is a king = 1

Putting all values in equation (i) we will get:

Application of Bayes' theorem in Artificial intelligence:

Following are some applications of Bayes' theorem:

o It is used to calculate the next step of the robot when the already executed step
is given.
o Bayes' theorem is helpful in weather forecasting.
o It can solve the Monty Hall problem.

Bayesian Belief Network in artificial intelligence

Bayesian belief network is key computer technology for dealing with probabilistic events
and to solve a problem which has uncertainty. We can define a Bayesian network as:

"A Bayesian network is a probabilistic graphical model which represents a set of

variables and their conditional dependencies using a directed acyclic graph."
It is also called a Bayes network, belief network, decision network, or Bayesian
model.

Bayesian networks are probabilistic, because these networks are built from a probability
distribution, and also use probability theory for prediction and anomaly detection.

Real world applications are probabilistic in nature, and to represent the relationship
between multiple events, we need a Bayesian network. It can also be used in various
tasks including prediction, anomaly detection, diagnostics, automated insight,
reasoning, time series prediction, and decision making under uncertainty.

Bayesian Network can be used for building models from data and experts opinions, and
it consists of two parts:

o Directed Acyclic Graph

o Table of conditional probabilities.

The generalized form of Bayesian network that represents and solve decision problems
under uncertain knowledge is known as an Influence diagram.

A Bayesian network graph is made up of nodes and Arcs (directed links), where:
 o Each node corresponds to the random variables, and a variable can
be continuous or discrete.
o Arc or directed arrows represent the causal relationship or conditional probabilities
between random variables. These directed links or arrows connect the pair of nodes in
the graph.
These links represent that one node directly influence the other node, and if there is no
directed link that means that nodes are independent with each other
o In the above diagram, A, B, C, and D are random variables represented by
the nodes of the network graph.
o If we are considering node B, which is connected with node A by a directed
arrow, then node A is called the parent of Node B.
o Node C is independent of node A.

Note: The Bayesian network graph does not contain any cyclic graph. Hence, it is known as
a directed acyclic graph or DAG.

The Bayesian network has mainly two components:

o Causal Component
o Actual numbers

Each node in the Bayesian network has condition probability

distribution P(Xi |Parent(Xi) ), which determines the effect of the parent on that node.

Bayesian network is based on Joint probability distribution and conditional probability.

So let's first understand the joint probability distribution:

Joint probability distribution:

If we have variables x1, x2, x3,....., xn, then the probabilities of a different combination of
x1, x2, x3.. xn, are known as Joint probability distribution.

P[x1, x2, x3,....., xn], it can be written as the following way in terms of the joint
probability distribution.

= P[x1| x2, x3,....., xn]P[x2, x3,....., xn]

= P[x1| x2, x3,....., xn]P[x2|x3,....., xn]....P[xn-1|xn]P[xn].

In general for each variable Xi, we can write the equation as:

P(Xi|Xi-1,........., X1) = P(Xi |Parents(Xi ))

Explanation of Bayesian network:

Let's understand the Bayesian network through an example by creating a directed
acyclic graph:

Example: Harry installed a new burglar alarm at his home to detect burglary. The alarm
reliably responds at detecting a burglary but also responds for minor earthquakes. Harry
has two neighbors David and Sophia, who have taken a responsibility to inform Harry at
work when they hear the alarm. David always calls Harry when he hears the alarm, but
sometimes he got confused with the phone ringing and calls at that time too. On the
other hand, Sophia likes to listen to high music, so sometimes she misses to hear the
alarm. Here we would like to compute the probability of Burglary Alarm.

Problem:

Calculate the probability that alarm has sounded, but there is neither a burglary,
nor an earthquake occurred, and David and Sophia both called the Harry.

Solution:

o The Bayesian network for the above problem is given below. The network structure is
showing that burglary and earthquake is the parent node of the alarm and directly
affecting the probability of alarm's going off, but David and Sophia's calls depend on
alarm probability.
o The network is representing that our assumptions do not directly perceive the burglary
and also do not notice the minor earthquake, and they also not confer before calling.
o The conditional distributions for each node are given as conditional probabilities table or
CPT.
o Each row in the CPT must be sum to 1 because all the entries in the table represent an
exhaustive set of cases for the variable.
o In CPT, a boolean variable with k boolean parents contains 2 K probabilities. Hence, if
there are two parents, then CPT will contain 4 probability values

List of all events occurring in this network:

 o Burglary (B)
o Earthquake(E)
o Alarm(A)
o David Calls(D)
o Sophia calls(S)

We can write the events of problem statement in the form of probability: P[D, S, A, B,
E], can rewrite the above probability statement using joint probability distribution:

P[D, S, A, B, E]= P[D | S, A, B, E]. P[S, A, B, E]

=P[D | S, A, B, E]. P[S | A, B, E]. P[A, B, E]

= P [D| A]. P [ S| A, B, E]. P[ A, B, E]

= P[D | A]. P[ S | A]. P[A| B, E]. P[B, E]

= P[D | A ]. P[S | A]. P[A| B, E]. P[B |E]. P[E]

Let's take the observed probability for the Burglary and earthquake component:
P(B= True) = 0.002, which is the probability of burglary.

P(B= False)= 0.998, which is the probability of no burglary.

P(E= True)= 0.001, which is the probability of a minor earthquake

P(E= False)= 0.999, Which is the probability that an earthquake not occurred.

We can provide the conditional probabilities as per the below tables:

Conditional probability table for Alarm A:

The Conditional probability of Alarm A depends on Burglar and earthquake:

B E P(A= True) P(A= False)

True True 0.94 0.06

True False 0.95 0.04

False True 0.31 0.69

False False 0.001 0.999

Conditional probability table for David Calls:

The Conditional probability of David that he will call depends on the probability of
Alarm.

A P(D= True) P(D= False)

True 0.91 0.09

False 0.05 0.95

Conditional probability table for Sophia Calls:

The Conditional probability of Sophia that she calls is depending on its Parent Node
"Alarm."

A P(S= True) P(S= False)

True 0.75 0.25

False 0.02 0.98

From the formula of joint distribution, we can write the problem statement in the form
of probability distribution:

ADVERTISEMENT

P(S, D, A, ¬B, ¬E) = P (S|A) *P (D|A)*P (A|¬B ^ ¬E) *P (¬B) *P (¬E).

= 0.75* 0.91* 0.001* 0.998*0.999

= 0.00068045.

Hence, a Bayesian network can answer any query about the domain by using Joint
distribution.

The semantics of Bayesian Network:

There are two ways to understand the semantics of the Bayesian network, which is given
below:

1. To understand the network as the representation of the Joint probability

distribution.

It is helpful to understand how to construct the network.

2. To understand the network as an encoding of a collection of conditional

independence statements.

It is helpful in designing inference procedure.

3.3probabilistic reasoning over time:

Probabilistic reasoning over time, often referred to as temporal probabilistic

reasoning, is a branch of probability theory that deals with modeling uncertainty
and making predictions about events or states of a system that evolve over time.
This field is particularly relevant in various domains such as finance, weather
forecasting, epidemiology, and artificial intelligence.

Here are some key concepts and methods used in probabilistic reasoning over time:

Markov Models: Markov models are a class of probabilistic models that describe
the transition of a system from one state to another over discrete time steps. In a
Markov model, the probability of transitioning to a future state depends only on the
current state of the system and not on the previous history of states.

Hidden Markov Models (HMMs): Hidden Markov Models are a type of Markov
model where the states of the system are not directly observable but are associated
with observable outputs. HMMs are widely used in speech recognition,
bioinformatics, and other applications where the underlying state of a system is
hidden.

Kalman Filters: Kalman filters are a set of mathematical equations that provide an
efficient recursive solution to the problem of estimating the state of a linear
dynamic system observed with noisy measurements. Kalman filters are extensively
used in fields such as control systems, navigation, and signal processing.

Bayesian Networks: Bayesian networks are graphical models that represent the
probabilistic relationships among a set of variables. Bayesian networks can be used
to model complex systems where variables evolve over time, and inference
algorithms can be applied to make predictions or update beliefs about the system
state given new evidence.

Particle Filters (Sequential Monte Carlo Methods): Particle filters are a class of
sequential Monte Carlo methods used for estimating the state of a nonlinear and
non-Gaussian dynamic system. Particle filters represent the posterior distribution
of the system state using a set of weighted particles, which are updated recursively
based on new observations.

Dynamic Bayesian Networks (DBNs): Dynamic Bayesian Networks extend the

concept of Bayesian networks to model temporal dependencies among variables.
DBNs are composed of a set of interconnected nodes representing variables that
evolve over time, and conditional probability distributions are defined over these
variables.

These methods and techniques allow probabilistic reasoning over time by

capturing the uncertainty inherent in the evolution of dynamic systems and making
predictions or inferences based on observed data and prior knowledge about the
system dynamics.
Temporal probabilistic reasoning involves reasoning about uncertainty in events and states of a
system as they evolve over time. It's a framework for modeling, predicting, and reasoning about
probabilistic relationships in dynamic systems. Here's a breakdown of its description:

Modeling Time-dependent Processes: Temporal probabilistic reasoning begins by constructing

models that represent the dynamics of systems over time. These models capture how variables
change and interact with each other as time progresses. The behavior of these variables may be
influenced by external factors, historical data, or stochastic processes.

Incorporating Uncertainty: Uncertainty is inherent in dynamic systems due to incomplete

information, noise, or randomness in the underlying processes. Temporal probabilistic reasoning
explicitly accounts for this uncertainty by assigning probabilities to different possible states or
outcomes at each time step.

Dynamic Bayesian Networks (DBNs): DBNs are a popular framework for temporal probabilistic
reasoning. They extend traditional Bayesian networks to model dependencies between variables
across multiple time steps. In DBNs, nodes represent variables, and directed edges capture
temporal relationships between them. Conditional probability distributions are defined over each
variable given its parents in the network.

Inference and Prediction: Once a temporal probabilistic model is constructed, inference

algorithms are used to make predictions or reason about the system's behavior over time.
Inference involves updating beliefs about the system's state based on observed evidence,
historical data, and the model's probabilistic structure. Predictions about future states or events
can also be made based on the current state of the system and its dynamics.

Applications: Temporal probabilistic reasoning finds applications in various domains, including

finance, weather forecasting, healthcare, robotics, and natural language processing. For example,
it can be used to predict stock prices, forecast the spread of diseases, track moving objects in a
video sequence, or understand the structure of documents over time.

Challenges: Temporal probabilistic reasoning faces several challenges, including the complexity
of modeling dynamic systems, the curse of dimensionality in large-scale problems, and the need
for efficient inference algorithms. Addressing these challenges often requires a combination of
mathematical techniques, computational methods, and domain-specific knowledge.

In summary, temporal probabilistic reasoning provides a powerful framework for reasoning

about uncertainty in dynamic systems and making informed decisions in the face of incomplete
information and changing environments. It integrates concepts from probability theory, statistics,
and machine learning to model, predict, and understand the behavior of complex systems over
time.
3.4 Time and Uncertainty:
Time and uncertainty play significant roles in artificial intelligence (AI),
particularly in areas like machine learning, robotics, and natural
language processing. Let's explore their description in AI along with an
example:

Description:
Temporal Dynamics in AI: Many AI tasks involve processing data that
unfolds over time. For instance, in natural language processing,
understanding a conversation requires analyzing a sequence of words
spoken or written over time. Similarly, in robotics, navigating an
environment involves sensing and acting in a sequence of time steps.

Uncertainty in AI: Uncertainty arises due to incomplete or noisy

information, ambiguity in data, and the inherent unpredictability of real-
world phenomena. In AI, uncertainty often manifests in various forms,
such as uncertainty in sensor measurements, uncertainty in model
parameters, or uncertainty in decision-making processes.

Probabilistic Modeling and Inference: To reason about uncertainty in AI

systems, probabilistic models and inference methods are commonly
used. These models represent uncertainty explicitly, allowing AI
systems to make informed decisions even in the presence of incomplete
or uncertain information. Bayesian inference, Markov models, and
Bayesian networks are some examples of probabilistic approaches used
in AI.
Dynamic Systems and Adaptation: Many AI systems operate in dynamic
environments where conditions change over time. These systems need to
adapt to new information and evolving situations to perform effectively.
Reinforcement learning algorithms, for example, enable agents to learn
optimal policies by interacting with their environment over time and
adjusting their behavior based on feedback.

Example: Autonomous Driving

Consider the example of autonomous driving, where AI systems control
vehicles to navigate safely and efficiently in complex environments:

Temporal Dynamics: Autonomous vehicles operate in environments

where the surroundings change rapidly over time. They need to perceive
and respond to dynamic elements such as pedestrians, other vehicles,
and changing road conditions in real-time.

Uncertainty in Perception and Decision-making: Sensors on autonomous

vehicles, such as cameras, LiDAR, and radar, provide imperfect and
noisy information about the environment. Moreover, predicting the
behavior of other road users and anticipating future traffic conditions
involve inherent uncertainty.

Probabilistic Reasoning: Autonomous driving systems employ

probabilistic models to reason about uncertainty. For instance, they use
Bayesian filters like Kalman filters or particle filters to estimate the state
of the vehicle and its surroundings based on sensor measurements.
Bayesian networks can also be used to model complex relationships
between environmental factors and make probabilistic predictions about
future events.

Adaptive Behavior: Autonomous vehicles continuously adapt their

behavior based on changing environmental conditions and new
information. Reinforcement learning techniques enable vehicles to learn
from experience and optimize their driving policies over time,
considering factors like safety, efficiency, and passenger comfort.

In summary, time and uncertainty are fundamental aspects of AI

systems, influencing their ability to perceive, reason, and act in dynamic
and uncertain environments. By employing probabilistic models and
adaptive strategies, AI systems can effectively navigate uncertainty and
make reliable decisions over time.
3.5 Inference in temporal models
Inference in temporal models in artificial intelligence involves reasoning
about the evolution of events or states over time, accounting for
uncertainty and making predictions or decisions based on observed data.
Temporal models capture dependencies between variables across
different time steps and enable AI systems to understand and predict
how systems evolve dynamically. Here's a description of inference in
temporal models along with an example in artificial intelligence:

Description:
Temporal Dependencies: Temporal models represent how variables
change over time and capture dependencies between them. These
models often include dynamic Bayesian networks (DBNs), hidden
Markov models (HMMs), recurrent neural networks (RNNs), or other
structures that explicitly model temporal relationships.

Probabilistic Reasoning: Inference in temporal models involves

probabilistic reasoning to estimate the likelihood of different states or
events at future time steps given observed data. Bayesian inference
methods are commonly used to update beliefs about the system's state
over time based on new evidence and prior knowledge.

Dynamic Evolution: Temporal models account for the dynamic

evolution of systems by incorporating transition probabilities between
states or observations at different time points. This allows AI systems to
make predictions about future states or sequences of events based on the
current state and past observations.

Uncertainty Management: Temporal models explicitly represent

uncertainty in the system's evolution and observations. Inference
algorithms take into account this uncertainty to provide probabilistic
estimates of future states or events, enabling AI systems to make
decisions under uncertainty.

Example: Stock Price Prediction

Consider the task of predicting stock prices over time using temporal
models in artificial intelligence:

Temporal Data: Stock prices fluctuate over time in response to various

factors such as market trends, company performance, and economic
indicators. Historical stock price data forms a time series, where each
data point represents the price of the stock at a specific time.

Temporal Model: A recurrent neural network (RNN) can be used to

model the temporal dependencies in the stock price data. The RNN takes
as input historical stock prices and learns to capture patterns and trends
in the data over time.

Inference: Given historical stock price data, the RNN performs inference
to predict future stock prices. The RNN updates its internal state based
on observed data and uses this information to generate probabilistic
predictions about future stock prices.

Uncertainty Estimation: The RNN outputs not only point predictions of

future stock prices but also estimates the uncertainty associated with
each prediction. This uncertainty reflects the confidence level of the
model in its predictions and helps investors make informed decisions
about trading strategies.

Dynamic Adaptation: As new stock price data becomes available, the

RNN continuously updates its predictions and adapts to changing market
conditions. This dynamic adaptation allows the model to capture
evolving patterns in the stock market and adjust its predictions
accordingly.

In summary, inference in temporal models in artificial intelligence

enables systems to understand and predict the dynamic evolution of
events over time, accounting for uncertainty and making informed
decisions based on observed data. This capability has applications in
various domains, including finance, healthcare, and natural language
processing..
3.6 Hidden markov models:

Hidden Markov Model in Machine Learning

Hidden Markov Models (HMMs) are a type of probabilistic model that are commonly
used in machine learning for tasks such as speech recognition, natural language
processing, and bioinformatics. They are a popular choice for modelling sequences of
data because they can effectively capture the underlying structure of the data, even
when the data is noisy or incomplete. In this article, we will give a comprehensive
overview of Hidden Markov Models, including their mathematical foundations,
applications, and limitations.

What are Hidden Markov Models?

A Hidden Markov Model (HMM) is a probabilistic model that consists of a sequence
of hidden states, each of which generates an observation. The hidden states are
usually not directly observable, and the goal of HMM is to estimate the sequence of
hidden states based on a sequence of observations. An HMM is defined by the following
components:

o A set of N hidden states, S = {s1, s2, ..., sN}.

o A set of M observations, O = {o1, o2, ..., oM}.
 o An initial state probability distribution, ? = {?1, ?2, ..., ?N}, which specifies the
probability of starting in each hidden state.
o A transition probability matrix, A = [aij], defines the probability of moving from
one hidden state to another.
o An emission probability matrix, B = [bjk], defines the probability of emitting an
observation from a given hidden state.

The basic idea behind an HMM is that the hidden states generate the observations, and
the observed data is used to estimate the hidden state sequence. This is often referred
to as the forward-backwards algorithm.

Applications of Hidden Markov Models

Now, we will explore some of the key applications of HMMs, including speech
recognition, natural language processing, bioinformatics, and finance.

o Speech Recognition
One of the most well-known applications of HMMs is speech recognition. In this
field, HMMs are used to model the different sounds and phones that makeup
speech. The hidden states, in this case, correspond to the different sounds or
phones, and the observations are the acoustic signals that are generated by the
speech. The goal is to estimate the hidden state sequence, which corresponds to
the transcription of the speech, based on the observed acoustic signals. HMMs
are particularly well-suited for speech recognition because they can effectively
capture the underlying structure of the speech, even when the data is noisy or
incomplete. In speech recognition systems, the HMMs are usually trained on
large datasets of speech signals, and the estimated parameters of the HMMs are
used to transcribe speech in real time.
o Natural Language Processing
Another important application of HMMs is natural language processing. In this
field, HMMs are used for tasks such as part-of-speech tagging, named entity
recognition, and text classification. In these applications, the hidden states are
typically associated with the underlying grammar or structure of the text, while
the observations are the words in the text. The goal is to estimate the hidden
state sequence, which corresponds to the structure or meaning of the text, based
 on the observed words. HMMs are useful in natural language processing because
they can effectively capture the underlying structure of the text, even when the
data is noisy or ambiguous. In natural language processing systems, the HMMs
are usually trained on large datasets of text, and the estimated parameters of the
HMMs are used to perform various NLP tasks, such as text classification, part-of-
speech tagging, and named entity recognition.
o Bioinformatics
HMMs are also widely used in bioinformatics, where they are used to model
sequences of DNA, RNA, and proteins. The hidden states, in this case, correspond
to the different types of residues, while the observations are the sequences of
residues. The goal is to estimate the hidden state sequence, which corresponds to
the underlying structure of the molecule, based on the observed sequences of
residues. HMMs are useful in bioinformatics because they can effectively capture
the underlying structure of the molecule, even when the data is noisy or
incomplete. In bioinformatics systems, the HMMs are usually trained on large
datasets of molecular sequences, and the estimated parameters of the HMMs are
used to predict the structure or function of new molecular sequences.
o Finance
Finally, HMMs have also been used in finance, where they are used to model
stock prices, interest rates, and currency exchange rates. In these applications, the
hidden states correspond to different economic states, such as bull and bear
markets, while the observations are the stock prices, interest rates, or exchange
rates. The goal is to estimate the hidden state sequence, which corresponds to
the underlying economic state, based on the observed prices, rates, or exchange
rates. HMMs are useful in finance because they can effectively capture the
underlying economic state, even when the data is noisy or incomplete. In finance
systems, the HMMs are usually trained on large datasets of financial data, and the
estimated parameters of the HMMs are used to make predictions about future
market trends or to develop investment strategies.

Limitations of Hidden Markov Models

Now, we will explore some of the key limitations of HMMs and discuss how they can
impact the accuracy and performance of HMM-based systems.
o Limited Modeling Capabilities
One of the key limitations of HMMs is that they are relatively limited in their
modelling capabilities. HMMs are designed to model sequences of data, where
the underlying structure of the data is represented by a set of hidden states.
However, the structure of the data can be quite complex, and the simple
structure of HMMs may not be enough to accurately capture all the details. For
example, in speech recognition, the complex relationship between the speech
sounds and the corresponding acoustic signals may not be fully captured by the
simple structure of an HMM.
o Overfitting
Another limitation of HMMs is that they can be prone to overfitting, especially
when the number of hidden states is large or the amount of training data is
limited. Overfitting occurs when the model fits the training data too well and is
unable to generalize to new data. This can lead to poor performance when the
model is applied to real-world data and can result in high error rates. To avoid
overfitting, it is important to carefully choose the number of hidden states and to
use appropriate regularization techniques.
o Lack of Robustness
HMMs are also limited in their robustness to noise and variability in the data. For
example, in speech recognition, the acoustic signals generated by speech can be
subjected to a variety of distortions and noise, which can make it difficult for the
HMM to accurately estimate the underlying structure of the data. In some cases,
these distortions and noise can cause the HMM to make incorrect decisions,
which can result in poor performance. To address these limitations, it is often
necessary to use additional processing and filtering techniques, such as noise
reduction and normalization, to pre-process the data before it is fed into the
HMM.
o Computational Complexity
Finally, HMMs can also be limited by their computational complexity, especially
when dealing with large amounts of data or when using complex models. The
computational complexity of HMMs is due to the need to estimate the
parameters of the model and to compute the likelihood of the data given in the
model. This can be time-consuming and computationally expensive, especially for
large models or for data that is sampled at a high frequency. To address this
limitation, it is often necessary to use parallel computing techniques or to use
approximations that reduce the computational complexity of the model.

3.7 Kalman filter:

The Kalman filter is a powerful algorithm used in artificial

intelligence for state estimation and prediction in dynamic systems.
It's particularly useful when dealing with noisy sensor
measurements and uncertain dynamics. Here's how the Kalman
filter works along with an example in AI:

Description:

State Estimation: The Kalman filter is designed to estimate the

state of a linear dynamic system given noisy measurements. The
state of the system represents the internal variables that describe its
behavior, such as position, velocity, or orientation.

Predictive Model: The Kalman filter uses a predictive model to

forecast the future state of the system based on its current state and
known dynamics. This prediction is then refined using incoming
sensor measurements to update the state estimate.

Bayesian Framework: The Kalman filter operates within a

Bayesian framework, where it maintains a probability distribution
over possible states of the system. As new measurements arrive,
the filter updates this distribution using Bayes' rule, incorporating
both the prior state estimate and the likelihood of the
measurements given the predicted state.
Optimal Estimation: The Kalman filter provides the optimal
estimate of the system's state under the assumptions of linearity
and Gaussian noise. It minimizes the mean squared error between
the estimated state and the true state of the system.

Example: Object Tracking in Computer Vision

Consider the example of object tracking in computer vision, where

the goal is to estimate the position and velocity of a moving object
based on noisy measurements from a camera:

Dynamic System: The motion of the object can be described by a

simple linear dynamic model, such as a constant velocity model.
This model predicts the future position of the object based on its
current position and velocity.

Noisy Measurements: The camera provides noisy measurements of

the object's position at discrete time intervals. These measurements
are subject to errors due to factors such as sensor noise, occlusions,
and motion blur.

Kalman Filter: The Kalman filter is applied to estimate the state of

the object (i.e., its position and velocity) based on the dynamic
model and the noisy measurements from the camera.
Prediction Step: In the prediction step, the Kalman filter uses the
dynamic model to predict the next state of the object based on its
current state. This prediction includes estimates of the object's
position and velocity along with their uncertainties.

Update Step: In the update step, the Kalman filter incorporates the
noisy measurements from the camera to refine its estimate of the
object's state. It computes the Kalman gain, which determines how
much weight to assign to the prediction and the measurements, and
updates the state estimate accordingly.

Iterative Process: The Kalman filter iterates between prediction

and update steps as new measurements become available. With
each iteration, the filter refines its estimate of the object's state,
providing a more accurate and reliable tracking result.

In summary, the Kalman filter is a versatile tool in artificial

intelligence for state estimation and prediction in dynamic systems.
It finds applications in various domains, including object tracking,
navigation, sensor fusion, and robotics, where accurate estimation
of system states is essential for decision-making and control.