Session Overview
• Introduction to the Model of Cognition
• Visual Perception
• Visual Recognition
• Machine Learning, and Robot Cognition
Introduction to the Model of Cognition
• Cognitive models represent internal depictions of the external world.
• scenes, and task coordination.
Application in Robotics:
• Early robotic models used functional decomposition (sensing, planning, task coordination).
• Rodney A. Brooks introduced the subsumption architecture in 1986, inspired by
animal behaviours.
• Development of robots with simple behaviours (e.g., obstacle avoidance, navigation) that build
complex actions through hierarchical behaviour layers.
S TA G E S O F C O G N I T I O N
7 Mental States
• Visual Sensing and Acquisition
• Reasoning
• Attention
• Recognition
• Learning
• Planning
• Action and Coordination
S TA G E S O F C O G N I T I O N
Cognitive Memory:
• Long Term Memory (LTM)
• Short Term Memory (STM)
• LTM: Stores information over
extended periods, facilitating long-
term retention and recall.
• STM: Temporarily holds information
for immediate use, typically for
seconds to minutes.
S TA G E S O F C O G N I T I O N
Visual Sensing and Acquisition
• Reception and transformation of signals into a measurable form.
• Includes preprocessing and feature extraction from sensed data.
• Example: Visual information is filtered and features like size, shape, color are extracted and
stored in Short Term Memory (STM).
S TA G E S O F C O G N I T I O N
Reasoning
• Constructs high-level knowledge from low-level information.
• Analyses semantic behaviour and associations of knowledge.
• Commonsense reasoning, causal reasoning, non-monotonic reasoning, default reasoning, fuzzy
reasoning, spatial and temporal reasoning, and meta-level reasoning.
S TA G E S O F C O G N I T I O N
Recognition
• Identifying complex sensory stimuli from a scene.
• Sensory data is processed, transformed, organized, and compared with stored information in
Long Term Memory (LTM) for recognition.
S TA G E S O F C O G N I T I O N
Learning
• Classifies sensory stimuli without explicit rules.
• Example: A child distinguishing between a cat and a dog through examples.
• Types: Supervised, unsupervised, and reinforcement learning.
S TA G E S O F C O G N I T I O N
Action and Coordination
• Determines control commands for actuators to execute action plans.
• Coordinates between desired actions and input stimuli through supervised learning.
S TA G E S O F C O G N I T I O N
• Sub Types: Iconic memory (visual
information) and echoic
memory (audio
information).
• Alternative Classification:
• Episodic memory (facts as they
happen)
• Semantic memory (structured
knowledge)
• Procedural memory (decision-
making for actions).
S TA G E S O F C O G N I T I O N
Three Embedded Cycles:
• Acquisition Cycle: Sensing, Attention, STM, LTM
• Perception Cycle: Reasoning, Attention, Recognition, LTM
• Learning and Coordination Cycle: Learning, Planning, Action, LTM
VISUAL PERCEPTION
• Visual perception is the ability to interpret the surrounding
environment through photopic vision (daytime vision),
color vision, scotopic vision (night vision), and mesopic vision
(twilight vision), using light in the visible spectrum reflected by
objects in the environment.
VISUAL PERCEPTION
VISUAL PERCEPTIO
N
• Visual perception in cognitive robotics refers to the ability of robots to interpret and
understand their environment through visual sensors, such as cameras.
• This involves the processing of visual data to enable the robot to make decisions, navigate,
and interact with objects and other entities in its surroundings.
V I S U A L P E R C E P T I O N-
[Link] Organization
II. Perceptual Constancy
We perceive sizes, shapes, colors, and brightness to be the same even
though physical characteristics appear to be changing
II. Perceptual Constancy
II. Perceptual Constancy
Example - Robot Perception
Example - Recursive Bayesian estimation
Recursive Bayesian estimation, also known as a Bayes filter, is a general probabilistic approach
for estimating an unknown probability density function (PDF) recursively over time using incoming
measurements and a mathematical process model.
A Bayes filter is an algorithm used for calculating the probabilities of multiple beliefs to allow a robot to infer its
position and orientation
Bayes filters allow robots to continuously update their most likely position within a coordinate system, based
on the most recently acquired sensor data.
This is a recursive algorithm. It consists of two parts: prediction and Update.
Prediction: This step involves predicting the current state based on the previous state and the process
model.
Update: This step involves updating the predicted state based on the new measurement.
Example - Recursive Bayesian estimation
Bayesian filtering has several applications, including:
Kalman Filter: A recursive Bayesian filter for multivariate normal distributions. It is used for linear systems with
Gaussian noise.
Particle Filter: A sequential Monte Carlo (SMC) technique that models the PDF using a set of discrete points. It is suitable
for non-linear and non-Gaussian systems.
Basic concepts to understand Baye’s Theorem
• Joint Distribution
• A joint probability distribution allows us to calculate the probability of two or more variables co-
occurring.
• PMF(Prob mass Funct) by Chain rule
• Marginal Distributions
• Given a joint probability distribution for discrete random variables and , we
can determine the probability of an outcome for , irrespective of the value
of . We call this the marginal probability distribution of X
• We can also calculate the conditional distributions when given the joint
distribution, by taking the joint probability distribution and dividing by the
appropriate marginal
Baye’s Theorem
• Bayes’ Theorem, which allows us to infer knowledge about a variable, say the
robot state , given an observed sensor value
• Posterior probability distribution P(X|Z=z)
• Prior indicate knowledge we have about a variable before seeing evidence for it
State Estimation method with Bayes Filtering
DYNAMIC BAYES NETS IN ROBOTICS
• Simulation of a Robot
Note that this algorithm is generic for all robot models with this action/sensing structure.
Example:Modeling the world state
For our simple trash sorting robot, the only thing that matters at a given
moment is the category of the item of trash on the conveyor belt.
Remember that items of trash are presented individually to the robot, so
there is no clutter, and no circumstance in which multiple pieces of trash are
simultaneously in the workspace. Therefore, it is natural to define the world
state explicitly in terms of the category of the current item of trash.
We consider five possible categories:
• cardboard
• paper
• cans
• scrap metal
• bottle
Modeling the world state
For our simple trash sorting robot, the only thing that matters at a given moment is
the category of the item of trash on the conveyor belt. Remember that items of
trash are presented individually to the robot, so there is no clutter, and no
circumstance in which multiple pieces of trash are simultaneously in the workspace.
Therefore, it is natural to define the world state explicitly in terms of the category of
the current item of trash.
We consider five possible categories:
• cardboard
• paper
• cans
• scrap metal
• bottle
Action Model
Table 1
Let,
Sample probability mass function of cost for
• Cost -- If the category is not known with certainty, moving paper to its bin
then the cost of an action will be a random
variable
• Assume the cost as given in table 1
Expectation
…Continued
Expectation
Sensor Model
• Binary Sensors
• A binary sensor can be modeled using
conditional probability distributions (mass
function).
• Multi-Valued Sensors
• Multiple values, pmf
• Continuous Valued Sensors
• Probability density function
Perception
• When given one or more measurements, infer what the state
might be
• Maximum Likelihood
• Maximum likelihood estimation outputs the state that “agrees” most with the
measurement.
• Likelihood Function
• Likelihoods are not probabilities
• It is convenient to define the likelihood of a state X, given the measurement
(z) , as any function (L) that is proportional to the conditional density
…Continued
• Likelihood Factors
• We can use a factor to represent the likelihood of an unknown state
• Formally, a factor is simply a function from outcomes
Likelihood and Discrete Sensors
• Likelihood Factors
• We can use a factor to represent the likelihood of an unknown state
• Formally, a factor is simply a function from outcomes
• With likelihood, we can get the maximum likelihood but how to fuse multiple
sensor measurements, or how to incorporate prior information?
Likelihood and Discrete Sensors
• Likelihood Factors
• We can use a factor to represent the likelihood of an unknown state
• Formally, a factor is simply a function from outcomes
• With likelihood, we can get the maximum likelihood but how to fuse multiple
sensor measurements, or how to incorporate prior information?
• Baye’s Theorem
Another Example: A Robot Vacuum Cleaner
• Mobile Robot
• Have path planning, motion control etc
• Dependency between states
• What if the number states and the possible occurrence are too high? Finding the
conditionaljoint probability in such cases will be complex
• Due to the above two additional constraints/limitations, Bayes theorem is not
enough Dynamic Bayes Nets
How Bayes Nets reduce the full joint distribution table?
• Motivation behind Bayes nets: a technique for describing complex
joint distributions (models) using simple, local distributions
• More properly called graphical models
• Eg:
Another Eg
• Joint Probability with Bayes Net
• This is exponentially smaller than the joint
probability if we had none of the
independence assumptions given to us in
the Bayes Net.
VISUAL RECOGNITION
• Visual recognition is a crucial part of perception, involving
identifying and labelling objects.
• Example: A human or animal that navigates its environment based
on personal experiences and subjective interpretations, where its
actions and perceptions are deeply intertwined with its understanding
of the world.
VISUAL RECOGNITION CONCEPTS
• Computer Vision: The ability of a robot to interpret visual data
from the world.
• Image Processing: Techniques to enhance or transform images
(e.g., filtering, edge detection).
• Pattern Recognition: Identifying regularities in data (e.g., shapes,
textures).
• Machine Learning & Deep Learning: Use of algorithms to
improve recognition accuracy over time.
VISUAL RECOGNITION TECHNIQUES
• Object Detection: Identifying and locating objects within an image
or video.
• Image Segmentation: Partitioning an image into meaningful regions.
• Facial Recognition: Identifying or verifying a person from a digital
image.
• Optical Character Recognition (OCR): Converting different types
of documents into editable and searchable data.
• Scene Understanding: Comprehensive interpretation of the entire
environment.
V I S U A L R E C O G N I T I O N :M A C H I N E
LEARNING
• Supervised Learning: Training models on labeled data (e.g., object
classification).
• Unsupervised Learning: Finding patterns or structures in unlabeled data
(e.g., clustering).
• Reinforcement Learning: Learning through interaction with the
environment to maximize reward.
• Deep Learning: A subset of ML involving neural networks with many
layers (e.g., CNNs for image recognition).
HUMAN
VISION
V SR O B O
TIC
VISION
Example:A Trash Sorting Robot
• A Trash Sorting Robot
Example:A Warehouse Robots in 2D
A ROBOT VACUUM CLEANER
• We assume this robot is equipped with hardware and
software to perform navigation, motion planning, and
motion control.
• We will focus on high-level problems (for example,
deciding which room to clean next) without worrying
about low-level details (e.g., planning specific paths to
cover a particular room or navigating through a
doorway).
A ROBOT VACUUM CLEANER
• First, the effects of actions depend on the current world state; if a robot is
in the living room and moves to its left, it will arrive at a different location
than if it had started in the office.
• Second, the actions executed by the vacuum cleaning robot have
uncertain effects.
• Third, because the effects of actions depend on the state, achieving
goals in the future will depend on the actions the robot executes now
(since current actions affect future states).
• Therefore, this robot must consider how the world state evolves over
A ROBOT VACUUM CLEANER
•Here, states correspond to rooms in the house, and trajectories
correspond to the robot moving from room to room.
•We will model uncertain actions with conditional probability
distributions.
•Using these, we can propagate uncertainty forward in time for
specific sequences of actions and learn how to generate sample
trajectories from the corresponding probability distributions.
A ROBOT VACUUM CLEANER
• Probabilistic Modeling:
• Use conditional probability distributions to model uncertain actions and
propagate uncertainty over time.
• Perception with Limited Sensing:
• Combine action history and sensor data using hidden Markov Models (HMMs) and factor
graphs to estimate the state efficiently.
• Advanced Planning:
• Introduce Markov Decision Processes (MDPs) for sequence-based
planning and reinforcement learning to optimize action policies based on
operational data.
A ROBOT VACUUM CLEANER:
DEFINING STATES
• The representation of a robot’s state should include all information necessary for the robot to act
effectively in its environment and achieve its goals.
• For a vacuum cleaning robot, this might include the exact location of the robot (e.g., x-y
coordinates in a map of the house), the heading direction of the robot, whether the floor
is carpeted, or the location of any pets or children that might be moving throughout the house.
A ROBOT VACUUM CLEANE
R:
• Assumptions: DEFININGSTATES
• The robot can move in any direction, which means that its heading angle is not
important.
• The robot is equipped with low-level navigation software that will allow it to move
from one room to another, through doorways, etc. (though not with 100% reliability).
• The robot is equipped with path planning software to clean the floor in a particular
room (e.g., execute random motions, or follow a boustrophedon path).
• The robot is equipped with collision-avoidance software, so that it need not worry
about the presence of obstacles (e.g., furniture, small dogs, or children).
A ROBOT VACUUM CLEANER
A ROBOT VACUUM CLEANER:
DEFINING STATES
• Discrete-Time Approach:
• Focus on Discrete Moments: Time is considered at specific, distinct intervals, rather than as a
continuous flow.
• The system evolves at specific discrete moments in time, represented by time index k.
• State Representation: State changes are captured at moments of significance, such as when the
robot moves between rooms.
• Application to Vacuum Cleaning Robot:
• State Representation: Only the room the robot is currently in is tracked, ignoring continuous
motion within a room. The system's state at time tk is denoted by Xk.
• Interesting Moments: Key times are when the robot moves between rooms, executes an action,
or takes a sensor measurement.
• Probabilistic Inference for State Evolution:
• Initial State Knowledge: The robot’s initial state is known (e.g., starting in the office) - > P(X1 =
Office) = 1.
A ROBOT VACUUM CLEANER
• The State Space for Vacuum Cleaning Robot:
A ROBOT VACUUM CLEANER: STAT
ESPACE
• For the vacuum cleaning robot, room adjacency is an important relationship, and therefore it
should be encoded into our representation.
• This can be accomplished using a connectivity graph.
• Each vertex of this graph represents a state (i.e., a specific room in the house), and two
vertices, xi, xj , are connected by an edge if and only if the transition between these two states
is possible.
• Since our robot can move in any direction, if it can transition from xi to xj , it can also transition
from xj to xi (as is the case, e.g., for xi = Hallway and xj = Dining room).
• Therefore, we represent the state space by an undirected graph.
A ROBOT VACUUM CLEANER:
ACTIONS OVER TIME
• We assume that our robot is equipped with navigation software that implements four primitive
actions: move left, move right, move up, move down, which we will denote by L,R,U, and D.
• Together, these four actions define the action space. Therefore, we represent the state space by
an undirected graph.
A ROBOT VACUUM CLEANER:
PROBABILISTIC OUTCOMES OF ACTION
•Actions are uncertain.
•In the real world, robots do not always execute actions
flawlessly, for a variety of reasons. For example, a robot
may want to drive forward outdoors, but mud under its wheels
might prevenfrom traveling as far as we would like.
A ROBOT VACUUM CLEANER:
PROBABILISTIC OUTCOMES OF ACTION
• Similarly, in vacuum cleaning robot example a variety of things could go wrong: a particular
doorway might be blocked, the robot might get lost, or it might take an action that is simply
not available in a particular room - for example executing move up in the office.
• To model the uncertainty associated with executing an action, we will use conditional probability.
• The conditional probability P(Xk+1|Xk=x, A=a) to define the state transition model, i.e., the
conditional probability distribution for the next state Xk+1, given the value of the current state
X k, and the value a of the action A.
• Because the state space in this case involves navigation, we could also call the corresponding
conditional distribution P(Xk+1|Xk, A) a motion model.
A RO
BOTVAC
UUM
CLEANE
R :P R O B A B I L I S
TICOUTCOMES
OF
ACTION
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
•Dynamic Bayes nets are a special type of Bayes net used to
model systems that change over time, such as robots.
•They can be used to track a robot's state over time by
representing it’s state as a set of variables and the relationships
between those variables.
•Ancestral sampling is a method for simulating the behaviour of
a robot by sampling from a probability distribution.
• It works by sampling each variable in turn but making sure that the parents of each variable are
sampled first.
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
• Let us conceive of a simple light sensor that we can use to determine which room we are in.
• Specifically, let us assume for illustration’s sake that the living room and kitchen are on the south
side and , hence, have a lot of light on a typical day.
• The office and the dining room are on the north side, and the light sensor typically responds with
medium light levels.
• On the other hand, the hallway is very dark as it has few windows.
• We can capture these semantics with a conditional probability distribution P(Zt|Xt), where Xt is
the state at time t, and Zt ∈ {dark, medium, light} is the measured light
level.
A ROBO
T VA C U U
MCLEAN
E R : DY N A
MIC
BAYES
NETS
INROBO
TICS
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
• In perception, the game is to estimate which state we are in, given what we measured.
• For example, let's apply the maximum likelihood method here.
• In our case, given a sensor reading of, say, Z1=medium, the maximum likelihood estimate (MLE)
for the state X1 of the robot is dining room: it corresponds to the highest value
in the medium column. But that takes only one measurement into account, even if we take
many steps in the environment.
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
• We use a dynamic Bayes net, a special type of Bayes net, to reason about
multiple measurements.
• Intuitively, we should be able to use the robot’s history to obtain a better estimate of its location.
• For example, if the robot were just in the living room, it would not have gotten to the dining
room in a single step.
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
• DBN is a graphical model used to represent
systems that evolve over time.
• Here , the DBN consists of three types
of nodes:
• A nodes represent actions taken at each time
step.
• X nodes represent the system's state at each
time step.
• Z nodes represent observations
or measurements made at each time step
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
• A joint probability distribution is denoted as P(Z1:n, X1:n, A1:n).
• This represents the probability of all possible sequences of observations to Zn), states
(Z1 (X1 to Xn), and actions (A1 to An) occurring together.
• The joint probability can be factored into a product of simpler terms, thanks to the structure of
the DBN. This factorization is expressed as:
• AXZ Slice: An AXZ slice represents the conditional probabilities for a single time step
(k).
A ROBOT VACUUM CLEANER:
DYNAMIC BAYES NETS IN ROBOTICS
•Simulation of a Robot
•Simulation of a robot is then equivalent to ancestral sampling
from the DBN associated with the robot, i.e., the controlled
Markov chain augmented with the measurement CPTs.
•In this case one topological sort is rather obvious, and so is a
simulation algorithm to sample from