Duration: 240 mins BCSE427L Cognitive Robotics
Module 2: Cybernetic View of Robotic
Cognition and Perception
1. [Link]
2. Cybernetic View of Robot Cognition and Perception – Chapter 1
(PDF attached in MS Teams)
Session Overview
• Introduction to the Model of Cognition
• Visual Perception
• Visual Recognition
• Machine Learning, and Robot Cognition
Humans Vs Machines with respect to Visual Perception
Robots/Machines perform better in the below examples
Humans are superior in the below recognition
Robotic Perception / Machine Vision
Image Acquisition
How important is image acquisition?
Example 1: Example 2:
Electromagnetic Spectrum
Optical Imaging
Optical Imaging
Radiometry
• The field of photometry is concerned with measuring the intensity and
power of visible light with respect to the sensitivity of the human eye
What is Perception Model?
• It is the inverse of Sensor Model
• When given a sensor input, what is the state of the system →
Perception Model
Modeling the world state
For our simple trash sorting robot, the only thing that matters at a given moment is
the category of the item of trash on the conveyor belt. Remember that items of trash
are presented individually to the robot, so there is no clutter, and no circumstance in
which multiple pieces of trash are simultaneously in the workspace. Therefore, it is
natural to define the world state explicitly in terms of the category of the current item
of trash.
We consider five possible categories:
•cardboard
•paper
•cans
•scrap metal
•bottle
Action Model
Table 1
Let,
Sample probability mass function of cost for
• Cost -- If the category is not known with certainty, moving paper to its bin
then the cost of an action will be a random
variable
• Assume the cost as given in table 1
Expectation
…Continued
Expectation
Sensor Model
• Binary Sensors
• A binary sensor can be modled using conditional
probability distributions (mass function).
• Multi-Valued Sensors
• Multiple values, pmf
• Continuous Valued Sensors
• Probability density function
Perception
• When given one or more measurements, infer what the state might be
• Maximum Likelihood
• Maximum likelihood estimation outputs the state that “agrees” most with the
measurement.
• Likelihood Function
• Likelihoods are not probabilities
• It is convenient to define the likelihood of a state X, given the measurement
(z),as any function (L) that is proportional to the conditional density
…Continued
• Likelihood Factors
• We can use a factor to represent the likelihood of an unknown state
• Formally, a factor is simply a function from outcomes
•
Likelihood and Discrete Sensors
• Likelihood Factors
• We can use a factor to represent the likelihood of an unknown state
• Formally, a factor is simply a function from outcomes
•
• With likelihood, we can get the maximum likelihood but how to fuse multiple
sensor measurements, or how to incorporate prior information?
Likelihood and Discrete Sensors
• Likelihood Factors
• We can use a factor to represent the likelihood of an unknown state
• Formally, a factor is simply a function from outcomes
•
• With likelihood, we can get the maximum likelihood but how to fuse multiple
sensor measurements, or how to incorporate prior information?
• Baye’s Theorem
Basic concepts to understand Baye’s Theorem
• Joint Distribution
• A joint probability distribution allows us to calculate the probability of two or more variables co-
occurring.
• pmf by Chain rule
• Marginal Distributions
• Given a joint probability distribution for discrete random variables and , we can determine the
probability of an outcome for , irrespective of the value of . We call this the marginal probability
distribution of X
• We can also calculate the conditional distributions when given the joint distribution, by taking the joint
probability distribution and dividing by the appropriate marginal
•
Baye’s Theorem
• Bayes’ Theorem, which allows us to infer knowledge about a variable, say the robot state , given an
observed sensor value
• Posterior probability distribution P(X|Z=z)
• Prior → indicate knowledge we have about a variable before seeing evidence for it
Another Example: A Robot Vacuum Cleaner
• Mobile Robot
• Have path planning, motion control etc
• Dependency between states
• What if the number states and the possible occurrence are too high? Finding the
conditional joint probability in such cases will be complex
• Due to the above two additional constraints/limitations, Bayes theorem is not
enough → Dynamic Bayes Nets
How Bayes Nets reduce the full joint distribution table?
• Motivation behind Bayes nets: a technique for describing complex joint
distributions (models) using simple, local distributions
• More properly called graphical models
• Eg:
Another Eg
• Joint Probability with Bayes Net
• This is exponentially smaller than the joint
probability if we had none of the
independence assumptions given to us in the
Bayes Net.
…Continued….
State Space
…Continued….
• Sample CPT
State Transition Matrices and the Belief State
Controlled Markov Chain
• Previous State and Action leads to
current state
• Another Example
Eg:
• Previous State and Action leads to
current state
• Another Example
Example:
• Let P(W|X,Z)= 0.05, P(X|Y,Z)=0.81,
P(Y|Z)=0.75,P(Z)=0.998
• Then P(W, X, Y, Z)
=0.05*0.81*0.75*0.998
=0.03013
Another Example
• Can we answer the question below?
• Calculate the probability that alarm has sounded,
but there is neither a burglary, nor an earthquake
occurred, and David and Sophia both called the
Harry
• Also identify how many computations are reduced
0.001
• P(S, D, A, ~B, ~E) =
P(S|A).P(D|A).P(A|~B,~E).P(~B).P(~E)
• =0.75*0.91*0.001*0.998*0.999
• =0.006
States of Robotic Perception
Visual Perception
Robotic Visual Perception
• Rapidity
• Compactness
Human Visual System
• Robustness
Active Perception
• Active perception is the study of perception strategies including sensor and signal
processing cooperation, to achieve knowledge about the environment
Visual Recognition
• Representation
• Featurespace, predicates, graph, etc
• No. of Representations / object
• 2D or 3D
• No. of Classes for mapping to representation
• Spatial relationship to objects
• Amount of processing required
Representation
• The primary issues in the visual recognition process are representations and
search, which means how to develop an appropriate representation for the
objects and then how to search them efficiently for a match at the time of
recognition
• Traditional Representation & Recognition techniques
• Template Matching
• Feature Based Model
• Fourier Model
• Structural Model
Template Matching
• Template matching is the simplest form of representation in which a replica of the
retinal stimulation pattern projected by a shape is stored in LTM.
• The recognition process compares all stored object templates with the input array
by selecting the best match based on the ratio of matching to non-matching
objects
• Advantages: Simple
• Disadvantages: False results, Change will produce different outcome, Any
Occlusion, shadow result in accurate matching
Feature-Based Model
• Instead of storing templates for entire shapes, the feature-based model util- izes
a series of feature detectors
• Generally, the features included are of a geometric type such as vertical and
horizontal lines, curves and angles
• Feature detectors may be used either at every position in the input array, or may
be used for the global image
• In case of multiple feature detectors, the degree of matching is estimated for the
target feature with respect to each section of the input array
Fourier Model
• In the Fourier model, a two-dimensional input array is subjected to a spatial
Fourier analysis
• In this model, the original array is decomposed into a set of spatial frequency
components of various orientations and frequencies in the form of sinusoidal
waveforms
• The amplitude and phase are both recorded for the spectrum of spatial
frequencies and angles
• The feature of this model is that it gives no restriction on angles, frequencies and
no computational problems, such as aliasing
Structural Model
• The structural model contains information about the relative positions and
relationships between parts of an object
• This structural description is stored in memory in the form of a data structure
such as a list or tree or graph of predicates
• The advantage of structural representation is that it factors apart the information
in a scene without losing any part of it
Machine Learning
• A computer program is said to learn from experience E with respect to some class
of tasks T and performance measure P, if its performance at tasks in T as
measured by P improves with experience E
Factors to be considered
• Choosing the training experience
• Choosing the target function
• Choosing the representation of a target function
• Choosing a function approximation algorithm
Types of ML
• Supervised
• Unsupervised Learning
• Reinforcement Learning
• In reinforcement learning, the learner does not explicitly know the input–
output instances, but it receives some form of feedback from its environment
Modeling Cognition Using ANN
• Sensing the external environment and storing it in the form of knowledge
• Applying the knowledge stored to solve problems
• Acquiring new knowledge through experience
Fuzzy Logic in Robot Cognition
• The methodology of fuzzy logic appears very useful when the processes are too
complex for analysis by conventional quantitative techniques or when the
available sources of information are interpreted qualitatively, inexactly, or
uncertainly, which is the case with mobile robots
Genetic Algorithms in Robot Cognition
• Genetic algorithms (GAs) are stochastic in nature, and mimic the natural process
of biological evolution