Intelligent Agents
CHAPTER 2
Oliver Schulte
Outline
2
Agents and environments
Rationality
PEAS (Performance measure, Environment,
Actuators, Sensors)
Environment types
Agent types
Artificial Intelligence a modern approach
The PEAS Model
3
Artificial Intelligence a modern approach
Agents
4
• An agent is anything that can be viewed as
perceiving its environment through sensors and
acting upon that environment through actuators
• Human agent:
– eyes, ears, and other organs for sensors;
– hands, legs, mouth, and other body parts for actuators
• Robotic agent:
– cameras and infrared range finders for sensors
– various motors for actuators
Artificial Intelligence a modern approach
Agents and environments
5
• The agent function maps from percept histories to
actions:
[f: P* à A]
• The agent program runs on the physical architecture to
produce f
• agent = architecture + program
Artificial Intelligence a modern approach
Vacuum-cleaner world
6
Open Source Demo
Percepts: location and contents, e.g., [A,Dirty]
Actions: Left, Right, Suck, NoOp
Agent’s function à look-up table
¡ For many agents this is a very large table
Artificial Intelligence a modern approach
Rational agents
7
• Rationality
– Performance measuring success
– Agents prior knowledge of environment
– Actions that agent can perform
– Agent’s percept sequence to date
• Rational Agent: For each possible percept sequence, a
rational agent should select an action that is expected to
maximize its performance measure, given
• the evidence provided by the percept sequence, and
• whatever built-in knowledge the agent has.
Artificial Intelligence a modern approach
Rationality
8
Rational is different from omniscience
¡ Percepts may not supply all relevant information
¡ E.g., in card game, don’t know cards of others.
Rational is different from being perfect
¡ Rationality maximizes expected outcome while
perfection maximizes actual outcome.
Artificial Intelligence a modern approach
Autonomy in Agents
The autonomy of an agent is the extent to which its
behaviour is determined by its own experience,
rather than knowledge of designer.
Extremes
¡ No autonomy – ignores environment/data
¡ Complete autonomy – must act randomly/no program
Example: baby learning to crawl
Ideal: design agents to have some autonomy
¡ Possibly become more autonomous with experience
The PEAS Framework
10
PERFORMANCE MEASURE, ENVIRONMENT,
ACTUATORS, SENSORS
Artificial Intelligence a modern approach
PEAS
11
• PEAS: Performance measure, Environment,
Actuators, Sensors
• Specifies the setting for designing an intelligent
agent
Artificial Intelligence a modern approach
PEAS: Part-Picking Robot
12
Agent: Part-picking robot
Performance measure:
Percentage of parts in
correct bins
Environment: Conveyor
belt with parts, bins
Actuators: Jointed arm
and hand
Sensors: Camera, joint
angle sensors
Artificial Intelligence a modern approach
PEAS
13
Agent: Interactive Spanish tutor
Performance measure: Maximize student's score on
test
Environment: Set of students
Actuators: Screen display (exercises, suggestions,
corrections)
Sensors: Keyboard
Artificial Intelligence a modern approach
Discussion: Self-Driving Car
14
Performance measure:
Environment:
Actuators:
Sensors:
Artificial Intelligence a modern approach
Environments
15
Artificial Intelligence a modern approach
Environment types
16
• Fully observable (vs. partially observable)
• Deterministic (vs. stochastic)
• Episodic (vs. sequential)
• Static (vs. dynamic)
• Discrete (vs. continuous)
• Single agent (vs. multiagent).
Artificial Intelligence a modern approach
Fully observable (vs. partially observable)
17
Is everything an agent requires to choose its actions
available to it via its sensors?
¡ If so, the environment is fully observable
If not, parts of the environment are unobservable.
¡ Agent must make informed guesses about world.
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Fully Partially Fully Partially Partially Fully
Artificial Intelligence a modern approach
Deterministic (vs. stochastic)
18
Does the change in world state depend only on
current state and agent’s action?
Non-deterministic environments
¡ Have aspects beyond the control of the agent
¡ Utility functions have to guess at changes in world
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Deterministic Stochastic Stochastic Stochastic Stochastic Deterministic
Artificial Intelligence a modern approach
Episodic (vs. sequential):
19
Is the choice of current action
¡ Dependent on previous actions?
¡ If not, then the environment is episodic
In sequential environments:
¡ Agent has to plan ahead:
÷ Current choice will affect future actions
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Sequential Sequential Sequential Sequential Episodic Episodic
Artificial Intelligence a modern approach
Static (vs. dynamic):
20
Static environments don’t change
¡ While the agent is deliberating over what to do
Dynamic environments do change
¡ So agent should/could consult the world when choosing actions
Semidynamic: If the environment itself does not change
with the passage of time but the agent's performance
score does.
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Static Static Static Dynamic Dynamic Semi
Another example: off-line route planning vs. on-board
navigation system
Artificial Intelligence a modern approach
Discrete (vs. continuous)
21
A limited number of distinct, clearly defined percepts and
actions vs. a range of values (continuous)
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Discrete Discrete Discrete Conti Conti Conti
Artificial Intelligence a modern approach
Single agent (vs. multiagent):
22
An agent operating by itself in an environment vs.
there are many agents working together
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Single Multi Multi Multi Single Single
Artificial Intelligence a modern approach
Discussion: Self-Driving Car
23
Observable Deterministic Episodic Static Discrete Agents
Self-Driving partially non- sequential dynamic continuous multi-
Car deterministic agent
Apple self-driving car was rear-ended by Nissan Leaf
Artificial Intelligence a modern approach
Summary.
Observable Deterministic Episodic Static Discrete Agents
Cross Word Fully Deterministic Sequential Static Discrete Single
Poker Partially Stochastic Sequential Static Discrete Multi
Backgammon Fully Stochastic Sequential Static Discrete Multi
Taxi driver Partially Stochastic Sequential Dynamic Conti Multi
Part picking Partially Stochastic Episodic Dynamic Conti Single
robot
Image Fully Deterministic Episodic Semi Conti Single
analysis
Artificial Intelligence a modern approach
Agents
26
AGENT TYPES
LEARNING
Artificial Intelligence a modern approach
Agent types
27
Four basic types in order of increasing generality:
¡ Simple reflex agents
¡ Reflex agents with state/model
¡ Goal-based agents
¡ Utility-based agents
¡ All these can be turned into learning agents
Artificial Intelligence a modern approach
Simple reflex agents
28
Artificial Intelligence a modern approach
Vacuum Cleaner Reflex Agent
29
Robot forgets past, knows only current square
History State Action
[A,Clean] [A,Clean] Right
[A,Clean,Right [B,Dirty] Suck
;
B, Dirty]
[A,Clean,Right [B, Clean] Left
;
B, Dirty, Suck;
B, Clean]
[A,Clean,Right [A,
; Clean]
B, Dirty, Suck;
B, Clean, Left;
Artificial Intelligence a modern approach
A, Clean]
Simple reflex agents
30
Simple but very limited intelligence.
Action does not depend on percept history, only on
current percept.
¡ Thermostat.
ª Therefore no memory requirements.
Infinite loops
¡ Suppose vacuum cleaner does not observe location. What
do you do given location = clean? Left on A or right on B -
> infinite loop.
¡ Fly buzzing around window or light.
¡ Possible Solution: Randomize action.
Artificial Intelligence a modern approach
States: Beyond Reflexes
31
• Recall the agent function that maps from percept
histories to actions:
[f: P* à A]
An agent program can implement an agent function
by maintaining an internal state (memory)
¡ e.g. cell phone knows its battery usage
The internal state can contain information about the
state of the external environment.
The state depends on the history of percepts and on
the history of actions taken:
[f: P*, A*à S àA] where S is the set of states.
Artificial Intelligence a modern approach
State-based reflex agents
32
Update state = remember history
• Many (most?) state-of-the-art
systems for open-world problems
follow this architecture
• (e.g. translation)
• No thinking
Artificial Intelligence a modern approach
Model-based reflex agents
33
Know how world evolves
¡ Overtaking car gets closer from
behind
Predict how agents actions affect
the world
¡ Wheel turned clockwise takes you
right
Model-based agents predict
consequences of their actions
state ç UPDATE-STATE(state,action,percept,model)
Artificial Intelligence a modern approach
Goal-based agents
34
• knowing state and environment? Enough?
Car can go left, right, straight
–
• Has a goal
¡ A destination to get to
Uses knowledge about a goal to guide its actions
¡ E.g., Search, planning
Artificial Intelligence a modern approach
Goal-based agents
35
• Reflex agent brakes when it sees brake lights. Goal based agent
reasons
– Brake light -> car in front is stopping -> I should stop -> I should use brake
Artificial Intelligence a modern approach
Example
36
The Monkey and Banana Problem
Monkeys can use a stick to grasp a hanging banana
Artificial Intelligence a modern approach
Utility-based agents
37
Goals are not always enough
¡ Many action sequences get car to destination
¡ Consider other things. How fast, how safe…..
A utility function maps a state onto a real number
which describes the associated degree of
“happiness”, “goodness”, “success”.
Where does the utility measure come from?
¡ Economics: money.
¡ Biology: number of offspring.
¡ Your life?
Artificial Intelligence a modern approach
Utility for Self-Driving Cars
38
What is the performance metric?
Safety - No accidents
Time to destination
What if accident is unavoidable? E.g.
¡ is it better to crash into an old person than into a child?
¡ How about 2 old people vs. 1 child?
Artificial Intelligence a modern approach
Utility-based agents
39
Artificial Intelligence a modern approach
Learning agents
40
Performance element is
what was previously the
whole agent
Input sensor
Output action
Learning element
Modifies performance
element.
Artificial Intelligence a modern approach
Learning agents (Self-Driving Car)
42
¡ Performance element
÷ How it currently drives
¡ Actuator (steering): Makes quick lane change
¡ Sensors observe
÷ Honking
÷ Sudden Proximity to other cars in the same lane
¡ Learning element tries to modify performance elements for future
÷ Problem generator suggests experiment: try out something called
Signal Light
¡ Exploration vs. Exploitation
÷ Exploration: try something new
+ Improved Performance in the long run
- Cost in the short run
Artificial
The Big Picture: AI for Model-Based Agents
43
Planning
Action Reinforcement
Decision Theory
Learning
Game Theory
Knowledge Learning
Logic Machine Learning
Probability Statistics
Heuristics
Inference
Artificial Intelligence a modern approach
The Picture for Reflex-Based Agents
44
Action
Reinforcement
Learning
Learning
• Studied in AI, Cybernetics, Control Theory, Biology,
Psychology.
• Skinner box
Artificial Intelligence a modern approach
Discussion Question
45
Model-based reasoning has a large overhead.
Our large brains are very expensive from an
evolutionary point of view.
Why would it be worthwhile to base behaviour on a
model rather than “hard-code” it?
For what types of organisms in what type of
environments?
¡ The dodo is an example of an inflexible animal
Artificial Intelligence a modern approach
Summary
46
Agents can be described by their PEAS.
Environments can be described by several key properties:
64 Environment Types.
A rational agent maximizes the performance measure for
their PEAS.
The performance measure depends on the agent
function.
The agent program implements the agent function.
4 main architectures for agent programs.
In this course we will look at some of the common and
useful combinations of environment/agent architecture.
Artificial Intelligence a modern approach