Chapter Two

Intelligent Agent
 Objective:
 Define what an agent is in general
 Agents and environments
 Gives ideas about agent, agent function, agent program and
architecture, environment, percept, sensor, actuator (effectors),
 Gives idea on how agent should act
 How to measure agent success
 Rational agent, autonomous agent
 Types of environment
 Types of agent

1
 What is an agent?
 Is any thing that can be viewed as perceiving its environment
through sensors and acting upon the environment through the
effectors
 Examples: Human agent; Robotic agent; Software agent

 Human-Agent: A human agent has eyes, ears, and other organs

which work for sensors and hand, legs, vocal tract work for
actuators.
 Robotic Agent: A robotic agent can have cameras, infrared
range finder, NLP for sensors and various motors for actuators.
 A software agent receives keystrokes, file contents, and
network packets as sensory inputs and acts on the environment
by displaying on the screen, writing files, and sending network
packets.
2
  Agents and environments
 Sensor: is a device which detects the change in the
environment and sends the information to other electronic
devices. An agent observes its environment through sensors.
 Actuators: are the component of machines that converts energy
into motion. The actuators are only responsible for moving and
controlling a system. An actuator can be an electric motor,
gears, rails, etc.
 Effectors: are the devices which affect the environment.
Effectors can be legs, wheels, arms, fingers, wings, fins, and
display screen.

External characterization :
Agents interact with
environments through sensors
and actuators.
3
 Cont’d…
 Percept refer to the agent’s perceptual inputs at any given
instant.
 Percept Sequence is the complete history of everything that the
agent has ever perceived.
 The current percept, or a sequence of percept can influence the
actions of an agent.
 Mathematically, we say that an agent’s behavior is described by
the agent function that maps any given percept sequence/
history to an action. [f: P ∗ → A]
 Internally, the agent function for an artificial agent will be
implemented by an agent program.
 The agent program runs on the physical architecture to produce
f. Agent = architecture + program
4
  Ideal Example of Agent
 The vacuum cleaner world
 has just two locations: squares A and B.
 The vacuum agent
 Perceives which square it is in and whether there is dirt in the square.
 Can choose to move left, move right, suck up the dirt or do nothing
 Agent function
 If the current square is dirty, then suck; otherwise move to the other square.

A vacuum-cleaner world with just two

locations. Partial tabulation of a simple agent function for the
5 vacuum-cleaner world
 How should Agents act?
 Agents may be rational or human like
 We have seen how human act or think is difficult to understand
since due to the complex structure of human intelligence.
 Our agent should be designed from rationality view that act
rationally.
 A rational agent is an agent that does the right thing for the
perceived data from the environment - conceptually speaking,
every entry in the table for the agent function is filled out
correctly.
 What is right is an ambiguous concept but we can consider the
right thing as the one that makes the agent more successful.
 Success is also measured by using performance measure
 Question: how and when do you measure success in
6
performance?
 Performance measure (how?)
 Subjective measure using the agent
 How happy is the agent at the end of the action
 Agent should answer based on his opinion
 Some agents are unable to answer, some are delude them
selves, some over estimate and some under estimate their
success
 Therefore, subjective measure is not a better way.
 Objective Measure imposed by some authority is an
alternative

7
 Objective Measure

Needs standard to measure success
 Provides quantitative value of success measure of agent
 Involves factors that affect performance and weight to each
factors
 E.g., performance measure of a vacuum-cleaner agent could be
 amount of dirt cleaned up,
 amount of time taken,
 amount of electricity consumed,
 amount of noise generated, etc.
 The time to measure performance is also important for success.
 It may include knowing starting time, finishing time, duration of
job, etc
8
 Omniscience vs Rational Agent
 Omniscience agent is distinct from Rational agent
 Omniscience agent is an agent that knows the actual outcome
of its action and can act accordingly; but perfection is
impossible in reality.
 Rationality is NOT the same as perfection.
 Rationality maximizes expected performance, while perfection
maximizes actual performance
 Rational agent could make a mistake because of unpredictable
factors at the time of making decision.
 A rational agent not only to gather information (exploration)
but also to learn as much as possible from what it perceives.
 Omniscient agent that act and think rationally never make a
mistake
 An agent relies on the prior knowledge of its designer rather
than on its own percepts, we say that the agent lacks autonomy.
A rational agent should be autonomous.
9
 Factors to measure rationality of agents
 What is rational at any given time depends on four things:
1. Percept sequence perceived so far (do we have the entire
history of how the world evolve or not)
2. The set of actions that the agent can perform (agents
designed to do the same job with different action set will
have different performance)
3. Performance measures ( is it subjective or objective? What
are the factors and their weights)
4. The agent prior knowledge about the environment (what
kind of sensor does the agent have? Does the agent knows
every thing about the environment or not)
 This leads to the concept of Ideal Rational Agent //definition
of a rational agent
10
 Ideal rational Agent
 For each possible percept sequence, an ideal rational agent
should do what every action is expected to maximize its
performance measure, on the basis of the evidence
provided by the percept sequence and what ever built-in
knowledge the agent has.
 Ideal rational agent implementation require perfection
 In real situation such agent is difficult to achieve
 Why car accident happened? Because drivers are not
perfect agent

11
 Autonomy
 An agent is autonomous if its behavior is determined by its
own experience (with ability to learn and adapt)
 Agent that lacks autonomous, if its actions are based
completely on built-in knowledge
 Example: student grade decider agent:
 Knowledge base given: rules for converting numeric grade to letter
grade
 Case 1: agent always follows the rule (lacks autonomous)
 Case 2: agent that modify the rules by learning exceptions from the
knowledge base as well as grade distribution.

12
 Specifying the task environment
 In designing an agent, the first step must always be to specify the
task environment as fully as possible.
 Task environment is the description of performance, environment,
actuators, and sensors.
 Performance measure  Environment  Actuators  Sensors are
abbreviated as PEAS
 Percepts Actions Goal  Environment are abbreviated as
PAGE

13
 Examples of agents structure and sample PEAS
 Agent: automated taxi driver:
 Environment: Roads, traffic, pedestrians, customers
 Sensors: Cameras, sonar, speedometer, GPS, odometer, engine
sensors, keyboard
 Actuators: Steering wheel, accelerator, brake, signal, horn
 Performance measure: Safe, fast, legal, comfortable trip,
maximize profits
 Agent: Medical diagnosis system
 Environment: Patient, hospital, physician, nurses, …
 Sensors: Keyboard (percept can be symptoms, findings, patient's
answers)
 Actuators: Screen display (action can be questions, tests,
diagnoses, treatments, referrals)
 Performance measure: Healthy patient, minimize costs, lawsuits
14
 Examples of agents structure and sample PEAS
 Agent: Interactive English tutor
 Environment: Set of students
 Sensors: Keyboard (typed words)
 Actuators: Screen display (exercises, suggestions, corrections)
 Performance measure: Maximize student's score on test
 Agent: Satellite image analysis system
 Environment: Images from orbiting
 Sensors: Pixels of varying intensity, color
 Actuators: print categorization of scene
 Performance measure: Correct categorization
 Agent: Part picking robot
 Environment: Conveyor belt with parts
 Sensors: pixels of varying intensity
 Actuators: pickup parts and sort into bins
 Performance measure: place parts in correct bins
15
 Types of Environment
 Based on the portion of the environment observable
 Fully observable: An agent's sensors give it access to the complete
state of the environment at each point in time.
 Fully observable environments are convenient because the agent
need not maintain any internal state to keep track of the world.
 Partially observable: An environment might be partially observable
because of noisy and inaccurate sensors.
 Fully unobservable
 Based on the effect of the agent action
 Deterministic : The next state of the environment is completely
determined by the current state and the action executed by the agent.
 Strategic: If the environment is deterministic except for the actions of
other agents, then the environment is strategic
16
 Stochastic or probabilistic
Types of Environment cont …
 Based on the number agent involved
 Single agent A single agent operating by itself in an
environment.
 Multi-agent: multiple agents are involved in the environment

 Based on the state, action and percept space pattern

 Discrete: A limited number of distinct, clearly defined state,
percepts and actions.
 Continuous: state, percept and action are consciously changing
variables
 Note: one or more of them can be discrete or continuous

17
 Types of Environment cont …
 Based on the effect of time
 Static: The environment is unchanged while an agent is deliberating.
 Dynamic: The environment changes while an agent is not
deliberating.
 semi-dynamic: The environment is semi-dynamic if the environment
itself does not change with the passage of time but the agent's
performance score does
 Based on loosely dependent sub-objectives
 Episodic: The agent's experience is divided into atomic "episodes"
(each episode consists of the agent perceiving and then performing a
single action), and the choice of action in each episode depends only
on the episode itself.
 Sequential: The agent's experience is a single atomic "episodes"
18
 Environment types example

Chess with Chess without Taxi driving

a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No

 The environment type largely determines the agent design

 The real world is (of course) partially observable, stochastic,
sequential, dynamic, continuous, multi-agent

19
 Structure of Intelligent Agent
 Structure of AI Agent refers to the design of intelligent agent
program (function that implement agent mapping from percept to
actions) that will run on some sort of computing device called
architecture
 This course focus on intelligent agent program function theory,
design and implementation
 Design of intelligent agent needs prior knowledge of
 Performance measure or Goal the agent supposed to achieve,
 On what kind of Environment it operates
 What kind of Actuators it has (what are the possible Actions),
 What kind of Sensors the agent has (what are the possible
Percepts)

20
 Agent programs
 An agent is completely specified by the agent function that maps
percept sequences into actions
 Aim: find a way to implement the rational agent function
concisely
 Skeleton of the Agent
FUNCTION SKELETON-AGENT (percept) returns action
static memory, the agent’s memory of the world
memory UPDATE-MEMORY (memory, percept)
action  CHOOSE-BEST-ACTION (memory)
memory UPDATE-MEMORY (memory, action)
RETURN action
Note:
1. the function gets only a single percept at a time
21
 Table-lookup agent
 Table lookup agent store all the percept sequences –action pair
into the table
 For each percept, this type of agent will search for the percept
entry and return the corresponding actions.
 Table look up couldn’t be the right option to implement
successful agent
 Why?
 Drawbacks:
 Huge table
 Take a long time to build the table
 No autonomy
 Even with learning, need a long time to learn the table entries
22
 Agent types
 Based on memory of the agent, and the way the agent takes
action we can divide agents into five basic types:
 These are (according to their increasing order of generality) :
1. Simple reflex agents
2. Model-based reflex agents
3. Goal-based agents
4. Utility-based agents
5. Learning agent
Each will be discussed soon with models.
Notation of model:
 Rectangles: used to represent the current internal state of the
agent decision process
 Ovals: used to represent the background information used in
23
the process
Simple reflex agents

 These agents take decisions on the basis of the current

percepts and ignore the rest of the percept history.
 These agents only succeed in the fully observable
environment.
 The Simple reflex agent does not consider any part of
percepts history during their decision and action process.
 The Simple reflex agent works on Condition-action rule,
which means it maps the current state to action. Such as a
Room Cleaner agent, it works only if there is dirt in the
room.

24
Simple reflex agents
 Problems for the simple reflex agent design approach:
 They have very limited intelligence
 They do not have knowledge of non-perceptual parts of the
current state
 Mostly too big to generate and to store.
 Not adaptive to changes in the environment.

25
 Simple reflex agent function prototype
 function SIMPLE-REFLEX-AGENT(percept) returns an action
persistent: rules, a set of condition–action rules

state ← INTERPRET-INPUT(percept)
rule ← RULE-MATCH(state, rules)
action ← [Link]
return action

 A simple reflex agent. It acts according to a rule whose

condition matches the current state, as defined by the percept.

26
 Model-based reflex agents
(also called a reflex agent with internal state)
 It can handle partially observable environments.
 Needs memory for storing the percept history, it uses the
percept history to help revealing the current unobservable
aspects of the environment (maintains internal state).
 The agent combines current percept with the internal state to
generate updated description of the current state.
 Updating the state requires information about
 How the world evolves independently from the agent
 How the agent actions affects the world.

27
Model-based reflex agents
(also called a reflex agent with internal state)

28
 Model-based reflex agents
 function MODEL-BASED-REFLEX-AGENT(percept) returns an action

 persistent: state, the agent’s current conception of the world state

model, a description of how the next state depends on current state and action
rules, a set of condition–action rules
action, the most recent action, initially none

state ← UPDATE-STATE(state, action, percept, model)
rule ← RULE-MATCH(state, rules)
action ← [Link]
return action

 A model-based reflex agent. It keeps track of the current state of the world,
using an internal model. It then chooses an action in the same way as the reflex
agent.
29
Goal-based agents
 Knowing current state of environment often is not enough to
decide an action
 Need of goal information
 Combine goal info with possible actions to choose that
achieve the goal
 Not easy always to decide best action to take
 Use search planning to find action sequences that achieve
goal
To finding action sequences
that achieve the agent's goals.

30
Goal-based agents

31
 Goal-based agents structure
 Function GOAL_BASED_AGENT(PERCEPT) return action
static state, a description of the current world state
goal, a description of the goal to achieve may be in
terms of state
stateUPDATE_STATE(state, percept)
actionSetPOSSIBLE_ACTIONS(state)
actionACTION_THAT_LEADS_TO_GOAL(actionSet)
stateUPDATE_STATE(state,action)
return action

32
Utility-based agents
 Goals not enough for high-quality behavior.
 In Goal-based agents states are classified as successful
unsuccessful
 We need a method to distinguish the level of utility or gain
in a state
 Utility: A function which maps a state (successful) into a
real number (describes associated degree of success)
 Utility functions allow for:
 Specifying tradeoffs in conflicting or alternative goals
 Specifying a way in which likelihood of success can be
weighed up against the importance of alternative goals

33
Utility-based agents

34
Utility-based agents structure
 Function UTILITY_BASED_AGENT(PERCEPT) return action
static state, a description of the current world state
goal, a description of the goal to achieve may be in
terms of state
stateUPDATE_STATE(state, percept)
actionSetPOSSIBLE_ACTIONS(state)
actionBEST_ACTION(actionSet)
stateUPDATE_STATE(state,action)
return action

35
Learning agents
 is an agent that augments the performance element which
determines actions from percept sequences / past experiences or
it has learning capabilities.
 Four conceptual components:
 Learning element : makes improvements to the agent’s
knowledge
 Critic: gives feedback to the learning element based on an
external performance standard (from user or examples, good or
not?)
 Problem generator: suggests actions that lead to new and
informative experiences.
 Performance element: Selecting external actions or what we
took for the whole agent above.
36
Learning agents

37
Summary
 Agents interact with environments through actuators and
sensors
 The agent function describes what the agent does in all
circumstances
 The performance measure evaluates the environment sequence
 A rational agent maximizes expected performance
 Agent programs implement (some) agent functions
 PEAS descriptions define task environments
 Environments are categorized along several dimensions:
 observable? deterministic? episodic? static? discrete? single-agent?

 Several basic agent architectures exist:

 reflex, reflex with state, goal-based, utility-based
38