CHAPTER TWO

INTELLIGENT AGENT

1
OBJECTIVE
 What is intelligent agent

 How it is related to its environment,

 Concept of rationality and how the performance of a rational agent is

evaluated.

 How to build intelligent agent.

 Types of environment

 Types of agent

2
• What is an agent?

• Is any thing that can be viewed as perceiving its

environment through sensors and acting upon
the environment through the effectors.

3
• Human as an agent has eyes, ears, and other organs for
sensors; and hands, legs, mouth, and other organs as
effectors.

• Robots as an agent has camera, sound recorder, infrared

range finder for sensors; and various motors for effectors.

• Software as an agent has Keystrokes, file contents,

received network packages which act as sensory inputs and
displays on the screen, writing files, and sent network
packets as output. 4
 HOW AGENTS SHOULD ACT?
• A rational agent is one that does the right thing.
Obviously, this is better than doing the wrong thing, but
what does it mean?

• As a first approximation, we will say that the right action is

the one that will cause the agent to be most successful.

• That leaves us with the problem of deciding how and

when to evaluate the agent's success.
5
 HOW AGENTS SHOULD ACT
• We use the term performance measure for the how criteria that determine how
successful an agent is.

 Performance measure (how?)

 Subjective measure using the agent

 How happy is the agent at the end of the action

 Agent should answer based on his opinion. Some agents are unable to
answer, some are delude them selves, some over estimate and some under
estimate their success. Therefore, subjective measure is not a better way.

 Objective Measure imposed by some authority is an alternative

• In other words, we as outside observers establish a standard of what it means
6
to be successful in an environment and use it to measure the performance of
agents.
 Objective Measure

 Needs standard to measure success

 Provides quantitative value of success measure of agent

 Involves factors that affect performance and weight to each

factors

The time to measure performance is also important for success.

It may include knowing starting time, finishing time, duration of

7
job, etc
 IDEAL EXAMPLE OF AGENT
❖Vacuum-cleaner world
How the agent should behave
• Percepts: location and ✓ amount of dirt cleaned up
contents ✓ amount of time taken,
✓ amount of electricity consumed,
• e.g., [A, Dirty]
✓ amount of noise generated, etc.
⚫ Actions: Left, Right, Suck Performance measure based on what
one actually want
✓ If we measured how much dirt the
agent had cleaned up in the first
hour of the day, we would be
rewarding those agents that start
fast (even if they do little or no
work later on), and punishing those
that work consistently.
✓ Thus, we want to measure
performance over the long run. 8
• Consider the following example:

• I am walking along the street one day and I see an old friend
across the street. There is no traffic nearby and I'm not
otherwise engaged, so, being rational, I start to cross the street.
Meanwhile, at 33,000 feet, a cargo door falls off a passing
airliner, and before I make it to the other side of the street I am
flattened. Was I irrational to cross the street?

❖ Rationality is concerned with expected success given what

has been perceived
9
• We cannot blame an agent for failing to take into
account something it could not perceive, or for failing
to take an action that it is incapable of taking.

• But relaxing the requirement of perfection is not just a

question of being fair to agents.

• The point is that if we specify that an intelligent agent should

always do what is actually the right thing, it will be impossible to
design an agent to fulfill this specification unless we improve the
performance of crystal balls. 10
• What is rational at any given time depends on four things:

✓The performance measure that defines degree of success.

✓Everything that the agent has perceived so far. We will call

this complete perceptual history the percept sequence.

✓What the agent knows about the environment.

✓The actions that the agent can perform.

11
 IDEAL RATIONAL AGENT

▪ For each possible percept sequence, an ideal rational agent should

Select an action that is expected to maximize its performance

measure, on the basis of the evidence provided by the percept

sequence and what ever built-in knowledge the agent has.

• An omniscient agent is an agent which knows the actual outcome of its action

in advance. However, such agents are impossible in the real world.

 Ideal rational agent implementation require perfection

 In real situation such agent is difficult to achieve 12

 Why car accident happened? Because drivers are not perfect agent
• The notion of an agent is meant to be a tool for analyzing

systems, not an absolute characterization that divides the world

into agents and non-agents.

• Consider a clock. It can be thought of as just an inanimate object,

or it can be thought of as a simple agent.

• As an agent, most clocks always do the right action: moving their

hands (or displaying digits) in the proper fashion.

13
• Clocks are a kind of degenerate agent in that their
percept sequence is empty; no matter what happens
outside, the clock's action should be unaffected.

• Well, this is not quite true. If the clock and its owner
take a trip from Ethiopia to Australia, the right thing for
the clock to do would be to turn itself back eight hours.

• We do not get upset at our clocks for failing to do this

because we realize that they are acting rationally, given
their lack of perceptual equipment. 14
• Once we realize that an agent's behavior depends only
on its percept sequence to date, then we can describe
any particular agent by making a table of the action it
takes in response to each possible percept sequence.

• Such a list is called a mapping from percept sequences to actions.

• if mappings describe agents, then ideal mappings

describe ideal agents.

• Specifying which action an agent ought to take in

response to any given percept sequence provides a 15

design for an ideal agent.

• If the agent's actions are based completely on built-in

knowledge, then we say that the agent lacks autonomy.

• For example, if the clock manufacturer was prescient enough to

know that the clock's owner would be going to Australia at some

particular date, then a mechanism could be built in to adjust the

hands automatically by eight hours at just the right time.

• This would certainly be successful behavior, but the intelligence seems to

belong to the clock's designer rather than to the clock itself. 16

• An agent's behavior can be based on both its own experience and

the built-in knowledge used in constructing the agent for the

particular environment in which it operates.

• A system is autonomous to the extent that its behavior is

determined by its own experience.

• An agent that operates on the basis of built-in assumptions will

only operate successfully when those assumptions hold, and thus

lacks flexibility. 17
 Autonomy

 An agent is autonomous if its behavior is determined by its

own experience (with ability to learn and adapt).

 Agent that lacks autonomous, if its actions are based

completely on built-in knowledge.

 Example: student grade decider agent:

 Knowledge base given: rules for converting numeric

grade to letter grade

 Case 1: agent always follows the rule (lacks autonomous)

 Case2: agent that modify the rules by learning exceptions 18

from the knowledge base as well as grade distribution.

 STRUCTURE OF INTELLIGENT AGENT
• To understand the structure of Intelligent Agents, we should be
familiar with Architecture and Agent programs.

• The job of AI is to design the agent program: a function that

implements the agent mapping from percepts to actions.

• We assume this program will run on some sort of

ARCHITECTURE computing device, which we will call the
architecture.

• Architecture is the machinery that the agent executes on. It is a

device with sensors and actuators, for example, a robotic car, a
19
camera, a PC.
AGENTS AND ENVIRONMENTS
• Percept: the agent’s perceptual inputs.

• Percept sequence: the complete history of everything the agent has

perceived.

• The agent function maps from percept histories to actions:

[f: P* → A]

• The agent program runs on the physical architecture to produce f. 20

agent = architecture + program

STRUCTURE OF INTELLIGENT AGENT

 Design of intelligent agent needs prior knowledge of

 Performance measure or Goal the agent supposed to achieve,

 On what kind of Environment it operates

 What kind of Actuators it has (what are the possible Actions),

 What kind of Sensors the agent has (what are the possible

Percepts)

 Performance measure → Environment → Actuators → Sensors 21

are abbreviated as PEAS.

EXAMPLES OF AGENTS STRUCTURE AND SAMPLE PEAS

 Agent: automated taxi driver:

 Environment: Roads, traffic, pedestrians, customers

 Sensors: Cameras, sonar, speedometer, GPS, odometer, engine sensors,

keyboard

 Actuators: Steering wheel, accelerator, brake, signal, horn, display

 Performance measure: Safe, fast, legal, comfortable trip, maximize profits.

22
EXAMPLES OF AGENTS STRUCTURE AND SAMPLE PEAS

 Agent: Medical diagnosis system

 Environment: Patient, hospital, physician, nurses, …

 Sensors: Keyboard entry of (percept can be symptoms, findings, patient's

answers)

 Actuators: Screen display of (action can be questions, tests, diagnoses,

treatments, referrals)
23

 Performance measure: Healthy patient, minimize costs, lawsuits.

EXAMPLES OF AGENTS STRUCTURE AND SAMPLE PEAS

 Agent: Interactive English tutor

 Environment: Set of students, testing agency

 Sensors: Keyboard (typed words)

 Actuators: Screen display (exercises, suggestions, corrections)

 Performance measure: Maximize student's score on test

24
EXAMPLES OF AGENTS STRUCTURE AND SAMPLE PEAS

 Agent: Robot Soccer Player.

 Performance Measure (P): To Play, Make Goal & Win the Game.

 Environment (E): Team Members, Opponents, Referee, Audience and

Soccer Field.

 Actuators (A): Navigator, Legs of Robot, View Detector for Robot.

 Sensors (S): Camera, Communicators and Orientation & Touch Sensors.

25
 AGENT PROGRAMS
 An agent is completely specified by the agent function that maps percept
sequences into actions.
FUNCTION SKELETON-AGENT (percept) returns action
static memory, the agent’s memory of the world
memory UPDATE-MEMORY (memory, percept)
action  CHOOSE-BEST-ACTION (memory)
memory UPDATE-MEMORY (memory, action)
RETURN action
Note:
1. the function gets only a single percept at a time 26

2. The goal or performance measure is not part of the skeleton

TYPES OF ENVIRONMENT
• An environment in artificial intelligence is the surrounding of

the agent.

• A task environment is a problem to which a rational agent is

designed as a solution.

• The agent takes input from the environment through sensors

27

and delivers the output to the environment through actuators.

TYPES OF ENVIRONMENT
• Based on the portion of the environment observable

• Fully observable: if the agent sensors give it access to the

complete state of the environment at each point in time. (chess

vs. driving)

• Partially observable: if the agent does not have complete and

relevant information of the environment, then the task

environment is partially observable. 28

TYPES OF ENVIRONMENT
• Example: In the Checker Game, the agent observes the

environment completely while in Poker Game, the agent partially

observes the environment because it cannot see the cards of the

other agent.

• Maintaining a fully observable environment is easy as there is no

need to keep track of the history of the surrounding.

• An environment is called unobservable when the agent has no 29

sensors in all environments.

TYPES OF ENVIRONMENT
• Based on the effect of the agent action

• Deterministic : The next state of the environment is completely

determined by the current state and the action executed by the agent.

✓Strategic: If the environment is deterministic except for the actions

of other agents, then the environment is strategic.

• Stochastic or probabilistic: is random in nature which is not unique

and cannot be completely determined by the agent.

• Example: Chess – there would be only a few possible moves for a coin
at the current state and these moves can be determined.

• Self Driving Cars: the actions of a self-driving car are not unique, it 30

varies time to time.

TYPES OF ENVIRONMENT
• Based on the number agent involved

• Single agent A single agent operating by itself in an

environment.

• Multi-agent: multiple agents are involved in the environment.

• Based on the state, the way time is handled, and to the percept and
action of the agent

• Discrete: A limited number of distinct, clearly defined state,

percepts and actions.

• Continuous: state, percept and action are consciously changing 31

variables
TYPES OF ENVIRONMENT
• Based on environment change

• Static: The environment is unchanged while an agent is

deliberating.

• Dynamic: The environment changes while an agent is not

deliberating.

• semi-dynamic: The environment is semi-dynamic if the

environment itself does not change with the passage of time
but the agent's performance score does.

32
TYPES OF ENVIRONMENT

• Based on loosely dependent sub-objectives

• Episodic: The agent's experience is divided into atomic

"episodes" (each episode consists of the agent perceiving and

then performing a single action), and the choice of action in

each episode depends only on the episode itself.

33
• Sequential: The agent's experience is a single atomic "episodes"
ENVIRONMENT TYPES EXAMPLE
• The environment type largely determines the agent design. The real
world is partially observable, stochastic, sequential, dynamic, continuous,
multi-agent.
Agent Observable Deterministic Episodic Static Discrete

Taxi driver No No No No No
Medical diagnosis No No No No No
Chess with clock Yes yes No Semi Yes
Chess without Yes Yes No yes Yes
clock
Interactive tutor No No No No Yes
34
AGENT TYPES
• Based on memory of the agent, and they way the agent
takes action we can divide agents into five basic types:

• These are (according to their increasing order of generality) :

1. Simple reflex agents

2. Model-based reflex agents

3. Goal-based agents

4. Utility-based agents
35

5. Learning agent
SIMPLE REFLEX AGENTS
• It is the simplest agent which acts according to the current percept
only, pays no attention to the rest of the percept history.

• The agent function of this type relies on the condition-action

rule – “If condition, then action.”

• It makes correct decisions only if the environment is fully

observable.

• It works by finding a rule whose condition matches the current

situation (as defined by the percept) and then doing the action
associated with that rule.
36
 SIMPLEX REFLEX AGENT
Function SIMPLE-REFLEX-AGENT
(percept) returns action
static: rules, a set of condition-action
rules
state<—INTERPRET_INPUT(percept)
rule<- RULE-MATCH(state, rules)
action <- RULE-ACTION
return action

• Simple reflex agents do not maintain the internal state and do not depend on the 37

percept theory.
 MODEL-BASED REFLEX AGENTS
• These type of agents can handle partially observable
environments by maintaining some internal states.
• The internal state depends on the percept history, which
reflects at least some of the unobserved aspects of the
current state.
• Therefore, as time passes, the internal state needs to be
updated which requires two types of knowledge or
information to be encoded in an agent program
38

+ the evolution of the world on its own

+ the effects of the agent’s actions.
 MODEL-BASED REFLEX AGENTS
• Function MODEL_BASED_AGENT(PERCEPT) return action
static state, a description of the current world state
rules, a set of condition action rules
stateUPDATE_STATE(state, percept)
ruleRULE_MATCH(state, rues)
action[Link]
return action

39

Example: When a person walks in a lane, he maps the pathway in his mind.
GOAL-BASED AGENTS
• It is not sufficient to have the current state information unless
the goal is not decided. Therefore, a goal-based agent selects a
way among multiple possibilities that helps it to reach its goal.

40
GOAL-BASED AGENTS STRUCTURE
• Function GOAL_BASED_AGENT(PERCEPT) return action

static state, a description of the current world state

goal, a description of the goal to achieve may be in

terms of state

stateUPDATE_STATE(state, percept)

actionSetPOSSIBLE_ACTIONS(state)

actionACTION_THAT_LEADS_TO_GOAL(actionSet)

return action

41
 UTILITY-BASED AGENTS
• These types of agents are concerned about the performance
measure. The agent selects those actions which maximize the
performance measure and devote towards the goal.

42
UTILITY-BASED AGENTS STRUCTURE
• Function UTILITY_BASED_AGENT(PERCEPT) return action

static state, a description of the current world state

goal, a description of the goal to achieve may be in terms of state

stateUPDATE_STATE(state, percept)

actionSetPOSSIBLE_ACTIONS(state)

actionBEST_ACTION(actionSet)
stateUPDATE_STATE(state,action)

return action

43
LEARNING AGENTS
• Critic: It provides feedback to the learning agent about how well
the agent is doing, which could maximize the performance
measure in the future.

44
LEARNING AGENTS
• The main task of these agents is to teach the agent machines to
operate in an unknown environment and gain as much
knowledge as they can. A learning agent is divided into four
conceptual components:

• Learning element: This element is responsible for making

improvements.

• Performance element: It is responsible for selecting external

actions according to the percepts it takes.

• Problem Generator: It suggests actions which could lead to 45

new and informative experiences.

DISCUSSION
1. Is your environment observable?

2. Is your environment next action determined by the agent

action? (deterministic, probabilistic)

3. Is your environment discrete or continuous?

4. Is your environment is unchanged while an agent is

deliberating.

5. Is your environment affected by the choice of action in

each episode depends only on the episode itself ? 46
DISCUSSION
1. What makes the agent
✓ Simple reflex agent

✓ Model based

✓ Goal based

✓ Utility based

✓ Learning

47
SUMMARY
• An agent is something that perceives and acts in an environment. We
split an agent into an architecture and an agent program.

• An ideal agent is one that always takes the action that is expected to
maximize its performance measure, given the percept sequence it has
seen so far.

• An agent is autonomous to the extent that its action choices depend on

its own experience, rather than on knowledge of the environment that
has been built-in by the designer.

• An agent program maps from a percept to an action, while updating an

internal state.
48
SUMMARY
• There exists a variety of basic agent program designs, depending on the kind of
information made explicit and used in the decision process.
• The designs vary in efficiency, compactness, and flexibility.
• The appropriate design of the agent program depends on the percepts, actions,
goals, and environment.
• Reflex agents respond immediately to percepts, goal-based agents act so that they
will achieve their goal(s), and utility-based agents try to maximize their own
happiness.
• The process of making decisions by reasoning with knowledge is central to AI and
to successful agent design. This means that representing knowledge is important.
• Some environments are more demanding than others. Environments that are
inaccessible, nondeterministic, nonepisodic, dynamic, and continuous are the most
challenging. 49
SUMMARY

Observable

Fully observable: Partially observable:

It access the complete It does not have
state of the complete state of the
environment environment

50
SUMMARY

Effect of agent action

Deterministic: next state of the

environment is completely Probabilistic: next state
determined by the current state of the environment is
and the action executed by the undetermined
agent.
51
SUMMARY

Environment

State, percept
No of Agent and action

Single Multiple
Discrete: Continuous:
limited number Changed variables
52
SUMMARY

Static: environment is Dynamic: environment

unchanged while an agent changes while an agent is
is deliberating. not deliberating.

Effect of time

semi-dynamic: environment itself does not

change with the passage of time but the agent's
performance score does. 53
SUMMARY

Episodic: single
the choice of action in each
episode depends only on the
episode itself.
Loosely dependent sub
objectives

Sequential: multiple

54
QUESTION?
THE END

55