Artificial Intelligence

Intelligent Agents

Dr. Muhammad Awais

Outline
• Agents and Environments
• Concept of Rationality
• Nature of Environment
• Structure of Agents
• Summary
Agents and Environment
• Agents
• Human Agents
• Inputs (Perception, Sensors)
• Listen, Taste, Observe, Smell, touch
• Percept Sequence : Complete history of observations
• Outputs (Actuators)
• Hands, Feet, Mouth
• Software Agents
• Inputs
• Keystroke, File contents, network packets, etc.
• Outputs
• Displaying on Screen, Writing Files, Sending network packets, etc.
• Robotic Agents
• Inputs (Perception, Sensors)
• Cameras, Infra-red and Laser range finders, etc.
• Outputs (Actuators)
• Motors, Wheels, Mechanical hands and / or legs, etc.
• Agents
• Agent function (Behavior)
• Abstract Mathematical description
• Mapping of the percepts (percept sequence) to actions
• Nothing to do with implementation
• Naïve approach ( tabulation of percepts and actions)
• Can be infinite table
• External Characterization
• Describe the agent’s Behaviour

• Agent program
• Internal Implementation of Agent function
• Implementation of perception
• Implementation of reaction
• Control of actuators, motors, etc.
• Agent example
1. Localize itself
2. If square dirty suck the dirt
3. Move left
4. Move right
5. Do nothing
• Agent example
Good Behaviour: The Concept of Rationality
• A rational agent should always do the right thing

• In connection to the previous discussion the table

should contain all the required entries with correct
and rational mappings

• What is the right thing OR what is the measure of

the rationality for the agent for the current task

• The current task is defined by the description of the

environment, sensors and actuators of the agent
Performance measures
Embodiment of the success criterion of Agent’s Behaviour
General Principle for Performance measure:
• Dependence on the change (sequence of changes) in the environment
rather than the expectation how the agent should behave
Vacuum Cleaner Example:
• Performance measure defined in terms of the cleaned dust
• Performance measure can be defined
– Giving the reward if the floor is clean
– Providing the penalty for electricity consumed and noise generated
• In this way the robot can optimize its performance by keeping the floor
clean and consuming less electricity and producing less noise
• An example Performance measure:
– Average cleanliness measure with respect to time
– Results can be achieved by two different kinds of agents
• A slow agent doing the job in a mediocre style
• An energetic agent doing the job swiftly but require long brakes
• Thus the performance measure totally depends on the current task and
the trade off exists between the selection, E.g., a reckless life with high
and lows, and safe life with humdrum
Rationality:
Rationality depends on
• Performance measure (success criterion)
• Agent’s prior knowledge of environment
• Actions that the agent can perform
• Agent’s percept sequence to date
Rationality
Definition: For each possible percept sequence, a rational agent
should select an action that is expected to maximize its performance
measure, given the evidence provided by the percept sequence and
whatever built-in knowledge the agent has.
Vacuum Cleaner Example
A claimed rational agent with the given
• Performance measure
• One point reward for a clean square
• Prior information of environment
• Geography of environment is known
• Can move Left and Right and remains in the task-space
• Sensors and Actuators
• Move Left and Right, Suck and do nothing
• Perception of location and dust
• Irrationality under different circumstances
• Needless oscillation after dirt clean up
• Penalty as performance measure on move left and right etc.
Omniscience, Learning and Autonomy

• A rational agent needs not to be Omniscience (Perfectionist, impossible

in rationality)
– Omniscience agent knows the outcome of its actions

• Rational agent maximizes the expected performance

• Rational agent gather information to maximize the expected performance

– E.g., look in both directions before crossing the street

• Rational agent increases its (prior) information about the environment

– Agent uses its percept sequence to learn new information

– New information can modify and augment agent behavior

• A rational agent is autonomous if

– Increases its information

– Can update and modify the designer

Nature of Environments
• Specification of the task environment
• Properties of task environments
• Fully observable vs Partially observable
• Deterministic vs Stochastic
• Episodic vs Sequential
• Static vs Dynamic
• Discrete vs Continuous
• Single agent vs Multiagent
Task Environment
• Task Environment Specification
– Performance
– Environment
– Actuators
– Sensors
Different task environments
Fully observable:
• Sensor of agent can detect the full state of environment at a
point of time
• All the aspect of environment concerning the action selection is
observable
• Agent does not need to maintain the internal state concerning
the environment

Partially observable:
• State of environment is not fully visible at any point of time
• Partial visibility may be due to
– noise
– inaccurate sensors, etc.
• Partially observable environment
– Local dirt sensors of Vacuum cleaner
– Taxi driver can not read other driver’s mind
Deterministic vs stochastic:
• Next state of environment is completely determined by the action
and the current state

• An observable environment is deterministic

• The vacuum cleaner example is a deterministic environment
• A deterministic environment with unpredictable other agents is called
strategic environment

 A partially observable (complex) environment is stochastic

 The taxi driver environment is stochastic as the traffic is not predictable
 The tires of taxi can blow, engine can malfunction, etc.
 Environment is considered deterministic or stochastic with respect to
the agent.
Episodic
• An episodic task is divided into episodes
• In an episode agent perceives and performs a single action
• Consecutive episodes are independent from each other, i.e., the
action in each episode is independent from actions in other episodes
• Assembly line tasks are episodic as each task actions are
independent from each other

Sequential:
• Current decision could effect all the future decisions.
• The actions in the sequential environment are not necessarily
independent
• Taxi and Chess are sequential tasks examples
Dynamic
• Environment changes while the agent deliberates (without the
agent’s actions)
• Agents needs to continuously monitor the environment
• Dynamic environment actively interact with the agent
• Taxi driver is a dynamic environment concerning other taxi drivers

Static
• Environment does not change without the agent’s action
• Agent is not required to monitor the environment
• Chess is static (without time)

Semi-dynamic environment:
• If the environment does not change explicitly but the
performance score of the agent changes with respect to time
• Chess is semi-dynamic (with time)
Discrete vs Continuous
• Distinction can be applied based on state of environment
• Handling of time, percepts and actions of agent can be continuous
or discrete based on the agent
• Chess is discrete with discrete percept, action and state
• Taxi driver is continuous with continuous time and continuous
state
• Camera input is discrete
Single agent vs Multiagent
• Clearly one agent and multiple agent is recognizable from the Single and
multiple presence of the agents
• Selection of the entities in the existing environment as ‘agents’ or
‘stochastic’ objects
 Agents : Other taxi drivers
 Stochastic objects : Waves on the beach, blowing leaves in wind
• Multiple classes in the multi-agent cases
• Competitive (chess)
• Partially Cooperative (Taxi driver example)
Properties of different environments
Structure of Agents:
• Task of AI is to design optimal agent programs
• Agent = program + Architecture
• Agent Program : implementation of agent function that maps
the percepts to the actions
• Architecture and program compatibility is required
• Program requiring legged robots is not appropriate for
robots with wheel
Skeleton of the Agents:
• Agent accepts the percepts from the environment and maps the
percepts to the action and apply the actions to the environment
• Agent can use the current percept as well as the percept
sequence for the action selection
Agent program:

Note: Simple Agent function is a table of actions indexed by

the percept or percept sequence
Advantages of table approach:
• Simple
• Easy to implement
Disadvantages of table approach:
• P = Possible Percepts
• T = Total life time entries
• Table Size = 𝑇𝑡=1 𝑃𝑡
• Taxi driver with camera (30 fps, 640 x 480, 24 bits color
information) construct a table of 10250,000,000,000 for 1 hour
• Look-up table of chess 10150 entries
• Memory space problem for table storage
• Designer problem to create and fill the entries in the table
• Impossible for the agent to learn the table entries
• Task of AI is to develop programs implementing the rational
behavior of agent with smaller agent programs
• Huge tables of square root replaced by Five line code due to
Newton's method
Basic Agent Programs:

• Simplex reflex agents

• Model based reflex agents

• Goal based agents

• Utility based agents

Simplex reflex agents:

• Simplex reflex agents are very simple and simple to implement

• It works if the environment is fully observable and correct decisions are
made
• E.g., false understanding of car brake light will make the taxi to brake
continuously or wrongly
Simplex reflex agents
• Selection of action based on current percept
• Complexity is reduced due to the usage of current percept than the
complete percept sequence
• Condition-action rule
Taxi example: if “car in front brakes” Then “initiate braking”
Model-based reflex agent:
• Agent use the internal state
• Internal state correspond to some of unobserved aspects of current state
depending on the percept history
• E.g., for lane changing the taxi driver needs to keep track of the
previous Vehicle situation as well as self position
• Internal state update depends
• How the world evolves independent of the agent
• How agent’s own action effect the world
• World-Model (how world works) is used by the model-based reflex agents
Model-based reflex agent:
Model-based reflex agent:
Goal based agents
• Knowing the environment state is not enough to solve a task
• Taxi is at a road junction, can move left, right, straight on
• Agent can make optimal decision given goal and environment state
• Planning and search specialized subfields of AI
• Goal based action-sequence selection
• Goal based action selection replaces goal for condition-action rule
• More flexible
• Explicit representation of supporting Knowledge
• Modifiable representation
• Future consideration concerning optimal solution, E.g.,
• What happens if Agent does A
• What makes agent happy
Goal based agents
Utility based agents:
• Goals do not guide the agent what path to follow if more than one paths are
available
• Utility (preference of A over B) tells the agent which path is optimal to goal
w.r.t available measures, e.g., taxi measures (quicker, safer, reliable)
• Utility function maps a state (sequence of states) onto a real number
• Number corresponds to the happiness of an agent
• Complete specification allows rational decisions in conflicting goals (some
can be achieved) as utility function gives better trade Off
• Given multiple uncertain goals, utility weighs important goals w.r.t success
certainty
Utility based agents:
Learning agents:
Learning element:
• Responsible for making improvements
Performance element:
• Responsible for action selection
Critic element:
• Responsible for modification in performance element for
improvement
Problem generator
• Responsible for exploration (Suggestion of new actions)
Learning agents: