Module 2: Intelligent Agent

Agent:

“Agent” is an independent program or entity that interacts with its environment by

perceiving its surroundings via sensors, then acting through actuators or effectors.

Agents use their actuators to run through a cycle of perception, thought, and action. Examples
of agents in general terms include:

 Software: This Agent has file contents, keystrokes, and received network packages that
function as sensory input, then act on those inputs, displaying the output on a screen.

 Human: Yes, we’re all agents. Humans have eyes, ears, and other organs that act as
sensors, and hands, legs, mouths, and other body parts act as actuators.

 Robotic: Robotic agents have cameras and infrared range finders that act as sensors, and
various servos and motors perform as actuators.

Intelligent agents in AI are autonomous entities that act upon an environment using sensors
and actuators to achieve their goals. In addition, intelligent agents may learn from the
environment to achieve those goals. Driverless cars and the Siri virtual assistant are examples
of intelligent agents in AI.

These are the main four rules all AI agents must adhere to:

 Rule 1: An AI agent must be able to perceive the environment.

 Rule 2: The environmental observations must be used to make decisions.

 Rule 3: The decisions should result in action.

 Rule 4: The action taken by the AI agent must be a rational. Rational actions are actions that
maximize performance and yield the best positive outcome.

Agent and Environment:

The term "environment '' is quite well known to everyone. As per definition from oxford
dictionary: "environment is the surroundings or conditions in which a person, animal, or plant
lives or operates." But when it comes to computing, it is the surrounding where a computing
device works or operates. In the context of Artificial intelligence, an Environment is simply
the surrounding of an agent and is where the agent operates.

Now, let's consider a real-life example of driving a car on the road. Can you guess who will be
the agent and environment? Yes,

 Agent will be the car

  The environment will be the road.

 The driver is the one who senses all alerts and operates the car in a dangerous
environment to get the desired output - safe driving.

Similarly, in AI, we have an environment that contains an agent, a sensor, and an actuator.

The above figure shows the simplest diagrammatic representation of an agent-environment

interaction. The agent is within the environment. There are sensors to sense the environment
and they provide sensory inputs to the agent. The agent then takes actions for the respective
inputs and provides the output back to the environment.

For AI, the problem which is to be solved itself creates a great challenge. Understanding the
given problem itself is a challenging task for AI. And apart from reasoning, the most
challenging aspect of an AI problem is the environment.

Agent and Environment can be said as two hooks where AI is hanging. Or more simply the
environment is considered as the problem then the agent is the solution for the problem or
‘agent’ is the game played on the ground ‘environment’.

Example for agents and their environments in AI

Some examples for agents and their environments are shown above for a clear understanding.
For a given task of driving, the vehicle is the agent and the road is the environment to drive on.
The sensor devices like cameras, radar, lidar, etc will collect information about the road like
the presence of pedestrians, the number of other vehicles present on the road about the traffic
signal, etc. The vehicle will then act concerning that information like whether the brake pedal
or the accelerator pedal has to be pushed or have to take a turn etc.

If a machine is an agent, then its working place is its environment. If we consider a Cooling
System as the agent, then the industry it works for is the environment. The coolant temperature
sensor will then collect information and the machine will act upon that information.

Why an agent environment is important?

AI can also be called the study of rational agents and their environments. From this statement
itself, it is clear how important an agent and agent environment is for AI. The working of an
AI system can be stated in the least words. Through different types of sensors, the agent will
sense the environment and through actuators, the agent will act on their environment. The
structure of an agent environment will affect the entire AI system itself.

Agents do not have full control over their environment but have partial control. This means the
environment is influenced by the agent. But the agent’s performance is directly influenced by
the changes in the environment. I.e. an agent performing the same task in two different
environments will result entirely in two different outcomes. An agent is preprogramed with a
set of abilities that it can apply in different situations it may face in its environment. These
preprogramed abilities are called effector capabilities. The sensors of the agents are strictly
influenced by the environment. The agent uses its effector capabilities and the information from
the sensor to perform decision making. Every action is not suited for every situation. The
precondition must be suitable to perform the particular task. For example, the agent has to open
a door and it can do so. But it can be performed only if the door is unlocked (precondition). So,
deciding the action to perform at a given instance to achieve the goal or at least work towards
it is the biggest problem of an agent in an environment. The properties of an environment
directly affect the complexity in the process of decision-making.

Rationality

Rationality is nothing but status of being reasonable, sensible, and having good sense of
judgment.

Rationality is concerned with expected actions and results depending upon what the agent has
perceived. Performing actions with the aim of obtaining useful information is an important part
of rationality.

What is Ideal Rational Agent?

An ideal rational agent is the one, which is capable of doing expected actions to maximize its
performance measure, on the basis of −
  Its percept sequence

 Its built-in knowledge base

Rationality of an agent depends on the following −

 The performance measures, which determine the degree of success.

 Agent’s Percept Sequence till now.

 The agent’s prior knowledge about the environment.

 The actions that the agent can carry out.

A rational agent always performs right action, where the right action means the action that
causes the agent to be most successful in the given percept sequence. The problem the agent
solves is characterized by Performance Measure, Environment, Actuators, and Sensors
(PEAS).

Types of environment

The agent environment in artificial Intelligence is classified into different types. The
environment is categorized based on how the agent is dealing with it.
Classification is as follows:

1. Fully observable & Partially observable

2. Deterministic & Stochastic

3. Episodic & Sequential

4. Static & Dynamic

5. Discrete & Continuous

6. Single-agent & Multi-agent

1. FULLY OBSERVABLE & PARTIALLY OBSERVABLE

As with the name itself the environment of the agent is observed all the time. At each point in
time, the complete state of the environment is sensed or accessed by the sensor. This type of
completely observed environment is called fully observable and else if it is not sensed or
observed continuously with varying time then it is partially observable. An environment that
is not at all observed or accessed by any sensor for any time is called an unobservable
environment. Since it doesn't need to have a track over the world, a fully observable
environment is more convenient.

In real life, Chess is an example of fully observable because each player of the chess game gets
to see the whole board. Another example of a fully observable environment is the road, while
driving a car on the road (environment), the driver (agent) can see all the traffic signals,
conditions, and pedestrians on the road at a given time and drive accordingly.

A card game can be considered as an example of a partially observable. Here some of the cards
are discarded into a pile face down. The user is only able to see his cards. The used cards, cards
reserved for the future, are not visible to the user.

2. DETERMINISTIC AND STOCHASTIC

An environment that ignores uncertainty is called a deterministic environment. For a

deterministic environment, the upcoming condition or state can be determined by the present
condition or state of the environment and the present action of the agent or the action selected
by the agent. An environment with a random nature is called a stochastic environment. The
upcoming state cannot be determined by the current state or by the agent. Most of the real-
world Ai applications are classified under stochastic type. An environment is stochastic only if
it is partially observable.

For each piece on the chessboard, the present position of them can set the next coming action.
There is no case of uncertainty. Which all steps can be taken by a piece from the present
position can be determined and so, it can be grouped under a deterministic environment.

But for a self-driving car, the coming actions can’t be determined in the present state because
the environment is varying continuously. Maybe the car has to push the brake or maybe push
the accelerator fully depending on the environment at that time. actions cannot be determined
and is an example of a stochastic environment.
[Link] & SEQUENTIAL

An environment with a series of actions where the current action of an agent will not make any
influence on future action. It is also called the non-sequential environment/Episodic
environment. Sequential or non-episodic environments are where the current action of the agent
will affect the future action.

For a classification task, the agent will receive the information from the environment
concerning the time, and actions are performed only on those pieces of information. The current
action doesn't have any influence on the future one, so it can be grouped under an episodic
environment.
But for a chess game, the current action of a particular piece can influence the future action. If
the coin takes a step forward now, the next coming actions depend on this action where to
move. And it is sequential.

4. STATIC AND DYNAMIC

An environment that remains always unchanged by the action of the agent is called a static
environment. A static environment is the simplest one which is easy to deal with since the agent
doesn’t need to keep track of the world during an action. But an environment is said to be
dynamic if it changes by the action of the agent. A dynamic environment keeps constantly
changing. An environment that keeps constant with time and the performance score of the agent
will change with time is called a semi-dynamic environment.

The crossword puzzle can be considered as an example of a Static environment since the
problem in the crossword puzzle is set paused at the beginning Crossword puzzle, the
environment remains constant and the environment doesn't expand or shrink it remains the
same.
For a Dynamic environment, we can consider a roller coaster ride as an example. The
environment keeps changing for every instant as it is set in motion. The height, mass, velocity,
different energies (kinetic, potential), centripetal force, etc will vary from time to time.

[Link] & CONTINUOUS

An environment with a finite number of possibilities is called a discrete environment. For a

discrete environment, there is a finite number of actions or precepts to be performed to reach
the final goal. For a continuous environment, the number of precepts remains unknown and
continuous.
In a chess game, the possible movements for each piece are finite. Like the king can move only
one square in any direction until that square is not attacked by an opponent piece. So the
possible movements for the particular piece are fixed and it can be considered as an example
for a Discrete Environment but the number of movements will vary for each game.

Self-driving cars are an example of a continuous environment. The surroundings change over
time, traffic rush, speed of other vehicles on the road, etc will vary continuously over time.
6. SINGLE-AGENT vs MULTI-AGENT

An environment that consists of only a single agent is called a single-agent environment. All
the operations over the environment are performed and controlled by this agent only. If the
environment consists of more than one agent or multiple agents for conducting the operations,
then such an environment is called a multiple agent environment.

In a vacuum cleaning environment, the vacuum cleaner is the only agent involved in the
environment. And it can be considered as an example of a single-agent environment.

Multi-Agent Systems, computer-based environments with multiple interacting agents are the
best example of a multi-agent environment. Computer games are the common MAS
application. Biological agents, Robotic agents, computational agents, software agents, etc are
some of the agents sharing the environment in a computer game.

Structure of Agents in Artificial Intelligence

An AI agent comprises of Architecture and an Agent program. Architecture involves

machinery for execution of tasks by agents. It consists of a device with sensors and effectors
or actuators. An agent program refers to the process of implementation of an agent function,
which is map of the percept sequence or the perceptual history of the agent for a particular
action.

Interaction of Agents with Environment

Interaction of the Agent with the environment uses Sensors and Effectors. Sensors perceive the
environment and the actuators or effectors act upon that environment.

This interaction can occur in two different ways:

1. Perception: Perception is a passive interaction between the agent and the environment
where the environment remains unchanged when the agent takes up information from
the environment. This involves gaining information using 'Sensors' from the
surroundings without any change to the surroundings.

2. Action: Action is an active interaction between the agent and the environment where
the environment changes when the action is performed. This involves utilization of an
'Effector' or an 'Actuator' which completes an action but leads to changes in the
surroundings while doing so.

For example, in case of a virtual agent, when the virtual agent reads and interprets the
information provided by the user, it is known as 'Perception' while when it replies to the user
based on the interpretation it is known as 'Action'.
Action of Agents in Artificial Intelligence

Agents in Artificial Intelligence act by:

1. Mapping of the Percept sequences or Perceptual history to the Actions: Mapping refers
to a list that maps a particular percept sequence to the action. The design for an ideal
agent can be figured out by specifying an action corresponding to the percept sequence
or the perceptual history.

2. Autonomy: The agent designer determines the behaviour of the agent by determining
its experience and its built-in knowledge. Autonomy refers to taking actions based on
the experience of the agent. If the system comprises of an autonomous intelligent agent,
then it is able to operate and adapt successfully in a wide range of environments.

Types of Agents

Based on their degree of perceived intelligence and capability, types of agents in artificial
intelligence can be divided into:

1. Simple Reflex Agents

2. Model-Based Agents

3. Goal-Based Agents

4. Utility-Based Agents

Performance can be improved and better action can be generated for each of these types of
agents in AI.

Simple Reflex Agents

1. This is a simple type of agent which works on the basis of current percept and not based
on the rest of the precepts history.

2. The agent function, in this case, is based on condition-action rule where the condition
or the state is mapped to the action such that action is taken only when condition is true
or else it is not.

3. If the environment associated with this agent is fully observable, only then is the agent
function successful, if it is partially observable, in that case the agent function enters
into infinite loops that can be escaped only on randomization of its actions.
 4. The problems associated with this type include very limited intelligence, No knowledge
of non-perceptual parts of the state, huge size for generation and storage and inability
to adapt to changes in the environment.

5. Example: A thermostat in a heating system.

This can be illustrated using the following image:

Model-Based Agents

1. Model-based agent utilizes the condition-action rule, where it works by finding a rule
that will allow the condition, which is based on the current situation, to be satisfied.

2. Irrespective of the first type, it can handle partially observable environments by tracking
the situation and using a particular model related to the world.

3. It consists of two important factors, which are Model and Internal State.

4. Model provides knowledge and understanding of the process of occurrence of different

things in the surroundings such that the current situation can be studied and a condition
can be created. Actions are performed by the agent based on this model.

5. Internal State uses the perceptual history to represent a current percept. The agent keeps
a track of this internal state and is adjusted by each of the precepts. The current internal
state is stored by the agent inside it to maintain a kind of structure that can describe the
unseen world.
 6. The state of the agent can be updated by gaining information about how the world
evolves and how the agent's action affects the world.

7. Example: A vacuum cleaner that uses sensors to detect dirt and obstacles and moves
and cleans based on a model.

This can be illustrated as:

Goal-Based Agents

1. This type takes decisions on the basis of its goal or desirable situations so that it can
choose such an action that can achieve the goal required.

2. It is an improvement over model based agent where information about the goal is also
included. This is because it is not always sufficient to know just about the current state,
knowledge of the goal is a more beneficial approach.

3. The aim is to reduce the distance between action and the goal so that the best possible
way can be chosen from multiple possibilities. Once the best way is found, the decision
is represented explicitly which makes the agent more flexible.

4. It carries out considerations of different situations called searching and planning by

considering long sequence of possible actions for confirming its ability to achieve the
goal. This makes the agent proactive.

5. It can easily change its behaviour if required.

6. Example: A chess-playing AI whose goal is winning the game.

This can be illustrated as follows:

Utility-Based Agents

1. Utility agent have their end uses as their building blocks and is used when best action
and decision needs to be taken from multiple alternatives.

2. It is an improvement over goal based agent as it not only involves the goal but also the
way the goal can be achieved such that the goal can be achieved in a quicker, safer,
cheaper way.

3. The extra component of utility or method to achieve a goal provides a measure of

success at a particular state that makes the utility agent different.

4. It takes the agent happiness into account and gives an idea of how happy the agent is
because of the utility and hence, the action with maximum utility is considered. This
associated degree of happiness can be calculated by mapping a state onto a real number.

5. Mapping of a state onto a real number with the help of utility function gives the
efficiency of an action to achieve the goal.

6. Example: A delivery drone that delivers packages to customers efficiently while

optimizing factors like delivery time, energy consumption, and customer satisfaction.

This can be illustrated as follows:

Learning Agent

How does it work?

AI learning agents follow a cycle of observing, learning, and acting based on feedback. They
interact with their environment, learn from feedback, and modify their behavior for future
interactions.
 Here’s how the cycle works:

 Observation: The learning agent observes its environment through sensors or other inputs.

 Learning: The agent analyses data using algorithms and statistical models, learning from

feedback on its actions and performance.

 Action: Based on what it has learned, the agent acts in its environment to decide how to behave.

 Feedback: The agent receives feedback about their actions and performance through rewards,

penalties, or environmental cues.

 Adaptation: Using feedback, the agent changes its behaviour and decision-making processes,

updating its knowledge and adapting to its environment.

This cycle repeats over time, allowing the agent to continuously improve its performance and
adapt to changing circumstances.

Example
A good example of a learning agent program is AutoGPT, created by Significant Gravitas.
Imagine you want to purchase a smartphone. So, you give AutoGPT a prompt to conduct
market research on the top ten smartphones, providing insights on their pros and cons.

Once given this task, AutoGPT analyses the pros and cons of the top ten smartphones by
exploring various websites and sources. It evaluates the authenticity of websites using a sub-

agent program. Finally, it generates a detailed report summarizing the findings and listing the
pros and cons of the top ten smartphone companies.

Advantages of learning agents

 The agent can convert ideas into action based on AI decisions

 Learning intelligent agents can follow basic commands, like spoken instructions, to perform

tasks

 Unlike classic agents that perform predefined actions, learning agents can evolve with time

 AI agents consider utility measurements, making them more realistic

Disadvantages of learning agents

 Prone to biased or incorrect decision-making

 High development and maintenance costs

 Requires significant computing resources

 Dependence on large amounts of data

 Lack of human-like intuition and creativity