Module 2
What is an agent and environment? Explain the concept of Rationality. What is Rational Agent?
1. An agent is anything that can be viewed as perceiving its environment through sensors and acting
upon that environment through effectors.
2. A human agent has eyes, ears, and other organs for sensors, and hands, legs, mouth, and other body
parts for effectors.
3. A robotic agent substitutes cameras and infrared range finders for the sensors and various motors for
the effectors.
4. An environment in artificial intelligence is the surrounding of the agent.
5. Rational dictionary meaning is something logical or sensible.
6. A rational agent is one that does the right thing to be most successful.
7. For each possible percept sequence, an ideal rational agent should do whatever action is expected to
maximize its performance measure, on the basis of the evidence provided by the percept sequence
and whatever built-in knowledge the agent has.
8. Example of a rational agent:
a. Performance measure
o Awards one point for each clean square at each time step, over 10000-time steps
b. Prior knowledge about the environment
o the geography of the environment
o Only two squares
o the effect of the actions
c. Actions that can perform
o Left, Right, Suck and No Operation
d. Percept sequences
o Where is the agent?
o Whether the location contains dirt?
o Under this circumstance, the agent is rational
Properties of environment: -
1. Fully observable vs. partially observable:
a. If an agent’s sensors give it access to the complete state of the environment at each point in time,
then the environment is effectively and fully observable.
b. Fully observable environments are convenient because agent need not maintain any internal state
to keep track of the world.
c. For example, chess board or crossword puzzle
d. An environment might be partially observable because of noisy and inaccurate sensors or because
parts of the state are simply missing from the sensor data.
e. For example, a vacuum agent with only a local dirt sensor cannot tell whether there is dirt in other
squares, and an automated taxi cannot see what other drivers are thinking.
2. Deterministic vs. nondeterministic (stochastic):
a. If the next state of the environment is completely determined by the current state and the actions
selected by the agents, then we say the environment is deterministic.
b. If the environment is inaccessible, however, then it may appear to be nondeterministic.
c. E.g. Taxi driving (nondeterministic), Teacher’s time table (deterministic)
3. Episodic vs. Sequential:
a. In an episodic environment, the agent’s experience is divided into “episodes.” Each episode
consists of the agent perceiving and then acting.
b. The quality of its action depends just on the episode itself, because subsequent episodes do not
depend on what actions occur in previous episodes.
c. For example, an agent that has to spot defective parts on an assembly line bases each decision on
the current part, regardless of previous decisions
d. In a sequential environment current decisions could affect all future decisions. E.g. chess
4. Static vs. dynamic:
a. If the environment can change while an agent is deliberating, then we say the environment is
dynamic for that agent; otherwise, it is static.
b. Static environments are easy to deal with because the agent need not keep looking at the world
while it is deciding on an action, nor need it worry about the passage of time.
c. If the environment does not change with the passage of time but the agent’s performance score
does, then we say the environment is semi dynamic.
d. Crossword puzzles are static whereas taxi driving is clearly dynamic.
5. Discrete vs. continuous:
a. If there are a limited number of distinct, clearly defined percepts and actions we say that the
environment is discrete.
b. Chess is discrete—there are a fixed number of possible moves on each turn.
c. Taxi driving is continuous—the speed and location of the taxi and the other vehicles sweep through
a range of continuous values.
6. Single agent vs. Multi agent:
a. When there is only one agent in a defined environment, it is named the Single-Agent System
(SAS). This agent acts and interacts only with its environment.
b. If there is more than one agent and they interact with each other and their environment, the
system is called the Multi-Agent System.
c. For example, an agent solving a crossword puzzle by itself is clearly in a single-agent
environment.
d. Chess is a multi-agent environment.
7. Competitive vs. Collaborative:
a. An agent is said to be in a competitive environment when it competes against another agent to
optimize the output. For example, Chess game
b. An agent is said to be in a collaborative environment when multiple agents cooperate to produce
the desired output. For example, when multiple self-driving cars are found on the roads, they
cooperate with each other to avoid collisions and reach their destination.
Types of Agents: -
1. Simple Reflex Agent:
a. Simple reflex agents act only on the basis of the current percept, ignoring the rest of the percept
history.
b. The agent function is based on the condition-action rule: if condition then action.
c. This agent function only succeeds when the environment is fully observable.
d. Infinite loops are often unavoidable for simple reflex agents operating in partially observable
environments
2. Model-based Reflex Agent:
a. A model-based agent can handle a partially observable environment.
b. Its current state is stored inside the agent maintaining some kind of structure which describes the
part of the world which cannot be seen. This knowledge about "how the world works" is called a
model of the world, hence the name "model-based agent".
c. A model-based reflex agent should maintain some sort of internal model that depends on the
percept history and thereby reflects at least some of the unobserved aspects of the current state.
3. Goal-based Agent:
a. Goal-based agents further expand on the capabilities of the model-based agents, by using "goal"
information.
b. This allows the agent a way to choose among multiple possibilities, selecting the one which
reaches a goal state.
c. In some instances, the goal-based agent appears to be less efficient; it is more flexible because
the knowledge that supports its decisions is represented explicitly and can be modified.
4. Utility-based Agent:
a. It is possible to define a measure of how desirable a particular state is. This measure can be
obtained through the use of a utility function which maps a state to a measure of the utility of the
state.
b. A more general performance measure should allow a comparison of different world states
according to exactly how happy they would make the agent.
c. The term utility can be used to describe how "happy" the agent is.
d. A utility-based agent has to model and keep track of its environment, tasks that have involved a
great deal of research on perception, representation, reasoning, and learning
5. Learning Agents:
a. A learning agent in AI is the type of agent that can learn from its past experiences or it has
learning capabilities. It starts to act with basic knowledge and then is able to act and adapt
automatically through learning.
b. Learning has an advantage that it allows the agents to initially operate in unknown environments
and to become more competent than its initial knowledge alone might allow.
c. For example, in school, the test is the critic. The teacher would mark the test and see what could
be improved and instruct how to do better next time, so the teacher is the learning element and
the tester is the performance element.
d. A learning agent has mainly four conceptual components:
i. Learning element: It is responsible for making improvements by learning from the
environment
ii. Critic: The learning element takes feedback from critics which describes how well the agent
is doing with respect to a fixed performance standard.
iii. Performance element: It is responsible for selecting external action
iv. Problem Generator: This component is responsible for suggesting actions that will lead to
new and informative experiences.