Categories of AI
Different properties of task environment
Fully Observable vs. Partially Observable:
• Fully Observable: The agent has access to the complete state of the environment at each point in time.
No information about the environment is hidden.
• Partially Observable: The agent does not have access to the complete state of the environment, leading
to uncertainty.
Deterministic vs. Stochastic:
• Deterministic: The next state of the environment is entirely determined by the current state and the
action taken by the agent. There is no randomness involved.
• Stochastic: The environment involves some degree of randomness or uncertainty, and the outcomes of
actions are not guaranteed.
Episodic vs. Sequential:
• Episodic: The agent's actions are divided into episodes, where each action is independent of the
previous ones. The outcome of one episode does not affect future episodes.
• Sequential: Actions are linked in a sequence, meaning that one action affects the future state and the
agent's subsequent actions.
Static vs. Dynamic:
• Static: The environment remains unchanged while the agent is deliberating. The world only changes as
a result of the agent’s actions.
• Dynamic: The environment can change independently of the agent’s actions, even while the agent is
deciding its next action.
Discrete vs. Continuous:
• Discrete: The environment consists of a finite number of distinct states, actions, or time intervals.
• Continuous: The environment has a potentially infinite number of states, actions, or time intervals,
often requiring more complex decision-making.
Single Agent vs. Multi-Agent:
• Single Agent: There is only one agent acting in the environment.
• Multi-Agent: Multiple agents operate in the environment, potentially with competitive or cooperative
interactions. Multi-agent environments introduce additional complexities, such as game theory and
coordination.
Vacuum cleaner agents as rational agent
• To illustrate these ideas, we use a very simple example—the vacuum-cleaner world
• This particular world has just two locations: squares A and B. The vacuum agent perceives which
square it is in and whether there is dirt in the square. It can choose to move left, move right, suck up
the dirt, or do nothing.
• One very simple agent function is the following: if the current square is dirty, then suck; otherwise,
move to the other square.
PEAS description for automated taxi
Applications of AI
• Robotic vehicles: A driverless robotic car “STANLEY”
• Speech recognition: A traveler calling United Airlines to book a flight
• Autonomous planning and scheduling: A hundred million miles from Earth, NASA’s Remote Agent
program became the first on-board autonomous planning program to control the scheduling of
operations for a spacecraft
• Game playing: IBM’s DEEP BLUE became the first computer program to defeat the world champion
in a chess match
• Spam fighting: Each day, learning algorithms classify over a billion messages as spam
• Logistics planning: DART (Cross and Walker, 1994), to do automated logistics planning and scheduling
for transportation (Persian Gulf crisis of 1991,)
• Robotics: Roomba robotic vacuum cleaners for home use
• Machine Translation: A computer program automatically translates from Arabic to English
Simple reflex agent with a code snippet
• The simplest kind of agent is the simple reflex agent. These agents select actions on the basis of the
current percept, ignoring the rest of the percept history.
• Simple reflex agents have the admirable property of being simple, but they turn out to be of limited
intelligence.
• The agent will work only if the correct decision can be made on the basis of only the current percept—
that is, only if the environment is fully observable. Even a little bit of unobservability can cause serious
trouble.
• For example, the braking rule given earlier assumes that the condition car-in-front-is-braking can be
determined from the current percept—a single frame of video.
Problem solving agent with a snippet
• Problem-solving agent
– A kind of goal-based agent
– It solves problem by
• finding sequences of actions that lead to desirable states (goals)
– To solve a problem,
• the first step is the goal formulation, based on the current situation
• Goal Formation: The goal is formulated
– as a set of world states, in which the goal is satisfied
• Reaching from initial state → goal state
– Actions are required
• Actions are the operators
– causing transitions between world states
– Actions should be abstract enough at a certain degree, instead of very detailed
– E.g., turn left VS turn left 30 degrees, etc.
• Problem formation: The process of deciding
– what actions and states to consider
• E.g., driving Amman → Zarqa
– in-between states and actions defined
– States: Some places in Amman & Zarqa
– Actions: Turn left, turn right, go straight, accelerate & brake, etc.
• Search: Because there are many ways to achieve the same goal
– Those ways are together expressed as a tree
– Multiple options of unknown value at a point,
• the agent can examine different possible sequences of actions, and choose the best
– This process of looking for the best sequence is called search
– The best sequence is then a list of actions, called solution.
State space with its problem components for vacuum world.
• States: The state is determined by both the agent location and the dirt locations. The agent is in one of two
locations, each of which might or might not contain dirt. Thus, there are 2 × 2 2 = 8 possible world states. A
larger environment with n locations has n · 2 n states.
• Initial state: Any state can be designated as the initial state.
• Actions: In this simple environment, each state has just three actions: Left, Right, and Suck. Larger
environments might also include Up and Down.
• Transition model: The actions have their expected effects, except that moving Left in the leftmost square,
moving Right in the rightmost square, and Sucking in a clean square have no effect.
• Goal test: This checks whether all the squares are clean.
• Path cost: Each step costs 1, so the path cost is the number of steps in the path.
Real world problems of AI
• States: Each state obviously includes a location (e.g., an airport) and the current time. Furthermore, because
the cost of an action (a flight segment) may depend on previous segments, their fare bases, and their status as
domestic or international, the state must record extra information about these “historical” aspects.
• Initial state: This is specified by the user’s query.
• Actions: Take any flight from the current location, in any seat class, leaving after the current time, leaving
enough time for within-airport transfer if needed.
• Transition model: The state resulting from taking a flight will have the flight’s destination as the current
location and the flight’s arrival time as the current time.
• Goal test: Are we at the final destination specified by the user?
•Path cost: This depends on monetary cost, waiting time, flight time, customs and immigration procedures, seat
quality, time of day, type of airplane, frequent-flyer mileage awards, and so on.
Problem components with respect to Arad Bucharest example
• Initial state: The starting point of the agent, such as "In(Arad)" in the Romania example.
• Actions: The set of possible actions from a given state, like {Go(Sibiu), Go(Timisoara), Go(Zerind)} from
Arad.
• Transition model: Describes the result of an action, where a function returns the next state, such as
transitioning from "In(Arad)" to "In(Zerind)" after the action Go(Zerind).
• State space: The set of all states reachable from the initial state, forming a directed graph of states and
actions.
• Goal test: Determines if the current state is the goal state, like checking if the agent is "In(Bucharest)."
Model based agent with its snippet and diagram
• The Model-based agent can work in a partially observable environment, and track the situation.
• A model-based agent has two important factors:
• Model: It is knowledge about "how things happen in the world," so it is called a Model-based
agent.
• Internal State: It is a representation of the current state based on percept history.
• These agents have the model, "which is knowledge of the world" and based on the model they perform
actions.
• Updating the agent state requires information about:
• How the world evolves
•
•
•
•
•
•
•
•
• How the agent's action affects the world.
Concepts of agents and environments along with a block diagram
• An “agent” is anything that can be viewed as perceiving its environment through sensors and acting
upon that environment through actuators.
• Percept refers to the agent’s perceptual inputs at a given time instant; an agent’s perceptual sequence is
the complete history of everything the agent has ever perceived.
• In general, an agent’s choice of action at any given instant can depend on the entire precept sequence
observed to date, but not on anything it hasn’t perceived.
• Agents include humans, robots, softbots, thermostats, etc.
• The agent function maps from percept histories to actions: f : P ∗→ A
• The agent program runs on the physical architecture to produce f.
Applications of AI
• Robotic vehicles: A driverless robotic car “STANLEY”
• Speech recognition: A traveler calling United Airlines to book a flight
• Autonomous planning and scheduling: A hundred million miles from Earth, NASA’s Remote Agent
program became the first on-board autonomous planning program to control the scheduling of
operations for a spacecraft
• Game playing: IBM’s DEEP BLUE became the first computer program to defeat the world champion
in a chess match
• Spam fighting: Each day, learning algorithms classify over a billion messages as spam
• Logistics planning: DART (Cross and Walker, 1994), to do automated logistics planning and scheduling
for transportation (Persian Gulf crisis of 1991,)
• Robotics: Roomba robotic vacuum cleaners for home use
• Machine Translation: A computer program automatically translates from Arabic to English
Categories of AI with respect to thinking and acting
Rational agent for vacuum cleaner with its dependencies
The performance measure that defines the criterion of success.
• The agent’s prior knowledge of the environment.
• The actions that the agent can perform.
• The agent’s percept sequence to date.
Ex: • The performance measure awards one point for each clean square at each time step, over a “lifetime” of
1000-time steps.
• The “geography” of the environment is known a priori but the dirt distribution and the initial location of the
agent are not. Clean squares stay clean and sucking cleans the current square. The Left and Right actions move
the agent left and right except when this would take the agent outside the environment, in which case the agent
remains where it is.
• The only available actions are Left, Right, and Suck.
• The agent correctly perceives its location and whether that location contains dirt.
Learning agent and explain its components
• A learning agent in AI is the type of agent which can learn from its past experiences, or it has learning
capabilities.
• It starts to act with basic knowledge and then able to act and adapt automatically through learning.
• A learning agent has mainly four conceptual components, which are:
• Learning element: It is responsible for making improvements by learning from environment
• Critic: Learning element takes feedback from critic which describes that how well the agent is doing with
respect to a fixed performance standard.
• Performance element: It is responsible for selecting external action
• Problem generator: This component is responsible for suggesting actions that will lead to new and
informative experiences.
• Hence, learning agents are able to learn, analyze performance, and look for new ways to improve the
performance.
Problem-solving agent for Arad example with respect to goal formulation, problem formulation, and the
process of searching for solution with its assumptions
• Initial state: The starting point of the agent, such as "In(Arad)" in the Romania example.
• Actions: The set of possible actions from a given state, like {Go(Sibiu), Go(Timisoara), Go(Zerind)} from
Arad.
• Transition model: Describes the result of an action, where a function returns the next state, such as
transitioning from "In(Arad)" to "In(Zerind)" after the action Go(Zerind).
• State space: The set of all states reachable from the initial state, forming a directed graph of states and
actions.
• Goal test: Determines if the current state is the goal state, like checking if the agent is "In(Bucharest)."
Airline travel problem.
Route-finding algorithms are used in a variety of applications. Some, such as Web sites and in-car systems that
provide driving directions, are relatively straightforward extensions of the Romania example. Others, such as
routing video streams in computer networks, military operations planning, and airline travel-planning systems,
involve much more complex specifications.
The airline travel problems that must be solved by a travel-planning Web site:
• States: Each state obviously includes a location (e.g., an airport) and the current time. Furthermore, because
the cost of an action (a flight segment) may depend on previous segments, their fare bases, and their status as
domestic or international, the state must record extra information about these “historical” aspects.
• Initial state: This is specified by the user’s query.
• Actions: Take any flight from the current location, in any seat class, leaving after the current time, leaving
enough time for within-airport transfer if needed.
• Transition model: The state resulting from taking a flight will have the flight’s destination as the current
location and the flight’s arrival time as the current time.
• Goal test: Are we at the final destination specified by the user?
• Path cost: This depends on monetary cost, waiting time, flight time, customs and immigration procedures, seat
quality, time of day, type of airplane, frequent-flyer mileage awards, and so on
Toy problems of AI
• States: A state description specifies the location of each of the eight tiles and the blank in one of the nine
squares.
• Initial state: Any state can be designated as the initial state. Note that any given goal can be reached from
exactly half of the possible initial states
• Actions: The simplest formulation defines the actions as movements of the blank space Left, Right, Up, or
Down. Different subsets of these are possible depending on where the blank is. • Transition model: Given a
state and action, this returns the resulting state; for example, if we apply Left to the start state in Figure, the
resulting state has the 5 and the blank switched. • Goal test: This checks whether the state matches the goal
configuration shown in Figure (Other goal configurations are possible.)
• Path cost: Each step costs 1, so the path cost is the number of steps in the path.
Formulation of problems with respect to abstraction for Arad Romania example
• The problem of finding a route to Bucharest is formulated using abstract components like the initial state,
actions, transition model, goal test, and path cost. However, this is a simplified model that omits real-world
details such as traveling companions, road conditions, and weather, which are irrelevant to the problem.
This simplification is called abstraction. Abstraction also simplifies actions, focusing only on changes in
location, while ignoring other effects like fuel consumption or road safety.
• A good abstraction should be broad enough to encompass many detailed states and actions, yet specific
enough to ensure that solutions can be easily translated into real-world actions. The goal is to remove
unnecessary details while retaining the core information needed to solve the problem effectively. Without
abstraction, agents would be overwhelmed by the complexity of the real world.