Chapter No 2.

Introduction to AI Agents and Learning and Adaptation of

Agents
What are AI Agents, Lang Chain Overview, Agent Architecture, Agent Autonomy,
Reinforcement learning in agentic systems, Learning from environment and user feedback,
Meta-cognition and self-adaptation in agents. Goal modelling and reasoning, planning
under uncertainty, Integrating symbolic and sub-symbolic reasoning.

What are AI Agents?

An AI Agent is a computational system that perceives its environment through sensors,
processes information, makes decisions, and acts upon the environment through actuators
to achieve specific goals autonomously.

Key Components:
 Perception: Interpreting sensor data
 Decision-Making: Using rules, logic, or learning algorithms
 Actuation: Executing actions to affect the environment
 Learning: Updating internal models based on feedback

Example: A chatbot that answers queries and improves its answers over time by learning
from user interactions.

Types of AI Agents
 Simple reflex agent
  Model based agent
 Goal based agent
 Utility based agent
 Learning agent

Simple Reflex Agent

• Simple reflex agent is said to be the simplest kind of agent. These agents select an
action based on the current percept ignoring the rest of the percept history.
• condition-action rules
if {set of percepts} then {set of actions}
For example,
if it is raining then put up umbrella
Example:
A home thermostat that knows how to start heating or cooling your house based on
reaching a certain temperature.

Model based agent

Model-based agent is known as Reflex agents with an internal state.
The cycle works like this:
• Sensors gather information about the current state of the environment.
• Internal state – It has two information.
1. Some information related to how the world evolves independently to agent
and
2. How world is affected by the action of agent.
• The agent updates its understanding of "what the world is now like"
• Using its knowledge of "how the world evolves" and "what my action do," it
determines "what action should I do now"
• Actuator executes this action in the environment
• Environment changes, and the cycle repeats
Example of Model based Agent
Roomba vacuum cleaner and the autonomous car known as Waymo.
• Both interact with their environments by using what they know--an internal model of
the world--and their on-board sensors as well, to make moment-to-moment
decisions about their actions.

Goal based agent

Unlike reactive agents that just respond to current conditions, this agent predicts
consequences before acting.
Goal based Agent enhances decision process as follows:
1. State Assessment: The agent determines "What the world is now like?" using sensor
input and its internal state model.
2. Predictive Modeling: The agent asks "What it will be if I take action A?" This involves:
i. Using knowledge of "How does the world evolve?"
ii. Combining this with understanding of "What my action do?"
iii. Predicting future states for different possible actions.
3. Goal-Directed Selection: The agent decides "What action should I do now?" by:
Comparing predicted outcomes against its Goal.
Example:
A self-driving car with the goal of "reach destination safely" would:
• Assess current traffic conditions
• Predict what happens if it changes lanes vs. stays in current lane
• Choose the action that best balances progress toward destination with safety
constraints.

Utility based agent

A utility-based agent is an intelligent system that makes decisions by evaluating how "happy"
or satisfied it will be with different outcomes.
This approach goes beyond simple goal-seeking by considering the relative desirability of
various states.
Core Component:
Utility Function: Measures satisfaction or "happiness" for different states
Decision Process:
• World Model: The agent maintains knowledge of how the world currently looks and
how it evolves over time
• Action Prediction: For each possible action, it predicts the resulting world state
• Utility Evaluation: It calculates how happy or satisfied it would be in each predicted
state
• Action Selection: It chooses the action that leads to the highest utility (greatest
satisfaction).
Key Advantages
 • Handles Trade-offs: Unlike simple goal-based agents that treat goals as binary
(achieved/not achieved), utility-based agents can weigh competing objectives. For
example, a delivery robot might balance speed versus energy consumption.
• Rational Decision Making: When multiple goals conflict or when there are degrees of
success, the utility function provides a principled way to choose the best action.
• Flexible Preferences: The same agent architecture can exhibit different behaviors
simply by changing the utility function, making it adaptable to various scenarios.
Example:
A self-driving car, for instance, has many goals to consider when heading toward its
destination: choosing the quickest route, ensuring the safety of its passengers, avoiding road
closures or traffic jams, among others. This makes it a utility-based agent.

Learning agent
A learning agent in AI is the type of agent that can learn from past experiences or has
learning capabilities. It starts to act with basic knowledge and then is able to act and adapt
automatically through learning.

Core Components
• Performance Element
• Learning Element
 • Critic
• Problem Generator
Performance Element: This is the "doing" part of the agent - it selects actions and
interacts with the environment based on current knowledge. It's essentially the decision-
making component that chooses what actions to take in any given situation.
Learning Element: This is the "improving" part that modifies the performance element
based on feedback. It analyzes experiences, identifies patterns, and updates the agent's
knowledge or decision-making strategies to perform better in the future.
Critic: This component evaluates how well the agent is performing by comparing actual
outcomes against the performance standard. It provides feedback to the learning
element about whether actions were good or bad, and by how much.
Problem Generator: This component suggests exploratory actions that might lead to new
and informative experiences. It helps the agent discover better strategies by encouraging
exploration rather than always exploiting current knowledge.
Learning Cycle
The process works as a continuous feedback loop:
• Sensor input provides information about the current environment state
• Performance Element selects and executes actions through the actuator
• Critic evaluates the results against the performance standard and provides feedback
• Learning Element uses this feedback to update knowledge and improve future
decision-making
• Problem Generator suggests new experiences to explore, contributing to learning
goals
Example:
• Recommendation engines that learn user preferences
• Autonomous vehicles that improve driving behavior through experience
• Game-playing AI that discovers new strategies
• Chatbots that become better at understanding and responding to user queries over
time.

Multi-Agent Systems (MAS)

MAS involves multiple intelligent agents working in the same environment. Each agent can
think, perceive, decide, and act independently. MAS is inspired by natural systems like ant
colonies, bird flocks, or human teamwork.
How do the agents behave?
• Agents can cooperate to reach a shared goal (e.g., rescue robots)
• They may act independently to achieve personal goals (e.g., delivery drones)
• Agents can also be competitive or adversarial (e.g., in games or simulations)
Key features of MAS:
• Autonomy – Each agent makes its own decisions.
• Interaction – Agents communicate and influence one another.
• Distributed Control – No central controller; decisions are spread among agents.
• Scalability – New agents can be added easily without breaking the system.
Example:
• Traffic management: Smart traffic lights coordinating flow
• Online marketplaces: Buyers and sellers interacting
• Distributed computing: Multiple computers solving problems together
• Smart grids: Energy systems balancing supply and demand
• Video games: Multiple AI characters with different behaviors

LangChain Overview
What is LangChain?
LangChain is a framework for developing applications powered by language models. It
provides abstractions to chain together language model queries and tool calls in an agent
loop. It offers modules for connecting to external data sources, maintaining long-term
memory, and managing multi-step conversations or reasoning processes. Because of its
modular architecture, LangChain makes it easier to integrate various components and allows
flexible orchestration of complex AI workflows. A developer can easily create an agent using
LangChain, for instance, that reads a user's question, determines it needs to perform a
computation or retrieve data from a URL, calls a tool to accomplish this, and then utilizes the
outcome to create a final response. All of this is coordinated within the LangChain agent
loop. It features agent and chain abstractions that codify best practices for tool-using AI
agents, and its design places a strong emphasis on extensibility (allowing integration with
numerous LLM providers and tools).
Core Components
Lang-Chain Application (Top Layer)
• The final application that users interact with
• It can be a chatbot, document analyzer, code generator, or any AI-powered tool
• Built by combining all the components below.
Main Building Blocks
Chains (Green):
• Sequences of operations that process information step-by-step
• Like a recipe that tells the system what to do first, second, third, etc.
• Example: "Read document → Summarize → Answer questions about summary"
Agents (Blue):
• Decision-making components that can choose which tools to use
• Act like intelligent assistants that can reason about what action to take next
• Can dynamically decide whether to search the web, run code, or call other tools
Tools (Orange):
 • External capabilities the system can use
• Examples: web search, calculators, APIs, databases, code execution
• Extend what the language model can do beyond just generating text
Memory (Purple):
• Stores conversation history and context
• Helps maintain continuity across multiple interactions
• Remembers what was discussed earlier in the conversation
Foundation Layer
Language Models (Red):
• The core AI models like GPT, Claude, or other LLMs
• Provide natural language understanding and generation capabilities
• The "brain" that processes and generates text
Vector Stores (Brown):
• Databases for storing text embeddings
• Enable semantic search and similarity matching
• Help find relevant information from large document collections
Document Loaders (Green):
• Import data from various sources
• Can read PDFs, websites, databases, text files, etc.
• Convert different file formats into a standard format the system can work with.
How It All Works Together
• Document Loaders bring in data from various sources
• Vector Stores organize and index this information for quick retrieval
• Language Models provide the core intelligence
• Memory maintains context across interactions
• Tools extend capabilities beyond text generation
• Agents make smart decisions about which tools to use
• Chains orchestrate the entire workflow
• LangChain Application presents the final user experience
Real-World Example
 Customer service chatbot:
 Imagine you're chatting with an e-commerce store bot:
 User: Where’s my order?
 Memory: Recognizes user has already authenticated
 Tool: Calls order database
 LLM: Interprets the intent (tracking request)
 Agent: Decides to fetch real-time data
 Response: “Your order #12345 is currently in transit and will be delivered on Aug 2.”
Key Technologies Used in AI Agents

AI Agent Frameworks and Tools

Agent Architecture:
AI Agent Architecture defines how different modules interact:

The Three-Layer Architecture of intelligent agents, helps agents operate effectively in

dynamic environments.
Reactive Layer
Purpose: Handles quick, low-level responses.
Key Functions:
• Immediate Response – Fast reaction to changes or threats.
• Reflex Actions – Predefined actions (like avoiding obstacles).
• Emergency Handling – Responding to critical situations instantly.
Think of it as the agent’s reflexes.
Deliberative Layer
Purpose: Handles high-level thinking and reasoning.
Key Functions:
• Planning – Deciding future steps or strategies.
• Goal Management – Choosing and prioritizing goals.
• Decision Making – Selecting the best action based on knowledge.
This is the “thinking” part of the agent.
Meta-Level
Purpose: Oversees and improves the agent's own behavior.
Key Functions:
 • Self-Reflection – Analyzing past actions and performance.
• Learning Control – Adjusting learning processes.
• Strategy Adaptation – Changing strategies based on outcomes.
This is the agent “thinking about its thinking.”
Example:
In a self-driving car:
• Reactive Layer ensures safety in real time,
• Deliberative Layer ensures smart navigation and decisions,
• Meta-Level ensures long-term improvement and adaptability.
Reactive: Immediate brake when sensor detects obstacle Deliberative: Plan route from A to
B considering traffic Meta-Level: Learn that certain routes are consistently slow and adapt
planning algorithm

Agent Autonomy
Agent autonomy refers to an agent’s ability to operate on its own, without direct human
control, while making its own decisions based on its goals, knowledge, and environment.
Full Autonomy (Top - Red)
 The system operates entirely on its own.
 No human input is needed.
 Example: Autonomous drones or self-driving cars that make decisions in real-time
without human help.
Supervised Autonomy (Blue)
 The system performs tasks independently but under human supervision.
 A human can intervene if necessary.
 Example: AI surveillance systems monitored by operators.
Semi-Autonomy (Green)
 The system and human share control.
 The system may handle routine tasks, while humans manage complex or critical
decisions.
 Example: Advanced driver-assistance systems (ADAS) in cars.
Human-in-the-Loop (Orange)
 Human input is required during the system’s operation.
 The machine supports the human but doesn't act independently.
 Example: AI systems suggesting decisions but requiring human confirmation.
Manual Control (Bottom - Purple)
 The human controls all actions directly.
 No automation involved.
 Example: Driving a non-automated car or manually operating industrial machinery.

Factors Affecting Autonomy

 Domain Complexity: Simple vs complex environments.
 Simple domains (e.g., sorting emails, cleaning a floor) allow higher autonomy
because rules are clear and the range of possible actions is limited.

Complex domains (e.g., air traffic control, medical diagnostics) involve many
variables, constraints, and interdependencies, making autonomous decision-making
more challenging.

As domain complexity increases, the need for human oversight, error handling, and
contextual awareness grows.

Example: A robot vacuum can work autonomously in a home, but a surgical AI

requires human guidance in complex and dynamic medical settings.

 Risk Level: High-stakes vs low-stakes decisions.

High-stakes decisions (e.g., autonomous weapons, emergency medical response)

demand more caution, explanation, and oversight due to potential harm or
irreversible consequences.

Low-stakes decisions (e.g., playlist generation, product recommendations) pose

minimal risk and can be delegated with greater autonomy.

The potential impact of failure determines how freely the AI can act. Example:
Autonomy in selecting a music track is acceptable, but not in deciding a patient's
cancer treatment plan.

 Uncertainty: Predictable vs unpredictable scenarios

In predictable environments, the AI can act confidently with minimal supervision

because outcomes are stable and patterns are known.

In unpredictable or stochastic environments (e.g., natural disasters, live

negotiations), autonomous behavior must be constrained or supported with fallback
mechanisms.

Systems must be able to handle or adapt to unexpected events to justify higher

autonomy. Example: An AI thermostat functions well in predictable indoor
conditions, but an autonomous drone navigating during a storm requires adaptive
responses and limits on its freedom.

 Ethical Considerations: Moral and legal implications

Autonomy must be constrained when actions involve human rights, fairness,

accountability, or legal consequences.
 Ethical principles such as non-maleficence, justice, and transparency guide how
much autonomy is appropriate.

This includes compliance with regulations like GDPR, AI Act, or medical ethics.
Example: A hiring algorithm must not autonomously reject applicants without
explainable criteria, as this could introduce bias and legal issues.

 Technical Maturity: Reliability of AI systems.

The robustness, accuracy, and generalizability of the AI system directly impact its
level of autonomy.

Mature systems with well-tested models, error handling, and fail-safes can operate
more independently.

Immature systems may need tight constraints or human-in-the-loop control to avoid

unintended behaviour. Example: A mature self-parking system in a car may operate
fully autonomously in most conditions, while a beta-version navigation system may
require human confirmation at every step.

Factor Low Autonomy High Autonomy

Domain
Complex, multi-variable Simple, rule-based
Complexity
Low-stakes
High-stakes (health, safety,
Risk Level (entertainment,
finance)
recommendations)
Uncertainty Unpredictable, volatile Stable, predictable
Ethical Strong moral/legal
Ethically neutral tasks
Considerations implications
Technical Proven, reliable
Experimental or unreliable
Maturity systems

Reinforcement Learning in Agentic Systems

Reinforcement Learning (RL) is a machine learning paradigm where an agent learns
optimal behaviour through trial and error by interacting with its environment. The
agent’s goal is to maximize cumulative reward over time.
RL Cycle:
 State: Current situation
 Action: Move taken
 Reward: Feedback received
 Policy: Strategy mapping states to actions
The RL Cycle in Action
 The agent observes the current state of the environment.
 It uses its policy to choose an action.
 The environment responds, producing a new state and a reward.
 The agent learns from the reward and updates its policy to improve future decisions.
This cycle continues until the agent learns an optimal policy — a strategy that yields the
highest cumulative reward over time.

Example:
Reinforcement Learning Cycle:
Agent observes the current state
The robot sees:
 Its GPS location
 A map of the environment
 Nearby pedestrians or obstacles
 Traffic rules
Agent uses policy to choose an action
 It decides:
 “Turn left,”
 “Speed up,”
 or “Wait for a pedestrian to pass.”
Environment responds with new state and reward
 After turning:
 The robot moves closer to the delivery spot (positive reward).
 Or it gets stuck behind a crowd (no reward or even negative if delayed).
Agent updates its policy
 It learns that slowing down near crowds avoids penalties, while choosing less
crowded paths improves delivery time.
 Over time, it adjusts its decision-making strategy to favor faster and safer routes.
Cycle Continues
 Every trip improves the robot’s behavior.
 Eventually, it learns an optimal policy

Learning from Environment and User Feedback

This diagram illustrates a Multi-Source Learning Architecture for intelligent or autonomous
agents.
It extends the traditional reinforcement learning cycle by incorporating multiple sources of
feedback to enhance learning and adaptability.

Top: Learning Agent

 This is the central intelligent system (e.g., an AI model, robot, or software agent).
 It receives feedback, learns from it, and updates its behavior accordingly.
Middle Layer: Multiple Feedback Sources
These are the inputs that influence the agent’s learning:
Environment Feedback
 Traditional RL feedback (rewards, penalties) based on actions in the environment.
  Example: A robot bumps into a wall → negative reward.
User Feedback
 Feedback directly from human users, such as ratings, corrections, or preferences.
 Example: A user says “this answer is incorrect” to an AI assistant.
Peer Feedback
 Input or insights from other agents or systems.
 Example: One AI system shares a learned strategy with another.
These diverse feedback streams make the agent more flexible, accurate, and responsive
than relying on a single source.
Bottom Layer: Learning Outcomes
These are the mechanisms the agent uses to learn and evolve from feedback:
Experience Replay
 The agent stores and reuses past experiences to reinforce learning.
 Helps stabilize and improve learning over time.
Knowledge Update
 The agent adjusts its internal models or policies based on new feedback.
 Enables continual learning and memory refinement.
Behavior Adaptation
 The agent modifies its future behavior to align better with goals or preferences.
 Makes the agent more dynamic and context-aware.

Feedback Types and Processing

Meta-cognition in Agents

Meta-cognitive (“thinking about thinking”) architecture is a two-layer system where:

• The Object-Level does the work, and the Meta-Level makes sure it’s done well.
Together, they create agents that don’t just act — they learn how to learn and
improve how they act.
Meta-Level (Self-Reflection)
Function: Monitors, evaluates, and adjusts the Object-Level's behavior.
Includes:
• Goal assessment
• Strategy evaluation
• Error detection
• Adaptation and learning
Example: The agent notices it's making repeated errors and decides to change its
approach.
Object-Level (Task Execution)
Function: Directly performs tasks and interacts with the environment. This is the agent’s
operational level where problem-solving, reasoning, and actions happen.
Example: Navigating, responding to queries, making decisions, etc.
Environment
Function: The real or virtual world the agent interacts with.
Provides input and receives actions from the Object-Level.
Example: Sensors feeding data to a robot; user queries in a chatbot environment.
Bidirectional Flow
Meta-Level ↔ Object-Level:
The Meta-Level monitors and modifies the Object-Level, while also receiving feedback
from it to improve future strategies
Object-Level ↔ Environment:
The Object-Level perceives the environment and acts within it.
This architecture enables agents to:
• Self-monitor and adapt in complex or changing conditions
• Handle uncertainty and failure better
• Exhibit more robust, intelligent, and human-like behavior

Self Adaptation in Agents

Self-adaptation refers to an agent’s ability to adjust its behaviour and strategies in
response to changes in the environment or its own internal state.
Key Mechanisms of Self-Adaptation:
• Environmental Feedback: Adjusting behavior based on observed changes or rewards.
• Policy Adjustment: Changing decision-making rules or heuristics.
• Learning from Experience: Using reinforcement learning, meta-learning, or online
learning.
• Resource Awareness: Adapting based on computational limits or energy constraints.
Example:
An autonomous drone encountering unexpected wind conditions adjusts its flight path
and control algorithms to maintain stability and safety.

Relation Between Meta-Cognition and Self-Adaptation

Meta-cognition enables self-adaptation by providing the mechanisms for:
• Assessing when adaptation is needed
• Selecting how to adapt (strategy-level decisions)
• Evaluating the success of the adaptation
Goal modelling and reasoning
Goal modeling defines what the agent wants to do. Goal reasoning figures out how, when,
and whether it should do it. This capability is central to autonomous, intelligent, and
purposeful behavior in agentic AI systems.
Together, Goal Modeling + Reasoning Allow Agents to:
• Act deliberately (not reactively)
• Adapt to changes or failure
• Pursue complex tasks in multi-step, multi-agent environments

Planning under Uncertainty

• Planning in AI refers to the process where an agent selects a sequence of actions to

transition from an initial state to a goal state.
• In real-world scenarios, environments are rarely perfect — they may be uncertain,
incomplete, or changing over time.
• To act effectively under such conditions, agents need to plan in a way that accounts
for risks, unknowns, and change.

Challenges in Planning Under Uncertainty

Incomplete Knowledge

 The agent doesn’t have full visibility into the environment or its current state.
 Causes: sensor limitations, hidden variables, lack of prior data.
 Planning must incorporate assumptions or seek information to reduce uncertainty.
Example: A delivery robot doesn't know if a hallway is blocked until it gets closer.
Stochastic Environments
 Actions have probabilistic outcomes — the same action may not always lead to the
same result.
 Requires modeling the likelihood of different outcomes and choosing actions that
maximize expected utility.
Example: A drone may attempt to land, but wind conditions could cause it to miss
the target slightly.
Dynamic Changes
 The environment changes over time, possibly due to other agents or external
factors.
 Plans may need to be re-evaluated and updated in real time.
Example: A warehouse robot’s path must adapt if a human worker unexpectedly
blocks its route.

Approaches to Planning Under Uncertainty

Probabilistic Planning (e.g., POMDPs)

  POMDP (Partially Observable Markov Decision Process) is a formal framework that
models:
o Uncertain action outcomes
o Partial observability
o Sequential decision-making
 The agent maintains a belief state (a probability distribution over possible actual
states) and selects actions that maximize expected rewards.
Use Case: Healthcare decision support systems where the patient’s true health state
isn't fully known, but actions like testing or treatment must be planned.

Contingency Planning
 Also known as "if-then" planning.
 The agent prepares branches in its plan to handle different possible future events.
 Especially useful in dynamic or partially known environments where the agent
cannot commit to a single fixed path.
Example:
"If the door is locked, then try the window. If the window is closed, then find another
entrance."
Use Case: Rescue robots in disaster zones where obstacles or hazards are not fully
known ahead of time.

Heuristic-Based Planning
 Uses domain-specific heuristics (rules of thumb) to guide planning efficiently in large
or complex spaces.
 Focuses on reducing search complexity and prioritizing promising actions.
 Often used in real-time or constrained scenarios where full computation isn't
feasible.
Example: A game-playing AI estimating that "moving toward the center" gives better
control of the board.
Use Case: Navigation systems, game AI, and robotics where quick decisions are
needed.

Comparison Table

Approach Strengths Limitations

Rigorous handling of
Computationally expensive,
POMDPs uncertainty and partial
hard to scale
info

Contingency Flexible to environmental Can become complex with

Planning changes many branches
 Approach Strengths Limitations

Fast, scalable, and May not handle deep

Heuristic
practical in large uncertainty or guarantee
Planning
environments optimality

Integrating Symbolic and Sub-symbolic Reasoning

Symbolic: Rule-based, logic-driven (e.g., Prolog)

Definition: Relies on explicit rules, logic, and symbols to represent knowledge and perform
reasoning.

Characteristics:

o Uses structured representations like facts, predicates, and rules.

o Supports deductive reasoning, theorem proving, and interpretable decision-
making.
o Well-suited for domains where rules and logic are clear and well-defined.

Example Technology:

o Prolog — a logic programming language based on symbolic reasoning.

o Expert systems with rule bases.

Strengths:

o Transparency: Decisions are explainable because they follow clear logic.

o Reasoning: Can perform complex symbolic inference and handle abstract
concepts.

Limitations:
 o Poor at handling noisy, unstructured data (e.g., images, speech).
o Difficult to scale to real-world messy environments.

Sub-symbolic: Pattern-based, data-driven (e.g., Neural Networks)

Definition: Relies on data-driven pattern recognition rather than explicit symbolic
manipulation.
Characteristics:
Uses statistical models like neural networks to learn from raw data.
Excels at perception tasks such as image recognition, speech understanding, and
natural language processing.
Learns implicit representations rather than human-readable rules.
Example Technology:
Convolutional Neural Networks (CNNs) for image analysis.
Deep learning models for text and audio.
Strengths:
 Adaptation: Learns from data and improves performance over time.
 Handles noisy, ambiguous, and high-dimensional data effectively.
Limitations:
 Opacity: Often considered “black boxes” with limited explainability.
 Weak at explicit logical reasoning or symbolic manipulation.
Hybrid Systems combine the strengths of both:
 Symbolic = Transparency + Reasoning
 Sub-symbolic = Adaptation + Perception