1

MODULE 6

Planning & Learning

Mrs. Rujuta Vartak

2
PLANNIG PROPLEM
• Problem: Get books from store.
• Given:
a. Initial State: The agent is at home without book
b. Output state: The agent is at home with book.
• Predicate Calculus Convert
a. State: At(x)
b. Actions: Go(y), Buy(z)
3
Components of planning
• Choose best rule to apply
• Apply rules
• Detecting a solution
• Detecting dead ends
• Repairing an almost correct solution
4
Partial order planning
• Approach to automated planning that maintains a partial ordering between
actions and only commits ordering between actions when the actions are partial
• The planning does not specify which actions will come out first when two
actions are processed
• A partial order plan or partial plan specifies all the actions that need to be taken,
but only specifies the order between actions when necessary
• Also called a non linear planner
5

• Components:
1. A set of actions(operators)
2. A set of Preconditions: → a set of actions together with partial ordering ,
representing a before relation on actions
3. A set of casual links : → specifies which actions meet which preconditions of
other actions act 0 < act 1
4. A set of open preconditions
5. Casual links
 • E.g. Partial order planning of Wearing Shoe
6
 • Set of actions:
7 • These are the steps of plan.
• For e.g.: Set of Actions = {Start, Rightsock, Rightshoe ,Leftsock, Leftshoe, Finish}
• Set of ordering constraints/preconditions:
• i. Preconditions are considered as ordering constraints.(i.e. without performing action “x”
we cannot perform action “y”)
• ii. For e.g.: Set of ordering = {Right-sock <right-shoe; left-sock<left-shoe}="" that="" is=""
in="" order="" to="" wear="" shoe,="" first="" we="" should="" wear="" a="" sock.<=""
p="">
• Set of causal links:
• Action A achieves effect “E” for action B
• For e.g. Set of Causal Links = {Right-sock-> Right-sock-on->Right shoe}
8

• Set of open preconditions:

• i. Preconditions are called open if it cannot be achieved by some actions in the plan.
• ii. Consistent Plan is a Solution for POP Problem
• iii. A consistent Plan doesn’t have cycle of constraints; it doesn’t have conflicts in the
causal links and doesn’t have open preconditions so it can provide a solution for POP
problem.
9
Working
• Begin with the action start and finish and partial order start < finish
• Set of (P, A) pairs where A is an action in the plan and P is the precondition of A
that must be achieved
• At each stage in the planning process , a pair (G, act 1) is selected from the
agenda
• Then an action act0 is chosen to achieve P
• IT could be the start action or new action
• Act0 must happen before act1
10

• Adds a casual link that records that act0 achieves P for action act1
• If act0 is a new action, its preconditions are added and the process continues till
the agenda is empty
11
Total order Planning
• Also called Linear Planner
• Forward/backward sate space searches are forms of totally order plan search.
• Explore only strictly linear sequences of actions directly connected to the start or
goal
12 Example
13
Learning agent
• 1. Physical agents
• Act in a physical environment
• Eg: sending a agent to a building, it must know where it is , it must move and search
• Consists of : -
• Computers
• Sensors
• Effectors
• Auxilliary equipment → tools
14 Continue…
 Key Components of Learning Agents
This Learning Agents is enabled by the synergy of different components:
15 1. Sensors/Perceptors: Sensors or perceptors collect information from the environment and
send it to the agent, allowing for decision-making and acquisition of knowledge.

2. Critic: The critic assesses and offers feedback on the agent's performance based on pre-
established goals or a predetermined reward system. The critic supports the learner by
providing feedback on the quality of their decisions, allowing them to enhance their skills
through various activities.

3. Learning Element: This part acts as the central cognitive hub of the agent, responsible
for analyzing the experiences acquired from interactions with the surroundings. Through
the use of different machine learning algorithms like reinforcement learning
or supervised learning, the learning component consistently updates the agent's internal
model or knowledge base, consequently improving its decision-making abilities.
16 • Performance Element: The performance element requires the learning element
and critic feedback so as to manage the agent's activities in an environment. In
selecting those actions that are most likely to help it achieve its goals, the
performance element takes the agent to the best possible outcomes.
• Actuators/Effectors: Effectors, also called actuators, carry out tasks selected by
the performance element. They adjust behaviors based on individuals'
judgments as conducted in response to choices made by them. Actuators can
come in various types depending on the designs of various agents.
• Problem Generator: The problem generator is in charge of creating challenges or
activities for the agent to complete. It consists of situations that require the agent
to apply the knowledge and skills it has gained, hence improving ongoing
learning and talent development.
17
Supervised learning
• Learning that takes place based on a class of examples
• Learning based on labelled data
• While learning, the system has knowledge of a set of labelled data
• Comprised of a series of algorithms that build mathematical models of certain
data sets that are capable of containing both inputs and the desired output for
that particular machine
18

• The data being inputted into the supervised learning method, is known as
training data
• Training example is represented by an array , also known as vector or a feature
vector
• Training data is represented by a matrix

• Uses classification and regression techniques

19
Working
• Models are trained using labelled dataset, where the model learns about each
type of data.
• Once the training process is completed, the model is tested on the basis of test
data and then predicts the output
20
steps
• 1. Determine the type of training dataset
• 2. Collect/gather the labelled training data
• 3. Split the training dataset, test dataset and validation dataset
• 4. Determine the input features of the training dataset, which should have enough
knowledge so that the model can accurately predict the output
• 5. Determine the suitable algorithm for the model, such as SVM, decision tree. Etc
• 6. Execute the algorithm on the training dataset
• 7. Evaluate the accuracy of the model by providing the test set
21

• Advantages :
• Model can predict the output on the basis of prior experiences
• Helps to solve various real world problems such as fraud detection, spam filtering
etc

• Disadvantages :
• Not suitable for complex task
• Cannot predict, if the test data is different
22
Unsupervised learning
• Refers to learning from unlabelled data
• All similar items are clustered together in a particular class where the label of a
class is not known
• Set of algorithms, where the only information being uploaded is inputs
• The device itself then is responsible for grouping together and creating ideal
outputs based on the data it discovers
• Types : → clustering , Association
23

• Clustering
• Method of grouping the objects into clusters such that objects with most
similarities remain into a group and has less or no similarities with the objects of
other group

• Association
• Used for finding the relationship between variables in the large database. It
determines the set of items that occur together in the dataset
24

• Advantages: → used for more complex task

• Disadvantages :→ the result might be less accurate
25
Semi supervised learning
• Combines a small amount of labelled data with a large amount of unlabelled
data during training
• Can produce considerable improvement in learning accuracy
26
Ensemble learning
• A machine learning model that combines the predictions from two or more
models
• The models that contribute to the ensemble are called as ensemble members
• They may be of the same type or different type
• They may or may not be trained on the same training data
27
Reinforcement learning
• Feedback based machine learning technique in which an agent learns to behave
in an environment by performing the actions and seeing the results of actions
• For each good action, the agent gets positive feedback
• Negative for each bad action
• Learns automatically using feedbacks without any labelled data
28
Approaches to rl
• 1. Value based → to find the optimal value function
• Agent expects the long term return at any state under policy (pi)

• 2. Policy based → to find the optimal policy

• 3. Model based → a virtual model is created for the environment

29
expert system
• AI based system that converts the knowledge of an expert in a specific subject
into a software code
• This code can be merged with other such codes based on the knowledge of other
experts, and used for answering queries submitted through a computer
• Expert system is a piece of software which use databases of expert knowledge to
offer advice or make decisions in such areas as medical diagnosis
• Computer program that contains a knowledge base and a set of algorithms or
rules that infer new facts from knowledge and from incoming data
30
31

• Expert system = knowledge + Inference engine

• Components :
• 1. Knowledge base :→ It is a warehouse of the domain specific knowledge
captured from the human expert via the knowledge acquisition model
• 2. Inference engine : → also called rule interpreter, performs the task of matching
antecedents from the responses given by the user and firing rules
32

• 3. User interface
• 4. Knowledge acquisition facility :→ for the expert to share the expertise with the
system
• 5. External interface → provides the communication link between the ES and
external environment
• 6. Explanation facility → explaining the reasoning process to user
33
Knowledge Acquisition
• In artificial intelligence, knowledge acquisition is the process of gathering,
selecting, and interpreting information and experiences to create and maintain
knowledge within a specific domain. It is a key component of machine learning
and knowledge-based systems.
34
Techniques of knowledge acquisition
• 3 kinds of cases :
• 1 ) Current → by watching the expert perform a task
• 2) Historical → by discussing with the expert a task that was performed in the
past
• 3) Hypothetical → by having the expert describe how a task should be
performed in a hypothetical situation
35
techniques
[Link] generation techniques → This method includes many types of
interviews, reporting and observational techniques
• 2. Protocol analysis techniques → used with transcripts of interviews or text
based info to identify basic knowledge objects within a protocol such as goals,
decisions , relationship , attributes(Expert is asked to perform verbalized
process)
• 3. Hierarchy generation techniques → involves creation, reviewing and
modification of hierarchical knowledge
36

• 4. Matrix based techniques → 2D matrix

37
Inference engine
• Program part of expert system
• Represents a problem solving model which uses the rules in the KB, and the
situation specific knowledge
• A rule can be fired if the antecedents are satisfied
• There are rules for which consequents match the current goal of the system
• Out of the rules in the KB, the engine selects one rule based on some predefined
criteria
38

• 2 types
• 1) deterministic → the conclusions drawn from this type are assumed to be true,
based on facts and rules
• 2) Probabilistic → contain uncertainty

• Uses
• Forward chaining –what can happen next?
• Backward chaining- why this happened?
39
User interface
• Interface that helps a non expert user to communicate with the expert system to
find a solution
40
Working memory
• Represents the set of facts known about the domain
• Elements reflect the current state of the world
• Contains information about the particular instance of the problem being
addresses
41
Development of expert system
• Steps:
• Identification of problem .
• Decision about the mode of development
• Development of a prototype
• Test and refine prototype
• Develop and complete the Expert System
• Maintain the System.
42

Thank You.