Syllabus

CS3491 ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING

LIST OF EXPERIMENTS: TOTAL: 30 PERIODS LTPC 0011

1. Implementation of Uninformed search algorithms (BFS, DFS)

2. Implementation of Informed search algorithms (A*, memory-bounded A*)

3. Implement naïve Bayes models

4. Implement Bayesian Networks

5. Build Regression models

6. Build decision trees and random forests

7. Build SVM models

8. Implement ensembling techniques

9. Implement clustering algorithms

10 Implement EM for Bayesian networks

11 Build simple NN models
12 Build deep learning NN models
 CONTENTS
[Link]. Date Experiments Page Mark Sig
No.

1 Implementation of Uninformed search algorithms (BFS,

DFS)

2 Implementation of Informed search algorithms (A*,

memory-bounded A*)

3 Implement naïve Bayes models

4 Implement Bayesian Networks

5 Build Regression models

6 Build decision trees and random forests

7 Build SVM models

8 Implement ensembling techniques

9 Implement clustering algorithms

10 Implement EM for Bayesian networks

11 Build simple NN models

12 Build deep learning NN models

Content beyond Syllabus

13 Build deep learning NN models for digit classification

 Introduction
Machine learning

Machine learning is a subset of artificial intelligence in the field of computer science that often
uses statistical techniques to give computers the ability to "learn" (i.e., progressively improve
performance on a specific task) with data, without being explicitly programmed. In the past
decade, machine learning has given us self-driving cars, practical speech recognition, effective
web search, and a vastly improved understanding of the human genome.

Machine learning tasks

Machine learning tasks are typically classified into two broad categories, depending on whether
there is a learning "signal" or "feedback" available to a learning system:

1. Supervised learning: The computer is presented with example inputs and their desired
outputs, given by a "teacher", and the goal is to learn a general rule that maps inputs to outputs.
As special cases, the input signal can be only partially available, or restricted to special feedback:

2. Semi-supervised learning: the computer is given only an incomplete training signal: a

training set with some (often many) of the target outputs missing.

3. Active learning: the computer can only obtain training labels for a limited set of instances
(based on a budget), and also has to optimize its choice of objects to acquire labels for. When
used interactively, these can be presented to the user for labeling.

4. Reinforcement learning: training data (in form of rewards and punishments) is given only as
feedback to the program's actions in a dynamic environment, such as driving a vehicle or playing
a game against an opponent.

Page 3
5. Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own to
find structure in its input. Unsupervised learning can be a goal in itself (discovering hidden
patterns in data) or a means towards an end (feature learning).

Machine Learning Applications

In classification, inputs are divided into two or more classes, and the learner must produce a
model that assigns unseen inputs to one or more (multi-label classification) of these classes. This
is typically tackled in a supervised manner. Spam filtering is an example of classification, where
the inputs are email (or other) messages and the classes are "spam" and "not spam".

In regression, also a supervised problem, the outputs are continuous rather than discrete. In
clustering, a set of inputs is to be divided into groups. Unlike in classification, the groups are not
known beforehand, making this typically an unsupervised task. Density estimation finds the
distribution of inputs in some space.

Dimensionality reduction simplifies inputs by mapping them into a lower dimensional space.
Topic modeling is a related problem, where a program is given a list of human language
documents and is tasked with finding out which documents cover similar topics.

Machine learning Approaches

1. Decision tree learning

Decision tree learning uses a decision tree as a predictive model, which maps observations about
an item to conclusions about the item's target value.

Page 4
2. Association rule learning
Association rule learning is a method for discovering interesting relations between variables in
large databases.

3. Artificial neural networks

An artificial neural network (ANN) learning algorithm, usually called "neural network" (NN), is
a learning algorithm that is vaguely inspired by biological neural networks. Computations are
structured in terms of an interconnected group of artificial neurons, processing information using
a connectionist approach to computation. Modern neural networks are non-linear statistical data
modeling tools. They are usually used to model complex relationships between inputs and
outputs, to find patterns in data, or to capture the statistical structure in an unknown joint
probability distribution between observed variables.

4. Deep learning
Falling hardware prices and the development of GPUs for personal use in the last few years have
contributed to the development of the concept of deep learning which consists of multiple hidden
layers in an artificial neural network. This approach tries to model the way the human brain
processes light and sound into vision and hearing. Some successful applications of deep learning
are computer vision and speech Recognition.

5. Inductive logic programming

Inductive logic programming (ILP) is an approach to rule learning using logic Programming as a
uniform representation for input examples, background knowledge, and hypotheses. Given an
encoding of the known background knowledge and a set of examples represented as a logical
database of facts, an ILP system will derive a hypothesized logic program that entails all positive
and no negative examples. Inductive programming is a related field that considers any kind of
programming languages for representing hypotheses (and not only logic programming), such as
functional programs.
6. Support vector machines
Support vector machines (SVMs) are a set of related supervised learning methods used for
classification and regression. Given a set of training examples, each marked as belonging to one

Page 5
of two categories, an SVM training algorithm builds a model that predicts whether a new
example falls into one category or the other.

7. Clustering
Cluster analysis is the assignment of a set of observations into subsets (called clusters) so that
observations within the same cluster are similar according to some pre designated criterion or
criteria, while observations drawn from different clusters are dissimilar. Different clustering
techniques make different assumptions on the structure of the data, often defined by some
similarity metric and evaluated for example by internal compactness (similarity between
members of the same cluster) and separation between different clusters. Other methods are based
on estimated density and graph connectivity. Clustering is a method of unsupervised learning,
and a common technique for statistical data analysis.

8. Bayesian networks
A Bayesian network, belief network or directed acyclic graphical model is a probabilistic
graphical model that represents a set of random variables and their conditional independencies
via a directed acyclic graph (DAG). For example, a Bayesian network could represent the
probabilistic relationships between diseases and symptoms. Given symptoms, the network can be
used to compute the probabilities of the presence of various diseases. Efficient algorithms exist
that perform inference and learning.

9. Reinforcement learning
Reinforcement learning is concerned with how an agent ought to take actions in an environment
so as to maximize some notion of long-term reward. Reinforcement learning algorithms attempt
to find a policy that maps states of the world to the actions the agent ought to take in those states.
Reinforcement learning differs from the supervised learning problem in that correct input/output
pairs are never presented, nor sub-optimal actions explicitly corrected.
10. Similarity and metric learning
In this problem, the learning machine is given pairs of examples that are considered similar and
pairs of less similar objects. It then needs to learn a similarity function (or a distance metric
function) that can predict if new objects are similar. It is sometimes used in Recommendation
systems.

Page 6
11. Genetic algorithms
A genetic algorithm (GA) is a search heuristic that mimics the process of natural selection, and
uses methods such as mutation and crossover to generate new genotype in the hope of finding
good solutions to a given problem. In machine learning, genetic algorithms found some uses in
the 1980s and 1990s. Conversely, machine learning techniques have been used to improve the
performance of genetic and evolutionary algorithms.

12. Rule-based machine learning

Rule-based machine learning is a general term for any machine learning method that identifies,
learns, or evolves "rules" to store, manipulate or apply, knowledge. The defining characteristic of
a rule- based machine learner is the identification and utilization of a set of relational rules that
collectively represent the knowledge captured by the system. This is in contrast to other machine
learners that commonly identify a singular model that can be universally applied to any instance
in order to make a prediction. Rule-based machine learning approaches include learning
classifier systems, association rule learning, and artificial immune systems.

13. Feature selection approach

Feature selection is the process of selecting an optimal subset of relevant features for use in
model construction. It is assumed the data contains some features that are either redundant or
irrelevant, and can thus be removed to reduce calculation cost without incurring much loss of
information. Common optimality criteria include accuracy, similarity and information measures.

Page 7
 [Link] :1 IMPLEMENTATION OF UNINFORMED SEARCH ALGORITHMS
DATE:

1.1 AIM:
To Implement Uninformed search algorithms ( BFS and DFS )

1.2 APPARATUS REQUIRED

Python 3.0 / Colab

1.3 ALGORITHM

BFS Algorithm
Breadth-First Search (BFS) is an algorithm used for traversing graphs or trees.
Traversing means visiting each node of the graph. Breadth-First Search is a recursive
algorithm to search all the vertices of a graph or a tree. BFS in python can be
implemented by using data structures like a dictionary and lists. Breadth-First Search in
tree and graph is almost the same. The only difference is that the graph may contain
cycles, so we may traverse to the same node again.

 Step 1: Enqueue the starting node. The first step is to enqueue the starting node into a
queue data structure. ...
 Step 2: Dequeue a node and mark it as visited. ...
 Step 3: Enqueue all adjacent nodes of the dequeued node that are not yet visited. ...
 Step 4: Repeat steps 2-3 until the queue is empty.

DFS Algorithm
The recursive method of the Depth-First Search algorithm is implemented using stack.
A standard Depth-First Search implementation puts every vertex of the graph into one

Page 8
 in all 2 categories: 1) Visited 2) Not Visited. The only purpose of this algorithm is to
visit all the vertex of the graph avoiding cycles.
 Step:1 : We will start by putting any one of the graph's vertex on top of the stack.
 Step:2 : After that take the top item of the stack and add it to the visited list of the
vertex.
 Step:3 : Next, create a list of that adjacent node of the vertex. Add the ones which
aren't in the visited list of vertexes to the top of the stack.
 Step:4 : Lastly, keep repeating steps 2 and 3 until the stack is empty.
1.4 PROGRAM & OUTPUT

[Link] of Uninformed search algorithms (BFS)

# Using a Python dictionary to act as an adjacency list

graph = {
'5' : ['3','7'],
'3' : ['2', '4'],
'7' : ['8'],
'2' : [],
'4' : ['8'],
'8' : []
}
visited = [] # List for visited nodes.
queue = [] #Initialize a queue
def bfs(visited, graph, node): #function for BFS
[Link](node)
[Link](node)

while queue: # Creating loop to visit each node

m = [Link](0)
Page 9
 print (m, end = " ")

for neighbour in graph[m]:

if neighbour not in visited:
[Link](neighbour)
[Link](neighbour)

# Driver Code
print("Following is the Breadth-First Search")
bfs(visited, graph, '5') # function calling

OUTPUT

Following is the Breadth-First Search 5 3 7 2 4 8

[Link] of Uninformed search algorithms ( DFS)

# Using a Python dictionary to act as an adjacency list

graph = {
'5' : ['3','7'],
'3' : ['2', '4'],
'7' : ['8'],
'2' : [],
'4' : ['8'],
'8' : []
}

visited = set() # Set to keep track of visited nodes of graph.

Page 10
def dfs(visited, graph, node): #function for dfs
if node not in visited:
print (node)
[Link](node)
for neighbour in graph[node]:
dfs(visited, graph, neighbour)

# Driver Code
print("Following is the Depth-First Search")
dfs(visited, graph, '5')

OUTPUT

Following is the Depth-First Search 5 3 2 4 8 7

1.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the
hard copy

1.6 RESULT

Page 11
[Link] : 2 IMPLEMENTATION OF INFORMED SEARCH ALGORITHMS
DATE: (A* & AO*)
2.1 AIM:
To Implement Informed search algorithms ( BFS and DFS )

2.2 APPARATUS REQUIRED

Python 3.0 / Colab

2.3 ALGORITHM
A* Search Algorithm:
A* Search Algorithm is a Path Finding Algorithm. It is similar to Breadth First Search(BFS). It
will search shortest path using heuristic value assigned to node and actual cost from
Source_node to Dest_node

Real-life Examples
 Maps
 Games

Formula for A* Algorithm

h(n) = heuristic_value
g(n) = actual_cost
f(n) = actual_cost + heursitic_value
f(n) = g(n) + h(n)

AO* Search Algorithm:

AO* Search Algorithm is a Path Finding Algorithm and it is similar to A* star, other than AND is
used between two nodes along with OR. After getting shortest path it will return back to root node
and it will update it's heuristic value. It is similar to Depth First Search(DFS). It will search shortest
path using heuristic value assigned to node and actual cost from Source_node to Dest_node

Page 12
 Difference between A * and AO * algorithm
An A* algorithm represents an OR graph algorithm that is used to find a single solution (either this
or that). An AO* algorithm represents an AND-OR graph algorithm that is used to find more than
one solution by ANDing more than one branch.

Real-life Examples
 Maps
 Games
Formula for AO* Algorithm
h(n) = heuristic_value
g(n) = actual_cost
f(n) = actual_cost + heursitic_value
f(n) = g(n) + h(n)

2.4 PROGRAM & OUTPUT

[Link] A* Search Algorithm

Program :

def aStarAlgo(start_node, stop_node):

open_set = set(start_node) # {A}, len{open_set}=1
closed_set = set()
g = {} # store the distance from starting node
parents = {}
g[start_node] = 0
parents[start_node] = start_node # parents['A']='A"
while len(open_set) > 0 :
n = None
for v in open_set: # v='B'/'F'
if n == None or g[v] + heuristic(v) < g[n] + heuristic(n):
n = v # n='A'
if n == stop_node or Graph_nodes[n] == None:
pass
else:
Page 13
 for (m, weight) in get_neighbors(n):
# nodes 'm' not in first and last set are added to first
# n is set its parent
if m not in open_set and m not in closed_set:
open_set.add(m) # m=B weight=6 {'F','B','A'} len{open_set}=2
parents[m] = n # parents={'A':A,'B':A} len{parent}=2
g[m] = g[n] + weight # g={'A':0,'B':6, 'F':3} len{g}=2

#for each node m,compare its distance from start i.e g(m) to the
#from start through n node
else:
if g[m] > g[n] + weight:
#update g(m)
g[m] = g[n] + weight
#change parent of m to n
parents[m] = n

#if m in closed set,remove and add to open

if m in closed_set:
closed_set.remove(m)
open_set.add(m)
if n == None:
print('Path does not exist!')
return None
# if the current node is the stop_node
# then we begin reconstructin the path from it to the start_node
if n == stop_node:
path = []

while parents[n] != n:
[Link](n)
n = parents[n]

Page 14
 [Link](start_node)
[Link]()
print('Path found: {}'.format(path))
return path

# remove n from the open_list, and add it to closed_list

# because all of his neighbors were inspected
open_set.remove(n)# {'F','B'} len=2
closed_set.add(n) #{A} len=1

print('Path does not exist!')

return None

#define fuction to return neighbor and its distance

#from the passed node
def get_neighbors(v):
if v in Graph_nodes:
return Graph_nodes[v]
else:
return None
#for simplicity we ll consider heuristic distances given
#and this function returns heuristic distance for all nodes

def heuristic(n):
H_dist = {
'A': 10,
'B': 8,
'C': 5,
'D': 7,
'E': 3,
'F': 6,
'G': 5,

Page 15
 'H': 3,
'I': 1,
'J': 0
}

return H_dist[n]

#Describe your graph here

Graph_nodes = {

'A': [('B', 6), ('F', 3)],

'B': [('C', 3), ('D', 2)],
'C': [('D', 1), ('E', 5)],
'D': [('C', 1), ('E', 8)],
'E': [('I', 5), ('J', 5)],
'F': [('G', 1),('H', 7)] ,
'G': [('I', 3)],
'H': [('I', 2)],
'I': [('E', 5), ('J', 3)],

}
aStarAlgo('A', 'J')

Output

Path found: ['A', 'F', 'G', 'I', 'J']

2.b. Implementation of Informed search algorithms (memory-bounded A*)

Page 16
Program

class Graph:
def __init__(self, graph, heuristicNodeList, startNode): #instantiate graph object with graph
topology, heuristic values, start node

[Link] = graph
self.H=heuristicNodeList
[Link]=startNode
[Link]={}
[Link]={}
[Link]={}

def applyAOStar(self): # starts a recursive AO* algorithm

[Link]([Link], False)

def getNeighbors(self, v): # gets the Neighbors of a given node

return [Link](v,'')

def getStatus(self,v): # return the status of a given node

return [Link](v,0)

def setStatus(self,v, val): # set the status of a given node

[Link][v]=val

def getHeuristicNodeValue(self, n):

return [Link](n,0) # always return the heuristic value of a given node

def setHeuristicNodeValue(self, n, value):

self.H[n]=value # set the revised heuristic value of a given node

def printSolution(self):

Page 17
 print("FOR GRAPH SOLUTION, TRAVERSE THE GRAPH FROM THE
STARTNODE:",[Link])
print("------------------------------------------------------------")
print([Link])
print("------------------------------------------------------------")

def computeMinimumCostChildNodes(self, v): # Computes the Minimum Cost of child

nodes of a given node v
minimumCost=0
costToChildNodeListDict={}
costToChildNodeListDict[minimumCost]=[]
flag=True
for nodeInfoTupleList in [Link](v): # iterate over all the set of child node/s
cost=0
nodeList=[]
for c, weight in nodeInfoTupleList:
cost=cost+[Link](c)+weight
[Link](c)

if flag==True: # initialize Minimum Cost with the cost of first set of child node/s
minimumCost=cost
costToChildNodeListDict[minimumCost]=nodeList # set the Minimum Cost child
node/s
flag=False
else: # checking the Minimum Cost nodes with the current Minimum Cost
if minimumCost>cost:
minimumCost=cost
costToChildNodeListDict[minimumCost]=nodeList # set the Minimum Cost child
node/s

return minimumCost, costToChildNodeListDict[minimumCost] # return Minimum Cost

and Minimum Cost child node/s

Page 18
 def aoStar(self, v, backTracking): # AO* algorithm for a start node and backTracking
status flag

print("HEURISTIC VALUES :", self.H)

print("SOLUTION GRAPH :", [Link])
print("PROCESSING NODE :", v)

print("-----------------------------------------------------------------------------------------")

if [Link](v) >= 0: # if status node v >= 0, compute Minimum Cost nodes of v

minimumCost, childNodeList = [Link](v)
[Link](v, minimumCost)
[Link](v,len(childNodeList))
solved=True # check the Minimum Cost nodes of v are solved

for childNode in childNodeList:

[Link][childNode]=v
if [Link](childNode)!=-1:
solved=solved & False

if solved==True: # if the Minimum Cost nodes of v are solved, set the current node
status as solved(-1)
[Link](v,-1)
[Link][v]=childNodeList # update the solution graph with the solved
nodes which may be a part of solution

if v!=[Link]: # check the current node is the start node for backtracking the current
node value
[Link]([Link][v], True) # backtracking the current node value with
backtracking status set to true

Page 19
 if backTracking==False: # check the current call is not for backtracking
for childNode in childNodeList: # for each Minimum Cost child node
[Link](childNode,0) # set the status of child node to 0(needs exploration)
[Link](childNode, False) # Minimum Cost child node is further explored with
backtracking status as false

h1 = {'A': 1, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J':1, 'T': 3}
graph1 = {
'A': [[('B', 1), ('C', 1)], [('D', 1)]],
'B': [[('G', 1)], [('H', 1)]],
'C': [[('J', 1)]],
'D': [[('E', 1), ('F', 1)]],
'G': [[('I', 1)]]
}
G1= Graph(graph1, h1, 'A')
[Link]()
[Link]()

h2 = {'A': 1, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7} # Heuristic values of Nodes
graph2 = { # Graph of Nodes and Edges
'A': [[('B', 1), ('C', 1)], [('D', 1)]], # Neighbors of Node 'A', B, C & D with repective weights
'B': [[('G', 1)], [('H', 1)]], # Neighbors are included in a list of lists
'D': [[('E', 1), ('F', 1)]] # Each sublist indicate a "OR" node or "AND" nodes
}

G2 = Graph(graph2, h2, 'A') # Instantiate Graph object with graph, heuristic values and start
Node
[Link]() # Run the AO* algorithm
[Link]() # print the solution graph as AO* Algorithm search

Page 20
Output:

HEURISTIC VALUES : {'A': 1, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : B
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : G
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : B
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 7, 'J': 1,
'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : I
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 0, 'J': 1,
'T': 3}
SOLUTION GRAPH : {'I': []}
PROCESSING NODE : G
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 1,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I']}
PROCESSING NODE : B
-----------------------------------------------------------------------------------------

Page 21
HEURISTIC VALUES : {'A': 12, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 1,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 1,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : C
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 1,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 1,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : J
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 0,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G'], 'J': []}
PROCESSING NODE : C
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 1, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J': 0,
'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G'], 'J': [], 'C': ['J']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
FOR GRAPH SOLUTION, TRAVERSE THE GRAPH FROM THE STARTNODE: A
------------------------------------------------------------
{'I': [], 'G': ['I'], 'B': ['G'], 'J': [], 'C': ['J'], 'A': ['B', 'C']}
------------------------------------------------------------
HEURISTIC VALUES : {'A': 1, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : D
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}

Page 22
 PROCESSING NODE : E
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 0, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': []}
PROCESSING NODE : D
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 6, 'E': 0, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': []}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 7, 'B': 6, 'C': 12, 'D': 6, 'E': 0, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': []}
PROCESSING NODE : F
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 7, 'B': 6, 'C': 12, 'D': 6, 'E': 0, 'F': 0, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': [], 'F': []}
PROCESSING NODE : D
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 7, 'B': 6, 'C': 12, 'D': 2, 'E': 0, 'F': 0, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': [], 'F': [], 'D': ['E', 'F']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
FOR GRAPH SOLUTION, TRAVERSE THE GRAPH FROM THE STARTNODE: A
------------------------------------------------------------
{'E': [], 'F': [], 'D': ['E', 'F'], 'A': ['D']}
------------------------------------------------------------

2.5 PROCEDURE
 Open python 3.0 IDLE / Colab
 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the
hard copy

2.6 RESULT

[Link] : 3 IMPLEMENTATION OF NAÏVE BAYES MODELS

DATE:
Page 23
3.1 AIM
To Implement Naïve Bayes Models

3.2 APPARATUS REQUIRED

Python 3.0 / Colab

3.3 ALGORITHM
Conditional probability is defined as the likelihood of an event or outcome occurring, based
on the occurrence of a previous event or outcome. Conditional probability is calculated by
multiplying the probability of the preceding event by the updated probability of the
succeeding, or conditional, event

Bayes’ Rule
Bayes’ Rule. Bayes’ theorem which was given by Thomas Bayes, a British Mathema tician,
in 1763 provides a means for calculating the probability of an event given some information.
Mathematically Bayes’ theorem can be stated as:

Naive Bayes
Bayes’ rule provides us with the formula for the probability of Y given some feature X. In real-
world problems, we hardly find any case where there is only one feature. When the features are
independent, we can extend Bayes’ rule to what is called Naive Bayes which assumes that the
Page 24
features are independent that means changing the value of one feature doesn’t influence the
values of other variables and this is why we call this algorithm “NAIVE”. Naive Bayes can be
used for various things like face recognition, weather prediction, Medical Diagnosis, News
classification, Sentiment Analysis, and a lot more.

When there are multiple X variables, we simplify it by assuming that X’s are independent, so

For n number of X, the formula becomes Naive Bayes:

Which can be expressed as:

This formula can also be understood as:

Gaussian Naïve Bayes :

Gaussian Naïve Bayes is used when we assume all the continuous variables associated with
each feature to be distributed according to Gaussian Distribution. Gaussian Distribution is
also called Normal distribution. The conditional probability changes here since we have

Page 25
different values now. Also, the (PDF) probability density function of a normal distribution is
given by:

We can use this formula to compute the probability of likelihoods if our data is
continuous.

Problem statement:
– Given features X1 ,X2 ,…,Xn
– Predict a label Y
X = (Rainy, Hot,
High, False) y =

No

Or
Consider a random experiment of tossing 2 coins. The sample space here will be:
S = {HH, HT, TH, TT}

 P(H) is the probability of hypothesis H being true. This is known as the prior
probability.
 P(E) is the probability of the evidence(regardless of the hypothesis).
 P(E|H) is the probability of the evidence given that hypothesis is true.
 P(H|E) is the probability of the hypothesis given that the evidence is there.

3.4 PROGRAM & OUTPUT

##import library
import math
Page 26
import random
import pandas as pd
import numpy as np

from [Link] import make_classification

X, y = make_classification(
n_features=6,
n_classes=3,
n_samples=800,
n_informative=2,
random_state=1,
n_clusters_per_class=1,

import [Link] as plt

[Link](X[:, 0], X[:, 1], c=y, marker="*");

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.33, random_state=125
)

from sklearn.naive_bayes import GaussianNB

# Build a Gaussian Classifier

model = GaussianNB()

# Model training
[Link](X_train, y_train)

# Predict Output
predicted = [Link]([X_test[6]])

print("Actual Value:", y_test[6])

print("Predicted Value:", predicted[0])

Page 27
 from [Link] import (
accuracy_score,
confusion_matrix,
ConfusionMatrixDisplay,
f1_score,
)

y_pred = [Link](X_test)
accuray = accuracy_score(y_pred, y_test)
f1 = f1_score(y_pred, y_test, average="weighted")

print("Accuracy:", accuray)
print("F1 Score:", f1)

labels = [0,1,2]
cm = confusion_matrix(y_test, y_pred, labels=labels)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=labels)
[Link]();

OUTPUT

Accuracy: 0.8484848484848485
F1 Score: 0.8491119695890328

Page 28
3.5 PROCEDURE
 Open python 3.0 IDLE / Colab
 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the
hard copy

3.6 RESULT
.

Page 29
 [Link] : 4 IMPLEMENTATION OF BAYESIAN NETWORKS
DATE :

4.1AIM:
To Implement Bayesian Networks

4.2 APPARATUS REQUIRED

Python 3.0 / Colab

4.3 ALGORITHM

Previous notebooks showed how Bayesian networks economically encode a probability

distribution over a set of variables, and how they can be used e.g. to predict variable states, or to
generate new samples from the joint distribution.

This section will be about obtaining a Bayesian network, given a set of sample data. Learning a
Bayesian network can be split into two problems:

Parameter learning: Given a set of data samples and a DAG that captures the dependencies
between the variables, estimate the (conditional) probability distributions of the individual
variables.

Structure learning: Given a set of data samples, estimate a DAG that captures the dependencies
between the variables.

This notebook aims to illustrate how parameter learning and structure learning can be done with
pgmpy. Currently, the library supports:

Parameter learning for discrete nodes:

 Maximum Likelihood Estimation

 Bayesian Estimation

Structure learning for discrete, fully observed networks:

Page 30
  Score-based structure estimation (BIC/BDeu/K2 score; exhaustive search, hill
climb/tabu search)
 Constraint-based structure estimation (PC)
 Hybrid structure estimation (MMHC)

Maximum Likelihood Estimation

 A natural estimate for the CPDs is to simply use the relative frequencies, with which the
variable states have occured. We observed 7 apples among a total of 14 fruits, so we
might guess that about 50% of fruits are apples.

 This approach is Maximum Likelihood Estimation (MLE). According to MLE, we should

fill the CPDs in such a way, that P(data|model) is maximal. This is achieved when using
the relative frequencies.

Bayesian Parameter Estimation

 The Bayesian Parameter Estimator starts with already existing prior CPDs, that express
our beliefs about the variables before the data was observed. Those "priors" are then
updated, using the state counts from the observed data.

 One can think of the priors as consisting in pseudo state counts, that are added to the
actual counts before normalization. Unless one wants to encode specific beliefs about
the distributions of the variables, one commonly chooses uniform priors, i.e. ones that
deem all states equiprobable.

 A very simple prior is the so-called K2 prior, which simply adds 1 to the count of every
single state. A somewhat more sensible choice of prior is BDeu (Bayesian Dirichlet
equivalent uniform prior). For BDeu we need to specify an equivalent sample size N and
then the pseudo-counts are the equivalent of having observed N uniform samples of each
variable (and each parent configuration).

4.4 PROGRAM & OUTPUT

*Parameter Learning *

Suppose we have the following data:

Parameter learning is the task to estimate the values of the conditional probability distributions
(CPDs), for the variables fruit, size, and tasty.

Page 31
data = [Link](data={'fruit': ["banana", "apple", "banana", "apple",
"banana","apple", "banana",
"apple", "apple", "apple", "banana",
"banana", "apple", "banana",],
'tasty': ["yes", "no", "yes", "yes", "yes",
"yes", "yes",
"yes", "yes", "yes", "yes", "no",
"no", "no"],
'size': ["large", "large", "large", "small",
"large", "large", "large",
"small", "large", "large", "large",
"large", "small", "small"]})

Program :
!pip install pgmpy
!pip install pandas
!pip install numpy

import pandas as pd
data = [Link](data={'fruit': ["banana", "apple", "banana", "apple",
"banana","apple", "banana",
"apple", "apple", "apple", "banana",
"banana", "apple", "banana",],
'tasty': ["yes", "no", "yes", "yes", "yes",
"yes", "yes",
"yes", "yes", "yes", "yes", "no", "no",
"no"],
'size': ["large", "large", "large", "small",
"large", "large", "large",
"small", "large", "large", "large",
"large", "small", "small"]})
print(data)

from [Link] import BayesianModel

model = BayesianModel([('fruit', 'tasty'), ('size', 'tasty')]) # fruit ->

tasty <- size

from [Link] import ParameterEstimator

pe = ParameterEstimator(model, data)
print("\n", pe.state_counts('fruit')) # unconditional
print("\n", pe.state_counts('tasty')) # conditional on fruit and size

from [Link] import MaximumLikelihoodEstimator

Page 32
 mle = MaximumLikelihoodEstimator(model, data)
print(mle.estimate_cpd('fruit')) # unconditional
print(mle.estimate_cpd('tasty')) # conditional

# Calibrate all CPDs of `model` using MLE:

[Link](data, estimator=MaximumLikelihoodEstimator)

from [Link] import BayesianEstimator

est = BayesianEstimator(model, data)
print(est.estimate_cpd('tasty', prior_type='BDeu',
equivalent_sample_size=10))

OUTPUT

fruit tasty size

0 banana yes large
1 apple no large
2 banana yes large
3 apple yes small
4 banana yes large
5 apple yes large
6 banana yes large
7 apple yes small
8 apple yes large
9 apple yes large
10 banana yes large
11 banana no large
12 apple no small
13 banana no small

fruit
apple 7
banana 7

fruit apple banana

size large small large small
tasty
no 1.0 1.0 1.0 1.0
yes 3.0 2.0 5.0 0.0

+---------------+-----+
| fruit(apple) | 0.5 |
+---------------+-----+
| fruit(banana) | 0.5 |
+---------------+-----+
+------------+--------------+--------------------+---------------------
+---------------+

Page 33
| fruit | fruit(apple) | fruit(apple) | fruit(banana) |
fruit(banana) |
+------------+--------------+--------------------+---------------------
+---------------+
| size | size(large) | size(small) | size(large) |
size(small) |
+------------+--------------+--------------------+---------------------
+---------------+
| tasty(no) | 0.25 | 0.3333333333333333 | 0.16666666666666666 | 1.0
|
+------------+--------------+--------------------+---------------------
+---------------+
| tasty(yes) | 0.75 | 0.6666666666666666 | 0.8333333333333334 | 0.0
|
+------------+--------------+--------------------+---------------------+--
+------------+---------------------+--------------------+--------------------+---------------------+
| fruit | fruit(apple) | fruit(apple) | fruit(banana) | fruit(banana) |
+------------+---------------------+--------------------+--------------------+---------------------+
| size | size(large) | size(small) | size(large) | size(small) |
+------------+---------------------+--------------------+--------------------+---------------------+
| tasty(no) | 0.34615384615384615 | 0.4090909090909091 | 0.2647058823529412 |
0.6428571428571429 |
+------------+---------------------+--------------------+--------------------+---------------------+
| tasty(yes) | 0.6538461538461539 | 0.5909090909090909 | 0.7352941176470589 |
0.35714285714285715 |
+------------+---------------------+--------------------+--------------------+---------------------+
4.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program & Run the program
 Observe the output and take the hard copy
 Write the program for various example / application ,
 Observe the output and take the hard copy

4.6 RESULT

Page 34
 [Link] : 5 BUILD REGRESSION MODELS
DATE:

5.1AIM:
To Build Regression Models

5.2 PPARATUS REQUIRED

Python 3.0 / Colab

5.3 ALGORITHM
Regression analysis is a commonly used statistical technique for predicting the relationship between a
dependent variable and one or more independent variables. In the field of machine learning, regression
algorithms are used to make predictions about continuous variables, such as housing prices, student scores,
or medical outcomes. Python, being one of the most widely used programming languages in data science
and machine learning, has a variety of powerful libraries for implementing regression algorithms.

[Link] linear regression is a statistical method used to model the relationship between a
dependent variable and two or more independent variables. It is an extension of simple linear
regression, where only one independent variable is used to predict the dependent variable.

[Link] regression is a form of regression analysis in which the relationship between the
independent variable x and the dependent variable y is modeled as an nth degree polynomial. It
allows for more flexibility to model non-linear relationships between variables, unlike linear
regression which assumes that the relationship is linear. Below you can see the generalized
equation for polynomial regression, where y is the dependent variable, and the x values would be
the independent variables. Notice how we could expand this by choosing higher orders of

Page 35
polynomials (to some order k) and we could have also included interaction terms.

3. Ridge Regression is a variation of linear regression that addresses some of the issues of linear
regression. Linear regression can be prone to overfitting when the number of independent
variables is large, this is because the coefficients of the independent variables can become very
large leading to a complex model that fits the noise of the data. Ridge Regression solves this issue
by adding a term to the linear regression equation called L2 regularization term, also known as
Ridge Penalty, which is the sum of the squares of the coefficients multiplied by a regularization
parameter lambda.

4. LASSO (Least Absolute Shrinkage And Selection Operator) is another variation of linear
regression that addresses some of the issues of linear regression. It is used to solve the problem of
overfitting when the number of independent variables is large. Lasso Regression adds a term to
the linear regression equation called L1 regularization term, also known as Lasso Penalty, which
is the sum of the absolute values of the coefficients multiplied by a regularization parameter
lambda.

5. Elastic Net Regression is a hybrid of Ridge Regression and Lasso Regression that combines
the strengths of both. It addresses the problem of overfitting when the number of independent
variables is large by adding both L1 and L2 regularization terms to the linear regression equation .

6. Decision tree based regression is a method that uses decision trees to model the relationship
between a dependent variable and one or more independent variables. Decision Trees are widely
used machine learning algorithms that can be used for both classification and regression problems
in python. A decision tree is a tree-like structure where each internal node represents a test on an
attribute, each branch represents an outcome of the test, and each leaf node represents a predicted
Page 36
 value or class
[Link] Vector Regression (SVR) is a type of Support Vector Machine (SVM) algorithm,
which is a supervised learning algorithm that can be used for regression problems. SVR is a linear
model that aims to find the hyperplane that maximally separates the data points into two classes,
while at the same time minimizing the classification error. In SVR, the goal is to find the
hyperplane that maximally separates the data points from the prediction error, while at the same
time minimizing the margin of deviation between the predicted value and the true value of the
dependent variable. The optimization problem of SVR can be formulated as:

5.4 PROGRAM & OUTPUT

1. linear regression
# linear regression
import numpy as np
from sklearn.linear_model import LinearRegression
X = [Link]([[1, 1], [1, 2], [2, 2], [2, 3]])
# y = 1 * x_0 + 2 * x_1 + 3
y = [Link](X, [Link]([1, 2])) + 3
reg = LinearRegression().fit(X, y)
[Link](X, y)

reg.coef_

reg.intercept_

[Link]([Link]([[3, 5]]))

OUTPUT:
array([16.])

2. Polynomial regression
# polynomial
import numpy as np
from [Link] import PolynomialFeatures
X = [Link](6).reshape(3, 2)
X

poly = PolynomialFeatures(2)

Page 37
poly.fit_transform(X)

poly = PolynomialFeatures(interaction_only=True)
poly.fit_transform(X)

OUTPUT:
array([[ 1., 0., 1., 0.],
[ 1., 2., 3., 6.],
[ 1., 4., 5., 20.]])

3. Ridge regression
from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 10, 5
rng = [Link](0)
y = [Link](n_samples)
X = [Link](n_samples, n_features)
clf = Ridge(alpha=1.0)
[Link](X, y)

OUTPUT:
Ridge
Ridge()

4. Lasso regression
#lasso
from sklearn import linear_model
clf = linear_model.Lasso(alpha=0.1)
[Link]([[0,0], [1, 1], [2, 2]], [0, 1, 2])

print(clf.coef_)

print(clf.intercept_)

OUTPUT:
[0.85 0. ]
0.15000000000000002

5. Elastic Net regression

#Elastic Net
from sklearn.linear_model import ElasticNet

Page 38
from [Link] import make_regression

X, y = make_regression(n_features=2, random_state=0)
regr = ElasticNet(random_state=0)
[Link](X, y)

print(regr.coef_)

print(regr.intercept_)

print([Link]([[0, 0]]))

OUTPUT:

[18.83816048 64.55968825]
1.4512607561653996
[1.45126076]

6. Decision tree regression

. #Decision Tree
from [Link] import load_diabetes
from sklearn.model_selection import cross_val_score
from [Link] import DecisionTreeRegressor
X, y = load_diabetes(return_X_y=True)
regressor = DecisionTreeRegressor(random_state=0)
cross_val_score(regressor, X, y, cv=10)

OUTPUT:
array([-0.39292219, -0.46749346, 0.02768473, 0.06441362, -0.50323135,
0.16437202, 0.11242982, -0.73798979, -0.30953155, -0.00137327])

[Link] Vector regression

#SVR
from [Link] import SVR
from [Link] import make_pipeline
from [Link] import StandardScaler
import numpy as np
n_samples, n_features = 10, 5
rng = [Link](0)
y = [Link](n_samples)
X = [Link](n_samples, n_features)
regr = make_pipeline(StandardScaler(), SVR(C=1.0, epsilon=0.2))
[Link](X, y)
Page 39
OUTPUT:

Application _ LR

from sklearn.model_selection import train_test_split

import [Link] as plt
import numpy as np
from sklearn import datasets, linear_model, metrics
import math
import random
import pandas as pd

data_url ="[Link]
raw_df = pd.read_csv(data_url, sep="\s+",
skiprows=22, header=None)

X = [Link]([raw_df.values[::2, :],
raw_df.values[1::2, :2]])
y = raw_df.values[1::2, 2]

X_train, X_test,\
y_train, y_test = train_test_split(X, y,
test_size=0.4,
random_state=1)

reg = linear_model.LinearRegression()
[Link](X_train, y_train)

regression coefficients
print('Coefficients: ', reg.coef_)

# variance score: 1 means perfect prediction

Page 40
print('Variance score: {}'.format([Link](X_test, y_test)))

0.0# plot for residual error

# setting plot style

[Link]('fivethirtyeight')

# plotting residual errors in training data

[Link]([Link](X_train),
[Link](X_train) - y_train,
color="green", s=10,
label='Train data')

# plotting residual errors in test data

[Link]([Link](X_test),
[Link](X_test) - y_test,
color="blue", s=10,
label='Test data')

# plotting line for zero residual error

[Link](y=0, xmin=0, xmax=50, linewidth=2)

# plotting legend
[Link](loc='upper right')

# plot title
[Link]("Residual errors")

# method call for showing the plot

[Link]()

OUTPUT:
Coefficients: [-8.95714048e-02 6.73132853e-02 5.04649248e-
02 2.18579583e+00
-1.72053975e+01 3.63606995e+00 2.05579939e-03 -
1.36602886e+00
2.89576718e-01 -1.22700072e-02 -8.34881849e-01 9.40360790e-
03
-5.04008320e-01]
Variance score: 0.7209056672661748

Page 41
5.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application ,
 Observe the output and take the hard copy

5.6 RESULT

[Link] : 6 BUILD DECISION TREES AND RANDOM FORESTS

Page 42
DATE:
6.1AIM:
To build Decision Trees and Random Forests

6.2 APPARATUS REQUIRED

Python 3.0 / Colab

6.3 ALGORITHM
A decision tree is a supervised machine-learning algorithm that can be used for
both classification and regression problems. Algorithm builds its model in the
structure of a tree along with decision nodes and leaf nodes. A decision tree is
simply a series of sequential decisions made to reach a specific result.
The Palmer Penguins dataset
This Colab uses the Palmer Penguins dataset, which contains size measurements for
three penguin species:
 Chinstrap
 Gentoo
 Adelie
 This is a classification problem—the goal is to predict the species of penguin
based on data in the Palmer's Penguins dataset. Let’s meet the penguins.

Page 43
 Application Ex: Bank loan approval

[Link]

Random Forest is a tree-based machine learning algorithm that leverages the

power of multiple decision trees for making decisions. As the name suggests, it
is a “forest” of trees!
But why do we call it a “random” forest? That’s because it is a forest
of randomly created decision trees. Each node in the decision tree works on a
random subset of features to calculate the output. The random forest then
combines the output of individual decision trees to generate the final output.
Bootstrapping is the process of randomly selecting items from the
training dataset. This is a haphazard technique. It assembles randomized
decisions based on several decisions and makes the final decision based on the
majority voting.

6.4 PROGRAM & OUTPUT

[Link]
!pip install tensorflow_decision_forests

import numpy as np
import pandas as pd
import tensorflow_decision_forests as tfdf

Page 44
path =
"[Link]
mer_penguins/[Link]"
pandas_dataset = pd.read_csv(path)

# Display the first 3 examples.

pandas_dataset.head(3)

label = "species"

classes = list(pandas_dataset[label].unique())
print(f"Label classes: {classes}")
# >> Label classes: ['Adelie', 'Gentoo', 'Chinstrap']

pandas_dataset[label] = pandas_dataset[label].map([Link])

[Link](1)
# Use the ~10% of the examples as the testing set
# and the remaining ~90% of the examples as the training set.
test_indices = [Link](len(pandas_dataset)) < 0.1
pandas_train_dataset = pandas_dataset[~test_indices]
pandas_test_dataset = pandas_dataset[test_indices]

print("Training examples: ", len(pandas_train_dataset))

# >> Training examples: 309

print("Testing examples: ", len(pandas_test_dataset))

# >> Testing examples: 35

tf_train_dataset =
[Link].pd_dataframe_to_tf_dataset(pandas_train_dataset, label=label)
model = [Link]()
[Link](tf_train_dataset)

tfdf.model_plotter.plot_model_in_colab

tfdf.model_plotter.plot_model_in_colab(model, max_depth=10)

bill_depth_mm = 16.35;
if bill_depth_mm > 16.35
classes = list(pandas_dataset[label].unique())
print(f"Label classes: {classes}")
else

Page 45
 bill_depth_mm < 16.35
classes = list(pandas_dataset[label].unique())
print(f"Label classes: {classes}")
end
end

[Link]("accuracy")
print("Train evaluation: ", [Link](tf_train_dataset,
return_dict=True))
# >> Train evaluation: {'loss': 0.0, 'accuracy': 0.96116}

tf_test_dataset =
[Link].pd_dataframe_to_tf_dataset(pandas_test_dataset, label=label)
print("Test evaluation: ", [Link](tf_test_dataset,
return_dict=True))
# >> Test evaluation: {'loss': 0.0, 'accuracy': 0.97142}

OUTPUT:

Label classes: ['Adelie', 'Gentoo', 'Chinstrap']

Training examples: 309

Testing examples: 35

Use /tmp/tmpwlyrflr1 as temporary training directory

Reading training dataset...
Training dataset read in 0:00:08.991866. Found 309 examples.
Training model...
Model trained in 0:00:00.036362
Compiling model...
Model compiled.
<[Link] at 0x7cf8f5b754e0>

tensorflow_decision_forests.component.model_plotter.model_plott
er.plot_model_in_colab
def plot_model_in_colab(model: InferenceCoreModel, **kwargs)

Page 46
 /usr/local/lib/python3.10/dist-packages/
tensorflow_decision_forests/component/model_plotter/
model_plotter.pyPlots a model structure in colab.

See "plot_model" for the available options.

Args:
model: The model to plot.
**kwargs: Arguments passed to "plot_model".

Returns:
A Colab HTML element showing the model.

1/1 [==============================] - 0s 419ms/step - loss:

0.0000e+00 - accuracy: 0.9709
Train evaluation: {'loss': 0.0, 'accuracy':
0.9708737730979919}
1/1 [==============================] - 0s 100ms/step - loss:
0.0000e+00 - accuracy: 0.9714
Test evaluation: {'loss': 0.0, 'accuracy': 0.9714285731315613}

6.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application ,
 Observe the output and take the hard copy

6.6 RESULT

Page 47
[Link] : 7 BUILD SVM MODELS
DATE:
7.1AIM:
To build SVM Models

7.2 APPARATUS REQUIRED

Python 3.0 / Colab

7.3 ALGORITHM
The main objective is to segregate the given dataset in the best possible way. The distance
between the either nearest points is known as the margin. The objective is to select a hyperplane
with the maximum possible margin between support vectors in the given dataset. SVM searches
for the maximum marginal hyperplane in the following steps:

1. Generate hyperplanes which segregates the classes in the best way. Left-hand side figure
showing three hyperplanes black, blue and orange. Here, the blue and orange have higher
classification error, but the black is separating the two classes correctly.

2. Select the right hyperplane with the maximum segregation from the either nearest data
points as shown in the right-hand side figure.

7.4 PROGRAM & OUTPUT

#import library

Page 48
import math
import random
import pandas as pd
import numpy as np
import [Link]
import requests

#Import scikit-learn dataset library

from sklearn import datasets

#Load dataset
cancer = datasets.load_breast_cancer()

#kernel implimentation
def K(x, xi):
# Choose one of the following implementations:

# Linear kernel
# return sum(x * xi)

# Polynomial kernel (degree 1)

# return 1 + sum(x * xi)

# Gaussian kernel
gamma = 1 # Set the kernel parameter
return exp(-gamma * sum((x_i - xi_i)**2 for x_i, xi_i in zip(x, xi)))

# print the names of the 30 features

print("Features: ", cancer.feature_names)

# print the label type of cancer('malignant' 'benign')

print("Labels: ", cancer.target_names)

# print data(feature)shape
[Link]

# print the cancer data features (top 5 records)

print([Link][0:5])
# print the cancer labels (0:malignant, 1:benign)
print([Link])

# Import train_test_split function

from sklearn.model_selection import train_test_split

# Split dataset into training set and test set

Page 49
X_train, X_test, y_train, y_test =
train_test_split([Link], [Link],
test_size=0.3,random_state=109) # 70% training and 30% test
#Import svm model
from sklearn import svm

#Create a svm Classifier

clf = [Link](kernel='linear') # Linear Kernel

#Train the model using the training sets

[Link](X_train, y_train)

#Predict the response for test dataset

y_pred = [Link](X_test)

#Import scikit-learn metrics module for accuracy calculation

from sklearn import metrics

# Model Accuracy: how often is the classifier correct?

print("Accuracy:",metrics.accuracy_score(y_test, y_pred))

# Model Precision: what percentage of positive tuples are labeled as such?

print("Precision:",metrics.precision_score(y_test, y_pred))

# Model Recall: what percentage of positive tuples are labelled as such?

print("Recall:",metrics.recall_score(y_test, y_pred))

OUTPUT:
Features: ['mean radius' 'mean texture' 'mean perimeter' 'mean area'
'mean smoothness' 'mean compactness' 'mean concavity'
'mean concave points' 'mean symmetry' 'mean fractal dimension'
'radius error' 'texture error' 'perimeter error' 'area error'
'smoothness error' 'compactness error' 'concavity error'
'concave points error' 'symmetry error' 'fractal dimension error'
'worst radius' 'worst texture' 'worst perimeter' 'worst area'
'worst smoothness' 'worst compactness' 'worst concavity'
'worst concave points' 'worst symmetry' 'worst fractal dimension']
Labels: ['malignant' 'benign']

[[1.799e+01 1.038e+01 1.228e+02 1.001e+03 1.184e-01 2.776e-01 3.001e-01

1.471e-01 2.419e-01 7.871e-02 1.095e+00 9.053e-01 8.589e+00 1.534e+02
6.399e-03 4.904e-02 5.373e-02 1.587e-02 3.003e-02 6.193e-03 2.538e+01
1.733e+01 1.846e+02 2.019e+03 1.622e-01 6.656e-01 7.119e-01 2.654e-01
4.601e-01 1.189e-01]
[2.057e+01 1.777e+01 1.329e+02 1.326e+03 8.474e-02 7.864e-02 8.690e-02
7.017e-02 1.812e-01 5.667e-02 5.435e-01 7.339e-01 3.398e+00 7.408e+01
5.225e-03 1.308e-02 1.860e-02 1.340e-02 1.389e-02 3.532e-03 2.499e+01
2.341e+01 1.588e+02 1.956e+03 1.238e-01 1.866e-01 2.416e-01 1.860e-01
2.750e-01 8.902e-02]

Page 50
 [1.969e+01 2.125e+01 1.300e+02 1.203e+03 1.096e-01 1.599e-01 1.974e-01
1.279e-01 2.069e-01 5.999e-02 7.456e-01 7.869e-01 4.585e+00 9.403e+01
6.150e-03 4.006e-02 3.832e-02 2.058e-02 2.250e-02 4.571e-03 2.357e+01
2.553e+01 1.525e+02 1.709e+03 1.444e-01 4.245e-01 4.504e-01 2.430e-01
3.613e-01 8.758e-02]
[1.142e+01 2.038e+01 7.758e+01 3.861e+02 1.425e-01 2.839e-01 2.414e-01
1.052e-01 2.597e-01 9.744e-02 4.956e-01 1.156e+00 3.445e+00 2.723e+01
9.110e-03 7.458e-02 5.661e-02 1.867e-02 5.963e-02 9.208e-03 1.491e+01
2.650e+01 9.887e+01 5.677e+02 2.098e-01 8.663e-01 6.869e-01 2.575e-01
6.638e-01 1.730e-01]
[2.029e+01 1.434e+01 1.351e+02 1.297e+03 1.003e-01 1.328e-01 1.980e-01
1.043e-01 1.809e-01 5.883e-02 7.572e-01 7.813e-01 5.438e+00 9.444e+01
1.149e-02 2.461e-02 5.688e-02 1.885e-02 1.756e-02 5.115e-03 2.254e+01
1.667e+01 1.522e+02 1.575e+03 1.374e-01 2.050e-01 4.000e-01 1.625e-01
2.364e-01 7.678e-02]]

[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0
1 0 1 0 0 1 1 1 0 0 1 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 1 1
1 1 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 0 1 1 1 1 0 1
1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 0
1 0 1 1 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 1
1 0 1 1 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1
1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 0 1 1
1 1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0
0 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1
1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 0 1 1
0 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0
1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 0 0 0 0 0 0 1]
Accuracy: 0.9649122807017544
Precision: 0.9811320754716981
Recall: 0.9629629629629629

7.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program / Run the program
 Observe the output and take the hard copy
 Write the program for various example / application ,
 Observe the output and take the hard copy

7.6 RESULT

Page 51
 [Link] : 8 IMPLEMENT ENSEMBLING TECHNIQUES
DATE:

8.1AIM:
To Implement Ensembling Techniques

8.2 APPARATUS REQUIRED

Python 3.0 / Colab

8.3 ALGORITHM
The steps of the EM algorithm are as follows:
1. We first consider a set of starting parameters given a set of incomplete (observed) data and
we assume that observed data come from a specific model
2. We then use the model to “estimate” the missing data . In other words after formulating some
parameters from observed data to build a model, we use this model to guess the missing
value/data. This step is called the expectation step.
3. Now we use the “complete” data that we have estimated to update parameters where using
the missing data and observed data, we find the most likely modified parameters to build the
modified model. This is called the maximization step .
4. We repeat steps 2 & 3 until convergence that is there is no change in the parameters of the
model and the estimated model fits the observed data.

EM – Coin Toss Example

Page 52
 The EM strategy can be explained with a coin toss example. This is the example we will be
using in subsequent iterations to explain the complete flow of the EM algorithm. In this
example we assume that we are tossing a number of coins sequentially to obtain a sequence
of Head or Tails. The context of the coin toss example is given in Table 33.1. Here the
problem is defined as X, the sequence of Heads and Tails that is observed, Y as the identifier
of the coin that is tossed in the sequence, which is hidden and finally θ which is the
parameter vector which is associated with the probabilities of the observed and hidden data.
Here if we assume three coins are tossed λ is the probability of coin 0 showing H (so 1 − λ is
the probability of it showing T), p1 is the probability of coin 1 showing H, and p2 is the
probability of coin 2 showing H.
Parameters of EM

Problem Coin toss

X(observed) Head-tail sequences
Y (hidden) Coin id sequences
Θ p1, p2, λ

E-step and M-step

8.4 PROGRAM & OUTPUT

import numpy as np

Page 53
def coin_em(rolls, theta_A=None, theta_B=None, maxiter=10):
# Initial Guess
theta_A = theta_A or [Link]()
theta_B = theta_B or [Link]()
thetas = [(theta_A, theta_B)]
# Iterate
for c in range(maxiter):
print("#%d:\t%0.2f %0.2f" % (c, theta_A, theta_B))
heads_A, tails_A, heads_B, tails_B = e_step(rolls, theta_A,
theta_B)
theta_A, theta_B = m_step(heads_A, tails_A, heads_B, tails_B)

[Link]((theta_A,theta_B))
return thetas, (theta_A,theta_B)

def e_step(rolls, theta_A, theta_B):

"""Produce the expected value for heads_A, tails_A, heads_B, tails_B
over the rolls given the coin biases"""

heads_A, tails_A = 0,0

heads_B, tails_B = 0,0
for trial in rolls:
likelihood_A = coin_likelihood(trial, theta_A)
likelihood_B = coin_likelihood(trial, theta_B)
p_A = likelihood_A / (likelihood_A + likelihood_B)
p_B = likelihood_B / (likelihood_A + likelihood_B)
heads_A += p_A * [Link]("H")
tails_A += p_A * [Link]("T")
heads_B += p_B * [Link]("H")
tails_B += p_B * [Link]("T")
return heads_A, tails_A, heads_B, tails_B

def m_step(heads_A, tails_A, heads_B, tails_B):

"""Produce the values for theta that maximize the expected number of
heads/tails"""

# Replace dummy values with your implementation

theta_A = heads_A / (heads_A + tails_A)
theta_B = heads_B / (heads_B + tails_B)
return theta_A, theta_B

def coin_likelihood(roll, bias):

# P(X | Z, theta)
numHeads = [Link]("H")
print(numHeads)
flips = len(roll)
print(flips)

Page 54
 return pow(bias, numHeads) * pow(1-bias, flips-numHeads)
# Call the functions
rolls = [ "HTTTHHTHTH", "HHHHTHHHHH", "HTHHHHHTHH",
"HTHTTTHHTT", "THHHTHHHTH" ]
thetas, _ = coin_em(rolls, 0.6, 0.5, maxiter=10)

type(thetas)
thet=thetas[1]

print(thetas)

rolls_p = "HHHTTHTHTH"
numHeads_p = rolls_p.count('H')
print('No. of Heads', numHeads_p)
flips_p = len(rolls_p)
print(flips_p)

l_a=pow(thet[0], numHeads_p) * pow(1-thet[0], flips_p-numHeads_p)

l_b=pow(thet[1], numHeads_p) * pow(1-thet[1], flips_p-numHeads_p)
print('Lilelihood of A coin : ',l_a)
print('Lilelihood of B coin : ',l_b)
p_a = l_a / (l_a +l_b)
p_b = l_b/ (l_a + l_b)
print('Probability of A coin : ',p_a)
print('Probability of B coin : ',p_b)

if p_a > p_b:

print('The boy tossed Coin A')
else:
print('The boy tossed Coin B')

OUTPUT:
9
10
9
10
8
10
8
10
4
10
4
10
7
10
7
Page 55
10
#8: 0.80 0.52
5
10
5
10
9
10
9
10
8
10
8
10
4
10
4
10
7
10
7
10
#9: 0.80 0.52
5
10
5
10
9
10
9
10
8
10
8
10
4
10
4
10
7
10
7
10

[(0.6, 0.5), (0.7967441494752115, 0.5196586622041124)]

No. of Heads 6
10
Lilelihood of A coin : 0.0004366017976005356
Lilelihood of B coin : 0.0010483562262602218
Probability of A coin : 0.2940162553991999
Probability of B coin : 0.7059837446008002

Page 56
 The boy tossed Coin B
8.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program / Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the
hard copy

8.6 RESULT

Page 57
[Link] : 9 IMPLEMENT CLUSTERING ALGORITHMS
DATE:
9.1AIM:
To Implement Clustering Algorithms

9.2 APPARATUS REQUIRED

Python 3.0 / Colab

9.3 ALGORITHM
Kmeans and EM algorithm
We can explain K means as an EM algorithm. First we initialize the k means (mk) of the Kmeans
algorithm. In the E Step we assign each point to a Cluster and during the M Step given the
Clusters we refine mean mk of each cluster k. This process is repeated until the change in means
is small.
K-means and Mixture of Gaussians
Now we know that in a general K-means which is essentially a classifier and we need to find the
parameter to fit data – that is we need to find the mean – µk as already discussed above.
However when we use mixture of Gaussians which is a probability model where we are defining
a “soft” classifier. Now the parameters that are to be determined to fit to data are the means µ k
and covariance Σk which define the Gaussians distributions and the mixing coefficient πk. Now
given the data set, find the mixing coefficients, means and covariance. If we knew which
component generated each data point, the maximum likelihood solution would involve fitting
each component to the corresponding cluster . However our problem is that the data set is
unlabelled or are hidden

9.4 PROGRAM & OUTPUT

from [Link] import KMeans

import numpy as np

Page 58
# Generate sample data
###[Link](0)
#X = [Link](19, 2)

X = [Link]([[1, 2], [2,4], [10,12], [11,15], [3,2], [12,13]])

print(X)
# Instantiate and fit the K-means model
#kmeans = KMeans(n_clusters=2, random_state=0, n_init='auto')
# [Link](X)
#label = kmeans.fit_predict(X)
#Build clustering model
kmeans = KMeans(n_clusters=2, random_state=0, n_init="auto").fit(X)

#Display the cluster centers

kmeans.cluster_centers_

#Displaythe label of the instances

kmeans.labels_
#Prediction on new customer income
[Link]([[8, 9]])

# Get the cluster labels and cluster centers

label = kmeans.labels_
centers = kmeans.cluster_centers_
# Display the cluster labels and centers
print("Cluster Labels:", label)
print("Cluster Centers:", centers)
#Getting unique labels

u_labels = [Link](label)
import [Link] as plt
#plotting the results:

for i in u_labels:
[Link](X[label == i , 0] , X[label == i , 1] , label = i)
[Link]()
[Link]("K-Means Clustering")
[Link]()

OUTPUT
[[ 1 2]
[ 2 4]
[10 12]
[11 15]
[ 3 2]
[12 13]]

Page 59
 array([[ 2. , 2.66666667],
[11. , 13.33333333]])

array([0, 0, 1, 1, 0, 1], dtype=int32)

array([1], dtype=int32)

Cluster Labels: [0 0 1 1 0 1]
Cluster Centers: [[ 2. 2.66666667]
[11. 13.33333333]]

9.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application ,
 Observe the output and take the hard copy

9.6 RESULT

Page 60
[Link] : 10 IMPLEMENT EM FOR BAYESIAN NETWORKS
DATE:
10.1AIM:
To Implement EM for Bayesian Networks

10.2 APPARATUS REQUIRED

Python 3.0 / Colab

10.3 ALGORITHM

Expectation–maximization (E–M) is a powerful algorithm that comes up in a

variety of contexts within data science. k-means is a particularly simple and easy-
to-understand application of the algorithm, and we will walk through it briefly
here. In short, the expectation–maximization approach here consists of the
following procedure:

1. Guess some cluster centers.

2. Repeat until converged:
E-step: Assign points to the nearest cluster center.
M-step: Set the cluster centers to the mean of their assigned points.

Here the E-step or expectation step is so named because it involves updating our
expectation of which cluster each point belongs to. The M-step or maximization
step is so named because it involves maximizing some fitness function that
defines the locations of the cluster centers—in this case, that maximization is
accomplished by taking a simple mean of the data in each cluster.

10.4 PROGRAM & OUTPUT

[Link]
PythonDataScienceHandbook/blob/master/notebooks/05.11-K-
[Link]#scrollTo=vu4yKpJKstWk

%matplotlib inline
import [Link] as plt
Page 61
[Link]('seaborn-whitegrid')
import numpy as np

from [Link] import make_blobs

X, y_true = make_blobs(n_samples=300, centers=4,
cluster_std=0.60, random_state=0)
[Link](X[:, 0], X[:, 1], s=50);

from [Link] import KMeans

kmeans = KMeans(n_clusters=4)
[Link](X)
y_kmeans = [Link](X)

[Link](X[:, 0], X[:, 1], c=y_kmeans, s=50, cmap='viridis')

centers = kmeans.cluster_centers_
[Link](centers[:, 0], centers[:, 1], c='black', s=200);

from [Link] import pairwise_distances_argmin

def find_clusters(X, n_clusters, rseed=2):

# 1. Randomly choose clusters
rng = [Link](rseed)
i = [Link]([Link][0])[:n_clusters]
centers = X[i]

while True:
# 2a. Assign labels based on closest center
labels = pairwise_distances_argmin(X, centers)

# 2b. Find new centers from means of points

new_centers = [Link]([X[labels == i].mean(0)
for i in range(n_clusters)])

# 2c. Check for convergence

if [Link](centers == new_centers):
break
centers = new_centers

return centers, labels

centers, labels = find_clusters(X, 4)

[Link](X[:, 0], X[:, 1], c=labels,
s=50, cmap='viridis');

Page 62
centers, labels = find_clusters(X, 4, rseed=0)
[Link](X[:, 0], X[:, 1], c=labels,
s=50, cmap='viridis');

labels = KMeans(6, random_state=0).fit_predict(X)

[Link](X[:, 0], X[:, 1], c=labels,
s=50, cmap='viridis');

from [Link] import make_moons

X, y = make_moons(200, noise=.05, random_state=0)

labels = KMeans(2, random_state=0).fit_predict(X)

[Link](X[:, 0], X[:, 1], c=labels,
s=50, cmap='viridis');

from [Link] import SpectralClustering

model = SpectralClustering(n_clusters=2, affinity='nearest_neighbors',
assign_labels='kmeans')
labels = model.fit_predict(X)
[Link](X[:, 0], X[:, 1], c=labels,
s=50, cmap='viridis');

Figure: 1

Figure: 2

Page 63
Figure: 3

Figure: 4
Page 64
Figure: 5

Figure: 6
Page 65
Figure: 7

Page 66
10.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program
 Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the
hard copy

10.6 RESULT

Page 67
[Link] : 11 BUILD NEURAL NETWORK MODELS
DATE:
11.1AIM:
To build Neural Network (BP) Models

11.2 APPARATUS REQUIRED

Python 3.0 / Colab

11.3 ALGORITHM
Neural Networks are computational models that mimic the complex functions of the human
brain. The neural networks consist of interconnected nodes or neurons that process and learn
from data, enabling tasks such as pattern recognition and decision making in machine
learning. The article explores more about neural networks, their working, architecture and
more.

11.4 PROGRAM & OUTPUT

import numpy as np
X = [Link](([2, 9], [1, 5], [3, 6]), dtype=float)

Page 68
y = [Link](([92], [86], [89]), dtype=float)
X = X/[Link](X,axis=0) # maximum of X array longitudinally y = y/100

#Sigmoid Function
def sigmoid (x):
return (1/(1 + [Link](-x)))
#Derivative of Sigmoid Function
def derivatives_sigmoid(x):
return x * (1 - x)

#Variable initialization
epoch=7000 #Setting training iterations
lr=0.1 #Setting learning rate
inputlayer_neurons = 2 #number of features in data set
hiddenlayer_neurons = 3 #number of hidden layers neurons
output_neurons = 1 #number of neurons at output layer
#weight and bias initialization
wh=[Link](size=(inputlayer_neurons,hiddenlayer_neurons))
bh=[Link](size=(1,hiddenlayer_neurons))
wout=[Link](size=(hiddenlayer_neurons,output_neurons))
bout=[Link](size=(1,output_neurons)) # draws a random range
of numbers uniformly of dim x*y
#Forward Propagation
for i in range(epoch):
hinp1=[Link](X,wh)
hinp=hinp1 + bh
hlayer_act = sigmoid(hinp)
outinp1=[Link](hlayer_act,wout)
outinp= outinp1+ bout
output = sigmoid(outinp)
#Backpropagation
EO = y-output
outgrad = derivatives_sigmoid(output)
d_output = EO* outgrad
EH = d_output.dot(wout.T)
hiddengrad = derivatives_sigmoid(hlayer_act)
#how much hidden layer wts contributed to error
d_hiddenlayer = EH * hiddengrad
wout += hlayer_act.[Link](d_output) *lr
# dotproduct of nextlayererror and currentlayerop
bout+= [Link](d_output, axis=0,keepdims=True) *lr
wh += [Link](d_hiddenlayer) *lr
bh += [Link](d_hiddenlayer, axis=0,keepdims=True) *lr

print("Input: \n" + str(X))

print("Actual Output: \n" + str(y))

Page 69
print("Predicted Output: \n" ,output)

OUTPUT:
Input:
[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]
Actual Output:
[[92.]
[86.]
[89.]]
Predicted Output:
[[0.99999894]
[0.99999822]
[0.99999887]]

Model: 2
import numpy as np
# array of any amount of numbers. n = m
X = [Link]([[1, 2, 3],
[3, 4, 1],
[2, 5, 3]])

# multiplication
y = [Link]([[.5, .3, .2]])

# transpose of y
y = y.T

# sigma value
sigm = 2

# find the delta

delt = [Link]((3, 3)) - 1

for j in range(100):
# find matrix 1. 100 layers.
m1 = (y - (1/(1 + [Link](-([Link]((1/(1 + [Link](
-([Link](X, sigm))))), delt))))))*((1/(
1 + [Link](-([Link]((1/(1 + [Link](
Page 70
 -([Link](X, sigm))))), delt)))))*(1-(1/(
1 + [Link](-([Link]((1/(1 + [Link](
-([Link](X, sigm))))), delt)))))))

# find matrix 2
m2 = [Link](delt.T) * ((1/(1 + [Link](-([Link](X, sigm)))))
* (1-(1/(1 + [Link](-([Link](X, sigm)))))))
# find delta
delt = delt + (1/(1 + [Link](-([Link](X, sigm))))).[Link](m1)

# find sigma
sigm = sigm + ([Link](m2))

# print output from the matrix

print(1/(1 + [Link](-([Link](X, sigm)))))

OUTPUT:
[[0.99999319 0.99999374 0.99999351]
[0.99999988 0.99999989 0.99999988]
[1. 1. 1. ]]

11.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program & Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the
hard copy

11.6 RESULT

Page 71
[Link] : 12 BUILD DEEP LEARNING MODELS
DATE:
12.1 AIM:
To build deep Neural Network Models

12.2 APPARATUS REQUIRED

Python 3.0 / Colab

12.3 ALGORITHM
Simple Convolutional Neural Network (CNN) to classify CIFAR images

The CIFAR10 dataset contains 60,000 color images in 10 classes, with 6,000 images in each
class. The dataset is divided into 50,000 training images and 10,000 testing images. The classes
are mutually exclusive and there is no overlap between them.

The 6 lines of code below define the convolutional base using a common pattern: a stack
of Conv2D and MaxPooling2D layers.

As input, a CNN takes tensors of shape (image_height, image_width, color_channels), ignoring

the batch size. If you are new to these dimensions, color_channels refers to (R,G,B). In this
example, you will configure your CNN to process inputs of shape (32, 32, 3), which is the
format of CIFAR images. You can do this by passing the argument input_shape to your first
layer.

12.4 PROGRAM & OUTPUT

import tensorflow as tf
from [Link] import datasets, layers, models
import [Link] as plt
from [Link] import Adam, SGD, RMSprop
(train_images, train_labels), (test_images, test_labels) =
datasets.cifar10.load_data()

# Normalize pixel values to be between 0 and 1

train_images, test_images = train_images / 255.0, test_images / 255.0

class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',

'dog', 'frog', 'horse', 'ship', 'truck']

[Link](figsize=(8,8))
Page 72
for i in range(25):
[Link](5,5,i+1)
[Link]([])
[Link]([])
[Link](False)
[Link](train_images[i])
# The CIFAR labels happen to be arrays,
#which is why we need the extra index
[Link](class_names[train_labels[i][0]])
[Link]()

model = [Link]()
[Link](layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32,
3)))
[Link](layers.MaxPooling2D((2, 2)))
[Link](layers.Conv2D(64, (3, 3), activation='relu'))
[Link](layers.MaxPooling2D((2, 2)))
[Link](layers.Conv2D(64, (3, 3), activation='relu'))

[Link]()

[Link]([Link]())
[Link]([Link](64, activation='relu'))
[Link]([Link](10))

[Link]()

# Adam is the best among the adaptive optimizers in most of the cases
[Link](optimizer='adam',

loss=[Link](from_logits=True),
metrics=['accuracy'])

# An epoch means training the neural network with all the

# training data for one cycle. Here I use 10 epochs
history = [Link](train_images, train_labels, epochs=10,
validation_data=(test_images, test_labels))

[Link]([Link]['accuracy'],label='accuracy')
[Link]([Link]['val_accuracy'],label = 'val_accuracy')
[Link]('Epoch')
[Link]('Accuracy')
[Link]([0.5, 1])
[Link](loc='lower right')

Page 73
 test_loss, test_acc = [Link](test_images,
test_labels,
verbose=2)

print('Test Accuracy is',test_acc)

OUTPUT:

Downloading data from [Link]

170498071/170498071 [==============================] - 4s 0us/step

Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
Page 74
 =================================================================
conv2d (Conv2D) (None, 30, 30, 32) 896

max_pooling2d (MaxPooling2 (None, 15, 15, 32) 0

D)

conv2d_1 (Conv2D) (None, 13, 13, 64) 18496

max_pooling2d_1 (MaxPoolin (None, 6, 6, 64) 0

g2D)

conv2d_2 (Conv2D) (None, 4, 4, 64) 36928

=================================================================
Total params: 56320 (220.00 KB)
Trainable params: 56320 (220.00 KB)
Non-trainable params: 0 (0.00 Byte)

Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 30, 30, 32) 896

max_pooling2d (MaxPooling2 (None, 15, 15, 32) 0

D)

conv2d_1 (Conv2D) (None, 13, 13, 64) 18496

max_pooling2d_1 (MaxPoolin (None, 6, 6, 64) 0

g2D)

conv2d_2 (Conv2D) (None, 4, 4, 64) 36928

flatten (Flatten) (None, 1024) 0

dense (Dense) (None, 64) 65600

dense_1 (Dense) (None, 10) 650

=================================================================
Total params: 122570 (478.79 KB)
Trainable params: 122570 (478.79 KB)
Non-trainable params: 0 (0.00 Byte)

Epoch 1/10
1563/1563 [==============================] - 38s 24ms/step - loss: 1.5158 -
accuracy: 0.4451 - val_loss: 1.1963 - val_accuracy: 0.5729
Epoch 2/10
1563/1563 [==============================] - 37s 24ms/step - loss: 1.1395 -
accuracy: 0.5940 - val_loss: 1.0595 - val_accuracy: 0.6315
Epoch 3/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.9965 -
accuracy: 0.6494 - val_loss: 1.0275 - val_accuracy: 0.6462
Epoch 4/10

Page 75
1563/1563 [==============================] - 36s 23ms/step - loss: 0.8967 -
accuracy: 0.6829 - val_loss: 0.9410 - val_accuracy: 0.6737
Epoch 5/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.8347 -
accuracy: 0.7081 - val_loss: 0.8940 - val_accuracy: 0.6955
Epoch 6/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.7794 -
accuracy: 0.7269 - val_loss: 0.8578 - val_accuracy: 0.7054
Epoch 7/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.7296 -
accuracy: 0.7446 - val_loss: 0.8526 - val_accuracy: 0.7099
Epoch 8/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.6908 -
accuracy: 0.7581 - val_loss: 0.8534 - val_accuracy: 0.7132
Epoch 9/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.6577 -
accuracy: 0.7676 - val_loss: 0.8581 - val_accuracy: 0.7144
Epoch 10/10
1563/1563 [==============================] - 36s 23ms/step - loss: 0.6192 -
accuracy: 0.7823 - val_loss: 0.8458 - val_accuracy: 0.7162

313/313 - 2s - loss: 0.8458 - accuracy: 0.7162 - 2s/epoch - 6ms/step

Test Accuracy is 0.7161999940872192

12.5 PROCEDURE

 Open python 3.0 IDLE / Colab

 Write the program / Run the program
 Observe the output and take the hard copy
 Write the program for various example / application , Observe the output and take the

Page 76
 hard copy

12.6 RESULT

[Link] : 13 DIGITS CLASSIFICATION Page 77

DATE:
13.1 AIM:
To build deep Neural Network for digit classification

13.2 APPARATUS REQUIRED

Python 3.0 / Colab

13.3 PROGRAM / OUTPUT

import numpy as np
from [Link] import Sequential
from [Link] import Dense
from [Link] import to_categorical
from sklearn.model_selection import train_test_split
from [Link] import load_digits

# Load the digits dataset

digits = load_digits()

# Extract the data and target labels

X = [Link]
y = [Link]

# One-hot encode the target labels

y = to_categorical(y)

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

# Build the neural network model

model = Sequential()
[Link](Dense(64, activation='relu', input_shape=(64,)))
[Link](Dense(10, activation='softmax'))

# Compile the model

[Link](loss='categorical_crossentropy', optimizer='adam',
metrics=['accuracy'])

# Train the model

Page 78
[Link](X_train, y_train, epochs=5, batch_size=32)

# Evaluate the model on the testing set

loss, accuracy = [Link](X_test, y_test)

print('Loss:', loss)
print('Accuracy:', accuracy)
pred = [Link](X[0,:].reshape(1, -1))
print(pred)
print(y[0,:])

from [Link] import load_digits

dgts = load_digits()
print([Link])
import [Link] as plt
[Link]()
[Link]([Link][0])
[Link]()

OUTPUT
Epoch 1/5
45/45 [==============================] - 1s 2ms/step - loss: 5.0013 -
accuracy: 0.2289
Epoch 2/5
45/45 [==============================] - 0s 2ms/step - loss: 1.1459 -
accuracy: 0.6354
Epoch 3/5
45/45 [==============================] - 0s 2ms/step - loss: 0.5060 -
accuracy: 0.8462
Epoch 4/5
45/45 [==============================] - 0s 2ms/step - loss: 0.3240 -
accuracy: 0.9040
Epoch 5/5
45/45 [==============================] - 0s 2ms/step - loss: 0.2358 -
accuracy: 0.9283
12/12 [==============================] - 0s 2ms/step - loss: 0.2600 -
accuracy: 0.9250
Loss: 0.2599562704563141
Accuracy: 0.925000011920929
1/1 [==============================] - 0s 52ms/step
[[9.9925131e-01 8.5686344e-07 7.1382141e-07 4.9433766e-06 6.5290674e-06
5.4754998e-04 2.5058553e-06 3.0487461e-06 3.8330847e-05 1.4414983e-04]]
[1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

(1797, 64)
<Figure size 640x480 with 0 Axes>

Page 79
Page 80