Program 1:

Aim: To Implement the FIND-S Algorithm for finding the most specific hypothesis based on
given training data which is in the form of CSV file.

Description:
The find-S algorithm is a basic concept learning algorithm in machine learning. The find-S
algorithm finds the most specific hypothesis that fits all the positive examples. We have to
note here that the algorithm considers only those positive training example. The find-S
algorithm starts with the most specific hypothesis and generalizes this hypothesis each
time it fails to classify an observed positive training data. Hence, the Find-S algorithm
moves from the most specific hypothesis to the most general hypothesis .

Algorithm:

1. Initialize h to the most specific hypothesis in H

2. For each positive training instance x For each attribute constraint a in h

i

If the constraint ai is satisfied by x Then do nothing

Else replace ai in h by the next more general constraint that is satisfied by x

3. Output hypothesis h
Program:
import pandas as pd
import numpy as np
#to read the data in the csv file
data = pd.read_csv("/content/[Link]")
print(data,"n")
#making an array of all the attributes
d = [Link](data)[:,:-1]
print("n The attributes are: ",d)
#segragating the target that has positive and negative examples
target = [Link](data)[:,-1]
print("n The target is: ",target)
#training function to implement find-s algorithm
def train(c,t):
for i, val in enumerate(t):
if val == "Yes":
specific_hypothesis = c[i].copy()
break

for i, val in enumerate(c):

if t[i] == "Yes":
 for x in range(len(specific_hypothesis)):
if val[x] != specific_hypothesis[x]:
specific_hypothesis[x] = '?'
else:
pass

return specific_hypothesis
#obtaining the final hypothesis
print("n The final hypothesis is:",train(d,target))

Dataset:
Time Weather Temperature Company Humidity Wind Goes

Morning Sunny Warm Yes Mild Strong Yes

Evening Rainy Cold No Mild Normal No

Morning Sunny Moderate Yes Normal Normal Yes

Evening Sunny Cold Yes High Strong Yes

Output:
Time Weather Temperature Company Humidity Wind Goes
0 Morning Sunny Warm Yes Mild Strong Yes
1 Evening Rainy Cold No Mild Normal No
2 Morning Sunny Moderate Yes Normal Normal Yes
3 Evening Sunny Cold Yes High Strong Yes n
n The attributes are: [['Morning' 'Sunny' 'Warm' 'Yes' 'Mild'
'Strong']
['Evening' 'Rainy' 'Cold' 'No' 'Mild' 'Normal']
['Morning' 'Sunny' 'Moderate' 'Yes' 'Normal' 'Normal']
['Evening' 'Sunny' 'Cold' 'Yes' 'High' 'Strong']]
n The target is: ['Yes' 'No' 'Yes' 'Yes']
n The final hypothesis is: ['?' 'Sunny' '?' 'Yes' '?' '?']
Program 2:
Aim: Applying the candidate elimination algorithm to find out whether the set of all
hypothesis are consistent or not.

Description:
The candidate elimination algorithm incrementally builds the version space given a
hypothesis space H and a set E of examples. The examples are added one by one; each
example possibly shrinks the version space by removing the hypotheses that are
inconsistent with the example. The candidate elimination algorithm does this by updating
the general and specific boundary for each new example.

Algorithm:
Step1: Load Data set
Step2: Initialize General Hypothesis and Specific Hypothesis.
Step3: For each training example
Step4: If example is positive example
if attribute_value == hypothesis_value:
Do nothing
else:
replace attribute value with '?' (Basically generalizing it)
Step5: If example is Negative example
Make generalize hypothesis more specific.

Program:

import numpy as np
import pandas as pd
data = pd.read_csv('/content/[Link]')
print(" Given data : \n\n",data)
concepts = [Link]([Link][:,0:-1])
print("\nInstances are:\n\n",concepts)
target = [Link]([Link][:,-1])
print("\nTarget Values are: ",target)
def learn(concepts, target):
specific_h = concepts[0].copy()
print("\nInitialization of specific_h and genearal_h")
print("\nSpecific Boundary: ", specific_h)
general_h = [["?" for i in range(len(specific_h))] for i in
range(len(specific_h))]
print("\nGeneric Boundary: ",general_h)
for i, h in enumerate(concepts):
print("\nInstance", i+1 , "is ", h)
if target[i] == "yes":
print("Instance is Positive ")
for x in range(len(specific_h)):
if h[x]!= specific_h[x]:
specific_h[x] ='?'
general_h[x][x] ='?'
 if target[i] == "no":
print("Instance is Negative ")
for x in range(len(specific_h)):
if h[x]!= specific_h[x]:
general_h[x][x] = specific_h[x]
else:
general_h[x][x] = '?'

print("Specific Bundary after ", i+1, "Instance is ", specific_h)

print("Generic Boundary after ", i+1, "Instance is ", general_h)
print("\n")
indices = [i for i, val in enumerate(general_h) if val == ['?', '?',
'?', '?', '?', '?']]
for i in indices:
general_h.remove(['?', '?', '?', '?', '?', '?'])
return specific_h, general_h
s_final, g_final = learn(concepts, target)
print("Final Specific_h: ", s_final, sep="\n")
print("Final General_h: ", g_final, sep="\n")

Output:
Given data :

[Link] Sky Air temp Humidity Wind Water Forecast Enjoy Sport
0 1 Suuny Warm Normal Strong Warm Same Yes
1 2 Sunny Warm High Strong Warm Same Yes
2 3 Rainy Cold High Strong Warm Change No
3 4 Sunny Warm High Strong Cool Change Yes

Instances are:

[[1 'Suuny' 'Warm' 'Normal' 'Strong' 'Warm' 'Same']

[2 'Sunny' 'Warm' 'High' 'Strong' 'Warm' 'Same']
[3 'Rainy' 'Cold' 'High' 'Strong' 'Warm' 'Change']
[4 'Sunny' 'Warm' 'High' 'Strong' 'Cool' 'Change']]

Target Values are: ['Yes' 'Yes' 'No' 'Yes']

Initialization of specific_h and genearal_h

Specific Boundary: [1 'Suuny' 'Warm' 'Normal' 'Strong' 'Warm' 'Same']

Generic Boundary: [['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?',
'?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?',
'?', '?', '?', '?', '?', '?']]

Instance 1 is [1 'Suuny' 'Warm' 'Normal' 'Strong' 'Warm' 'Same']

Instance 2 is [2 'Sunny' 'Warm' 'High' 'Strong' 'Warm' 'Same']

Instance 3 is [3 'Rainy' 'Cold' 'High' 'Strong' 'Warm' 'Change']

Instance 4 is [4 'Sunny' 'Warm' 'High' 'Strong' 'Cool' 'Change']

Specific Bundary after 4 Instance is ['?' '?' '?' '?' 'Strong' '?' '?']
 Generic Boundary after 4 Instance is [['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'],
['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?',
'?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?']]

Final Specific_h:
['?' '?' '?' '?' 'Strong' '?' '?']
Final General_h:
[['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?',
'?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?',
'?']]

Dataset:

[Link]. SKY AIR TEMP HUMIDITY WIND WATER FORECAST ENJOY SPORT

1. Sunny Warm Normal Strong Warm Same Yes

2. Sunny Warm High Strong Warm Same Yes
3. Rainy Cold High Strong Warm Change No
4. Sunny Warm High Strong Cool Change Yes
Program 3:
Aim: To implement decision tree based ID3 algorithm and to know the working principle of
decision trees.

Description:

ID3 stands for Iterative Dichotomiser 3 and is named such because the algorithm iteratively

(repeatedly) dichotomizes(divides) features into two or more groups at each step.

Invented by Ross Quinlan, ID3 uses a top-down greedy approach to build a decision tree. In

simple words, the top-down approach means that we start building the tree from the top and

the greedy approach means that at each iteration we select the best feature at the present

moment to create a node.

Most generally ID3 is only used for classification problems with nominal features only.

Algorithm:
1. Calculate entropy for dataset.

2. For each attribute/feature.

2.1. Calculate entropy for all its categorical values.
2.2. Calculate information gain for the feature.

3. Find the feature with maximum information gain.

4. Repeat it until we get the desired tree.

Program:
import numpy as np
import [Link] as plt
import pandas as pd
import math
import copy

dataset = pd.read_csv('/content/[Link]')
X = [Link][:, 1:].values
# print(X)
attribute = ['outlook', 'temp', 'humidity', 'wind']
class Node(object):
def __init__(self):
[Link] = None
[Link] = None
[Link] = None

def findEntropy(data, rows):

yes = 0
no = 0
ans = -1
idx = len(data[0]) - 1
entropy = 0
for i in rows:
if data[i][idx] == 'Yes':
yes = yes + 1
else:
no = no + 1

x = yes/(yes+no)
y = no/(yes+no)
if x != 0 and y != 0:
entropy = -1 * (x*math.log2(x) + y*math.log2(y))
if x == 1:
ans = 1
if y == 1:
ans = 0
return entropy, ans

def findMaxGain(data, rows, columns):

maxGain = 0
retidx = -1
entropy, ans = findEntropy(data, rows)
if entropy == 0:
"""if ans == 1:
print("Yes")
else:
print("No")"""
return maxGain, retidx, ans

for j in columns:
mydict = {}
idx = j
for i in rows:
key = data[i][idx]
if key not in mydict:
mydict[key] = 1
else:
 mydict[key] = mydict[key] + 1
gain = entropy

# print(mydict)
for key in mydict:
yes = 0
no = 0
for k in rows:
if data[k][j] == key:
if data[k][-1] == 'Yes':
yes = yes + 1
else:
no = no + 1
# print(yes, no)
x = yes/(yes+no)
y = no/(yes+no)
# print(x, y)
if x != 0 and y != 0:
gain += (mydict[key] * (x*math.log2(x) +
y*math.log2(y)))/14
# print(gain)
if gain > maxGain:
# print("hello")
maxGain = gain
retidx = j

return maxGain, retidx, ans

def buildTree(data, rows, columns):

maxGain, idx, ans = findMaxGain(X, rows, columns)

root = Node()
[Link] = []
# print(maxGain
#
# )
if maxGain == 0:
if ans == 1:
[Link] = 'Yes'
else:
[Link] = 'No'
return root

[Link] = attribute[idx]
mydict = {}
for i in rows:
key = data[i][idx]
if key not in mydict:
 mydict[key] = 1
else:
mydict[key] += 1

newcolumns = [Link](columns)
[Link](idx)
for key in mydict:
newrows = []
for i in rows:
if data[i][idx] == key:
[Link](i)
# print(newrows)
temp = buildTree(data, newrows, newcolumns)
[Link] = key
[Link](temp)
return root

def traverse(root):
print([Link])
print([Link])

n = len([Link])
if n > 0:
for i in range(0, n):
traverse([Link][i])

def calculate():
rows = [i for i in range(0, 14)]
columns = [i for i in range(0, 4)]
root = buildTree(X, rows, columns)
[Link] = 'Start'
traverse(root)

calculate()

Output:
Start
outlook
D1
No
D2
No
D3
Yes
D4
Yes
D5
Yes
 D6
No
D7
Yes
D8
No
D9
Yes
D10
Yes
D11
Yes
D12
Yes
D13
Yes
D14
No

Dataset:
Outlook Temparature Humidity Wind Answer
sunny Hot High Weak No
Sunny Hot High Strong No
Overcast Hot High Weak Yes
Rain Mild High Weak Yes
Rain Cool Normal Weak Yes
Rain Cool Normal Strong No
Overcast Cool Normal Strong Yes
Sunny Mild High Weak No
Sunny Cool Normal Weak Yes
Rain Mild Normal Weak Yes
Sunny Mild Normal Strong Yes
Overcast Mild High Strong Yes
Overcast Hot Normal Weak Yes
Rain Mild High Strong No

Program 4
Aim: Learning the concept of Artificial neural network by applying the Back-Propagation
algorithm.

Description:

Backpropagation, or backward propagation of errors, is an algorithm that is designed to test

for errors working back from output nodes to input nodes. It is an important mathematical
tool for improving the accuracy of predictions in data mining and machine learning.
Essentially, backpropagation is an algorithm used to calculate derivatives quickly.

There are two leading types of backpropagation networks:

1. Static backpropagation. Static backpropagation is a network developed to map

static inputs for static outputs. Static backpropagation networks can solve static
classification problems, such as optical character recognition (OCR).

2. Recurrent backpropagation. The recurrent backpropagation network is used for

fixed-point learning. Recurrent backpropagation activation feeds forward until it
reaches a fixed value.

Algorithm:
Step 1: Inputs X, arrive through the preconnected path.
Step 2: The input is modeled using true weights W. Weights are usually chosen randomly.
Step 3: Calculate the output of each neuron from the input layer to the hidden layer to the
output layer.
Step 4: Calculate the error in the outputs
Backpropagation Error= Actual Output – Desired Output
Step 5: From the output layer, go back to the hidden layer to adjust the weights to reduce
the error.
Step 6: Repeat the process until the desired output is achieved.

Program:

import numpy as np
X = [Link](([2, 9], [1, 5], [3, 6]), dtype=float)
y = [Link](([92], [86], [89]), dtype=float)
X = X/[Link](X,axis=0) # maximum of X array longitudinally
y = y/100
#Sigmoid Function
def sigmoid (x):
return 1/(1 + [Link](-x))
#Derivative of Sigmoid Function
def derivatives_sigmoid(x):
return x * (1 - x)
#Variable initialization
epoch=5000 #Setting training iterations
lr=0.1 #Setting learning rate
inputlayer_neurons = 2 #number of features in data set
hiddenlayer_neurons = 3 #number of hidden layers neurons
output_neurons = 1 #number of neurons at output layer
#weight and bias initialization
wh=[Link](size=(inputlayer_neurons,hiddenlayer_neurons))
bh=[Link](size=(1,hiddenlayer_neurons))
wout=[Link](size=(hiddenlayer_neurons,output_neurons))
bout=[Link](size=(1,output_neurons))
#draws a random range of numbers uniformly of dim x*y
for i in range(epoch):
#Forward Propogation
hinp1=[Link](X,wh)
hinp=hinp1 + bh
hlayer_act = sigmoid(hinp)
outinp1=[Link](hlayer_act,wout)
outinp= outinp1+ bout
output = sigmoid(outinp)
#Backpropagation
EO = y-output
outgrad = derivatives_sigmoid(output)
d_output = EO* outgrad
EH = d_output.dot(wout.T)
#how much hidden layer wts contributed to error
hiddengrad = derivatives_sigmoid(hlayer_act)
d_hiddenlayer = EH * hiddengrad
# dotproduct of nextlayererror and currentlayerop
wout += hlayer_act.[Link](d_output) *lr
wh += [Link](d_hiddenlayer) *lr
print("Input: \n" + str(X))
print("Actual Output: \n" + str(y))
print("Predicted Output: \n" ,output)

Output:
Input:
[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]
Actual Output:
[[0.92]
[0.86]
[0.89]]
Predicted Output:
[[0.89591615]
[0.87528844]
[0.8982718 ]]
Program 5
Aim: Finding the accuracy of a classification model by applying the Naïve Baysian
classifier.

Description:
The Naïve Bayes classifier is a supervised machine learning algorithm, which is used for
classification tasks, like text classification. It is also part of a family of generative learning
algorithms, meaning that it seeks to model the distribution of inputs of a given class or
category. Unlike discriminative classifiers, like logistic regression, it does not learn which
features are most important to differentiate between classes.

Algorithm:

Working of Naïve Bayes' Classifier can be understood with the help of the below example:

Suppose we have a dataset of weather conditions and corresponding target variable

"Play". So using this dataset we need to decide that whether we should play or not on a
particular day according to the weather conditions. So to solve this problem, we need to
follow the below steps:

1. Convert the given dataset into frequency tables.

2. Generate Likelihood table by finding the probabilities of given features.
3. Now, use Bayes theorem to calculate the posterior probability.

Program:

import pandas as pd
from sklearn import tree
from [Link] import LabelEncoder
from sklearn.naive_bayes import GaussianNB

# load data from CSV

data = pd.read_csv('/content/[Link]')
print("THe first 5 values of data is :\n",[Link]())

# obtain Train data and Train output

X = [Link][:,:-1]
print("\nThe First 5 values of train data is\n",[Link]())

y = [Link][:,-1]
print("\nThe first 5 values of Train output is\n",[Link]())

# Convert then in numbers

le_outlook = LabelEncoder()
[Link] = le_outlook.fit_transform([Link])

le_Temperature = LabelEncoder()
[Link] = le_Temperature.fit_transform([Link])
le_Humidity = LabelEncoder()
[Link] = le_Humidity.fit_transform([Link])

le_Windy = LabelEncoder()
[Link] = le_Windy.fit_transform([Link])

print("\nNow the Train data is :\n",[Link]())

le_PlayTennis = LabelEncoder()
y = le_PlayTennis.fit_transform(y)
print("\nNow the Train output is\n",y)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X,y,
test_size=0.20)

classifier = GaussianNB()
[Link](X_train,y_train)

from [Link] import accuracy_score

print("Accuracy is:",accuracy_score([Link](X_test),y_test))

Dataset:

Outloo Temperatur Humidit

k e y Windy PlayTennis
2 Sunny Hot High FALSE No
3 Sunny Hot High TRUE No
4 Overcast Hot High FALSE Yes
5 Rainy Mild High FALSE Yes
6 Rainy Cool Normal FALSE Yes
7 Rainy Cool Normal TRUE No
8 Overcast Cool Normal TRUE Yes
9 Sunny Mild High FALSE No
10 Sunny Cool Normal FALSE Yes
11 Rainy Mild Normal FALSE Yes
12 Sunny Mild Normal TRUE Yes
13 Overcast Mild High TRUE Yes
14 Overcast Hot Normal FALSE Yes
15 Rainy Mild High TRUE No

Output:
THe first 5 values of data is :
Outlook Temperature Humidity Windy PlayTennis Unnamed: 5
0 2 Sunny Hot High False No
1 3 Sunny Hot High True No
2 4 Overcast Hot High False Yes
3 5 Rainy Mild High False Yes
4 6 Rainy Cool Normal False Yes

The First 5 values of train data is

Outlook Temperature Humidity Windy PlayTennis
0 2 Sunny Hot High False
1 3 Sunny Hot High True
2 4 Overcast Hot High False
3 5 Rainy Mild High False
4 6 Rainy Cool Normal False

The first 5 values of Train output is

0 No
1 No
2 Yes
3 Yes
4 Yes
Name: Unnamed: 5, dtype: object

Now the Train data is :

Outlook Temperature Humidity Windy PlayTennis
0 0 2 1 0 False
1 1 2 1 0 True
2 2 0 1 0 False
3 3 1 2 0 False
4 4 1 0 1 False

Now the Train output is

[0 0 1 1 1 0 1 0 1 1 1 1 1 0]
Accuracy is: 0.3333333333333333

Program 6
Aim: Finding the Accuracy,Precision and Recall of textual document by implementing the
Naïve Bayesian Classifier model.

Description:

The Naïve Bayes algorithm is comprised of two words Naïve and Bayes, Which can be
described as:

o Naïve: It is called Naïve because it assumes that the occurrence of a certain feature
is independent of the occurrence of other features. Such as if the fruit is identified on
the bases of color, shape, and taste, then red, spherical, and sweet fruit is
recognized as an apple. Hence each feature individually contributes to identify that it
is an apple without depending on each other.
o Bayes: It is called Bayes because it depends on the principle of Bayes' Theorem.

Bayes' Theorem:

o Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to
determine the probability of a hypothesis with prior knowledge. It depends on the
conditional probability.
o The formula for Bayes' theorem is given as:

Where,

P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.

P(B|A) is Likelihood probability: Probability of the evidence given that the probability of a
hypothesis is true.

Algorithm:
Working of Naïve Bayes' Classifier can be understood with the help of the below example:

Suppose we have a dataset of weather conditions and corresponding target variable

"Play". So using this dataset we need to decide that whether we should play or not on a
particular day according to the weather conditions. So to solve this problem, we need to
follow the below steps:

1. Convert the given dataset into frequency tables.

2. Generate Likelihood table by finding the probabilities of given features.
3. Now, use Bayes theorem to calculate the posterior probability.
Program:

from [Link] import fetch_20newsgroups

from [Link] import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn import metrics

news = fetch_20newsgroups()
print("All Targets\n", news["target_names"])

categories = ['[Link]', '[Link]', '[Link]',

'[Link]']
news_train = fetch_20newsgroups(subset='train', categories=categories,
shuffle='true')
news_test = fetch_20newsgroups(subset='test', categories=categories,
shuffle='true')
print("Target Names", news_train.target_names)

text_clf = Pipeline([('vect', TfidfVectorizer()), ('clf',

MultinomialNB())])
text_clf.fit(news_train.data, news_train.target)
predicted = text_clf.predict(news_test.data)

print("Accuracy", metrics.accuracy_score(news_test.target, predicted))

print(metrics.classification_report(news_test.target, predicted,
target_names=news_test.target_names))

print("Confusion Matrix:\n", metrics.confusion_matrix(news_test.target,

predicted))

Output:
All Targets
['[Link]', '[Link]', '[Link]',
'[Link]', '[Link]', '[Link].x',
'[Link]', '[Link]', '[Link]', '[Link]',
'[Link]', '[Link]', '[Link]', '[Link]',
'[Link]', '[Link]', '[Link]',
'[Link]', '[Link]', '[Link]']
Target Names ['[Link]', '[Link]', '[Link]',
'[Link]']
Accuracy 0.8348868175765646

precision recall f1-score support

[Link] 0.97 0.60 0.74 319
 [Link] 0.96 0.89 0.92 389
[Link] 0.97 0.81 0.88 396
[Link] 0.65 0.99 0.78 398

accuracy 0.83 1502

macro avg 0.89 0.82 0.83 1502
weighted avg 0.88 0.83 0.84 1502

Confusion Matrix:
[[192 2 6 119]
[ 2 347 4 36]
[ 2 11 322 61]

[ 2 2 1 393]]

Program 7
Aim: Applying Bayesian network on heart disease to find a correct classification label.

Description:
In Bayesian networks, we deal with a number of interrelated (random) variables. We explore
how the joint distribution of the variables can be described by exploiting what we know about
their natural interrelationships via conditional distributions. We use graph theory to explain
their interrelationship. If data are available on the random variables, we fit a Bayesian
network model which describes their relationship in a succinct way. Bayesian networks are a
marriage between probability theory and graphs. One of the main goals in Bayesian
networks is prediction. Bayes theorem plays a crucial part in this connection.

Algorithm:

There are three main steps to create a BN :

1. First, identify which are the main variable in the problem to solve. Each variable

corresponds to a node of the network. It is important to choose the number states for each

variable, for instance, there are usually two states (true or false).

2. Second, define structure of the network, that is, the causal relationships between all the

variables (nodes).

3. Third, define the probability rules governing the relationships between the variables.

Program:
import numpy as np
import csv
import pandas as pd
from [Link] import BayesianModel
from [Link] import MaximumLikelihoodEstimatorfrom
[Link] import
VariableElimination
#read Cleveland Heart Disease data
heartDisease = pd.read_csv('[Link]')
heartDisease = [Link]('?',[Link])
#display the data
print('Few examples from the dataset are given below')
print([Link]())
#Model Bayesian Network
Model=BayesianModel([('age','trestbps'),('age','fbs'),
('sex','trestbps'),('exang','trestbps'),('trestbps','heartdise
ase'),('fbs','heartdisease'),('heartdisease','restecg'),
('heartdisease','thalach'),('heartdisease','chol')])
#Learning CPDs using Maximum Likelihood Estimators
print('\n Learning CPD using Maximum likelihood estimators')
[Link](heartDisease,estimator=MaximumLikelihoodEstimator)
# Inferencing with Bayesian Network
print('\n Inferencing with Bayesian Network:') HeartDisease_infer =
VariableElimination(model)
#computing the Probability of HeartDisease given Age
print('\n 1. Probability of HeartDisease given Age=30')
q=HeartDisease_infer.query(variables=['heartdisease'],evidence
={'age':28})
print(q['heartdisease'])
#computing the Probability of HeartDisease given cholesterol print('\n
2. Probability of HeartDisease
given cholesterol=100')
q=HeartDisease_infer.query(variables=['heartdisease'],evidence
={'chol':100})
print(q['heartdisease'])

Output:

Inferencing with Bayesian Network:

1. Probability of HeartDisease given Age=28

╒════════════════╤═════════════════════╕ │ heartdisease │

phi(heartdisease) │ ╞════════════════╪═════════════════════╡ │
heartdisease_0 │ 0.6791 │ ├────────────────┼─────────────────────┤ │

heartdisease_1 │ 0.1212 │ ├────────────────┼─────────────────────┤ │

heartdisease_2 │ 0.0810 │ ├────────────────┼─────────────────────┤ │

heartdisease_3 │ 0.0939 │ ├────────────────┼─────────────────────┤ │

heartdisease_4 │ 0.0247 │

╘════════════════╧═════════════════════╛

2. Probability of Heart Disease given cholesterol=100

╒════════════════╤═════════════════════╕ │ heartdisease │
 phi(heartdisease) │ ╞════════════════╪═════════════════════╡ │
heartdisease_0 │ 0.5400 │ ├────────────────┼─────────────────────┤ │

heartdisease_1 │ 0.1533 │ ├────────────────┼─────────────────────┤ │

heartdisease_2 │ 0.1303 │ ├────────────────┼─────────────────────┤ │

heartdisease_3 │ 0.1259 │ ├────────────────┼─────────────────────┤ │

heartdisease_4 │ 0.0506 │ ╘════════════════╧═══════------------------------

Dataset:
age sex cp trestbps chol fbs restec g thalac h exan g oldpea k slop e ca thal Heartdiseas e

63 1 1 145 233 1 2 150 0 2.3 3 0 6 0

67 1 4 160 286 0 2 108 1 1.5 2 3 3 2
67 1 4 120 229 0 2 129 1 2.6 2 2 7 1
41 0 2 130 204 0 2 172 0 1.4 1 0 3 0
62 0 4 140 268 0 2 160 0 3.6 3 2 3 3
60 1 4 130 206 0 2 132 1 2.4 2 2 7 4

Program 8
Aim: Comparing between EM algorithm and K-Means algorithm to find which of them is
best algorithm.

Description:
EM is a very general algorithm for learning models with hidden variables. EM optimizes the
marginal likelihood of the data (likelihood with hidden variables summed out). Like K-means,
it's iterative, alternating two steps, E and M, which correspond to estimating hidden variables
given the model and then estimating the model given the hidden variable estimates. Unlike
K-means, the cluster assignments in EM for Gaussian mixtures are soft. Let's consider the
simplest case, closest to K-means.

Algorithm:
K-means:
Step 1: Select the Number of Clusters, k. ...
Step 2: Select k Points at Random. ...
Step 3: Make k Clusters. ...
Step 4: Compute New Centroid of Each Cluster. ...
Step 5: Assess the Quality of Each Cluster. ...
Step 6: Repeat Steps 3–5.

EM Algorithm:
o 1st Step: The very first step is to initialize the parameter values. Further, the system
is provided with incomplete observed data with the assumption that data is obtained
from a specific model.

o 2nd Step: This step is known as Expectation or E-Step, which is used to estimate or
guess the values of the missing or incomplete data using the observed data. Further,
E-step primarily updates the variables.
o 3rd Step: This step is known as Maximization or M-step, where we use complete data
obtained from the 2nd step to update the parameter values. Further, M-step primarily
updates the hypothesis.
o 4th step: The last step is to check if the values of latent variables are converging or
not. If it gets "yes", then stop the process; else, repeat the process from step 2 until
the convergence occurs.

Program:

from [Link] import KMeans

from sklearn import preprocessing
from [Link] import GaussianMixture
from [Link] import load_iris
import [Link] as sm
import pandas as pd
import numpy as np
import [Link] as plt

dataset=load_iris()
# print(dataset)

X=[Link]([Link])
[Link]=['Sepal_Length','Sepal_Width','Petal_Length','Petal_Width']
y=[Link]([Link])
[Link]=['Targets']
# print(X)

[Link](figsize=(14,7))
colormap=[Link](['red','lime','black'])

# REAL PLOT
[Link](1,3,1)
[Link](X.Petal_Length,X.Petal_Width,c=colormap[[Link]],s=40)
[Link]('Real')

# K-PLOT
[Link](1,3,2)
model=KMeans(n_clusters=3)
[Link](X)
predY=[Link](model.labels_,[0,1,2]).astype(np.int64)
[Link](X.Petal_Length,X.Petal_Width,c=colormap[predY],s=40)
[Link]('KMeans')

# GMM PLOT
scaler=[Link]()
[Link](X)
xsa=[Link](X)
xs=[Link](xsa,columns=[Link])
gmm=GaussianMixture(n_components=3)
[Link](xs)

y_cluster_gmm=[Link](xs)
[Link](1,3,3)
[Link](X.Petal_Length,X.Petal_Width,c=colormap[y_cluster_gmm],s=40
)
[Link]('GMM Classification')

Output:
/usr/local/lib/python3.10/dist-packages/sklearn/cluster/_kmeans.py:870:
FutureWarning: The default value of `n_init` will change from 10 to 'auto'
in 1.4. Set the value of `n_init` explicitly to suppress the warning
[Link](
Text(0.5, 1.0, 'GMM Classification')
Program 9
Aim: To implement the K-Nearest Neighbour algorithm on iris dataset to check whether the
algorithm predicts the output correct or not.
Description:

The k-nearest neighbors algorithm, also known as KNN or k-NN, is a non-parametric,

supervised learning classifier, which uses proximity to make classifications or predictions
about the grouping of an individual data point. While it can be used for either regression or
classification problems, it is typically used as a classification algorithm, working off the
assumption that similar points can be found near one another.

Algorithm:
The K-NN working can be explained on the basis of the below algorithm:

o Step-1: Select the number K of the neighbors

o Step-2: Calculate the Euclidean distance of K number of neighbors
o Step-3: Take the K nearest neighbors as per the calculated Euclidean distance.
o Step-4: Among these k neighbors, count the number of the data points in each
category.
o Step-5: Assign the new data points to that category for which the number of the
neighbor is maximum.
o Step-6: Our model is ready.

Program:

from [Link] import load_iris

from [Link] import KNeighborsClassifier
from sklearn.model_selection import train_test_split
import numpy as np

dataset=load_iris()
#print(dataset)
X_train,X_test,y_train,y_test=train_test_split(dataset["data"],dataset[
"target"],random_state=0)

kn=KNeighborsClassifier(n_neighbors=1)
[Link](X_train,y_train)

for i in range(len(X_test)):
x=X_test[i]
x_new=[Link]([x])
prediction=[Link](x_new)
print("TARGET=",y_test[i],dataset["target_names"]
[y_test[i]],"PREDICTED=",prediction,dataset["target_names"]
[prediction])
print([Link](X_test,y_test))

Output:
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 2 virginica PREDICTED= [2] ['virginica']
TARGET= 1 versicolor PREDICTED= [1] ['versicolor']
TARGET= 0 setosa PREDICTED= [0] ['setosa']
TARGET= 1 versicolor PREDICTED= [2] ['virginica']
0.9736842105263158

Program 10
Aim: To Implement the non-parametric locally weighted regression algorithm to fit the data

points by using the appropriate datset .

Description:
Locally Weighted Linear Regression (LWLR) is a non-parametric regression technique that
aims to fit a linear regression model to a dataset by giving more weight to nearby data
points. For example, consider a dataset of temperature readings and corresponding energy
consumption. LWLR can be used to predict the energy consumption for a given temperature
reading by fitting a linear regression model to the training data, where the weight assigned to
each training data point is inversely proportional to its distance from the query point. This
means that training data points that are closer to the query point will have a higher weight
and contribute more to the linear regression model.

Algorithm:
 Read the Given data Sample to X and the curve (linear or non linear) to Y.
 Set the value for Smoothening parameter or Free parameter say τ
 Set the bias /Point of interest set x0 which is a subset of X.
 Determine the weight matrix using :
 Determine the value of model term parameter β using:
 Prediction = x0*β

Program:
import [Link] as plt
import pandas as pd
import numpy as np
def kernel(point, xmat, k):
m,n = [Link](xmat)
weights = [Link]([Link]((m)))
for j in range(m):
diff = point - X[j]
weights[j,j] = [Link](diff*diff.T/(-2.0*k**2))
return weights
def localWeight(point, xmat, ymat, k):
wei = kernel(point,xmat,k)
W = (X.T*(wei*X)).I*(X.T*(wei*ymat.T))
return W

def localWeightRegression(xmat, ymat, k):

m,n = [Link](xmat)
ypred = [Link](m)
for i in range(m):
ypred[i] =xmat[i]*localWeight(xmat[i],xmat,ymat,k)
return ypred

# load data points

data = pd.read_csv('/content/[Link]')
bill = [Link](data.total_bill)
tip = [Link]([Link])
#preparing and add 1 in bill
mbill = [Link](bill)
mtip = [Link](tip)
m= [Link](mbill)[1]
one = [Link]([Link](m))
X = [Link]((one.T,mbill.T))
#set k here
ypred = localWeightRegression(X,mtip,0.5)
SortIndex = X[:,1].argsort(0)
xsort = X[SortIndex][:,0]
fig = [Link]()
ax = fig.add_subplot(1,1,1)
[Link](bill,tip, color='green')
[Link](xsort[:,1],ypred[SortIndex], color = 'red', linewidth=5)
[Link]('Total bill')
[Link]('Tip')
[Link]();

Dataset:
total_bil
l tip sex smoker day time size
0 16.99 1.01 Female No Sun Dinner 2
1 10.34 1.66 Male No Sun Dinner 3
2 21.01 3.5 Male No Sun Dinner 3
3 23.68 3.31 Male No Sun Dinner 2
4 24.59 3.61 Female No Sun Dinner 4
5 22.67 4.34 Female No Sun Dinner 3
239 29.03 5.92 Male No Sat Dinner 3
240 27.18 2 Female Yes Sat Dinner 2
241 22.67 2 Male Yes Sat Dinner 2
242 17.82 1.75 Male No Sat Dinner 2

Output: