DEPARTMENTOF ARTIFICIAL INTELLIGENCE & DATA SCIENCE

LABORATORY MANUAL

AD3461-MACHINE LEARNING LABORATORY

REGULATION-2021

II YEAR / IV SEMESTER
 INDEX

[Link] LISTOF PROGRAMS PAGE

NO
1 IMPLEMENTATIONOFCANDIDATE–
ELIMINATIONALGORITHM

2 IMPLEMENTATIONOFDECISIONTREEINID3
ALGORITHM

3 IMPLEMENTATION OF BACK PROPAGATION

ALGORITHMTOBUILDANARTIFICIALNEURAL
NETWORK

4 IMPLEMENTATION OF NAÏVE BAYESIAN

CLASSIFIERFORASAMPLETRAININGDATASET AND
TO COMPUTE ACCURACY

5 IMPLEMENTATION OF NAÏVE BAYESIAN

CLASSIFIER MODEL TO CLASSIFY A SET OF
DOCUMENTSANDTOMEASURETHEACCURACY,
PRECISION, AND RECALL

6 CONSTRUCTIONOFABAYESIANNETWORKTO
DIAGNOSE CORONA INFECTION USING
STANDARD WHO DATA SET

7 COMPARISON OF CLUSTERING IN EM
ALGORITHMANDK-MEANSALGORITHMUSING
THE SAME DATA SETS

8 IMPLEMENTATIONOFK-NEARESTNEIGHBOUR
ALGORITHM TO CLASSIFY THE IRIS DATA SET

9 IMPLEMENTATION OF THE NON-PARAMETRIC

LOCALLYWEIGHTEDREGRESSIONALGORITHM
IN ORDER TO FIT DATA POINTS
 AD3461 MACHINE LEARNING LABORATORY

OBJECTIVE:
 To understand the data sets and apply suitable algorithms for selecting the appropriate features
for analysis.
 To learn to implement supervised machine learning algorithms on standard datasets and evaluate
the performance.
 To experiment the unsupervised machine learning algorithms on standard datasets and evaluate
the performance.
 To build the graph based learning models for standard data sets.
 To compare the performance of different ML algorithms and select the suitable one based on
the application
LIST OF EXPERIMENTS
1. For a given set of training data examples stored in a .CSV file, implement and demonstrate the Candidate-
Elimination algorithm to output a description of the set of all hypotheses consistent with the training examples.
2. Write a program to demonstrate the working of the decision tree based ID3 algorithm. Use an appropriate data
set for building the decision tree and apply this knowledge to classify a new sample.
3. Build an Artificial Neural Network by implementing the Backpropagation algorithm and test the same
using appropriate data sets.
4. Write a program to implement the naïve Bayesian classifier for a sample training data set stored as a .
CSV file and compute the accuracy with a few test data sets.
5. Implement naïve Bayesian Classifier model to classify a set of documents and measure the accuracy,
precision, and recall.
6. Write a program to construct a Bayesian network to diagnose CORONA infection using standard
WHO Data Set.
7. Apply EM algorithm to cluster a set of data stored in a .CSV file. Use the same data set for clustering
using the k-Means algorithm. Compare the results of these two algorithms.
8. Write a program to implement k-Nearest Neighbor algorithm to classify the iris data set. Print both correct
and wrong predictions.
9. Implement the non-parametric Locally Weighted Regression algorithm in order to fit data points. Select
an appropriate data set for your experiment and draw graphs.

COURSE OUTCOMES:

 Apply suitable algorithms for selecting the appropriate features for analysis.
 Implement supervised machine learning algorithms on standard datasets and evaluate the performance.
 Apply unsupervised machine learning algorithms on standard datasets and evaluate the performance.
 Build the graph based learning models for standard data sets.
 Assess and compare the performance of different ML algorithms and select the suitable one based on the
application.
Expt. No : 1 IMPLEMENTATION OF CANDIDATE–ELIMINATION ALGORITHM
Date :

AIM:
ToimplementanddemonstratetheCandidate-Eliminationalgorithm,foragivensetof
training data examples stored in a .CSV file, to output a description of the set of all hypotheses
consistent with the training examples.
ALGORITHM:

1. LoadDataset.
2. InitializeGeneralHypothesisandSpecificHypothesis.
3. Foreachtrainingexample
4. Ifexampleispositiveexample
ifattribute_value==hypothesis_value:
Donothing
else:
replaceattributevaluewith'?'(Basicallygeneralizingit)
5. IfexampleisNegativeexample
Makegeneralizehypothesismorespecific.

PROGRAM:

[Link]

1
importnumpyasnp
importpandasaspd
#LoadingDatafromaCSVFile
data=[Link](data=pd.read_csv('E:\BALA\AI\Labprograms\pgms\[Link]')) print(data)

#SeparatingconceptfeaturesfromTarget
concepts = [Link]([Link][:,0:-1])
print(concepts)

#IsolatingtargetintoaseparateDataFrame #
copying last column to target array
target=[Link]([Link][:,-1])
print(target)

2
deflearn(concepts,target):
'''
learn()functionimplementsthelearningmethodoftheCandidateeliminationalgorithm. Arguments:
concepts-adataframewithall thefeatures
target-adataframewithcorrespondingoutputvalues
'''
#InitialiseS0 with thefirstinstancefrom concepts
#.copy()makessureanewlistiscreated insteadofjustpointing tothesamememorylocation
specific_h = concepts[0].copy()
print("\nInitializationofspecific_handgeneral_h")
print(specific_h)
#h=["#"foriinrange(0,5)]
#print(h)

general_h=[["?"foriinrange(len(specific_h))]foriinrange(len(specific_h))] print(general_h)
#Thelearningiterations
fori,hinenumerate(concepts):
#Checkingifthehypothesishasapositivetarget if
target[i] == "Yes":
forxin range(len(specific_h)):
#ChangevaluesinS&Gonlyifvalueschange if
h[x] != specific_h[x]:
specific_h[x] = '?'
general_h[x][x]='?'
#Checkingifthehypothesis hasapositivetarget

3
 iftarget[i]=="No":
forxin range(len(specific_h)):
#Fornegativehyposthesischangevalues onlyinG if
h[x] != specific_h[x]:
general_h[x][x]=specific_h[x]
else:
general_h[x][x]= '?'
print("\nStepsofCandidateEliminationAlgorithm",i+1)
print(specific_h)
print(general_h)
#findindiceswherewehaveemptyrows,meaningthosethatareunchanged
indices =[ifor i,val inenumerate(general_h)ifval==['?', '?','?', '?', '?','?']] for i
in indices:
# remove those rows from
general_hgeneral_h.remove(['?','?','?','?'
,'?','?'])
#Returnfinal values
returnspecific_h,general_h
s_final, g_final = learn(concepts, target)
print("\nFinalSpecific_h:",s_final,sep="\n")
print("\nFinalGeneral_h:",g_final,sep="\n")

4
OUTPUT:

5
6
RESULT:
Thus the Candidate-Elimination algorithm, to test all the hypotheses with the training sets using
python was executed and verified successfully.

7
Expt. No: 2 IMPLEMENTATIONOFDECISIONTREEINID3ALGORITHM
Date:

AIM:
TobuildDecisiontreeinID3algorithmtoclassifyanewsampleusing python.
ALGORITHM:

1. Observethedataset. Importthenecessarybasicpythonlibraries.
2. Readthedataset.
3. Calculate theEntropyofthewholedataset.
4. Calculate theEntropyofthefiltered dataset.
5. Calculate theInformationgain forthefeature(outlook).
6. Findingthemost informativefeature(featurewithhighestinformation gain).
7. Addinganodeto thetree.
8. PerformID3algorithmandgenerateatree.
9. Findinguniqueclassesofthelabel.
10. Predictingfromthetree.
11. Evaluatingthetestdataset.
12. Checkingthetestdataset.

PROGRAM:
importnumpyasnp import
math
importcsv
defread_data(filename):
withopen(filename,'r')ascsvfile:
datareader=[Link](csvfile,delimiter=',')
headers = next(datareader)
metadata=[]
traindata=[]
for name in headers:
[Link](name)

8
[Link](row)

return(metadata,traindata)
classNode:
definit(self,attribute):
[Link] = attribute
[Link] = []
[Link] = ""

defstr(self):
[Link]
defsubtables(data,col,delete):
dict = {}
items=[Link](data[:,col])
count=[Link](([Link][0],1),dtype=np.int32) for
x in range([Link][0]):
foryinrange([Link][0]):
ifdata[y, col]== items[x]:
count[x]+=1
forxin range([Link][0]):
dict[items[x]]=[Link]((int(count[x]),[Link][1]),dtype="|S32") pos
=0
foryinrange([Link][0]):
ifdata[y, col]== items[x]:
dict[items[x]][pos]=data[y]
pos += 1
if delete:
dict[items[x]]=[Link](dict[items[x]],col,1)
return items, dict

9
defentropy(S):
items=[Link](S)
if [Link] == 1:
return0
counts=[Link](([Link][0],1))
sums = 0
forxin range([Link][0]):
counts[x]=sum(S==items[x])/([Link]*1.0) for
count in counts:
sums+=-1*count*[Link](count,2) return
sums
defgain_ratio(data, col):
items,dict=subtables(data,col,delete=False)
total_size = [Link][0]
entropies=[Link](([Link][0],1))
intrinsic = [Link](([Link][0], 1))
for x in range([Link][0]):
ratio = dict[items[x]].shape[0]/(total_size * 1.0)
entropies[x]=ratio*entropy(dict[items[x]][:,-1])
intrinsic[x] = ratio * [Link](ratio, 2)
total_entropy=entropy(data[:,-1]) iv
= -1 * sum(intrinsic)
forxinrange([Link][0]):
total_entropy -= entropies[x]
returntotal_entropy/iv
defcreate_node(data,metadata):
if([Link](data[:,-1])).shape[0]==1:
node = Node("")
 [Link]=[Link](data[:,-1])[0]
return node
gains=[Link](([Link][1]-1,1)) for
col in range([Link][1] - 1):
gains[col]=gain_ratio(data,col)
split = [Link](gains)
node= Node(metadata[split])
metadata=[Link](metadata,split,0)
items,dict=subtables(data,split,delete=True)
for x in range([Link][0]):
child=create_node(dict[items[x]],metadata)
[Link]((items[x], child))
return node
defempty(size):
s= ""
forxinrange(size): s
+= ""
returns
defprint_tree(node,level): if
[Link] != "":
print(empty(level),[Link])
return
print(empty(level),[Link])
for value, n in [Link]:
print(empty(level+1),value)
print_tree(n, level + 2)
metadata,traindata=read_data("E:\BALA\AI\Labprograms\pgms\[Link]") data
= [Link](traindata)

10
OUTPUT:

11
RESULT:
ThustheprogramtoimplementdecisiontreebasedID3algorithmusingpythonwas executed
and verified successfully.

12
Expt. No. 3 IMPLEMENTATIONOFBACKPROPAGATIONALGORITHMTOBUILDAN
ARTIFICIAL NEURAL NETWORK
Date:

AIM:
ToimplementtheBackPropagationalgorithmtobuildanArtificialNeuralNetwork.
ALGORITHM:

1. InputsX,arrivethroughthepreconnectedpath.
2. [Link].
3. Calculatetheoutputforeveryneuronfromtheinputlayer,to thehiddenlayers,to the output
layer.
4. Calculatetheerrorin theoutputs
5. Travelbackfrom theoutput layer tothehiddenlayer toadjusttheweightssuchthatthe errors
is decreased. Keep repeating the process until the desired output is achieved.

PROGRAM:

from math import exp

fromrandomimportseed
fromrandomimportrandom #
Initialize a network
definitialize_network(n_inputs,n_hidden,n_outputs):

13
 network=list()
hidden_layer=[{'weights':[random()foriinrange(n_inputs+1)]}foriinrange(n_hidden)]
[Link](hidden_layer)
output_layer=[{'weights':[random()foriinrange(n_hidden+1)]}foriinrange(n_outputs)]
[Link](output_layer)
return network
#Calculateneuronactivationforaninput def
activate(weights, inputs):
activation=weights[-1]
foriinrange(len(weights)-1):
activation+=weights[i]*inputs[i]
return activation
#Transferneuronactivation
def transfer(activation):
return1.0/(1.0+exp(-activation)) #
Forward propagate input to a network output
defforward_propagate(network, row):
inputs=row
forlayerin network:
new_inputs =
[]forneuroninlayer:
activation=activate(neuron['weights'],inputs)
neuron['output'] = transfer(activation)
new_inputs.append(neuron['output'])
inputs=new_inputs
return inputs
#Calculatethederivativeofanneuron output

14
deftransfer_derivative(output):
returnoutput*(1.0-output) #
Backpropagate error and store in neurons
defbackward_propagate_error(network, expected):
foriinreversed(range(len(network))):
layer=network[i]
errors = list()
ifi!=len(network)-1:
forjinrange(len(layer)):
error = 0.0
forneuroninnetwork[i+1]:
error+=(neuron['weights'][j]*neuron['delta'])
[Link](error)
else:
forjinrange(len(layer)):
neuron = layer[j]
[Link](neuron['output']-expected[j])
forjin range(len(layer)):
neuron=layer[j]
neuron['delta']=errors[j]*transfer_derivative(neuron['output']) #
Update network weights with error
defupdate_weights(network,row,l_rate):
foriinrange(len(network)):
inputs = row[:-1]
ifi!= 0:
inputs=[neuron['output']forneuroninnetwork[i-1]] for
neuron in network[i]:
forjin range(len(inputs)):

15
 neuron['weights'][j]-=l_rate*neuron['delta']*inputs[j]
neuron['weights'][-1] -= l_rate * neuron['delta']
#Trainanetworkforafixednumberofepochs
deftrain_network(network,train,l_rate,n_epoch,n_outputs): for
epoch in range(n_epoch):
sum_error = 0
forrowintrain:
outputs=forward_propagate(network,row)
expected = [0 for i in range(n_outputs)]
expected[row[-1]] = 1
sum_error += sum([(expected[i]-outputs[i])**2 for i in
range(len(expected))])
backward_propagate_error(network,expected)
update_weights(network, row, l_rate)
print('>epoch=%d,lrate=%.3f,error=%.3f'%(epoch,l_rate,sum_error)) #
Test training backprop algorithm
seed(1)
dataset=[[2.7810836,2.550537003,0],
[1.465489372,2.362125076,0],
[3.396561688,4.400293529,0],
[1.38807019,1.850220317,0],
[3.06407232,3.005305973,0],
[7.627531214,2.759262235,1],
[5.332441248,2.088626775,1],
[6.922596716,1.77106367,1],
[8.675418651,-0.242068655,1],
[7.673756466,3.508563011,1]]
n_inputs=len(dataset[0])-1
n_outputs=len(set([row[-1]forrowindataset]))

16
OUTPUT:
>epoch=0,lrate=0.500,error=6.350
>epoch=1,lrate=0.500,error=5.531
>epoch=2,lrate=0.500,error=5.221
>epoch=3,lrate=0.500,error=4.951
>epoch=4,lrate=0.500,error=4.519
>epoch=5,lrate=0.500,error=4.173
>epoch=6,lrate=0.500,error=3.835
>epoch=7,lrate=0.500,error=3.506
>epoch=8,lrate=0.500,error=3.192
>epoch=9,lrate=0.500,error=2.898
>epoch=10,lrate=0.500,error=2.626
>epoch=11,lrate=0.500,error=2.377
>epoch=12,lrate=0.500,error=2.153
>epoch=13,lrate=0.500,error=1.953
>epoch=14,lrate=0.500,error=1.774
>epoch=15,lrate=0.500,error=1.614
>epoch=16,lrate=0.500,error=1.472
>epoch=17,lrate=0.500,error=1.346
>epoch=18,lrate=0.500,error=1.233
>epoch=19,lrate=0.500,error=1.132
[{'weights':[-1.4688375095432327,1.850887325439514,1.0858178629550297],'output':0.029
980305604426185,'delta':0.0059546604162323625},{'weights':[0.37711098142462157,-0.06
25909894552989,0.2765123702642716],'output':0.9456229000211323,'delta':-0.0026279652
850863837}]
[{'weights':[2.515394649397849,-0.3391927502445985, -0.9671565426390275],'output':0.23
648794202357587,'delta':0.04270059278364587},{'weights':[-2.5584149848484263,1.00364
22106209202,0.42383086467582715],'output':0.7790535202438367,'delta':-0.038031325964
37354}]

17
RESULT:
ThustheBackpropagationalgorithmtobuildanArtificialNeuralnetworkswas implemented successfully.

18
Expt. No : 4 IMPLEMENTATIONOFNAÏVEBAYESIANCLASSIFIERFORASA
MPLE TRAINING DATASET AND TO COMPUTE ACCURACY
Date :

AIM:

To implement Naïve Bayesian classifier for Tennis data set and to compute the accuracy
with few datasets.

ALGORITHM:

1. Convertthedatasetintoafrequencytable.
2. Create likelihood table by finding the probabilitieslike overcast probability = 0.29
andprobability of plating is 0.64.
3. Now,[Link] class
with the highest posterior probability is the outcome of prediction.

Problem:[Link]? We can

solve it using above discussed method of posterior probability.

P(Yes|Sunny) =P( Sunny |Yes)* P(Yes) /P(Sunny)

HerewehaveP(Sunny|Yes)=3/9=0.33, P(Sunny)=5/14=0.36,P(Yes)=9/14=0.64 Now, P

(Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, which has higher probability.
4. Exit.

PROGRAM:

import pandas as pd
fromsklearnimporttree
[Link] from
sklearn.naive_bayes import GaussianNB
data=pd.read_csv("E:\BALA\AI\Labprograms\pgms\[Link]")
print("The first 5 values of data is :\n",[Link]())

19
#obtainTraindataandTrainoutput X
= [Link][:,:-1]
print("\nTheFirst5valuesoftraindatais\n",[Link]())

y=[Link][:,-1]
print("\nThefirst5valuesofTrainoutputis\n",[Link]())

# Convert then in numbers

le_outlook=LabelEncoder()
[Link]=le_outlook.fit_transform([Link])
le_Temperature = LabelEncoder()
[Link]=le_Temperature.fit_transform([Link])
le_Humidity = LabelEncoder()
[Link]=le_Humidity.fit_transform([Link])
le_Windy = LabelEncoder()
[Link]=le_Windy.fit_transform([Link])
print("\nNow the Train data is :\n",[Link]())

20
le_PlayTennis=LabelEncoder()
y = le_PlayTennis.fit_transform(y)
print("\nNowtheTrainoutputis\n",y)

fromsklearn.model_selectionimporttrain_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.20)

classifier = GaussianNB()
[Link](X_train,y_train)

fromsklearn.metricsimportaccuracy_score
print("Accuracyis:",accuracy_score([Link](X_test),y_test))

21
OUTPUT:

Accuracyis:0.6666666666666666

22
RESULT:
Thus the program to implement Naïve Bayesian classifier to compute the accuracy with
few datasets using python was executed and verified successfully.

23
[Link] IMPLEMENTATIONOFNAÏVEBAYESIANCLASSIFIERMODELTOCLASSIFYA
SET OF DOCUMENTS AND TO MEASURE THE ACCURACY, PRECISION,
AND RECALL
Date :

AIM:

ToclassifyasetofdocumentsusingNaïveBayesianclassifierandtomeasuretheaccuracy
and precision

ALGORITHM:

1. Importbasiclibraries.
2. Importingthedataset.
3. Datapreprocessing.
4. Trainingthemodel.
5. Testingandevaluationofthemodel.
6. Visualizingthemodel.

PROGRAM:

fromsklearn.datasetsimportfetch_20newsgr

oups from [Link] import

confusion_matrix from [Link]

import classification_report import numpy

as np

categories = ['[Link]', '[Link]','[Link]', '[Link]']

twenty_train=fetch_20newsgroups(subset='train',categories=categories,shuffle=

True) twenty_test =

fetch_20newsgroups(subset='test',categories=categories,shuffle=True)

print(len(twenty_train.data))

print(len(twenty_test.data))

24
print(twenty_train.target_names)

print("\n".join(twenty_train.data[0].split("\n")))

print(twenty_train.target[0])

fromsklearn.feature_extraction.textimportCountVectori

zercount_vect = CountVectorizer()

X_train_tf =

count_vect.fit_transform(twenty_train.data)

fromsklearn.feature_extraction.textimportTfidfTransfo

rmertfidf_transformer = TfidfTransformer()

X_train_tfidf= tfidf_transformer.fit_transform(X_train_tf)

X_train_tfidf.shape

fromsklearn.naive_bayesimportMultinomialNB

fromsklearn.metricsimportaccuracy_

score from sklearn import metrics

mod = MultinomialNB()

[Link](X_train_tfidf,twenty_train.t

arget)

X_test_tf =

count_vect.transform(twenty_test.data)

X_test_tfidf=tfidf_transformer.transform(X_tes

t_tf) predicted = [Link](X_test_tfidf)

25
OUTPUT:

OUTPUT:

26
RESULT:

Thus the accuracy and precision was measured by Naïve Bayesian classifier model.

27
Expt. No : 6 CONSTRUCTION OF A BAYESIAN NETWORK TO
DIAGNOSE CORONA INFECTION USING STANDARD
WHO DATA SET
Date :
AIM:
To construct a Bayesian network to diagnose corona infection using WHO data set.

ALGORITHM:

ThisNaiveBayesisbrokendowninto5parts: 1:
Separate by Class.
2:SummarizeDataset.
3:SummarizeDataby Class.
4:GaussianProbabilityDensityFunction. 5:
Class Probabilities.
PROGRAM

importnumpyasnp
importpandasaspd
[Link]
import pandas as pd
[Link]
frompandasimportset_optionp
[Link]('ggplot')
from sklearn.model_selection import train_test_split
fromsklearn.linear_modelimportLogisticRegression
from sklearn.feature_selection import RFE
fromsklearn.model_selectionimportKFold
fromsklearn.model_selectionimportGridSearchCV

28
fromsklearn.model_selectionimportRandomizedSearchCV
from [Link] import StandardScaler
[Link]
[Link]
import xgboost as xgb
fromxgboostimportXGBClassifier
fromsklearn.naive_bayesimport GaussianNB
fromsklearn.model_selectionimportcross_val_score
from [Link] import confusion_matrix
[Link]
from [Link] import DecisionTreeClassifier
from [Link] import ExtraTreesClassifier
fromsklearn.feature_selectionimportSelectFromModel
from sklearn import metrics
importwarnings
[Link]("ignore",category=FutureWarning)
from [Link] import classification_report
covid_19_data=pd.read_csv("E:\BALA\AI\Labprograms\pgms\covid_19_data.csv")
print(f'The shape of the dataframe is {covid_19_data.shape}')
print()

print(covid_19_data.info())
print()

covid_19_data.replace(to_replace='?',value=[Link],inplace=True)
print(covid_19_data.describe(include='all'))
print()

importseabornassns
29
[Link](x='Country/Region',data=covid_19_data,linewidth=3)
[Link]()
covid_19_data[['ObservationDate', 'Province/State', 'Country/Region','LastUpdate','Confirmed',
'Deaths', 'Recovered']].hist(bins=50, figsize=(15,8))
[Link]()
covid_19_data['Country/Region'].fillna(covid_19_data['Country/Region'].mode()[0],
inplace=True)
covid_19_data['Confirmed'].fillna(covid_19_data['Confirmed'].mode()[0], inplace=True)
X = covid_19_data.drop(['Deaths'],axis=1)
y = covid_19_data.Recovered
X=X[['confirmed','Recovered']]
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2)
NB_classifier = GaussianNB()
NB_classifier.fit(X_train, y_train)
y_predict=NB_classifier.predict(X_test)

cm = confusion_matrix(y_test, y_predict)
[Link](cm, annot=True, cmap='Blues')
print(classification_report(y_test, y_predict))

30
31
OUTPUT:

32
RESULT:

Thus the program to diagnose corona infection using Bayesian network was successfully
implemented using python.

33
Expt. No : 1 COMPARISON OF CLUSTERING IN EM ALGORITHM AND
K-MEANS ALGORITHM USING THE SAME DATA SETS

Date :
AIM:
To compare the clustering in EM algorithm and K-means algorithm using the same data
Sets.
ALGORITHM:

The K-mean simple mentation is as follows:

1. Choosethenumberofclustersk.
2. Selectkrandompoints fromthedataas centroids.
3. Assignallthepointstotheclosestclustercentroid.
4. Recomputethe centroidsofnewlyformed clusters.
5. Repeatsteps 3and4.

TheEMimplementationisasfollows:

1. Expectationstep(E-step):Itinvolvestheestimation(guess)ofallmissingvaluesinthe dataset
so that after completing this step, there should not be any missing value.
2. Maximization step (M - step):This step involves the use of estimated data in the E-step
and updating the parameters.
3. Repeat E-step and M-step until the convert gence of the values occurs.

PROGRAM:
[Link]
from sklearn import preprocessing
[Link] from
[Link] import load_iris
[Link]
import pandas as pd
importnumpyasnp
[Link]

34
dataset=load_iris()
# print(dataset)

X=[Link]([Link])
[Link]=['Sepal_Length','Sepal_Width','Petal_Length','Petal_Width']
y=[Link]([Link])
[Link]=['Targets']
# print(X)

[Link](figsize=(14,7))
colormap=[Link](['red','lime','black'])

#REALPLOT
[Link](1,3,1)
[Link](X.Petal_Length,X.Petal_Width,c=colormap[[Link]],s=40)
[Link]('Real')

#K-PLOT
[Link](1,3,2)
model=KMeans(n_clusters=3)
[Link](X)
predY=[Link](model.labels_,[0,1,2]).astype(np.int64)
[Link](X.Petal_Length,X.Petal_Width,c=colormap[predY],s=40)
[Link]('KMeans')

#GMM PLOT
scaler=[Link]()
[Link](X)

35
OUTPUT:

36
RESULT:
Thus theprogram to compare clustering in EM and K-means algorithm with few datasets
was performed successfully.

37
Expt. No : 7 IMPLEMENTATION OF K –NEAREST NEIGHBOR ALGORITHM TO
CLASSIFY THE IT IS DATA SET
Date :
AIM:
To implement K-Nearest Neighbor algorithm to classify iris dataset.

ALGORITHM:

1. Calculate the Information Gain of each feature.

2. Considering that all rows don’t belong to the same class, split the dataset Sin to subsets
using the feature for which the Information Gain is maximum.

3. MakeadecisiontreenodeusingthefeaturewiththemaximumInformationgain.

4. Ifallrowsbelongtothesameclass,makethecurrentnodeasaleafnodewiththeclassasits label.

5. Repeatfortheremainingfeaturesuntilwerunoutofallfeatures,orthedecisiontreehasall leaf
nodes.

PROGRAM:
fromsklearn.datasetsimportload_iris
[Link]
fromsklearn.model_selection importtrain_test_split
import numpy as np
dataset=load_iris()
X_train,X_test,y_train,y_test=train_test_split(dataset["data"],dataset["target"],random_state=0)
kn=KNeighborsClassifier(n_neighbors=1)
[Link](X_train,y_train)
foriin range(len(X_test)):

38
 x=X_test[i]
x_new=[Link]([x])
prediction=[Link](x_new)
print("TARGET=",y_test[i],dataset["target_names"]
[y_test[i]],"PREDICTED=",prediction,dataset["target_names"][prediction])
print([Link](X_test,y_test))

OUTPUT:
TARGET=2virginicaPREDICTED=[2]['virginica']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=2virginicaPREDICTED=[2]['virginica']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=2virginicaPREDICTED=[2]['virginica']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=2virginica PREDICTED=[2]['virginica']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=2virginicaPREDICTED=[2]['virginica']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=2virginicaPREDICTED=[2]['virginica']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=1versicolorPREDICTED=[1]['versicolor']
TARGET=0setosaPREDICTED=[0]['setosa']
TARGET=2virginicaPREDICTED=[2]['virginica']
TARGET=1versicolorPREDICTED=[1]['versicolor']

39
RESULT:
ThustheprogramforK-NearestNeighbouralgorithmwasimplementedsuccessfullyusing an
iris data set.

40
Expt. No. :9 IMPLEMENTATION OF THE NON-PARAMETRICLOCALLY
WEIGHTED REGRESSION ALGORITHM IN ORDER TO FIT DATA
POINTS
Date :
AIM:
Toimplementthenon-parametricLocallyWeightedRegressionalgorithminordertofit
datapoints.
ALGORITHM:

1. ReadtheGivendataSampletoXandthecurve(linearornonlinear)toY

2. SetthevalueforSmootheningparameterorFreeparametersayτ

3. Setthebias /Pointof interestsetx0which isasubsetof X

4. Determinetheweightmatrixusing:

5. Determinethevalueofmodeltermparameterβusing:

6. Prediction= x0*β

PROGRAM:
from math import ceil
import numpy as np
fromscipyimportlinalg
deflowess(x,y,f,iterations): n
= len(x)
r=int(ceil(f*n))
h=[[Link]([Link](x -x[i]))[r] foriin range(n)]

41
 w=[Link]([Link]((x[:,None]-x[None,:])/h),0.0,1.0) w = (1
- w ** 3) ** 3
yest=[Link](n)
delta=[Link](n)
foriterationinrange(iterations):
for i in range(n):
weights=delta*w[:,i]
b=[Link]([[Link](weights *y),[Link](weights *y*x)])
A = [Link]([[[Link](weights), [Link](weights * x)],[[Link](weights * x),
[Link](weights * x * x)]])
beta=[Link](A,b)
yest[i]=beta[0]+beta[1]*x[i]
residuals = y - yest
s= [Link]([Link](residuals))
delta=[Link](residuals /(6.0*s),-1,1)
delta=(1-delta**2)**2
return yest
importmath
n = 100
x=[Link](0,2*[Link],n)
y=[Link](x)+0.3*[Link](n) f
=0.25
iterations=3
yest=lowess(x,y,f,iterations)
import [Link] as
[Link](x,y,"r.")
[Link](x,yest,"b-")

42
OUTPUT:

43
RESULT:
Thus the non-parametric Locally Weighted Regression algorithm to fit data
pointswasimplemented successfully.

44