21ECE305J-MACHINE LEARNING ALGORITHMS

LABORATORY MANUAL

offered by
DEPARTMENT OF
ELECTRONICS AND COMMUNICATION ENGINEERING
FACULTY OF ENGINEERING AND TECHNOLOGY
SRM INSTITUTE OF SCIENCE AND TECHNOLOGY
Ramapuram, Chennai 600089

Academic year: 2024 – 2025

Semester: Even

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 SRM Institute of Science and Technology, Ramapuram Campus
Faculty of Engineering& Technology
Department of ECE

21ECE305J-MACHINE LEARNING ALGORITHMS LAB

2024-2025 (Even Semester)

Department : ECE

Year/Semester : III/VI

Name of the Student :

Register Number :

Date of Submission :

Staff Name :
Signature :

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 21ECE305J-MACHINE LEARNING ALGORITHMS
LABORATORY

List of Experiments

[Link] Date of Name of the Experiment Marks Faculty

Experiment signature
1. Linear Regression
2. Support Vector Machines
3. Decision Trees
4. K-Means clustering
5. Bayes Network
6. Hierarchical Clustering
7. Logistic Regression
8. Voting Classifier
9. CNN
10. Mini Project

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 LINEAR REGRESSION
EXPERIMENT No: 1 Date:

Aim:To write a python program for linear regression using scikit learn module and diabetes data
and to predict the disease progression.

Software Required:
Google colab

Theory:
Simple Linear Regression allows us to study the relationship between two variables. In simple
linear regression a single independent variable is used to predict the value of a dependent
variable. i.e. One independent variable (X) and One Dependent variable (Y).
This can be denoted by

In school we became familiar with equations like the one shown below, then how it is different
from the equation above?

Well both are the same type of equation, we just changed the name of the variable and added
something extra, the ‘e’ to minimize the chance of error.
Where m was slope, and c was Intercept. In the first equation b0 is the intercept, and b1 is the
slope.
Slope direction
The slope of a line can be positive, negative, zero or undefined.
Positive slope: y increases as x increases, so the line slopes upwards to the right.
Negative slope: y decreases as x increases, so the line slopes downwards to the right. If you
remember from the previous examples, we have seen an example of this, where the Age of the
Car increases, the price decreases.
Zero slope: y does not change as x increases, so the line remains horizontal. The slope of any
horizontal line is always zero.
Undefined slope: When the line is exactly vertical, it does not have a defined slope. The two x
coordinates are the same, so the difference is zero.

Program:
import [Link] as plt
import numpy as np
from sklearn import datasets,linear_model
from [Link] import mean_squared_error,r2_score

#load the diabetes dataset

diabetes_X,diabetes_y=datasets.load_diabetes(return_X_y=True)
#use only one feature
diabetes_x=diabetes_X[:,[Link],2]

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

#split the data into training/testing sets
diabetes_x_train=diabetes_x[:-20]
diabetes_x_test=diabetes_x[-20:]
#split the targets into training/testing sets
diabetes_y_train=diabetes_y[:-20]
diabetes_y_test=diabetes_y[-20:]
#create linear regression object
regr = linear_model.LinearRegression()
#Train the model using the training sets
[Link](diabetes_x_train,diabetes_y_train)
#Make predictions using the testing set
diabetes_y_pred=[Link](diabetes_x_test)
#The coeficients
print("Coefficient:\n",regr.coef_)
#The mean Squared Error
print("Mean Square Error: %2f"%mean_squared_error(diabetes_y_test,diabetes_y_pred))
#The coeficient of determination: 1 is the perfect prediction
print("Coefficient of Determination: %2f"%r2_score(diabetes_y_test,diabetes_y_pred))
#plot outputs
[Link](diabetes_x_test,diabetes_y_test,color="black")
[Link](diabetes_x_test,diabetes_y_pred,color="blue",linewidth=3)
[Link](())
[Link](())
[Link]

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

Output:

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result:
Thus by using python program for linear regression using scikit learn module and diabetes data
the disease progression was predicted.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Support Vector Machines
EXPERIMENT No:2 Date:

Aim:To write a python program for support vector machine using scikit learn module

Software Required:
Google colab

Theory:

A Support Vector Machine (SVM) is a binary linear classification whose decision boundary is
explicitly constructed to minimize generalization error. It is a very powerful and versatile
Machine Learning model, capable of performing linear or nonlinear classification, regression and
even outlier [Link] is well suited for classification of complex but small or medium sized
datasets.

How does SVM classify?

It’s important to start with the intuition for SVM with the special linearly
separable classification [Link] classification of observations is “linearly separable”, SVM fits
the “decision boundary” that is defined by the largest margin between the closest points for each
class. This is commonly called the “maximum margin hyperplane (MMH)”.
If the functioning of SVM classifier is to be understood mathematically then it can be understood
in the following ways-

Step 1: SVM algorithm predicts the classes. One of the classes is identified as 1 while the other
is identified as -1.
Step 2: As all machine learning algorithms convert the business problem into a mathematical
equation involving unknowns. These unknowns are then found by converting the problem into an
optimization problem. As optimization problems always aim at maximizing or minimizing
something while looking and tweaking for the unknowns, in the case of the SVM classifier, a
loss function known as the hinge loss function is used and tweaked to find the maximum margin.
Step 3: For ease of understanding, this loss function can also be called a cost function whose
cost is 0 when no class is incorrectly predicted. However, if this is not the case, then error/loss is
calculated. The problem with the current scenario is that there is a trade-off between maximizing
margin and the loss generated if the margin is maximized to a very large extent. To bring these
concepts in theory, a regularization parameter is added.
Step 4: As is the case with most optimization problems, weights are optimized by calculating the
gradients using advanced mathematical concepts of calculus viz. partial derivatives.
Step 5: The gradients are updated only by using the regularization parameter when there is no
error in the classification while the loss function is also used when misclassification happens.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

Step 6: The gradients are updated only by using the regularization parameter when there is no
error in the classification, while the loss function is also used when misclassification happens.
Program:
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import SVC

from matplotlib import pyplot as plt

%matplotlib inline
url = '[Link]
Neighbors/master/Dataset/[Link]'
df = pd.read_csv(url) # Dataset - Breast Cancer Wisconsin Data
df['diagnosis'] = df['diagnosis'].map({
'M': 1,
'B': 2
}) # Label values - 1 for Malignant and 2 for Benign
labels = df['diagnosis'].tolist()
df['Class'] = labels #Cpying values of diagnosis to newly clreated labels column
df = [Link](['id', 'Unnamed: 32', 'diagnosis'],
axis=1) #Dropping unncessary columns
[Link]() #Displaying first five rows of the dataset
target_names = ['', 'M', 'B']
df['attack_type'] = [Link](lambda x: target_names[x])
[Link]()
df1 = df[[Link] == 1]
df2 = df[[Link] == 2]
[Link]('radius_mean')
[Link]('texture_mean')
[Link](df1['radius_mean'], df1['texture_mean'], color='green', marker='+')
[Link](df2['radius_mean'], df2['texture_mean'], color='blue', marker='.')
X = [Link](['Class', 'attack_type'], axis='columns')
[Link]()
y = [Link]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
print(len(X_train))
print(len(X_test))
model = SVC(kernel='linear')
[Link](X_train, y_train)
SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
decision_function_shape='ovr', degree=3, gamma='auto_deprecated',
kernel='linear', max_iter=-1, probability=False, random_state=None,
shrinking=True, tol=0.001, verbose=False)
predictions = [Link](X_test)
print(predictions)
percentage = [Link](X_test, y_test)
from [Link] import confusion_matrix

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

res = confusion_matrix(y_test, predictions)
print("Confusion Matrix")
print(res)
print(f"Test Set: {len(X_test)}")
print(f"Accuracy = {percentage*100} %")

Output:

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for support vector machine is simulated and verified using scikit learn
module .

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Decision Trees
EXPERIMENT No: 3 Date:

Aim: To write a python program for Decision Trees using scikit learn module

Software Required:
Google colab

Theory:
Decision Trees (DTs) are a non-parametric supervised learning method used
for classification and regression. The goal is to create a model that predicts the value of a
target variable by learning simple decision rules inferred from the data features. A tree can be
seen as a piecewise constant approximation.
Decision Tree Implementation
o Problem Analysis
o Step 1: Importing the libraries
o Step 2: Importing the dataset
o Step 3: Splitting the dataset into the Training set and Test set
o Step 4: Training the Decision Tree Classification model on the Training Set
o Step 5: Predicting the Test Set Results
o Step 6: Comparing the Real Values with Predicted Values
o Step 7: Confusion Matrix and Accuracy
o Step 8: Visualizing the Decision Tree Classifier

Program:
import pandas as pd
import pydotplus #pip install pydotplus
from [Link] import export_graphviz
from [Link] import DecisionTreeClassifier
import numpy as np
data = [Link]({'P_Movies': [17,64,18,20,38,49,55,25,29,31,33],
'Gender': [1,0,1,0,1,0,0,1,1,0,1]})
data =data.sort_values('P_Movies')
data
def tree_graph_to_png(tree, feature_names, png_file_to_save):
tree_str = export_graphviz(tree, feature_names=feature_names,
filled=True, out_file=None)
graph = pydotplus.graph_from_dot_data(tree_str)
graph.write_png(png_file_to_save)
#define Decision Tree
dt = DecisionTreeClassifier(criterion = 'entropy')
#Define input vectors

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

#X is the features in this dataset
X = data['P_Movies'].[Link](-1, 1)
#Y is the vector with our Target Variables
Y = data['Gender'].values
#start fitting process
[Link](X, Y)
tree_graph_to_png(dt, feature_names=['P_Movies'],
png_file_to_save='[Link]')
d = [Link]([7, 15, 43, 45])
d=[Link](-1, 1)
[Link](d)

Output:
array([1, 1, 1, 0])

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for Decision Trees is simulated and verified using scikit learn module

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 K means clustering
EXPERIMENT No:4 Date:

Aim:To write a python program for K-Means clustering using scikit learn module

Software Required:
Google colab

Theory:

K-Means Clustering is an Unsupervised Learning algorithm, which groups the unlabeled dataset
into different clusters. Here K defines the number of pre-defined clusters that need to be created
in the process, as if K=2, there will be two clusters, and for K=3, there will be three clusters, and
so on. It allows us to cluster the data into different groups and a convenient way to discover the
categories of groups in the unlabeled dataset on its own without the need for any training.

It is a centroid-based algorithm, where each cluster is associated with a centroid. The main aim
of this algorithm is to minimize the sum of distances between the data point and their
corresponding clusters.

The algorithm takes the unlabeled dataset as input, divides the dataset into k-number of clusters,
and repeats the process until it does not find the best clusters. The value of k should be
predetermined in this algorithm.

The k-means clustering algorithm mainly performs two tasks:

o Determines the best value for K center points or centroids by an iterative process.
o Assigns each data point to its closest k-center. Those data points which are near to the
particular k-center, create a cluster.

Hence each cluster has datapoints with some commonalities, and it is away from other clusters.

How does the K-Means Algorithm Work?

The working of the K-Means algorithm is explained in the below steps:
Step-1: Select the number K to decide the number of clusters.
Step-2: Select random K points or centroids. (It can be other from the input dataset).
Step-3: Assign each data point to their closest centroid, which will form the predefined K
clusters.
Step-4: Calculate the variance and place a new centroid of each cluster.
Step-5: Repeat the third steps, which means reassign each datapoint to the new closest centroid
of each cluster.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.
Step-7: The model is ready.

Program:
import [Link] as plt
from [Link] import make_blobs
# create dataset
X, y = make_blobs(
n_samples=150, n_features=2,
centers=3, cluster_std=0.5,
shuffle=True, random_state=0
)

# plot
[Link](
X[:, 0], X[:, 1],
c='white', marker='o',
edgecolor='black', s=50
)
[Link]()

from [Link] import KMeans

km = KMeans(
n_clusters=3, init='random',
n_init=10, max_iter=300,
tol=1e-04, random_state=0
)
y_km = km.fit_predict(X)

# plot the 3 clusters

[Link](
X[y_km == 0, 0], X[y_km == 0, 1],
s=50, c='lightgreen',
marker='s', edgecolor='black',
label='cluster 1'
)

[Link](
X[y_km == 1, 0], X[y_km == 1, 1],
s=50, c='orange',
marker='o', edgecolor='black',
label='cluster 2'
)
[Link](
X[y_km == 2, 0], X[y_km == 2, 1],

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 s=50, c='lightblue',
marker='v', edgecolor='black',
label='cluster 3'
)

# plot the centroids

[Link](
km.cluster_centers_[:, 0], km.cluster_centers_[:, 1],
s=250, marker='*',
c='red', edgecolor='black',
label='centroids'
)
[Link](scatterpoints=1)
[Link]()
[Link]()

# calculate distortion for a range of number of cluster

distortions = []
for i in range(1, 11):
km = KMeans(
n_clusters=i, init='random',
n_init=10, max_iter=300,
tol=1e-04, random_state=0
)
[Link](X)
[Link](km.inertia_)

# plot
[Link](range(1, 11), distortions, marker='o')
[Link]('Number of clusters')
[Link]('Distortion')
[Link]()

Output:

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual
Parameter Program Written Execution Viva
Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for K-Means clustering is simulated and verified using scikit learn
module

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Bayes Network
EXPERIMENT No: 5 Date:

Aim: To write a python program for Bayes Network using scikit learn module

Software Required:
Google colab

Theory:
Naive Bayes is a statistical classification technique based on Bayes Theorem. It is one of the
simplest supervised learning algorithms. Naive Bayes classifier is the fast, accurate and reliable
algorithm. Naive Bayes classifiers have high accuracy and speed on large datasets.

Naive Bayes classifier assumes that the effect of a particular feature in a class is independent of
other features. For example, a loan applicant is desirable or not depending on his/her income,
previous loan and transaction history, age, and location. Even if these features are
interdependent, these features are still considered independently. This assumption simplifies
computation, and that's why it is considered as naive. This assumption is called class conditional
independence.

 P(h): the probability of hypothesis h being true (regardless of the data). This is known as
the prior probability of h.

 P(D): the probability of the data (regardless of the hypothesis). This is known as the prior
probability.

 P(h|D): the probability of hypothesis h given the data D. This is known as posterior
probability.

 P(D|h): the probability of data d given that the hypothesis h was true. This is known as
posterior probability.

How Naive Bayes classifier works?

Let’s understand the working of Naive Bayes through an example. Given an example of weather
conditions and playing sports. You need to calculate the probability of playing sports. Now, you
need to classify whether players will play or not, based on the weather condition.

First Approach (In case of a single feature)

Naive Bayes classifier calculates the probability of an event in the following steps:

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

  Step 1: Calculate the prior probability for given class labels
 Step 2: Find Likelihood probability with each attribute for each class
 Step 3: Put these value in Bayes Formula and calculate posterior probability.
 Step 4: See which class has a higher probability, given the input belongs to the higher
probability class.

For simplifying prior and posterior probability calculation you can use the two tables frequency
and likelihood tables. Both of these tables will help you to calculate the prior and posterior
probability. The Frequency table contains the occurrence of labels for all features. There are two
likelihood tables. Likelihood Table 1 is showing prior probabilities of labels and Likelihood
Table 2 is showing the posterior probability.

Now suppose you want to calculate the probability of playing when the weather is overcast.

Probability of playing:

P(Yes | Overcast) = P(Overcast | Yes) P(Yes) / P (Overcast) .....................(1)

1. Calculate Prior Probabilities:
P(Overcast) = 4/14 = 0.29
P(Yes)= 9/14 = 0.64
1. Calculate Posterior Probabilities:
P(Overcast |Yes) = 4/9 = 0.44
1. Put Prior and Posterior probabilities in equation (1)
P (Yes | Overcast) = 0.44 * 0.64 / 0.29 = 0.98(Higher)

Similarly, you can calculate the probability of not playing:

Probability of not playing:

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

P(No | Overcast) = P(Overcast | No) P(No) / P (Overcast) .....................(2)
1. Calculate Prior Probabilities:
P(Overcast) = 4/14 = 0.29
P(No)= 5/14 = 0.36
1. Calculate Posterior Probabilities:
P(Overcast |No) = 0/9 = 0
1. Put Prior and Posterior probabilities in equation (2)
P (No | Overcast) = 0 * 0.36 / 0.29 = 0

The probability of a 'Yes' class is higher. So you can determine here if the weather is overcast
than players will play the sport.

Program

# Assigning features and label variables

weather=['Sunny','Sunny','Overcast','Rainy','Rainy','Rainy','Overcast','Sunny','Sunny',
'Rainy','Sunny','Overcast','Overcast','Rainy']

temp=['Hot','Hot','Hot','Mild','Cool','Cool','Cool','Mild','Cool','Mild','Mild','Mild','Hot','Mild']

play=['No','No','Yes','Yes','Yes','No','Yes','No','Yes','Yes','Yes','Yes','Yes','No']

# Import LabelEncoder
from sklearn import preprocessing
#creating labelEncoder
le = [Link]()
# Converting string labels into numbers.
wheather_encoded=le.fit_transform(weather)
print ("Whether:",wheather_encoded)

# Converting string labels into numbers

temp_encoded=le.fit_transform(temp)
label=le.fit_transform(play)
print ("Temp:",temp_encoded)
print ("Play:",label)

#Combinig weather and temp into single listof tuples

import numpy as np
features=list(zip(wheather_encoded,temp_encoded))
print ("\nCombine:",features)

from sklearn.naive_bayes import GaussianNB

#Create a Gaussian Classifier

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

model = GaussianNB()

# Train the model using the training sets

[Link](features,label)

#Predict Output
predicted= [Link]([[2,1]])
print ("\nPredicted Value:", predicted)

OUTPUT
Whether: [2 2 0 1 1 1 0 2 2 1 2 0 0 1]
Temp: [1 1 1 2 0 0 0 2 0 2 2 2 1 2]
Play: [0 0 1 1 1 0 1 0 1 1 1 1 1 0]

Combine: [(2, 1), (2, 1), (0, 1), (1, 2), (1, 0), (1, 0), (0, 0), (2, 2), (2, 0), (1, 2), (2, 2), (0, 2), (0, 1),
(1, 2)]

Predicted Value: [0]

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for Bayes Network is simulated and verified using scikit learn module

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Hierarchical Clustering
EXPERIMENT No: 6 Date:

Aim: To write a python program for Hierarchical Clustering using scikit-learn module

Software Required:
Google colab

Theory:
Hierarchical clustering is a connectivity-based clustering model that groups the data points
together that are close to each other based on the measure of similarity or distance. The
assumption is that data points that are close to each other are more similar or related than data
points that are farther apart.
A dendrogram, a tree-like figure produced by hierarchical clustering, depicts the hierarchical
relationships between groups. Individual data points are located at the bottom of the
dendrogram, while the largest clusters, which include all the data points, are located at the top.
In order to generate different numbers of clusters, the dendrogram can be sliced at various
heights.
The dendrogram is created by iteratively merging or splitting clusters based on a measure of
similarity or distance between data points. Clusters are divided or merged repeatedly until all
data points are contained within a single cluster, or until the predetermined number of clusters
is attained.
Hierarchical Agglomerative Clustering
It is also known as the bottom-up approach or hierarchical agglomerative clustering (HAC). A
structure that is more informative than the unstructured set of clusters returned by flat
clustering. This clustering algorithm does not require us to prespecify the number of clusters.
Bottom-up algorithms treat each data as a singleton cluster at the outset and then successively
agglomerate pairs of clusters until all clusters have been merged into a single cluster that
contains all data.
Program
from [Link] import AgglomerativeClustering
import numpy as np
# randomly chosen dataset
X = [Link]([[1, 2], [1, 4], [1, 0],
[4, 2], [4, 4], [4, 0]])

# here we need to mention the number of clusters

# otherwise the result will be a single cluster
# containing all the data
clustering = AgglomerativeClustering(n_clusters=2).fit(X)

# print the class labels

print(clustering.labels_)

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

Output
[1 1 1 0 0 0]

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for Hierarchical clustering is simulated and verified using scikit learn
module.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Logistic Regression
EXPERIMENT No: 7 Date:

Aim: To write a python program for Hierarchical Clustering using scikit-learn module

Software Required:
Google colab

Theory:

Logistic regression aims to solve classification problems. It does this by predicting categorical
outcomes, unlike linear regression that predicts a continuous outcome.

In the simplest case there are two outcomes, which is called binomial, an example of which is
predicting if a tumor is malignant or benign. Other cases have more than two outcomes to
classify, in this case it is called multinomial. A common example for multinomial logistic
regression would be predicting the class of an iris flower between 3 different [Link] we
will be using basic logistic regression to predict a binomial variable. This means it has only two
possible outcomes.

Program

import numpy
from sklearn import linear_model

#Reshaped for Logistic function.

X = [Link]([3.78, 2.44, 2.09, 0.14, 1.72, 1.65, 4.92, 4.37, 4.96, 4.52, 3.69, 5.88]).reshape(-1,1)
y = [Link]([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1])

logr = linear_model.LogisticRegression()
[Link](X,y)

#predict if tumor is cancerous where the size is 3.46mm:

predicted = [Link]([Link]([3.46]).reshape(-1,1))
print(predicted)

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

Output
[0]

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for Logistic Regression is simulated and verified using sklearn learn
module.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Voting Classifier
EXPERIMENT No: 8 Date:

Aim: To write a python program for Voting Classifier using scikit-learn module

Software Required:
Google colab

Theory:
A Voting Classifier is a machine learning model that trains on an ensemble of numerous
models and predicts an output (class) based on their highest probability of chosen class as the
output.
It simply aggregates the findings of each classifier passed into Voting Classifier and predicts
the output class based on the highest majority of voting. The idea is instead of creating
separate dedicated models and finding the accuracy for each them, we create a single model
which trains by these models and predicts output based on their combined majority of voting
for each output class.
Voting Classifier supports two types of votings.
1. Hard Voting: In hard voting, the predicted output class is a class with the highest
majority of votes i.e the class which had the highest probability of being predicted by each
of the classifiers. Suppose three classifiers predicted the output class(A, A, B), so here the
majority predicted A as output. Hence A will be the final prediction.
2. Soft Voting: In soft voting, the output class is the prediction based on the average of
probability given to that class. Suppose given some input to three models, the prediction
probability for class A = (0.30, 0.47, 0.53) and B = (0.20, 0.32, 0.40). So the average for
class A is 0.4333 and B is 0.3067, the winner is clearly class A because it had the highest
probability averaged by each classifier.

Program
# importing libraries
from [Link] import VotingClassifier
from sklearn.linear_model import LogisticRegression
from [Link] import SVC
from [Link] import DecisionTreeClassifier
from [Link] import load_iris
from [Link] import accuracy_score
from sklearn.model_selection import train_test_split

# loading iris dataset

iris = load_iris()
X = [Link][:, :4]
Y = [Link]

# train_test_split

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

X_train, X_test, y_train, y_test = train_test_split(X,
Y,
test_size = 0.20,
random_state = 42)

# group / ensemble of models

estimator = []
[Link](('LR',
LogisticRegression(solver ='lbfgs',
multi_class ='multinomial',
max_iter = 200)))
[Link](('SVC', SVC(gamma ='auto', probability = True)))
[Link](('DTC', DecisionTreeClassifier()))

# Voting Classifier with hard voting

vot_hard = VotingClassifier(estimators = estimator, voting ='hard')
vot_hard.fit(X_train, y_train)
y_pred = vot_hard.predict(X_test)

# using accuracy_score metric to predict accuracy

score = accuracy_score(y_test, y_pred)
print("Hard Voting Score % d" % score)

# Voting Classifier with soft voting

vot_soft = VotingClassifier(estimators = estimator, voting ='soft')
vot_soft.fit(X_train, y_train)
y_pred = vot_soft.predict(X_test)

# using accuracy_score
score = accuracy_score(y_test, y_pred)
print("Soft Voting Score % d" % score)

Output
Hard Voting Score 1
Soft Voting Score 1

Parameter Program Written Execution Viva

Maximum Marks 3 5 2
Obtained Marks

Result

Thus a python program for Voting Classifier is simulated and verified using sklearn learn
module.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

 Convolutional neural network
(Content Beyond Syllabus)
EXPERIMENT No: 9 Date:

Aim: To write a python program for Convolution neural network using scikit learn module

Software Required:
Google colab

Theory:
Convolutional neural network

A neural network in which at least one layer is a convolutional layer. A typical convolutional
neural network consists of some combination of the following layers:

 convolutional layers
 pooling layers
 dense layers

Convolutional neural networks have had great success in certain kinds of problems, such as
image recognition.

The CIFAR10 dataset contains 60,000 color images in 10 classes, with 6,000 images in each
class. The dataset is divided into 50,000 training images and 10,000 testing images. The classes
are mutually exclusive and there is no overlap between them.

Verify the data

To verify that the dataset looks correct, let's plot the first 25 images from the training set and
display the class name below each image:
Create the convolutional base
The 6 lines of code below define the convolutional base using a common pattern: a stack
of Conv2D and MaxPooling2D layers.
As input, a CNN takes tensors of shape (image_height, image_width, color_channels), ignoring
the batch size. If you are new to these dimensions, color_channels refers to (R,G,B). In this
example, you will configure your CNN to process inputs of shape (32, 32, 3), which is the format
of CIFAR images. You can do this by passing the argument input_shape to your first layer.
Add Dense layers on top
To complete the model, you will feed the last output tensor from the convolutional base (of
shape (4, 4, 64)) into one or more Dense layers to perform classification. Dense layers take
vectors as input (which are 1D), while the current output is a 3D tensor. First, you will flatten (or
unroll) the 3D output to 1D, then add one or more Dense layers on top. CIFAR has 10 output
classes, so you use a final Dense layer with 10 outputs.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

The network summary shows that (4, 4, 64) outputs were flattened into vectors of shape (1024)
before going through two Dense layers.
Compile and train the model
Evaluate the model
Program
import tensorflow as tf

from [Link] import datasets, layers, models

import [Link] as plt
(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()

# Normalize pixel values to be between 0 and 1

train_images, test_images = train_images / 255.0, test_images / 255.0
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
'dog', 'frog', 'horse', 'ship', 'truck']

[Link](figsize=(10,10))
for i in range(25):
[Link](5,5,i+1)
[Link]([])
[Link]([])
[Link](False)
[Link](train_images[i])
# The CIFAR labels happen to be arrays,
# which is why you need the extra index
[Link](class_names[train_labels[i][0]])
[Link]()
model = [Link]()
[Link](layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
[Link](layers.MaxPooling2D((2, 2)))
[Link](layers.Conv2D(64, (3, 3), activation='relu'))
[Link](layers.MaxPooling2D((2, 2)))
[Link](layers.Conv2D(64, (3, 3), activation='relu'))
[Link]([Link]())
[Link]([Link](64, activation='relu'))
[Link]([Link](10))
[Link]()
[Link](optimizer='adam',
loss=[Link](from_logits=True),
metrics=['accuracy'])

history = [Link](train_images, train_labels, epochs=10,

validation_data=(test_images, test_labels))
[Link]([Link]['accuracy'], label='accuracy')
[Link]([Link]['val_accuracy'], label = 'val_accuracy')

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

[Link]('Epoch')
[Link]('Accuracy')
[Link]([0.5, 1])
[Link](loc='lower right')

test_loss, test_acc = [Link](test_images, test_labels, verbose=2)

print(test_acc)

Output:

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual

Parameter Program Written Execution Viva
Maximum Marks 3 5 2
Obtained Marks

Result:
Thus a python program for Convolution neural network is simulated and verified using scikit
learn module.

21ECE305J- Machine Learning Algorithms /SRMIST-Ramapuram Lab manual