MACHINE

MACHINE LEARNING WITH PYTHON

1. Implement and demonstrate the FIND-S algorithm for finding the most specific
hypothesis based on a given set of training data samples. Read the training data from a .CSV
file.

FIND-S Algorithm
1. Initialize h to the most specific hypothesis in H
2. For each positive training instance x
For each attribute constraint ai in h

If the constraint ai is satisfied by x

Then do nothing
Else replace ai in h by the next more general constraint that is satisfied by x
3. Output hypothesis h

Training Examples:

Example Sky AirTemp Humidity Wind Water Forecast EnjoySport

1 Sunny Warm Normal Strong Warm Same Yes

2 Sunny Warm High Strong Warm Same Yes

3 Rainy Cold High Strong Warm Change No

4 Sunny Warm High Strong Cool Change Yes

1 Tirumala Engineering college

MACHINE
MACHINE LEARNING WITH PYTHON

Program:

import csv a =

[]

with open('[Link]', 'r') as csvfile: for row in

[Link](csvfile):
[Link](row)
print(a)

print("\n The total number of training instances are : ",len(a)) num_attribute = len(a[0])-1

print("\n The initial hypothesis is : ") hypothesis =

['0']*num_attribute print(hypothesis)

for i in range(0, len(a)):

if a[i][num_attribute] == 'yes':
for j in range(0, num_attribute):
if hypothesis[j] == '0' or hypothesis[j] == a[i][j]: hypothesis[j] = a[i][j]
else:
hypothesis[j] = '?'
print("\n The hypothesis for the training instance {} is :
\n" .format(i+1),hypothesis)

print("\n The Maximally specific hypothesis for the training instance is ")
print(hypothesis)

2 Tirumala Engineering college

 Machine learning with python

Data Set:

sunny warm normal strong warm same yes

sunny warm high strong warm same yes
rainy cold high strong warm change no
sunny warm high strong cool change yes

Output:

The Given Training Data Set

['sunny', 'warm', 'normal', 'strong', 'warm', 'same', 'yes']

['sunny', 'warm', 'high', 'strong', 'warm', 'same', 'yes']
['rainy', 'cold', 'high', 'strong', 'warm', 'change', 'no']
['sunny', 'warm', 'high', 'strong', 'cool', 'change', 'yes'] The total number of training

instances are : 4

The initial hypothesis is :

['0', '0', '0', '0', '0', '0']

The hypothesis for the training instance 1 is : ['sunny', 'warm', 'normal', 'strong',
'warm', 'same']

The hypothesis for the training instance 2 is : ['sunny', 'warm', '?', 'strong', 'warm',
'same']

The hypothesis for the training instance 3 is : ['sunny', 'warm', '?', 'strong', 'warm',
'same']

The hypothesis for the training instance 4 is : ['sunny', 'warm', '?',

'strong', '?', '?']

The Maximally specific hypothesis for the training instance is

['sunny', 'warm', '?', 'strong', '?', '?']

1 Tirumala Engineering College

 Machine learning with python

2. For a given set of training data examples stored in a .CSV file, implement and demonstrate
the Candidate-Elimination algorithm to output a description of the set of all hypotheses
consistent with the training examples.

CANDIDATE-ELIMINATION Learning Algorithm

The CANDIDATE-ELIMINTION algorithm computes the version space containing all

hypotheses from H that are consistent with an observed sequence of training examples.

Initialize G to the set of maximally general hypotheses in H

Initialize S to the set of maximally specific hypotheses in H
For each training example d, do
• If d is a positive example
• Remove from G any hypothesis inconsistent with d
• For each hypothesis s in S that is not consistent with d
• Remove s from S
• Add to S all minimal generalizations h of s such that
• h is consistent with d, and some member of G is more general than h
• Remove from S any hypothesis that is more general than another hypothesis in S

• If d is a negative example
• Remove from S any hypothesis inconsistent with d
• For each hypothesis g in G that is not consistent with d
• Remove g from G
• Add to G all minimal specializations h of g such that
• h is consistent with d, and some member of S is more specific than h
• Remove from G any hypothesis that is less general than another hypothesis in G

CANDIDATE- ELIMINTION algorithm using version spaces

Training Examples:

Example Sky AirTemp Humidity Wind Water Forecast EnjoySport

1 Sunny Warm Normal Strong Warm Same Yes

2 Sunny Warm High Strong Warm Same Yes
3 Rainy Cold High Strong Warm Change No
4 Sunny Warm High Strong Cool Change Yes

2 Tirumala Engineering College

 Machine learning with python

Program:

import numpy as np import

pandas as pd
data = [Link](data=pd.read_csv('[Link]')) concepts =
[Link]([Link][:,0:-1])
print(concepts)
target = [Link]([Link][:,-1]) print(target)

def learn(concepts, target): specific_h =

concepts[0].copy()
print("initialization of specific_h and general_h") print(specific_h)
general_h = [["?" for i in range(len(specific_h))] for i in range(len(specific_h))]
print(general_h)
for i, h in enumerate(concepts): if target[i] ==
"yes":
for x in range(len(specific_h)): if h[x]!=
specific_h[x]:
specific_h[x] ='?' general_h[x][x]
='?'
print(specific_h)
print(specific_h)
if target[i] == "no":
for x in range(len(specific_h)): if h[x]!=
specific_h[x]:
general_h[x][x] = specific_h[x] else:
general_h[x][x] = '?'
print(" steps of Candidate Elimination Algorithm",i+1) print(specific_h)
print(general_h)
indices = [i for i, val in enumerate(general_h) if val == ['?', '?', '?', '?', '?', '?']]
for i in indices:
general_h.remove(['?', '?', '?', '?', '?', '?']) return specific_h, general_h
s_final, g_final = learn(concepts, target) print("Final Specific_h:",
s_final, sep="\n") print("Final General_h:", g_final, sep="\n")

3 Tirumala Engineering College

 Machine Learning with python

Data Set:

Sky AirTemp Humidity Wind Water Forecast EnjoySport

sunny warm normal strong warm same yes

sunny warm high strong warm same yes
rainy cold high strong warm change no
sunny warm high strong cool change yes

Output:

Final Specific_h:
['sunny' 'warm' '?' 'strong' '?' '?']

Final General_h:
[['sunny', '?', '?', '?', '?', '?'],
['?', 'warm', '?', '?', '?', '?']]

Tirumala Engineering college

 Machine Learning with python

3. Write a program to demonstrate the working of the decision tree based ID3 algorithm. Use
an appropriate data set for building the decision tree and apply this knowledge to classify a
new sample.

ID3 Algorithm

ID3(Examples, Target_attribute, Attributes)

Examples are the training examples. Target_attribute is the attribute whose value is to
be predicted by the tree. Attributes is a list of other attributes that may be tested by the
learned decision tree. Returns a decision tree that correctly classifies the given
Examples.

4. Create a Root node for the tree

5. If all Examples are positive, Return the single-node tree Root, with label = +
6. If all Examples are negative, Return the single-node tree Root, with label = -
7. If Attributes is empty, Return the single-node tree Root, with label = most common value
of Target_attribute in Examples

8. Otherwise Begin
• A ← the attribute from Attributes that best* classifies Examples
• The decision attribute for Root ← A
• For each possible value, vi, of A,
• Add a new tree branch below Root, corresponding to the test A = vi
• Let Examples vi, be the subset of Examples that have value vi for A
• If Examples vi , is empty
• Then below this new branch add a leaf node with label = most common
value of Target_attribute in Examples
• Else below this new branch add the subtree
ID3(Examples vi, Targe_tattribute, Attributes –
{A}))
9. End
10. Return Root

[Link] best attribute is the one with highest information gain

Tirumala Engineering college

 Machine Learning with python

ENTROPY:
Entropy measures the impurity of a collection of examples.

Where, p+ is the proportion of positive examples in S

p- is the proportion of negative examples in S.

INFORMATION GAIN:

• Information gain, is the expected reduction in entropy caused by partitioning the

examples according to this attribute.
• The information gain, Gain(S, A) of an attribute A, relative to a collection of examples
S, is defined as

Training Dataset:

Day Outlook Temperature Humidity Wind PlayTennis

D1 Sunny Hot High Weak No
D2 Sunny Hot High Strong No
D3 Overcast Hot High Weak Yes
D4 Rain Mild High Weak Yes
D5 Rain Cool Normal Weak Yes
D6 Rain Cool Normal Strong No
D7 Overcast Cool Normal Strong Yes
D8 Sunny Mild High Weak No
D9 Sunny Cool Normal Weak Yes
D10 Rain Mild Normal Weak Yes
D11 Sunny Mild Normal Strong Yes
D12 Overcast Mild High Strong Yes
D13 Overcast Hot Normal Weak Yes
D14 Rain Mild High Strong No

Test Dataset:

Day Outlook Temperature Humidity Wind

T1 Rain Cool Normal Strong
T2 Sunny Mild Normal Strong

Tirumala Engineering college

 Machine Learning with python

Program:

import math
import csv

def load_csv(filename):
lines=[Link](open(filename,"r"));
dataset = list(lines) headers =
[Link](0) return
dataset,headers

class Node:
def init (self,attribute): [Link]=attribute
[Link]=[] [Link]=""

def subtables(data,col,delete):
dic={}
coldata=[row[col] for row in data]
attr=list(set(coldata))

counts=[0]*len(attr)
r=len(data) c=len(data[0])
for x in range(len(attr)): for y in
range(r):
if data[y][col]==attr[x]: counts[x]
+=1

for x in range(len(attr)):
dic[attr[x]]=[[0 for i in range(c)] for j in range(counts[x])]
pos=0
for y in range(r):
if data[y][col]==attr[x]: if delete:
del data[y][col] dic[attr[x]]
[pos]=data[y] pos+=1
return attr,dic

Tirumala Engineering college

 Machine Learning with python

def entropy(S):
attr=list(set(S))
if len(attr)==1: return 0

counts=[0,0]
for i in range(2):
counts[i]=sum([1 for x in S if attr[i]==x])/(len(S)*1.0)

sums=0
for cnt in counts:
sums+=-1*cnt*[Link](cnt,2) return sums

def compute_gain(data,col):
attr,dic = subtables(data,col,delete=False)

total_size=len(data)
entropies=[0]*len(attr)
ratio=[0]*len(attr)

total_entropy=entropy([row[-1] for row in data]) for x in

range(len(attr)):
ratio[x]=len(dic[attr[x]])/(total_size*1.0)
entropies[x]=entropy([row[-1] for row in
dic[attr[x]]])
total_entropy-=ratio[x]*entropies[x] return
total_entropy

def build_tree(data,features):
lastcol=[row[-1] for row in data] if(len(set(lastcol)))==1:
node=Node("")
[Link]=lastcol[0] return
node

n=len(data[0])-1
gains=[0]*n
for col in range(n): gains[col]=compute_gain(data,col)
split=[Link](max(gains))
node=Node(features[split])
fea = features[:split]+features[split+1:] attr,dic=subtables(data,split,delete=True)

Tirumala Engineering college

 Machine Learning with python

for x in range(len(attr)): child=build_tree(dic[attr[x]],fea)

[Link]((attr[x],child))
return node

def print_tree(node,level):
if [Link]!="":
print(" "*level,[Link]) return

print(" "*level,[Link]) for value,n

in [Link]:
print(" "*(level+1),value)
print_tree(n,level+2)

def classify(node,x_test,features):
if [Link]!="":
print([Link]) return
pos=[Link]([Link]) for value, n in
[Link]:
if x_test[pos]==value:
classify(n,x_test,features)

'''Main program'''
dataset,features=load_csv("[Link]") node1=build_tree(dataset,features)

print("The decision tree for the dataset using ID3 algorithm is")
print_tree(node1,0) testdata,features=load_csv("data3_test.csv") for
xtest in testdata:
print("The test instance:",xtest)
print("The label for test instance:",end=" ")
classify(node1,xtest,features)

Tirumala Engineering college

 Machine Learning with python

Output:

The decision tree for the dataset using ID3 algorithm is

Outlook
rain
Wind
strong
no
weak
yes
overcast
yes

Tirumala Engineering college

 Machine Learning with python

sunny
Humidity
normal
yes
high
no

The test instance: ['rain', 'cool', 'normal', 'strong']

The label for test instance: no

The test instance: ['sunny', 'mild', 'normal', 'strong']

The label for test instance: yes

1 TIRUMALA ENGINEERING COLLEGE

 Machine Learning with python

[Link] an Artificial Neural Network by implementing the Backpropagation algorithm and

test the same using appropriate data sets.

BACKPROPAGATION Algorithm

BACKPROPAGATION (training_example, ƞ, nin, nout, nhidden )

Each training example is a pair of the form (⃗𝑥 𝑡 ), where (𝑥) is the vector of network
,
input values, (𝑡 ) and is the vector of target network output values.
ƞ is the learning rate (e.g., .05). ni, is the number of network inputs, nhidden the number of
units in the hidden layer, and nout the number of output units.
The input from unit i into unit j is denoted xji, and the weight from unit i to unit j is
denoted wji

• Create a feed-forward network with ni inputs, nhidden hidden units, and nout output
units.
• Initialize all network weights to small random numbers
• Until the termination condition is met, Do

• For each (⃗𝑥

𝑡 ), in training examples, Do
,

1. Input the instance ⃗𝑥 , to the network and compute the output ou of every
Propagate the input forward through the network:

unit u in the network.

Propagate the errors backward through the network:

2 TIRUMALA ENGINEERING COLLEGE

 Machine Learning with python

Training Examples:

Expected % in
Example Sleep Study
Exams
1 2 9 92
2 1 5 86
3 3 6 89

Normalize the input

Expected %
Example Sleep Study
in Exams
1 2/3 = 0.66666667 9/9 = 1 0.92
2 1/3 = 0.33333333 5/9 = 0.55555556 0.86
3 3/3 = 1 6/9 = 0.66666667 0.89

Program:

import numpy as np
X = [Link](([2, 9], [1, 5], [3, 6]), dtype=float)
y = [Link](([92], [86], [89]), dtype=float)
X = X/[Link](X,axis=0) # maximum of X array longitudinally y = y/100

#Sigmoid Function def

sigmoid (x):
return 1/(1 + [Link](-x))

#Derivative of Sigmoid Function def

derivatives_sigmoid(x):
return x * (1 - x)

#Variable initialization
epoch=5000 #Setting training iterations lr=0.1
#Setting learning rate
inputlayer_neurons = 2 #number of features in data set
hiddenlayer_neurons = 3 #number of hidden layers neurons output_neurons
=1 #number of neurons at output layer

3 TIRUMALA ENGINEERING COLLEGE

 Machine Learning with python

#weight and bias initialization

wh=[Link](size=(inputlayer_neurons,hiddenlayer_neur ons))
bh=[Link](size=(1,hiddenlayer_neurons))
wout=[Link](size=(hiddenlayer_neurons,output_neuron s))
bout=[Link](size=(1,output_neurons))

#draws a random range of numbers uniformly of dim x*y for i in range(epoch):

#Forward Propogation
hinp1=[Link](X,wh)
hinp=hinp1 + bh
hlayer_act = sigmoid(hinp)
outinp1=[Link](hlayer_act,wout) outinp= outinp1+
bout
output = sigmoid(outinp)

#Backpropagation EO =
y-output
outgrad = derivatives_sigmoid(output) d_output = EO*
outgrad
EH = d_output.dot(wout.T)

#how much hidden layer wts contributed to error hiddengrad =

derivatives_sigmoid(hlayer_act) d_hiddenlayer = EH *
hiddengrad

# dotproduct of nextlayererror and currentlayerop wout +=

hlayer_act.[Link](d_output) *lr
wh += [Link](d_hiddenlayer) *lr

print("Input: \n" + str(X)) print("Actual Output: \n" +

str(y)) print("Predicted Output: \n" ,output)

4 TIRUMALA ENGINEERING COLLEGE

 Machine Learning with python

Output:

Input:
[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]

Actual Output:
[[0.92]
[0.86]
[0.89]]

Predicted Output:
[[0.89726759]
[0.87196896]
[0.9000671]]

1 Tirumala engineering college

 Machine Learning with python

[Link] a program to implement k-Nearest Neighbour algorithm to classify the iris data set.
Print both correct and wrong predictions. Java/Python ML library classes can be used for this
problem.

K-Nearest Neighbor Algorithm

Training algorithm:
• For each training example (x, f (x)), add the example to the list training
examples Classification algorithm:
• Given a query instance xq to be classified,
• Let x1 . . .xk denote the k instances from training examples that are nearest to xq
• Return

• Where, f(xi) function to calculate the mean value of the k nearest training
examples.

Data Set:

Iris Plants Dataset: Dataset contains 150 instances (50 in each of three classes)
Number of Attributes: 4 numeric, predictive attributes and the Class

2 Tirumala engineering college

 Machine Learning with python

Program:

from sklearn.model_selection import train_test_split from [Link]

import KNeighborsClassifier
from [Link] import classification_report, confusion_matrix from sklearn import datasets

""" Iris Plants Dataset, dataset contains 150 (50 in each of three
classes)Number of Attributes: 4 numeric, predictive attributes and
the Class
"""
iris=datasets.load_iris()

""" The x variable contains the first four columns of the dataset
(i.e. attributes) while y contains the labels.
"""
x = [Link] y =
[Link]

print ('sepal-length', 'sepal-width', 'petal-length', 'petal-width') print(x)

print('class: 0-Iris-Setosa, 1- Iris-Versicolour, 2- Iris-Virginica') print(y)

""" Splits the dataset into 70% train data and 30% test data. This
means that out of total 150 records, the training set will contain
105 records and the test set contains 45 of those records
"""
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3)

#To Training the model and Nearest nighbors K=5

classifier = KNeighborsClassifier(n_neighbors=5) [Link](x_train, y_train)

#to make predictions on our test data

y_pred=[Link](x_test)

""" For evaluating an algorithm, confusion matrix, precision, recall

and f1 score are the most commonly used metrics.
"""
print('Confusion Matrix') print(confusion_matrix(y_test,y_pred))
print('Accuracy Metrics') print(classification_report(y_test,y_pred))

3 Tirumala engineering college

 Machine Learning with python

Output:

sepal-length sepal-width petal-length petal-width

[[5.1 3.5 1.4 0.2]
[4.9 3. 1.4 0.2]
[4.7 3.2 1.3 0.2]
[4.6 3.1 1.5 0.2]
[5. 3.6 1.4 0.2]
. . . . .
. . . . .

[6.2 3.4 5.4 2.3]

[5.9 3. 5.1 1.8]]

class: 0-Iris-Setosa, 1- Iris-Versicolour, 2- Iris-Virginica

[0 0 0 ………0 0 1 1 1 …………1 1 2 2 2 ………… 2 2]

Confusion Matrix
[[20 0 0]
[ 0 10 0]
[ 0 1 14]]

Accuracy Metrics

Precision recall f1-score support

0 1.00 1.00 1.00 20

1 0.91 1.00 0.95 10
2 1.00 0.93 0.97 15

avg / total 0.98 0.98 0.98 45

4 Tirumala engineering college

 Machine Learning with python

Basic knowledge
Confusion Matrix

True positives: data points labelled as positive that are actually positive
False positives: data points labelled as positive that are actually negative
True negatives: data points labelled as negative that are actually negative
False negatives: data points labelled as negative that are actually positive

Accuracy: how often is the classifier correct?

F1-Score:

Support: Total Predicted of Class.

Support = TP + FN

5 Tirumala engineering college

 Machine Learning with python

Example:

• Support _ A = TP_A + FN_A

= 30 + (20 + 10)
= 60

6 Tirumala engineering college

 Machine Learning with python

Experiment-4: Exercise to solve the real world problems using the following machine
learning methods:(a) Linear Regression (b) Logistic Regression
(c) Binary classifier
Regression
The term regression is used when you try to find the relationship between variables.

In Machine Learning, and in statistical modeling, that relationship is used to predict the outcome of future events.

Linear Regression
Linear regression uses the relationship between the data-points to draw a straight line through all them.

This line can be used to predict future values.

In Machine Learning, predicting the future is very important.

How Does it Work?

Python has methods for finding a relationship between data-points and to draw a line of linear regression. We
will show you how to use these methods instead of going through the mathematic formula.

In the example below, the x-axis represents age, and the y-axis represents speed. We have registered the age and
speed of 13 cars as they were passing a tollbooth. Let us see if the data we collected could be used in a linear
regression:

Example

7 Tirumala engineering college

 Machine Learning with python

import sys

import matplotlib

[Link]('Agg')

import [Link] as plt

x = [5,7,8,7,2,17,2,9,4,11,12,9,6]

y = [99,86,87,88,111,86,103,87,94,78,77,85,86]

[Link](x, y)

[Link]()

[Link]([Link])

[Link]()

Result:

Logistic Regression:

8 Tirumala engineering college

 Machine Learning with python

Logistic regression aims to solve classification problems. It does this by predicting categorical outcomes, unlike
linear regression that predicts a continuous outcome.

In the simplest case there are two outcomes, which is called binomial, an example of which is predicting if a
tumor is malignant or benign. Other cases have more than two outcomes to classify, in this case it is called
multinomial. A common example for multinomial logistic regression would be predicting the class of an iris
flower between 3 different species.

Here we will be using basic logistic regression to predict a binomial variable. This means it has only two possible
outcomes.

How does it work?

In Python we have modules that will do the work for us. Start by importing the NumPy module.

import numpy

Store the independent variables in X.

Store the dependent variable in y.

Below is a sample dataset:

#X represents the size of a tumor in centimeters.

X=
[Link]([3.78, 2.44, 2.09, 0.14, 1.72, 1.65, 4.92, 4.37, 4.96, 4.52, 3.69, 5.88]).reshape(-
1,1)

#Note: X has to be reshaped into a column from a row for the LogisticRegression() function
to work.
#y represents whether or not the tumor is cancerous (0 for "No", 1 for "Yes").
y = [Link]([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1])

We will use a method from the sklearn module, so we will have to import that module as
well:

from sklearn import linear_model

9 Tirumala engineering college

 Machine Learning with python

From the sklearn module we will use the LogisticRegression() method to create a
logistic regression object.

This object has a method called fit() that takes the independent and dependent values as
parameters and fills the regression object with data that describes the relationship:

logr = linear_model.LogisticRegression()
[Link](X,y)

Now we have a logistic regression object that is ready to whether a tumor is cancerous
based on the tumor size:

#predict if tumor is cancerous where the size is 3.46mm:

predicted = [Link]([Link]([3.46]).reshape(-1,1))

Program:

import numpy

from sklearn import linear_model

X = [Link]([3.78, 2.44, 2.09, 0.14, 1.72, 1.65, 4.92, 4.37, 4.96, 4.52, 3.69, 5.88]).reshape(-1,1)

y = [Link]([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1])

logr = linear_model.LogisticRegression()

[Link](X,y)

1 Tirumala engineering college

0
 Machine Learning with python

#predict if tumor is cancerous where the size is 3.46mm:

predicted = [Link]([Link]([3.46]).reshape(-1,1))

print(predicted)

Result:
[[4.03541657]]

1 Tirumala engineering college

1
 Machine Learning with python

(c) Binary classifier

What is a Binary Classifier?

Let’s consider a scenario where you are told to seperate a basket full of Apples and
Oranges into two seperate baskets.

So, what do you do?

 You might look at the color

 You might look at the shape or the dimensions
 You might feel the difference in the texture
 You might feel the difference in the weights

Afer you find the difference between the two, then you’ll seperate them.

Now, let’s explain the Binary Classifier from the above scenario.

1. Firstly, you get the data to solve your problem. (Basket full of Apples and
Oranges)
2. Secondly, you create a feature set, which uniquely defines each data. (Your
assumptions like color, size, weights and etc.)
3. Thirdly, you are able to label or categorize each data. (Apple or Orange)
4. Fourthly, you have learnt to differentiate the data during the entire process. (In
future, you’ll be able to differentiate between an Apple and a Orange)

1 Tirumala engineering college

2
 Machine Learning with python

A Classifier in Machine Learning is an algorithm, that will determine the class to

which the input data belongs to based on a set of features.

Types of problems in Machine Learning:

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

A Binary Classifier is an instance of Supervised Learning. In Supervised Learning

we have a set of input data and a set of labels, our task is to map each data with a
label. A Binary Classifier classifies elements into two groups, either Zero or One.

Machine Learning Model

1. Data Preprocessing
2. Learning
3. Evaluation
4. Prediction

1. Data Preprocessing

1 Tirumala engineering college

3
 Machine Learning with python

As Machine Learning algorithms learn from the data, we are obliged to feed them
the right kind of data. So, the step towards achieving that is via Data
Preprocessing.

Data Preprocessing is a data mining technique that involves transforming the raw
data into an understandable format. Real-world data is often incomplete, noisy,
inconsistent or unreliable and above all it might be unstructured.

In simple terms, Data Preprocessing implies grooming the raw data according to
your requirement using certain techniques.

Steps involved in Data Preprocessing:

1. Data Cleaning — Fill in the missing values, detect and remove noisy data and
outliers.
2. Data Transformation — Normalize data to reduce dimensions and noise.
3. Data Reduction — Sample data records or attributes for easier data handling.
4. Data Discretization — Convert continuous attributes to categorical attributes
for ease of use with certain machine learning methods.
5. Text Cleaning — Remove embedded characters which may cause data
misalignment, for e.g., embedded tabs in a tab-separated data file, embedded
new lines which may break records, etc.

2. Learning

Once you have your dataset after preprocessing, then it’s time to select a learning
algorithm to perform your desired task. In our case it’s Binary Classifier or a
Perceptron.

Parameters to consider, while choosing a learning algorithm:

1 Tirumala engineering college

4
 Machine Learning with python

1. Accuracy
2. Training Time
3. Linearity
4. Number of Parameters

3. Evaluation

The metrics that you choose to evaluate the machine learning algorithm are very
important. The choice of metrics influences how the performance of machine
learning is measured and compared.

Classification Metrics

1. Classification Accuracy
2. Logarithmic Loss
3. Area Under ROC Curve
4. Confusion Matrix
5. Classification Report

Regression Metrics

1. Mean Absolute Error

2. Mean Squared Error
3. R-Squared

Implementing the Perceptron

A Perceptron is an algorithm for learning a binary classifier: a function that maps

it’s input x to an output value f(x)

Algorithm

1 Tirumala engineering college

5
 Machine Learning with python

Where,

1. w is a vector of real-value weights

2. w.x is a dot product
3. b is the bias

The value of f(x) is either 0 or 1, which is used to classify x as either a positive or a

negative instance.

Implementation

Let’s implement the perceptron to predict the outcome of an OR gate.

1. Let’s initialize an array with initial weights equal to 0. The length of the array is
equal to number of features + 1. The additional feature is the “threshold”.

self.weight_matrix = [Link](1 + [Link][1])

2. The loop “iterates” multiple times over the training data to optimize the weights
of the dataset.

for _ in range(number_of_iterations):

1 Tirumala engineering college

6
 Machine Learning with python

3. We loop over each training data point and it’s target. The target is the desired
output which we want the algorithm to predict. As it’s a binary classifier, the
targeted ouput is either a 0 or 1.

The prediction calculation is a matrix multiplication of the features with the

appropirate weights. To this multiplication we add the “threshold” value.

If the resulting value is above 0, then the predicted category is 1.

If the resulting value is below 0, the the predicted category is 0.

At each iteration, if the prediction is not accurate, the algorithm will adjust the
weights. The adjustment of the weights will be done proportionally to the
difference between the target and predicted value.

The difference is then mulitplied by the learning rate (rate). Higher the value of rate,
larger the correction of weights. The algorithm will stop to adjust the weights when
the predicted value becomes accurate.

Program:

import numpy as np

class Perceptron(object):
""" Perceptron Classifier

Parameters
------------
rate : float
Learning rate (ranging from 0.0 to 1.0)
number_of_iteration : int
Number of iterations over the input dataset.

Attributes:
------------

1 Tirumala engineering college

7
 Machine Learning with python

weight_matrix : 1d-array
Weights after fitting.

error_matrix : list
Number of misclassification in every epoch(one full training cycle on the training set)

"""

def __init__(self, rate = 0.01, number_of_iterations = 100):

[Link] = rate
self.number_of_iterations = number_of_iterations

def fit(self, X, y):

""" Fit training data

Parameters:
------------
X : array-like, shape = [number_of_samples, number_of_features]
Training vectors.
y : array-like, shape = [number_of_samples]
Target values.

Returns
------------
self : object

"""

self.weight_matrix = [Link](1 + [Link][1])

self.errors_list = []

for _ in range(self.number_of_iterations):
errors = 0
for xi, target in zip(X, y):
update = [Link] * (target - [Link](xi))
self.weight_matrix[1:] += update * xi
self.weight_matrix[0] += update
errors += int(update != 0.0)
self.errors_list.append(errors)
return self

def dot_product(self, X):

""" Calculate the dot product """
return ([Link](X, self.weight_matrix[1:]) + self.weight_matrix[0])

def predict(self, X):

""" Predicting the label for the input data """
return [Link](self.dot_product(X) >= 0.0, 1, 0)

1 Tirumala engineering college

8
 Machine Learning with python

if __name__ == '__main__':
X = [Link]([[0, 0, 0], [0, 0, 1], [0, 1, 0], [0, 1, 1], [1, 0, 0], [1, 0, 1], [1, 1, 0]])
y = [Link]([0, 1, 1, 1, 1, 1, 1])
p = Perceptron()
[Link](X, y)
print("Predicting the output of [1, 1, 1] = {}".format([Link]([1, 1, 1])))

Experiment -6:

Write a program to implement Categorical Encoding ,one-hot Encoding

Categorical encoding using Label-Encoding and One-Hot-Encoder

In many Machine-learning or Data Science activities, the data set might contain text or categorical

values (basically non-numerical values). For example, color feature having values like red, orange, blue,

white etc. Meal plan having values like breakfast, lunch, snacks, dinner, tea etc. Few algorithms such as

CATBOAST, decision-trees can handle categorical values very well but most of the algorithms expect

numerical values to achieve state-of-the-art results.

Over your learning curve in AI and Machine Learning, one thing you would notice that most of the

algorithms work better with numerical inputs. Therefore, the main challenge faced by an analyst is to

convert text/categorical data into numerical data and still make an algorithm/model to make sense out of

it. Neural networks, which is a base of deep-learning, expects input values to be numerical.

There are many ways to convert categorical values into numerical values. Each approach has its own

trade-offs and impact on the feature set. Hereby, I would focus on 2 main methods: One-Hot-Encoding

1 Tirumala engineering college

9
 Machine Learning with python

and Label-Encoder. Both of these encoders are part of SciKit-learn library (one of the most widely used

Python library) and are used to convert text or categorical data into numerical data which the model

expects and perform better with.

Code snippets in this article would be of Python since I am more comfortable with Python. If you need

for R (another widely used Machine-Learning language) then say so in comments.

Label Encoding

This approach is very simple and it involves converting each value in a column to a number. Consider a

dataset of bridges having a column names bridge-types having below values. Though there will be many

more columns in the dataset, to understand label-encoding, we will focus on one categorical column

only.

BRIDGE-TYPE
Arch
Beam
Truss
Cantilever
Tied Arch
Suspension
Cable

We choose to encode the text values by putting a running sequence for each text values like below:

2 Tirumala engineering college

0
 Machine Learning with python

With this, we completed the label-encoding of variable bridge-type. That’s all label encoding is about.

But depending upon the data values and type of data, label encoding induces a new problem since it uses

number sequencing. The problem using the number is that they introduce relation/comparison between

them. Apparently, there is no relation between various bridge type, but when looking at the number, one

might think that ‘Cable’ bridge type has higher precedence over ‘Arch’ bridge type. The algorithm might

misunderstand that data has some kind of hierarchy/order 0 < 1 < 2 … < 6 and might give 6X more

weight to ‘Cable’ in calculation then than ‘Arch’ bridge type.

Let’s consider another column named ‘Safety Level’. Performing label encoding of this column also

induces order/precedence in number, but in the right way. Here the numerical order does not look out-of-

box and it makes sense if the algorithm interprets safety order 0 < 1 < 2 < 3 < 4 i.e. none < low <

medium < high < very high.

2 Tirumala engineering college

1
 Machine Learning with python

Label Encoding in Python

Using category codes approach:

This approach requires the category column to be of ‘category’ datatype. By default, a non-numerical

column is of ‘object’ type. So you might have to change type to ‘category’ before using this approach.

# import required libraries

import pandas as pd
import numpy as np# creating initial dataframe
bridge_types = ('Arch','Beam','Truss','Cantilever','Tied Arch','Suspension','Cable')
bridge_df = [Link](bridge_types, columns=['Bridge_Types'])# converting type of columns to
'category'
bridge_df['Bridge_Types'] = bridge_df['Bridge_Types'].astype('category')# Assigning numerical values and
storing in another column
bridge_df['Bridge_Types_Cat'] = bridge_df['Bridge_Types'].[Link]
bridge_df

Using sci-kit learn library approach:

Another common approach which many data analyst perform label-encoding is by using SciKit learn

library.

import pandas as pd
import numpy as np
from [Link] import LabelEncoder# creating initial dataframe
bridge_types = ('Arch','Beam','Truss','Cantilever','Tied Arch','Suspension','Cable')
bridge_df = [Link](bridge_types, columns=['Bridge_Types'])# creating instance of labelencoder
labelencoder = LabelEncoder()# Assigning numerical values and storing in another column
bridge_df['Bridge_Types_Cat'] = labelencoder.fit_transform(bridge_df['Bridge_Types'])
bridge_df

2 Tirumala engineering college

2
 Machine Learning with python

bridge_df with categorical caolumn and label-encoded column values

One-Hot Encoder

Though label encoding is straight but it has the disadvantage that the numeric values can be

misinterpreted by algorithms as having some sort of hierarchy/order in them. This ordering issue is

addressed in another common alternative approach called ‘One-Hot Encoding’. In this strategy, each

category value is converted into a new column and assigned a 1 or 0 (notation for true/false) value to the

column. Let’s consider the previous example of bridge type and safety levels with one-hot encoding.

Above are the one-hot encoded values of categorical column ‘Bridge-Type’. In the same way, let's check

for ‘Safety-Level’ column.

2 Tirumala engineering college

3
 Machine Learning with python

Rows which have the first column value (Arch/None) will have ‘1’ (indicating true) and other value’s

columns will have ‘0’ (indicating false). Similarly, for other rows matching value with column value.

Though this approach eliminates the hierarchy/order issues but does have the downside of adding more

columns to the data set. It can cause the number of columns to expand greatly if you have many unique

values in a category column. In the above example, it was manageable, but it will get really challenging

to manage when encoding gives many columns.

One-Hot Encoding in Python

Using sci-kit learn library approach:

OneHotEncoder from SciKit library only takes numerical categorical values, hence any value of string

type should be label encoded before one hot encoded. So taking the dataframe from the previous

example, we will apply OneHotEncoder on column Bridge_Types_Cat.

import pandas as pd
import numpy as np
from [Link] import OneHotEncoder# creating instance of one-hot-encoder
enc = OneHotEncoder(handle_unknown='ignore')# passing bridge-types-cat column (label encoded values of
bridge_types)
enc_df = [Link](enc.fit_transform(bridge_df[['Bridge_Types_Cat']]).toarray())# merge with main df
bridge_df on key values
bridge_df = bridge_df.join(enc_df)
bridge_df

2 Tirumala engineering college

4
 Machine Learning with python

Bridge_Type column encoded using SciKit OneHotEncoder

Columns ‘Bridge_Types_Cat’ can be dropped from the dataframe.

Using dummies values approach:

This approach is more flexible because it allows encoding as many category columns as you would like

and choose how to label the columns using a prefix. Proper naming will make the rest of the analysis just

a little bit easier.

import pandas as pd
import numpy as np# creating initial dataframe
bridge_types = ('Arch','Beam','Truss','Cantilever','Tied Arch','Suspension','Cable')
bridge_df = [Link](bridge_types, columns=['Bridge_Types'])# generate binary values using
get_dummies
dum_df = pd.get_dummies(bridge_df, columns=["Bridge_Types"], prefix=["Type_is"] )# merge with main
df bridge_df on key values
bridge_df = bridge_df.join(dum_df)
bridge_df

Bridge_Type values encoded using dummies approach

2 Tirumala engineering college

5
 Machine Learning with python

Experiment - 9. Implement the non-parametric Locally Weighted Regression algorithm in Python in

order to fit data points. Select the appropriate data set for your experiment and draw graphs.

Locally Weighted Regression Algorithm

Regression:

 Regression is a technique from statistics that are used to predict values of the desired target
quantity when the target quantity is continuous.
 In regression, we seek to identify (or estimate) a continuous variable y associated with a given
input vector x.
 y is called the dependent variable.
 x is called the independent variable.

2 Tirumala engineering college

6
 Machine Learning with python

Loess/Lowess Regression:

Loess regression is a nonparametric technique that uses local weighted regression to fit a smooth curve
through points in a scatter plot.

Lowess Algorithm:

Locally weighted regression is a very powerful nonparametric model used in statistical learning.

Given a dataset X, y, we attempt to find a model parameter β(x) that minimizes residual sum of
weighted squared errors.

The weights are given by a kernel function (k or w) which can be chosen arbitrarily

Algorithm

1. Read the Given data Sample to X and the curve (linear or non linear) to Y

2. Set the value for Smoothening parameter or Free parameter say τ

2 Tirumala engineering college

7
 Machine Learning with python

3. Set the bias /Point of interest set x0 which is a subset of X

4. Determine the weight matrix using :

5. Determine the value of model term parameter β using:

6. Prediction = x0*β

Python Program to Implement and Demonstrate Locally Weighted Regression Algorithm

import [Link] as plt

import pandas as pd

import numpy as np

def kernel(point, xmat, k):

m,n = [Link](xmat)

weights = [Link]([Link]((m)))

for j in range(m):

diff = point - X[j]

weights[j,j] = [Link](diff*diff.T/(-2.0*k**2))

return weights

def localWeight(point, xmat, ymat, k):

wei = kernel(point,xmat,k)

W = (X.T*(wei*X)).I*(X.T*(wei*ymat.T))

2 Tirumala engineering college

8
 Machine Learning with python

return W

def localWeightRegression(xmat, ymat, k):

m,n = [Link](xmat)

ypred = [Link](m)

for i in range(m):

ypred[i] = xmat[i]*localWeight(xmat[i],xmat,ymat,k)

return ypred

# load data points

data = pd.read_csv('[Link]')

bill = [Link](data.total_bill)

tip = [Link]([Link])

#preparing and add 1 in bill

mbill = [Link](bill)

mtip = [Link](tip)

m= [Link](mbill)[1]

one = [Link]([Link](m))

X = [Link]((one.T,mbill.T))

#set k here

2 Tirumala engineering college

9
 Machine Learning with python

ypred = localWeightRegression(X,mtip,0.5)

SortIndex = X[:,1].argsort(0)

xsort = X[SortIndex][:,0]

fig = [Link]()

ax = fig.add_subplot(1,1,1)

[Link](bill,tip, color='green')

[Link](xsort[:,1],ypred[SortIndex], color = 'red', linewidth=5)

[Link]('Total bill')

[Link]('Tip')

[Link]();

Output

Data set:

total_bill tip sex smoker day time size

16.99 1.01 Female No Sun Dinner 2
10.34 1.66 Male No Sun Dinner 3
21.01 3.5 Male No Sun Dinner 3
23.68 3.31 Male No Sun Dinner 2
24.59 3.61 Female No Sun Dinner 4

3 Tirumala engineering college

0
 Machine Learning with python

25.29 4.71 Male No Sun Dinner 4

8.77 2 Male No Sun Dinner 2
26.88 3.12 Male No Sun Dinner 4
15.04 1.96 Male No Sun Dinner 2
14.78 3.23 Male No Sun Dinner 2
10.27 1.71 Male No Sun Dinner 2
35.26 5 Female No Sun Dinner 4
15.42 1.57 Male No Sun Dinner 2
18.43 3 Male No Sun Dinner 4
14.83 3.02 Female No Sun Dinner 2
21.58 3.92 Male No Sun Dinner 2
10.33 1.67 Female No Sun Dinner 3
16.29 3.71 Male No Sun Dinner 3
16.97 3.5 Female No Sun Dinner 3
20.65 3.35 Male No Sat Dinner 3
17.92 4.08 Male No Sat Dinner 2
20.29 2.75 Female No Sat Dinner 2
15.77 2.23 Female No Sat Dinner 2
39.42 7.58 Male No Sat Dinner 4
19.82 3.18 Male No Sat Dinner 2
17.81 2.34 Male No Sat Dinner 4
13.37 2 Male No Sat Dinner 2
12.69 2 Male No Sat Dinner 2
21.7 4.3 Male No Sat Dinner 2
19.65 3 Female No Sat Dinner 2
9.55 1.45 Male No Sat Dinner 2
18.35 2.5 Male No Sat Dinner 4
15.06 3 Female No Sat Dinner 2
20.69 2.45 Female No Sat Dinner 4
17.78 3.27 Male No Sat Dinner 2
24.06 3.6 Male No Sat Dinner 3
16.31 2 Male No Sat Dinner 3
16.93 3.07 Female No Sat Dinner 3
18.69 2.31 Male No Sat Dinner 3
31.27 5 Male No Sat Dinner 3
16.04 2.24 Male No Sat Dinner 3
17.46 2.54 Male No Sun Dinner 2
13.94 3.06 Male No Sun Dinner 2
9.68 1.32 Male No Sun Dinner 2
30.4 5.6 Male No Sun Dinner 4
18.29 3 Male No Sun Dinner 2
22.23 5 Male No Sun Dinner 2
32.4 6 Male No Sun Dinner 4
28.55 2.05 Male No Sun Dinner 3
18.04 3 Male No Sun Dinner 2

3 Tirumala engineering college

1
 Machine Learning with python

12.54 2.5 Male No Sun Dinner 2

10.29 2.6 Female No Sun Dinner 2
34.81 5.2 Female No Sun Dinner 4
9.94 1.56 Male No Sun Dinner 2
25.56 4.34 Male No Sun Dinner 4
19.49 3.51 Male No Sun Dinner 2
38.01 3 Male Yes Sat Dinner 4
26.41 1.5 Female No Sat Dinner 2
11.24 1.76 Male Yes Sat Dinner 2
48.27 6.73 Male No Sat Dinner 4
20.29 3.21 Male Yes Sat Dinner 2
13.81 2 Male Yes Sat Dinner 2
11.02 1.98 Male Yes Sat Dinner 2
18.29 3.76 Male Yes Sat Dinner 4
17.59 2.64 Male No Sat Dinner 3
20.08 3.15 Male No Sat Dinner 3
16.45 2.47 Female No Sat Dinner 2
3.07 1 Female Yes Sat Dinner 1
20.23 2.01 Male No Sat Dinner 2
15.01 2.09 Male Yes Sat Dinner 2
12.02 1.97 Male No Sat Dinner 2
17.07 3 Female No Sat Dinner 3
26.86 3.14 Female Yes Sat Dinner 2
25.28 5 Female Yes Sat Dinner 2
14.73 2.2 Female No Sat Dinner 2
10.51 1.25 Male No Sat Dinner 2
17.92 3.08 Male Yes Sat Dinner 2
27.2 4 Male No Thur Lunch 4
22.76 3 Male No Thur Lunch 2
17.29 2.71 Male No Thur Lunch 2
19.44 3 Male Yes Thur Lunch 2
16.66 3.4 Male No Thur Lunch 2
10.07 1.83 Female No Thur Lunch 1
32.68 5 Male Yes Thur Lunch 2
15.98 2.03 Male No Thur Lunch 2
34.83 5.17 Female No Thur Lunch 4
13.03 2 Male No Thur Lunch 2
18.28 4 Male No Thur Lunch 2
24.71 5.85 Male No Thur Lunch 2
21.16 3 Male No Thur Lunch 2
28.97 3 Male Yes Fri Dinner 2
22.49 3.5 Male No Fri Dinner 2
5.75 1 Female Yes Fri Dinner 2
16.32 4.3 Female Yes Fri Dinner 2
22.75 3.25 Female No Fri Dinner 2

3 Tirumala engineering college

2
 Machine Learning with python

40.17 4.73 Male Yes Fri Dinner 4

27.28 4 Male Yes Fri Dinner 2
12.03 1.5 Male Yes Fri Dinner 2
21.01 3 Male Yes Fri Dinner 2
12.46 1.5 Male No Fri Dinner 2
11.35 2.5 Female Yes Fri Dinner 2
15.38 3 Female Yes Fri Dinner 2
44.3 2.5 Female Yes Sat Dinner 3
22.42 3.48 Female Yes Sat Dinner 2
20.92 4.08 Female No Sat Dinner 2
15.36 1.64 Male Yes Sat Dinner 2
20.49 4.06 Male Yes Sat Dinner 2
25.21 4.29 Male Yes Sat Dinner 2
18.24 3.76 Male No Sat Dinner 2
14.31 4 Female Yes Sat Dinner 2
14 3 Male No Sat Dinner 2
7.25 1 Female No Sat Dinner 1
38.07 4 Male No Sun Dinner 3
23.95 2.55 Male No Sun Dinner 2
25.71 4 Female No Sun Dinner 3
17.31 3.5 Female No Sun Dinner 2
29.93 5.07 Male No Sun Dinner 4
10.65 1.5 Female No Thur Lunch 2
12.43 1.8 Female No Thur Lunch 2
24.08 2.92 Female No Thur Lunch 4
11.69 2.31 Male No Thur Lunch 2
13.42 1.68 Female No Thur Lunch 2
14.26 2.5 Male No Thur Lunch 2
15.95 2 Male No Thur Lunch 2
12.48 2.52 Female No Thur Lunch 2
29.8 4.2 Female No Thur Lunch 6
8.52 1.48 Male No Thur Lunch 2
14.52 2 Female No Thur Lunch 2
11.38 2 Female No Thur Lunch 2
22.82 2.18 Male No Thur Lunch 3
19.08 1.5 Male No Thur Lunch 2
20.27 2.83 Female No Thur Lunch 2
11.17 1.5 Female No Thur Lunch 2
12.26 2 Female No Thur Lunch 2
18.26 3.25 Female No Thur Lunch 2
8.51 1.25 Female No Thur Lunch 2
10.33 2 Female No Thur Lunch 2
14.15 2 Female No Thur Lunch 2
16 2 Male Yes Thur Lunch 2
13.16 2.75 Female No Thur Lunch 2

3 Tirumala engineering college

3
 Machine Learning with python

17.47 3.5 Female No Thur Lunch 2

34.3 6.7 Male No Thur Lunch 6
41.19 5 Male No Thur Lunch 5
27.05 5 Female No Thur Lunch 6
16.43 2.3 Female No Thur Lunch 2
8.35 1.5 Female No Thur Lunch 2
18.64 1.36 Female No Thur Lunch 3
11.87 1.63 Female No Thur Lunch 2
9.78 1.73 Male No Thur Lunch 2
7.51 2 Male No Thur Lunch 2
14.07 2.5 Male No Sun Dinner 2
13.13 2 Male No Sun Dinner 2
17.26 2.74 Male No Sun Dinner 3
24.55 2 Male No Sun Dinner 4
19.77 2 Male No Sun Dinner 4
29.85 5.14 Female No Sun Dinner 5
48.17 5 Male No Sun Dinner 6
25 3.75 Female No Sun Dinner 4
13.39 2.61 Female No Sun Dinner 2
16.49 2 Male No Sun Dinner 4
21.5 3.5 Male No Sun Dinner 4
12.66 2.5 Male No Sun Dinner 2
16.21 2 Female No Sun Dinner 3
13.81 2 Male No Sun Dinner 2
17.51 3 Female Yes Sun Dinner 2
24.52 3.48 Male No Sun Dinner 3
20.76 2.24 Male No Sun Dinner 2
31.71 4.5 Male No Sun Dinner 4
10.59 1.61 Female Yes Sat Dinner 2
10.63 2 Female Yes Sat Dinner 2
50.81 10 Male Yes Sat Dinner 3
15.81 3.16 Male Yes Sat Dinner 2
7.25 5.15 Male Yes Sun Dinner 2
31.85 3.18 Male Yes Sun Dinner 2
16.82 4 Male Yes Sun Dinner 2
32.9 3.11 Male Yes Sun Dinner 2
17.89 2 Male Yes Sun Dinner 2
14.48 2 Male Yes Sun Dinner 2
9.6 4 Female Yes Sun Dinner 2
34.63 3.55 Male Yes Sun Dinner 2
34.65 3.68 Male Yes Sun Dinner 4
23.33 5.65 Male Yes Sun Dinner 2
45.35 3.5 Male Yes Sun Dinner 3
23.17 6.5 Male Yes Sun Dinner 4
40.55 3 Male Yes Sun Dinner 2

3 Tirumala engineering college

4
 Machine Learning with python

20.69 5 Male No Sun Dinner 5

20.9 3.5 Female Yes Sun Dinner 3
30.46 2 Male Yes Sun Dinner 5
18.15 3.5 Female Yes Sun Dinner 3
23.1 4 Male Yes Sun Dinner 3
15.69 1.5 Male Yes Sun Dinner 2
19.81 4.19 Female Yes Thur Lunch 2
28.44 2.56 Male Yes Thur Lunch 2
15.48 2.02 Male Yes Thur Lunch 2
16.58 4 Male Yes Thur Lunch 2
7.56 1.44 Male No Thur Lunch 2
10.34 2 Male Yes Thur Lunch 2
43.11 5 Female Yes Thur Lunch 4
13 2 Female Yes Thur Lunch 2
13.51 2 Male Yes Thur Lunch 2
18.71 4 Male Yes Thur Lunch 3
12.74 2.01 Female Yes Thur Lunch 2
13 2 Female Yes Thur Lunch 2
16.4 2.5 Female Yes Thur Lunch 2
20.53 4 Male Yes Thur Lunch 4
16.47 3.23 Female Yes Thur Lunch 3
26.59 3.41 Male Yes Sat Dinner 3
38.73 3 Male Yes Sat Dinner 4
24.27 2.03 Male Yes Sat Dinner 2
12.76 2.23 Female Yes Sat Dinner 2
30.06 2 Male Yes Sat Dinner 3
25.89 5.16 Male Yes Sat Dinner 4
48.33 9 Male No Sat Dinner 4
13.27 2.5 Female Yes Sat Dinner 2
28.17 6.5 Female Yes Sat Dinner 3
12.9 1.1 Female Yes Sat Dinner 2
28.15 3 Male Yes Sat Dinner 5
11.59 1.5 Male Yes Sat Dinner 2
7.74 1.44 Male Yes Sat Dinner 2
30.14 3.09 Female Yes Sat Dinner 4
12.16 2.2 Male Yes Fri Lunch 2
13.42 3.48 Female Yes Fri Lunch 2
8.58 1.92 Male Yes Fri Lunch 1
15.98 3 Female No Fri Lunch 3
13.42 1.58 Male Yes Fri Lunch 2
16.27 2.5 Female Yes Fri Lunch 2
10.09 2 Female Yes Fri Lunch 2
20.45 3 Male No Sat Dinner 4
13.28 2.72 Male No Sat Dinner 2
22.12 2.88 Female Yes Sat Dinner 2

3 Tirumala engineering college

5
 Machine Learning with python

24.01 2 Male Yes Sat Dinner 4

15.69 3 Male Yes Sat Dinner 3
11.61 3.39 Male No Sat Dinner 2
10.77 1.47 Male No Sat Dinner 2
15.53 3 Male Yes Sat Dinner 2
10.07 1.25 Male No Sat Dinner 2
12.6 1 Male Yes Sat Dinner 2
32.83 1.17 Male Yes Sat Dinner 2
35.83 4.67 Female No Sat Dinner 3
29.03 5.92 Male No Sat Dinner 3
27.18 2 Female Yes Sat Dinner 2
22.67 2 Male Yes Sat Dinner 2
17.82 1.75 Male No Sat Dinner 2
18.78 3 Female No Thur Dinner 2

Experiment-10 : Assuming a set of documents that need to be classified, use the naïve Bayesian
Classifier model to perform this task. Built-in Java classes/API can be used to write the program.
Calculate the accuracy, precision, and recall for your data set.

Bayes’ Theorem is stated as:

Where,

P(h|D) is the probability of hypothesis h given the data D. This is called the posterior probability

P(D|h) is the probability of data d given that the hypothesis h was true.

3 Tirumala engineering college

6
 Machine Learning with python

P(h) is the probability of hypothesis h being true. This is called the prior probability of h. P(D) is the
probability of the data. This is called the prior probability of D

See also 18CS76 Machine Learning Laboratory VTU ML Lab

After calculating the posterior probability for a number of different hypotheses h, and is interested in
finding the most probable hypothesis h ∈ H given the observed data D. Any such maximally probable
hypothesis is called a maximum a posteriori (MAP) hypothesis.

Bayes theorem to calculate the posterior probability of each candidate hypothesis is hMAP is a MAP
hypothesis provided.

(Ignoring P(D) since it is a constant)

CLASSIFY_NAIVE_BAYES_TEXT (Doc)

Return the estimated target value for the document Doc. ai denotes the word found in the ith position
within Doc.

 positions ← all word positions in Doc that contain tokens found in Vocabulary
 Return VNB, where

Data set:
Save dataset in .csv format

Text Documents Label

1 I love this sandwich pos

2 This is an amazing place pos

3 Tirumala engineering college

7
 Machine Learning with python

3 I feel very good about these beers pos

4 This is my best work pos

5 What an awesome view pos

6 I do not like this restaurant neg

7 I am tired of this stuff neg

8 I can’t deal with this neg

9 He is my sworn enemy neg

10 My boss is horrible neg

11 This is an awesome place pos

12 I do not like the taste of this juice neg

13 I love to dance pos

14 I am sick and tired of this place neg

15 What a great holiday pos

16 That is a bad locality to stay neg

17 We will have good fun tomorrow pos

18 I went to my enemy’s house today neg

Python Program to Implement and Demonstrate Naïve Bayesian Classifier using API for
document classification

"""

6. Assuming a set of documents that need to be classified, use the naïve Bayesian Classifier model to
perform this task.

Built-in Java classes/API can be used to write the program. Calculate the accuracy, precision, and recall
for your data set

"

3 Tirumala engineering college

8
 Machine Learning with python

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.feature_extraction.text import CountVectorizer

from sklearn.naive_bayes import MultinomialNB

from sklearn import metrics

msg=pd.read_csv('[Link]',names=['message','label'])

print('The dimensions of the dataset',[Link])

msg['labelnum']=[Link]({'pos':1,'neg':0})

X=[Link]

y=[Link]

#splitting the dataset into train and test data

xtrain,xtest,ytrain,ytest=train_test_split(X,y)

print ('\n the total number of Training Data :',[Link])

print ('\n the total number of Test Data :',[Link])

#output the words or Tokens in the text documents

cv = CountVectorizer()

xtrain_dtm = cv.fit_transform(xtrain)

3 Tirumala engineering college

9
 Machine Learning with python

xtest_dtm=[Link](xtest)

print('\n The words or Tokens in the text documents \n')

print(cv.get_feature_names())

df=[Link](xtrain_dtm.toarray(),columns=cv.get_feature_names())

# Training Naive Bayes (NB) classifier on training data.

clf = MultinomialNB().fit(xtrain_dtm,ytrain)

predicted = [Link](xtest_dtm)

#printing accuracy, Confusion matrix, Precision and Recall

print('\n Accuracy of the classifier is',metrics.accuracy_score(ytest,predicted))

print('\n Confusion matrix')

print(metrics.confusion_matrix(ytest,predicted))

print('\n The value of Precision', metrics.precision_score(ytest,predicted))

print('\n The value of Recall', metrics.recall_score(ytest,predicted))

Output

The dimensions of the dataset (18, 2)

1. I love this sandwich

2. This is an amazing place

3. I feel very good about these beers

4. This is my best work

5. What an awesome view

4 Tirumala engineering college

0
 Machine Learning with python

6. I do not like this restaurant

7. I am tired of this stuff

8. I can’t deal with this

9. He is my sworn enemy

10. My boss is horrible

11. This is an awesome place

12. I do not like the taste of this juice

13. I love to dance

14. I am sick and tired of this place

15. What a great holiday

16. That is a bad locality to stay

17. We will have good fun tomorrow

18. I went to my enemy’s house today

Name: message, dtype: object 0 1

1 1

2 1

3 1

4 1

5 0

6 0

7 0

8 0

9 0

4 Tirumala engineering college

1
 Machine Learning with python

10 1

11 0

12 1

13 0

14 1

15 0

16 1

17 0

Name: labelnum, dtype: int64

The total number of Training Data: (13,) The total number of Test Data: (5,)

The words or Tokens in the text documents

[‘about’, ‘am’, ‘amazing’, ‘an’, ‘and’, ‘awesome’, ‘beers’, ‘best’, ‘can’, ‘deal’, ‘do’, ‘enemy’, ‘feel’,

‘fun’, ‘good’, ‘great’, ‘have’, ‘he’, ‘holiday’, ‘house’, ‘is’, ‘like’, ‘love’, ‘my’, ‘not’, ‘of’, ‘place’,

‘restaurant’, ‘sandwich’, ‘sick’, ‘sworn’, ‘these’, ‘this’, ‘tired’, ‘to’, ‘today’, ‘tomorrow’, ‘very’, ‘view’,
‘we’, ‘went’, ‘what’, ‘will’, ‘with’, ‘work’]

Accuracy of the classifier is 0.8

Confusion matrix

[[2 1]

[0 2]]

The value of Precision 0.6666666666666666

The value of Recall 1.0

4 Tirumala engineering college

2
 Machine Learning with python

Experiment-11: Apply EM algorithm to cluster a set of data stored in a .CSV file. Use the same
data set for clustering using the k-Means algorithm. Compare the results of these two algorithms and
comment on the quality of clustering. You can add Java/Python ML library classes/API in the program.

Python Program to Implement and Demonstrate K-Means and EM Algorithm Machine

Learning

from [Link] import KMeans

from [Link] import GaussianMixture

import [Link] as metrics

import pandas as pd

import numpy as np

import [Link] as plt

names = ['Sepal_Length','Sepal_Width','Petal_Length','Petal_Width', 'Class']

dataset = pd.read_csv("[Link]", names=names)

4 Tirumala engineering college

3
 Machine Learning with python

X = [Link][:, :-1]

label = {'Iris-setosa': 0,'Iris-versicolor': 1, 'Iris-virginica': 2}

y = [label[c] for c in [Link][:, -1]]

[Link](figsize=(14,7))

colormap=[Link](['red','lime','black'])

# REAL PLOT

[Link](1,3,1)

[Link]('Real')

[Link](X.Petal_Length,X.Petal_Width,c=colormap[y])

# K-PLOT

model=KMeans(n_clusters=3, random_state=0).fit(X)

[Link](1,3,2)

[Link]('KMeans')

[Link](X.Petal_Length,X.Petal_Width,c=colormap[model.labels_])

print('The accuracy score of K-Mean: ',metrics.accuracy_score(y, model.labels_))

print('The Confusion matrixof K-Mean:\n',metrics.confusion_matrix(y, model.labels_))

4 Tirumala engineering college

4
 Machine Learning with python

# GMM PLOT

gmm=GaussianMixture(n_components=3, random_state=0).fit(X)

y_cluster_gmm=[Link](X)

[Link](1,3,3)

[Link]('GMM Classification')

[Link](X.Petal_Length,X.Petal_Width,c=colormap[y_cluster_gmm])

print('The accuracy score of EM: ',metrics.accuracy_score(y, y_cluster_gmm))

print('The Confusion matrix of EM:\n ',metrics.confusion_matrix(y, y_cluster_gmm))

Output

The accuracy score of K-Mean: 0.24

The Confusion matrixof K-Mean:

[[ 0 50 0]
[48 0 2]
[14 0 36]]

The accuracy score of EM: 0.36666666666666664

The Confusion matrix of EM:

[[50 0 0]
[ 0 5 45]
[ 0 50 0]]

4 Tirumala engineering college

5
 Machine Learning with python

Data Set:

5.1 3.5 1.4 0.2 Iris-setosa

4.9 3 1.4 0.2 Iris-setosa
4.7 3.2 1.3 0.2 Iris-setosa
4.6 3.1 1.5 0.2 Iris-setosa
5 3.6 1.4 0.2 Iris-setosa
5.4 3.9 1.7 0.4 Iris-setosa
4.6 3.4 1.4 0.3 Iris-setosa
5 3.4 1.5 0.2 Iris-setosa
4.4 2.9 1.4 0.2 Iris-setosa
4.9 3.1 1.5 0.1 Iris-setosa
5.4 3.7 1.5 0.2 Iris-setosa
4.8 3.4 1.6 0.2 Iris-setosa
4.8 3 1.4 0.1 Iris-setosa
4.3 3 1.1 0.1 Iris-setosa
5.8 4 1.2 0.2 Iris-setosa
5.7 4.4 1.5 0.4 Iris-setosa
5.4 3.9 1.3 0.4 Iris-setosa
5.1 3.5 1.4 0.3 Iris-setosa
5.7 3.8 1.7 0.3 Iris-setosa
5.1 3.8 1.5 0.3 Iris-setosa
5.4 3.4 1.7 0.2 Iris-setosa
5.1 3.7 1.5 0.4 Iris-setosa
4.6 3.6 1 0.2 Iris-setosa
5.1 3.3 1.7 0.5 Iris-setosa
4.8 3.4 1.9 0.2 Iris-setosa
5 3 1.6 0.2 Iris-setosa

4 Tirumala engineering college

6
 Machine Learning with python

5 3.4 1.6 0.4 Iris-setosa

5.2 3.5 1.5 0.2 Iris-setosa
5.2 3.4 1.4 0.2 Iris-setosa
4.7 3.2 1.6 0.2 Iris-setosa
4.8 3.1 1.6 0.2 Iris-setosa
5.4 3.4 1.5 0.4 Iris-setosa
5.2 4.1 1.5 0.1 Iris-setosa
5.5 4.2 1.4 0.2 Iris-setosa
4.9 3.1 1.5 0.1 Iris-setosa
5 3.2 1.2 0.2 Iris-setosa
5.5 3.5 1.3 0.2 Iris-setosa
4.9 3.1 1.5 0.1 Iris-setosa
4.4 3 1.3 0.2 Iris-setosa
5.1 3.4 1.5 0.2 Iris-setosa
5 3.5 1.3 0.3 Iris-setosa
4.5 2.3 1.3 0.3 Iris-setosa
4.4 3.2 1.3 0.2 Iris-setosa
5 3.5 1.6 0.6 Iris-setosa
5.1 3.8 1.9 0.4 Iris-setosa
4.8 3 1.4 0.3 Iris-setosa
5.1 3.8 1.6 0.2 Iris-setosa
4.6 3.2 1.4 0.2 Iris-setosa
5.3 3.7 1.5 0.2 Iris-setosa
5 3.3 1.4 0.2 Iris-setosa
Iris-
7 3.2 4.7 1.4
versicolor
Iris-
6.4 3.2 4.5 1.5
versicolor
Iris-
6.9 3.1 4.9 1.5
versicolor
Iris-
5.5 2.3 4 1.3
versicolor
Iris-
6.5 2.8 4.6 1.5
versicolor
Iris-
5.7 2.8 4.5 1.3
versicolor
Iris-
6.3 3.3 4.7 1.6
versicolor
Iris-
4.9 2.4 3.3 1
versicolor
Iris-
6.6 2.9 4.6 1.3
versicolor
Iris-
5.2 2.7 3.9 1.4
versicolor
Iris-
5 2 3.5 1
versicolor
Iris-
5.9 3 4.2 1.5
versicolor
Iris-
6 2.2 4 1
versicolor

4 Tirumala engineering college

7
 Machine Learning with python

Iris-
6.1 2.9 4.7 1.4
versicolor
Iris-
5.6 2.9 3.6 1.3
versicolor
Iris-
6.7 3.1 4.4 1.4
versicolor
Iris-
5.6 3 4.5 1.5
versicolor
Iris-
5.8 2.7 4.1 1
versicolor
Iris-
6.2 2.2 4.5 1.5
versicolor
Iris-
5.6 2.5 3.9 1.1
versicolor
Iris-
5.9 3.2 4.8 1.8
versicolor
Iris-
6.1 2.8 4 1.3
versicolor
Iris-
6.3 2.5 4.9 1.5
versicolor
Iris-
6.1 2.8 4.7 1.2
versicolor
Iris-
6.4 2.9 4.3 1.3
versicolor
Iris-
6.6 3 4.4 1.4
versicolor
Iris-
6.8 2.8 4.8 1.4
versicolor
Iris-
6.7 3 5 1.7
versicolor
Iris-
6 2.9 4.5 1.5
versicolor
Iris-
5.7 2.6 3.5 1
versicolor
Iris-
5.5 2.4 3.8 1.1
versicolor
Iris-
5.5 2.4 3.7 1
versicolor
Iris-
5.8 2.7 3.9 1.2
versicolor
Iris-
6 2.7 5.1 1.6
versicolor
Iris-
5.4 3 4.5 1.5
versicolor
Iris-
6 3.4 4.5 1.6
versicolor
Iris-
6.7 3.1 4.7 1.5
versicolor
Iris-
6.3 2.3 4.4 1.3
versicolor
Iris-
5.6 3 4.1 1.3
versicolor
Iris-
5.5 2.5 4 1.3
versicolor

4 Tirumala engineering college

8
 Machine Learning with python

Iris-
5.5 2.6 4.4 1.2
versicolor
Iris-
6.1 3 4.6 1.4
versicolor
Iris-
5.8 2.6 4 1.2
versicolor
Iris-
5 2.3 3.3 1
versicolor
Iris-
5.6 2.7 4.2 1.3
versicolor
Iris-
5.7 3 4.2 1.2
versicolor
Iris-
5.7 2.9 4.2 1.3
versicolor
Iris-
6.2 2.9 4.3 1.3
versicolor
Iris-
5.1 2.5 3 1.1
versicolor
Iris-
5.7 2.8 4.1 1.3
versicolor
Iris-
6.3 3.3 6 2.5
virginica
Iris-
5.8 2.7 5.1 1.9
virginica
Iris-
7.1 3 5.9 2.1
virginica
Iris-
6.3 2.9 5.6 1.8
virginica
Iris-
6.5 3 5.8 2.2
virginica
Iris-
7.6 3 6.6 2.1
virginica
Iris-
4.9 2.5 4.5 1.7
virginica
Iris-
7.3 2.9 6.3 1.8
virginica
Iris-
6.7 2.5 5.8 1.8
virginica
Iris-
7.2 3.6 6.1 2.5
virginica
Iris-
6.5 3.2 5.1 2
virginica
Iris-
6.4 2.7 5.3 1.9
virginica
Iris-
6.8 3 5.5 2.1
virginica
Iris-
5.7 2.5 5 2
virginica
Iris-
5.8 2.8 5.1 2.4
virginica
Iris-
6.4 3.2 5.3 2.3
virginica
Iris-
6.5 3 5.5 1.8
virginica

4 Tirumala engineering college

9
 Machine Learning with python

Iris-
7.7 3.8 6.7 2.2
virginica
Iris-
7.7 2.6 6.9 2.3
virginica
Iris-
6 2.2 5 1.5
virginica
Iris-
6.9 3.2 5.7 2.3
virginica
Iris-
5.6 2.8 4.9 2
virginica
Iris-
7.7 2.8 6.7 2
virginica
Iris-
6.3 2.7 4.9 1.8
virginica
Iris-
6.7 3.3 5.7 2.1
virginica
Iris-
7.2 3.2 6 1.8
virginica
Iris-
6.2 2.8 4.8 1.8
virginica
Iris-
6.1 3 4.9 1.8
virginica
Iris-
6.4 2.8 5.6 2.1
virginica
Iris-
7.2 3 5.8 1.6
virginica
Iris-
7.4 2.8 6.1 1.9
virginica
Iris-
7.9 3.8 6.4 2
virginica
Iris-
6.4 2.8 5.6 2.2
virginica
Iris-
6.3 2.8 5.1 1.5
virginica
Iris-
6.1 2.6 5.6 1.4
virginica
Iris-
7.7 3 6.1 2.3
virginica
Iris-
6.3 3.4 5.6 2.4
virginica
Iris-
6.4 3.1 5.5 1.8
virginica
Iris-
6 3 4.8 1.8
virginica
Iris-
6.9 3.1 5.4 2.1
virginica
Iris-
6.7 3.1 5.6 2.4
virginica
Iris-
6.9 3.1 5.1 2.3
virginica
Iris-
5.8 2.7 5.1 1.9
virginica
Iris-
6.8 3.2 5.9 2.3
virginica

5 Tirumala engineering college

0
 Machine Learning with python

Iris-
6.7 3.3 5.7 2.5
virginica
Iris-
6.7 3 5.2 2.3
virginica
Iris-
6.3 2.5 5 1.9
virginica
Iris-
6.5 3 5.2 2
virginica
Iris-
6.2 3.4 5.4 2.3
virginica
Iris-
5.9 3 5.1 1.8
virginica

Experiment-12: Exploratory Data Analysis for Classification using pandas or Matplotlib.

Introduction to EDA

5 Tirumala engineering college

1
 Machine Learning with python

The main objective of this article is to cover the steps involved in Data pre-processing,
Feature Engineering, and different stages of Exploratory Data Analysis, which is an
essential step in any research analysis.

Data pre-processing, Feature Engineering, and EDA are fundamental early steps after data
collection. Still, they are not limited to where the data is simply visualized, plotted, and
manipulated, without any assumptions, to assess the quality of the data and building models.

Data Pre-processing and Feature Engineering

We spend a lot of time refining our raw data. Data pre-processing and Feature Engineering
plays a key role in any data process

Become a Full Stack Data Scientist

5 Tirumala engineering college

2
 Machine Learning with python

Transform into an expert and significantly impact the world of data science.

Download Brochure

Data Pre-processing refers to Data Integration, Data Analysis, Data cleaning, Data
Transformation, and Dimension Reduction

Data preprocessing is the process of cleaning and preparing the raw data to enable
feature engineering

Feature Engineering covers various data engineering techniques such as adding/removing

relevant features, handling missing data, encoding the data, handling categorical variables,
etc

Feature Engineering is one of the most crucial tasks and plays a major role in determining
the outcome of a model

Feature engineering involves the creation of features, whereas preprocessing involves

cleaning the data.

The Data pre-processing, Feature Engineering, and EDA steps will be carried out in this
article using Python.

5 Tirumala engineering college

3
 Machine Learning with python

Import Python Libraries

The first step involved in ML using python is understanding and playing around with our
data using libraries. Here is the link to the dataset.

Import all libraries which are required for our analysis, such as Data Loading, Statistical
analysis, Visualizations, Data Transformations, Merge and Joins, etc.

Pandas and Numpy have been used for Data Manipulation and numerical Calculations

5 Tirumala engineering college

4
 Machine Learning with python

Matplotlib and Seaborn have been used for Data visualizations.

import pandas as pd
import numpy as np
import [Link] as plt
import seaborn as sns
#to ignore warnings
import warnings
[Link]('ignore')

Reading Dataset
The Pandas library offers a wide range of possibilities for loading data into the pandas
DataFrame from files like JSON, .csv, .xlsx, .sql, .pickle, .html, .txt, images etc.

Most of the data are available in a tabular format of CSV files. It is trendy and easy to
access. Using the read_csv() function, data can be converted to a pandas DataFrame.

In this article, the data to predict Used car price is being used as an example. In this
dataset, we are trying to analyze the used car’s price and how EDA focuses on identifying
the factors influencing the car price. We have stored the data in the DataFrame data.

data = pd.read_csv("used_cars.csv")

Analyzing the data

Before we make any inferences, we listen to our data by examining all variables in the data.

The main goal of data understanding is to gain general insights about the data, which covers
the number of rows and columns, values in the data, datatypes, and Missing values in the
dataset.

shape – shape will display the number of observations(rows) and features(columns) in the
dataset

There are 7253 observations and 14 variables in our dataset

5 Tirumala engineering college

5
 Machine Learning with python

head() will display the top 5 observations of the dataset

[Link]()

tail() will display the last 5 observations of the dataset

[Link]()

5 Tirumala engineering college

6
 Machine Learning with python

info() helps to understand the data type and information about data, including the number of
records in each column, data having null or not null, Data type, the memory usage of the
dataset

[Link]()

5 Tirumala engineering college

7
 Machine Learning with python

[Link]() shows the variables Mileage, Engine, Power, Seats, New_Price, and Price have
missing values. Numeric variables like Mileage, Power are of datatype as float64 and int64.
Categorical variables like Location, Fuel_Type, Transmission, and Owner Type are of
object data type

5 Tirumala engineering college

8
 Machine Learning with python

Check for duplication

nunique() based on several unique values in each column and the data description, we can
identify the continuous and categorical columns in the data. Duplicated data can be handled
or removed based on further analysis

[Link]()

Missing values Calculation

isnull() is widely been in all pre-processing steps to identify null values in the data

In our example, [Link]().sum() is used to get the number of missing records in each
column

5 Tirumala engineering college

9
 Machine Learning with python

[Link]().sum()

The below code helps to calculate the percentage of missing values in each column

([Link]().sum()/(len(data)))*100

6 Tirumala engineering college

0
 Machine Learning with python

The percentage of missing values for the columns New_Price and Price is ~86% and ~17%,
respectively.

Data Reduction
Some columns or variables can be dropped if they do not add value to our analysis.

In our dataset, the column [Link] have only ID values, assuming they don’t have any
predictive power to predict the dependent variable.

# Remove [Link]. column from data

data = [Link](['[Link].'], axis = 1)
[Link]()

6 Tirumala engineering college

1
 Machine Learning with python

We start our Feature Engineering as we need to add some columns required for analysis.

Feature Engineering
Feature engineering refers to the process of using domain knowledge to select and transform
the most relevant variables from raw data when creating a predictive model using machine
learning or statistical modeling. The main goal of Feature engineering is to create
meaningful data from raw data.

6 Tirumala engineering college

2
 Machine Learning with python

Creating Features
We will play around with the variables Year and Name in our dataset. If we see the sample
data, the column “Year” shows the manufacturing year of the car.

It would be difficult to find the car’s age if it is in year format as the Age of the car is a
contributing factor to Car Price.

Introducing a new column, “Car_Age” to know the age of the car

from datetime import date

[Link]().year
data['Car_Age']=[Link]().year-data['Year']
[Link]()

Since car names will not be great predictors of the price in our current data. But we can
process this column to extract important information using brand and Model names. Let’s
split the name and introduce new variables “Brand” and “Model”

data['Brand'] = [Link]().[Link](0)
data['Model'] = [Link]().[Link](1) + [Link]().[Link](2)

6 Tirumala engineering college

3
 Machine Learning with python

data[['Name','Brand','Model']]

Data Cleaning/Wrangling
Some names of the variables are not relevant and not easy to understand. Some data may
have data entry errors, and some variables may need data type conversion. We need to fix
this issue in the data.

6 Tirumala engineering college

4
 Machine Learning with python

In the example, The brand name ‘Isuzu’ ‘ISUZU’ and ‘Mini’ and ‘Land’ looks incorrect. This needs to be
corrected

print([Link]())
print([Link]())

searchfor = ['Isuzu' ,'ISUZU','Mini','Land']

data[[Link]('|'.join(searchfor))].head(5)

6 Tirumala engineering college

5
 Machine Learning with python

data["Brand"].replace({"ISUZU": "Isuzu", "Mini": "Mini Cooper","Land":"Land Rover"}, inplace=True)

We have done the fundamental data analysis, Featuring, and data clean-up. Let’s move to
the EDA process

Voila!! Our Data is ready to perform EDA.

EDA Exploratory Data Analysis

6 Tirumala engineering college

6
 Machine Learning with python

Exploratory Data Analysis refers to the crucial process of performing initial investigations
on data to discover patterns to check assumptions with the help of summary statistics and
graphical representations.

 EDA can be leveraged to check for outliers, patterns, and trends in the given data.
 EDA helps to find meaningful patterns in data.
 EDA provides in-depth insights into the data sets to solve our business problems.
 EDA gives a clue to impute missing values in the dataset

Statistics Summary
The information gives a quick and simple description of the data.

Can include Count, Mean, Standard Deviation, median, mode, minimum value, maximum
value, range, standard deviation, etc.

Statistics summary gives a high-level idea to identify whether the data has any outliers, data
entry error, distribution of data such as the data is normally distributed or left/right skewed

In python, this can be achieved using describe()

describe() function gives all statistics summary of data

describe()– Provide a statistics summary of data belonging to numerical datatype such as

int, float

[Link]().T

6 Tirumala engineering college

7
 Machine Learning with python

From the statistics summary, we can infer the below findings :

 Years range from 1996- 2019 and has a high in a range which shows used cars contain both
latest models and old model cars.
 On average of Kilometers-driven in Used cars are ~58k KM. The range shows a huge
difference between min and max as max values show 650000 KM shows the evidence of an
outlier. This record can be removed.
 Min value of Mileage shows 0 cars won’t be sold with 0 mileage. This sounds like a data
entry issue.
 It looks like Engine and Power have outliers, and the data is right-skewed.

6 Tirumala engineering college

8
 Machine Learning with python

 The average number of seats in a car is 5. car seat is an important feature in price
contribution.
 The max price of a used car is 160k which is quite weird, such a high price for used cars.
There may be an outlier or data entry issue.

describe(include=’all’) provides a statistics summary of all data, include object, category etc

[Link](include='all').T

6 Tirumala engineering college

9
 Machine Learning with python

Before we do EDA, lets separate Numerical and categorical variables for easy analysis

cat_cols=data.select_dtypes(include=['object']).columns
num_cols = data.select_dtypes(include=[Link]).[Link]()

7 Tirumala engineering college

0
 Machine Learning with python

print("Categorical Variables:")
print(cat_cols)
print("Numerical Variables:")
print(num_cols)

EDA Univariate Analysis

Analyzing/visualizing the dataset by taking one variable at a time:

Data visualization is essential; we must decide what charts to plot to better understand the
data. In this article, we visualize our data using Matplotlib and Seaborn libraries.

Matplotlib is a Python 2D plotting library used to draw basic charts we use Matplotlib.

Seaborn is also a python library built on top of Matplotlib that uses short lines of code to
create and style statistical plots from Pandas and Numpy

Univariate analysis can be done for both Categorical and Numerical variables.

Categorical variables can be visualized using a Count plot, Bar Chart, Pie Plot, etc.

Numerical Variables can be visualized using Histogram, Box Plot, Density Plot, etc.

In our example, we have done a Univariate analysis using Histogram and Box Plot for
continuous Variables.

7 Tirumala engineering college

1
 Machine Learning with python

In the below fig, a histogram and box plot is used to show the pattern of the variables, as
some variables have skewness and outliers.

for col in num_cols:

print(col)
print('Skew :', round(data[col].skew(), 2))
[Link](figsize = (15, 4))
[Link](1, 2, 1)
data[col].hist(grid=False)
[Link]('count')
[Link](1, 2, 2)
[Link](x=data[col])
[Link]()

7 Tirumala engineering college

2
 Machine Learning with python

Price and Kilometers Driven are right skewed for this data to be transformed, and all
outliers will be handled during imputation

categorical variables are being visualized using a count plot. Categorical variables provide
the pattern of factors influencing car price

fig, axes = [Link](3, 2, figsize = (18, 18))

[Link]('Bar plot for all categorical variables in the dataset')
[Link](ax = axes[0, 0], x = 'Fuel_Type', data = data, color = 'blue',
order = data['Fuel_Type'].value_counts().index);
[Link](ax = axes[0, 1], x = 'Transmission', data = data, color = 'blue',
order = data['Transmission'].value_counts().index);
[Link](ax = axes[1, 0], x = 'Owner_Type', data = data, color = 'blue',
order = data['Owner_Type'].value_counts().index);
[Link](ax = axes[1, 1], x = 'Location', data = data, color = 'blue',
order = data['Location'].value_counts().index);
[Link](ax = axes[2, 0], x = 'Brand', data = data, color = 'blue',
order = data['Brand'].head(20).value_counts().index);
[Link](ax = axes[2, 1], x = 'Model', data = data, color = 'blue',
order = data['Model'].head(20).value_counts().index);
axes[1][1].tick_params(labelrotation=45);
axes[2][0].tick_params(labelrotation=90);
axes[2][1].tick_params(labelrotation=90);

7 Tirumala engineering college

3
 Machine Learning with python

7 Tirumala engineering college

4
 Machine Learning with python

From the count plot, we can have below observations

 Mumbai has the highest number of cars available for purchase, followed by Hyderabad and Coimbatore

 ~53% of cars have fuel type as Diesel this shows diesel cars provide higher performance
 ~72% of cars have manual transmission
 ~82 % of cars are First owned cars. This shows most of the buyers prefer to purchase first-
owner cars
 ~20% of cars belong to the brand Maruti followed by 19% of cars belonging to Hyundai
 WagonR ranks first among all models which are available for purchase

Data Transformation
Before we proceed to Bi-variate Analysis, Univariate analysis demonstrated the data pattern
as some variables to be transformed.

Price and Kilometer-Driven variables are highly skewed and on a larger scale. Let’s do log
transformation.

Log transformation can help in normalization, so this variable can maintain standard scale
with other variables:

# Function for log transformation of the column

def log_transform(data,col):
for colname in col:
if (data[colname] == 1.0).all():
data[colname + '_log'] = [Link](data[colname]+1)
else:
data[colname + '_log'] = [Link](data[colname])
[Link]()
log_transform(data,['Kilometers_Driven','Price'])
#Log transformation of the feature 'Kilometers_Driven'
[Link](data["Kilometers_Driven_log"], axlabel="Kilometers_Driven_log");

7 Tirumala engineering college

5
 Machine Learning with python

EDA Bivariate Analysis

Now, let’s move ahead with bivariate analysis. Bivariate Analysis helps to understand how
variables are related to each other and the relationship between dependent and independent
variables present in the dataset.

For Numerical variables, Pair plots and Scatter plots are widely been used to do Bivariate
Analysis.

7 Tirumala engineering college

6
 Machine Learning with python

A Stacked bar chart can be used for categorical variables if the output variable is a
classifier. Bar plots can be used if the output variable is continuous

In our example, a pair plot has been used to show the relationship between two Categorical
variables.

[Link](figsize=(13,17))
[Link](data=[Link](['Kilometers_Driven','Price'],axis=1))
[Link]()

7 Tirumala engineering college

7
 Machine Learning with python

Pair Plot provides below in

7 Tirumala engineering college

8
 Machine Learning with python

7 Tirumala engineering college

9
 Machine Learning with python

sights:

 The variable Year has a positive correlation with price and mileage
 A year has a Negative correlation with kilometers-Driven
 Mileage is negatively correlated with Power
 As power increases, mileage decreases
 Car with recent make is higher at prices. As the age of the car increases price decreases
 Engine and Power increase, and the price of the car increases

A bar plot can be used to show the relationship between Categorical variables and
continuous variables

fig, axarr = [Link](4, 2, figsize=(12, 18))

[Link]('Location')['Price_log'].mean().sort_values(ascending=False).[Link](ax=axarr[0][0],
fontsize=12)
axarr[0][0].set_title("Location Vs Price", fontsize=18)
[Link]('Transmission')['Price_log'].mean().sort_values(ascending=False).[Link](ax=axarr[0][1],
fontsize=12)
axarr[0][1].set_title("Transmission Vs Price", fontsize=18)
[Link]('Fuel_Type')['Price_log'].mean().sort_values(ascending=False).[Link](ax=axarr[1][0],
fontsize=12)
axarr[1][0].set_title("Fuel_Type Vs Price", fontsize=18)
[Link]('Owner_Type')['Price_log'].mean().sort_values(ascending=False).[Link](ax=axarr[1][1],
fontsize=12)
axarr[1][1].set_title("Owner_Type Vs Price", fontsize=18)
[Link]('Brand')['Price_log'].mean().sort_values(ascending=False).head(10).[Link](ax=axarr[2][0],
fontsize=12)
axarr[2][0].set_title("Brand Vs Price", fontsize=18)
[Link]('Model')['Price_log'].mean().sort_values(ascending=False).head(10).[Link](ax=axarr[2][1],
fontsize=12)
axarr[2][1].set_title("Model Vs Price", fontsize=18)
[Link]('Seats')['Price_log'].mean().sort_values(ascending=False).[Link](ax=axarr[3][0],
fontsize=12)
axarr[3][0].set_title("Seats Vs Price", fontsize=18)
[Link]('Car_Age')['Price_log'].mean().sort_values(ascending=False).[Link](ax=axarr[3][1],
fontsize=12)
axarr[3][1].set_title("Car_Age Vs Price", fontsize=18)
plt.subplots_adjust(hspace=1.0)
plt.subplots_adjust(wspace=.5)
[Link]()

8 Tirumala engineering college

0
 Machine Learning with python

8 Tirumala engineering college

1
 Machine Learning with python

Observations

 The price of cars is high in Coimbatore and less price in Kolkata and Jaipur
 Automatic cars have more price than manual cars.
 Diesel and Electric cars have almost the same price, which is maximum, and LPG cars have
the lowest price
 First-owner cars are higher in price, followed by a second
 The third owner’s price is lesser than the Fourth and above
 Lamborghini brand is the highest in price
 Gallardocoupe Model is the highest in price
 2 Seater has the highest price followed by 7 Seater
 The latest model cars are high in price

EDA Multivariate Analysis

As the name suggests, Multivariate analysis looks at more than two variables. Multivariate
analysis is one of the most useful methods to determine relationships and analyze patterns
for any dataset.

A heat map is widely been used for Multivariate Analysis

Heat Map gives the correlation between the variables, whether it has a positive or negative
correlation.

In our example heat map shows the correlation between the variables.

[Link](figsize=(12, 7))
[Link]([Link](['Kilometers_Driven','Price'],axis=1).corr(), annot = True, vmin = -1, vmax = 1)
[Link]()

8 Tirumala engineering college

2
 Machine Learning with python

From the Heat map, we can infer the following:

 The engine has a strong positive correlation to Power 0.86

 Price has a positive correlation to Engine 0.69 as well Power 0.77
 Mileage has correlated to Engine, Power, and Price negatively
 Price is moderately positive in correlation to year.
 Kilometer driven has a negative correlation to year not much impact on the price
 Car age has a negative correlation with Price
 car Age is positively correlated to Kilometers-Driven as the Age of the car increases; then
the kilometer will also increase of car has a negative correlation with Mileage this makes
sense

Impute Missing values

Missing data arise in almost all statistical analyses. There are many ways to impute missing
values; we can impute the missing values by their Mean, median, most frequent, or zero
values and use advanced imputation algorithms like KNN, Regularization, etc.

8 Tirumala engineering college

3
 Machine Learning with python

We cannot impute the data with a simple Mean/Median. We must need business knowledge
or common insights about the data. If we have domain knowledge, it will add value to the
imputation. Some data can be imputed on assumptions.

In our dataset, we have found there are missing values for many columns like Mileage,
Power, and Seats.

We observed earlier some observations have zero Mileage. This looks like a data entry
issue. We could fix this by filling null values with zero and then the mean value of Mileage
since Mean and Median values are nearly the same for this variable chosen Mean to impute
the values.

[Link][data["Mileage"]==0.0,'Mileage']=[Link]
[Link]().sum()
data['Mileage'].fillna(value=[Link](data['Mileage']),inplace=True)

Similarly, imputation for Seats. As we mentioned earlier, we need to know common insights
about the data.

Let’s assume some cars brand and Models have features like Engine, Mileage, Power, and
Number of seats that are nearly the same. Let’s impute those missing values with the
existing data:

[Link]().sum()
data['Seats'].fillna(value=[Link],inplace=True)
data['Seats']=[Link](['Model','Brand'])['Seats'].apply(lambda x:[Link]([Link]()))
data['Engine']=[Link](['Brand','Model'])['Engine'].apply(lambda x:[Link]([Link]()))
data['Power']=[Link](['Brand','Model'])['Power'].apply(lambda x:[Link]([Link]()))

In general, there are no defined or perfect rules for imputing missing values in a dataset.
Each method can perform better for some datasets but may perform even worse. Only
practice and experiments give the knowledge which works better.

Conclusion
In this article, we tried to analyze the factors influencing the used car’s price.

8 Tirumala engineering college

4
 Machine Learning with python

 Data Analysis helps to find the basic structure of the dataset.

 Dropped columns that are not adding value to our analysis.
 Performed Feature Engineering by adding some columns which contribute to our analysis.
 Data Transformations have been used to normalize the columns.
 We used different visualizations for EDA like Univariate, Bi-Variate, and Multivariate
Analysis.

Through EDA, we got useful insights, and below are the factors influencing the price
of the car and a few takeaways

 Most of the customers prefer 2 Seat cars hence the price of the 2-seat cars is higher than
other cars.
 The price of the car decreases as the Age of the car increases.
 Customers prefer to purchase the First owner rather than the Second or Third.
 Due to increased Fuel price, the customer prefers to purchase an Electric vehicle.
 Automatic Transmission is easier than Manual.

This way, we perform EDA on the datasets to explore the data and extract all possible
insights, which can help in model building and better decision making.

However, this was only an overview of how EDA works; you can go deeper into it and
attempt the stages on larger datasets.

If the EDA process is clear and precise, our model will work better and gives higher
accuracy!

8 Tirumala engineering college

5
 Machine Learning with python

Experiment -13: Write a python to construct a Bayesian network considering medical [Link]
this model to demonstract the diagnosis of heart patients using standard heart disease Data Set.

Theory

A Bayesian network is a directed acyclic graph in which each edge corresponds to a conditional
dependency, and each node corresponds to a unique random variable.

Bayesian network consists of two major parts: a directed acyclic graph and a set of conditional
probability distributions

 The directed acyclic graph is a set of random variables represented by nodes.

 The conditional probability distribution of a node (random variable) is defined for every possible
outcome of the preceding causal node(s).

For illustration, consider the following example. Suppose we attempt to turn on our computer, but the
computer does not start (observation/evidence). We would like to know which of the possible causes of
computer failure is more likely. In this simplified illustration, we assume only two possible causes of
this misfortune: electricity failure and computer malfunction.

The corresponding directed acyclic graph is depicted in below figure.

The goal is to calculate the posterior conditional probability distribution of each of the possible
unobserved causes given the observed evidence, i.e. P [Cause | Evidence].

Data Set:

Title: Heart Disease Databases

The Cleveland database contains 76 attributes, but all published experiments refer to using a subset of
14 of them. In particular, the Cleveland database is the only one that has been used by ML researchers
to this date. The “Heartdisease” field refers to the presence of heart disease in the patient. It is integer
valued from 0 (no presence) to 4.

8 Tirumala engineering college

6
 Machine Learning with python

Database: 0 1 2 3 4 Total

Cleveland: 164 55 36 35 13 303

Attribute Information:

1. age: age in years

2. sex: sex (1 = male; 0 = female)
3. cp: chest pain type
1. Value 1: typical angina
2. Value 2: atypical angina
3. Value 3: non-anginal pain
4. Value 4: asymptomatic
4. trestbps: resting blood pressure (in mm Hg on admission to the hospital)
5. chol: serum cholestoral in mg/dl
6. fbs: (fasting blood sugar > 120 mg/dl) (1 = true; 0 = false)
7. restecg: resting electrocardiographic results
1. Value 0: normal
2. Value 1: having ST-T wave abnormality (T wave inversions and/or ST elevation or
depression of > 0.05 mV)
3. Value 2: showing probable or definite left ventricular hypertrophy by Estes’ criteria
8. thalach: maximum heart rate achieved
9. exang: exercise induced angina (1 = yes; 0 = no)
10. oldpeak = ST depression induced by exercise relative to rest
11. slope: the slope of the peak exercise ST segment
1. Value 1: upsloping
2. Value 2: flat
3. Value 3: downsloping
12. thal: 3 = normal; 6 = fixed defect; 7 = reversable defect
13. Heartdisease: It is integer valued from 0 (no presence) to 4.

Some instance from the dataset:

ag se c trestbp cho fb restec thalac exan oldpea slop c tha Heartdiseas

e x p s l s g h g k e a l e

63 1 1 145 233 1 2 150 o 2.3 3 o 6 o

67 1 4 160 286 o 2 108 1 1.5 2 3 3 2

67 1 4 120 229 o 2 129 1 2.6 2 2 7 1

41 o 2 130 204 o 2 172 o 1.4 1 o 3 o

62 o 4 140 268 o 2 160 o 3.6 3 2 3 3

8 Tirumala engineering college

7
 Machine Learning with python

60 1 4 130 206 o 2 132 1 2.4 2 2 7 4

Python Program to Implement and Demonstrate Bayesian network using pgmpy Machine
Learning

import numpy as np

import pandas as pd

import csv

from [Link] import MaximumLikelihoodEstimator

from [Link] import BayesianModel

from [Link] import VariableElimination

heartDisease = pd.read_csv('[Link]')

heartDisease = [Link]('?',[Link])

print('Sample instances from the dataset are given below')

print([Link]())

print('\n Attributes and datatypes')

print([Link])

model= BayesianModel([('age','heartdisease'),('sex','heartdisease'),('exang','heartdisease'),
('cp','heartdisease'),('heartdisease','restecg'),('heartdisease','chol')])

print('\nLearning CPD using Maximum likelihood estimators')

[Link](heartDisease,estimator=MaximumLikelihoodEstimator)

8 Tirumala engineering college

8
 Machine Learning with python

print('\n Inferencing with Bayesian Network:')

HeartDiseasetest_infer = VariableElimination(model)

print('\n 1. Probability of HeartDisease given evidence= restecg')

q1=HeartDiseasetest_infer.query(variables=['heartdisease'],evidence={'restecg':1})

print(q1)

print('\n 2. Probability of HeartDisease given evidence= cp ')

q2=HeartDiseasetest_infer.query(variables=['heartdisease'],evidence={'cp':2})

print(q2)

Output

8 Tirumala engineering college

9
 Machine Learning with python

age gender cp trestbps chol fbs restecg thalach exang oldpeak

63 1 1 145 233 1 2 150 0 2.3
67 1 4 160 286 0 2 108 1 1.5
67 1 4 120 229 0 2 129 1 2.6
37 1 3 130 250 0 0 187 0 3.5
41 0 2 130 204 0 2 172 0 1.4
56 1 2 120 236 0 0 178 0 0.8
62 0 4 140 268 0 2 160 0 3.6
57 0 4 120 354 0 0 163 1 0.6
63 1 4 130 254 0 2 147 0 1.4
53 1 4 140 203 1 2 155 1 3.1
57 1 4 140 192 0 0 148 0 0.4
56 0 2 140 294 0 2 153 0 1.3
56 1 3 130 256 1 2 142 1 0.6
44 1 2 120 263 0 0 173 0 0
52 1 3 172 199 1 0 162 0 0.5
57 1 3 150 168 0 0 174 0 1.6
48 1 2 110 229 0 0 168 0 1
54 1 4 140 239 0 0 160 0 1.2
48 0 3 130 275 0 0 139 0 0.2
49 1 2 130 266 0 0 171 0 0.6
64 1 1 110 211 0 2 144 1 1.8
58 0 1 150 283 1 2 162 0 1
58 1 2 120 284 0 2 160 0 1.8
58 1 3 132 224 0 2 173 0 3.2
60 1 4 130 206 0 2 132 1 2.4
50 0 3 120 219 0 0 158 0 1.6
58 0 3 120 340 0 0 172 0 0
66 0 1 150 226 0 0 114 0 2.6
43 1 4 150 247 0 0 171 0 1.5
40 1 4 110 167 0 2 114 1 2

9 Tirumala engineering college

0
 Machine Learning with python

69 0 1 140 239 0 0 151 0 1.8

60 1 4 117 230 1 0 160 1 1.4
64 1 3 140 335 0 0 158 0 0
59 1 4 135 234 0 0 161 0 0.5
44 1 3 130 233 0 0 179 1 0.4
42 1 4 140 226 0 0 178 0 0
43 1 4 120 177 0 2 120 1 2.5
57 1 4 150 276 0 2 112 1 0.6
55 1 4 132 353 0 0 132 1 1.2
61 1 3 150 243 1 0 137 1 1
65 0 4 150 225 0 2 114 0 1
40 1 1 140 199 0 0 178 1 1.4
71 0 2 160 302 0 0 162 0 0.4
59 1 3 150 212 1 0 157 0 1.6
61 0 4 130 330 0 2 169 0 0
58 1 3 112 230 0 2 165 0 2.5
51 1 3 110 175 0 0 123 0 0.6
50 1 4 150 243 0 2 128 0 2.6
65 0 3 140 417 1 2 157 0 0.8
53 1 3 130 197 1 2 152 0 1.2
41 0 2 105 198 0 0 168 0 0
65 1 4 120 177 0 0 140 0 0.4
44 1 4 112 290 0 2 153 0 0
44 1 2 130 219 0 2 188 0 0
60 1 4 130 253 0 0 144 1 1.4
54 1 4 124 266 0 2 109 1 2.2
50 1 3 140 233 0 0 163 0 0.6
41 1 4 110 172 0 2 158 0 0
54 1 3 125 273 0 2 152 0 0.5
51 1 1 125 213 0 2 125 1 1.4
51 0 4 130 305 0 0 142 1 1.2
46 0 3 142 177 0 2 160 1 1.4
58 1 4 128 216 0 2 131 1 2.2
54 0 3 135 304 1 0 170 0 0
54 1 4 120 188 0 0 113 0 1.4
60 1 4 145 282 0 2 142 1 2.8
60 1 3 140 185 0 2 155 0 3
54 1 3 150 232 0 2 165 0 1.6
59 1 4 170 326 0 2 140 1 3.4
46 1 3 150 231 0 0 147 0 3.6
65 0 3 155 269 0 0 148 0 0.8
67 1 4 125 254 1 0 163 0 0.2
62 1 4 120 267 0 0 99 1 1.8
65 1 4 110 248 0 2 158 0 0.6
44 1 4 110 197 0 2 177 0 0
65 0 3 160 360 0 2 151 0 0.8

9 Tirumala engineering college

1
 Machine Learning with python

60 1 4 125 258 0 2 141 1 2.8

51 0 3 140 308 0 2 142 0 1.5
48 1 2 130 245 0 2 180 0 0.2
58 1 4 150 270 0 2 111 1 0.8
45 1 4 104 208 0 2 148 1 3
53 0 4 130 264 0 2 143 0 0.4
39 1 3 140 321 0 2 182 0 0
68 1 3 180 274 1 2 150 1 1.6
52 1 2 120 325 0 0 172 0 0.2
44 1 3 140 235 0 2 180 0 0
47 1 3 138 257 0 2 156 0 0
53 0 3 128 216 0 2 115 0 0
53 0 4 138 234 0 2 160 0 0
51 0 3 130 256 0 2 149 0 0.5
66 1 4 120 302 0 2 151 0 0.4
62 0 4 160 164 0 2 145 0 6.2
62 1 3 130 231 0 0 146 0 1.8
44 0 3 108 141 0 0 175 0 0.6
63 0 3 135 252 0 2 172 0 0
52 1 4 128 255 0 0 161 1 0
59 1 4 110 239 0 2 142 1 1.2
60 0 4 150 258 0 2 157 0 2.6
52 1 2 134 201 0 0 158 0 0.8
48 1 4 122 222 0 2 186 0 0
45 1 4 115 260 0 2 185 0 0
34 1 1 118 182 0 2 174 0 0
57 0 4 128 303 0 2 159 0 0
71 0 3 110 265 1 2 130 0 0
49 1 3 120 188 0 0 139 0 2
54 1 2 108 309 0 0 156 0 0
59 1 4 140 177 0 0 162 1 0
57 1 3 128 229 0 2 150 0 0.4
61 1 4 120 260 0 0 140 1 3.6
39 1 4 118 219 0 0 140 0 1.2
61 0 4 145 307 0 2 146 1 1
56 1 4 125 249 1 2 144 1 1.2
52 1 1 118 186 0 2 190 0 0
43 0 4 132 341 1 2 136 1 3
62 0 3 130 263 0 0 97 0 1.2
41 1 2 135 203 0 0 132 0 0
58 1 3 140 211 1 2 165 0 0
35 0 4 138 183 0 0 182 0 1.4
63 1 4 130 330 1 2 132 1 1.8
65 1 4 135 254 0 2 127 0 2.8
48 1 4 130 256 1 2 150 1 0
63 0 4 150 407 0 2 154 0 4

9 Tirumala engineering college

2
 Machine Learning with python

51 1 3 100 222 0 0 143 1 1.2

55 1 4 140 217 0 0 111 1 5.6
65 1 1 138 282 1 2 174 0 1.4
45 0 2 130 234 0 2 175 0 0.6
56 0 4 200 288 1 2 133 1 4
54 1 4 110 239 0 0 126 1 2.8
44 1 2 120 220 0 0 170 0 0
62 0 4 124 209 0 0 163 0 0
54 1 3 120 258 0 2 147 0 0.4
51 1 3 94 227 0 0 154 1 0
29 1 2 130 204 0 2 202 0 0
51 1 4 140 261 0 2 186 1 0
43 0 3 122 213 0 0 165 0 0.2
55 0 2 135 250 0 2 161 0 1.4
70 1 4 145 174 0 0 125 1 2.6
62 1 2 120 281 0 2 103 0 1.4
35 1 4 120 198 0 0 130 1 1.6
51 1 3 125 245 1 2 166 0 2.4
59 1 2 140 221 0 0 164 1 0
59 1 1 170 288 0 2 159 0 0.2
52 1 2 128 205 1 0 184 0 0
64 1 3 125 309 0 0 131 1 1.8
58 1 3 105 240 0 2 154 1 0.6
47 1 3 108 243 0 0 152 0 0
57 1 4 165 289 1 2 124 0 1
41 1 3 112 250 0 0 179 0 0
45 1 2 128 308 0 2 170 0 0
60 0 3 102 318 0 0 160 0 0
52 1 1 152 298 1 0 178 0 1.2
42 0 4 102 265 0 2 122 0 0.6
67 0 3 115 564 0 2 160 0 1.6
55 1 4 160 289 0 2 145 1 0.8
64 1 4 120 246 0 2 96 1 2.2
70 1 4 130 322 0 2 109 0 2.4
51 1 4 140 299 0 0 173 1 1.6
58 1 4 125 300 0 2 171 0 0
60 1 4 140 293 0 2 170 0 1.2
68 1 3 118 277 0 0 151 0 1
46 1 2 101 197 1 0 156 0 0
77 1 4 125 304 0 2 162 1 0
54 0 3 110 214 0 0 158 0 1.6
58 0 4 100 248 0 2 122 0 1
48 1 3 124 255 1 0 175 0 0
57 1 4 132 207 0 0 168 1 0
52 1 3 138 223 0 0 169 0 0
54 0 2 132 288 1 2 159 1 0

9 Tirumala engineering college

3
 Machine Learning with python

35 1 4 126 282 0 2 156 1 0

45 0 2 112 160 0 0 138 0 0
70 1 3 160 269 0 0 112 1 2.9
53 1 4 142 226 0 2 111 1 0
59 0 4 174 249 0 0 143 1 0
62 0 4 140 394 0 2 157 0 1.2
64 1 4 145 212 0 2 132 0 2
57 1 4 152 274 0 0 88 1 1.2
52 1 4 108 233 1 0 147 0 0.1
56 1 4 132 184 0 2 105 1 2.1
43 1 3 130 315 0 0 162 0 1.9
53 1 3 130 246 1 2 173 0 0
48 1 4 124 274 0 2 166 0 0.5
56 0 4 134 409 0 2 150 1 1.9
42 1 1 148 244 0 2 178 0 0.8
59 1 1 178 270 0 2 145 0 4.2
60 0 4 158 305 0 2 161 0 0
63 0 2 140 195 0 0 179 0 0
42 1 3 120 240 1 0 194 0 0.8
66 1 2 160 246 0 0 120 1 0
54 1 2 192 283 0 2 195 0 0
69 1 3 140 254 0 2 146 0 2
50 1 3 129 196 0 0 163 0 0
51 1 4 140 298 0 0 122 1 4.2
43 1 4 132 247 1 2 143 1 0.1
62 0 4 138 294 1 0 106 0 1.9
68 0 3 120 211 0 2 115 0 1.5
67 1 4 100 299 0 2 125 1 0.9
69 1 1 160 234 1 2 131 0 0.1
45 0 4 138 236 0 2 152 1 0.2
50 0 2 120 244 0 0 162 0 1.1
59 1 1 160 273 0 2 125 0 0
50 0 4 110 254 0 2 159 0 0
64 0 4 180 325 0 0 154 1 0
57 1 3 150 126 1 0 173 0 0.2
64 0 3 140 313 0 0 133 0 0.2
43 1 4 110 211 0 0 161 0 0
45 1 4 142 309 0 2 147 1 0
58 1 4 128 259 0 2 130 1 3
50 1 4 144 200 0 2 126 1 0.9
55 1 2 130 262 0 0 155 0 0
62 0 4 150 244 0 0 154 1 1.4
37 0 3 120 215 0 0 170 0 0
38 1 1 120 231 0 0 182 1 3.8
41 1 3 130 214 0 2 168 0 2
66 0 4 178 228 1 0 165 1 1

9 Tirumala engineering college

4
 Machine Learning with python

52 1 4 112 230 0 0 160 0 0

56 1 1 120 193 0 2 162 0 1.9
46 0 2 105 204 0 0 172 0 0
46 0 4 138 243 0 2 152 1 0
64 0 4 130 303 0 0 122 0 2
59 1 4 138 271 0 2 182 0 0
41 0 3 112 268 0 2 172 1 0
54 0 3 108 267 0 2 167 0 0
39 0 3 94 199 0 0 179 0 0
53 1 4 123 282 0 0 95 1 2
63 0 4 108 269 0 0 169 1 1.8
34 0 2 118 210 0 0 192 0 0.7
47 1 4 112 204 0 0 143 0 0.1
67 0 3 152 277 0 0 172 0 0
54 1 4 110 206 0 2 108 1 0
66 1 4 112 212 0 2 132 1 0.1
52 0 3 136 196 0 2 169 0 0.1
55 0 4 180 327 0 1 117 1 3.4
49 1 3 118 149 0 2 126 0 0.8
74 0 2 120 269 0 2 121 1 0.2
54 0 3 160 201 0 0 163 0 0
54 1 4 122 286 0 2 116 1 3.2
56 1 4 130 283 1 2 103 1 1.6
46 1 4 120 249 0 2 144 0 0.8
49 0 2 134 271 0 0 162 0 0
42 1 2 120 295 0 0 162 0 0
41 1 2 110 235 0 0 153 0 0
41 0 2 126 306 0 0 163 0 0
49 0 4 130 269 0 0 163 0 0
61 1 1 134 234 0 0 145 0 2.6
60 0 3 120 178 1 0 96 0 0
67 1 4 120 237 0 0 71 0 1
58 1 4 100 234 0 0 156 0 0.1
47 1 4 110 275 0 2 118 1 1
52 1 4 125 212 0 0 168 0 1
62 1 2 128 208 1 2 140 0 0
57 1 4 110 201 0 0 126 1 1.5
58 1 4 146 218 0 0 105 0 2
64 1 4 128 263 0 0 105 1 0.2
51 0 3 120 295 0 2 157 0 0.6
43 1 4 115 303 0 0 181 0 1.2
42 0 3 120 209 0 0 173 0 0
67 0 4 106 223 0 0 142 0 0.3
76 0 3 140 197 0 1 116 0 1.1
70 1 2 156 245 0 2 143 0 0
57 1 2 124 261 0 0 141 0 0.3

9 Tirumala engineering college

5
 Machine Learning with python

44 0 3 118 242 0 0 149 0 0.3

58 0 2 136 319 1 2 152 0 0
60 0 1 150 240 0 0 171 0 0.9
44 1 3 120 226 0 0 169 0 0
61 1 4 138 166 0 2 125 1 3.6
42 1 4 136 315 0 0 125 1 1.8
52 1 4 128 204 1 0 156 1 1
59 1 3 126 218 1 0 134 0 2.2
40 1 4 152 223 0 0 181 0 0
42 1 3 130 180 0 0 150 0 0
61 1 4 140 207 0 2 138 1 1.9
66 1 4 160 228 0 2 138 0 2.3
46 1 4 140 311 0 0 120 1 1.8
71 0 4 112 149 0 0 125 0 1.6
59 1 1 134 204 0 0 162 0 0.8
64 1 1 170 227 0 2 155 0 0.6
66 0 3 146 278 0 2 152 0 0
39 0 3 138 220 0 0 152 0 0
57 1 2 154 232 0 2 164 0 0
58 0 4 130 197 0 0 131 0 0.6
57 1 4 110 335 0 0 143 1 3
47 1 3 130 253 0 0 179 0 0
55 0 4 128 205 0 1 130 1 2
35 1 2 122 192 0 0 174 0 0
61 1 4 148 203 0 0 161 0 0
58 1 4 114 318 0 1 140 0 4.4
58 0 4 170 225 1 2 146 1 2.8
58 1 2 125 220 0 0 144 0 0.4
56 1 2 130 221 0 2 163 0 0
56 1 2 120 240 0 0 169 0 0
67 1 3 152 212 0 2 150 0 0.8
55 0 2 132 342 0 0 166 0 1.2
44 1 4 120 169 0 0 144 1 2.8
63 1 4 140 187 0 2 144 1 4
63 0 4 124 197 0 0 136 1 0
41 1 2 120 157 0 0 182 0 0
59 1 4 164 176 1 2 90 0 1
57 0 4 140 241 0 0 123 1 0.2
45 1 1 110 264 0 0 132 0 1.2
68 1 4 144 193 1 0 141 0 3.4
57 1 4 130 131 0 0 115 1 1.2
57 0 2 130 236 0 2 174 0 0
38 1 3 138 175 0 0 173 0 0
Data Set:

9 Tirumala engineering college

6
 Machine Learning with python

Experiment -14:Write a program to implement support vector Machines and Principal

Component Analysis

Introduction
A Support Vector Machine (SVM) is a very powerful and versatile Machine Learning
model, capable of performing linear or nonlinear classification, regression, and even outlier
detection. With this tutorial, we learn about the support vector machine technique and how
to use it in scikit-learn. We will also discover the Principal Component Analysis and its
implementation with scikit-learn.

1. Support Vector Machine

Another simple approach that any machine learning expert should know about is the support
vector machine. Many people prefer the support vector machine because it produces great
accuracy while using less computing power. SVM (Support Vector Machine) can be used
for both regression and classification. However, it is widely applied in classifications
objectives.

What is a Support Vector Machine?

The objective of the support vector machine algorithm is to find a hyperplane in N-
dimensional space(N — the number of features) that distinctly classifies the data points.

9 Tirumala engineering college

7
 Machine Learning with python

There are numerous hyper-planes from which to choose to split the two kinds of data points.
Our goal is to discover a plane with the greatest margin, or the greatest distance between
data points from both classes. Maximizing the margin distance adds some reinforcement,
making it easier to classify future data points.

Hyper-planes and Support Vectors

Hyper-planes are decision-making boundaries that help in data classification. Different

classes can be assigned to data points on either side of the hyperplane. The hyperplane’s
dimension is also determined by the number of features. If there are only two input
characteristics, the hyperplane is simply a line. The hyperplane becomes a two-dimensional

9 Tirumala engineering college

8
 Machine Learning with python

plane when the number of input features reaches three. When the number of features
exceeds three, it becomes impossible to imagine.

Become a Full Stack Data Scientist

Transform into an expert and significantly impact the world of data science.

Download Brochure

Support vectors are data points that are closer to the hyperplane and have an influence on
the hyperplane’s position and orientation. We increase the classifier’s margin by using these

9 Tirumala engineering college

9
 Machine Learning with python

support vectors. The hyperplane’s position will be altered if the support vectors are deleted.
These are the points that will assist us in constructing our SVM.

2. SVM Implementation in Python

We will use a support vector machine in Predicting if the cancer diagnosis is benign or
malignant based on several observations/features.

import pandas as pd
import numpy as np
import [Link] as plt
import seaborn as sns
%matplotlib inline
sns.set_style('whitegrid')

Python Code:

print(cancer.target_names)

[Link]()

1 Tirumala engineering college

0
 Machine Learning with python

[Link]()

Visualizing The Data

[Link]

[Link](df, hue='target', vars=['mean radius', 'mean texture', 'mean perimeter', 'mean area',
'mean smoothness', 'mean compactness', 'mean concavity',
'mean concave points', 'mean symmetry', 'mean fractal dimension'])

1 Tirumala engineering college

0
 Machine Learning with python

1 Tirumala engineering college

0
 Machine Learning with python

[Link](df['target'], label = "Count")

[Link](figsize=(10, 8))
[Link](x = 'mean area', y = 'mean smoothness', hue = 'target', data = df)

# Let's check the correlation between the variables

# Strong correlation between the mean radius and mean perimeter, mean area and mean primeter
[Link](figsize=(20,10))
[Link]([Link](), annot=True)

1 Tirumala engineering college

0
 Machine Learning with python

3. Model Training and finding problem solution

we use SVM sklearn for selection and for training, sklearn support vector machine to do
cross_val_score, train_test_split data.

from sklearn.model_selection import cross_val_score, train_test_split

from [Link] import Pipeline
from [Link] import StandardScaler, MinMaxScaler
X = [Link]('target', axis=1)
y = [Link]
print(f"'X' shape: {[Link]}")
print(f"'y' shape: {[Link]}")
pipeline = Pipeline([
('min_max_scaler', MinMaxScaler()),
('std_scaler', StandardScaler())
])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

from [Link] import accuracy_score, confusion_matrix, classification_report

def print_score(clf, X_train, y_train, X_test, y_test, train=True):
if train:
pred = [Link](X_train)
clf_report = [Link](classification_report(y_train, pred, output_dict=True))
print("Train Result:n================================================")
print(f"Accuracy Score: {accuracy_score(y_train, pred) * 100:.2f}%")

1 Tirumala engineering college

0
 Machine Learning with python

print("_______________________________________________")
print(f"CLASSIFICATION REPORT:n{clf_report}")
print("_______________________________________________")
print(f"Confusion Matrix: n {confusion_matrix(y_train, pred)}n")
elif train==False:
pred = [Link](X_test)
clf_report = [Link](classification_report(y_test, pred, output_dict=True))
print("Test Result:n================================================")
print(f"Accuracy Score: {accuracy_score(y_test, pred) * 100:.2f}%")
print("_______________________________________________")
print(f"CLASSIFICATION REPORT:n{clf_report}")
print("_______________________________________________")
print(f"Confusion Matrix: n {confusion_matrix(y_test, pred)}n")

Support Vector Machines (Kernels)

C parameter: Controls trade-off between classifying training points correctly and having a
smooth decision boundary.

 Small C (loose) makes the cost (penalty) of misclassification low (soft margin)
 Large C (strict) makes the cost of misclassification high (hard margin), forcing the model to
explain input data stricter and potentially over its

gamma parameter: Controls how far the influence of a single training set reaches.

 Large gamma: close reach (closer data points have high weight)
 Small gamma: far reach (more generalized solution)

degree parameter: Degree of the polynomial kernel function (‘poly’). Ignored by all other
kernels.
Grid search is a popular way to find the right hyper-parameter values. Performing a large
grid search first, then a refined grid search centred on the best results is frequently faster.
Knowing what each hyper-parameter does can also help you identify the right part of the
hyper-parameter space to search for.

SVM Kernel type:

1. Linear Kernel SVM

from [Link] import LinearSVC

model = LinearSVC(loss='hinge', dual=True)

1 Tirumala engineering college

0
 Machine Learning with python

[Link](X_train, y_train)
print_score(model, X_train, y_train, X_test, y_test, train=True)
print_score(model, X_train, y_train, X_test, y_test, train=False)

2. Polynomial Kernel SVM

This code trains an SVM classifier using a 2nd-degree polynomial kernel.

from [Link] import SVC

# The hyperparameter coef0 controls how much the model is influenced by high degree ploynomials
model = SVC(kernel='poly', degree=2, gamma='auto', coef0=1, C=5)
[Link](X_train, y_train)
print_score(model, X_train, y_train, X_test, y_test, train=True)
print_score(model, X_train, y_train, X_test, y_test, train=False)

1 Tirumala engineering college

0
 Machine Learning with python

3. Radial Kernel SVM

Just like the polynomial features method, the similarity features can be useful with any

model = SVC(kernel='rbf', gamma=0.5, C=0.1)

[Link](X_train, y_train)
print_score(model, X_train, y_train, X_test, y_test, train=True)
print_score(model, X_train, y_train, X_test, y_test, train=False)

1 Tirumala engineering college

0
 Machine Learning with python

Other kernels are available, but they are used far less frequently. Some kernels, for example,
are specialized to particular data structures. When identifying text documents based on
DNA sequences, string kernels are sometimes used.

How do you determine which kernel to use when there are so many options? If the training
set is big or has a lot of characteristics, you should always attempt the linear kernel first.
You should also try the Gaussian RBF kernel if the training set isn’t too big.

Support Vector Machine Hyper-parameter tuning

from sklearn.model_selection import GridSearchCV
param_grid = {'C': [0.01, 0.1, 0.5, 1, 10, 100],
'gamma': [1, 0.75, 0.5, 0.25, 0.1, 0.01, 0.001],
'kernel': ['rbf', 'poly', 'linear']}
grid = GridSearchCV(SVC(), param_grid, refit=True, verbose=1, cv=5, iid=True)
[Link](X_train, y_train)
best_params = grid.best_params_
print(f"Best params: {best_params}")
svm_clf = SVC(**best_params)

1 Tirumala engineering college

0
 Machine Learning with python

svm_clf.fit(X_train, y_train)
print_score(svm_clf, X_train, y_train, X_test, y_test, train=True)
print_score(svm_clf, X_train, y_train, X_test, y_test, train=False)

4. Principal Component Analysis

PCA is.

 Singular Value Decomposition is used to reduce the data’s dimensionality and project it to a
lower-dimensional environment.
 Unsupervised Machine Learning
 A transformation of your data and attempts to find out what features explain the most
variance in your data. For example:

1 Tirumala engineering college

0
 Machine Learning with python

robpca

[Link]()

1 Tirumala engineering college

1
 Machine Learning with python

PCA Visualization
Because it’s difficult to represent high-dimensional data using a single scatter-plot, we may
use PCA to determine the first two main components and visualize the data in this new,
two-dimensional space. However, we must first scale our data to ensure that each feature
has single unit variance.

scaler = StandardScaler()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)

PCA in Scikit Learn works in a similar way to the other preprocessing methods in Scikit
Learn. We create a PCA object, use the fit method to discover the principle components,
and then use transform to rotate and reduce the dimensionality.

When building the PCA object, we can additionally indicate how many components we
wish to create.

from [Link] import PCA

pca = PCA(n_components=3)
scaler = StandardScaler()
X_train = pca.fit_transform(X_train)
X_test = [Link](X_test)
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)
[Link](figsize=(8,6))
[Link](X_train[:,0],X_train[:,1],c=y_train,cmap='plasma')
[Link]('First principal component')
[Link]('Second Principal Component')

1 Tirumala engineering college

1
 Machine Learning with python

Using these two components we can easily separate these two classes.

Interpreting the components

Unfortunately, this remarkable power of dimensionality reduction comes at the cost of not
being able to understand what these components represent easily.
Components are maintained as an attribute of the fitted PCA object and correspond to
combinations of the original features:
param_grid = {'C': [0.01, 0.1, 0.5, 1, 10, 100],
'gamma': [1, 0.75, 0.5, 0.25, 0.1, 0.01, 0.001],
'kernel': ['rbf', 'poly', 'linear']}
grid = GridSearchCV(SVC(), param_grid, refit=True, verbose=1, cv=5, iid=True)
[Link](X_train, y_train)
best_params = grid.best_params_
print(f"Best params: {best_params}")
svm_clf = SVC(**best_params)
svm_clf.fit(X_train, y_train)
print_score(svm_clf, X_train, y_train, X_test, y_test, train=True)
print_score(svm_clf, X_train, y_train, X_test, y_test, train=False)

1 Tirumala engineering college

1
 Machine Learning with python

1 Tirumala engineering college

1
 Machine Learning with python

Experiment-15: Write a program to implement Principal Component Analysis.

Principal Component Analysis (PCA): is an algebraic technique for converting

a set of observations of possibly correlated variables into the set of values of liner
uncorrelated variables.

All principal components are chosen to describe most of the available variance in
the variable, and all principal components are orthogonal to each other. In all the
sets of the principal component first principal component will always have the
maximum variance.

Different Uses of Principal Component Analysis:

o PCA can be used for finding interrelations between various variables in the
data.
o PCA can be used for interpreting and visualizing the data sets.
o PCA can also be used for visualizing genetic distance and connection
between populations.
o PCA also makes analysis simple with the decrease in the number of
variables.

Principal component analysations are usually executed on a square symmetric

matrix, and this can be a pure sum of squares and cross products matrix or
correlation matrix or covariance matrix. The correlation matrix is used if there is a
major difference in the individual variance.

What are the Objectives of Principal Component

Analysis?
The basic objectives of PCA are as follows:

o PCA is a nondependent method can be used for reducing attribute space

from a larger number of variables of the set to a smaller number of factors.
o It is a dimension reducing technique but with no assurance whether the
dimension would be interpretable.
o In PCA, the main job is selecting the subset of variables from a larger set,
depending on which original variables will have the highest correlation with
the principal amount.

Principal Axis Method: Principal Component Analysis searches for the linear
combination of the variable for extracting maximum variance from the variables.
Once the PCA is done with the process, it will move forward to another linear

1 Tirumala engineering college

1
 Machine Learning with python

combination which will explain the maximum ratio of the remaining variance,
which would lead to orthogonal factors of the sets. This method is used for
analysing total variance in the variables of the set.

Eigen Vector: It is a nonzero vector that remains parallel after multiplying the
matrix. Suppose 'V' is an eigen vector of dimension R of matrix K with dimension R
* R. If KV and V are parallel. Then the user has to solve KV = PV where both V and
P are unknown for solving eigen vector and eigen value.

Eigen Value: It is also known as "characteristic roots" in PCA. This is used for
measuring the variance in all the variables of the set, which is reported for by that
factor. The proportion of eigen value is the ratio of descriptive importance of the
factors concerning the variables. If the factor is low, then it subsidises less to the
description of variables.

Now, we will Discuss Principal Component Analysis with Python.

Following are the Steps for Using PCA with Python:

In this tutorial, we will use [Link] Dataset.

Step 1: We will import the libraries.

1. import numpy as nmp

2. import [Link] as mpltl
3. import pandas as pnd

Step 2: We will import the dataset ([Link])

First, we will import the dataset and distribute it into X and Y components for data
analysis.

1. DS = pnd.read_csv('[Link]')
2.
3. # Now, we will distribute the dataset into two components "X" and "Y"
4.
5. X = [Link][: , 0:13].values
6. Y = [Link][: , 13].values

Step 3: In this step, we will split the dataset into the training set and testing set.

1 Tirumala engineering college

1
 Machine Learning with python

1. from sklearn.model_selection import train_test_split as tts

2.
3. X_train, X_test, Y_train, Y_test = tts(X, Y, test_size = 0.2, random_state = 0)

Step 4: Now, we will Feature Scaling.

In this step, we will do the re-processing on the training and testing set, for
example, fitting the standard scale.

1. from [Link] import StandardScaler as SS

2. SC = SS()
3.
4. X_train = SC.fit_transform(X_train)
5. X_test = [Link](X_test)

Step 5: Then, Apply the PCA function

We will apply the PCA function into the training set and testing set for analysis.

1. from [Link] import PCA

2.
3. PCa = PCA (n_components = 1)
4.
5. X_train = PCa.fit_transform(X_train)
6. X_test = [Link](X_test)
7.
8. explained_variance = PCa.explained_variance_ratio_

Step 6: Now, we will fit Logistic Regression for the training set

1. from sklearn.linear_model import LogisticRegression as LR

2.
3. classifier_1 = LR (random_state = 0)
4. classifier_1.fit(X_train, Y_train)

Output:

1 Tirumala engineering college

1
 Machine Learning with python

LogisticRegression(random_state=0)

Step 7: Here, we will predict the testing set result:

1. Y_pred = classifier_1.predict(X_test)

Step 8: We will create the confusion matrix.

1. from [Link] import confusion_matrix as CM

2.
3. c_m = CM (Y_test, Y_pred)

Step 9: Then, predict the result of the training set.

1. from [Link] import ListedColormap as LCM

2.
3. X_set, Y_set = X_train, Y_train
4. X_1, X_2 = [Link]([Link](start = X_set[:, 0].min() - 1,
5. stop = X_set[: , 0].max() + 1, step = 0.01),
6. [Link](start = X_set[: , 1].min() - 1,
7. stop = X_set[: , 1].max() + 1, step = 0.01))
8.
9. [Link](X_1, X_2, classifier_1.predict([Link]([X_1.ravel(),
10. X_2.ravel()]).T).reshape(X_1.shape), alpha = 0.75,
11. cmap = LCM (('yellow', 'grey', 'green')))
12.
13. [Link] (X_1.min(), X_1.max())
14. [Link] (X_2.min(), X_2.max())
15.
16. for s, t in enumerate([Link](Y_set)):
17. [Link](X_set[Y_set == t, 0], X_set[Y_set == t, 1],
18. c = LCM (('red', 'green', 'blue'))(s), label = t)
19.
20. [Link]('Logistic Regression for Training set: ')
21. [Link] ('PC_1') # for X_label

1 Tirumala engineering college

1
 Machine Learning with python

22. [Link] ('PC_2') # for Y_label

23. [Link]() # for showing legend
24.
25. # show scatter plot
26. [Link]()

Output:

Step 10: At last, we will visualize the result of the testing set.

1. from [Link] import ListedColormap as LCM

2.
3. X_set, Y_set = X_test, Y_test
4.
5. X_1, X_2 = [Link]([Link](start = X_set[: , 0].min() - 1,
6. stop = X_set[: , 0].max() + 1, step = 0.01),
7. [Link](start = X_set[: , 1].min() - 1,
8. stop = X_set[: , 1].max() + 1, step = 0.01))
9.
10. [Link](X_1, X_2, classifier_1.predict([Link]([X_1.ravel(),
11. X_2.ravel()]).T).reshape(X_1.shape), alpha = 0.75,
12. cmap = LCM(('pink', 'grey', 'aquamarine')))
13.
14. [Link](X_1.min(), X_1.max())
15. [Link](X_2.min(), X_2.max())

1 Tirumala engineering college

1
 Machine Learning with python

16.
17. for s, t in enumerate([Link](Y_set)):
18. [Link](X_set[Y_set == t, 0], X_set[Y_set == t, 1],
19. c = LCM(('red', 'green', 'blue'))(s), label = t)
20.
21. # title for scatter plot
22. [Link]('Logistic Regression for Testing set')
23. [Link] ('PC_1') # for X_label
24. [Link] ('PC_2') # for Y_label
25. [Link]()
26.
27. # show scatter plot
28. [Link]()

Output:

1 Tirumala engineering college

1