Scribd
Upload
15 views14 pages

KNN Algorithm: Distance Metrics & Usage

The K Nearest Neighbour (KNN) algorithm is a supervised machine learning method primarily used for classification tasks, characterized as a lazy and non-parametric learning algorithm. It calculates distances using various metrics such as Euclidean, Manhattan, and Hamming to determine the nearest neighbors, and the choice of K is crucial for its performance, typically suggested as k = sqrt(N)/2. The algorithm involves loading data, calculating distances, sorting them, and then determining the output based on the nearest K entries.

Uploaded by

sravane1608
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
15 views14 pages

KNN Algorithm: Distance Metrics & Usage

The K Nearest Neighbour (KNN) algorithm is a supervised machine learning method primarily used for classification tasks, characterized as a lazy and non-parametric learning algorithm. It calculates distances using various metrics such as Euclidean, Manhattan, and Hamming to determine the nearest neighbors, and the choice of K is crucial for its performance, typically suggested as k = sqrt(N)/2. The algorithm involves loading data, calculating distances, sorting them, and then determining the output based on the nearest K entries.

Uploaded by

sravane1608
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd

K Nearest Neighbour Algorithm

• Introduction

• Euclidean, Manhattan or Hamming distance

• Working of KNN Algorithm

• Choice of K
INTRODUCTION
• K-nearest neighbors (KNN) algorithm is a type of supervised ML
algorithm which can be used for both classification as well as regression
predictive problems.
• However, it is mainly used for classification predictive problems in
industry.
• Lazy learning algorithm − KNN is a lazy learning algorithm because it
does not have a specialized training phase and uses all the data for
training while classification.
• Non-parametric learning algorithm − KNN is also a non-parametric
learning algorithm because it doesn’t assume anything about the
underlying data.
Similarity Distance Measure
• The similarity measure is the measure of how much alike two data objects are.
Similarity measure in a data mining context is a distance with dimensions
representing features of the objects. If this distance is small, it will be the high
degree of similarity where large distance will be the low degree of similarity.
• Two main consideration about similarity:
• Similarity = 1 if X = Y (Where X, Y are two objects)
• Similarity = 0 if X ≠ Y
Minkowski distance
• The distance can be calculated using the below formula:-

Some common values of ‘p’ are:-


•p = 1, Manhattan Distance
•p = 2, Euclidean Distance
•p = infinity, Chebychev Distance
Euclidean Distance:
• Euclidean distance is the straight line distance between 2 data points
in a plane.

• It is calculated using the Minkowski Distance formula by


Euclidean Distance
Actual data set with target variable
(class) & independent variables

X1 X2 DISTANCE CLASS
8 4 1 A
6 5 2 A
5 6 8 B
7 7 9 A
4 5 10 B
4 6 13 B

We take a point (7,4)


Manhattan Distance:
• Distance between two data points in a grid-like path

X1 X2 Distance CLASS
8 4 1 A
6 5 2 A
7 7 3 A
5 6 4 B
4 5 4 B
4 6 5 B
Hamming Distance
KNN Algorithm: Working
• Load the data
• Initialize K to your chosen number of neighbors
• For each example in the data
• Calculate the distance between the query example and the current example
from the data.
• Add the distance and the index of the example to an ordered collection
• Sort the ordered collection of distances and indices from smallest to largest (in
ascending order) by the distances
• Pick the first K entries from the sorted collection
• Get the labels of the selected K entries
• If regression, return the mean of the K labels
• If classification, return the mode of the K labels
• k is non-parametric and a general rule of thumb in choosing the value of k is k
= sqrt(N)/2, where N stands for the number of samples in your training
CHOICE OF K
• k is non-parametric and a general rule of thumb in choosing the value
of k is k = sqrt(N)/2, where N stands for the number of samples in
your training dataset.
• Generally k is odd to avoid tie up.
• [Link]
-to-choose-k-value-in-knn-algo.18299/
• Accuracy score plot
CLASSIFICATION USING KNN

You might also like