Scribd
Upload
2 views9 pages

KNN Algorithm

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for classification and regression tasks, characterized as a lazy learner that stores the entire dataset and computes predictions at classification time. It involves calculating distances, selecting K nearest neighbors, and using majority voting for predictions, with advantages including ease of implementation and adaptability, but disadvantages such as inefficiency with large datasets and susceptibility to overfitting. KNN has various applications, including recommendation systems, finance forecasting, healthcare risk assessment, spam detection, customer segmentation, and speech recognition.

Uploaded by

jximhiadyjyxkcqgja
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views9 pages

KNN Algorithm

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for classification and regression tasks, characterized as a lazy learner that stores the entire dataset and computes predictions at classification time. It involves calculating distances, selecting K nearest neighbors, and using majority voting for predictions, with advantages including ease of implementation and adaptability, but disadvantages such as inefficiency with large datasets and susceptibility to overfitting. KNN has various applications, including recommendation systems, finance forecasting, healthcare risk assessment, spam detection, customer segmentation, and speech recognition.

Uploaded by

jximhiadyjyxkcqgja
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

KNN (K nearest Neighbor

algorithm
Tisha Cherian
Introduction
• Powerful supervised machine learning technique used for
both classification (generally) and regression tasks
• Classification ( predicting the class)
• Eg: Predicting Movie Genre- collect the data

• Predict – Barbie Movie- duration 114 and IMDb rating 7.4


• K-Nearest Neighbors is also called as
a lazy learner algorithm because it
does not learn from the training set
immediately instead, it stores the
entire dataset and performs
computations only at the time of
classification and make predictions
when needed
• Step 1: Calculate distance: Euclidean distance- distance between
the query point and the other data
• Step 2: Select K nearest neighbors- define the value of K-depends
on the input data. Data with more number of outliers or noise will
perform better with high values of K
• Use Odd numbers to minimise the chances of ties in classification
• Step 3: Majority Voting
Advantage
• Easy to implement
• Few hyper parameters- k value and distance metrics
• Adaptable – new data are added the algorithm adjusts to account
for any new data. The old data will be stored in memory
• Versatile: Works for both classification and regression problems.
Disadvantage

• Inability to Scale- as the data set grows, the algorithm becomes


less efficient due to increased computational complexity. Hence
KNN becomes a lazy algorithm (stores all training data and defers
the computation to the time of classification). Results in higher
usage and slow processing compared to other classifiers
• Slow with large data
• Curse of dimensionality- does not perform well with high-
dimensional data inputs ie, if more features are added (size, color,
wt etc)- the distance between the points becomes similar-
difficult to find meaningful neighbors- lead to peaking
phenomenon, ie, after reaching the optium no. of features (adding
more features just increase the noise and increase classification
errors
• Overfitting: lower values of K can overfit the data, higher values of
k tend to smooth out the prediction values- (struggles with many
features)
• Expensive
Applications
• K value is used for simple recommendation system
• KNN can be used for data pre-processing- missing data
• Finance- forecasting, money laundering analysis , stock
• Healthcare- risk of heart attacks, prostate cancer,
• Recommendation Systems: Suggests items like movies or products
by finding users with similar preferences.
• Spam Detection: Identifies spam emails by comparing new emails to
known spam and non-spam examples.
• Customer Segmentation: Groups customers by comparing their
shopping behavior to others.
• Speech Recognition: Matches spoken words to known patterns to
convert them into text.

You might also like