Scribd
Upload
2 views2 pages

KNN Algorithm Notes

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for classification and regression by predicting outputs based on the K nearest training examples. It is a lazy learning algorithm that relies on distance calculations, such as Euclidean distance, to determine the closest neighbors. While KNN is simple and effective for small datasets, it has disadvantages like high memory usage and sensitivity to irrelevant features.

Uploaded by

Anusha Av
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views2 pages

KNN Algorithm Notes

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for classification and regression by predicting outputs based on the K nearest training examples. It is a lazy learning algorithm that relies on distance calculations, such as Euclidean distance, to determine the closest neighbors. While KNN is simple and effective for small datasets, it has disadvantages like high memory usage and sensitivity to irrelevant features.

Uploaded by

Anusha Av
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

K-Nearest Neighbors (KNN) Algorithm -

NotebookLM Notes
1. Introduction
K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for both classification
and regression. It predicts the output for a new data point by examining the K nearest training
examples. It is called a lazy learning algorithm because it stores the training data instead of building
a model.

2. Learning Objectives
Define KNN; Explain how KNN works; Calculate Euclidean distance; Perform classification using
KNN; Choose an appropriate value of K; List advantages, disadvantages, and applications.

3. Types of Machine Learning


Supervised Learning: uses labeled data. Unsupervised Learning: discovers hidden patterns.
Reinforcement Learning: learns through rewards and penalties.

4. What is KNN?
KNN is a non-parametric, instance-based, distance-based, lazy learning algorithm. It predicts using
the nearest neighbors.

5. Workflow
Store training data → Receive new sample → Calculate distances → Sort distances → Select K
nearest neighbors → Majority vote (classification) or average (regression) → Prediction.

6. Euclidean Distance
Formula: d = sqrt((x2-x1)^2 + (y2-y1)^2). Commonly used to measure similarity between points.

7. Choosing K
Small K may overfit; large K may underfit. Odd values such as 3, 5, and 7 are commonly used.

8. Example
Training data: 2(Fail), 3(Fail), 5(Pass), 6(Pass), 7(Pass). New sample: 4, K=3. Nearest neighbors:
Fail, Pass, Pass. Prediction: Pass.

9. Advantages
Simple, easy to implement, no training phase, works for classification and regression, effective for
small datasets.

10. Disadvantages
Slow for large datasets, high memory usage, sensitive to irrelevant features and feature scaling,
affected by the curse of dimensionality.

11. Applications
Recommendation systems, image classification, handwriting recognition, face recognition, medical
diagnosis, fraud detection, spam filtering.

12. Python Example


from [Link] import KNeighborsClassifier; create model with n_neighbors=3; fit training
data; predict new sample.

13. Summary
KNN predicts by examining the K closest neighbors. It is simple and effective for small datasets, but
selecting the correct K is essential.

You might also like