Scribd
Upload
37 views6 pages

Example KNN Numerical Problem

The document explains the K-Nearest Neighbors (KNN) algorithm for classifying a new point based on its distance to existing points, using a dataset with specific classes. It also covers K-Means and K-Modes clustering techniques, highlighting their differences, applications, and steps involved in the algorithms. K-Means is used for numerical data, while K-Modes is designed for categorical data, focusing on grouping similar data points without labeled information.

Uploaded by

UTKARSH MISHRA
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
37 views6 pages

Example KNN Numerical Problem

The document explains the K-Nearest Neighbors (KNN) algorithm for classifying a new point based on its distance to existing points, using a dataset with specific classes. It also covers K-Means and K-Modes clustering techniques, highlighting their differences, applications, and steps involved in the algorithms. K-Means is used for numerical data, while K-Modes is designed for categorical data, focusing on grouping similar data points without labeled information.

Uploaded by

UTKARSH MISHRA
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Example KNN Numerical Problem

A dataset contains the following points with their classes:

Point X Y Class

P1 1 2 A

P2 2 3 A

P3 3 3 B

P4 6 5 B

P5 7 7 B

Classify the new point P = (3,4) using KNN with k = 3.


Use Euclidean Distance.

Step 1: Euclidean Distance Formula

Step 2: Calculate Distance from P(3,4)

Distance to P1 (1,2)

Distance to P2 (2,3)
Distance to P3 (3,3)

Distance to P4 (6,5)

Distance to P5 (7,7)

Step 3: Sort Distances

Point Distance Class

P3 1.00 B

P2 1.41 A

P1 2.83 A

P4 3.16 B

P5 5.00 B

Step 4: Select k = 3 Nearest Neighbors

Nearest points:

 P3 → B

 P2 → A
 P1 → A

Votes:

 A=2

 B=1

Clustering

Clustering is an unsupervised learning technique used to group similar data points together.

 Data points in the same cluster are similar

 Data points in different clusters are dissimilar

Unlike supervised learning, clustering does not use labeled data.

K – Means Clustering

K-Means Clustering groups similar data points into clusters without needing labeled data. It is used to
uncover hidden patterns when the goal is to organize data based on similarity.

 Helps identify natural groupings in unlabeled datasets

 Works by grouping points based on distance to cluster centers

 Commonly used in customer segmentation, image compression, and pattern discovery

 Useful when you need structure from raw, unorganized data


Working of K-Means Clustering
Steps of K-Means Algorithm

1. Choose number of clusters K

2. Select K random centroids

3. Assign each data point to the nearest centroid

4. Recalculate new centroids

5. Repeat steps 3–4 until clusters stop changing.

K – Mode Clustering

K-mode clustering is an unsupervised machine-learning used to group categorical data into k clusters
(groups). The K-Modes clustering partitions the data into two mutually exclusive groups. Unlike K-
Means which uses distances between numbers K-Modes uses the number of mismatches between
categorical values to decide how similar two data points are. For example:

 Data point 1: ["red", "small", "round"]

 Data point 2: ["blue", "small", "square"]

Note: K-Modes clustering is an extension of K-Means designed for categorical data.

K-Means works with numerical values, but K-Modes works with categories.

Steps

1. Select K modes

2. Assign data points to nearest mode

3. Update cluster mode


4. Repeat until convergence

Advantages

 Works well with categorical datasets

 Efficient for large categorical data

Example Applications

 Market segmentation

 Survey data analysis

 Recommendation systems

Difference Between K-Means and K-Modes

Feature K-Means K-Modes

Data Type Numerical Categorical

Center Mean Mode

Distance Euclidean Matching dissimilarity

Example Age, Salary Gender, City

You might also like