Unsupervised Machine Learning Models

➢ Unsupervised machine learning is a broad category of algorithms that aim to discover patterns, structures, or
relationships in data without the use of labeled training examples.
➢ While clustering is one common task within unsupervised learning, there are other types of unsupervised learning
tasks as well. Here are some examples:

➢ Clustering
➢ Dimensionality Reduction
➢ Anomaly Detection
➢ Density Estimation
➢ Generative Modeling
➢ Association Rule Learning
 Clustering Types
1. K-Means Clustering: (Example: Grouping customers into segments based on their purchase behavior.)

2. Hierarchical Clustering: (Example: Taxonomy in biology, where species are grouped into hierarchical categories.)

3. DBSCAN (Density-Based Spatial Clustering of Applications with Noise): (Example: Identifying hotspots of criminal activity in
a city based on crime incident data.)

4. Mean Shift: (Example: Object tracking in computer vision.)

5. Gaussian Mixture Model (GMM): (Example: Segmenting a dataset of various audio signals to identify different sound sources.)

6. Self-Organizing Maps (SOM): (Example: Analyzing and visualizing high-dimensional data like images or genetic data.)

7. OPTICS (Ordering Points To Identify the Clustering Structure): (Example: Identifying clusters in a large dataset with varying
densities, such as network traffic analysis.)

8. Affinity Propagation: (Example: Clustering news articles into topics for news recommendation systems.)

9. BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies): (Example: Clustering customer data for market
segmentation in retail.)
 1. K-Means Clustering
• K-means clustering is an iterative algorithm used to partition a set of data points into K clusters. The algorithm's goal
is to minimize the sum of squared distances between data points and their respective cluster centroids. Here are the
mathematical calculations and steps behind K-means clustering.

• K-means clustering is an iterative algorithm used to partition a set of data points into K clusters. The algorithm's goal
is to minimize the sum of squared distances between data points and their respective cluster centroids. Here are the
mathematical calculations and steps behind K-means clustering.

•K: The number of clusters.

•N: The number of data points.
•xi​: The i-th data point in N-dimensional space.
•ck​: The centroid of cluster k.
•Sk​: The set of data points assigned to cluster k.
•d(x,y): The Euclidean distance between points x and y.
• Example

• A1 (2, 10),
• A2(2, 5),
• A3 (8, 4),
• B1(5, 8),
• B2(7, 5),
• B3(6, 4),
• C1(1, 2),
• C2(4, 9)
• A1 (2, 10), A1
• A2(2, 5), C2
• A3 (8, 4),
B1
• B1(5, 8),
• B2(7, 5), A2
B2
• B3(6, 4), B3
• C1(1, 2), A3
C1
• C2(4, 9)
• A1 (2, 10),
• A2(2, 5), Distance to Centroid New
Cluster
• A3 (8, 4), Cluster
• B1(5, 8), A1 2 10
• B2(7, 5), A2 2 5
• B3(6, 4), A3 8 4
• C1(1, 2), B1 5 8
• C2(4, 9) B2 7 5
B3 6 4
C1 1 2
C2 4 9
• A1 (2, 10),
• A2(2, 5), Distance to Centroid New
Cluster
• A3 (8, 4), 2 10 5 8 1 2 Cluster
• B1(5, 8), A1 2 10
• B2(7, 5), A2 2 5
• B3(6, 4), A3 8 4
• C1(1, 2), B1 5 8
• C2(4, 9) B2 7 5
B3 6 4
C1 1 2
C2 4 9
• A1 (2, 10),
• A2(2, 5), Distance to Centroid New
Cluster
• A3 (8, 4), 2 10 5 8 1 2 Cluster
• B1(5, 8), A1 2 10 0 3.61 8.06
• B2(7, 5), A2 2 5 5.0 4.24 3.16
• B3(6, 4), A3 8 4 8.49 5 7.28
• C1(1, 2), B1 5 8 3.61 0 7.21
• C2(4, 9) B2 7 5 7.07 3.61 6.71
B3 6 4 7.21 4.12 5.39
C1 1 2 8.06 7.21 0.00
C2 4 9 2.24 1.41 7.62
• A1 (2, 10),
• A2(2, 5), Distance to Centroid New
Cluster
• A3 (8, 4), 2 10 5 8 1 2 Cluster
• B1(5, 8), A1 2 10 0 3.61 8.06 1
• B2(7, 5), A2 2 5 5.0 4.24 3.16 3
• B3(6, 4), A3 8 4 8.49 5 7.28 2
• C1(1, 2), B1 5 8 3.61 0 7.21 2
• C2(4, 9) B2 7 5 7.07 3.61 6.71 2
B3 6 4 7.21 4.12 5.39 2
C1 1 2 8.06 7.21 0.00 3
C2 4 9 2.24 1.41 7.62 2
• New Centroids
Distance to Centroid New
Cluster
• A1 (2, 10), 2 10 6 6 1.5 3.5 Cluster
A1 2 10
• B1 (6, 6),
A2 2 5
• C1 (1.5, 3.5), A3 8 4
B1 5 8
B2 7 5
B3 6 4
C1 1 2
C2 4 9
• New Centroids
Distance to Centroid New
Cluster
• A1 (2, 10), 2 10 6 6 1.5 3.5 Cluster
A1 2 10 1
• B1 (6, 6),
A2 2 5 3
• C1 (1.5, 3.5), A3 8 4 2
B1 5 8 2
B2 7 5 2
B3 6 4 2
C1 1 2 3
C2 4 9 2
• New Centroids
Distance to Centroid New
Cluster
• A1 (2, 10), 2 10 6 6 1.5 3.5 Cluster
A1 2 10 0 5.66 6.52 1 1
• B1 (6, 6),
A2 2 5 5 4.12 1.58 3 3
• C1 (1.5, 3.5), A3 8 4 8.49 2.83 6.52 2 2
B1 5 8 3.61 2.24 5.70 2 2
B2 7 5 7.01 1.41 5.70 2 2
B3 6 4 7.21 2.00 4.53 2 2
C1 1 2 8.06 6.40 1.58 3 3
C2 4 9 2.24 3.61 6.04 2 1
• New Centroids
Distance to Centroid New
Cluster
• A1 (3, 9.5), 3 9.5 6.5 5.25 1.5 3.5 Cluster
A1 2 10
• B1 (6.5, 5.25),
A2 2 5
• C1 (1.5, 3.5), A3 8 4
B1 5 8
B2 7 5
B3 6 4
C1 1 2
C2 4 9
• New Centroids
Distance to Centroid New
Cluster
• A1 (3, 9.5), 3 9.5 6.5 5.25 1.5 3.5 Cluster
A1 2 10 1.12 6.54 6.52 1
• B1 (6.5, 5.25),
A2 2 5 4.61 4.51 1.58 3
• C1 (1.5, 3.5), A3 8 4 7.43 1.95 6.52 2
B1 5 8 2.50 3.13 5.70 2
B2 7 5 6.02 0.56 5.70 2
B3 6 4 6.26 1.35 4.53 2
C1 1 2 7.76 6.39 1.58 3
C2 4 9 1.12 4.51 6.04 1
• New Centroids
Distance to Centroid New
Cluster
• A1 (3, 9.5), 3 9.5 6.5 5.25 1.5 3.5 Cluster
A1 2 10 1.12 6.54 6.52 1 1
• B1 (6.5, 5.25),
A2 2 5 4.61 4.51 1.58 3 3
• C1 (1.5, 3.5), A3 8 4 7.43 1.95 6.52 2 2
B1 5 8 2.50 3.13 5.70 2 1
B2 7 5 6.02 0.56 5.70 2 2
B3 6 4 6.26 1.35 4.53 2 2
C1 1 2 7.76 6.39 1.58 3 3
C2 4 9 1.12 4.51 6.04 1 1
• New Centroids
Distance to Centroid New
Cluster
• A1 (3.67, 9), 3.67 9 7 4.33 1.5 3.5 Cluster
A1 2 10
• B1 (7, 4.33),
A2 2 5
• C1 (1.5, 3.5), A3 8 4
B1 5 8
B2 7 5
B3 6 4
C1 1 2
C2 4 9
• New Centroids
Distance to Centroid New
Cluster
• A1 (3.67, 9), 3.67 9 7 4.33 1.5 3.5 Cluster
A1 2 10 1.94 7.56 6.52 1
• B1 (7, 4.33),
A2 2 5 4.33 5.04 1.58 3
• C1 (1.5, 3.5), A3 8 4 6.62 1.05 6.52 2
B1 5 8 1.67 4.18 5.70 1
B2 7 5 5.21 0.67 5.70 2
B3 6 4 5.52 1.05 4.53 2
C1 1 2 7.49 6.44 1.58 3
C2 4 9 0.33 5.55 6.04 1
• New Centroids
Distance to Centroid New
Cluster
• A1 (3.67, 9), 3.67 9 7 4.33 1.5 3.5 Cluster
A1 2 10 1.94 7.56 6.52 1 1
• B1 (7, 4.33),
A2 2 5 4.33 5.04 1.58 3 3
• C1 (1.5, 3.5), A3 8 4 6.62 1.05 6.52 2 2
B1 5 8 1.67 4.18 5.70 1 1
B2 7 5 5.21 0.67 5.70 2 2
• New Centroids B3 6 4 5.52 1.05 4.53 2 2
• A1 (3.67, 9), C1 1 2 7.49 6.44 1.58 3 3
C2 4 9 0.33 5.55 6.04 1 1
• B1 (7, 4.33),

• C1 (1.5, 3.5),
• A1 (2, 10),
• A2(2, 5),
• A3 (8, 4),
• B1(5, 8),
• B2(7, 5),
• B3(6, 4),
• C1(1, 2),
• C2(4, 9)
Elbow Method for Knowing the K in K Means Clustering
1. 3192.6,
2. 1515.4,
3. 887.1,
4. 541.0,
5. 478.9,
6. 423.6,
7. 367.2,
8. 316.5,
9. 285.5
Elbow Method for Knowing the K in K Means Clustering

19953.76,
9416.21,
2110.41,
564.91,
513.97,
460.91,
412.20,
364.38,
324.59,
298.08
Elbow Method for Knowing the K in K Means Clustering

• A1 (2, 10),
• A2(2, 5),
• A3 (8, 4),
• B1(5, 8),
• B2(7, 5),
• B3(6, 4),
• C1(1, 2),
• C2(4, 9)
K=1, Inertia=100.75
K=2, Inertia=51.47
K=3, Inertia=14.33
K=4, Inertia=8.67
K=5, Inertia=3.67
K=6, Inertia=2.00
 Hierarchical Clustering
➢ Also called Hierarchical cluster analysis or HCA is an unsupervised clustering algorithm which involves
creating clusters that have predominant ordering from top to bottom

➢ The algorithm groups similar objects into groups called clusters.

The endpoint is a set of clusters or groups, where each cluster is
distinct from each other cluster, and the objects within each
cluster are broadly similar to each other.

1. Agglomerative Hierarchical Clustering

2. Divisive Hierarchical Clustering
 Agglomerative Hierarchical Clustering
• The Agglomerative Hierarchical Clustering is the most common type of hierarchical clustering used to group objects
in clusters based on their similarity. It’s also known as AGNES (Agglomerative Nesting). It's a “bottom-up”
approach: each observation starts in its own cluster, and pairs of clusters are merged as one moves up the
hierarchy

Steps to follow

1. Make each data point a single-point cluster → forms N clusters

2. Take the two closest data points and make them one cluster → forms N-1 clusters
3. Take the two closest clusters and make them one cluster → Forms N-2 clusters.
4. Repeat step-3 until you are left with only one cluster.
Agglomerative Hierarchical Clustering…
Agglomerative Hierarchical Clustering…
 Agglomerative Hierarchical Clustering…
Hierarchical agglomerative clustering with 12 data points that form 3 clusters.

Data points (X, Y coordinates): Let's consider the following data points:

1. (2, 3)
2. (3, 3)
3. (3, 4)
4. (4, 4)
5. (6, 2)
6. (6, 3)
7. (7, 3)
8. (7, 4)
9. (8, 4)
10. (10, 5)
11. (11, 5)
12. (11, 6)
 1 (2, 3) 2(3,3) 3(3, 4) 4(4, 4) 5(6, 2) 6(6, 3) 7(7, 3) 8(7, 4) 9(8, 4) 10(10,5) 11(11,5) 12(11,6)

1(2,3) 0

2(3, 3) 0

3 (3, 4) 0

4 (4, 4) 0

5 (6, 2) 0

6 (6, 3) 0

7 (7, 3) 0

8 (7, 4) 0

9 (8, 4) 0

10 (10,5) 0

11 (11,5) 0

12 (11,6) 0
 1 (2, 3) 2(3,3) 3(3, 4) 4(4, 4) 5(6, 2) 6(6, 3) 7(7, 3) 8(7, 4) 9(8, 4) 10(10,5) 11(11,5) 12(11,6)

1(2,3) 0.0 1.0 1.4 2.2 4.1 4.0 5.0 5.1 6.1 8.2 9.2 9.5
2(3, 3) 1.0 0.0 1.0 1.4 3.2 3.0 4.0 4.1 5.1 7.3 8.2 8.5
3 (3, 4) 1.4 1.0 0.0 1.0 3.6 3.2 4.1 4.0 5.0 7.1 8.1 8.2
4 (4, 4) 2.2 1.4 1.0 0.0 2.8 2.2 3.2 3.0 4.0 6.1 7.1 7.3
5 (6, 2) 4.1 3.2 3.6 2.8 0.0 1.0 1.4 2.2 2.8 5.0 5.8 6.4
6 (6, 3) 4.0 3.0 3.2 2.2 1.0 0.0 1.0 1.4 2.2 4.5 5.4 5.8
7 (7, 3) 5.0 4.0 4.1 3.2 1.4 1.0 0.0 1.0 1.4 3.6 4.5 5.0
8 (7, 4) 5.1 4.1 4.0 3.0 2.2 1.4 1.0 0.0 1.0 3.2 4.1 4.5
9 (8, 4) 6.1 5.1 5.0 4.0 2.8 2.2 1.4 1.0 0.0 2.2 3.2 3.6
10 (10,5) 8.2 7.3 7.1 6.1 5.0 4.5 3.6 3.2 2.2 0.0 1.0 1.4
11 (11,5) 9.2 8.2 8.1 7.1 5.8 5.4 4.5 4.1 3.2 1.0 0.0 1.0
12 (11,6) 9.5 8.5 8.2 7.3 6.4 5.8 5.0 4.5 3.6 1.4 1.0 0.0
 1 (2, 3) 2(3,3) 3(3, 4) 4(4, 4) 5(6, 2) 6(6, 3) 7(7, 3) 8(7, 4) 9(8, 4) 10(10,5) 11(11,5) 12(11,6)

1(2,3) 0.0
2(3, 3) 1.0 0.0
3 (3, 4) 1.4 1.0 0.0
4 (4, 4) 2.2 1.4 1.0 0.0
5 (6, 2) 4.1 3.2 3.6 2.8 0.0
6 (6, 3) 4.0 3.0 3.2 2.2 1.0 0.0
7 (7, 3) 5.0 4.0 4.1 3.2 1.4 1.0 0.0
8 (7, 4) 5.1 4.1 4.0 3.0 2.2 1.4 1.0 0.0
9 (8, 4) 6.1 5.1 5.0 4.0 2.8 2.2 1.4 1.0 0.0
10 (10,5) 8.2 7.3 7.1 6.1 5.0 4.5 3.6 3.2 2.2 0.0
11 (11,5) 9.2 8.2 8.1 7.1 5.8 5.4 4.5 4.1 3.2 1.0 0.0
12 (11,6) 9.5 8.5 8.2 7.3 6.4 5.8 5.0 4.5 3.6 1.4 1.0 0.0
 Other Example

One Dimensional data set [1, 5, 8, 10, 2]

 Divisive Hierarchical Clustering

• In Divisive or DIANA(DIvisive ANAlysis

Clustering) is a top-down clustering method
where we assign all of the observations to a
single cluster and then partition the cluster
to two least similar clusters. Finally, we
proceed recursively on each cluster until
there is one cluster for each observation. So
this clustering approach is exactly opposite
to Agglomerative clustering
Divisive Hierarchical Clustering

Problem:
Suppose you have the following data points:

Data Points (8 points):

1. (2, 2)
2. (2, 4)
3. (3, 3)
4. (5, 2)
5. (5, 4)
6. (6, 3)
7. (8, 2)
8. (8, 4)
 Initial Cluster: [1, 2, 3, 4, 5, 6, 7, 8]
1. (2, 2)
➢ Centroid: (4.875, 3.0) 2. (2, 4)
3. (3, 3)
4. (5, 2)
5. (5, 4)
Distance from Centroid: 6. (6, 3)
7. (8, 2)
1. 3.04 8. (8, 4)
2. 3.04
3. 1.88
4. 1.01
Subcluster 1: [1,2,7,8]
5. 1.01 Subcluster 2: [3,4,5,6]
6. 1.13
7. 3.28
8. 3.28
DBSCAN (Density-Based Spatial Clustering of Applications with Noise):
DBSCAN (Density-Based Spatial Clustering of Applications with Noise):

P1: (3, 7)
P2: (4,6)
P1: (3, 7)
P3: (5,5)
P4: (6,4) P2: (4,6)
P10: (2,6)
P5: (7,3) Create the cluster with
P3: (5,5)
minPts = 4 P11: (3,5)
P6: (6,2)
Epsilon (e) = 1.9 P4: (6,4)
P7: (7,2) P12: (2,4) P8: (8,4)

P8: (8,4) P5: (7,3)

P9: (3,3)
P9: (3,3)
P6: (6,2) P7: (7,2)
P10: (2,6)
P11: (3,5)
P12: (2,4)
DBSCAN (Density-Based Spatial Clustering of Applications with Noise):

P1: (3, 7) P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

P1 0.0 1.4 2.8 4.2 5.7 5.8 6.4 5.8 4.0 1.4 2.0 3.2
P2: (4,6)
P2 1.4 0.0 1.4 2.8 4.2 4.5 5.0 4.5 3.2 2.0 1.4 2.8
P3: (5,5)
P3 2.8 1.4 0.0 1.4 2.8 3.2 3.6 3.2 2.8 3.2 2.0 3.2
P4: (6,4)
P4 4.2 2.8 1.4 0.0 1.4 2.0 2.2 2.0 3.2 4.5 3.2 4.0
P5: (7,3)
P5 5.7 4.2 2.8 1.4 0.0 1.4 1.0 1.4 4.0 5.8 4.5 5.1
P6: (6,2)
P6 5.8 4.5 3.2 2.0 1.4 0.0 1.0 2.8 3.2 5.7 4.2 4.5
P7: (7,2)
P7 6.4 5.0 3.6 2.2 1.0 1.0 0.0 2.2 4.1 6.4 5.0 5.4
P8: (8,4)
P8 5.8 4.5 3.2 2.0 1.4 2.8 2.2 0.0 5.1 6.3 5.1 6.0
P9: (3,3) P9 4.0 3.2 2.8 3.2 4.0 3.2 4.1 5.1 0.0 3.2 2.0 1.4
P10: (2,6) P10 1.4 2.0 3.2 4.5 5.8 5.7 6.4 6.3 3.2 0.0 1.4 2.0
P11: (3,5) P11 2.0 1.4 2.0 3.2 4.5 4.2 5.0 5.1 2.0 1.4 0.0 1.4
P12: (2,4) P12 3.2 2.8 3.2 4.0 5.1 4.5 5.4 6.0 1.4 2.0 1.4 0.0
DBSCAN (Density-Based Spatial Clustering of Applications with Noise):

P1: (3, 7) P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P1: P2, P10

P1 0.0 1.4 2.8 4.2 5.7 5.8 6.4 5.8 4.0 1.4 2.0 3.2
P2: (4,6) P2: P1, P3, P11
P2 1.4 0.0 1.4 2.8 4.2 4.5 5.0 4.5 3.2 2.0 1.4 2.8
P3: (5,5) P3: P3, P5
P3 2.8 1.4 0.0 1.4 2.8 3.2 3.6 3.2 2.8 3.2 2.0 3.2
P4: (6,4) P4: P3, P5
P4 4.2 2.8 1.4 0.0 1.4 2.0 2.2 2.0 3.2 4.5 3.2 4.0
P5: (7,3) P5: P4, P6, P7, P8
P5 5.7 4.2 2.8 1.4 0.0 1.4 1.0 1.4 4.0 5.8 4.5 5.1
P6: (6,2) P6: P5, P7
P6 5.8 4.5 3.2 2.0 1.4 0.0 1.0 2.8 3.2 5.7 4.2 4.5
P7: (7,2) P7: P5, P6
P7 6.4 5.0 3.6 2.2 1.0 1.0 0.0 2.2 4.1 6.4 5.0 5.4
P8: (8,4) P8: P5
P8 5.8 4.5 3.2 2.0 1.4 2.8 2.2 0.0 5.1 6.3 5.1 6.0
P9: (3,3) P9 4.0 3.2 2.8 3.2 4.0 3.2 4.1 5.1 0.0 3.2 2.0 1.4
P9: P12
P10: (2,6) P10 1.4 2.0 3.2 4.5 5.8 5.7 6.4 6.3 3.2 0.0 1.4 2.0 P10: P1, P11
P11: (3,5) P11 2.0 1.4 2.0 3.2 4.5 4.2 5.0 5.1 2.0 1.4 0.0 1.4 P11: P2, P10, P12
P12: (2,4) P12 3.2 2.8 3.2 4.0 5.1 4.5 5.4 6.0 1.4 2.0 1.4 0.0 P12: P9, P11
DBSCAN (Density-Based Spatial Clustering of Applications with Noise):
P1: P2, P10
P2: P1, P3, P11
P1: (3, 7)
P3: P3, P5
P4: P3, P5
P10: (2,6) P2: (4,6)
P5: P4, P6, P7, P8
P3: (5,5)
P6: P5, P7 P11: (3,5)

P7: P5, P6 P12: (2,4) P4: (6,4) P8: (8,4)

P8: P5
P5: (7,3)
P9: (3,3)
P9: P12
P7: (7,2)
P10: P1, P11 P6: (6,2)

P11: P2, P10, P12

P12: P9, P11
 P1: (3, 7)

P10: (2,6) P2: (4,6)

P3: (5,5)
P11: (3,5)

P4: (6,4)
P12: (2,4) P8: (8,4)

P5: (7,3)
P9: (3,3)

P7: (7,2)
P6: (6,2)
 Silhouette (clustering) / Coefficient
• Silhouette refers to a method of interpretation and validation of consistency within clusters of data. The technique
provides a succinct graphical representation of how well each object has been classified.[1] It was proposed by
Belgian statistician Peter Rousseeuw in 1987.
• The silhouette value is a measure of how similar an object is to its own cluster (cohesion) compared to other clusters
(separation). The silhouette ranges from −1 to +1, where a high value indicates that the object is well matched to its
own cluster and poorly matched to neighboring clusters
Silhouette (clustering) / Coefficient

Silhouette Score: 0.5011

Silhouette (clustering) / Coefficient

Silhouette Score: 0.7916

Principal Component Analysis

● Principal Component Analysis (PCA) is a statistical procedure

that uses a orthogonal transformation that converts a set of
correlated variables to a set of uncorrelated variables.
● Also used to examine the interrelations among a set of
variables.
1. Principal Component Analysis (PCA) is a technique for dimensionality reduction that identifies
a set of orthogonal axes, called principal components, that capture the maximum variance in
the data. The principal components are linear combinations of the original variables in the
dataset and are ordered in decreasing order of importance. The total variance captured by all
the principal components is equal to the total variance in the original dataset.
2. The first principal component captures the most variation in the data, but the second
principal component captures the maximum variance that is orthogonal to the first principal
component
Step-By-Step Explanation of PCA (Principal Component Analysis)

Step 1: Standardization
First, we need to standardize our dataset to ensure that each variable has a mean
of 0 and a standard deviation of 1.
Step2: Covariance Matrix Computation
Covariance measures the strength of joint variability between two or
more variables, indicating how much they change in relation to each
other.
The value of covariance can be positive, negative, or zeros.
● Positive: As the x1 increases x2 also increases.
● Negative: As the x1 increases x2 also decreases.
● Zeros: No direct relation
Step 3: Compute Eigenvalues and Eigenvectors of Covariance Matrix to Identify
Principal Components

Let A be a square nXn matrix and X be a non-zero vector for which :

Ax = λx

Here, λ is the eigenvalue of matrix A and X is the eigenvector of matrix A

 How Principal Component Analysis(PCA) works?

PCA employs a linear transformation that is based on preserving the

most variance in the data using the least number of dimensions. It
involves the following steps:
Original
Dataframe shape
: (569, 31)
Inputs
Dataframe
shape :
(569, 30)
Now we will apply the first most step which is to standardize the data and for that, we
will have to first calculate the mean and standard deviation of each feature in the
feature space

# Mean
X_mean = [Link]()

# Standard deviation
X_std = [Link]()

# Standardization
Z = (X - X_mean) / X_std
The covariance matrix helps us visualize how strong the dependency of two features is
with each other in the feature space

# covariance
c = [Link]()

# Plot the covariance matrix

import [Link] as plt
import seaborn as sns
[Link](c)
[Link]()
Computing the eigenvalues and vectors

eigenvalues, eigenvectors = [Link](c)

print('Eigen values:\n', eigenvalues)
print('Eigen values Shape:', [Link])
print('Eigen Vector Shape:', [Link])
Output:
Eigen values:
[1.32816077e+01 5.69135461e+00 2.81794898e+00 1.98064047e+00
1.64873055e+00 1.20735661e+00 6.75220114e-01 4.76617140e-01
4.16894812e-01 3.50693457e-01 2.93915696e-01 2.61161370e-01
2.41357496e-01 1.57009724e-01 9.41349650e-02 7.98628010e-02
5.93990378e-02 5.26187835e-02 4.94775918e-02 1.33044823e-04
7.48803097e-04 1.58933787e-03 6.90046388e-03 8.17763986e-03
1.54812714e-02 1.80550070e-02 2.43408378e-02 2.74394025e-02
3.11594025e-02 2.99728939e-02]
Eigen values Shape: (30,)
Eigen Vector Shape: (30, 30)
Dimensionality Reduction

Dimensionality reduction is a technique used to reduce the number of

features in a dataset while retaining as much of the important information
as possible. In simple its a process of transforming high-dimensional data
into a lower-dimensional space that still preserves the essence of the
original data.
There are two main approaches to dimensionality reduction:
● Feature selection
● Feature extraction.
● Feature extraction: This reduces the data in a high dimensional space to a lower

dimension space, i.e. a space with lesser no. of dimensions.

● Feature selection: We try to find a subset of the original set of variables, or features, to
get a smaller subset which can be used to model the problem
Methods of Dimensionality Reduction

The various methods used for dimensionality reduction include:

● Principal Component Analysis (PCA)

● Linear Discriminant Analysis (LDA)
● Generalized Discriminant Analysis (GDA)