0% found this document useful (0 votes)
12 views57 pages

Unit - 5 Unsupervised Learning

The document discusses Unsupervised Learning in machine learning, focusing on clustering techniques such as K-Means and Hierarchical Clustering, their definitions, advantages, and applications. It highlights the importance of clustering in data analysis for pattern discovery, data simplification, and anomaly detection, along with challenges faced in unsupervised learning. Additionally, it outlines the steps involved in K-Means clustering and the hierarchical approach to clustering, including agglomerative and divisive methods.

Uploaded by

sanchita2303101
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
12 views57 pages

Unit - 5 Unsupervised Learning

The document discusses Unsupervised Learning in machine learning, focusing on clustering techniques such as K-Means and Hierarchical Clustering, their definitions, advantages, and applications. It highlights the importance of clustering in data analysis for pattern discovery, data simplification, and anomaly detection, along with challenges faced in unsupervised learning. Additionally, it outlines the steps involved in K-Means clustering and the hierarchical approach to clustering, including agglomerative and divisive methods.

Uploaded by

sanchita2303101
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Machine Learning

Unit -5 Unsupervised Learning -16 Marks

MAL - 316316 Mrs. Supriya [Link]


TY – Computer Engineering Computer Engineering Dept.
MAL-316316
MAL-316316

Unit –V Unsupervised Learning


5.1 Clustering Techniques:-Define Clustering, Importance of clustering in
data analysis, Applications of Clustering
5.2 K-Means Clustering: Definition and working principle, Steps involved in
the K-Means algorithm, Advantages of K-Means, Disadvantages of K-Means,
Hierarchical Clustering: Definition and types, Steps in Hierarchical Clustering,
Advantages of Hierarchical Clustering, Disadvantages of Hierarchical
Clustering, Comparing K-Means and Hierarchical Clustering
5.3 Dimensionality Reduction: Importance of Dimensionality Reduction
5.4 PCA -Definition and fundamental principles of PCA, Eigenvectors and
eigenvalues, Steps in PCA, Explained Variance, Choosing the Optimal
Dimensionality, Advantages and Disadvantages of PCA, Applications of PCA
5.0 Unsupervised Learning
• Unsupervised Learning is a type of machine learning
where the model works without labelled data. It
learns patterns on its own by grouping similar data
points or finding hidden structures without any
human intervention.
• It is used for tasks like clustering, dimensionality
reduction and Association Rule Learning.
• Helps identify hidden patterns in data
• Useful for grouping, compression and anomaly
detection
• The image shows set of animals like elephants, camels and cows
that represents raw data that the unsupervised learning algorithm
will process.
• The "Interpretation" stage signifies that the algorithm doesn't
have predefined labels or categories for the data. It needs to
figure out how to group or organize the data based on inherent
patterns.
• An algorithm represents unsupervised learning process that helps
to identify patterns in the data.
• The processing stage shows the algorithm working on the data.
• The output shows the results of the unsupervised learning
process. In this case, the algorithm might have grouped the
animals into clusters based on their species (elephants, camels,
cows).
Working of Unsupervised Learning
The working of unsupervised machine learning can be explained in these steps:
1. Collect Unlabeled Data
Gather a dataset without predefined labels or categories.
Example: Images of various animals without any tags.
2. Select an Algorithm
Choose a suitable unsupervised algorithm such as clustering like K-Means, association rule learning
like Apriori or dimensionality reduction like PCA based on the goal.
3. Train the Model on Raw Data
Feed the entire unlabeled dataset to the algorithm.
The algorithm looks for similarities, relationships or hidden structures within the data.
4. Group or Transform Data
The algorithm organizes data into groups (clusters), rules or lower-dimensional forms without human
input.
Example: It may group similar animals together or extract key patterns from large datasets.
5. Interpret and Use Results
Analyze the discovered groups, rules or features to gain insights or use them for further tasks like
visualization, anomaly detection or as input for other models.
Advantages -
• No need for labeled data: Works with raw, unlabeled
data hence saving time and effort on data annotation.
• Discovers hidden patterns: Finds natural groupings
and structures that might be missed by humans.
• Handles complex and large datasets: Effective for
high-dimensional or vast amounts of data.
• Useful for anomaly detection: Can identify outliers
and unusual data points without prior examples.
Challenges/Disadvantages
Here are the key challenges of unsupervised learning:
• Noisy Data: Outliers and noise can distort patterns and
reduce the effectiveness of algorithms.
• Overfitting Risk: Overfitting can occur when models
capture noise instead of meaningful patterns in the data.
• Limited Guidance: The absence of labels restricts the
ability to guide the algorithm toward specific outcomes.
• Cluster Interpretability: Results such as clusters may lack
clear meaning or alignment with real-world categories.
Applications of Unsupervised learning
• Unsupervised learning has diverse applications across industries and
domains. Key applications include:
• Customer Segmentation: Algorithms cluster customers based on
purchasing behavior or demographics, enabling targeted marketing
strategies.
• Anomaly Detection: Identifies unusual patterns in data, aiding fraud
detection, cybersecurity and equipment failure prevention.
• Recommendation Systems: Suggests products, movies or music by
analyzing user behavior and preferences.
• Image and Text Clustering: Groups similar images or documents for tasks
like organization, classification or content recommendation.
• Social Network Analysis: Detects communities or trends in user
interactions on social media platforms.
5.1 Clustering Techniques:-Define Clustering
• Clustering is an unsupervised machine
learning technique that groups similar data
points together into clusters based on their
characteristics, without using any labeled data.
• The objective is to ensure that data points within
the same cluster are more similar to each other
than to those in different clusters, enabling the
discovery of natural groupings and hidden
patterns in complex datasets.
• Goal: Discover the natural grouping or structure in
unlabeled data without predefined categories.
• How: Data points are assigned to clusters based on
similarity or distance measures.
• Similarity Measures: Can include Euclidean distance,
cosine similarity or other metrics depending on data
type and clustering method.
• Output: Each group is assigned a cluster ID,
representing shared characteristics within the cluster.
• For example, if we have customer purchase
data, clustering can group customers with
similar shopping habits.
• These clusters can then be used for targeted
marketing, personalized recommendations or
customer segmentation.
Importance of Clustering in Data Analysis
1) Data Simplification
Clustering divides a large and complex dataset into smaller, meaningful groups.
It reduces complexity by summarizing similar data points together.
This makes data easier to visualize, interpret, and analyze.
It helps analysts understand overall data distribution quickly.
2) Pattern Discovery
Clustering identifies hidden patterns and relationships in unlabeled data.
It reveals natural groupings that may not be visible initially.
This is very useful in exploratory data analysis (EDA).
It helps researchers and businesses understand behavior trends.

3) Data Compression
Clustering reduces redundancy by grouping similar records together.
Instead of analyzing individual data points, we analyze clusters.
This reduces storage requirements and computational cost.
It improves efficiency in large-scale data processing.
4) Improves Decision Making
By grouping similar data, organizations can take data-driven decisions.
For example, companies can identify high-value and low-value customers.
It supports strategic planning and targeted marketing.
Better grouping leads to better resource allocation.

5) Anomaly Detection
Clustering helps identify data points that do not belong to any group.
These points are treated as outliers or anomalies.
It is widely used in fraud detection, cyber security, and fault detection.
Abnormal patterns can be detected easily using clustering.

6) Preprocessing Step
Clustering is often used before classification and regression tasks.
It helps in understanding data structure before model building.
Feature engineering becomes easier after clustering.
This improves the performance and accuracy of machine learning models.
Applications of Clustering
1) Customer Segmentation
Clustering groups customers based on purchasing behavior, income, or interests.
Businesses use it to design targeted marketing strategies.
It helps in increasing sales and customer satisfaction.

2) Image Segmentation
Clustering divides an image into different regions based on color or texture.
It is used in medical imaging and object detection.
It helps in identifying important parts of an image.

3) Document Clustering
Clustering groups similar documents based on content.
It is used in search engines and digital libraries.
It helps users find relevant information quickly.
4) Social Network Analysis
Clustering identifies communities or groups in social networks.
It shows how users are connected based on interests or interactions.
It is useful for analyzing online behavior patterns.
5) Medical Diagnosis
Clustering groups patients with similar symptoms or medical history.
Doctors can identify disease patterns and risk groups.
It helps in early diagnosis and treatment planning.
6) Fraud Detection
Clustering detects unusual patterns in financial transactions.
Abnormal transactions that do not fit any cluster are flagged as fraud.
It improves security in banking and e-commerce systems.
7) Recommendation Systems
Clustering groups users with similar preferences or interests.
It is used in platforms like Netflix and Amazon for recommendations.
It helps in suggesting products or content to users.
8) Market Research
Clustering identifies different consumer groups in the market.
Companies use it to understand customer needs and preferences.
It helps in launching suitable products for target audiences.
Types of Clustering methods
1. Centroid-based Clustering (Partitioning Methods)
• Centroid-based clustering organizes data points around
central prototypes called centroids, where each cluster is
represented by the mean (or medoid) of its members. The
number of clusters is specified in advance and the
algorithm allocates points to the nearest centroid, making
this technique efficient for spherical and similarly sized
clusters but sensitive to outliers and initialization.
Algorithms:
• K-means: Iteratively assigns points to nearest centroid and
recalculates centroids to minimize intra-cluster variance.
[Link]-based Clustering (Model-based Methods)
• Density-based clustering defines clusters as contiguous
regions of high data density separated by areas of lower
density. This approach can identify clusters of arbitrary
shapes, handles noise well and does not require
predefining the number of clusters, though its
effectiveness depends on chosen density parameters.
Algorithms:
• DBSCAN (Density-Based Spatial Clustering of Applications
with Noise): Groups points with sufficient neighbors;
labels sparse points as noise.
3. Connectivity-based Clustering (Hierarchical Clustering)
• Connectivity-based (or hierarchical) clustering builds
nested groupings of data by evaluating how data points
are connected to their neighbors. It creates a dendrogram
a tree-like structure that reflects relationships at various
granularity levels and does not require specifying cluster
numbers in advance, but can be computationally
intensive.
Approaches:
• Agglomerative (Bottom-up): Start with each point as a
cluster; iteratively merge closest clusters.
• Divisive (Top-down): Start with one cluster; iteratively split
into smaller clusters.
5.2 K-Means Clustering:
• K-Means Clustering groups similar data points into
clusters without needing labeled data. It is used to
uncover hidden patterns when the goal is to organize
data based on similarity.
• Helps identify natural groupings in unlabeled datasets
• Works by grouping points based on distance to cluster
centers
• Commonly used in customer segmentation, image
compression, and pattern discovery
• Useful when you need structure from raw,
unorganized data
Working of K-Means Clustering
• Suppose we are given a data set of items with
certain features and values for these features
like a vector. The task is to categorize those
items into groups.
• To achieve this we will use the K-means
algorithm.
• " 𝑘 " represents the number of groups or
clusters we want to classify our items into.
The algorithm works as follows:
• Initialization: We begin by randomly selecting k cluster
centroids.
• Assignment Step: Each data point is assigned to the
nearest centroid, forming clusters.
• Update Step: After the assignment, we recalculate the
centroid of each cluster by averaging the points within it.
• Repeat: This process repeats until the centroids no longer
change or the maximum number of iterations is reached.
• The goal is to partition the dataset into 𝑘 clusters such
that data points within each cluster are more similar to
each other than to those in other clusters.
Steps in K-Means Clustering-
1. Select Number of Clusters (K)-First, choose the number of clusters K that you
want to divide the dataset into. Example: If K = 3, the algorithm will divide the data
into 3 clusters.
2. Initialize Centroids-Randomly select K data points from the dataset as the initial
centroids (center points of clusters).Each centroid represents the center of a cluster.
3. Assign Data Points to the Nearest Centroid-Calculate the distance between each
data point and the centroids (usually Euclidean distance).Each data point is assigned
to the cluster whose centroid is closest.
4. Update the Centroids-After assigning all data points to clusters: Recalculate the
centroid of each cluster.
The new centroid is the mean (average) of all data points in that cluster.
5. Repeat Assignment and Update-Repeat the following steps: Assign data points to
the nearest centroid. Recalculate centroids.
This process continues until centroids stop changing or clusters become stable.
6. Final Clusters Formed-When no more changes occur in centroids or cluster
assignments, the algorithm stops and produces the final clusters.
Advantages and Disadvantages of K-Means Clustering
Advantages of K-Means
• Simple and Easy to Implement-The algorithm is easy to understand and implement
compared to other clustering techniques.
• Fast and Efficient-It works efficiently on large datasets and has low computational
complexity.
• Scalable-Suitable for large datasets with many data points.
• Easy Interpretation-Results are easy to interpret because clusters are represented by
centroids.
• Works Well for Compact Clusters-Gives good results when clusters are spherical and well
separated.
Disadvantages of K-Means
• Need to Specify K Value-The number of clusters (K) must be defined before applying the
algorithm.
• Sensitive to Initial Centroids-Different initial centroid selections can give different results.
• Sensitive to Outliers-Outliers can significantly affect cluster centroids.
• Not Suitable for Non-Spherical Clusters-Does not perform well if clusters have irregular
shapes.
• May Converge to Local Optimum-Sometimes it does not give the global best clustering
solution.
Hierarchical Clustering:
• Hierarchical Clustering is an unsupervised learning
technique that groups data into a hierarchy of clusters
based on similarity. It builds a tree-like structure
(dendrogram) that helps visualize relationships and
decide the optimal number of clusters.
– Does not require pre-selecting the number of clusters.
– Uses agglomerative or divisive approaches.
– Commonly applied in data exploration and pattern
discovery.
– It is commonly used in pattern recognition, customer
segmentation and image grouping.
• Imagine we have four fruits with different weights: an
apple (100g), a banana (120g), a cherry (50g) and a
grape (30g). Hierarchical clustering starts by treating
each fruit as its own group.
– Start with each fruit as its own cluster.
– Merge the closest items: grape (30g) and cherry (50g) are
grouped first.
– Next, apple (100g) and banana (120g) are grouped.
– Finally, these two clusters merge into one.
– Finally all the fruits are merged into one large group,
showing how hierarchical clustering progressively
combines the most similar data points.
Types of Hierarchical Clustering
• Now we understand the basics of hierarchical
clustering. There are two main types of
hierarchical clustering.
1. Agglomerative Clustering
2. Divisive clustering
1. Hierarchical Agglomerative Clustering
• It is also known as the bottom-up approach

or hierarchical agglomerative
clustering (HAC). Bottom-up algorithms treat
each data as a singleton cluster at the outset
and then successively agglomerate pairs of
clusters until all clusters have been merged
into a single cluster that contains all data.
Workflow for Hierarchical Agglomerative clustering
• Start with individual points: Each data point is its own cluster. For
example if we have 5 data points we start with 5 clusters each containing
just one data point.
• Calculate distances between clusters: Calculate the distance between
every pair of clusters. Initially since each cluster has one point this is the
distance between the two data points.
• Merge the closest clusters: Identify the two clusters with the smallest
distance and merge them into a single cluster.
• Update distance matrix: After merging we now have one less cluster.
Recalculate the distances between the new cluster and the remaining
clusters.
• Repeat steps 3 and 4: Keep merging the closest clusters and updating
the distance matrix until we have only one cluster left.
• Create a dendrogram: As the process continues, we can visualize the
merging of clusters using a tree-like diagram called a dendrogram. It
shows the hierarchy of how clusters are merged.
2. Hierarchical Divisive clustering
• Divisive clustering is also known as a top-

down approach.
• Top-down clustering requires a method for

splitting a cluster that contains the whole


data and proceeds by splitting clusters
recursively until individual data have been
split into singleton clusters.
Workflow for Hierarchical Divisive clustering :
• Start with all data points in one cluster: Treat the entire dataset
as a single large cluster.
• Split the cluster: Divide the cluster into two smaller clusters. The
division is typically done by finding the two most dissimilar points
in the cluster and using them to separate the data into two parts.
• Repeat the process: For each of the new clusters, repeat the
splitting process: Choose the cluster with the most dissimilar
points and split it again into two smaller clusters.
• Stop when each data point is in its own cluster: Continue this
process until every data point is its own cluster or the stopping
condition (such as a predefined number of clusters) is met.
Advantages and Disadvantages of Hierarchical Clustering
Advantages of Hierarchical Clustering
• No Need to Specify Number of Clusters (K)
The number of clusters is not required in advance. It can be decided by analyzing the
dendrogram.
• Dendrogram Visualization
It produces a tree-like structure (dendrogram), which helps in understanding cluster
relationships.
• Works for Different Types of Data
Can be used for numerical as well as categorical data (with proper distance measures).
Disadvantages of Hierarchical Clustering
• High Computational Cost
Time complexity is high, so it is not suitable for very large datasets.
• Irreversible Process
Once clusters are merged (or split), the process cannot be undone.
• Sensitive to Noise and Outliers
Outliers can significantly affect the clustering result.
• Less Efficient for Large Data
Memory and processing requirements increase rapidly with dataset size.
Sr.
K-Means Clustering Hierarchical Clustering
No.

K-Means clustering is a partition-based clustering technique Hierarchical clustering is a tree-based clustering


1
that divides the dataset into K predefined clusters. technique that builds a hierarchy of clusters.

In K-Means clustering, the number of clusters (K) must be In Hierarchical clustering, the number of clusters is not
2
specified before applying the algorithm. required initially.

K-Means clustering assigns data points to clusters based on Hierarchical clustering groups data points based on
3
the nearest centroid. distance between clusters.

Hierarchical clustering produces a tree-like structure


4 K-Means clustering produces a flat cluster structure.
called a dendrogram.

K-Means clustering is computationally faster and suitable for Hierarchical clustering is slower and suitable mainly for
5
large datasets. small datasets.

In K-Means clustering, data points can change clusters during In Hierarchical clustering, once clusters are merged or
6
iterations. divided, they cannot be changed.

Hierarchical clustering requires more memory and


7 K-Means clustering requires less memory and computation.
computational resources.

K-Means clustering works well for spherical and well- Hierarchical clustering can detect clusters with complex
8
separated clusters. shapes.
5.3 Dimensionality Reduction:
• Dimensionality reduction helps to reduce the
number of features while retaining key
information.
• It converts high-dimensional data into a lower-
dimensional space while preserving important
details.
• This process makes data analysis more efficient
hence improving computation speed and
visualization while minimizing redundancy
Dimensionality Reduction Techniques
• Dimensionality reduction techniques can be broadly divided into
two categories:
1. Feature Selection
• Feature selection chooses the most relevant features from the
dataset without altering them. It helps remove redundant or
irrelevant features, improving model efficiency. Some common
methods are:
• Filter methods rank the features based on their relevance to the
target variable.
• Wrapper methods use the model performance as the criteria for
selecting features.
• Embedded methods combine feature selection with the model
training process.
2. Feature Extraction
Feature extraction involves creating new features by combining or transforming the
original features. These new features retain most of the dataset’s important
information in fewer dimensions. Common feature extraction methods are:
• Principal Component Analysis (PCA): Converts correlated variables into
uncorrelated principal components hence reducing dimensionality while
maintaining as much variance as possible enabling more efficient analysis.
• Missing Value Ratio: Variables with missing data beyond a set threshold are
removed, improving dataset reliability.
• Backward Feature Elimination: Starts with all features and removes the least
significant ones in each iteration. The process continues until only the most
impactful features remain, optimizing model performance.
• Forward Feature Selection: It begins with one feature, adds others incrementally
and keeps those improving model performance.
• Random Forest: Random forest uses decision trees to evaluate feature
importance, automatically selecting the most relevant features without the need
for manual coding, enhancing model accuracy.
Importance of Dimensionality Reduction
1. Reduces Overfitting-When a dataset has too many features, the model may learn noise instead of patterns.
Reducing dimensions helps improve generalization and reduces overfitting.

2. Improves Model Performance-With fewer features, algorithms run faster and require less memory. This increases
training speed and improves computational efficiency.

3. Removes Multicollinearity-Many features may be highly correlated. Techniques like Principal Component Analysis
convert correlated variables into uncorrelated principal components.

4. Handles Curse of Dimensionality-In high-dimensional data, distance-based algorithms (like KNN) perform poorly.
Dimensionality reduction helps overcome the curse of dimensionality.

5. Better Data Visualization-It helps visualize high-dimensional data in 2D or 3D space using techniques like:
Principal Component Analysis
t-Distributed Stochastic Neighbor Embedding
This makes pattern recognition easier.
6. Noise Reduction-Unimportant and irrelevant features are removed, improving data quality and model accuracy.
7. Simplifies Data Storage-Reduced features mean less storage space and easier data handling.
Advantages dimensionality reduction -
• Faster Computation: With fewer features machine learning algorithms
can process data more quickly. This results in faster model training and
testing which is particularly useful when working with large datasets.
• Better Visualization: As we saw in the earlier figure reducing dimensions
makes it easier to visualize data and reveal hidden patterns.
• Prevent Overfitting: With few features models are less likely to
memorize the training data and overfit. This helps the model generalize
better to new, unseen data improve its ability to make accurate
predictions.
Disadvantages dimensionality reduction -
• Data Loss & Reduced Accuracy: Some important information may be lost
during dimensionality reduction and affect model performance.
• Choosing the Right Components: Deciding how many dimensions to
keep is difficult as keeping too few may lose valuable information while
keeping too many can led to overfitting.
5.4 Principal Component Analysis (PCA)
Definition
• Principal Component Analysis is a
dimensionality reduction technique used to
transform a large set of correlated variables into
a smaller set of uncorrelated variables called
principal components, while retaining most of
the important information (variance) in the
dataset.
The fundamental principle of Principal Component
Analysis is:
• To transform correlated variables into a new set

of uncorrelated variables (principal components)


such that the first component captures the
maximum variance in the data, and each
subsequent component captures the remaining
maximum variance under the constraint of being
orthogonal to the previous components.
Eigenvalues and Eigenvectors
• In Principal Component Analysis (PCA), Eigenvalues and
Eigenvectors are very important because they help us find the
main patterns and directions of variation in the data.
1. Eigenvector (Direction of Data Spread)
• An Eigenvector is a vector that represents the direction in which
the data varies the most.
• It shows the new axis (principal component) of the dataset.
• Each eigenvector corresponds to a principal component (PC).
• The first eigenvector gives the direction of maximum variance in
the data.
• The second eigenvector gives the next maximum variance and is
perpendicular to the first.
2. Eigenvalue (Amount of Variance)
• An Eigenvalue tells how much variance exists
along the corresponding eigenvector.
• It represents the importance or strength of a
principal component.
• A larger eigenvalue means that component
explains more information from the dataset.
• PCA usually keeps only components with large
eigenvalues.
Steps in Principal Component Analysis (PCA)
Step 1: Standardize the Data
• Different features may have different units and scales like salary
vs. age. To compare them fairly PCA first standardizes the data by
making each feature have:
A mean of 0
A standard deviation of 1
𝑋−𝜇
𝑍=
𝜎
• where:
𝜇 is the mean of independent features 𝜇 = 𝜇1 𝜇2 ⋯ 𝜇𝑚
𝜎 is the standard deviation of independent features 𝜎
= 𝜎1 𝜎2 ⋯ 𝜎𝑚
Step 2: Calculate Covariance Matrix
• Next PCA calculates the covariance matrix to

see how features relate to each other


whether they increase or decrease together.
Cov 𝑋1 𝑋2
• If covariance is high → variables are highly

correlated.
Step 3: Calculate Eigenvalues and Eigenvectors
• Find eigenvalues and eigenvectors of the covariance matrix.
• Eigenvectors → Directions of principal components.
• Eigenvalues → Amount of variance in each direction.
• The eigenvector with the highest eigenvalue becomes Principal Component 1 (PC1).
Step 4: Sort Eigenvalues
• Arrange eigenvalues in descending order.
• Select top k eigenvectors with highest eigenvalues.
• These form the new feature space.
Step 5: Form Feature Vector
• Create a matrix using selected eigenvectors.
• This matrix is called the Feature Vector.
Step 6: Transform the Data
• Multiply original standardized data with feature vector.
• This gives new reduced dataset (Principal Components).
• New Data = Feature Vector𝑇 × Standardized Data
Explained Variance in Principal Component Analysis (PCA)

• Explained Variance is an important concept in


PCA that indicates how much of the total
variability (information) in the dataset is
captured by each principal component.
• When PCA transforms the original correlated
variables into a set of new uncorrelated
variables called principal components, each
component explains a certain portion of the
total variance of the data.
• Variance represents the spread or variability of data points
around the mean. In PCA, the goal is to find directions in which
the data varies the most.
• Each principal component captures some part of this variability.
The amount of variance captured by a principal component is
known as explained variance.
• Explained variance is useful because it:
– Helps in dimensionality reduction by selecting the most important
components.
– Identifies principal components that contain maximum
information.
– Removes redundant and less important features.
– Improves data visualization in 2D or 3D space.
– Helps in building efficient machine learning models with less
computation.
Advantages of PCA
1. Reduces Dimensionality-PCA reduces the number of features
while keeping maximum important information.
2. Removes Multicollinearity-It converts correlated variables into
uncorrelated principal components.
3. Improves Model Performance-With fewer features, models train
faster and require less memory.
4. Handles Curse of Dimensionality-Improves performance of
distance-based algorithms like KNN.
5. Better Visualization-High-dimensional data can be visualized in 2D
or 3D using principal components.
6. Noise Reduction-Less important features (low variance) can be
removed, reducing noise.
Disadvantages of PCA
1. Loss of Information-Some variance is lost when reducing
dimensions.
2. Difficult to Interpret-Principal components are linear
combinations of original features, so they are hard to
interpret.
3. Assumes Linearity-PCA works only for linear relationships.
4. Sensitive to Scaling-If data is not standardized, PCA may
give misleading results.
5. Affected by Outliers-Extreme values can significantly
change principal components.
Applications of PCA
PCA finds applications in various domains, including but not limited
to:
• Image Compression: Reducing the dimensionality of image data
while preserving essential information, which is crucial for image
storage and transmission.
• Bioinformatics: Analyzing high-dimensional gene expression data
to identify patterns and reduce noise.
• Face Recognition: Extracting essential facial features for
recognition tasks.
• Recommendation Systems: Reducing the dimensionality of user-
item interaction data for efficient recommendation algorithms.
• Finance: Analyzing financial data to identify underlying trends and
patterns.

You might also like