Module 5 Unsupervised Learning
Clustering in machine learning-
• Clustering or cluster analysis is a machine learning technique, which groups the unlabeled dataset.
• It can be defined as "A way of grouping the data points into different clusters, consisting of similar
data points. The objects with the possible similarities remain in a group that has less or no
similarities with another group."
• It does by finding some similar patterns in the unlabeled dataset such as shape, size, color, behavior,
etc., and divides them as per the presence and absence of those similar patterns.
• It is an unsupervised learning method, hence no supervision is provided to the algorithm, and it deals
with the unlabeled dataset.
Types of Clustering Methods-
• Partitioning Clustering
• Density-Based Clustering
• Distribution Model-Based Clustering
• Hierarchical Clustering
• Fuzzy Clustering
1) Partitioning Clustering
o It is a type of clustering that divides the data into non-hierarchical groups.
o It is also known as the centroid-based method.
o Common example of partitioning clustering is the K-Means Clustering algorithm.
o In this type, the dataset is divided into a set of k groups.
o K is used to define the number of pre-defined groups.
o The cluster center is created in such a way that the distance between the data points of
one cluster is minimum as compared to another cluster centroid.
2) Density-Based Clustering
o The density-based clustering method connects the highly dense areas into clusters.
o These algorithms can face difficulty in clustering the data points if the dataset has
varying densities and high dimensions.
3) Distribution Model-Based Clustering
o In this method, the data is divided based on the probability.
o Dataset belongs to a particular distribution.
o The grouping is done by assuming Gaussian Distribution.
o An example of this type is the Expectation-Maximization Clustering.
4) Hierarchical Clustering
o Hierarchical clustering can be used as an alternative for partitioned clustering.
o In this technique, the dataset is divided into clusters to create a tree-like structure,
which is also called a dendrogram.
o A common example of this method is the Agglomerative Hierarchical algorithm.
5) Fuzzy Clustering
o Fuzzy clustering is a type of soft method.
o which a data object may belong to more than one group or cluster.
o Each dataset has a set of membership coefficients.
o which depends on the degree of membership to be in a cluster.
o Fuzzy C-means algorithm or Fuzzy C-means algorithm is the example of this type of
clustering.
Applications of Clustering-
In Identification of Cancer Cells
In Search Engines
Customer Segmentation
In Biology
In Land Use
K-Means Clustering Algorithm-
K-Means Clustering is an Unsupervised Learning algorithm.
which groups the unlabeled dataset into different clusters.
It allows us to cluster the data into different groups and is a convenient way to discover the
categories of groups in the unlabeled dataset.
It is a centroid-based algorithm. Each cluster is associated with a centroid.
The main aim of this algorithm is to minimize the sum of distances between the data point and their
corresponding clusters.
o The k-means clustering algorithm mainly performs two tasks:
Determines the best value for K center points or centroids by an iterative process.
Assigns each data point to its closest k-center.
Those data points which are near to the particular k-center, create a cluster.
K-Means Clustering Algorithm-
• The working of the K-Means algorithm is explained in the below steps:
o Step-1: Select the number K to decide the number of clusters.
o Step-2: Select random K points or centroids. (It can be other from the input dataset).
o Step-3: Assign each data point to their closest centroid, which will form the predefined K
clusters.
o Step-4: Calculate the variance and place a new centroid of each cluster.
o Step-5: Repeat the third step, which means reassigning each datapoint to the new closest
centroid of each cluster.
o Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.
o Step-7: The model is ready.
Why do we need DBSCAN Clustering?
• K-Means and Hierarchical Clustering both fail in creating clusters of arbitrary shapes.
• They are not able to form clusters based on varying densities. That’s why we need DBSCAN
clustering.
What Exactly is DBSCAN Clustering?
• DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise.
• The most exciting feature of DBSCAN clustering is that it is robust to outliers.
• It also does not require the number of clusters.
• DBSCAN requires only two parameters: 1) epsilon and 2) minPoints.
o Epsilon is the radius of the circle to be created around each data point.
o minPoints is the minimum number of data points required inside that circle.
o that data point to be classified as a Core point.
o In higher dimensions the circle becomes hypersphere, epsilon becomes the radius of that
hypersphere. minPoints is the minimum number of data points required inside that
hypersphere.
• DBSCAN creates a circle of epsilon radius around every data point and classifies them
into Core point, Border point, and Noise.
• Core point if the circle around it contains at least ‘minPoints’ number of points.
• If the number of points is less than minPoints, then it is classified as Border Point
• If there are no other data points around any data point within the epsilon radius, then it treated
as Noise.
Reachability and Connectivity
• Reachability states if a data point can be accessed from another data point directly or indirectly,
• Connectivity states whether two data points belong to the same cluster or not.
• In terms of reachability and connectivity, two points in DBSCAN can be referred to as
o Directly Density-Reachable
o Density-Reachable
o Density-Connected
a) Directly Density-Reachable
• A point X is directly density-reachable from point Y w.r.t epsilon, minPoints
• X belongs to the neighborhood of Y, i.e, dist(X, Y) <= epsilon
• Y is a core point
b) Density-Reachable
• A point X is density-reachable from point Y w.r.t epsilon, minPoints if there
is a chain of points p1, p2, p3, …, pn and p1=X and pn=Y such that pi+1 is
directly density-reachable from pi.
c) Density-Connected
• A point X is density-connected from point Y w.r.t epsilon and minPoints if
there exists a point O such that both X and Y are density-reachable
from O w.r.t to epsilon and minPoints.
Algorithm-
• Choose a value for epsilons and Minpoints
• For a particular data point (x) calculate its distance from every other data point.
• Find all the neighborhood points of x which fall inside the circle of radius (eps).
• Treat x as visited.
• if the number of neighborhood points around x are greater or equal to MinPts then treat x as a core
point.
• If it is not assigned to any cluster, create a new cluster, and assign it to that.
• If the number of neighborhood points around x are less than MinPts and it has a core point in its
neighborhood, treat it as a border point.
Principal Component Analysis-
• Principle Component Analysis (PCA0 comes under the Unsupervised Machine Learning category.
• Reducing the number of variables in a data collection.
• PCA can be mainly used for Dimensionality Reduction and for important feature selection.
• Correlated features to independent features.
When to use PCA?
• Whenever we need to know our features are independent of each other.
• Whenever we need fewer features from higher features.
Basic Terminologies of PCA-
• Variance – for calculating the variation of data distributed across dimensionality of graph.
• Covariance – calculating dependencies and relationship between features.
• Standardizing data – Scaling our dataset within a specific range for unbiased output.
• Covariance matrix – Used for calculating interdependencies between the features or variables and
helps in reducing it to improve the performance.
• Eigenvalues and Eigenvectors – Eigenvectors’ purpose is to find out the largest variance that exists in
the dataset to calculate Principal Component. Eigenvalue means the magnitude of the Eigenvector.
Eigenvalue indicates variance in a particular direction and whereas eigenvector is expanding or
contracting X-Y (2D) graph without altering the direction.
• Dimensionality Reduction – Transpose of original data and multiply it by transposing of the derived
feature vector. Reducing the features without losing information.
How does PCA work?
• Original Data
• Normalize the original data (mean =0, variance =1)
• Calculating covariance matrix
• Calculating Eigen values, Eigen vectors, and normalized Eigenvectors
• Calculating Principal Component (PC)
• Plot the graph for orthogonality between PCs.
Advantage for Principal Component Analysis
• Used for Dimensionality Reduction
• PCA will assist you in eliminating all related features, sometimes referred to as multi-collinearity.
• The time required to train your model is now substantially shorter because of PCA’s reduction in the
number of features.
• PCA aids in overcoming overfitting by eliminating the extraneous features from your dataset.
Disadvantage for Principal Component Analysis-
• Useful for quantitative data but not effective with qualitative data.
• Interpretation of PC is difficult from original data.