CSI 3004 – Text Mining

Faculty Name : Yuvaraj Natarajan

School : SCOPE
Cabin : SJT 613A
1

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 CSI 3004 – Text Mining

Module:2 Clustering (8 hours)

Feature Selection and transformation Methods - Distance-based Clustering
Algorithms - Word and Phrase based Clustering - Probabilistic Document
Clustering and Topic Models - Online Clustering with Text Streams;
Multilingual document clustering - Multilingual LSA, LSA with term
alignments, LMSA with term alignments.

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Overview of Clustering in Text Mining

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Overview of Clustering in Text Mining

• Clustering in text mining is the process of grouping a set of text

documents into clusters (or groups) based on similarity.
• The goal is to bring similar documents together and separate dissimilar
ones, enabling better understanding and analysis of large text datasets.

Example:
News Article Categorization: Automatically grouping news articles into
topics like "sports," "politics," "technology," etc., so that users can easily
browse the relevant articles.

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Importance of Clustering for Text Data

Grouping Similar Documents: Clustering

helps organize large amounts of unstructured
text data by grouping similar documents
together.
Example: Product Review Categorization

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Importance of Clustering for Text Data

Information Retrieval: Text clustering, related documents

are grouped, making it easier for search engines to deliver
more relevant search results to users. Essential for enhancing
user experience.
Example: Search Engines

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Importance of Clustering for Text Data

Knowledge Discovery: Text clustering can reveal

hidden patterns and relationships in large dataset.
It allows data scientists to uncover trends, topics, or
themes that may not be immediately obvious.
Example: Customer Feedback Analysis

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Overview of Clustering in Text Mining

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Overview of Clustering Techniques

Distance-Based Clustering: Groups documents based on a similarity

measure, such as Euclidean distance, cosine similarity, or Manhattan
distance. Eg: K-means and DBSCAN.

Probabilistic Clustering: Each document is assumed as a mixture of

topics and try to uncover the underlying structure of the text.
Example: Latent Dirichlet Allocation (LDA)

Multilingual Clustering: Involves grouping documents written in

different languages while overcoming the challenge of language
differences.
9

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Overview of Clustering Techniques (Examples)

K-means for Document Clustering: Grouping thousands of research

papers based on their topics using K-means clustering.

Topic Modeling with LDA: In a collection of academic papers, LDA can

uncover hidden topics such as "AI," "robotics," and "machine learning," even
if the words used in each paper don't exactly match.

Cross-Language Document Clustering: For multinational companies,

clustering customer feedback in multiple languages (e.g., English, Spanish,
French) helps in analyzing global sentiment trends.

10

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Challenges in Clustering Text Data

v Handling Noise and Outliers: Text data often contains irrelevant

or noisy information that can confuse clustering algorithms.

v High Dimensionality: Text data typically involves thousands or

even millions of unique words, making it difficult to perform
clustering without dimensionality reduction.

11

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Selection and Transformation Methods

12

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Role of Features in Text Clustering
• In text mining, documents are represented as high dimensional
feature vectors
• Vocabulary size ranges from 10,000 to 100,000+ terms
• Not all features contribute meaningfully to clustering

Irrelevant and redundant features will:

Increase computational cost ; Degrade cluster quality;
Introduce noise and sparsity

Feature Selection Objective: Reduce dimensionality ; Preserve semantic

structure ; Improve clustering accuracy and interpretability

13

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 What is Feature Selection
Feature Selection – Process of selecting a subset of relevant
features from the original feature space
Eliminates: Rare terms ; Stop words ; Redundant terms

Why Feature Selection is Important for Clustering

• Clustering is unsupervised, sensitive to noise
• Distance based algorithms suffer in high dimensional spaces

Feature selection advantages:

Improves cluster separation;
Reduces dimensionality and Enhances scalability
14

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Categories of Feature Selection Methods
Filter methods - Evaluates features
Filter Methods
independently based on statistical tests or
•Information Gain
measures before the model is built.
•Chi Square Test
Wrapper Methods
Wrapper methods - Selects features by
•k means based
building and evaluating models on different
•Genetic algorithm
subsets of features.
Embedded Methods
•L1 regularization
Embedded methods
•Topic model
• Feature selection integrated into model learning
• Rare for classical clustering
In text clustering, filter based methods dominate. 15

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Selection Methods – Information Gain
Information Gain for Text Feature Selection -
Measures how much information a term contributes to
reducing uncertainty ; Based on entropy reduction

High information gain indicates:

• Term occurs selectively
• Term helps distinguish document groups

Usage in Clustering: Applying on known document

categories for preprocessing

Advantages: Simple ; Effective for large vocabularies

16

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Selection Methods – Chi Square Test

• Chi Square Test for Feature Selection : Measures

statistical dependence between term occurrence and
document distribution
• Tests whether the presence of a term is independent of
document groups

When the chi square value is high then Term is informative ;

Advantages: Robust to rare terms; Widely used in text mining

pipelines

17

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation

• Feature Transformation: Converts original features

into a new representation
• Unlike feature selection, no features are removed

Focuses on: Weighting ; Semantic embedding ;

Dimensionality compression

Importance for Clustering

• Improves similarity computation
• Captures semantic relationships
• Reduces sparsity
18

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using TF-IDF

TF IDF Transformation
Formula: TF IDF = TF × IDF
• TF measures term importance within a document
• IDF penalizes common terms across corpus

Impact on Clustering
• Highlights discriminative terms
• Improves distance based clustering
• Standard representation for text clustering

Limitation: Does not capture semantic similarity

19

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using word embeddings

20

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using word embeddings

• Word embeddings are a way to transform words into

dense vectors of real numbers where words with similar
meanings are placed close together in the vector space.
• Word embeddings make it easier for computers to
understand relationships between words.

• For example, "dog" and "cat" are related, and in word

embeddings, they will have similar vectors.

21

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using word embeddings

Example of Word Embedding Transformation

Before Word Embedding (One-Hot Encoding) - In one-hot
encoding, each word is represented by a vector where only one
element is 1, and all others are 0.
Let’s say we have a small vocabulary of 4 words:
Vocabulary = ["king", "queen", "man", "woman"]
The one-hot encoded vectors look like this:
•"king" = [1, 0, 0, 0]
•"queen" = [0, 1, 0, 0]
•"man" = [0, 0, 1, 0]
•"woman" = [0, 0, 0, 1]
22

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using word embeddings

Example of Word Embedding Transformation

After Word Embedding : Each word is transformed into a dense
vector (e.g., 3 numbers) that captures the word’s meaning.
Words with similar meanings have similar vectors.
Example after applying word embeddings (e.g., Word2Vec):
•"king" = [0.45, 0.67, 0.12]
•"queen" = [0.44, 0.66, 0.13]
•"man" = [0.20, 0.33, 0.54]
•"woman" = [0.21, 0.32, 0.55]

•"king" and "queen" are similar, so their vectors are close to each other.
•"man" and "woman" are also related, and their vectors are closer
23

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using Dimensionality Reduction

24

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using Dimensionality Reduction

Dimensionality Reduction using PCA

Principal Component Analysis
• Projects high dimensional data into lower dimensions
• Maximizes variance retention

Application in Text Clustering

• Applied after TF IDF or embeddings
• Reduces noise and redundancy
• Improves clustering speed

25

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Feature Transformation using PCA

Imagine you have a collection of thousands of text documents, such as

news articles or customer reviews. After preprocessing the text (like
removing stop words, stemming, and applying TF-IDF), each document
is represented by a high-dimensional vector where each dimension
corresponds to a unique word from the entire collection. This can
result in a very large feature space (e.g., thousands of dimensions),
where each document is described by a sparse vector of word
frequencies.

PCA transforms the high-dimensional document vectors into a lower-dimensional form,

improving clustering tasks by reducing noise and speeding up the process, while still
retaining most of the important information from the original text
26

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Applying PCA for Dimensionality Reduction:

Step 1: Feature Transformation: PCA helps reduce the high-

dimensional TF-IDF vectors to a smaller dimensions.
It does this by identifying the principal components. Essentially, PCA
finds the directions in which the data varies the most and uses them to
form a new, lower-dimensional representation.
Step 2 Improved Clustering: After applying PCA, the number of
dimensions is reduced, making it easier and faster to cluster similar
documents together.
Step 3 Noise and Redundancy Reduction: By projecting the data in
lower-dimensional, PCA removes less relevant variations making the
clustering process more efficient.
27

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Applying PCA for Dimensionality Reduction:

Example: Representing a Document as a Vector

Step 1: Initial High-Dimensional Vectors (Before PCA):
Imagine you have a collection of three short documents:
Document 1: "I love programming” : [1, 1, 1, 0, 0, 0, 0]
Document 2: "I enjoy coding”. : [1, 0, 0, 1, 1, 0, 0]
Document 3: "Coding is fun”. : [0, 0, 0, 0, 1, 1, 1]

Step 2: Center the Data:

Before applying PCA, center the data, meaning subtracting the mean of
each dimension (word frequency) from the respective values.

28

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Applying PCA for Dimensionality Reduction:
Step 3: Compute the Covariance Matrix: PCA analyzes the data's
covariance matrix, which describes how the features (words) vary
with respect to each other.

Step 4: Find the Principal Components (PCs) : PCA identifies

PCs - the new axes in the data that account for the most variance.

Step 5: Project the Documents onto PC’s : Project the original

high-dimensional data onto a lower-dimensional space defined by PC’s

Document 1: [0.8, -0.2] Document 2: [0.6, 0.4] Document 3: [-0.4, 0.8]

29

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Distance-Based Clustering Algorithms

30

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Distance-Based Clustering Algorithms

v Clustering groups data points into clusters where data points within a
cluster are more similar to each other than to those in other clusters.
v Distance-based algorithms (e.g., K-means, K-medoids and hierarchical )
measure the similarity between data points based on distance metrics.
v K-means Clustering: K-means aims to partition n data points into k
clusters, minimizing the variance within clusters. Each cluster is
represented by its centroid.
v K-medoids is similar to K-means but instead of using the mean to
represent a cluster, it uses an actual data point (medoid).

31

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Distance-Based Clustering Algorithms
Steps in K means clustering:
Ø Choose k: Select the number of clusters (k).
Ø Initialize centroids: Randomly assign k centroids.
Ø Assign points to nearest centroid: Each point is assigned to the
nearest centroid based on a distance measure (e.g., Euclidean).
Ø Update centroids: Recalculate centroids as the mean of all points
in the cluster.
Ø Repeat: Reassign points and update centroids until convergence.

Distance Metric: Euclidean distance is the most common:

Where 𝑥! & 𝑦! are coordinates of points x & y in the feature space.

32

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Distance-Based Clustering Algorithms

Steps in K-medoids Clustering

vChoose k: Select the number of clusters.
vSelect medoids: Choose k data points as medoids.
vAssign points to nearest medoid: Each point is assigned to the
nearest medoid.
vUpdate medoids: Reassign the medoid as the point minimizing the
average distance to all other points in the cluster.
vRepeat: Reassign points and update medoids until no changes.

33

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Distance Metrics

Distance Metrics in Text Mining

Euclidean Distance: Measures straight-line

distance between two points in vector space.
Common in general data clustering tasks.

Manhattan distance is the total distance

between two points measured by summing
the absolute differences of their Cartesian
coordinates

34

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Distance Metrics

Cosine Similarity: Measures the cosine

angle between two vectors. Ideal for text
mining as it focuses on orientation (meaning)
rather than the magnitude (length) of vectors.

𝑨⋅𝑩
Cosine Similarity =
∥ 𝑨 ∥∥ 𝑩 ∥

Where 𝐴 and 𝐵 are vectors of two

documents, and the formula measures the
cosine of the angle between them.
35

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means Clustering Example
Imagine the following 2D points that represent the positions
of students in a classroom based on their height and weight:
P1: (1, 2) P2: (1, 4) P3: (1, 0)
P4: (10, 12) P5: (10, 10) P6: (10, 8)
Divide these points into 2 clusters (k=2).

Step-by-Step Process:
Step 1: Initialize Centroids
Start by picking 2 random points as centroids
(Let us take P1 and P4 as centroids).
Centroid 1 (C1): P1 = (1, 2)
Centroid 2 (C2): P4 = (10, 12)
36

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means Clustering Example
Step 2: Assign Points to the Nearest Centroid - Calculate the distance of
each point from the two centroids. We'll use Euclidean distance for simplicity.
Distance from Centroid 1 (C1 = (1, 2)):
𝒅 𝑷 𝟏𝑪𝟏 = &𝟏 − 𝟏)𝟐 + *𝟐 − 𝟐)𝟐 = 𝟎

𝒅 𝑷 𝟐𝑪𝟏 = &𝟏 − 𝟏)𝟐 + *𝟒 − 𝟐)𝟐 = 𝟒 = 𝟐

𝒅 𝑷 𝟑𝑪𝟏 = &𝟏 − 𝟏)𝟐 + *𝟎 − 𝟐)𝟐 = 𝟒 = 𝟐

𝒅 𝑷 𝟒𝑪𝟏 = &𝟏𝟎 − 𝟏)𝟐 + *𝟏𝟐 − 𝟐)𝟐 = 𝟖𝟏 + 𝟏𝟎𝟎 = 𝟏𝟖𝟏 ≈ 𝟏𝟑. 𝟒𝟓

𝒅 𝑷 𝟓𝑪𝟏 = &𝟏𝟎 − 𝟏)𝟐 + *𝟏𝟎 − 𝟐)𝟐 = 𝟖𝟏 + 𝟔𝟒 = 𝟏𝟒𝟓 ≈ 𝟏𝟐. 𝟎𝟒

𝒅 𝑷 𝟔𝑪𝟏 = &𝟏𝟎 − 𝟏)𝟐 + *𝟖 − 𝟐)𝟐 = 𝟖𝟏 + 𝟑𝟔 = 𝟏𝟏𝟕 ≈ 𝟏𝟎. 𝟖𝟐

37

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means Clustering Example
Step 2: Assign Points to the Nearest Centroid : Calculate the distance
of each point from the two centroids. Use Euclidean distance
Distance from Centroid 2 (C2 = (10, 12)):
𝒅 𝑷 𝟏𝑪𝟐 = &𝟏 − 𝟏𝟎)𝟐 + *𝟐 − 𝟏𝟐)𝟐 = 𝟖𝟏 + 𝟏𝟎𝟎 = 𝟏𝟖𝟏 ≈ 𝟏𝟑. 𝟒𝟓

𝒅 𝑷 𝟐𝑪𝟐 = &𝟏 − 𝟏𝟎)𝟐 + *𝟒 − 𝟏𝟐)𝟐 = 𝟖𝟏 + 𝟔𝟒 = 𝟏𝟒𝟓 ≈ 𝟏𝟐. 𝟎𝟒

𝒅 𝑷 𝟑𝑪𝟐 = &𝟏 − 𝟏𝟎)𝟐 + *𝟎 − 𝟏𝟐)𝟐 = 𝟖𝟏 + 𝟏𝟒𝟒 = 𝟐𝟐𝟓 = 𝟏𝟓

𝒅 𝑷 𝟒𝑪𝟐 = &𝟏𝟎 − 𝟏𝟎)𝟐 + *𝟏𝟐 − 𝟏𝟎)𝟐 = 𝟎 + 𝟒 = 𝟐

𝒅 𝑷 𝟓𝑪𝟐 = &𝟏𝟎 − 𝟏𝟎)𝟐 + *𝟏𝟎 − 𝟏𝟎)𝟐 = 𝟎

𝒅 𝑷 𝟔𝑪𝟐 = &𝟏𝟎 − 𝟏𝟎)𝟐 + *𝟖 − 𝟏𝟐)𝟐 = 𝟎 + 𝟏𝟔 = 𝟒

38

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means Clustering Example
Step 3: Assign Points to Clusters : Assign each point to the nearest
centroid based on the calculated distances:
Cluster 1 (C1): P1, P2, P3 (because they are closest to C1)
Cluster 2 (C2): P4, P5, P6 (because they are closest to C2)

Step 4: Recalculate Centroids: Calculate the new centroids by averaging

the coordinates of the points assigned to each cluster.
Formula for Recalculating Centroid: New centroid is the mean of all
the points in that cluster. The formula for the new centroid is:
∑ 𝑥! ∑ 𝑦!
Centroid = 𝑛 𝑛
Where: 𝑥! and 𝑦! are the x and y coordinates of each point in the cluster.
𝑛is the number of points in the cluster.
39

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means Clustering Example
Step 4: Recalculate Centroids
New Centroid for Cluster 1:
1+1+1 2+4+0 3 6
C1 new = 3 3 = 3 3 = 12
New Centroid for Cluster 2:
10 + 10 + 10 12 + 10 + 8 30 30
C2 new = 3 3 = 3 3 = 10 10
Since the centroids haven’t changed, the algorithm has converged, and we
stop here.
Final Clusters:
Cluster 1: P1 = (1, 2), P2 = (1, 4), P3 = (1, 0)
Cluster 2: P4 = (10, 12), P5 = (10, 10), P6 = (10, 8)
40

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-medoids Clustering Example

41

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-medoids Clustering Example

Make use of the same points in the previous examples:

P1: (1, 2) ; P2: (1, 4) ; P3: (1, 0) ;
P4: (10, 12) ; P5: (10, 10) ; P6: (10, 8)
Perform K-medoids clustering with 𝑘 = 2.

Step-by-Step Process:
Step 1: Initialize Medoids
Randomly pick 2 points as medoids:
Medoid 1: P1 = (1, 2)
Medoid 2: P4 = (10, 12)

42

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-medoids Clustering Example

Step 2: Assign Points to Nearest Medoid : We calculate the distance

of each point from the two medoids using Euclidean distance.

Distance from Medoid 1 (P1):

d(P1, P1) = 0 d(P2, P1) = 2 d(P3, P1) = 2
d(P4, P1) ≈ 13.45 d(P5, P1) ≈ 12.04 d(P6, P1) ≈ 10.82

Distance from Medoid 2 (P4):

d(P1, P4) ≈ 13.45 d(P2, P4) ≈ 12.04 d(P3, P4) = 15
d(P4, P4) = 0 d(P5, P4) = 2 d(P6, P4) = 4

43

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-medoids Clustering Example
Step 3: Assign Points to Clusters
Cluster 1 (Medoid P1): P1, P2, P3 (because they are closest to Medoid 1)
Cluster 2 (Medoid P4): P4, P5, P6 (because they are closest to Medoid 2)

Step 4: Recalculate the medoids by choosing the point in each

cluster that minimizes the total distance to all other points in that cluster.
For Cluster 1: The new medoid would be P1 (since P1 already minimizes the
total distance to the other points).
For Cluster 2: Similarly, the medoid remains P4 (since P4 already minimizes
the total distance).
Since the medoids haven’t changed, the algorithm has converged.
Final Clusters: Cluster 1: P1 = (1, 2), P2 = (1, 4), P3 = (1, 0)
44

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means and K-medoids in Text Mining

45

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means and K-medoids in Text Mining

Working of K-means and K-medoids in Text Mining

Clustering algorithms used to group similar text documents into clusters,
helping to uncover patterns or topics from unstructured text data.

Steps Involved in Both K-means and K-medoids

Text Preprocessing:
• Convert text documents into numerical vectors using methods like
TF-IDF or word embeddings (e.g., Word2Vec).
• Each document is represented as a vector of numbers where each
dimension corresponds to a word or feature.

46

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means and K-medoids in Text Mining
Choosing k (Number of Clusters):
Decide how many clusters you want to
create (e.g., 2 clusters: Technology and
Sports).
Elbow Method, which helps find the
optimal number of clusters.
The Elbow Method helps by plotting
the Within-Cluster Sum of Squares
(WCSS) against increasing k values and
looking for a point where the
improvement slows down, this point is
called the "elbow."
47

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 K-means and K-medoids in Text Mining

Example in Text Mining:

Imagine you have a collection of 6 news articles, and you want to group them
into 2 clusters.
Step 1: Convert the text documents into TF-IDF vectors (numerical
representations of the content).
Step 2: Use either K-means or K-medoids to cluster the documents
based on their similarity.
Step 3: Each cluster will contain documents that are similar to each other.
For example, the Sports cluster might include documents about football and
basketball, while the Technology cluster includes articles about AI and
robotics.
48

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Hierarchical Clustering

49

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Hierarchical Clustering

Hierarchical clustering is a method of

clustering where data points are grouped into
a hierarchy based on their similarity.

It creates a tree-like structure called a

dendrogram, which shows how the clusters
are merged.

Two main types of hierarchical clustering:

Agglomerative Clustering (Bottom-up
approach)
Divisive Clustering (Top-down approach)
50

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Dendrogram of Hierarchical Clustering

51

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Agglomerative Clustering

[Link] with each data point as its own cluster: Initially, each data point
is treated as an individual cluster.
[Link] the Distance Matrix: Calculate the pairwise distance
(similarity) between every pair of clusters. Common distance measures are:
1. Euclidean Distance for numerical data.
2. Cosine Similarity for text data.
3. Manhattan Distance for some specific applications.
[Link] the Closest Clusters: The two clusters that are closest (based
on the distance measure) are merged into one cluster.

52

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Agglomerative Clustering

4. Update the Distance Matrix: After merging two clusters, update the
distance matrix to reflect the new cluster and its distance to other clusters.
The new distance can be calculated using different linkage methods:
1. Single Linkage: Minimum distance between any two points in the clusters.
2. Complete Linkage: Maximum distance between any two points in the clusters.
3. Average Linkage: Average of the pairwise distances between points in the clusters.

5. Repeat & plot: Continue merging the closest clusters until all data points
are grouped into a single cluster. The result is often visualized using a
dendrogram.

53

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Divisive Clustering

54

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Divisive Clustering

v Divisive Clustering is a hierarchical clustering method that follows a

top-down approach.

v Divisive Clustering begins with all data points in a single cluster and
recursively splits the cluster into smaller clusters until each data point is
its own cluster.

55

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Divisive Clustering

Steps Involved in Divisive Clustering:

1. Start with the entire dataset as a single cluster:
Initially, consider all the data points as part of a single, large cluster.

2. Choose the best cluster to split:

Identify the cluster that can be most meaningfully divided.
In the beginning, this is the entire dataset, but as the process proceeds, it
will be one of the smaller clusters.

56

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Divisive Clustering

Steps Involved in Divisive Clustering:

3. Find the optimal split: The key idea is to find a way to split the selected
cluster into two smaller clusters. The splitting can be done in various ways:
Partitioning based on distance or similarity measures (K-means or others)

Maximizing the dissimilarity between the two new clusters.

Different approaches are used to determine how to split cluster. For example:
• Maximizing intra-cluster dissimilarity (ensuring the two clusters have distinct
characteristics).
• Minimizing intra-cluster distance (making sure the data points in each new cluster
are as close as possible).

57

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Working of Divisive Clustering

Steps Involved in Divisive Clustering:

4. Recurse the splitting: After splitting, we have two smaller clusters.
These clusters are then treated as individual units and are further split if
necessary.
5. Repeat the splitting: Continue the process of dividing the clusters
recursively until each data point is in its own cluster or until a stopping
criterion is met.
6. Construct a Dendrogram: The resulting splits are often represented in a
dendrogram, a tree-like structure where:
Ø The root represents the original dataset.
Ø The branches show how the clusters and leaves represent individual data
points.
58

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

59

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example
•Given a set of 5 movie reviews, perform Agglomerative Clustering to group
them based on their sentiment (positive or negative).
•Use cosine similarity as the distance measure to calculate the pairwise
similarities between the reviews.
•Construct a dendrogram to visualize the clustering process and interpret
the resulting clusters.

The Reviews (Data Points):

[Link] 1: "The movie was fantastic and I loved the acting!"
[Link] 2: "An amazing plot and wonderful characters. Highly recommended!"
[Link] 3: "The movie was a complete disaster. Terrible acting and boring storyline."
[Link] 4: "It was a waste of [Link] dull and uninteresting."
[Link] 5: "Great cinematography, but the story was predictable."

60

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Step-by-Step Process with Cosine Similarity Values:

Preprocess the Text

We assume the following tokenized words after Stopword
Removal and TF-IDF Vectorization:
Review 1: "movie, fantastic, loved, acting"
Review 2: "amazing, plot, characters, highly, recommended"
Review 3: "disaster, terrible, acting, boring, storyline"
Review 4: "waste, time, dull, uninteresting"
Review 5: "cinematography, great, story, predictable"

61

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example
Cosine Similarity Calculation (Assumed Values):

R1 R2 = 0.3
R1 R3 = 0.1 Here, 1 means the
R1 R4 = 0.2 reviews are exactly
R1 R5 = 0.4 identical, and 0 means
R2 R3 = 0.2 there’s no similarity.
R2 R4 = 0.3
R2 R5 = 0.5
R3 R4 = 0.6
R3 R5 = 0.3
R4 R5 = 0.4 62

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Start with each review as a separate cluster

C1 = {R1}. C2 = {R2}. C3 = {R3}. C4 = {R4}. C5 = {R5}

Step 1: Merge the most similar pair

Highest similarity is R3 R4 = 0.6
So merge:
C34 = {R3, R4}
Now clusters are:
{R1}, {R2}, {R5}, {R3, R4}

63

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Step 2: Update similarities using average linkage

Average linkage between a single review and cluster {R3, R4} is the average
of its similarity with R3 and R4.
Similarity({R1}, {R3, R4}) = (R1 R3 + R1 R4) / 2 = (0.1 + 0.2) / 2 = 0.15
Similarity({R2}, {R3, R4}) = (R2 R3 + R2 R4) / 2 = (0.2 + 0.3) / 2 = 0.25
Similarity({R5}, {R3, R4}) = (R5 R3 + R5 R4) / 2 = (0.3 + 0.4) / 2 = 0.35
Other single to single similarities remain:
R2 R5 = 0.5. ; R1 R5 = 0.4. ; R1 R2 = 0.3

Similarity(Review 1, Review 2) + Similarity(Review 1, Review 5)

Avg Similarity =
2

64

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Step 3: Merge the next most similar pair

Highest remaining similarity is R2 R5 = 0.5
So merge:
C25 = {R2, R5}
Now clusters are:
{R1}, {R2, R5}, {R3, R4}

65

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Step 4: Update similarities again

Similarity({R1}, {R2, R5}) = (R1 R2 + R1 R5) / 2 = (0.3 + 0.4) / 2 = 0.35
Similarity({R2, R5}, {R3, R4}) = average of four pairs
= (R2 R3 + R2 R4 + R5 R3 + R5 R4) / 4
= (0.2 + 0.3 + 0.3 + 0.4) / 4
= 1.2 / 4
= 0.30
Similarity({R1}, {R3, R4}) was already computed as 0.15
So we have:
Sim({R1}, {R2, R5}) = 0.35
Sim({R2, R5}, {R3, R4}) = 0.30
Sim({R1}, {R3, R4}) = 0.15
66

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Step 5: Merge the next most similar pair

Highest is 0.35, so merge:
C125 = {R1, R2, R5}
Now clusters are:
{R1, R2, R5} and {R3, R4}

Step 6: Final merge (to complete hierarchy)

Merge the remaining two clusters:
{R1, R2, R5, R3, R4}

67

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

Final interpretation (sentiment based clusters)

Cluster A (mostly positive or mixed): R1, R2, R5
Cluster B (negative): R3, R4

Dendrogram merge order

1.(R3, R4) at similarity 0.6
2.(R2, R5) at similarity 0.5
3.(R1 joins {R2, R5}) at similarity 0.35
4.({R1, R2, R5} joins {R3, R4}) at similarity 0.30

68

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Agglomerative Clustering - Example

v In this example, Agglomerative Hierarchical Clustering was used to

group movie reviews based on their sentiment.
v Initially, each review is its own cluster, and clusters are progressively
merged based on similarity.
v The final clustering results in two main groups, which can be interpreted
as reviews with positive sentiment and negative sentiment.

Draw the dendrogram for this example

69

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Divisive Hierarchical Clustering – Example -1
Problem Statement: Divisive Hierarchical Clustering
Refer to the previously given set of 5 movie reviews and their associated
cosine similarity values.
1. Perform Divisive Hierarchical Clustering to group the reviews
based on sentiment (positive or negative).
2. Start with all reviews in a single cluster and recursively split the cluster
into smaller clusters using the provided cosine similarity values.
3. Construct a dendrogram to visualize the hierarchical splitting
process.
4. Interpret the resulting clusters as either positive or negative
sentiment reviews.

70

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Divisive Hierarchical Clustering – Example 2
Divisive Clustering Application
An online learning platform collects short textual feedback from learners enrolled in
various professional skill programs. The platform wants to segment the feedback into
thematically coherent groups in order to improve course design and policy
decisions.
The following learner feedback snippets are collected:
F1: “The new data science course helped me understand machine learning workflows effectively.”
F2: “Advanced programming modules demand more time due to increased assignment complexity.”
F3: “Sustainability focused courses are encouraging learners to adopt eco friendly practices.”
F4: “Electric mobility and renewable energy programs are gaining popularity among students.”
F5: “Recent education policies emphasize green skills and sustainable technology training.”

71

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Divisive Hierarchical Clustering – Example 2
After preprocessing the text using a Bag of Words representation, the following significant terms
are extracted:
data science, programming, assignments, sustainability, electric mobility, renewable energy, education
policy, green skills
Task:
1. Apply a Divisive Hierarchical Clustering (Top Down) approach to partition
the feedback into two meaningful clusters based on content similarity.
2. Use Euclidean distance to measure dissimilarity between feedback vectors.
3. Clearly explain how the initial single cluster is recursively split to form the final
two clusters.
4. Illustrate the hierarchical splits using a divisive clustering tree diagram.
5. Interpret each resulting cluster by describing the dominant theme reflected in
the feedback items assigned to it.
72

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Hierarchical Clustering – Example 3
A smart city traffic monitoring team is analyzing congestion patterns at
different road intersections during peak hours. Each intersection is
represented using two numerical indicators: average vehicle density and
average waiting time (in seconds). To identify groups of intersections
with similar traffic behavior, the team decides to apply hierarchical
agglomerative clustering.
The observed data for five major intersections is shown below.
Intersection ID Vehicle Density Waiting Time
I1 12.5 45
I2 13.8 50
I3 11.9 42
I4 30.2 95
I5 31.0 90 73

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Hierarchical Clustering – Example 3

Tasks:
• Compute the pairwise distances between intersections using the
Euclidean distance metric.
• Perform agglomerative hierarchical clustering using the average
linkage method.
• Show the clustering process step by step until a single cluster is formed.
• Draw the corresponding dendrogram and interpret the clustering result
in the context of traffic congestion patterns.

74

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Word and Phrase-Based Clustering

75

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Word and Phrase-Based Clustering - Introduction

Word and Phrase-Based Clustering:

• A technique in text mining where we
group similar words or phrases to
help identify patterns and themes
within documents.
• To understand relationships between
words/phrases and group documents
based on these relationships.

Applications: Topic modeling, document classification, search engine results.

76

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Word and Phrase-Based Clustering

Clustering with Frequent Word Patterns

v Frequent Word Patterns: Identify & cluster
words that appear frequently across documents.
v These often represent important themes or topics.
How it Works:
• Use techniques like TF-IDF or Word2Vec
to represent words numerically.
• Group words that frequently appear together
into clusters.

Example: In a set of articles about sports, words like "game", "team", "win", and
"score" would cluster together.
77

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Leveraging Word Clusters for Document Clusters

Word Clusters for Document Clusters: Once

words are grouped, they can be used to cluster entire
documents.
How it Works:
• Document Representation: Represent each
document as a vector of word clusters.
• Clustering: Apply clustering algorithms (like K-means)
to these document vectors to group similar documents.

Example: In a collection of news articles, documents about sports and

politics might be clustered separately based on the frequency of word
clusters like "team", "government", etc.
78

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Co-clustering Words and Documents

Co-clustering Words and Documents:

v Simultaneously cluster both words &
documents based on the patterns of co-
occurrence.
v Identifies relationships between words
and documents for more accurate
clustering.

Example: Words like "game", "score", "team" might cluster with documents
from the sports category.

79

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Clustering with Frequent Phrases

Clustering with Frequent Phrases:

v Instead of individual words, group frequent
multi-word phrases together.
v Phrases like "world cup", "national team", or
"super bowl" often carry significant meaning
and should be treated as single units.
v This helps improve the semantic
understanding of the document content.

80

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Word and Phrase-Based Clustering (Summary)

v Word and Phrase-Based Clustering

improves document clustering by focusing
on word patterns and phrases that
represent topics or themes.
v Co-clustering allows us to consider both
words and documents simultaneously for
more accurate results.

These techniques help in organizing large text corpora

and uncovering hidden patterns efficiently.
81

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 82

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Probabilistic Document Clustering

Probabilistic Document Clustering and Topic Models:

Probabilistic Latent Semantic Indexing (PLSI)
Latent Dirichlet Allocation (LDA)

83

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Probabilistic Document Clustering

What is Probabilistic Document Clustering?

• Probabilistic document clustering groups documents based on likelihood
rather than hard assignments.
• Each document can belong to multiple topics with different
probabilities, which reflects real world text more accurately.
• Example:
A news article may discuss technology (60%), business (30%), and
policy (10%).

84

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Probabilistic Document Clustering - LDA

Latent Dirichlet Allocation (LDA) – It is a probabilistic topic

modeling algorithm that discovers hidden topics in collection of documents.
How LDA Works
[Link] a fixed number of topics
Example: Sports, Politics, Technology
[Link] document has a probability distribution over topics
Example: Document 1 → Sports 0.7, Technology 0.3
[Link] topic has a probability distribution over words
Example: Sports → match, team, score
[Link] in documents are generated by selecting
Topic → then Word (based on probabilities)

Key logic: Documents are mixtures of topics, and topics are mixtures of words.
85

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Probabilistic Document Clustering - LDA

Why LDA is Useful for Clustering

• Documents are soft clustered, not forced into one group
• Handles large text collections well
• Captures hidden semantic structure
• Widely used in research and industry

Use Case: Clustering research papers by themes such as AI,

Networks, Security, Data Science

Key logic: Documents are mixtures of topics, and topics are mixtures of words.
86

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Probabilistic Document Clustering - pLSI

Probabilistic Latent Semantic Indexing (pLSI)

• pLSI is an earlier probabilistic model that explains documents
using latent topics, similar to LDA.
• Key idea: Words and documents are connected through
hidden topics using probability distributions.

How pLSI Works

• Each document is represented as a mixture of topics
• Each topic generates words probabilistically
• Topics are learned from word co occurrence patterns
87

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Probabilistic Document Clustering - pLSI

Aspect pLSI LDA

Topic Generated using prior
Document specific
distribution distribution
Generalization Poor for new documents Good for unseen documents
Overfitting More likely Less likely
Practical usage Limited Widely used

88

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Topic Models

89

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Topic Modeling

What is Topic Modeling?

Topic modeling is a text mining technique used to automatically discover hidden themes or
topics in a large collection of documents.
Instead of reading thousands of documents manually, topic models summarize what the documents
are about using probability and statistics.
Simple idea:
A topic is a group of words that frequently occur together.

Why Do We Need Topic Modeling?

Modern applications generate massive amounts of text:
• Social media posts
• Research papers
• News articles
• Customer reviews
Manual analysis is impossible.
Topic modeling helps computers understand themes
without human labeling. 90

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Topic Modeling

Explanation
[Link] documents talk about a
few hidden topics
[Link] document contains a mix
of topics
[Link] topic contains a set of
related words
[Link] model learns these
patterns automatically

LDA in Topic Modeling - It is the most widely used topic model.

What LDA does:
• Discovers topics automatically
• Assigns topic probabilities to documents
• Assigns word probabilities to topics
LDA does not need labeled data , It works in an unsupervised manner. 91

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Topic Modeling

Simple Example : Consider 1000

student feedback comments.
Topic modeling may uncover topics Applications of Topic Modeling
like: 1. Document Summarization
• Teaching quality • Identify key topics in long documents
2. Document Classification
• Exam difficulty • Group documents based on dominant
• Course content topics
• Infrastructure [Link] Analysis
• Track how topics change over time
• Placement support
Each feedback comment belongs to
multiple topics, not just one.

92

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Clustering with Text Streams

93

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Clustering with Text Streams

What is Online Clustering?

Online clustering is the process of grouping data continuously as it
arrives, instead of clustering a fixed dataset at once.
In text mining, this means clustering:
• Live tweets
• News feeds
• Chat messages
• User reviews
• Sensor or log text streams
The model updates clusters in real time, without reprocessing all past data.

94

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Clustering with Text Streams

Why Do We Need Online Clustering?

Modern applications generate text continuously and rapidly:
• Social media streams update every second
• News articles are published throughout the day
• Customer feedback arrives in real time
Traditional clustering fails because:
• Data size keeps growing
• Re clustering from scratch is expensive
• Results must be updated immediately
Online clustering enables scalable and real time analysis.

95

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Clustering with Text Streams

Challenges in Online Clustering

1. Continuous Data Arrival Incremental Clustering for Text
• Data never stops coming
• Cannot store all past documents Streams
• Model must learn on the fly Instead of clustering everything again:
2. Large Scale Data • Update existing clusters
• Millions of documents
• High dimensional text vectors • Adjust cluster centers incrementally
• Limited memory and computation • Assign new documents as they
3. Dynamic Topic Drift arrive
• Topics evolve over time This makes clustering fast, adaptive,
• Old topics may disappear
• New topics may emerge and memory efficient.

Example: Trending topics on social media during elections or disasters. 96

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Clustering with Text Streams

Simple Flow
[Link] document arrives
[Link] document to vector form
[Link] it to the nearest cluster
[Link] the cluster representation
[Link] or compress old
information

97

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Spherical k Means Algorithm
What is Spherical k Means?
Spherical k Means is a variation of k Means designed for text data.
Key idea:
• Uses cosine similarity instead of Euclidean distance
• Works on normalized document vectors
This makes it ideal for text clustering.

Why Online Spherical k Means?

• Efficient for high dimensional text
• Supports incremental updates
• Handles large text streams well

98

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Online Spherical k Means Algorithm

How Online Spherical k Means Works

[Link] cluster centroids
[Link] a new document
[Link] the document vector
[Link] it to the closest centroid using cosine similarity
[Link] the centroid incrementally
Each step is fast and suitable for streaming data.

99

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual Document Clustering

100

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual Document Clustering
What is Multilingual Document Clustering?
Multilingual document clustering is the task of grouping
documents written in different languages based on their
semantic similarity, not their surface words.
Key idea:
Documents discussing the same concept should be clustered
together even if they are written in different languages.
Example:
• English news about “climate change”
• French article on “changement climatique”
• Spanish report on “cambio climático”
All should belong to the same cluster.
101

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual Document Clustering
Challenges in Multilingual Document Clustering
1. Language Diversity
• Different scripts (English, Hindi, Arabic, Chinese)
• Different grammar and word order
• Different vocabulary for the same concept
2. Language-Specific Nuances
• Same word may have different meanings across languages
• Cultural expressions and idioms vary
• Morphology and inflections differ
Example: The meaning of a political term may vary across regions and languages.
3. Feature Space Mismatch
• Words in one language do not directly map to words in another
• Traditional vector representations fail across languages
Hence, semantic alignment becomes necessary.
102

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual Clustering Algorithms: Core Approach

103

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual Clustering Algorithms: Core Approach
Common Strategies
[Link]-Based Approaches
• Translate all documents into a single language
• Apply standard clustering
• Simple but error-prone and expensive
[Link] Semantic Space Approaches
• Map documents from different languages into a common latent space
• Cluster based on semantic similarity
• More robust and scalable
[Link]-Based Methods
• Align words or concepts across languages
• Preserve semantic relationships
104

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual LSA (Latent Semantic Analysis)
What is LSA?
Latent Semantic Analysis is a technique that:
• Reduces high dimensional text data
• Captures hidden semantic relationships
• Represents documents in a latent semantic space
LSA uses matrix factorization to uncover underlying topics.

Role of LSA in Document Clustering

• Documents with similar meanings are placed closer
• Noise due to vocabulary differences is reduced
• Works well for synonym handling
This makes LSA suitable for semantic clustering.
105

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Applying LSA for Multilingual Data

How Multilingual LSA Works

[Link] term document matrices for each language
[Link] them into a shared matrix
[Link] Singular Value Decomposition
[Link] all documents into a common semantic space
[Link] clustering in this space
Result: Documents across languages are clustered based on
meaning, not words
106

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Multilingual LSA

Multilingual LSA: Key Insight

LSA ignores surface language differences and
focuses on co occurrence patterns.
However, plain multilingual LSA assumes some
overlap, which may be weak for distant
languages.
This leads to the need for term alignment.

107

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 LSA with Term Alignments

What is Term Alignment?

Term alignment is the process of linking semantically equivalent
words across languages.
Examples:
• “computer” ↔ “ordinateur”
• “energy” ↔ “energía”
• “government” ↔ “gouvernement”
These alignments act as bridges between languages.

108

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 How Term Alignments Improve LSA

• Align related terms across languages

• Strengthen semantic connections
• Reduce language isolation in the latent
space
This leads to better clustering accuracy.

Applications of LSA with Term Alignments

• Machine translation support
• Cross language information retrieval
• Multilingual search engines
• International news analysis
109

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 LMSA with Term Alignments

What is LMSA?
Latent Multilingual Semantic Analysis is Why LMSA is Better than Basic LSA
• Handles diverse languages more
an advanced extension of LSA that:
effectively
• Explicitly integrates term alignments • Reduces bias toward dominant languages
• Models multiple languages jointly • Improves topic coherence across
languages
• Produces a unified semantic space
• Supports scalable multilingual clustering
LMSA is designed for large scale
multilingual corpora.

110

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 LMSA with Term Alignments: Conceptual Flow

• Build multilingual term document

matrices
• Introduce alignment constraints
between terms
• Learn shared latent semantic
representation
• Cluster documents across languages

111

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 LMSA with Term Alignments: Conceptual Flow

Use Cases of LMSA

• Clustering international research articles
• Multilingual social media monitoring
• Regional news aggregation
• Policy analysis across countries
• Global market intelligence systems

112

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT
 Thank you

113

PRESENTATION BY: YUVARAJ , SCOPE, VIT

Presentation by:Yuvaraj , SCOPE,VIT