Scribd
Upload
14 views2 pages

Text Mining: From Normalization to Topics

The document discusses techniques in text mining, focusing on document normalization and topic modeling methods such as TF-IDF and cosine similarity for similarity computation. It explores specialized clustering methods like K-means and co-clustering for efficient document and word cluster discovery. Additionally, it introduces PLSA for probabilistic topic modeling, emphasizing its use in uncovering hidden topic structures and dimensionality reduction.

Uploaded by

rehanaparveen479
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views2 pages

Text Mining: From Normalization to Topics

The document discusses techniques in text mining, focusing on document normalization and topic modeling methods such as TF-IDF and cosine similarity for similarity computation. It explores specialized clustering methods like K-means and co-clustering for efficient document and word cluster discovery. Additionally, it introduces PLSA for probabilistic topic modeling, emphasizing its use in uncovering hidden topic structures and dimensionality reduction.

Uploaded by

rehanaparveen479
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Text Mining: Document Normalization to Topic Modeling

Text Mining: Document Normalization to Topic Modeling

1. Document Normalization and Similarity Computation

- Inverse Document Frequency (IDF): Reduces the weight of common words. IDF = log(n/ni), where n is

total documents and ni is number of documents containing term i.

- Frequency Damping: Applies functions like log(x) or sqrt(x) to reduce the effect of very frequent words.

- TF-IDF: A combination of term frequency and inverse document frequency. Normalized term: h(xi) = f(xi) *

idi.

- Cosine Similarity: Measures angle between vectors to determine similarity.

- Jaccard Coefficient: Less common in text but used in sparse datasets.

2. Specialized Clustering Methods for Text

- Text data represented in high-dimensional, sparse vector space.

- K-means is commonly used but requires:

- Cosine similarity instead of Euclidean distance.

- Cluster digest: Retain only 200-400 top words in centroid.

- Combines well with hierarchical clustering for efficiency and topic clarity.

3. Simultaneous Document and Word Cluster Discovery (Co-Clustering)

- Discovers both document and word clusters simultaneously.

- Matrix factorization like Nonnegative Matrix Factorization (NMF) is useful.

- Co-clustering: Divides rows (documents) and columns (words) into non-overlapping clusters.

- Coherent word clusters lead to coherent document clusters.

- Represented as a bipartite graph (documents and words as nodes).

- Graph partitioning groups documents and words into meaningful blocks.

4. Topic Modeling (PLSA)

- PLSA (Probabilistic Latent Semantic Analysis): Probabilistic alternative to LSA.

- Uses Expectation Maximization (EM) to discover hidden topic structures.

- Generates document-word entries via latent topics (aspects).


Text Mining: Document Normalization to Topic Modeling

- Parameters: P(Gm), P(Xi|Gm), P(wj|Gm) need to be learned from data.

- Suitable for dimensionality reduction and interpretable topic modeling.

You might also like