Prog: 2
Use dimensionality reduction (e.g., PCA or t-SNE) to visualize word embeddings for Q
1. Select 10 words from a specific domain (e.g., sports, technology) and visualize their
embeddings. Analyze clusters and relationships. Generate contextually rich outputs
using embeddings. Write a program to generate 5 semantically similar words
for a given input.
Step 1: # pip install gensim matplotlib scikit-learn
Step 2:!pip install gensim
Step 3:
# Module or library install command (run this in terminal before
running the script)
# pip install gensim matplotlib scikit-learn
import [Link] as api
from [Link] import PCA
import [Link] as plt
# Load model
model = [Link]("word2vec-google-news-300")
# Select 10 domain-specific words (technology domain)
words = ['computer', 'internet', 'software', 'hardware',
'keyboard', 'mouse', 'server', 'network', 'programming',
'database']
vectors = [model[word] for word in words]
# Dimensionality reduction using PCA
pca = PCA(n_components=2)
reduced = pca.fit_transform(vectors)
# Generate 5 semantically similar words for a given input
input_word = 'computer'
similar_words = model.most_similar(input_word, topn=5)
# Print the similar words to terminal
print(f"Top 5 words similar to '{input_word}':")
for word, score in similar_words:
print(f"{word}: {score:.4f}")
# Plot the word embeddings
[Link](figsize=(8, 6))
for i, word in enumerate(words):
[Link](reduced[i, 0], reduced[i, 1])
[Link](word, (reduced[i, 0], reduced[i, 1]))
[Link]("PCA Visualization of Technology Word Embeddings")
[Link]("PC1")
[Link]("PC2")
[Link](True)
# Show the plot
[Link]()
OUTPUT
[==================================================] 100.0%
1662.8/1662.8MB downloaded Top 5 words similar to
'computer': computers: 0.7979
laptop: 0.6640
laptop_computer: 0.6549
Computer: 0.6473
com_puter: 0.6082
step-by-Step Explanation with Examples
🔹 Step 1: Understanding Word Embeddings
What are word embeddings?
Word embeddings are numerical vector representations of words where:
Similar words have similar vectors
Meaning and context are captured mathematically
Example:
Word Meaning
computer electronic device
laptop portable computer
➡ Their vectors are close in space because meanings are related.
🔹 Step 2: Loading Pre-trained Word Embeddings
We use Word2Vec (Google News) which converts each word into a 300-dimensional vector.
Why pre-trained?
Trained on millions of documents
Captures rich semantic relationships
Example:
computer → [0.21, -0.45, 0.33, ..., 0.12] (300 values)
👉 Humans can’t visualize 300 dimensions → we reduce dimensions
🔹 Step 3: Selecting Domain-Specific Words
We select 10 technology-related words:
computer, software, hardware, internet, network,database, cloud, security, algorithm,
artificial_intelligence
Why domain-specific?
Ensures meaningful semantic clustering
Helps analyze relationships within one domain
Example:
computer ↔ software → closely related
computer ↔ cloud → moderately related
🔹 Step 4: Creating Word Vectors
Each selected word is converted into a vector using Word2Vec.
Example:
Word Vector Size
computer 300
software 300
cloud 300
➡ Final matrix size:
10 words × 300 dimensions
🔹 Step 5: Dimensionality Reduction using PCA
What is PCA?
Principal Component Analysis (PCA) is a linear technique that:
Reduces dimensions
Retains maximum variance
Why PCA?
Simple
Fast
Good for initial visualization
Example:
300D vector → 2D vector
computer → (1.2, -0.8)
software → (1.0, -0.6)
🔹 Step 6: PCA Visualization Interpretation
When plotted:
Nearby points = similar meaning
Far points = unrelated meaning
Observations:
computer, hardware, software → cluster together
internet, network, cloud → form another cluster
algorithm, artificial_intelligence → appear close
👉 PCA shows global structure, but not fine details
🔹 Step 7: Dimensionality Reduction using t-SNE
What is t-SNE?
t-Distributed Stochastic Neighbor Embedding is a non-linear technique that:
Preserves local similarity
Shows clusters more clearly
Why t-SNE?
Better than PCA for semantic clustering
Commonly used for NLP visualization
🔹 Step 8: t-SNE Visualization Interpretation
Observations:
Stronger and clearer clusters than PCA
Words with similar usage appear very close
Example clusters:
{computer, laptop, hardware}
{internet, network, cloud}
{algorithm, artificial_intelligence}
👉 t-SNE reveals hidden semantic relationships
🔹 Step 9: Semantic Similarity Generation
What does this mean?
Given one word, find semantically closest words using cosine similarity.
Example:
Input word:
computer
Output:
computers, PC, laptop, desktop, machine
Explanation:
All outputs represent computing devices
Vectors point in similar directions
🔹 Step 10: Contextually Rich Output using Embeddings
Embeddings understand context, not just spelling.
Example:
king − man + woman = queen
Similarly:
computer → laptop → desktop
👉 This proves embeddings capture meaning, not just words
🔹 Final Result Explanation
PCA gives basic visualization
t-SNE provides clear semantic clusters
Similar-word generation validates embedding quality