Program 5:
Use word embeddings to create meaningful sentences for creative tasks. Retrieve similar words for a seed
word. Create a sentence or story using these words as a starting point. Write a program that: Takes a seed
word. Generates similar words. Construct a short paragraph using these words.
Title
Creative Text Generation Using Pre-trained Word Embeddings (GloVe) in Python
Aim
To generate a creative paragraph using a seed word by retrieving similar words from pre-trained GloVe
word embeddings.
Objective
To understand pre-trained word embeddings.
To use Gensim to load GloVe word vectors.
To retrieve semantically similar words.
To generate meaningful sentences and a paragraph using NLP techniques.
Theory
Natural Language Processing (NLP)
Natural Language Processing is a branch of Artificial Intelligence that enables computers to understand,
interpret, and generate human language.
Word Embeddings
Word embeddings are dense vector representations of words that capture semantic meaning. Words with
similar meanings have similar vector representations.
Example:
king – queen
man – woman
doctor – hospital
GloVe (Global Vectors for Word Representation)
GloVe is a pre-trained word embedding model developed by Stanford University. It learns word
representations by analyzing global word-word co-occurrence statistics.
In this experiment:
We use glove-wiki-gigaword-100
Each word is represented as a 100-dimensional vector
Gensim Library
Gensim is a Python library used for topic modeling and vector space modeling. It provides access to pre-
trained word embeddings.
Software Requirements
Python 3.x
Jupyter Notebook / Google Colab
Required Libraries:
o nltk
o gensim
o random
Algorithm
1. Install and import required libraries.
2. Download NLTK tokenizer resource ('punkt').
3. Load pre-trained GloVe word embeddings using Gensim.
4. Accept a seed word from the user.
5. Retrieve top 5 similar words using most_similar().
6. Generate meaningful sentences using sentence templates.
7. Combine sentences to form a paragraph.
8. Display the generated paragraph.
Source Code:
Continue…
OUTPUT:
Result
Thus, a creative paragraph was successfully generated using a seed word and pre-trained GloVe word
embeddings.
Applications
Content generation
Chatbots
Story writing
AI-based creative writing tools
NLP research
Conclusion
This experiment demonstrates how pre-trained word embeddings can be used to generate meaningful and
creative text. It highlights the importance of semantic similarity in Natural Language Processing
applications.
Viva Questions & Answers
1. What is Natural Language Processing (NLP)?
Answer:
Natural Language Processing (NLP) is a branch of Artificial Intelligence that enables computers to
understand, interpret, and generate human language in a meaningful way.
2. What are word embeddings?
Answer:
Word embeddings are numerical vector representations of words in a continuous vector space where
semantically similar words have similar vector representations.
3. What is GloVe?
Answer:
GloVe (Global Vectors for Word Representation) is a pre-trained word embedding model developed by
Stanford University. It learns word vectors using global word-word co-occurrence statistics from a large
corpus.
4. What is the dimension of the GloVe model used in this program?
Answer:
The program uses glove-wiki-gigaword-100, which means each word is represented as a 100-
dimensional vector.
5. What is the purpose of the most_similar() function?
Answer:
The most_similar() function retrieves words that are semantically similar to a given word based on cosine
similarity between word vectors.
6. What similarity measure is commonly used in word embeddings?
Answer:
Cosine similarity is commonly used to measure similarity between two-word vectors.
7. What happens if the seed word is not present in the vocabulary?
Answer:
A Key Error occurs. In this program, the exception is handled using a try-except block, and a message is
displayed to the user.
8. Why is pre-trained word embedding used instead of training from scratch?
Answer:
pre-trained embeddings save time and computational resources because they are already trained on large
datasets and capture rich semantic relationships.
9. What is the role of the Gensim library in this program?
Answer:
Gensim is used to download and load the pre-trained GloVe model and to compute similar words using
vector operations.
10. Why is NLTK used in this program?
Answer:
NLTK is used for natural language processing tasks such as sentence tokenization. The 'punkt' resource is
downloaded for sentence splitting.
11. What is the purpose of the random module?
Answer:
The random module is used to randomly select sentence templates, making the generated paragraph more
varied and creative.
12. What type of text generation is implemented in this program?
Answer:
This program implements template-based text generation using semantically similar words from word
embeddings.
13. What are the advantages of word embeddings over one-hot encoding?
Answer:
Dense representation
Captures semantic meaning
Lower dimensionality
Words with similar meaning are closer in vector space
14. What are the limitations of this program?
Answer:
Generated text may not always be grammatically perfect.
It depends only on similarity, not context.
It uses fixed templates instead of advanced language models.
15. What are some real-world applications of word embeddings?
Answer:
Machine translation
Sentiment analysis
Chatbots
Information retrieval
Text classification
16. What is cosine similarity?
Answer:
Cosine similarity measures the cosine of the angle between two vectors. It ranges from -1 to 1. Higher
values indicate greater similarity.
Formula:
A⋅B
Cosine Similarity =
∣∣ A ∣∣ ∣ ∣B ∣∣
17. How does this program generate a paragraph?
Answer:
It:
1. Takes a seed word.
2. Finds top 5 similar words.
3. Inserts them into predefined sentence templates.
4. Combines multiple sentences to form a paragraph.
18. What is the difference between GloVe and Word2Vec?
Answer:
GloVe Word2Vec
Uses global co-occurrence statistics Uses the local context window
Matrix factorization approach Neural network-based model
Faster training for large corpora Predictive model (CBOW/Skip-gram)
19. What is meant by vocabulary in NLP?
Answer:
Vocabulary refers to the set of unique words present in a trained model.
20. How can this program be improved?
Answer:
Use transformer-based models (like GPT or BERT)
Improve grammar using language models
Add context awareness
Use dynamic sentence generation instead of templates