Module 4: Semantic Analysis

- Lexical semantics:

- Introduction

- Relations among lexemes & their senses (homonymy, polysemy, synonymy, hyponymy, meronymy)

- Lexical resources: WordNet, FrameNet, Distributional Semantics

- Robust Word Sense Disambiguation (WSD):

- Rule-based approaches

- Supervised approaches

- Unsupervised approaches

- Neural models for semantics:

- Sequence-to-sequence models (e.g., machine translation)

- Attention mechanisms

- Transformers (BERT, T5)

- Language generation models (GPT, Variational Autoencoders)

- Advanced NLP techniques: Transfer learning, few-shot learning, zero-shot learning

Question Marks

What is lexical semantics? 2

Lexical semantics is a subfield of semantics that studies the meaning of individual words and the
relationships between them. It involves analyzing word meanings, word structures, and their usage in
context. Lexical semantics also explores how words relate through synonyms, antonyms, homonyms, and
hierarchical structures like hyponymy and hypernymy.

Define homonymy. 2

Homonymy refers to words that have the same spelling or pronunciation but different, unrelated
meanings. For example, the word “bat” can mean a flying mammal or a piece of sports equipment used in
cricket. These words cause ambiguity and their meaning can only be understood by looking at the
sentence context.

What is polysemy? 2

Polysemy occurs when a single word has multiple related meanings. For instance, the word “man” can
mean a male human, an adult male, or even humanity in general. Unlike homonyms, polysemous words
have meanings that are connected by a common origin or concept, making them contextually related.
Question Marks

Explain synonymy. 2

Synonymy refers to a relationship between words where two or more different words share similar or
identical meanings. For example, “big” and “large” are synonyms. Synonyms help in improving text
diversity and readability. They are essential in tasks like paraphrasing, information retrieval, and semantic
search in Natural Language Processing (NLP).

What is hyponymy? 2

Hyponymy is a semantic relationship in which a specific term (hyponym) falls under a broader category
(hypernym). For example, “rose” is a hyponym of “flower.” It shows a class-object relationship and helps
organize vocabulary hierarchically. Hyponymy is important in building knowledge bases and in tasks like
information retrieval.

What is WordNet? 2

WordNet is a lexical database for English that groups words into sets of synonyms called synsets. It
connects these words through semantic relationships such as hypernymy, hyponymy, antonymy, and
meronymy. WordNet is widely used in NLP tasks like word sense disambiguation, semantic similarity, and
automatic thesaurus generation.
Question Marks

Define word sense disambiguation (WSD). 2

Word Sense Disambiguation (WSD) is the process of identifying the correct meaning of a word based on
its context. Many words have multiple meanings, like “bank” (river bank or financial institution), and WSD
helps in selecting the most appropriate meaning. It is vital for tasks like translation, search, and chatbots.

Question Marks

What is a sequence-to-sequence model? 5

- Sequence-to-Sequence (Seq2Seq) models are deep learning models used to transform one sequence
into another, especially in tasks like machine translation and text summarization.
- It has two parts: Encoder, which processes the input sequence into a fixed representation, and Decoder,
which generates the output sequence.
- Originally based on RNNs and LSTMs, Seq2Seq models capture the context and handle variable-length
inputs and outputs.
- They are ideal for tasks like English-to-French translation or chatbot conversations.
- When enhanced with attention mechanisms, these models can focus on relevant input parts while
decoding.
- Seq2Seq is used in many NLP applications such as speech recognition, summarization, and grammar
correction.
Question Marks

- Their ability to remember and relate sequences makes them essential in modern NLP.

Describe the concept of WordNet. 5

- WordNet is a lexical database that organizes English words into synsets (sets of synonyms) representing
distinct concepts.
- It defines relationships like synonymy, antonymy, hyponymy, hypernymy, and meronymy between
words.
- Each synset contains a gloss (definition) and example usage, helping understand the sense of the word.
- Example: The word “car” may belong to a synset with “automobile,” and be linked to “vehicle” via
hypernymy.
- WordNet is widely used in word sense disambiguation (WSD), semantic similarity, and information
retrieval.
- It supports NLP applications like text summarization, search engines, question answering, and chatbots.
- It bridges lexical gaps in language understanding, making it easier for machines to understand human
language.

Explain robust word sense disambiguation (WSD). 5

- WSD is the task of identifying the correct meaning of a word based on the context in which it appears.
- For example, the word “bank” can mean a financial institution or a riverbank—WSD finds the right
meaning.
- Rule-based approaches use hand-written rules or dictionaries like WordNet.
Question Marks

- Supervised approaches train models using annotated data and use classifiers like SVM, Decision Trees,
and Neural Networks.
- Unsupervised approaches cluster word usages in similar contexts without labeled data.
- Knowledge-based approaches (e.g., Lesk Algorithm) match dictionary definitions with nearby words.
- Hybrid approaches combine two or more methods to improve accuracy.
- Robust WSD is vital in machine translation, information retrieval, and question-answering systems.

Discuss the sequence-to-sequence models used in NLP. 5

- Seq2Seq models are neural network architectures used to map one sequence to another.
- They consist of an encoder, which reads the input, and a decoder, which produces the output.
- RNNs and LSTMs were initially used in Seq2Seq models but now attention and transformers are more
common.
- Applications include translation (e.g., English to French), text summarization, speech recognition, and
chatbots.
- The attention mechanism allows the decoder to focus on important input tokens, improving
performance.
- Models like T5 and GPT have evolved from Seq2Seq concepts and deliver state-of-the-art results.
- These models can handle varying sequence lengths and maintain contextual relationships.
- Seq2Seq has revolutionized NLP by enabling machines to understand and generate complex language
sequences.

Explain the attention mechanisms in transformers. 5

- Attention in transformers allows models to focus on relevant parts of input when generating each output
token.
- It assigns weights to words based on importance in context, unlike RNNs that process sequentially.
- Self-attention enables each word to consider other words in a sentence to understand meaning better.
- For example, in the sentence “She saw the dog with the telescope,” attention helps identify who had the
telescope.
Question Marks

- Transformers use multi-head attention, allowing the model to capture different relationships
simultaneously.
- This mechanism improves performance in tasks like translation, summarization, and question
answering.
- Attention helps models overcome limitations of long-range dependencies in sequences.
- It is the core component of transformer-based models like BERT, GPT, and T5.

Describe the concept of language generation models. 5

- Language generation models are designed to produce human-like text based on context or prompts.
- They learn to predict the next word or generate entire sequences from input data.
- Examples include GPT, T5, and Variational Autoencoders (VAEs).
- These models are pre-trained on massive corpora and can generate responses, summaries, or
translations.
- Applications include chatbots, story generation, email writing, and automated content creation.
- T5 treats every NLP task as text-to-text, simplifying model structure and training.
- GPT excels at creative and open-ended generation, useful for interactive assistants and dialogue
systems.
- Language generation models are powerful tools for automating and enhancing written communication
in AI systems.

What is the difference between homonymy and polysemy? 5

- Homonymy occurs when two words share the same form (spelling or pronunciation) but have unrelated
meanings. Example: “bat” (animal) and “bat” (cricket equipment).
- Polysemy is when a single word has multiple related meanings. Example: “man” can refer to an adult
male or human species.
- Homonyms often confuse NLP models due to completely separate meanings and require context to
disambiguate.
- Polysemous words usually share a common origin, and their meanings are easier to infer from context.
- Homonymy is more problematic for machine translation and text understanding.
- Polysemy is natural and common in language and often captured better by-word embeddings.
- Both phenomena are important in word sense disambiguation.
- Distinguishing between them helps machines understand semantics more accurately.
 Question Marks

Last-minute revision cheat sheet for Module 4: Semantic Analysis (based on both 2-mark and 5-mark answers)

NLP Module 4 – Semantic Analysis Cheat Sheet

Lexical Semantics

• Lexical Semantics: Study of word meanings and their relationships.

• Homonymy: Same spelling/pronunciation, unrelated meanings (e.g., bat - animal vs cricket).

• Polysemy: One-word, multiple related meanings (e.g., man - male, human).

• Synonymy: Words with similar meanings (e.g., big ~ large).

• Hyponymy: Specific to general word hierarchy (rose is a hyponym of flower).

Link - Lexical Semantics, Synonymy, Antonymy, Hypernym, Hyponymy, Metonymy, Polysemy

Lexical Resources

• WordNet: Synonym sets (synsets) + semantic relations (e.g., synonymy, hyponymy).

• FrameNet: Connects words to their semantic frames (situation-based).

• Distributional Semantics: Words used in similar contexts have similar meanings (e.g., word embeddings).

Word Sense Disambiguation (WSD)

• Definition: Finding the correct meaning of a word in context.

• Approaches:

o Rule-Based: Uses dictionaries/rules (e.g., Lesk algorithm).

o Supervised: Trained on annotated data (SVM, Decision Trees).

o Unsupervised: Clustering based on usage patterns (no labeled data).

• Example: “Bank” – river or finance? WSD resolves this using context.

Neural Models for Semantics

• Seq2Seq Models: Encoder-decoder models for tasks like translation.

• Attention: Focus on relevant input parts during decoding.

• Transformers: Use self-attention for parallel processing (e.g., BERT, T5).

• Language Generation Models: GPT, T5, VAEs used for text creation (e.g., summaries, responses).

Advanced NLP Techniques

• Transfer Learning: Using pre-trained models on new tasks (e.g., fine-tuning BERT).

• Few-shot Learning: Model learns from a few examples.

• Zero-shot Learning: Model performs tasks it hasn’t seen before.

Quick Tips:

• Polysemy = related meanings

• Homonymy = unrelated meanings

 • WSD = resolves word ambiguity

• WordNet = semantic dictionary

• Transformers = parallel + attention-based

• GPT/T5 = text generation

10-mark answers

1. Explain the significance of lexical semantics in NLP.

Lexical semantics is a vital branch of semantics that deals with understanding the meaning of individual words and
the relationships between them. It focuses on how words convey meaning and how their interconnections contribute
to sentence-level interpretation. In Natural Language Processing (NLP), lexical semantics helps computers interpret
human language accurately by modeling meaning relationships like synonymy, antonymy, homonymy, polysemy,
hyponymy, and meronymy.

Lexical semantics is essential for various NLP tasks such as word sense disambiguation, sentiment analysis,
information retrieval, and question answering. For instance, in search engines, understanding that “car” and
“automobile” refer to the same concept improves search accuracy. Similarly, in sentiment analysis, knowing that
“happy” and “joyful” convey similar sentiments enhances classification precision.

Flow of Lexical Semantics in NLP:

Word → Context Identification → Relationship Mapping → Semantic Network Building → Meaning Interpretation

Example:
Input: “The bank raised interest rates.”
Output: Lexical semantics identifies “bank” as a financial institution (not riverbank) using context.

Thus, lexical semantics provides a structured foundation for meaning representation and reasoning in NLP. It allows
models to go beyond syntax and truly comprehend semantics, making natural language understanding, translation,
and summarization more contextually accurate and meaningful.

2. Describe how transformers have changed NLP.

Transformers have revolutionized NLP by introducing a new architecture based on self-attention mechanisms rather
than recurrent or convolutional structures. Unlike RNNs or LSTMs, which process data sequentially, transformers
process entire sequences in parallel, drastically improving speed and performance.

The transformer architecture includes two key components — Encoder and Decoder. The encoder reads the input
text, and the decoder generates the output. The self-attention mechanism allows each word to consider every other
word in the sequence, capturing long-range dependencies and contextual meaning efficiently.

Diagram (simplified flow):

Input Sentence → Encoder (Self-Attention + Feed Forward) → Decoder (Attention + Output Generation)

Impact on NLP:
Transformers power state-of-the-art models like BERT, GPT, and T5. These models are capable of understanding,
generating, and reasoning with human-like language. They enable tasks such as question answering, summarization,
translation, and sentiment analysis with high accuracy.
Example Input/Output:
Input: “Translate: I love learning NLP → French”
Output: “J’aime apprendre le traitement du langage naturel.”

Transformers have set the foundation for modern NLP through scalability, contextual understanding, and transfer
learning, replacing traditional sequence models in nearly every NLP application today.

3. Use WordNet to find synonyms for a given word.

WordNet is a large lexical database of English that organizes words into synsets (sets of synonyms) and interlinks
them through semantic relationships like hypernymy, hyponymy, and antonymy. It’s an essential resource in NLP for
semantic similarity, word sense disambiguation, and automatic thesaurus generation.

Process of Finding Synonyms using WordNet:

1. Input a target word.

2. Retrieve corresponding synsets.

3. Extract all lemmas (synonyms) from each synset.

4. Display results in a structured format.

Flowchart:
Input Word → Search in WordNet → Retrieve Synsets → Extract Lemmas → Output Synonyms

Example Implementation (Python):

from [Link] import wordnet as wn

word = "happy"

synonyms = []

for syn in [Link](word):

for lemma in [Link]():

[Link]([Link]())

print(set(synonyms))

Output:
{'happy', 'contented', 'joyful', 'pleased', 'glad'}

Real-life Application:
WordNet is widely used in search engines for query expansion, where user queries like “car” automatically include
synonyms like “automobile,” improving search relevance.

Thus, WordNet serves as a powerful semantic backbone for NLP systems, enriching text analysis and knowledge
representation.

4. Apply a sequence-to-sequence model for machine translation.

A Sequence-to-Sequence (Seq2Seq) model is a neural network framework designed to map one sequence (like a
sentence) to another, making it ideal for machine translation. It consists of two key components:

• Encoder: Reads and compresses the input sequence into a context vector.

• Decoder: Generates the target sequence from this vector.

Earlier Seq2Seq models used RNNs and LSTMs, but modern versions integrate attention mechanisms for improved
performance.

Workflow:
Input Sentence → Encoder (Context Representation) → Decoder (Output Translation) → Target Language Sentence

Example Implementation (Simplified):

Input: “I love language processing.”
Model (English → French)
Output: “J’aime le traitement du langage.”

Diagram:

English Input → Encoder (RNN/LSTM) → Context Vector → Decoder → French Output

Applications:

• Google Translate

• Speech-to-speech translation

• Multilingual chatbots

Seq2Seq models revolutionized NLP by handling variable-length inputs and outputs, enabling accurate translations
and forming the basis for advanced transformer architectures.

5. Use an attention mechanism in a transformer model.

The attention mechanism allows a model to focus on the most relevant parts of the input sequence when generating
output. Instead of giving equal importance to every word, attention assigns weights based on contextual relevance.

In transformers, self-attention helps each word in a sequence interact with all other words, capturing dependencies
regardless of distance. Multi-head attention enhances this by learning multiple relationships in parallel.

Flowchart:
Input Sentence → Query, Key, Value Vectors → Attention Scores → Weighted Sum → Output Representation

Example (Sentence Disambiguation):

Input: “She saw the man with the telescope.”
Attention focuses on linking “man” with “telescope” correctly, identifying who had the telescope.

Mathematical Core:
Attention(Q, K, V) = softmax(QKᵀ / √dₖ) × V

Real-life Application:

• Improves machine translation accuracy (aligns source-target words).

• Used in BERT, GPT, and T5 for tasks like summarization and question answering.

Attention enables models to “understand context” deeply, making transformers the most effective architecture for
contextual NLP understanding.

6. Apply transfer learning to improve an NLP model.

Transfer learning in NLP involves using pre-trained language models (like BERT, GPT, or RoBERTa) on large corpora
and fine-tuning them for specific downstream tasks such as sentiment analysis, question answering, or text
classification.
Process:

1. Pre-training: The model learns general language features from massive datasets.

2. Fine-tuning: The model is retrained on smaller, task-specific data to adapt to a particular domain.

Flowchart:
Large Corpus → Pre-trained Model → Fine-tuning → Task-Specific Model → Predictions

Example Implementation:
Input: Movie reviews dataset
Model: Pre-trained BERT fine-tuned for sentiment analysis
Output: Positive / Negative sentiment

Benefits:

• Requires less data and time for training.

• Achieves state-of-the-art accuracy.

• Handles domain-specific vocabulary better.

Applications:

• Chatbots (fine-tuned for company-specific responses)

• Medical text classification

• Financial document summarization

Transfer learning has made NLP more efficient, accurate, and adaptable, significantly reducing the cost of building
powerful language systems.

7. Compare different approaches to Word Sense Disambiguation (WSD).

Word Sense Disambiguation (WSD) determines the correct meaning of a word based on its context. There are
multiple approaches, each with distinct methodologies and strengths.

Approach Description Example / Tools Advantages Limitations

Rule-Based Uses hand-crafted rules and Lesk Algorithm Simple, Limited scalability
dictionaries like WordNet interpretable

Supervised Trains models on labeled SVM, Decision Tree, High accuracy on Needs large labeled
datasets using classifiers Neural Nets known data corpora

Unsupervised Clusters words based on Word embeddings, No labeled data Lower precision
usage context clustering needed

Hybrid Combines rule-based + ML Knowledge + Balanced accuracy Complex

methods Statistical Models implementation

Example:
Input: “He sat on the bank.”
Rule-based: Matches “bank” near “river” → riverbank.
Supervised: Trained classifier predicts financial vs river sense based on features.

Applications: Machine Translation, Search Engines, and Chatbots.

WSD ensures contextual clarity and meaning accuracy, making it foundational for semantic understanding in NLP.