Module 4

SEMANTICS AND PRAGMATICS : Requirements for representation, First-

Order Logic, description Logics – Syntax-driven Semantic analysis, Semantic
attachments – Word Senses, Relations between Senses, Thematic Roles,
selectional restrictions – Word Sense Disambiguation, WSD using Supervised,
Dictionary & Thesaurus, Bootstrapping methods – Word Similarity using
Thesaurus and distributional methods.
SEMANTICS AND PRAGMATICS
Introduction to Semantics and Pragmatics
Both Semantics and Pragmatics are branches of linguistics that deal with the
meaning of language — but they focus on different aspects of meaning.
What is Semantics?
Semantics is the study of the literal meaning of words, phrases, and
sentences.
It deals with:
• What words mean on their own.
• How meanings of words combine in sentences.
• Rules that decide how these meanings are structured.
Semantics = Dictionary meaning + Grammar meaning
Example of Semantics:
• Sentence: “The cat is on the mat.”
Semantically, this sentence means:
• There is an animal called a cat.
• It is located on top of something called a mat.
• The sentence is stating a fact.
Semantics focuses only on the literal meaning, not who said it, why, or in
what situation.
Components of Semantics:
1. Lexical Semantics: Meaning of individual words (like "cat" means a
small domesticated animal).
2. Compositional Semantics: How meanings of words combine to form
the meaning of the sentence.
3. Truth-Conditional Semantics: Whether a sentence is true or false in
a particular situation.

1
What is Pragmatics?
Pragmatics is the study of how context influences the interpretation of
meaning.
It looks at:
• Who is speaking?
• What is the situation?
• What is the speaker’s intention?
• What does the listener understand?
Pragmatics = Meaning based on context and intention
Example of Pragmatics:
• Sentence: “It’s cold in here.”
Semantically, it just means the temperature is low.
But pragmatically, the meaning could be:
• A request: "Please close the window."
• A complaint: "I’m not comfortable."
• A hint: "Let’s go somewhere warmer."
So pragmatics looks at what the speaker actually meant in a particular
context.
Key Concepts in Pragmatics:
1. Deixis: Words like here, there, you, now — whose meaning depends on
who is speaking and when.
2. Presupposition: Background assumptions.
o E.g., “John stopped smoking.” → Assumes John used to smoke.
3. Speech Acts: Saying something is doing something (e.g., "I promise" is
making a promise).
4. Implicature: Hidden meaning understood indirectly.
o E.g., “Can you pass the salt?” is actually a request, not a question
about ability.
Semantics vs Pragmatics – A Quick Comparison

Aspect Semantics Pragmatics

Focus Literal meaning Meaning in context

2
Aspect Semantics Pragmatics

Based on Words and grammar Situation, intention, tone, etc.

Example “It’s cold” = low temperature “It’s cold” = Close the window?

Interpretation Fixed and general Flexible and varies with context

Why are Semantics and Pragmatics Important?

• In natural language processing (NLP), understanding both is essential
for:
o Machine translation
o Chatbots
o Question answering
o Sentiment analysis
• Humans naturally use both semantic knowledge and pragmatic
understanding to communicate effectively.
Summary in Simple Words:
• Semantics = what the words say (the exact dictionary meaning).
• Pragmatics = what the words mean in the current situation.
Requirements for representation
What is "Representation" in NLP?
In NLP, representation means how we convert human language (like English,
Hindi, etc.) into a form that a computer can understand and work with.
This could be:

• Words ➝ numbers (e.g., embeddings like Word2Vec or BERT)

• Sentences ➝ trees (e.g., syntax trees)

• Meanings ➝ structures (e.g., feature structures, frames)

Why are Requirements Important in NLP?
Because human language is complex, we need a good representation to
ensure that:
• Computers understand the meaning correctly
• They respond logically
• They handle new sentences properly
• They don’t get confused by different meanings or grammar
3
Requirements for Good Representation in NLP
1. Correctness (Linguistic Accuracy)
• The representation should faithfully reflect the meaning of the text.
• It must preserve grammar, word order, and context.
Example:
The sentence “The cat chased the mouse”
should not be misunderstood as “The mouse chased the cat”
2. Completeness
• All the important elements (subject, object, verb, modifiers) must be
represented.
Example:
In “She quickly closed the door,” the adverb “quickly” must be captured, not
ignored.
3. Efficiency
• The representation must be compact and allow fast processing for real-
time applications like translation or speech recognition.
Example: Vector embeddings (like BERT) make it easier to compare words in
few dimensions instead of comparing full dictionaries.
4. Understandability (Human Interpretability)
• It should be possible for humans (linguists or developers) to
understand what the model is doing.
Example:
Parse trees or part-of-speech tagging are easier to read than raw neural
network weights.
5. Flexibility (Handling Variations)
• Should handle different forms, synonyms, tenses, passive voice, etc.
Example:
“Ravi eats an apple”
and
“An apple is eaten by Ravi”
should represent the same core meaning.
6. Inferability (Enable Reasoning)
• The system should be able to derive new facts or understand implicit
meaning.

4
Example:
If it knows:
"John is a doctor"
and "Doctors treat patients",
It should infer: "John treats patients".
7. Minimal Redundancy
• Should not store repetitive or duplicate meanings of a sentence
unnecessarily.
Example: Don't store the same word vector multiple times if the word appears
often.
8. Generalization (Handle New Sentences)
• Should work for new or unseen sentences or phrases, not just the ones
seen in training.
Example:
Even if the model never saw “The boy fed the giraffe,” it should still
understand it using knowledge of verbs and nouns.
9. Scalability
• Should support large vocabulary and many documents without slowing
down or crashing.
NLP models like GPT or BERT handle millions of sentences, so representation
must be scalable.
10. Consistency
• Words or phrases should always be represented in the same way in
similar contexts.
Example:
The word “bank” in the context of money should not be confused with the
riverbank.
This is often solved using contextual word embeddings (like BERT).
Summary Table (NLP-Focused)

Requirement NLP Meaning (Simple)

Correctness Sentence meaning must be represented accurately

Completeness Every detail like subject, verb, object must be included

Efficiency Fast to process in real-time NLP tasks

Understandability Easy for humans to follow and debug

5
Requirement NLP Meaning (Simple)

Flexibility Can handle different forms, voices, and synonyms

Inferability Supports drawing logical conclusions

Minimal Redundancy Avoid duplicate or repeated info

Generalization Works on unseen/new sentences

Scalability Handles large texts and vocabularies

Consistency Same word has same meaning in same context

Real NLP Example

Let’s take the sentence:
“Riya bought a new red dress yesterday.”
A good representation will include:
• Who? → Riya
• Did what? → bought
• What? → a new red dress
• When? → yesterday
And will also:
• Know that “dress” is an object, “bought” is a past action
• Store word meanings in a way that helps understand similar sentences
• Allow search systems or chatbots to respond appropriately or translate
it
First-Order Logic
First-Order Logic (FOL), also called Predicate Logic, is a formal mathematical
language used to represent:
• Facts about the world
• Relationships between objects
• Rules and conditions
• Logical reasoning
In simpler terms, it gives us a structured way to represent the meaning of
natural language sentences, which computers can understand and reason
about.
2. Why Do We Use FOL in NLP?

6
Natural language (like English, Tamil, or Hindi) is full of ambiguity and
complex structure. For example:
"Every student loves a teacher."
This sentence can have different meanings depending on how we interpret "a
teacher."
FOL helps by:
• Representing the exact meaning
• Removing confusion
• Allowing logical reasoning
So, in NLP, FOL is used for:
• Understanding sentence meaning (semantics)
• Question answering
• Information extraction
• Knowledge representation
• Chatbots and dialogue systems
3. Components of FOL
Let’s understand the building blocks of FOL in NLP.

Component Description Example

Constants Names of specific objects John, India, Red

Variables Represent general objects x, y, z

State properties or
Predicates Student(x), Loves(x, y)
relationships

FatherOf(x) means the

Functions Return an object
father of x

Express “for all” or “there

Quantifiers ∀ (for all), ∃ (exists)
exists”

Logical ∧ (and), ∨ (or), ¬ (not), →

Combine statements
Connectives (implies)

4. Converting Natural Language to FOL

Example 1: "John is a student."
• FOL: Student(John)

7
 • Student is a predicate, and John is a constant.
Example 2: "All students study."
• FOL: ∀x (Student(x) → Studies(x))
• Means: "For every x, if x is a student, then x studies."
Example 3: "Some students play football."
• FOL: ∃x (Student(x) ∧ PlaysFootball(x))
• Means: "There is at least one x who is a student and plays football."
Example 4: "Every teacher teaches some student."
• FOL: ∀x (Teacher(x) → ∃y (Student(y) ∧ Teaches(x, y)))
• Means: "For every x, if x is a teacher, then there exists a y such that y
is a student and x teaches y."
5. Logical Reasoning in NLP
Once we represent sentences in FOL, we can perform reasoning. That means
we can derive new facts from existing ones.
Example:
• ∀x (Student(x) → Studies(x))
• Student(Ravi)
We can deduce: Studies(Ravi)
This is called inference. It’s very useful in:
• Chatbots that answer questions
• Intelligent systems that make decisions
• Semantic search engines
6. FOL vs Natural Language

Feature Natural Language First-Order Logic

Ambiguity Yes No (very precise)

Easy for humans Yes Not directly (needs learning)

Easy for computers No Yes (structured & clear)

Can perform reasoning Hard Easy

So, FOL translates human language into a machine-understandable logic

format.

8
7. How NLP Systems Use FOL
Semantic Parsing
Semantic parsing is the task of converting a sentence into a logical form (like
FOL).
Example:
Sentence: “Which students play football?”
FOL: λx. Student(x) ∧ PlaysFootball(x)
This helps the system understand exactly what is being asked and how to
answer it.
Information Extraction
From text like:
“John lives in Paris and works at Google.”
FOL can extract and represent:
• LivesIn(John, Paris)
• WorksAt(John, Google)
These facts can be stored in a knowledge base for future queries.
Dialogue Systems and Chatbots
Chatbots can use FOL to:
• Understand user intentions
• Maintain logical consistency
• Give relevant responses
Example:
• User: “Does Ravi study?”
• Bot checks if Studies(Ravi) is true in its knowledge base using FOL.
Machine Translation
FOL helps preserve logical meaning when translating from one language to
another.
For example:
• English: “All birds can fly.”
• FOL: ∀x (Bird(x) → CanFly(x))

• Tamil translation: "அனைத்து பறனைகளும் பறக்கிை் றை."

• Logical meaning is preserved!

9
8. Limitations of FOL in NLP
• Human language is rich and vague.
• FOL is rigid and can’t capture emotion, sarcasm, or contextual meaning
easily.
• Handling uncertainty is difficult in FOL. (That’s why probabilistic logic
is also used in NLP.)
9. Summary

Point Explanation

What A logic system to represent facts and relationships

Use in NLP To convert language into a machine-readable logical form

Benefits Precise, allows reasoning, enables intelligent systems

Tasks Semantic parsing, question answering, information extraction

Limitations Can’t easily handle ambiguity, context, or emotion

Different Examples with Explanation

Example 1: "Every cat chases some mouse."
This involves two quantifiers:
• ∀x: for all cats
• ∃y: there exists some mouse
FOL: ∀x (Cat(x) → ∃y (Mouse(y) ∧ Chases(x, y)))
Meaning: For every x, if x is a cat, then there exists a y such that y is a mouse
and x chases y.
Example 2: "No student sleeps in class."
“No” means not even one — so we use negation.

FOL: ¬∃x (Student(x) ∧ SleepsInClass(x))

Or equivalently:

FOL: ∀x (Student(x) → ¬SleepsInClass(x))

Meaning: For all x, if x is a student, then x does not sleep in class.
Example 4: "If a person is a doctor, then they help patients."
This is a conditional sentence using implication.
FOL: ∀x (Doctor(x) → Helps(x, Patients))

10
Meaning: For every x, if x is a doctor, then x helps patients.
Example 5: "There is a boy who likes all kinds of fruits."
This has a universal quantifier inside an existential quantifier.

FOL: ∃x (Boy(x) ∧ ∀y (Fruit(y) → Likes(x, y)))

Meaning: There exists an x, such that x is a boy, and for all y, if y is a fruit,
then x likes y.
Natural Language Sentences and Their Logical Forms

English Sentence First-Order Logic Representation

"All humans are mortal." ∀x (Human(x) → Mortal(x))

"Sita likes chocolate." Likes(Sita, Chocolate)

"Someone is sleeping." ∃x (Sleeping(x))

"Every parent loves their child." ∀x (Parent(x) → Loves(x, ChildOf(x)))

"Some teachers teach in college." ∃x (Teacher(x) ∧ TeachesIn(x, College))

Where FOL Is Used in NLP

NLP Application How FOL Helps

Question Answering Helps interpret questions logically and search facts

Chatbots/Virtual
Maintains conversation logic, avoids contradictions
Agents

Knowledge Graphs Stores logical facts and rules in structured format

Text Understanding Breaks complex sentences into meaning parts

Finds documents using logical meaning, not just

Semantic Search
keywords

Limitations of FOL in NLP

• Cannot deal well with vague or fuzzy language
(e.g., “Ravi is probably late.”)
• Hard to capture emotions, sarcasm, or contextual meanings
• Doesn’t handle uncertainty (That’s what probabilistic logic helps with)
Description Logics
What Are Description Logics?

11
Description Logics (DL) are a family of formal knowledge representation
languages used to represent the structure of knowledge in a domain.
In simple words:
Description Logics help us describe concepts (things), their relationships, and
their properties in a clear, logical way — so that machines can understand
and reason about them.
They are used in Artificial Intelligence to build ontologies and knowledge-
based systems.
Where Are Description Logics Used?
• Ontologies (like in the Semantic Web)
• Knowledge Representation
• Reasoning Systems
• Medical Terminologies (e.g., SNOMED)
• OWL (Web Ontology Language) is based on DL
Simple Comparison

Area Description Logic (DL)

Purpose Represent knowledge about things

Focus Concepts (classes), Roles (relationships), Instances

Like in Programming Classes, properties, objects

Example Usage Define that “All Doctors are Humans” logically

Core Elements of Description Logic

Let’s break DL into its main components. Each part is like a building block.
1. Concepts (C) = Classes or Categories
Used to represent types of things.
Examples:
• Student
• Teacher
• Person
• Vehicle
2. Roles (R) = Relationships
Used to represent how two things are related.

12
Examples:
• hasChild
• teaches
• drives
3. Individuals (a, b, c) = Objects or Instances
Actual members of concepts.
Examples:
• Ravi, Car1, BTI_College
Example Knowledge Representation
Let’s write some common sentences in DL form.
Example 1: “All teachers are persons.”
In DL:
Teacher ⊑ Person
This means: Every teacher is a person
(Teacher is a subclass of Person)
Example 2: “Ravi is a student.”
In DL:
Student(Ravi)
Ravi is an individual of the class Student.
Example 3: “A student has at least one subject.”
In DL:
Student ⊑ ∃[Link]
Every student is related to at least one subject via the role hasSubject.
Example 4: “No student is a teacher.”
In DL:
Student ⊓ Teacher ⊑ ⊥
Means: The intersection of Student and Teacher is empty (no one can be both).
Summary of Symbols in Description Logics

Symbol Meaning Example

⊑ Subclass / Subconcept Cat ⊑ Animal

13
Symbol Meaning Example

⊓ AND / Intersection Doctor ⊓ Male

⊔ OR / Union Student ⊔ Teacher

¬ NOT / Negation ¬Employed

∃ There exists ∃[Link]

∀ For all ∀[Link] (All pets are dogs)

⊥ Bottom (empty class - contradiction) Student ⊓ Teacher ⊑ ⊥

How Does DL Help in AI or NLP?

Task Description Logic’s Role

Defines structure of knowledge (concepts, roles,

Ontology Design
instances)

Semantic Web OWL is based on DL; used in intelligent web search

Inference/Reasoning Can infer new facts from known facts

Used in building structured and explainable

Medical Systems
knowledge bases

Question Answering Helps machine "understand" domain meanings

Reasoning with Description Logic

Let’s say we define:
1. Teacher ⊑ Person
2. Person ⊑ Mammal
3. Ravi:Teacher
Now, we can infer:
• Ravi is a Person
• Ravi is a Mammal
This type of automated reasoning is one of the major strengths of DL.
Real-World Tools That Use DL

14
 Based on
Tool/Ontology Language Purpose
DL?

OWL (Web Ontology For creating web-based

Yes
Language) ontologies

Tool to build DL-based

Protégé Yes
ontologies

Fact++ / Pellet Yes DL-based reasoning engines

SNOMED CT Yes Clinical healthcare terminology

Summary (for College Students)

Feature Explanation

What is DL? A logical way to represent knowledge about a domain

Used For Ontologies, reasoning, knowledge representation

Concepts (classes), Roles (relationships), Individuals

Main Elements
(objects)

Why Important in Helps machines understand and reason about

NLP? meaning

Semantic Web, AI chatbots, medical diagnosis

Used in
systems

Syntax-driven Semantic analysis

What is Syntax-Driven Semantic Analysis?
Syntax-Driven Semantic Analysis is a method in Natural Language Processing
(NLP) where:
We use the grammar (syntax) of a sentence to find its meaning (semantics)
and then map that meaning into a formal system like a Description Logic (DL)
ontology.
In simple words:
We analyze the structure of a sentence using grammar rules, then translate
that structure into logical concepts so that a machine can understand and
reason about the sentence.
Why is this called a Description Logic (DL) Approach?
This is called a DL-based approach because:
• The final goal is to represent the sentence meaning in a DL ontology.

15
 • A DL ontology is a structured knowledge base made up of:
o Concepts (like Doctor, Person)
o Roles or Relationships (like hasSpecialization)
o Individuals (like John, Ravi)
Step-by-Step Explanation
Let’s break this down into steps, so it’s easy to understand.
Step 1: Use the Grammar (Syntax) of the Sentence
Grammar tells us how words are arranged and how they relate to each other.
Sentence: "John is a doctor who specializes in cardiology."
The grammar helps us break this sentence into parts:
• Main clause: "John is a doctor"
• Sub-clause: "who specializes in cardiology"
This structure tells us that:
• John is a person.
• John is a doctor.
• John has a specialization in cardiology.
So, we use the structure to guide the meaning.
Step 2: Map Parts to Logical Concepts (DL Ontology)
Now we map these parts into DL concepts and relationships.

What do we have?
• John: an individual
• Doctor: a concept (class)
• Cardiology: another concept
• specializes in: a relationship (role)

Now map it into logic:

In Description Logic form:
Doctor ⊑ Person → Every doctor is a person
John : Doctor → John is a doctor
hasSpecialization(John, Cardiology) → John specializes in cardiology
This is how we transform the sentence into logic using grammar.

16
What Does the Machine Understand?
Now that we’ve mapped the sentence into DL:
• The machine knows who John is.
• It knows that Doctor is a type of Person.
• It knows that John has a specialization in Cardiology.
So it can now:
• Answer questions like “Who specializes in Cardiology?”
• Classify people into roles (e.g., doctors, specialists)
• Draw conclusions using logical reasoning
Why Use This Approach?

Reason Explanation

Better understanding Grammar gives us clues about the meaning

Resolves confusion Helps deal with ambiguous or complex sentences

Meaning is stored in ontologies, which can be used

Connects to AI systems
in AI

Works with real-world Used in chatbots, medical systems, intelligent

data assistants

Real-Life Example
Let’s take another sentence:
Sentence: "Ravi is a teacher who teaches Artificial Intelligence."
1. Break into syntax:
o Main clause: "Ravi is a teacher"
o Sub-clause: "who teaches Artificial Intelligence"
2. Use DL concepts:
o Ravi: Individual
o Teacher: Concept
o Artificial Intelligence: Concept
o teaches: Role
3. DL representation:
Teacher ⊑ Person

17
Ravi : Teacher
teaches(Ravi, ArtificialIntelligence)
The sentence is now machine-readable, stored in a knowledge base, and ready
to be used in AI applications.
Key Assumption
The core belief of this approach is:
“The grammar of a sentence reflects its meaning.”
So if we understand the structure, we can figure out the meaning. This makes
the approach reliable, especially for AI systems that rely on logic and
reasoning.
Summary

Concept Explanation

Syntax-Driven Understanding the meaning of a sentence using its

Semantic Analysis grammar

A structured knowledge base with concepts,

DL Ontology
relationships, and individuals

Analyze sentence structure → Map each part to DL

Process
concepts → Build logical meaning

“John is a doctor who specializes in cardiology” →

Example
becomes logical facts the machine can use

For question answering, reasoning, chatbots, and

Why useful?
more

Makes NLP more accurate and logical using grammar

Main Benefit
as a guide

Semantic attachments
What are Semantic Attachments?
Semantic Attachments are a way of labeling who does what to whom in a
sentence.
They tell us the meaning-based roles that words or phrases play in a sentence,
based on the action (verb) and the participants involved in that action.
In simple words:
When you read a sentence, you can usually answer:
• Who is doing the action?

18
 • What is the action?
• What is being affected by the action?
• How or with what is the action being done?
These answers help us understand the roles that different parts of the
sentence play.
These roles are called:
• Agent
• Patient
• Instrument
• Location
• Experiencer, etc.
Each of these is a semantic attachment or semantic role.
Key Components
Let’s break this down:

Term Meaning

Semantic A label that tells the role a word/phrase plays in a

Attachment sentence

Verb The action that connects all the roles

Arguments Words or phrases connected to the verb that play roles

Roles Agent, Patient, Instrument, etc.

Common Semantic Roles (Theta Roles)

Example (in "John ate the pizza

Role Description
with a fork")

The one who does the

Agent John (he is eating)
action

The thing that receives the

Patient Pizza (it is being eaten)
action

The object used to perform

Instrument Fork (used to eat)
the action

Location Where the action happens "in the kitchen" (if added)

19
 Example (in "John ate the pizza
Role Description
with a fork")

One who feels or perceives "Mary felt happy" – Mary is

Experiencer
something experiencer

In "He moved the chair", chair is

Theme The entity moved or affected
the theme

Example Sentence Breakdown

Let’s analyze the sentence:
"John ate the pizza with a fork."
1. Verb = “ate” → This is the main action.
2. John → Agent (the one doing the eating)
3. Pizza → Patient (the thing being eaten)
4. Fork → Instrument (used to eat)
Semantic Attachment (Role Assignment):

Phrase Role

John Agent

ate Action

pizza Patient

with a fork Instrument

So, each phrase in the sentence is attached to the verb “ate” with a specific
semantic role.
Why Is It Important?
In Natural Language Processing (NLP):
Computers need to understand:
• Who is doing what?
• What objects are involved?
• What tools are being used?
• What is being affected?
By labeling roles with semantic attachments, NLP systems can:
• Understand sentence meaning
• Answer questions
20
 • Translate accurately
• Extract information from text
Real-Life Example for Students
Sentence: “Ravi gave the book to Priya.”

Word/Phrase Role

Ravi Agent (who gave)

the book Theme (what was given)

to Priya Recipient (who received)

So, we attach these roles to each word or phrase:

• Agent(Ravi)
• Theme(book)
• Recipient(Priya)
How This Helps in AI or Chatbots
Imagine a chatbot reads:
"I booked a table at the restaurant using my phone."
It can extract:
• Action: booked
• Agent: I
• Theme: table
• Location: restaurant
• Instrument: phone
Now the chatbot knows:
• You made a reservation
• For a table
• At a restaurant
• Using a phone
This makes it smart enough to reply like:
"Great! I’ve added your restaurant booking to your calendar."
Summary

21
Concept Explanation

Semantic
Labels that show the role of each part of a sentence
Attachments

Also called Semantic Roles or Theta Roles

Understanding the meaning of actions and

Used for
participants

Examples of roles Agent, Patient, Instrument, Theme, Location

NLP, machine translation, chatbots, question

Used in
answering

Helps machines understand the sentence like a

Why useful?
human would

Quick Practice Activity

Take this sentence:
"Meera opened the door with a key."
Can you identify:
• Agent = ?
• Patient = ?
• Instrument = ?
Answer:
• Agent = Meera
• Patient = Door
• Instrument = Key
Word Senses
What is a Word Sense?
A word sense is just a particular meaning of a word, depending on the
context (the words around it).

🔸 Example: The word “bank”

It can mean:
• A place where we keep money (like SBI or HDFC)
• Or the side of a river (like a riverbank)
So, the word “bank” has multiple meanings (or senses), and the correct
meaning is chosen based on the sentence.

22
What Are Semantic Attachments?
Semantic Attachments are a way to attach meaning to each word in a
sentence by:
• Looking at its role (like Agent, Patient, etc.)
• And also, by identifying its correct sense in context.
They help NLP systems (like chatbots, translators, search engines)
understand:
• What the word really means
• What concept or entity it refers to
Example in Detail
Let’s take the word “bank”.
Sentence 1: "I deposited money in the bank."
• Here, “bank” means a financial institution.
• The word is linked to concepts like:
o money
o savings
o loans
o accounts
So, the semantic attachment is:
• Word: bank
• Sense: financial institution
• Attachment: links to concepts like money, loan, ATM, etc.
Sentence 2: "The kids played near the bank of the river."
• Here, “bank” means the side of a river.
• It’s linked to:
o water
o shore
o trees
o sand
Semantic attachment becomes:
• Word: bank

23
 • Sense: river bank
• Attachment: links to river, shore, nature, etc.
Why Do We Need Semantic Attachments?
In Natural Language Processing (NLP), understanding the correct sense of a
word is very important.
Otherwise, a computer might:
• Think you're talking about a money bank when you actually mean
riverbank!
• Translate or respond incorrectly.
By using semantic attachments, the machine:
• Picks the correct meaning from possible meanings
• Connects it to the right concepts
This helps in:

Task How Semantic Attachments Help

Search engines Understand what you are really looking for

Machine translation Choose the correct word in the other language

Chatbots Reply with proper understanding

Sentiment analysis Know what emotion is tied to the context

Process Summary
When a sentence is given:
1. The system looks at the word (e.g., “bank”).
2. It checks the context (surrounding words like “money” or “river”).
3. It identifies the correct sense (financial institution or riverside).
4. It attaches this word to the correct concepts.
5. Now, the sentence can be understood correctly by the computer.
Real-Life Analogy for Students
Think of a dictionary.
A word like “light” has many meanings:
• Not heavy
• Bright

24
When you use the word in a sentence, your brain picks the right meaning by
looking at the full sentence.
Computers are not as smart as human brains. So, semantic attachments help
them do the same thing — understand the meaning based on context.
Practice Example
Sentence: “She saw the bat in the cave.”
• Word: bat
• Possible senses:
o An animal (linked to flying, night, caves)
o A cricket bat (linked to sports, games)
The word “cave” gives a clue → So, “bat” here refers to the animal.
Semantic attachment:
• Word: bat
• Sense: animal
• Attachment: linked to cave, wings, flying
Summary

Term Simple Meaning

Word Sense Specific meaning of a word depending on the context

Semantic Linking a word to its correct meaning and related

Attachment concept

Helps machines understand human language more

Why Important?
accurately

Search, Translation, Sentiment Analysis, Chatbots,

Used In
etc.

Relations between Senses

What Are “Relations Between Senses”?
In language, words can have different meanings (senses), and these senses
are often connected to each other.
These connections or relationships between meanings are called semantic
relations.

👉 Example:
The word “car” is related to:

25
 • “vehicle” (more general word)
• “automobile” (same meaning)
• “bus” or “bike” (similar category)
• “wheel” or “engine” (parts of a car)
These kinds of semantic relations help machines understand how words are
connected in meaning.
Why Is This Important in NLP?
When we speak or write:
• We use many words that mean similar things or are related in some
way.
• Computers and NLP systems need to understand these relations to
work properly in tasks like:
o Word sense disambiguation (choosing correct meaning)
o Information retrieval (finding related documents)
o Machine translation (using correct words in another language)
Types of Semantic Relations
Let’s understand the main types of relations between word senses, using very
simple examples.

🔸 1. Hyponymy / Hypernymy
Relation: "Is-a" relationship
• A Hyponym is a more specific word.
• A Hypernym is a more general word.

Example Explanation

"Dog" is a hyponym of "Animal" Dog is a type of animal

"Rose" is a hyponym of "Flower" Rose is a type of flower

"Vehicle" is a hypernym of "Car", "Bike", "Bus" These are all types of vehicles

Used in ontology creation and text classification.

🔸 2. Synonymy
Relation: Same or very similar meaning

26
Example Explanation

"Car" and "Automobile" Both mean the same thing

"Begin" and "Start" Can be used interchangeably in many contexts

👉 Used in search engines: When you search “automobile,” it also finds “car.”

🔸 3. Antonymy
Relation: Opposite meanings

Example Explanation

"Hot" and "Cold" Completely opposite temperatures

"Big" and "Small" Opposite sizes

"Win" and "Lose" Opposite outcomes

👉 Used in sentiment analysis, text polarity detection (positive/negative

meaning).

🔸 4. Meronymy / Holonymy
Relation: Part-to-whole relationship
• A Meronym is a part of something
• A Holonym is the whole

Example Explanation

"Wheel" is a meronym of "Car" A car has a wheel

"Finger" is a meronym of "Hand" Finger is part of a hand

"Tree" is a holonym of "Leaf" A tree has leaves

👉 Used in question answering systems, image labeling, etc.

🔸 5. Troponymy
Relation: Specific way of doing an action
• It’s used for verbs.
• A troponym is a specific kind of action.

Example Explanation

"Walk" is a troponym of "Move" Walking is one way to move

27
Example Explanation

"Run" is a troponym of "Walk" Running is a faster way of walking

"Whisper" is a troponym of "Speak" Whispering is a quiet way of speaking

👉 Used in action recognition, event detection, and dialog systems.

Putting It All Together — A Semantic Network
Think of a mind map (like a spider web), where:
• Words are nodes (points)
• Their meanings (senses) are linked by relationships

🔗 These connections help machines:

• Understand the context
• Pick the right meaning
• Suggest related terms
This is the foundation of WordNet, a popular lexical database used in many
NLP tools.
Real-Life Example
Let’s take the word “light”:
• It can mean not heavy (antonym of “heavy”)
• Or brightness (synonym of “illumination”)
• Or it can be part of “traffic light” (meronym of traffic signal system)
In a sentence:
“The bag is light, and I saw a bright light.”
The machine will:
• Use semantic relations to understand two different senses of “light”
• One refers to weight, another to brightness
Summary Table

Semantic Relation Meaning Example

Hyponymy Specific → General Dog → Animal

Hypernymy General → Specific Flower → Rose

Synonymy Similar meaning Car ↔ Automobile

28
Semantic Relation Meaning Example

Antonymy Opposite meaning Hot ↔ Cold

Meronymy Part of whole Wheel → Car

Holonymy Whole of parts Tree → Leaf

Troponymy Specific way of action Whisper → Speak

Applications in NLP

NLP Task How Semantic Relations Help

Choose correct word meaning based on

Word Sense Disambiguation
relations

Machine Translation Pick right translation depending on meaning

Retrieve related documents using synonyms or

Information Retrieval
hypernyms

Chatbots Improve understanding of user input

Ontology and Knowledge

Organize concepts for intelligent reasoning
Graphs

Thematic Roles
What are Thematic Roles?
When you write or say a sentence, you usually describe an action and the
people or things involved in that action.

👉 Example:
“John kicked the ball.”
• Someone (John) is doing the action.
• Something (the ball) is affected by the action.
These different participants in the sentence have different roles.
These roles are called thematic roles.
Why Are Thematic Roles Important?
In Natural Language Processing (NLP) and linguistics, machines need to:
• Understand who did what to whom
• Understand how entities are involved in an action or event
Thematic roles help us identify:

29
 • Who is doing the action?
• What is affected by the action?
• Where or how is the action happening?
Common Thematic Roles (with Simple Examples)
Let’s now look at the main types of thematic roles using simple sentences:

🔸 1. Agent
The person or thing that does the action.

Example Sentence Agent

“John kicked the ball.” John is doing the action (kicking)

“The dog barked.” The dog is the one doing the action

💡 Tip: The agent is usually active, voluntary, and has control.

🔸 2. Patient
The person or thing that is affected by the action.

Example Sentence Patient

“John kicked the ball.” The ball is affected by the action

“She broke the vase.” The vase got broken (was affected)

💡 Tip: The patient receives the action or undergoes a change.

🔸 3. Theme
The thing that is moved, given, or talked about.

Example Sentence Theme

“John gave the book to Mary.” The book is what is being given

“He sent a letter.” A letter is the theme (what is sent)

💡 Tip: Sometimes similar to patient, but more neutral — just what the
sentence is about.

🔸 4. Experiencer
The person or thing that feels or experiences something (emotions,
perceptions, etc.)

30
Example Sentence Experiencer

“John loves Mary.” John is feeling love

“Sarah heard the music.” Sarah is experiencing hearing

💡 Tip: The experiencer doesn’t do the action, but feels it.

🔸 5. Instrument
The thing that is used to perform the action.

Example Sentence Instrument

“John cut the bread with a knife.” A knife is used to cut

“He painted with a brush.” A brush is used to paint

💡 Tip: Tools, devices, or any means used to do something.

🔸 6. Location
The place where the action happens.

Example Sentence Location

“John is at the store.” The store is the place

“They danced in the hall.” The hall is the location

💡 Tip: It tells where the action occurs.

🔸 7. Source
The place or person from which something comes or starts.

Example Sentence Source

“John came from New York.” New York is the source

“He got the gift from his friend.” His friend is the source

💡 Tip: Source = starting point or origin.

🔸 8. Goal
The place or person towards which something moves or is directed.

Example Sentence Goal

“John went to the store.” The store is the goal

“He gave the gift to his friend.” His friend is the goal

31
💡 Tip: Goal = destination or endpoint.
Let’s See It All Together
Sentence:
“John gave the book to Mary at the library with a pen.”

Phrase Thematic Role

John Agent

the book Theme

Mary Goal

at the library Location

with a pen Instrument

Summary Table

Role Description Example

Agent Does the action John kicked the ball

Patient Affected by the action The ball was kicked

Theme What is moved or talked about He gave the book

Experiencer Feels or experiences something Sarah heard the music

Instrument Tool used to do something Cut with a knife

Location Where the action happens Danced in the hall

Source Where something comes from Came from New York

Goal Where something is going Gave it to Mary

Why Do Thematic Roles Matter in NLP?

NLP Task How Thematic Roles Help

Information
Identifying who did what, to whom, where
Extraction

Question Answering Finding correct answers based on roles

Using correct word order and roles in other

Machine Translation
languages

Semantic Parsing Understanding sentence meaning

32
NLP Task How Thematic Roles Help

Dialogue Systems Responding appropriately to user input

Selectional restrictions
What Are Selectional Restrictions?
Selectional restrictions (also known as semantic constraints) are rules about
which types of words can go together, especially when you're using verbs
(actions) and their arguments (subjects, objects, etc.).

👉 They make sure that the meaning of a sentence makes sense, not just its
grammar.
Simple Definition:
Selectional restrictions are rules that control which kinds of words can be
used with a verb, based on their meaning.
Example:

🔸 Sentence: “She ate an apple.”

This sentence is correct because:
• “ate” is a verb that needs something edible as an object.
• “apple” is edible → So, it fits the selectional restriction.

🔸 Sentence: “She ate a car.”

This sounds strange because:
• “car” is not edible, and the verb “eat” expects something that is.
• This violates the selectional restriction of the verb “eat”.
Why Are Selectional Restrictions Important?
1. They help us understand the correct meaning of a sentence.
2. They prevent nonsense or illogical combinations of words.
3. They help computers and AI systems avoid errors in translation, text
generation, speech synthesis, etc.
Types of Selectional Restrictions
Let’s understand the different types, each with a simple explanation and
example.

🔸 1. Syntactic Restrictions
Based on grammar and the structure of the sentence.
Example:

33
 • Transitive verb: needs two arguments (subject + object)
→ “She kicked the ball.”
• Intransitive verb: needs only one argument (subject)
→ “He sleeps.”
Incorrect use:
• “She kicked.” → Incomplete (missing object for a transitive verb)
• “He sleeps the bed.” → Wrong structure (intransitive verb used with
object)

🔸 2. Semantic Restrictions
Based on the meaning of the words involved.
Example:
• “The baby drank milk.” → Makes sense: milk is drinkable.
Example:
• “The baby drank rocks.” → Doesn’t make sense: rocks are not
drinkable.

🔸 3. Selectional Preferences
Some verbs prefer certain kinds of subjects or objects more than others.
Example:
• “The teacher gave the student a book.”
o “Give” likes three arguments: giver (agent), receiver (goal), and
thing given (theme).
Example:
• “The chair gave the student a book.” → Strange: a chair cannot give.

🔸 4. Prototypicality Effects
Some verbs prefer objects with typical features.
Example:
• “She ate rice.” – Makes sense.
• “He ate metal.” – Sounds weird.
Even though grammatically correct, semantically it feels odd, because “metal”
is not a typical food item.

🔸 5. Frame Semantics

34
Think of verbs as part of an event frame — a situation that includes
participants and roles.
Example (for “buy”):
• The buyer, the seller, the item, and the money are all part of the buying
frame.
→ “John bought a car from the dealer for $10,000.”
This helps NLP systems understand complete events, not just word meanings.
Use in NLP Applications

NLP Task How Selectional Restrictions Help

Machine Ensures the correct word is chosen in another

Translation language

Helps intonation and phrasing by detecting odd

Text-to-Speech
combinations

Information Improves search accuracy by understanding verb-

Retrieval object pairs

Error Detection Flags unusual or nonsensical combinations

What Happens If We Violate Selectional Restrictions?

Violating them creates semantic errors — the sentence may be grammatically
correct but nonsensical.

Is It
Sentence Why?
Valid?

“The man opened the door.” Yes Door can be opened.

“The man opened the cloud.” No Clouds cannot be opened.

“The doctor prescribed

Yes Makes sense.
medicine.”

“The medicine prescribed the Wrong roles; medicine can't

No
doctor.” prescribe.

Summary

Key Point Explanation

Selectional Semantic rules that decide which types of words can

Restrictions go together

Main Use Make sentences meaningful and logical

35
Key Point Explanation

Syntactic, Semantic, Preferences, Prototypicality,

Types
Frame Semantics

NLP tasks like translation, information retrieval, and

Helps In
text generation

Violations Help identify errors or nonsense in text

✏️ Final Analogy:
Think of a verb like a machine that accepts only certain inputs:
• The verb “eat” expects something like food.
• If you give it a stone, the machine doesn’t work — it’s an error.
Word Sense Disambiguation
What is Word Sense Disambiguation?
WSD stands for Word Sense Disambiguation.
It means:
Finding the correct meaning (sense) of a word when it has more than one
meaning, based on the context in which it is used.
Why is this needed?
Because many English words have multiple meanings, and we need to figure
out the correct one depending on the sentence.
Example:
Take the word "bank":
• In “He went to the bank to deposit money,” → bank = financial
institution
• In “She sat by the bank of the river,” → bank = riverbank
Both sentences are correct, but the word "bank" means different things.

👉 WSD helps us figure out which "bank" the sentence is talking about.
Why is WSD Important?
1. Ambiguity Resolution
Natural language is full of ambiguity. Words can mean many things.
→ WSD helps us remove confusion and choose the correct meaning.
2. Improves Accuracy
In tasks like translation or text analysis, if we get the meaning of a

36
 word wrong, the whole output can be wrong.
→ WSD ensures better accuracy.
3. Better Precision
Helps in reducing false positives (wrong results) and false negatives
(missed results) when retrieving or analyzing information.
4. Helps in Language Understanding
Computers need to understand context just like humans do.
→ WSD brings machines closer to human-like understanding.
5. Useful in Information Retrieval
When you search something on Google, WSD helps ensure that the
search engine shows relevant results, not random ones.
Techniques Used in WSD
WSD can be done using different methods or algorithms:

🔹 1. Rule-Based Methods
• Use a set of hand-written rules to determine meaning.
• For example:
o If the word “bank” is near words like money, deposit, then it
likely means financial bank.
o If it is near river, water, shore, then it means riverbank.

✅ Simple to create
❌ Can’t handle all cases well

🔹 2. Knowledge-Based Methods
• Use dictionaries, thesauruses (like WordNet), and definitions to
compare the context with word meanings.
Example: Using a dictionary, compare the surrounding words in the sentence
with each meaning of the word to find the best match.

🔹 3. Supervised Machine Learning

• Use labeled training data where the correct senses of words are
already marked.
• Train a model to learn patterns and predict the correct sense in new
sentences.

✅ Accurate when data is available

❌ Needs lots of labeled data

🔹 4. Unsupervised Machine Learning

37
 • Use unlabeled data, try to group word usages into clusters (each
cluster representing a sense).

✅ Doesn’t need labeled data

❌ Less accurate and harder to evaluate
Applications of WSD

Area Use of WSD

Helps choose correct translation (e.g., “bat” → 🦇

Machine Translation
or 🏏?)

Search engines give better results by

Information Retrieval
understanding the meaning of queries

Ensures correct pronunciation and intonation

Text-to-Speech
(some meanings change the tone)

Chatbots and Virtual

Helps understand user intent better
Assistants

Text Summarization Improves the meaning of the summary

Real-Life Analogy
Imagine you’re reading the word "crane".
• In a bird-watching blog, you guess it's a bird.
• In a construction site manual, you guess it's a machine.
• In a yoga guide, it could be a yoga pose.

👉 You understood the correct meaning each time using context — that’s
what WSD does!
Summary Table

Concept Explanation

WSD Identifying the correct meaning of a word with multiple senses

Why Words can be ambiguous; WSD removes confusion

How Rule-based, knowledge-based, supervised, unsupervised

Used In Translation, search engines, chatbots, summarization

Ambiguous context, lack of labeled data, subtle differences in

Challenges
meaning

🧪 Want to Test It?

38
Which “bat” is this?

• “The bat flew out of the cave.” → 🦇 Animal

• “He hit the ball with a bat.” → 🏏 Sports equipment

👉 That’s WSD in action!

WSD using Supervised, Dictionary & Thesaurus
Word Sense Disambiguation (WSD):
WSD means choosing the correct meaning (sense) of a word based on the
sentence or context in which it appears.
Different methods or strategies can be used to do this. Let's understand three
important approaches:

🔹 1. Supervised Approach
What is it?
• Just like teaching a child with examples, this approach trains a
machine using labeled data (sentences where the meanings of words
are already known).
• The model learns from those examples and then applies that learning
to predict the meaning of the same word in new sentences.
How it works:
• You prepare a training dataset where each sentence has a word whose
correct meaning is labeled.
• Then, you use a machine learning algorithm to train a model.
• Once trained, the model can read a new sentence and guess the correct
meaning.
Example:
Sentence:
“I went to the bank to deposit my paycheck.”
• The word bank could mean:
o A financial institution, or
o The side of a river
→ A supervised model uses examples like this, along with clue words like
“deposit,” “paycheck,” “money,” and learns that in this context, bank =
financial institution.

39
Common Algorithm:
Naive Bayes Classifier (a simple and fast machine learning algorithm used in
NLP)
Summary:

Feature Description

Data Required Large set of labeled examples

Accuracy High if trained well

Disadvantage Needs manual effort to label data

🔹 2. Dictionary-Based Approach
What is it?
• This approach uses dictionary definitions to understand the possible
meanings of a word.
• It compares those meanings with the words around the target word in
the sentence (context) and selects the best match.
Example:
Sentence:
“I love to play the bass guitar.”
• The word bass can mean:
o A type of fish
o A musical instrument with low pitch
→ The dictionary lists both meanings. But based on nearby words like "play"
and "guitar", we match it to the musical instrument sense.
Tools used:
Dictionaries like WordNet, Oxford, Merriam-Webster, etc.
Summary:

Feature Description

Data Used Dictionary entries and definitions

Strength No need for labeled training data

Disadvantage May struggle with very short or vague contexts

40
🔹 3. Thesaurus-Based Approach
What is it?
• This method uses a thesaurus or semantic network to find related or
similar words.
• Then it checks which group of related words best fits the sentence.
Example:
Sentence:
“The company's revenue has been steadily increasing.”
• The word increase could mean:
o A rise in profits
o A rise in size
o Or just a general growth
→ A thesaurus will group “increase” with growth, improvement, gain, etc.
→ Based on surrounding words like “revenue” and “steadily,” it guesses the
correct meaning: financial growth.
Tools used:
• Thesaurus
• WordNet
• Semantic networks (like ConceptNet)
Summary:

Feature Description

Data Used Words grouped by meaning or similarity

Strength Useful for capturing subtle relationships

Disadvantage Depends on the quality of the thesaurus

Quick Comparison Table

Needs
Approach Based On Pros Cons
Training?

Machine
High accuracy if Needs a lot of
Supervised learning with ✅ Yes
trained labeled data
labeled data

41
 Needs
Approach Based On Pros Cons
Training?

May not
Dictionary- Definitions of Easy to use,
❌ No handle vague
Based words human-readable
contexts well

Understands
Thesaurus- Groups of Might be less
❌ No word
Based similar words precise
relationships

Real-Life Analogy
Let’s say you’re learning the meaning of “bark”.
• Supervised: Your teacher gives you examples of “bark” as dog sound
and tree covering, so you learn when to use which.
• Dictionary-Based: You check a dictionary, read meanings and pick the
right one based on sentence.
• Thesaurus-Based: You group “bark” with similar words like “woof,”
“howl” or “tree,” and guess which group fits best.
Final Thoughts
Word Sense Disambiguation is key to understanding language correctly.
These approaches help computers and AI systems understand what we really
mean when we use ambiguous words.
Bootstrapping methods
What is Bootstrapping?
The word bootstrapping means starting with a little and growing gradually by
using what you already have.
In NLP (Natural Language Processing), bootstrapping refers to building a
system using a small amount of labeled data, and then automatically
improving it by learning from more unlabeled data.
Why is it useful?
Because labeling large datasets is expensive and time-consuming.
Bootstrapping helps create smarter models with less manual work.
Bootstrapping Methods
There are four main bootstrapping methods used in NLP:

1️⃣ Self-Training
What is it?

42
Self-training is like a student learning a few chapters first, then trying out
questions from more chapters using their current knowledge, and learning
from the new answers.
How it works:
1. You train a model using a small labeled dataset.
2. The model makes predictions on unlabeled data.
3. The confident predictions are added to the training set.
4. The model is retrained, and the cycle repeats.

✅ Example:
You want to train a model to do Part-of-Speech (POS) tagging.
• Start with 100 sentences where each word is labeled with its POS tag
(like noun, verb).
• Train the model.
• Now give it 1,000 unlabeled sentences.
• The model tags them.
• Take the best predictions (ones it is most confident about) and add them
to the training data.
• Train the model again — now it’s better!

2️⃣ Co-Training
What is it?
Co-training is like two friends learning from different books. Each teaches the
other what they know best.
How it works:
1. Train two (or more) models using different features (or perspectives) of
the same data.
2. Each model makes predictions on unlabeled data.
3. They share the predictions with each other.
4. They add new labeled data to their sets and retrain.

✅ Example:
You are doing sentiment analysis.
• Model A is trained using tweets.
• Model B is trained using news articles.

43
 • Both are trained with a small amount of labeled data.
• Now you feed them lots of unlabeled social media posts.
• Model A labels some; Model B labels others.
• They exchange their best predictions, retrain, and get better.

3️⃣ Active Learning

What is it?
Active learning is like a student saying:
👉 “I don’t understand this part, can you explain it to me?”
The model asks humans for help when it is most uncertain.
How it works:
1. Train a model on a small labeled dataset.
2. Use the model to select difficult or confusing examples from unlabeled
data.
3. A human annotator labels those examples.
4. Add them to the training set and retrain the model.
Example:
You want to detect Named Entities (like names of people, places, etc.).
• Train a model on 100 labeled sentences.
• From 1,000 unlabeled sentences, the model identifies 50 that are most
confusing (e.g., “Is ‘Amazon’ a river or a company?”).
• A human labels them correctly.
• Add them to the dataset and retrain — now the model is more accurate.

4️⃣ Semi-Supervised Learning

What is it?
Semi-supervised learning is like using both textbooks (labeled data) and extra
reading materials (unlabeled data) to learn better.
How it works:
• You train the model using both:
o A small labeled dataset
o A large unlabeled dataset
• The model learns patterns from the unlabeled data and applies it to
improve the predictions on the labeled part.

44
Example:
You are building a Machine Translation model (say, English to French).
• You have 1,000 sentence pairs (English ↔ French) that are labeled.
• You also have 10,000 sentences in English and French separately (not
aligned).
• The model learns structure and grammar from the unlabeled data.
• This extra knowledge helps translate more accurately.
Summary Table

Human
Method Idea Data Needed Example
Help

Model teaches itself Small labeled

Self-
using confident + large ❌ POS tagging
training
predictions unlabeled

Two models teach Small labeled

Sentiment
Co-training each other using + large ❌
analysis
different views unlabeled

Small labeled
Active Model asks human to Named Entity
+ large ✅
learning label confusing data Recognition
unlabeled

Model learns from

Small labeled
Semi- both labeled and Machine
+ large ❌
supervised unlabeled data Translation
unlabeled
together

Real-Life Analogy

Bootstrapping
Real-Life Analogy
Method

A student solves more problems on their own after

Self-training
studying a little.

Two friends with different strengths help each other

Co-training
understand better.

A student asks the teacher for help with only the tough
Active learning
questions.

A student uses both class notes and reference books to

Semi-supervised
study.

45
Final Thoughts
Bootstrapping is a smart way to train models with less effort and less labeled
data. It helps in building efficient, scalable NLP applications even when
human-labeled data is limited.
Word Similarity using Thesaurus and distributional methods:
Word Similarity: What is it?
Word similarity is the task of finding how similar two words are in terms of
meaning.
For example:
• "Car" and "automobile" are very similar.
• "Car" and "banana" are not similar.
Why is Word Similarity Important in NLP?
Word similarity is used in many applications like:
• Search engines – to show results even if the words are not exactly the
same.
• Chatbots – to understand user inputs better.
• Machine translation – to choose correct words in the target language.
• Spelling correction – to suggest similar words when a mistake is made.

🔹 1. Thesaurus-Based Method
What is a Thesaurus?
A thesaurus is like a dictionary that shows:
• Synonyms (words with similar meanings),
• Antonyms (opposite meanings), and
• Word relationships (like part-whole or type-kind).
WordNet – A Popular Thesaurus
• WordNet is a digital thesaurus.
• It organizes words into synsets (sets of synonyms that express the same
concept).
How Thesaurus-Based Methods Work:
1. Words are grouped into synsets.
2. Synsets are arranged in a graph (like a tree or network).
3. The distance between two words in this graph is measured.

46
 o Shorter distance → more similar meaning.
o Longer distance → less similar.
Example:
• "Dog" and "Wolf" might belong to similar synsets.
• In WordNet, you might find only 1 or 2 steps between them → They are
highly similar.
• "Dog" and "Pen" might be far apart in WordNet → Not similar.

🔹 2. Distributional-Based Method
Key Idea:
Words that appear in similar contexts have similar meanings.
For example:
• "Teacher" and "professor" often appear with words like “class”,
“students”, “lecture”.
• So, they are likely to be similar.
How Distributional Methods Work:
1. A huge text dataset (corpus) is used.
2. Each word is represented as a vector (a list of numbers) based on how
it appears with other words.
3. We then compare the vectors of two words using mathematical
formulas.
Common Similarity Measures:
• Cosine similarity – measures angle between vectors (0° means they are
identical).
• Euclidean distance – measures straight-line distance.
• Mahalanobis distance – advanced distance taking into account the
spread of data.
Popular Distributional Tools:
• Word2Vec
• GloVe
• BERT
These are all machine learning models that create word embeddings
(numerical vector representations of words).
Example:

47
 • "King" and "Queen" will have vectors close to each other in space.
• "King" and "Apple" will have vectors far apart → they are not similar.
Comparison of Thesaurus-Based and Distributional Methods

Feature Thesaurus-Based Distributional-Based

Source of Pre-defined thesaurus

Large text corpora
knowledge (e.g., WordNet)

Based on meaning &

Type of similarity Based on context usage
definitions

High for nuanced/context-

Precision High for known words
based similarity

Limited to words in Flexible, can learn new word

Flexibility
thesaurus meanings

Computational High (needs training or

Low to medium
cost embeddings)

Dog ↔ Canine (short Teacher ↔ Professor (similar

Example
path in WordNet) usage context)

Summary (in Simple Words):

• Thesaurus-based methods use human-created lists of similar words
(like WordNet) to measure similarity.
• Distributional-based methods look at how words are used in real texts
to find which words are used in similar ways.
• Both methods help computers understand meaning, but they work
differently and are useful in different situations.

48