[Link] Natural Language Processing (NLP).

Compare the
rationalist and empiricist approaches to modeling human
language understanding.
Natural Language Processing (NLP)
1. Natural Language Processing (NLP) is a branch of computer
science and artificial intelligence concerned with processing
human language.
2. NLP develops computational models for understanding and
processing natural languages such as English, Hindi, etc.
3. The main objectives of NLP are:
o To develop automated tools for language processing.
o To gain better understanding of human communication.
4. NLP requires knowledge of how humans acquire, store, and
process language.
5. Applications of NLP include machine translation, speech
recognition, information retrieval, question answering, and text
summarization.

Rationalist and Empiricist Approaches

1. Rationalist Approach
• The rationalist approach is an early approach to NLP.
• It assumes the existence of a special language faculty in the
human brain.
• According to this approach, humans cannot learn complex
language only from limited sensory input.
• It emphasizes innate linguistic knowledge and predefined
grammatical structures.
2. Empiricist Approach
 • The empiricist approach does not believe in an inbuilt
language faculty.
• It believes language learning happens through experience and
sensory input.
• This approach uses principles such as pattern recognition,
association, and generalization.
• Detailed language structures are learned from data and
observations.

Comparison Between Rationalist and Empiricist Approaches

Rationalist Approach Empiricist Approach
Believes in innate language Does not believe in innate language
faculty faculty
Language knowledge is inborn Language is learned from experience
Focuses on data and learning
Focuses on grammar and rules
patterns
Uses predefined linguistic Uses sensory inputs and
structures observations
Supports data-driven and statistical
Supports rule-based models
models

• Both approaches have greatly influenced the

development of modern NLP systems and
computational language models.
2. Explain the five levels of language processing (lexical,
syntactic, semantic, discourse, and pragmatic) with suitable
examples.
Five Levels of Language Processing in NLP
1. Lexical Analysis
 • Lexical analysis deals with the analysis of words in a language.
• It focuses on word-level processing and requires morphological
knowledge about the structure and formation of words from
morphemes.
• The rules for forming words differ from one language to another.
• Example:
o “Unhappiness” can be divided into:
▪ “un” (prefix)
▪ “happy” (root word)
▪ “ness” (suffix)
2. Syntactic Analysis
• Syntactic analysis studies the structure of sentences and the
relationships among words.
• It checks whether a sentence follows grammatical rules such as
word order and agreement.
• This level requires syntactic knowledge about how words
combine to form phrases and sentences.
• Examples:
o “I went to the market” → grammatically correct.
o “Went the I market to” → grammatically incorrect.
3. Semantic Analysis
• Semantic analysis is concerned with understanding the meaning
of language.
• It creates meaningful representations of linguistic inputs and
identifies the correct meaning of words in context.
• A sentence may be syntactically correct but semantically
meaningless.
 • Example:
• “Colorless green ideas sleep furiously.”
• The sentence is grammatically correct but meaningless
semantically.

4. Discourse Analysis
• Discourse analysis studies the structure and meaning of larger
text units such as paragraphs and documents.
• It helps in resolving references between sentences and
understanding the flow of information.
• It requires identification of anaphoric references and discourse
structure.
• Example:
• “The district administration refused permission because they
feared violence.”
• Here, “they” refers to the district administration.
5. Pragmatic Analysis
• Pragmatic analysis is the highest level of language processing.
• It deals with the purposeful use of language in real-life
situations and depends on context and world knowledge.
• It helps understand speaker intention and implied meaning
beyond literal words.
 • Example:
• “Can you open the door?”
• Pragmatically, it is a request, not a question about ability.

3. Discuss the major challenges in NLP, such as ambiguity,

idioms, evolving language, and ellipses, and explain how
context helps in resolving these issues.
Major Challenges in Natural Language Processing (NLP)
1. Ambiguity in Natural Language
• Natural languages are highly ambiguous, making it difficult for
computers to identify the exact meaning of words and
sentences.
• Ambiguity can occur at different levels such as word ambiguity
and structural ambiguity.
• Word ambiguity occurs when a word has multiple meanings.
o Example:
▪ “Bank” may refer to a financial institution or the side
of a river.
▪ “Taj” may refer to a monument, tea brand, or hotel.
• Structural ambiguity occurs when a sentence can have more
than one interpretation.
o Example:
 ▪ “Stolen rifle found by tree.”
• NLP systems use context, probabilistic parsing, and word sense
disambiguation techniques to resolve ambiguity.

2. Idioms and Metaphors

• Idioms and metaphors make NLP difficult because their
meanings cannot be understood literally.
• Example of idiom:
o “Kicked the bucket” means “to die,” not literally kicking a
bucket.
• Example of metaphor:
o “Time is a thief” suggests that time takes away valuable
moments.
• Context helps NLP systems understand the intended figurative
meaning instead of the literal meaning.

3. Evolving Nature of Language

• Language constantly changes with the addition of new words
and new meanings for existing words.
• Words may acquire different meanings depending on social and
cultural changes.
• Example:
o “9/11” now commonly refers to the terrorist attack on the
World Trade Center.
• NLP systems must continuously learn from updated data and
contexts to understand modern usage.

4. Ellipses
 • Ellipses refer to omission of words or phrases that are
understood from context.
• Example:
• “I’m going to the store, and you’re going to the party, right?”
• “Yes, I am…”
• The omitted words must be inferred correctly for proper
understanding.

Role of Context in Resolving NLP Challenges

• Context plays an important role in identifying the correct
meaning of words and sentences.
• The context of a word is determined by surrounding or co-
occurring words.
• Context helps in:
• Resolving ambiguity
• Understanding idioms and metaphors
• Identifying omitted words in ellipses
• Determining correct meaning of evolving words
• Frequency of word usage in a particular sense also helps
machines learn the intended meaning.
• Therefore, context-based learning is essential for improving the
accuracy and efficiency of NLP systems.
4. What is the difference between language and grammar?
How does Chomsky’s transformational grammar help in
parsing natural language?
Difference Between Language and Grammar
Language
• Language is the primary medium of communication used
by humans to express ideas, emotions, and knowledge.
• It is a system of sounds, words, and sentences that convey
meaning to listeners or readers.
• Language is considered the outer form through which
content and knowledge are expressed.
• Different languages can express the same content in
different forms.
• Natural languages continuously evolve with new words
and changing meanings.

Grammar
• Grammar is a set of rules that defines the structure of a
language.
• It helps in parsing and generating meaningful and
grammatically correct sentences.
• Grammar specifies how words combine to form phrases
and sentences.
• Different grammar frameworks include transformational
grammar, lexical functional grammar, and dependency
grammar.
• Grammar acts as the foundation for Natural Language
Processing systems.
Chomsky’s Transformational Grammar
• Noam Chomsky proposed transformational grammar in
Syntactic Structures (1957).
• He argued that phrase structure grammar alone is
insufficient for describing natural language.
• According to Chomsky, every sentence has:
o Deep structure
o Surface structure
• Deep structure represents the actual meaning of the
sentence, while surface structure represents the spoken
or written form.
• Different surface structures can share the same deep
meaning.
• Example:
• “Pooja plays veena.”
• “Veena is played by Pooja.”
fig:surface &deep structure

• How Transformational Grammar Helps in Parsing

Natural Language
• Transformational grammar helps NLP systems understand
relationships between different sentence forms.
• It converts one sentence structure into another using
transformation rules.
• Example:
• Active sentence → Passive sentence
• It consists of three main components:
o Phrase structure grammar
o Transformational rules
o Morphophonemic rules
• Phrase structure grammar generates the syntactic
structure of sentences.

• Transformational rules modify sentence structures while

preserving meaning.
• Example:
• “The police will catch the snatcher.”
• “The snatcher will be caught by the police.”
• Morphophonemic rules adjust word forms and
pronunciation during transformations.
• Example:
• “catch + en” becomes “caught.”
• Thus, transformational grammar improves natural
language parsing by identifying hidden relationships
between sentence structures and meanings.
[Link] is Karaka Theory? Illustrate at least four Karaka
roles with examples in an Indian language sentence.
Karaka Theory
• Karaka Theory is the central concept of the Paninian
Grammar (PG) framework.
• It explains the relationship between verbs and different
participants involved in an action.
• Karaka relations are assigned based on the role played by
participants in the main activity of a sentence.
• Paninian Grammar is mainly used for Indian languages,
especially languages with Subject-Object-Verb (SOV)
structure.
• Important Karaka roles include:
o Karta (Subject/Doer)
o Karma (Object)
o Karana (Instrument)
o Sampradana (Beneficiary)
o Apadan (Separation)
o Adhikaran (Location)

Example Sentence in Hindi

“म ाँ बच्ची को आं गन में ह थ से रोटी खिल ती है ।”
(Maan bachchi ko aangan mein haath se rotii khilaatii hai.)
Meaning: “The mother feeds bread to the child by hand in
the courtyard.”

Four Karaka Roles with Examples

1. Karta (Doer/Subject)
• Karta is the performer of the action in a sentence.
• In the example sentence, “म ाँ (Maan)” meaning mother is
the Karta because she performs the action of feeding.

2. Karma (Object)
• Karma is the object that receives the effect of the action.
• In the sentence, “रोटी (Roti)” meaning bread is the Karma
because it is the thing being fed.

3. Karana (Instrument)
• Karana refers to the instrument or means used to perform
the action.
• In the sentence, “ह थ से (haath se)” meaning by hand is
the Karana because the hand is used as the instrument
for feeding.

4. Sampradana (Beneficiary/Recipient)
• Sampradana indicates the beneficiary or receiver of the
action.
• In the sentence, “बच्ची को (bachchi ko)” meaning to the
child is the Sampradana because the child receives the
food.

5. Adhikaran (Location)
• Adhikaran represents the location or place where the
action occurs.
• In the sentence, “आं गन में (aangan mein)” meaning in the
courtyard is the Adhikaran because it specifies the place
of action.
 Conclusion
• Karaka Theory helps in understanding semantic and
syntactic relations in Indian languages.
• It plays an important role in Natural Language Processing
for analyzing sentence structure and meaning in Indian
language systems.
[Link] are the applications of NLP in real-world systems?
Briefly explain at least three: Machine Translation, Question
Answering, and Text Summarization.
Applications of NLP in Real-World Systems
• Natural Language Processing (NLP) is widely used for
processing, understanding, and generating human
language in computer systems.
• NLP applications help computers interact with humans in
a natural and meaningful way.
• Important applications of NLP include:
o Machine Translation
o Speech Recognition
o Information Retrieval
o Question Answering
o Text Summarization
o Information Extraction

1. Machine Translation
 • Machine Translation refers to the automatic translation of
text from one human language to another.
• It requires understanding of:
o Words and phrases
o Grammar of both languages
o Semantics and vocabulary knowledge
• NLP techniques analyze sentence structure and meaning
before generating translation in the target language.
• Examples:
o Google Translate
o Translation systems used in multilingual websites
and mobile apps.
• Machine Translation helps users communicate across
different languages efficiently.

2. Question Answering (QA)

• Question Answering systems attempt to provide precise
answers to user questions from a collection of documents
or databases.
• Unlike normal search engines, QA systems provide direct
answers instead of only listing documents.
• These systems require:
• Precise analysis of questions
• Semantic understanding
• Background knowledge processing
 • Examples:
• Siri
• Google Assistant
• AI chatbots and virtual assistants.
• QA systems are used in customer support, education,
healthcare, and search engines.

3. Text Summarization
• Text Summarization is the process of generating a short
and meaningful summary of a large document.
• It reduces lengthy information into concise text while
preserving important ideas.
• Text summarization involves:
• Syntactic processing
• Semantic processing
• Discourse-level analysis
• Examples:
• News summary generators
• Research paper summarizers
• Email and document summarization tools.
• It helps users quickly understand important information
without reading the entire document.

Conclusion
• NLP applications are widely used in modern real-world
systems for improving human-computer interaction.
• Machine Translation, Question Answering, and Text
Summarization are important NLP applications that help
users communicate, retrieve information, and process
large amounts of text efficiently.
[Link] Grammar-based and Statistical language
models. Discuss n-gram models and how sentence
probability is estimated using bigrams.
Comparison Between Grammar-Based and Statistical
Language Models
Grammar-Based Language
Statistical Language Model
Model
Uses probability and
Uses grammar rules of a
statistics to model
language to create models.
language.
Represents syntactic
Learns patterns from a large
structure using hand-coded
training corpus.
rules.
Depends on linguistic Depends on frequency and
knowledge and grammar probability of word
frameworks. occurrence.
Difficult to design because
Easier to adapt with large
rules must be manually
amounts of data.
written.
Focuses mainly on syntax Focuses on predicting word
and structure. sequences statistically.
Grammar-Based Language
Statistical Language Model
Model
Examples: Generative
Grammar, Paninian Example: n-gram model.
Framework, LFG.

n-gram Models
• An n-gram model is a statistical language model used to
predict the probability of a word based on the previous
words in a sequence.
• It simplifies language modelling by considering only the
previous (n-1) words instead of the entire sentence
history.
• Applications of n-gram models include:
o Speech recognition
o Spelling correction
o Machine translation
o Handwriting recognition
• In an n-gram model, the probability of a word depends on
previous words only.
• Types of n-gram models:
o Unigram: depends on no previous word
o Bigram: depends on one previous word
o Trigram: depends on two previous words
Bigram Model
• A bigram model predicts the probability of a word using
only the immediately previous word.
• The bigram probability is represented as:
P(w_i\mid w_{i-1})
• Sentence probability is calculated by multiplying
conditional probabilities of all word pairs.
• For a sentence:
“The Arabian knights are the fairy tales of the east”
the probability is estimated as:
• P(The/⟨s⟩)×P(Arabian/The)×P(knights/Arabian)×P(are/kni
ghts)×P(the/are)×P(fairy/the)×P(tales/fairy)×P(of/tales)×
P(the/of)×P(east/the)
• Using the probabilities from the training corpus:
[
0.67 \times 0.5 \times 1.0 \times 1.0 \times 0.5 \times 0.2
\times 1.0 \times 1.0 \times 1.0 \times 0.2
]
• Final sentence probability becomes:
P=0.0067

Limitations of n-gram Models

• n-gram models suffer from data sparsity because unseen
word combinations get zero probability.
 • They fail to capture long-distance dependencies in
sentences.
• Smoothing techniques such as Add-one smoothing and
Good-Turing smoothing are used to solve these
problems.
8. Describe and differentiate between Add-one smoothing
and Good-Turing smoothing in statistical language modeling.
Add-one Smoothing and Good-Turing Smoothing in
Statistical Language Modeling
Add-one Smoothing (Laplace Smoothing)
• Add-one smoothing is a technique used to handle zero
probabilities in n-gram models.
• In this method, one is added to every n-gram frequency
before converting counts into probabilities.
• It ensures that even unseen n-grams receive a non-zero
probability.
• The probability formula for Add-one smoothing is:
P(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V}
Where:
• (C(w_{i-1}, w_i)) = count of the bigram
• (C(w_{i-1})) = count of previous word
• (V) = vocabulary size
• Example:
o Corpus:
1. “I love programming”
 2. “I love coding”
o Bigram “I love” appears 2 times.
• Add-one smoothing adjusts the probability by adding one
to the frequency count.
• Advantage:
o Prevents zero probabilities for unseen word
combinations.
• Limitation:
o It gives equal probability to all unseen n-grams, even
when some are more likely than others.
Good-Turing Smoothing
• Good-Turing smoothing improves probability estimation
by adjusting frequencies based on the occurrence of rare
n-grams.
• It estimates probabilities for unseen n-grams using the
frequency distribution of observed n-grams.
• The adjusted count is calculated as:
f^*=\frac{(f+1)N_{f+1}}{N_f}
Where:
• (f) = original frequency
• (f^*) = adjusted frequency
• (N_f) = number of n-grams occurring (f) times
• (N_{f+1}) = number of n-grams occurring (f+1) times
• Example:
 • If 25,108 n-grams occur 4 times and 20,542 occur 5 times,
Good-Turing smoothing adjusts the frequency
accordingly.
• Advantage:
• Provides better estimation for unseen and low-frequency
n-grams.
• Limitation:
• More computationally complex than Add-one smoothing.

Difference Between Add-one and Good-Turing Smoothing

Add-one Smoothing Good-Turing Smoothing

Adds one to every n-gram Adjusts counts based on

count. frequency distribution.
Simple and easy to More complex and statistically
implement. accurate.
Gives equal probability to Estimates unseen probabilities
unseen n-grams. more realistically.
May distort probabilities for Preserves probability
frequent words. distribution better.
Less effective for large More effective for sparse data
corpora. problems.

Both smoothing techniques are used to solve the data sparsity

problem in statistical language models.