Natural Language Processing

Detailed Study Notes

Module-I, Unit-I - Foundations of Natural Language Processing

Department of CSE, School of Computing & Informatics

VFSTR Deemed to be University

Document date: 2026-07-01

Dept. of CSE, SoCI Module-I | Unit-I Page 1

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Introduction to Natural Language Processing

1.1 Preamble

Human beings communicate primarily through language. Every day we produce and consume
billions of words - in conversation, documents, messages, searches, and commands. For a
computer, however, language is not native. Text on a screen is merely a sequence of character
codes until an algorithm interprets it. Natural Language Processing (NLP) is the scientific and
engineering discipline devoted to closing that gap: enabling machines to process human
language in ways that are useful, accurate, and scalable.

This topic establishes the conceptual foundation for the entire unit. We define NLP precisely,
explain why it cannot be reduced to a single parent discipline, and introduce the two
complementary capacities every practical language system must possess - understanding
incoming language and generating outgoing language. We also preview the processing pipeline
that reappears throughout later topics, and we situate NLP within the modern data economy
where unstructured text dominates enterprise information.

• Additional solved examples for this topic: Problem 1.1, Problem 1.2. Full solutions appear
at the end of this unit.

User Input (text or speech)

|
[NLU] ---> intent, entities, slots
|
Backend / database / planner
|
[NLG] ---> fluent response text
Figure 0.1: NLU and NLG in a language technology system

1.2 Formal Definition and Scope

Natural Language Processing may be defined as follows:

• Definition: Natural Language Processing (NLP) is a branch of Artificial Intelligence concerned

with the automatic analysis, understanding, transformation, and generation of human language in
textual or spoken form.

The scope of NLP is broad. At the lowest level, systems segment continuous text into words and
sentences. At intermediate levels, they assign grammatical categories, build syntactic structure,
and extract entities and relations. At higher levels, they infer intent, sentiment, and discourse
meaning.

• Generation tasks reverse the direction: from structured data or internal representations to
fluent sentences, summaries, translations, or dialogue responses.

Dept. of CSE, SoCI Module-I | Unit-I Page 2

Module-I | Unit-I VFSTR Deemed to be University

NLP is not synonymous with "using ChatGPT." Large language models are one family of
techniques within a much older intellectual tradition that includes formal grammars, statistical
models, information retrieval, and speech technology. A well-trained student must understand
both the classical pipeline view and the modern end-to-end neural view, because industry
systems still combine them.

1.3 The Interdisciplinary Foundation

NLP is genuinely interdisciplinary. Three fields supply its intellectual core.

Linguistics contributes theories of how language is structured and interpreted. Syntax tells us how
words combine into phrases and sentences. Morphology explains how roots and affixes create
word forms. Semantics studies literal meaning; pragmatics studies meaning in context. Phonology
and phonetics matter when NLP interfaces with speech. Without linguistic theory, computational
systems become ad hoc collections of heuristics with no explanatory power.

Computer Science contributes algorithms and data structures. Efficient tokenization, indexing,
graph search, dynamic programming for parsing, and distributed processing for web-scale
corpora are all CS contributions.

• Complexity analysis matters: parsing algorithms, training neural networks, and serving
real-time dialogue systems all have computational cost profiles that engineers must manage.

Statistics and Machine Learning contribute methods for learning from data and quantifying
uncertainty. Language is variable; rules alone cannot cover all constructions people actually use.
Probabilistic models - from Naive Bayes classifiers to Hidden Markov Models to transformers -
estimate parameters from annotated corpora and generalize to new inputs. Evaluation metrics
such as precision, recall, and F1 are statistical tools.

1.4 NLU and NLG - The Two Halves of Language Technology

It is useful to divide language technology into two complementary processes.

Natural Language Understanding (NLU) maps from unstructured language input to structured
internal representation. Given the utterance "Book a flight to Hyderabad on Friday," an NLU
module might produce:

• Intent: BOOK_FLIGHT

• Origin: (unspecified - system may ask)

• Destination: Hyderabad

• Date: next Friday (resolved against calendar)

Dept. of CSE, SoCI Module-I | Unit-I Page 3

Module-I | Unit-I VFSTR Deemed to be University

NLU includes intent classification, named entity recognition, slot filling, sentiment analysis, and
semantic parsing.

• The central question NLU answers is: What did the speaker or writer mean?

Natural Language Generation (NLG) maps from internal representation to fluent language output.
Given a database row or a planner decision, NLG produces sentences a human can read or hear.

• The central question NLG answers is: How should this meaning be expressed in natural
language?

Most deployed systems require both. A voice assistant performs speech recognition (signal to
text), NLU (text to meaning), backend reasoning, NLG (meaning to reply text), and speech
synthesis (text to audio).

• Students should memorize the directionality: NLU is language in; NLG is language out.

• Illustrative contrast: Sentiment analysis is NLU (classify polarity from review text). Weather
report narration from structured forecast data is NLG. Machine translation uses NLU-like analysis
of the source and NLG-like production of the target.

1.5 The NLP Processing Pipeline - Conceptual Architecture

Although modern neural models sometimes blur stage boundaries, the classical NLP pipeline
remains the best pedagogical framework. Text passes through a sequence of analyses of
increasing abstraction:

1. Lexical analysis - segmentation into tokens (words, punctuation, subwords).

2. Morphological analysis - internal structure of words; lemmas and inflections.

3. Syntactic analysis - grammatical structure, often as a tree or dependency graph.

4. Semantic analysis - literal meaning representations; roles of entities and events.

5. Pragmatic / discourse analysis - intended meaning using context, prior sentences, and world
knowledge.

• Consider the sentence: I am feeling hungry.

• Lexical: tokens [I, am, feeling, hungry]

• Morphological: feel+ing (progressive aspect); hungry (adjective, base form)

• Syntactic: sentence consists of subject NP(I) and predicate VP(am feeling hungry)

Dept. of CSE, SoCI Module-I | Unit-I Page 4

Module-I | Unit-I VFSTR Deemed to be University

• Semantic: experiencer = speaker; internal state = hunger; aspect = ongoing

• Pragmatic: in a dialogue system, likely intent = FOOD_REQUEST or restaurant search

Changing one word - I am feeling tired - preserves the pipeline structure but alters semantic and
pragmatic output (REST suggestion instead of FOOD). This distinction is frequently tested in
university exams.

Lexical analysis
|
Morphological analysis
|
Syntactic analysis (parse tree)
|
Semantic analysis
|
Pragmatic / discourse analysis
Figure 0.2: The classical NLP processing pipeline

1.6 Economic and Social Significance

Human language constitutes the largest repository of unstructured digital information. Industry
estimates suggest that more than eighty percent of enterprise data is unstructured, predominantly
text. Search engines, recommendation systems, compliance monitoring, clinical documentation,
legal discovery, and citizen-facing government services all depend on NLP.

In India, initiatives such as Bhashini under MeitY aim to deliver digital government and
commercial services in all scheduled languages through speech and translation interfaces. For a
student at an Indian university, NLP is therefore not merely an academic subject but a nationally
strategic technology area.

The global NLP market has expanded rapidly with transformer-based large language models.
Nevertheless, foundational skills - tokenization, tagging, parsing, evaluation metrics, ambiguity
analysis - remain essential because they explain why models succeed or fail.

Dept. of CSE, SoCI Module-I | Unit-I Page 5

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Neighbors of NLP - AI, ML, and Related

Fields

2.1 Introduction

Natural Language Processing does not exist in isolation. It draws tools, theories, and evaluation
traditions from neighboring disciplines. Confusing these fields leads to poor system design - for
example, applying deep learning tooling to a problem that requires only a finite-state
morphological analyzer, or conversely attempting to hand-code rules for a task that demands
statistical generalization.

This topic maps the intellectual neighborhood of NLP and clarifies the nested relationship among
Artificial Intelligence, Machine Learning, Deep Learning, and NLP applications.

• Additional solved examples for this topic: Problem 2.1. Full solutions appear at the end of
this unit.

2.2 Related Academic Disciplines

Linguistics supplies formal models of language structure and meaning. Cognitive science informs
computational psycholinguistics - how humans parse and produce language in real time.
Mathematics and statistics provide probability theory, information theory, and formal language
theory (regular, context-free, and beyond). Speech processing handles acoustic signals, phonetic
transcription, and pronunciation modeling - historically a sibling field that converges with NLP in
modern voice assistants.

Artificial intelligence broadly encompasses any system performing tasks associated with human
intelligence: planning, perception, reasoning, and language. NLP is a major subfield of AI focused
specifically on linguistic input and output.

2.3 AI, Machine Learning, and Deep Learning - Precise Distinctions

Students must internalize the following hierarchy:

• Artificial Intelligence: the umbrella discipline.

• Machine Learning: a subset of AI in which behavior is learned from data rather than explicitly
programmed.

• Deep Learning: a subset of ML using neural networks with multiple layers of representation
learning.

Examples for examination:

Dept. of CSE, SoCI Module-I | Unit-I Page 6

Module-I | Unit-I VFSTR Deemed to be University

- A regex-based email validator is AI (pattern matching) but not ML.

- A logistic-regression spam filter is AI and ML but not necessarily DL.

- BERT or GPT-based text classifiers are AI, ML, and DL.

NLP tasks may be solved at any of these levels depending on data availability, interpretability
requirements, latency constraints, and accuracy targets. Industry practice increasingly favors DL
for high-accuracy open-domain tasks, but rule-based and classical statistical components persist
in production pipelines (e.g., regex normalization before neural inference).

Artificial Intelligence (broadest)

|
Machine Learning (learn from data)
|
Deep Learning (neural networks)
|
NLP tasks (MT, tagging, dialogue, ...)
Figure 0.1: Nested relationship among AI, ML, and Deep Learning

2.4 Information-Theoretic View of Language Modeling

Probability and entropy appear throughout NLP. The entropy of a discrete distribution over
intents, words, or tags measures average uncertainty in bits. Maximum entropy occurs when all
outcomes are equally likely. Skewed distributions - common in language due to Zipf's law - have
lower entropy because a few high-frequency events dominate.

Understanding entropy prepares students for language modeling, cross-entropy loss in neural
networks, and perplexity evaluation in later modules.

Dept. of CSE, SoCI Module-I | Unit-I Page 7

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Origins and Evolution of NLP

3.1 Historical Overview and the Symbolic Era

Natural Language Processing did not emerge fully formed in the age of large language models.
Its intellectual history spans more than seven decades and is commonly organized into three
overlapping eras: symbolic, statistical, and neural. Understanding this evolution is essential
because examination questions at the university level frequently ask students to compare eras,
identify representative systems, and explain why each paradigm rose and eventually gave way to
the next.

The symbolic era, roughly from the 1950s through the 1980s, treated language as a formal object
governed by explicit rules. Researchers believed that if one could write down a comprehensive
grammar and a lexicon, a computer could analyze and generate sentences by applying logical
inference. Alan Turing's 1950 paper posed the imitation game, which later became known as the
Turing Test, framing conversational competence as a benchmark for machine intelligence. The
Georgetown-IBM machine translation experiment in 1954 translated a limited set of Russian
sentences into English and generated enormous public optimism. Leaders predicted that general
machine translation would be solved within a few years.

Early dialogue systems illustrated both the promise and the illusion of symbolic NLP. ELIZA,
developed by Joseph Weizenbaum at MIT in the 1966, pattern-matched user input against
scripted templates and produced responses that mimicked a Rogerian psychotherapist. Users
often attributed genuine understanding to the program, even though ELIZA possessed no
semantic model whatsoever. SHRDLU, Terry Winograd's 1970 blocks-world system, combined a
restricted grammar with a simulated physical environment and demonstrated that constrained
domains could support surprisingly rich linguistic interaction.

The fundamental weakness of purely symbolic approaches was brittleness. Hand-crafted rules
covered only the constructions their authors anticipated. Real language exhibits productive
ambiguity, idiomatic usage, spelling variation, and domain-specific terminology. A rule that
handles one passive construction may fail on a superficially similar sentence with a different verb
class. Maintenance cost grew combinatorially as rule bases expanded. By the late 1980s, the AI
winter had cooled enthusiasm for expert-system-style NLP, though formal grammars and
knowledge representation never disappeared entirely. They remain important for morphological
analyzers, controlled languages, and hybrid pipelines that combine rules with learned
components.

• Additional solved examples for this topic: Problem 3.1, Problem 3.2, Problem 3.3. Full
solutions appear at the end of this unit.

3.2 The Statistical Revolution

The second era of NLP, dominant from roughly the late 1980s through the early 2010s, replaced

Dept. of CSE, SoCI Module-I | Unit-I Page 8

Module-I | Unit-I VFSTR Deemed to be University

or supplemented hand-written rules with models estimated from annotated corpora.

• Two developments made this shift possible: the availability of large machine-readable text
collections and affordable computing power sufficient to estimate parameters for probabilistic
models.

Hidden Markov Models became the workhorse of sequence labeling. For part-of-speech tagging,
each word generates an observed token while a hidden state represents the grammatical
category. Transition probabilities capture tag sequences such as determiner followed by noun;
emission probabilities capture how likely each word is under each tag. The Viterbi algorithm finds
the most probable tag sequence for a sentence in polynomial time. Similar HMM architectures
underpinned early speech recognition systems, linking NLP tightly to speech technology.

Probabilistic context-free grammars extended statistical reasoning to syntactic structure. Instead

of admitting all parses permitted by a CFG, a PCFG assigns probabilities to production rules.
When multiple parse trees are grammatically legal, the parser ranks them by joint probability.

• This was a direct response to syntactic ambiguity: grammar alone underdetermines

structure, but corpus statistics prefer certain attachments and constituent expansions.

Machine translation underwent a parallel transformation. IBM alignment models, developed by

researchers including Brown, Cocke, Pietra, and Mercer, learned word-level translation
probabilities from parallel bilingual corpora. Phrase-based systems extended this idea to
multi-word units, dramatically improving fluency over word-by-word substitution. Evaluation shifted
from anecdotal examples to automatic metrics such as BLEU, which compare system output
against reference translations using n-gram overlap.

The statistical era delivered measurable gains on real-world text, but it imposed its own costs.

• Feature engineering consumed enormous researcher effort: designing lexical, syntactic,

and orthographic features for each new task. Sparse high-dimensional feature spaces required
careful regularization.

• Models were opaque in a different way from neural networks: performance depended on
feature templates whose interaction was difficult to reason about. Nevertheless, statistical NLP
established the empirical discipline that persists today. No modern system ignores data,
evaluation metrics, or train-test methodology, regardless of whether its internal representation is a
logistic regression or a transformer.

3.3 The Neural and Transformer Era

The third era, still unfolding, replaces hand-designed features with learned distributed
representations. Neural networks map discrete linguistic symbols to continuous vectors and
compose them through layers of nonlinear transformations. When training data and compute are
sufficient, these models discover features that would have taken years to engineer manually.

Dept. of CSE, SoCI Module-I | Unit-I Page 9

Module-I | Unit-I VFSTR Deemed to be University

Word2Vec, introduced by Mikolov and colleagues around 2013, demonstrated that a shallow
neural network trained on a word prediction objective could learn embeddings capturing semantic
regularities.

• Vector arithmetic on embeddings produced famous analogies: king minus man plus woman
approximates queen. GloVe offered an alternative factorization of co-occurrence statistics. These
static embeddings assigned one vector per word type, limiting their ability to represent
context-sensitive meaning.

Sequence-to-sequence models with attention, particularly the architecture proposed by

Bahdanau, Cho, and Bengio in 2014, revolutionized machine translation. An encoder reads the
source sentence into a sequence of hidden states; a decoder generates the target sentence while
attending to relevant encoder positions. Attention solved the bottleneck of compressing an entire
source sentence into a single fixed-length vector.

The transformer architecture, introduced in the 2017 paper "Attention Is All You Need" by
Vaswani and colleagues, removed recurrent structure in favor of self-attention. Each token
attends to all other tokens in a layer, enabling parallel training on modern hardware. BERT,
released in 2018, pretrained a bidirectional transformer on masked language modeling and
next-sentence prediction, then fine-tuned it on downstream tasks with remarkable sample
efficiency. GPT-style decoder-only models scaled pretraining on web text and became
general-purpose text generators.

The neural era raised accuracy on many benchmarks by large margins, but introduced new
challenges. Training large models requires substantial GPU or TPU clusters and energy
expenditure.

• Model behavior is difficult to interpret: attention weights are not explanations. Generative
models hallucinate plausible but false statements. Bias present in training corpora propagates
into system outputs. For VFSTR students, the pedagogical point is not to declare neural methods
victorious in all circumstances, but to understand what problem each era solved and what
trade-offs remain.

3.4 Pedagogical Synthesis: Comparing the Three Eras

A compact comparison helps consolidate the historical narrative for revision and examination
preparation.

The symbolic era prioritized linguistic insight encoded as explicit rules. Strengths include
interpretability, precise control in narrow domains, and zero training data requirement for rule-only
components. Weaknesses include poor coverage of open-domain text, high maintenance cost,
and inability to rank competing analyses when multiple rules fire.

The statistical era prioritized empirical estimation from corpora. Strengths include robust
performance on noisy real-world input, principled handling of ambiguity through probability, and

Dept. of CSE, SoCI Module-I | Unit-I Page 10

Module-I | Unit-I VFSTR Deemed to be University

systematic evaluation methodology. Weaknesses include heavy feature engineering, sensitivity to

domain shift when test data differs from training data, and limited ability to capture long-range
dependencies without elaborate feature design.

The neural era prioritizes representation learning at scale. Strengths include state-of-the-art
accuracy on many tasks, reduced manual feature design after pretraining, and unified
architectures serving multiple tasks through fine-tuning or prompting. Weaknesses include data
and compute requirements, opacity of internal decisions, and risks of hallucination and bias in
generation.

• Remark for classroom discussion: production systems at major technology companies rarely
use only one era. A commercial email filter may apply regular-expression normalization
(symbolic), a logistic regression or gradient-boosted model on TF-IDF features (statistical), and a
transformer-based phishing detector (neural) in a staged pipeline. The educated engineer selects
components based on latency, interpretability, data availability, and accuracy targets rather than
ideological commitment to a single paradigm.

• Timeline anchors worth memorizing: Turing Test 1950; ELIZA 1966; statistical MT
resurgence 1990s; Word2Vec 2013; attention-based seq2seq 2014; transformers 2017; BERT
2018. Examinations may ask you to place a named system or paper in the correct era and justify
your classification.

• Revision exercise: Draw three columns labeled Symbolic, Statistical, Neural with rows for data
need, interpretability, ambiguity handling, and failure mode. Avoid teleological history; explain
which bottleneck each era addressed.

Symbolic era ----> Statistical era ----> Neural era

(hand rules) (corpus counts) (embeddings)
Figure 0.1: Three eras of NLP compared

3.5 Research Directions Emerging from Historical Lessons

History teaches that no paradigm eliminates the need for linguistic theory, empirical evaluation,
and engineering discipline. Current research directions reflect lessons from all three eras.

Low-resource and multilingual NLP addresses the fact that most languages lack the billion-token
corpora available for English. Transfer learning from high-resource languages, multilingual
pretraining, and community-driven dataset creation (including Indian government initiatives)
extend neural methods beyond a handful of wealthy-language benchmarks.

Efficient NLP seeks to compress large models through distillation, quantization, and sparse
architectures so that mobile devices and edge deployments can run language technology without
cloud latency. This reconnects with the computational theme introduced in later topics.

Interpretability and controllability research asks whether systems can expose the reasoning

Dept. of CSE, SoCI Module-I | Unit-I Page 11

Module-I | Unit-I VFSTR Deemed to be University

behind a classification or allow users to constrain outputs through formal specifications. Symbolic
methods never lost their appeal for regulated industries requiring audit trails.

Retrieval-augmented generation combines neural fluency with symbolic or database grounding: a

generator conditions on retrieved documents rather than relying solely on parametric memory.
This directly attacks hallucination, one of the neural era's most visible failure modes.

For the undergraduate student, the historical arc is not a story of obsolescence but of
accumulation. Finite-state morphological analyzers written in the 1980s still run in Indian
language pipelines. TF-IDF remains a strong baseline for document classification. Transformers
dominate leaderboard tasks but do not absolve the engineer from understanding tokenization,
ambiguity, or evaluation metrics developed across decades of prior work.

Dept. of CSE, SoCI Module-I | Unit-I Page 12

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Applications and Case Studies

4.1 Landscape of NLP Applications

Natural Language Processing has moved from laboratory demonstrations to infrastructure that
billions of people interact with daily. Mapping an application to its underlying NLP task is a core
examination skill because product names change while task definitions remain stable.

Search engines perform query understanding, document indexing, and relevance ranking. Web
pages are tokenized, stemmed or lemmatized, and inverted indexes map terms to documents.
Modern systems add neural re-rankers that score query-document pairs using contextual
embeddings.

Machine translation converts text or speech from a source language to a target language while
preserving meaning. Google Translate, Microsoft Translator, and India's Bhashini platform
exemplify deployment at national scale. Translation quality varies by language pair depending on
parallel corpus availability.

Virtual assistants such as Siri, Alexa, and Google Assistant chain speech recognition, intent
classification, slot filling, backend API calls, and natural language generation. Each stage is an
NLP or speech subtask with distinct error profiles.

Grammar and writing assistants (Grammarly, language tools for Indian languages) combine spell
checking, grammatical error detection, and style suggestions. Clinical NLP extracts diagnoses,
medications, and procedures from electronic health records. Legal technology searches millions
of documents for relevant clauses during discovery.

Sentiment and opinion mining monitor brand perception on social media. Chatbots handle
customer support for banks, telecom operators, and e-commerce platforms. Content moderation
detects hate speech, spam, and policy violations in user-generated text.

The unifying design pattern across these applications is transformation of unstructured language
into structured action: a ranked list, a database query, a translated string, a classification label, or
a generated response. Students should practice stating the input modality, output representation,
and evaluation metric for any named application.

• Application mapping drill: For each product category write Input, Output, Primary task, Metric
on one line. Legal e-discovery, clinical coding, and citizen grievance portals in Indian
e-governance consume NLP at scale.

• Additional solved examples for this topic: Problem 4.1, Problem 4.2, Problem 4.3. Full
solutions appear at the end of this unit.

4.2 Case Study - Email Spam Detection

Dept. of CSE, SoCI Module-I | Unit-I Page 13

Module-I | Unit-I VFSTR Deemed to be University

Spam detection is the canonical introductory case study because it illustrates the full pipeline from
raw text to deployed classifier with interpretable features and well-understood evaluation metrics.

• Problem definition: Given an incoming email message, assign a binary label spam or ham
(legitimate). Constraints include processing millions of messages per hour with low false-positive
rate, because legitimate mail wrongly blocked harms user trust.

Preprocessing begins with parsing MIME structure to extract subject and body text. HTML tags
are stripped. Text is lowercased for consistency, though case can be a feature for shouting
detection. Tokenization splits on whitespace and punctuation. Stopword removal is optional; for
spam, rare tokens such as drug names may be informative and should be retained.

Feature extraction historically used bag-of-words counts or TF-IDF weighting. Term frequency
measures how often a token appears in a document. Inverse document frequency down-weights
terms that appear in many messages because they lack discriminative power. A message
containing "winner," "claim," and "prize" with high TF-IDF weights for those terms receives
elevated spam score.

Classification algorithms span eras. Naive Bayes with multinomial event model was an early
baseline assuming conditional independence of features given the class. Logistic regression and
linear SVMs improved accuracy with calibrated probabilities. Modern systems may add character
n-grams to catch obfuscation ("pr!ze") and neural embeddings for semantic similarity to known
spam campaigns.

Evaluation uses precision, recall, and F1 on a held-out test set. Precision measures what fraction
of predicted spam is truly spam. Recall measures what fraction of actual spam is caught. For
email providers, false positives (ham classified as spam) are often costlier than false negatives,
so precision may be weighted heavily in product decisions.

• Deployment considerations include adversarial adaptation: spammers mutate wording to

evade filters, requiring continuous retraining on fresh labeled data. Privacy regulations may
restrict storing message content, pushing filters toward on-device or federated learning
approaches.

• Spam extension: TF-IDF maps documents into sparse space where linear classifiers separate
classes efficiently. Discuss class imbalance and adversarial adaptation. Privacy-conscious
designs hash tokens or classify on-device.

4.3 Case Study - Neural Machine Translation

Neural machine translation (NMT) replaced phrase-based statistical systems on many language
pairs during the mid-2010s. Understanding NMT at a conceptual level prepares students for
India's multilingual digital infrastructure goals.

• Architecture: An encoder network reads the source sentence token by token, producing a

Dept. of CSE, SoCI Module-I | Unit-I Page 14

Module-I | Unit-I VFSTR Deemed to be University

sequence of hidden representations. A decoder network generates target tokens one at a time,
conditioned on prior generated tokens and on the encoder states. Attention mechanisms allow the
decoder to focus on relevant source words when producing each target word, solving the
information bottleneck of fixed-length encoding.

• Consider English to Hindi translation for the sentence: "The students are reading books."

English uses subject-verb-object order with auxiliary "are" marking progressive aspect.

• Hindi typically places the verb finally: "Vidyaarthi kitaabein padh rahe hain." The encoder
must represent plural subject, progressive aspect, and object noun phrase. The decoder must
generate postpositions, inflect the verb for gender and number of the subject, and place the verb
at sentence end. Attention weights often align "students" with "vidyaarthi" and "reading" with
"padh rahe hain."

• Training requires parallel corpora: sentence pairs aligned across languages. Government
ministries, United Nations proceedings, and web-crawled bilingual sites supply data.
Low-resource pairs (English to a scheduled Indian language with limited parallel text) benefit from
multilingual models trained jointly on many languages, enabling transfer of syntactic knowledge.

Evaluation combines automatic metrics and human judgment. BLEU compares n-gram overlap
between system output and one or more reference translations. BLEU correlates imperfectly with
fluency; human evaluators rate adequacy (meaning preserved) and fluency (grammaticality) on
sample outputs.

Failure modes include hallucination of content not present in the source, mishandling of named
entities, and dropping of negation.

• Domain shift hurts performance: a model trained on news may fail on medical discharge
summaries. Post-editing by human translators remains common in professional workflows.

• NMT extension: Subword tokenization handles rare names. Back-translation augments parallel
data for low-resource Indian language pairs. Human evaluation with bilingual annotators remains
essential for high-stakes domains.

4.4 General Design Pattern for NLP Applications

Nearly every NLP application, regardless of domain glamour, follows a repeatable engineering
lifecycle. Mastery of this pattern separates students who memorize product names from those
who can design systems.

• Step one: Problem formulation. Define input (single sentence, document, dialogue turn, speech
utterance), output (label, span, tree, translation, generated text), and success metric (accuracy,
F1, BLEU, user satisfaction score). Ambiguous problem statements produce systems that
optimize the wrong objective.

Dept. of CSE, SoCI Module-I | Unit-I Page 15

Module-I | Unit-I VFSTR Deemed to be University

• Step two: Data acquisition and annotation. Supervised tasks require labeled examples.
Annotation guidelines must be written before labeling begins to ensure consistency.
Inter-annotator agreement (Cohen's kappa) quantifies label quality. For Indian language
applications, script normalization and code-mixing policies must be decided upfront.

• Step three: Preprocessing and feature extraction or representation learning. Choices made
here propagate downstream. Aggressive stemming may destroy morphological cues needed for
Indian languages. Subword tokenization (BPE, SentencePiece) balances vocabulary size with
coverage of rare words.

• Step four: Model selection and training. Baseline with a simple model before investing in large
neural architectures. Compare against the baseline on the same test split to justify complexity.

• Step five: Evaluation and error analysis. Aggregate metrics hide systematic failures. Manual
inspection of false positives and false negatives reveals whether errors cluster on long sentences,
named entities, or informal register.

• Step six: Deployment and monitoring. Production systems face domain shift when user
language evolves. Monitoring drift in input distribution and periodic retraining are operational
necessities, not optional polish.

• Lifecycle emphasis: Maintenance dominates long-term cost. Domain shift breaks static
models. Monitoring confidence drift triggers retraining. Ethics review for employee or clinical text
is mandatory.

4.5 Indian Context - Bhashini and Multilingual Service Delivery

India's linguistic diversity makes NLP a strategic national capability rather than a convenience
feature. The Bhashini mission under the Ministry of Electronics and Information Technology aims
to build a national public digital platform for languages, enabling citizens to access government
and commercial services in their preferred scheduled language through speech and text
interfaces.

The engineering challenge is scale across 22 scheduled languages with differing scripts,
morphological complexity, and digital resource availability. English-medium NLP tooling does not
transfer without adaptation. Automatic speech recognition must model phonetic inventories of
Dravidian and Indo-Aryan languages. Machine translation must handle SOV word order and rich
verbal inflection. Text-to-speech requires recorded speech corpora and prosody models for each
target language.

For VFSTR students, connecting global NLP case studies (spam filtering, NMT) to India's
public-sector deployment context demonstrates applied relevance. A spam filter protects email; a
Hindi-English machine translation API enables a farmer in a rural district to read agricultural
advisories. The underlying tasks are the same; the societal impact differs.

Dept. of CSE, SoCI Module-I | Unit-I Page 16

Module-I | Unit-I VFSTR Deemed to be University

• Bhashini context: Multilingual APIs lower barriers for voice interfaces in scheduled languages.
Exam questions may ask how case studies differ when script is Devanagari and morphology is
rich.

Dept. of CSE, SoCI Module-I | Unit-I Page 17

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Themes of NLP

5.1 Representation - Encoding Language for Computation

The first enduring theme of Natural Language Processing is representation: how to encode
linguistic objects so that algorithms can process them. Computers manipulate numbers, vectors,
graphs, and symbolic structures. Human language arrives as utterances, characters, and sounds.
The bridge between the two determines what a system can learn and how efficiently it operates.

At the most primitive level, text is a sequence of Unicode code points. Tokenization segments
character sequences into words, subwords, or morphemes depending on language and
application. For English, whitespace splitting with punctuation rules is a starting point. For
agglutinative Indian languages, token boundaries may not align with whitespace, requiring
morphological segmenters.

One-hot encoding represents each word type as a vector of vocabulary size with a single 1 and
all other entries 0. One-hot vectors are sparse, high-dimensional, and treat every word pair as
equally dissimilar. They cannot express that "cat" and "dog" are semantically related.

TF-IDF sparse vectors represent documents in a term space. Each dimension corresponds to a
vocabulary term weighted by frequency in the document and rarity across the corpus. TF-IDF
remains competitive for document classification and information retrieval baselines because it
captures lexical salience without training.

Dense word embeddings map each word to a low-dimensional real vector learned from
co-occurrence or prediction objectives.

• Word2Vec and GloVe produce static embeddings: one vector per word type regardless of
context. Contextual embeddings from ELMo, BERT, and successors produce different vectors for
the same word in different sentences, essential for resolving lexical ambiguity.

Structured representations include parse trees, dependency graphs, semantic role structures, and
knowledge graph triples. A dependency graph links heads to dependents with labeled relations
such as subject, object, and modifier. These structures support reasoning and generation tasks
that bag-of-words cannot address.

Representation choice involves trade-offs among memory footprint, training data requirement,
interpretability, and suitability for downstream task. Mobile keyboards may use compact subword
models; legal analytics may combine neural embeddings with explicit citation graphs.

• Representation deep dive: Subword tokenization balances open vocabulary with finite
embedding tables.

• Choose representation to match task: sparse for retrieval, dense for similarity, trees for
parsing.

Dept. of CSE, SoCI Module-I | Unit-I Page 18

Module-I | Unit-I VFSTR Deemed to be University

• Additional solved examples for this topic: Problem 5.1, Problem 5.2, Problem 5.3. Full
solutions appear at the end of this unit.

5.2 Ambiguity Resolution

The second theme is that natural language is systematically ambiguous at every level, and NLP
systems must resolve or rank competing interpretations. Ambiguity is not a bug in human
communication but a consequence of efficient encoding: speakers reuse forms with multiple
senses and rely on context to disambiguate.

Lexical ambiguity occurs when a single word form maps to multiple senses. "Bank" may denote a
financial institution or a river edge. "Bat" may be sports equipment or an animal. Word sense
disambiguation selects the intended sense using surrounding words and domain.

Syntactic ambiguity arises when a sentence admits multiple parse trees.

• Prepositional phrase attachment is the standard classroom example: "I saw the man with
the telescope" allows attachment of the PP to the verb or to the noun phrase. Both analyses are
grammatically well formed.

Semantic ambiguity includes scope ambiguities in sentences with quantifiers and modifiers.
"Every student read a book" may mean one shared book or different books per student depending
on quantifier scope.

Pragmatic ambiguity concerns intended speech acts. "Can you pass the salt?" is grammatically a
question about ability but pragmatically a request. Sarcasm inverts literal sentiment.

• NLP systems resolve ambiguity through knowledge sources: selectional restrictions (verbs
prefer certain object types), statistical preferences learned from treebanks, discourse context, and
world knowledge. Neural language models encode contextual preferences implicitly in hidden
states; symbolic systems apply explicit rules and lexicons.

Failure to handle ambiguity produces humorous errors in early MT systems and serious errors in
medical or legal NLP. The theme of ambiguity connects directly to Topic 12, which taxonomizes
ambiguity types in detail.

Ambiguity theme links to Topic 12. Ranked senses or parses beat silent wrong choices.
Calibration helps dialogue systems decide when to clarify.

Lexical --> Syntactic --> Semantic --> Pragmatic

Figure 0.1: Ambiguity arises at every linguistic level

5.3 Context and Discourse

The third theme is that sentence meaning is rarely self-contained. Interpretation depends on

Dept. of CSE, SoCI Module-I | Unit-I Page 19

Module-I | Unit-I VFSTR Deemed to be University

linguistic context (prior sentences), situational context (time, place, participants), and shared
background knowledge between speaker and hearer.

Anaphora and coreference require linking pronouns to antecedents. In "Rama met Sita. He
greeted her warmly," resolving "He" to Rama and "her" to Sita uses gender stereotypes, world
knowledge, and discourse salience. Systems use coreference resolution models trained on
annotated corpora such as OntoNotes.

Deixis anchors meaning to the speech situation. "Tomorrow," "here," and "this" require a
reference time, location, and pointing gesture or prior mention. Dialogue systems maintain a
discourse state tracking resolved entities and temporal anchors.

Discourse relations connect sentences beyond simple coreference. In "The storm knocked out
power. Classes were cancelled," the reader infers causation though no explicit connective
appears. Rhetorical structure theory and discourse parsing research formalize such relations.

Register and genre shape interpretation. The sentence "Patient presents with acute dyspnea" is
clinical; "Can't breathe, need help" is colloquial emergency language. Models trained only on
newswire may fail on social media or clinical notes.

Transformer self-attention provides a mechanism for relating tokens across long spans, partially
addressing discourse phenomena within a single forward pass. Multi-turn dialogue systems must
additionally persist state across utterances. The context theme explains why isolated sentence
benchmarks sometimes overstate real-world system capability.

• Discourse extension: Long-context models increase window size but do not guarantee correct
antecedent selection. Code-mixed Indian chat requires in-domain fine-tuning.

5.4 Computation - Engineering Constraints on NLP

• The fourth theme is computation: NLP is not only a science of language but an engineering
discipline constrained by time, memory, energy, and cost. An algorithmically correct parser that
cannot process web-scale input in real time is not a deployable solution.

Time complexity matters for parsing. Context-free parsing with CKY runs in O(n^3) in sentence
length n. Dependency parsing with transition-based systems can approach linear amortized time
with feature-rich models. Long documents require chunking or hierarchical processing.

Space complexity matters for vocabulary and model storage. A language model with 175 billion
parameters requires hundreds of gigabytes in full precision, impractical on edge devices.
Quantization and distillation compress models at some accuracy cost.

Throughput and latency govern user experience. Speech assistants target sub-second
end-to-end response. Batch offline processing of legal archives tolerates hours of runtime.

Dept. of CSE, SoCI Module-I | Unit-I Page 20

Module-I | Unit-I VFSTR Deemed to be University

• System design must match latency budget to architecture: streaming ASR, cached
embeddings, and approximate nearest-neighbor search are engineering responses to
computational theme.

Energy consumption has become a societal concern as large models train on thousand-GPU
clusters for weeks. Efficient architectures, smaller models for narrow tasks, and conditional
computation (routing easy inputs to lightweight models) are active research and product
strategies.

Parallelization exploits modern hardware. Transformers train efficiently on GPUs because matrix
operations parallelize; recurrent models with sequential dependencies parallelize less well. Data
parallelism, model parallelism, and pipeline parallelism partition work across devices.

Students should connect computational theme to India's mobile-first internet: NLP for feature
phones and low-bandwidth environments cannot assume cloud-only inference. On-device models
for keyboard prediction and offline translation exemplify computation-aware design.

• Computation extension: Training and inference have different bottlenecks. Edge deployment
uses quantization. Relate latency budgets to dialogue user experience from Topic 1.

5.5 Integration of the Four Themes

Representation, ambiguity, context, and computation are not independent modules but interacting
forces in every NLP system.

A representation that ignores context (static word embeddings) struggles on ambiguous words
until replaced or augmented with contextual models. A highly accurate model that ignores
computation cannot serve real-time speech interfaces. A discourse-aware model with rich
structure may be slow to parse, forcing approximate methods at scale.

Design exercises for examinations often present a scenario and ask which theme dominates. For
a spell checker on a mobile phone, computation and representation (compact lexicon) dominate.
For literary metaphor interpretation, ambiguity and context dominate with computation secondary.

• Integration capstone: Mobile keyboards stress computation; contracts stress ambiguity and
context. Write one paragraph linking all four themes for 5-mark practice.

Dept. of CSE, SoCI Module-I | Unit-I Page 21

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Language and Knowledge

6.1 Why Knowledge Is Inseparable from Language

Language does not float free of the world it describes. To understand an utterance, a hearer or an
NLP system must bring both linguistic knowledge (how words and structures combine) and world
knowledge (facts about objects, events, social norms, and physical causation). Purely syntactic
analysis can tell you that a sentence is well formed without telling you whether it is plausible,
truthful, or intended sincerely.

• Consider the sentence: "The chicken is ready to eat." Syntactically, this is ambiguous. One
reading treats "the chicken" as the agent of eating (the chicken will eat something). Another
reading treats "the chicken" as the patient (someone will eat the chicken). Syntax alone permits
both bracketings. World knowledge strongly favors the edible reading in a dinner context because
chickens are commonly prepared as food, while chickens eating meals is unusual though not
impossible (one might be describing farm feeding time).

Similarly, "The trophy would not fit in the suitcase because it was too big" suggests the trophy is
big, while substituting "too small" flips the referent of "it" to the suitcase. Humans resolve such
cases using physical reasoning about typical object sizes. NLP systems without world models
reproduce pronoun attachment errors that children outgrow early in development.

This inseparability explains why knowledge graphs, commonsense databases, and retrieval from
document corpora appear in modern NLP stacks. Large language models encode implicit world
knowledge in parameters learned from text, but that knowledge is incomplete and may be wrong.
Symbolic knowledge bases offer precision but limited coverage. Hybrid approaches remain an
active research and engineering frontier.

• Additional solved examples for this topic: Problem 6.1, Problem 6.2, Problem 6.3. Full
solutions appear at the end of this unit.

6.2 The Six Knowledge Layers

Pedagogically, language understanding is decomposed into six interacting knowledge layers.

Examinations at VFSTR often request definitions and examples for each layer.

Phonological and phonetic knowledge concerns sound patterns. Which sound sequences are
legal words in a language? How is orthography mapped to pronunciation? This layer matters for
speech recognition, text-to-speech, and spell checking. English "knight" preserves silent letters
historically; Hindi schwa deletion rules affect pronunciation of written conjuncts.

• Morphological knowledge concerns word structure: roots, affixes, inflection, derivation.

Knowing that "unhappiness" decomposes into un- + happy + -ness enables analysis of unseen
words. Indian languages with agglutinative morphology require morphological analyzers before

Dept. of CSE, SoCI Module-I | Unit-I Page 22

Module-I | Unit-I VFSTR Deemed to be University

syntactic parsing.

• Syntactic knowledge concerns grammatical combination: subject-verb agreement, case

government, constituent structure. A parser uses syntactic knowledge to reject "The cat sleep on
mat" and prefer "The cat sleeps on the mat."

• Semantic knowledge concerns literal meaning: who did what to whom, in what manner,
when, and where. Semantic role labeling maps verbs to agents, patients, instruments, and
locations. "Rama broke the window with a rock" assigns Rama as breaker, window as broken
thing, rock as instrument.

• Pragmatic knowledge concerns intended force in context: requests, promises, threats,

irony. "It is cold in here" may be a statement or an indirect request to close the window depending
on context.

• General world knowledge encompasses facts not encoded in grammar: Paris is in France,
fire is hot, students submit assignments to instructors. Question answering and fact checking
depend on this layer.

Full comprehension integrates all applicable layers. A speech interface must traverse from
phonetics through pragmatics; a spam filter may stop at lexical and semantic cues without deep
world modeling.

• Memorize six layers with examples: phonetics in ASR, morphology in lemmatization, syntax in
parsing, semantics in SRL, pragmatics in intent, world knowledge in QA.

6.3 Classic Pedagogical Examples

Certain minimal pairs and ambiguous sentences appear repeatedly in NLP curricula because they
isolate one knowledge layer at a time.

"I saw her duck." Lexical ambiguity: "duck" as noun (animal) or verb (lower head).

• Syntactic structure differs: duck as object noun vs duck as embedded verb in reduced relative
clause "I saw her [duck]."

"John broke the window with a rock." PP attachment ambiguity: "with a rock" may modify the verb
(instrument used to break) or the noun window (window somehow associated with a rock, less
plausible). Semantics and world knowledge prefer instrument reading.

"Flying planes can be dangerous." Structural ambiguity: planes that are flying vs the activity of
flying planes. Both readings are grammatically licensed.

"Time flies like an arrow." Garden-path humor exploits temporary misparse. Initial parse may treat
"flies" as verb before reanalysis.

Dept. of CSE, SoCI Module-I | Unit-I Page 23

Module-I | Unit-I VFSTR Deemed to be University

Students should explain such examples in examinations with bracketings or dependency

sketches, not merely name ambiguity types.

• Faculty expect integration: identify layer, show competing analyses, state resolution strategy.

Practice bracket notation for duck and window examples without only naming ambiguity types.
Faculty reward integrated analysis.

6.4 Knowledge in Neural versus Symbolic Systems

Symbolic NLP systems store knowledge explicitly in lexicons, ontologies, and rule bases.
WordNet provides synonym sets and lexical relations. ConceptNet encodes commonsense
triples. Medical terminologies such as SNOMED CT standardize clinical concepts. Advantages
include inspectability and precise inference. Disadvantages include coverage gaps and
maintenance burden.

Neural systems distribute knowledge across millions or billions of parameters.

• Pretraining on web text absorbs statistical associations: capitals with countries, symptoms
with diseases. Advantages include broad coverage and graceful handling of paraphrase.
Disadvantages include hallucination, difficulty updating facts without retraining, and opaque
failure modes.

Retrieval-augmented generation explicitly queries a document store or knowledge base at

inference time, conditioning generation on retrieved passages. This architecture reconnects
language models with curated knowledge for enterprise QA over policy manuals or university
regulations.

For Indian language NLP, lexical resources may be sparse compared to English. Morphological
lexicons and bilingual dictionaries become critical symbolic assets complementing neural models
trained on limited corpora.

Compare symbolic WordNet-style knowledge with parametric LLM knowledge on updateability,

precision, and coverage trade-offs.

Key Points for Review

• Stemming is fast but approximate; lemmatization is slower but linguistically correct.

6.5 Examination-Oriented Synthesis

When answering long-form questions on language and knowledge, use a template: (1) quote or
restate the example sentence, (2) identify which knowledge layers are engaged, (3) present
competing analyses if ambiguous, (4) explain which world knowledge disambiguates, (5) note
how an NLP module would operationalize the analysis (parser, WSD, coreference system).

Dept. of CSE, SoCI Module-I | Unit-I Page 24

Module-I | Unit-I VFSTR Deemed to be University

Use the five-step examination template for every long-form knowledge-layer answer in tests.

Dept. of CSE, SoCI Module-I | Unit-I Page 25

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Challenges in NLP

7.1 Ambiguity as a Core Challenge

Ambiguity pervades natural language at lexical, syntactic, semantic, and pragmatic levels. Unlike
programming languages with deliberately unambiguous grammars, human languages tolerate
massive underspecification that hearers repair using context, world knowledge, and statistical
expectations. For NLP engineers, ambiguity is not a corner case but the default condition that
every module must handle explicitly or implicitly.

Lexical ambiguity arises when a single orthographic form maps to multiple senses. The word
"bank" may denote a financial institution or the edge of a river. Part-of-speech tagging itself is a
form of lexical disambiguation when "book" may be noun or verb. Word sense disambiguation
systems use surrounding collocations, dictionary resources, and contextual embeddings to select
the intended sense.

Syntactic ambiguity multiplies parse trees for the same token sequence. Attachment ambiguities
involving prepositional phrases, relative clauses, and coordination are ubiquitous. A parser that
returns only one tree without confidence scoring risks downstream semantic errors that are
difficult to trace.

Semantic and pragmatic ambiguity add further layers. Scope ambiguities involving quantifiers,
negation, and modals affect logical interpretation of sentences in contracts and regulations.
Indirect speech acts, sarcasm, and metaphor invert or extend literal meaning. A sentiment
classifier that reads only surface polarity fails on ironic praise.

Combined ambiguities grow combinatorially. A sentence with three lexical ambiguities and two
attachment choices can admit dozens of joint analyses. Exhaustive enumeration is infeasible at
scale. Statistical and neural methods assign high probability mass to contextually appropriate
readings while suppressing others, but they can still fail on adversarial or rare constructions.

• Remark for examinations: when asked to discuss ambiguity as a challenge, always specify the
ambiguity type, provide an original example, name the module that primarily addresses it, and
note that perfect disambiguation is neither achievable nor required in all applications. Some
systems abstain or request clarification when confidence is low.

When discussing ambiguity as a challenge, specify type, example, module, and abstention policy
for low-confidence cases.

• Additional solved examples for this topic: Problem 7.1, Problem 7.2, Problem 7.3. Full
solutions appear at the end of this unit.

7.2 Variability and Paraphrase

Dept. of CSE, SoCI Module-I | Unit-I Page 26

Module-I | Unit-I VFSTR Deemed to be University

The same communicative intent can be expressed with radically different surface forms.

• Consider weather queries: "What's the temperature?", "How hot is it outside?", "Tell me the
weather.", "Is it raining now?" A dialogue system trained predominantly on the first phrasing may
classify the others as out-of-domain despite identical user goals. This variability challenge affects
every NLU task from intent detection to semantic parsing.

Paraphrase is not random noise but a systematic property of language economy and register
variation. Formal written requests differ from casual speech. Indian English adds politeness
markers and code-mixed constructions absent from American newswire training corpora. Robust
systems therefore require diverse training data, data augmentation through back-translation or
paraphrase models, and representation learning that maps varied strings to nearby points in
embedding space.

Morphological variability compounds lexical variability especially in Indian languages. English

irregular verbs such as go, went, gone are a small illustration. Hindi or Telugu verbs may inflect
for person, gender, number, tense, aspect, and mood in ways that multiply surface forms per
lemma. A tokenizer that treats each inflection as unrelated increases data sparsity and hurts
generalization.

Normalization through stemming or lemmatization reduces sparsity but must preserve

meaning-bearing distinctions. Over-aggressive stemming merges unrelated words to the same
root. Under-segmentation in morphologically rich languages leaves analyzers blind to case
markers critical for parsing.

Domain variability presents another axis. Medical discharge summaries, legal contracts, Twitter
posts, and classroom lectures differ in vocabulary, syntax length, and noise profile. A model
trained on one domain degrades on another unless fine-tuned or adapted. Domain adaptation
and continual learning are active responses to variability challenge.

For product design, variability implies that accuracy on a single test set is insufficient. Evaluation
must include paraphrase suites and cross-register samples. Otherwise systems appear excellent
in demos yet fail on real user phrasing.

Paraphrase suites in evaluation prevent demo-grade NLU that fails on real user wording.

7.3 Context Dependence and Non-Literal Language

Sentence meaning is rarely self-contained. Interpretation depends on who speaks, to whom,

when, where, and what was said earlier in the discourse. Deictic expressions such as "here,"
"tomorrow," and "this" anchor to the speech situation and cannot be resolved from isolated
strings. A calendar assistant must resolve "tomorrow" against the user's timezone and current
date.

Coreference chains link pronouns and definite descriptions to antecedents across sentences. In

Dept. of CSE, SoCI Module-I | Unit-I Page 27

Module-I | Unit-I VFSTR Deemed to be University

multi-turn dialogue, "change it to the later one" refers to an option presented two turns earlier.
Systems without discourse state misinterpret such elliptical commands.

Non-literal language violates compositional literal semantics. Idioms such as "kick the bucket"
cannot be understood by composing word meanings.

• Metaphor maps concepts across domains: "the market crashed" is financial, not physical.

• Sarcasm inverts polarity: "Great job, you failed again" expresses criticism through positive
words. Each phenomenon requires pragmatic inference beyond syntactic and semantic analysis.

Social and cultural context shapes taboo, politeness, and honorific selection. Indian languages
distinguish formal and informal second-person address. A Telugu assistant that uses overly
familiar pronouns to elders violates social norms even if grammatically well formed. Gendered
language and respectful titles matter in customer service chatbots.

Situational context includes user goals and physical environment. "It's too loud" in a music app
means volume; in a complaint form about construction it means environmental noise. Intent
classifiers must integrate dialogue history and metadata, not only the latest utterance.

NLP responses to context dependence include coreference resolvers, discourse parsers,

dialogue state trackers, and large language models pretrained on long contexts. None eliminate
the challenge entirely; long documents still exceed context windows, and models may attend to
irrelevant context while missing critical antecedents.

Indirect speech acts require pragmatic classifiers; literal semantic parse is insufficient for
assistants.

7.4 Linguistic Diversity and Resource Inequality

Approximately 7,000 languages are spoken worldwide, yet NLP resources exist at industrial scale
for only a few dozen. English dominates annotated treebanks, pretrained models, evaluation
benchmarks, and academic publication.

• This imbalance is itself a challenge: techniques proven on English do not automatically

transfer to scheduled Indian languages with different scripts, morphologies, and word orders.

Resource inequality manifests as scarcity of labeled data, pretrained checkpoints, spell checkers,
and keyboard input methods. Telugu may have orders of magnitude less annotated NER data
than English. Low-resource conditions inflate error rates unless mitigated through multilingual
training, transfer learning, or community annotation campaigns.

Script diversity requires Unicode expertise, font rendering, and OCR pipelines tuned per script.
Devanagari conjuncts, Telugu ligatures, and Tamil pulli marks each present segmentation and
recognition difficulties absent in Latin script processing.

Dept. of CSE, SoCI Module-I | Unit-I Page 28

Module-I | Unit-I VFSTR Deemed to be University

Dialectal and regional variation within a single language further complicates modeling. Hindi
spoken in different states mixes regional vocabulary. Telugu literary register differs from
WhatsApp chat. Models trained on one register fail on another.

Government initiatives such as India's Bhashini mission explicitly target multilingual digital
inclusion. Academic curricula should connect global NLP challenges to national priorities: building
equitable language technology is an engineering and ethical obligation, not an optional
specialization.

Evaluation must be language-specific. High English BLEU does not imply high Malayalam BLEU.
Per-language F1, native-speaker judgment, and failure analysis by linguistic phenomenon are
mandatory before deployment in public services.

• Resource inequality is ethical and technical: evaluate each scheduled language separately
before deployment.

7.5 Non-Standard and Noisy Input

Real user text deviates sharply from edited newswire on which many classical models were
trained. Typos, autocorrect errors, missing punctuation, elongated emphasis ("sooooo good"),
emoji, hashtags, and URL fragments are routine on social media. Models optimized for clean text
exhibit sudden accuracy drops on noisy input unless training data includes similar noise profiles.

• Code-mixing blends languages within a single utterance or sentence: "Kal meeting hai at 5
pm" mixes Hindi and English. Tokenizers and language models trained monolingually may assign
wrong language IDs, split mixed spans incorrectly, or ignore morphological boundaries at script
switches. Indian urban digital text is among the most code-mixed globally, making this challenge
locally urgent.

Abbreviations and domain jargon evolve faster than lexicons update. Medical shorthand, student
slang, and product-specific acronyms challenge named entity recognition and spell checking.

Adversarial inputs deliberately exploit model weaknesses. Spam uses homoglyphs (Cyrillic '?'
resembling Latin 'a'), invisible Unicode characters, and paraphrased phishing text to evade filters.
Security-sensitive NLP requires red-team adversarial testing, not only clean validation accuracy.

Normalization pipelines must balance cleaning against information loss. Stripping all punctuation
destroys emoticons that carry sentiment. Aggressive lowercasing erases case-sensitive entity
cues.

• Engineering decisions should be task-specific: search may normalize heavily; authorship

attribution may preserve quirks.

Robustness techniques include training on noised data, character-level models, subword

tokenization, and human-in-the-loop review for high-risk decisions. No single preprocessor solves

Dept. of CSE, SoCI Module-I | Unit-I Page 29

Module-I | Unit-I VFSTR Deemed to be University

non-standard input; continuous monitoring of production traffic reveals emerging patterns

requiring model updates.

Adversarial testing complements clean-set accuracy for spam and moderation systems.

7.6 Mapping Challenges to Pipeline Modules

No single NLP module solves all challenges; the pipeline architecture exists precisely because
linguistic difficulties factor into stages of analysis. Tokenizers and script normalizers address
segmentation, encoding, and noisy orthography. Morphological analyzers combat inflectional
variability especially in Indian languages. Syntactic parsers resolve structural ambiguity where
possible. Word sense disambiguation, semantic role labeling, and coreference systems attack
meaning-level challenges. Intent classifiers, sentiment analyzers, and dialogue managers handle
pragmatic phenomena.

Error propagation means early-stage failure cascades irreversibly unless n-best lists or joint
models preserve ambiguity. A tokenizer that splits a multiword named entity into two tokens
prevents the NER module from ever recovering the correct span. Engineering practice therefore
profiles per-stage accuracy and invests in the weakest link.

End-to-end neural models learn implicit pipelines but remain vulnerable to the same linguistic
phenomena, often without interpretable failure traces. Debugging a wrong translation from a
transformer may require attention visualization or contrastive examples because internal stages
are not explicitly labeled.

• Hybrid systems combine strengths: regex normalization before neural inference,

morphological features fed into parsers, retrieval grounding for factual QA.

• Challenge-to-module mapping is a standard examination question format: present a

phenomenon, ask which component addresses it, and justify.

Map each challenge to pipeline module in revision tables; error propagation is a favorite exam
subquestion.

Dept. of CSE, SoCI Module-I | Unit-I Page 30

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Language and Grammar

8.1 Role of Grammar in NLP

Grammar is a formal specification of which word sequences constitute well-formed sentences and
how those words group into constituents bearing grammatical relations. Parsers consume
grammars or learned approximations to construct syntactic structure that downstream semantic
modules require. Flat bag-of-words representations cannot distinguish "dog bites man" from "man
bites dog"; syntax encodes who acts upon whom.

In NLP, grammar serves both analysis and generation. Analysis parsers assign structure to input
sentences. Generation traverses grammatical rules or neural decoders constrained by syntactic
well-formedness to produce fluent output. Machine translation benefits from syntactic reordering
models especially when source and target languages differ in word order, as English SVO versus
Hindi SOV.

Grammatical theory also delimits search spaces. Without grammar, sequence models must learn
implausible word orders from data alone. With grammar, invalid structures can be filtered before
semantic interpretation, reducing error rates in regulated domains.

Students should distinguish descriptive grammar (what speakers actually produce) from
prescriptive grammar (what style guides recommend). NLP systems model descriptive usage
learned from corpora; prescriptive rules appear only in grammar-checking products targeting
formal writing.

Distinguish descriptive grammar modeled from corpora versus prescriptive grammar in style
checkers.

• Additional solved examples for this topic: Problem 8.1, Problem 8.2, Problem 8.3. Full
solutions appear at the end of this unit.

8.2 Phrase-Structure and Context-Free Grammars

Phrase-structure grammar expands non-terminal symbols into sequences of terminals and

non-terminals until only words remain. A context-free grammar (CFG) consists of production rules
such as S -> NP VP, NP -> Det N, VP -> V NP. Constituent trees visualize noun phrases, verb
phrases, and clauses as hierarchical groupings.

CFGs are context-free because the left-hand side is a single non-terminal without surrounding
context symbols. This restriction enables efficient parsing algorithms while covering much of
natural language syntax. Programming language compilers routinely use CFGs; NLP adopted
them for linguistic analysis with probabilistic extensions.

The CKY algorithm parses CFGs in O(n^3) time using dynamic programming over chart cells

Dept. of CSE, SoCI Module-I | Unit-I Page 31

Module-I | Unit-I VFSTR Deemed to be University

recording possible constituents spanning token subsequences. Earley's algorithm handles

left-recursive grammars common in linguistic formalisms. In practice, broad-coverage parsers use
treebank-induced grammars rather than tiny pedagogical rule sets.

Probabilistic CFGs attach probabilities to rules estimated by counting expansions in annotated

corpora such as the Penn Treebank. When multiple parses exist, the parser prefers the
highest-probability tree. This unifies grammar with the statistical era's disambiguation strategy.

Limitations matter for examinations. Cross-serial dependencies in some languages, agreement

across long distances, and certain morphosyntactic interactions may require mildly
context-sensitive formalisms. Nevertheless CFG-based parsers with lexicalized or neural
reranking remain industrial baselines and teaching foundations.

Students must practice deriving simple sentences from given rules and drawing bracketed tree
notation, skills tested in 5-mark questions at university level.

• Practice CKY chart intuition: fills triangular table of spans. State O(n^3) complexity.

8.3 Dependency Grammar

Dependency grammar represents syntax as a network of directed labeled arcs from heads to
dependents. In "Rama saw the man," the verb "saw" is the head; "Rama" attaches as subject
(nsubj); "man" attaches as object (obj). Dependencies expose predicate-argument structure
directly, aligning with semantic role labeling and relation extraction.

Unlike constituency trees that introduce abstract phrase nodes, dependency graphs stay close to
lexical items.

• This parsimony aids multilingual parsing: subject and object relations are labeled
consistently even when Hindi places the verb finally and English places it medially.

Two major parsing paradigms dominate. Transition-based parsers maintain a stack and buffer,
applying shift-reduce actions learned from training data. Graph-based parsers score all possible
trees using structured prediction, often with dynamic programming for projective trees.
Non-projective dependencies, common in languages with discontinuous constructions, require
extended algorithms.

Universal Dependencies (UD) provides a cross-linguistically consistent label set adopted in Indian
language treebank efforts. UD treebanks for Hindi, Telugu, and Tamil enable training parsers that
transfer features across related languages.

Neural dependency parsers represent words with contextual embeddings, then predict arcs and
labels in one forward pass. Accuracy approaches human inter-annotator agreement on some
languages, though long sentences and rare constructions remain difficult.

Dept. of CSE, SoCI Module-I | Unit-I Page 32

Module-I | Unit-I VFSTR Deemed to be University

For examinations, students compare constituency and dependency views of the same sentence,
explain why dependencies suit SOV languages, and name one parsing algorithm with its
complexity class.

Draw dependency graph for same sentence as constituency tree to compare formalisms.

8.4 Prepositional Phrase Attachment

Prepositional phrase attachment is the canonical classroom example of syntactic ambiguity

because minimal edits to structure yield different meanings without any single word being lexically
ambiguous.

• Consider: "I saw the man with the telescope."

In the verb-attached (VP attachment) analysis, the prepositional phrase "with the telescope"
modifies the verb phrase headed by "saw." The interpretation is that the speaker used a
telescope as an instrument to see the man.

• Bracket notation: [I saw [the man] [with the telescope]] where PP attaches high to VP.

In the noun-attached (NP attachment) analysis, the PP modifies the noun phrase "the man." The
interpretation is that the man possessed or was associated with a telescope.

• Bracket notation: [I saw [the man [with the telescope]]].

Both analyses are syntactically well formed. Grammar alone underdetermines the reading.
Disambiguation draws on selectional preferences (verbs of perception often take instrument PPs),
corpus statistics (treebanks record attachment frequencies), and discourse context. If the prior
sentence mentions climbing an observatory hill, instrument reading is primed.

Automated systems learn attachment preferences from annotated data. Neural parsers implicitly
encode such preferences in hidden representations. Symbolic systems may apply heuristics such
as RASP attachment rules or lexical affinity scores.

Similar ambiguities arise with PP, adverbial, and relative clause attachment: "I ate pizza with
anchovies" (topping vs separate instrument), "The professor lectured the students with slides"
(professor or students possess slides). Students should generate original examples and diagram
both attachments for examination practice.

Generate two original PP attachment examples beyond telescope sentence for oral exams.

VP-attach: speaker used telescope to see the man

NP-attach: the man possessed the telescope
Figure 0.1: Two analyses for PP attachment

Dept. of CSE, SoCI Module-I | Unit-I Page 33

Module-I | Unit-I VFSTR Deemed to be University

8.5 Transformational and Other Theories - Pedagogical Role

Transformational grammar, historically associated with Noam Chomsky, posits deep syntactic
structures transformed by movement operations into surface strings. Passive voice,
wh-questions, and relative clauses were analyzed as derived from simpler underlying forms
through movement and deletion rules. Full transformational machinery is not implemented in most
industrial parsers today, but the pedagogical insights remain valuable for NLP students.

• Consider active and passive pairs: "Rama broke the window" versus "The window was
broken by Rama." A question-answering system must recognize that both describe the same
event with Rama as agent and window as patient. Neural models learn such equivalences from
parallel data; symbolic pipelines may store grammar interchange rules or semantic
representations independent of voice.

Government and binding theory, lexical-functional grammar, head-driven phrase structure

grammar, and categorial grammar offer alternative formalisms emphasizing different linguistic
generalizations. Broad-coverage parsers often use treebank learning rather than enforcing a
single linguistic theory, but university syllabi still expect familiarity with CFG and dependency
frameworks plus awareness that transformational ideas explain syntactic alternations.

Link grammar and combinatory categorial grammar appear in specialized parsers. For Indian
languages, linguistic descriptions in Paninian tradition influence morphological analysis even
when parsers use UD dependency formalism.

• Examination preparation should include: drawing constituency and dependency

representations, deriving sentences from CFG rules, analyzing PP attachment, and explaining
how grammatical formalisms support semantic analysis without replacing it.

Active-passive pair shows why semantic representation may abstract away from surface voice.

Dept. of CSE, SoCI Module-I | Unit-I Page 34

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Processing Indian Languages

9.1 Structural Features of Indian Languages

Languages of the Indian subcontinent, whether Indo-Aryan (Hindi, Bengali, Marathi) or Dravidian
(Telugu, Tamil, Kannada, Malayalam), share structural properties that distinguish them from
analytic languages like English. NLP pipelines designed for English require fundamental retooling.

Free or flexible word order is common, especially SOV default in Hindi and Telugu. Subject,
object, and adjuncts may permute for focus and topicalization while case marking preserves
grammatical relations. Parsers cannot rely solely on position adjacent to the verb.

Rich inflectional morphology encodes gender, number, person, tense, aspect, and case on verbs
and nouns. A single Telugu verb form may bundle what English expresses with multiple words
and auxiliaries.

• Agglutination stacks suffixes on roots: Hindi '????' (karta) patterns vs more extreme
agglutination in some South Indian languages. Segmentation is non-trivial.

Multiple native scripts (Devanagari, Telugu, Tamil, Bengali, etc.) require script-specific tokenizers,
OCR, and font handling. Romanized transliteration appears in informal digital text.

• Code-mixing with English is pervasive in urban social media: Latin script English tokens
embedded in native script sentences or vice versa.

List SOV, morphology, scripts, sandhi, and code-mixing as five pillars of Indian language NLP.

• Additional solved examples for this topic: Problem 9.1, Problem 9.2, Problem 9.3. Full
solutions appear at the end of this unit.

9.2 Comparison with English-Centric Assumptions

English NLP often assumes whitespace tokenization, fixed SVO order, minimal inflection, and
Porter stemming adequacy. None transfer directly.

• English: "The student reads books."

• Hindi (SOV): "Vidyaarthi kitaabein padhta hai." (student books reads)

Word alignment for MT must reorder constituents. Morphological agreement links verb 'padhta' to
masculine singular subject.

English plural is often -s; Hindi marks plural on nouns, adjectives, and verbs coherently. Missing
morphological analysis yields spurious unknown words and broken agreement features for

Dept. of CSE, SoCI Module-I | Unit-I Page 35

Module-I | Unit-I VFSTR Deemed to be University

generation.

English function words are separate; Hindi postpositions attach phonologically to phrases.
Segmenting postpositions incorrectly attaches them to wrong heads in dependency parsing.

Contrast English whitespace tokenization with Hindi postpositions and agreement on verbs.

9.3 Engineering Implications for NLP Pipelines

• Tokenizer: Use script-aware normalization (Unicode NFC), handle zero-width joiners in

Devanagari, avoid naive split-on-space for compounds.

• Morphological analyzer: Finite-state transducers (HFST, Xerox tools) or neural morphological

taggers segment roots and suffixes, generate feature bundles for downstream use.

• POS tagging and parsing: Feature models must include case markers, not just word order.
Universal Dependencies treebanks for Indian languages provide training data but remain smaller
than English Penn Treebank.

• Machine translation and ASR: Government missions (Bhashini) fund parallel corpora and
speech data; academic projects must document low-resource mitigation (transfer learning,
back-translation, multilingual models).

• Evaluation: Perplexity and BLEU on English do not substitute for human evaluation by native
speakers on in-domain Indian language text.

• Pipeline order: normalize script, morph analyze, POS, parse, then semantics. Skipping morph
hurts most.

9.4 Script, Orthography, and Digital Text Issues

Spelling variation arises from archaic orthography, lack of standardized spell checkers, and
phonetic typing in Roman script. Named entity recognition must handle person names spanning
scripts.

Sandhi and schwa deletion in Hindi affect pronunciation modeling for TTS and ASR.
Grapheme-to-phoneme rules differ per language.

OCR of scanned Indian language documents suffers from conjunct characters and font diversity.
Post-OCR correction uses language models and lexicons.

OCR plus language model correction is standard for scanned government forms in Indian
languages.

Dept. of CSE, SoCI Module-I | Unit-I Page 36

Module-I | Unit-I VFSTR Deemed to be University

9.5 Research and National Initiatives

India's AI mission and Bhashini platform coordinate datasets, models, and APIs across scheduled
languages. VFSTR students benefit from aligning coursework with nationally prioritized
multilingual technology.

Research directions include multilingual transformers (mBERT, IndicBERT, MuRIL), code-mixed

language modeling (Spanglish analogs for Hinglish), and community annotation for
under-resourced languages.

Align semester projects with Bhashini or Indic NLP resources when possible for societal
relevance.

Dept. of CSE, SoCI Module-I | Unit-I Page 37

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Components of NLP Systems

10.1 Modular Architecture Overview

A full-featured NLP system decomposes into interacting components, each responsible for a level
of linguistic analysis. Modular design enables specialization, independent upgrade, and
interpretable debugging even when end-to-end neural models blur boundaries in practice.

• The classical stack includes: (1) lexical analyzer or tokenizer, (2) morphological analyzer, (3)
syntactic parser, (4) semantic analyzer, (5) discourse and pragmatic analyzer, (6) knowledge
base with reasoning component. Speech systems prepend acoustic processing and append
synthesis.

Commercial platforms may collapse steps inside a single neural model, but production pipelines
often retain modular preprocessing (normalization, language ID) and post-processing (entity
linking to databases).

Draw six-box pipeline diagram labeling input/output types between modules for 5-mark diagram
questions.

• Additional solved examples for this topic: Problem 10.1, Problem 10.2, Problem 10.3. Full
solutions appear at the end of this unit.

Tokenizer -> Morphology -> Parser -> Semantics

-> Discourse / pragmatics -> Knowledge base
Figure 0.1: Modular architecture of an NLP system

10.2 Lexical and Morphological Components

• The lexical analyzer segments character sequences into tokens: words, numbers,
punctuation, emoticons. It may perform sentence boundary detection. For social media,
tokenizers handle hashtags, mentions, and URLs as distinct token types.

The morphological analyzer returns lemmas, part-of-speech tags, and morphological features
(tense, gender, case). Output feeds POS taggers and parsers. In Indian languages this
component is indispensable, not optional.

• Example flow: "????? ??? ??? ???" yields tokens with lemmas ????? (child), ??? (play),
progressive aspect, plural subject agreement.

Tokenization for social media preserves hashtags as tokens; morph output includes feature
bundles for Indian verbs.

10.3 Syntactic and Semantic Components

Dept. of CSE, SoCI Module-I | Unit-I Page 38

Module-I | Unit-I VFSTR Deemed to be University

The syntactic parser builds constituency trees or dependency graphs. Output enables semantic
role labeling, relation extraction, and grammar checking.

• The semantic analyzer constructs meaning representations: who did what to whom,
temporal and spatial modifiers, negation scope. Frame semantics and AMR (Abstract Meaning
Representation) are research formalisms; industrial systems often use slot-filling templates.

For question answering, semantic parsing maps questions to database queries (semantic parsing
for SQL or SPARQL).

Semantic templates for slots in assistants are industrial simplification of full logical forms.

10.4 Discourse, Pragmatics, and Knowledge Integration

Discourse component tracks entities across sentences, resolves coreference, and identifies
rhetorical relations. Pragmatic component classifies speech acts and sentiment.

The knowledge base stores facts, ontologies, and domain rules. Reasoner answers queries,
detects contradictions, and supports explanation. Retrieval-augmented systems treat document
indexes as external knowledge.

Dialogue managers maintain state across turns, deciding when to clarify versus answer.

Retrieval-augmented generation treats search index as knowledge component external to model

weights.

10.5 Interaction, Error Propagation, and Modern Hybrids

Components form a directed pipeline; errors propagate forward. A mis-tokenized word becomes
unknown to the morphological analyzer, yielding wrong POS, wrong parse, and wrong semantic
roles.

• Mitigation strategies: n-best lists passed between stages, joint models optimizing multiple
levels, end-to-end training with intermediate supervision, and confidence-based abstention.

• Modern hybrid: BERT tokenizer and encoder replace hand-built features for multiple tasks
simultaneously, but explicit JSON schema validation and database lookup remain symbolic
components in enterprise assistants.

Throughput profiling identifies bottlenecks. If semantic role labeling is fifty times slower than
tokenization, batching and GPU acceleration target SRL first.

Profile each module latency; optimize bottleneck first. Hybrid symbolic validation after neural NLG
is common in banking.

Dept. of CSE, SoCI Module-I | Unit-I Page 39

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Phases of Processing

11.1 The Five Phases Defined

Phases of processing describe the sequence of analyses applied to text from raw characters to
situated meaning. Although neural models may learn implicit phase-like representations, the
pedagogical framework remains essential for examinations and debugging.

• Phase 1 - Lexical analysis: segmentation into tokens and sentence boundaries.

• Phase 2 - Morphological analysis: internal structure of words, lemmas, inflections.

• Phase 3 - Syntactic analysis: grammatical structure as tree or dependencies.

• Phase 4 - Semantic analysis: literal meaning, roles, relations, scope of negation.

• Phase 5 - Pragmatic/discourse analysis: intended meaning using context and world

knowledge.

Each phase increases abstraction; earlier phases supply structure later phases consume.

Phases are pedagogical; neural models blur boundaries but exams still require five-phase traces.

• Additional solved examples for this topic: Problem 11.1, Problem 11.2, Problem 11.3. Full
solutions appear at the end of this unit.

[Link] [Link] [Link]

[Link] [Link] / discourse
Figure 0.1: Five phases of processing in order

11.2 Worked Analysis - Example A

• Sentence: I am feeling hungry.

• Lexical phase: Tokens [I, am, feeling, hungry]. Sentence boundary detected.

• Morphological phase: feel+ing (progressive participle); hungry (adjective lemma hungry); am

(auxiliary be, 1st person singular present).

• Syntactic phase: Sentence -> NP(I) + VP(am feeling hungry). VP -> auxiliary + verb phrase
headed by feeling with adjective complement hungry (predicative adjective structure).

• Semantic phase: Experiencer role assigned to speaker (I); internal state predicate HUNGRY;
aspect ongoing via progressive.

Dept. of CSE, SoCI Module-I | Unit-I Page 40

Module-I | Unit-I VFSTR Deemed to be University

• Pragmatic phase: In dialogue system, likely user intent expresses need for food or restaurant
search; politeness neutral; temporal reference present.

Reproduce hungry sentence analysis without notes as weekly drill.

11.3 Worked Analysis - Example B

• Sentence: The students are submitting their assignments.

• Lexical: six tokens identified.

• Morphological: plural noun students; progressive auxiliary are; verb submit+ing; plural
possessive their; plural noun assignments.

• Syntactic: NP subject "The students"; VP predicate with progressive aspect; direct object NP
"their assignments"; possessive anaphor their coindexed with students.

• Semantic: Event SUBMIT; agents = students; patients = assignments; possession links

assignments to students.

• Pragmatic: Academic institutional context; deadline scenario plausible; no sarcasm.

Coindex possessive their with students explicitly in syntactic and semantic phases.

11.4 Worked Analysis - Example C

• Sentence: Can you pass the salt?

• Lexical: five tokens.

• Morphological: modal can; pronoun you; verb pass; determiner the; noun salt.

• Syntactic: Yes-no question structure; VP pass the salt; NP object the salt.

• Semantic: Literal ability question: are you able to pass salt?

• Pragmatic: Indirect speech act classified as request, not information question, given dining
context. NLU should map to ACTION_PASS_OBJECT(salt), not ABILITY_QUERY.

Salt sentence is canonical indirect speech act example; contrast literal semantic vs pragmatic
intent labels.

11.5 Algorithms and Typical Errors per Phase

Dept. of CSE, SoCI Module-I | Unit-I Page 41

Module-I | Unit-I VFSTR Deemed to be University

• Lexical: regex tokenizers, BPE subword models.

• Errors: split contractions wrongly, break URLs.

• Morphological: stemmers, FST analyzers, lemmatizers with POS.

• Errors: over-stemming unrelated words to same stem.

• Syntactic: CKY, transition-based dependency parsers.

• Errors: PP attachment mistakes.

• Semantic: SRL, NER, relation extraction.

• Errors: wrong argument attachment to verb.

• Pragmatic: intent classifiers, coreference resolvers.

• Errors: sarcasm misread as literal praise.

• Evaluation metrics differ: token F1, morph accuracy, labeled attachment score, semantic role
F1, intent accuracy.

Match each phase to one algorithm and one typical error type in tabular revision notes.

Dept. of CSE, SoCI Module-I | Unit-I Page 42

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - The Problem of Ambiguity

12.1 Why Ambiguity Is Central to NLP

Ambiguity is not an occasional defect but a systematic property of human language enabling
concise expression. Speakers reuse word forms across senses, omit redundant material, and rely
on hearers to supply missing detail. For NLP systems, ambiguity is the default operational
condition rather than an exception requiring special handling.

When a machine translation system chooses the wrong sense of a polysemous word, output may
be fluent yet factually wrong. When a parser attaches a prepositional phrase incorrectly, semantic
role labeling assigns instruments to wrong predicates. When a dialogue system interprets indirect
speech literally, user goals are missed. Each failure traces to unresolved ambiguity at some
linguistic level.

Unlike formal languages designed for unambiguous interpretation, natural languages trade
precision for flexibility. Legal drafters attempt to minimize ambiguity, yet litigation still disputes
sentence scope. Poetry exploits ambiguity intentionally. NLP systems cannot eliminate ambiguity
without eliminating natural language itself.

Engineering responses include maintaining n-best analyses rather than committing early,
attaching confidence scores to decisions, and requesting clarification in interactive settings. Large
language models distribute ambiguity resolution across billions of parameters, but hallucination
shows they still guess rather than truly know in many cases.

Pedagogically, Topic 12 taxonomizes ambiguity into six types examined at VFSTR. Students
must define each type, supply original examples, and describe resolution strategies. This topic
connects grammar (Topic 8), challenges (Topic 7), and processing phases (Topic 11) into a
unified framework.

Ambiguity is default condition; systems guess with confidence or abstain.

• Additional solved examples for this topic: Problem 12.1, Problem 12.2, Problem 12.3. Full
solutions appear at the end of this unit.

Lexical | Syntactic | Semantic

Pragmatic | Anaphoric | Discourse
Figure 0.1: Taxonomy of ambiguity types

12.2 Six Types of Ambiguity

University examinations require precise definitions of six ambiguity classes with illustrative
examples. Master this taxonomy with original sentences, not only textbook repeats.

• Type 1 - Lexical ambiguity: one word form maps to multiple senses or parts of speech.

Dept. of CSE, SoCI Module-I | Unit-I Page 43

Module-I | Unit-I VFSTR Deemed to be University

• Example: "bank" as financial institution versus river edge; "bat" as animal versus sports
equipment. Resolution employs word sense disambiguation using collocations, dictionaries such
as WordNet, and contextual embeddings.

• Type 2 - Syntactic ambiguity: the same token sequence admits multiple parse trees.

• Example: "Visiting relatives can be boring" where relatives may be subject of visiting or object
being visited. Resolution uses statistical parsers, neural parsers, and grammar filters.

• Type 3 - Semantic ambiguity: logical scope and quantifier interactions yield multiple truth
conditions.

• Example: "Every student read a book" may assert one shared book or many books. Resolution
builds logical forms with explicit scope or applies corpus preferences.

• Type 4 - Pragmatic and referential ambiguity: intended referent or speech act is unclear.

• Example: "John told Bill that he failed" leaves "he" ambiguous between John and Bill.
Resolution combines coreference models, gender constraints, and world knowledge.

• Type 5 - Discourse ambiguity: relation between sentences is underspecified.

• Example: "The power failed. Classes were cancelled." may describe causation or mere
temporal sequence. Resolution uses discourse parsers and temporal reasoning.

• Type 6 - Anaphoric ambiguity: a pronoun or definite description has multiple antecedents

satisfying grammatical constraints.

• Classic Winograd schema: "The trophy would not fit in the suitcase because it was too big"
where "it" refers to trophy or suitcase depending on adjective size.

• Create flashcards: type name, definition, example, resolver on four lines each.

12.3 Resolution Strategies

Ambiguity resolution combines knowledge sources, statistical preferences, and interaction design.
No single strategy achieves perfect accuracy; production systems layer multiple signals.

• Knowledge-based methods apply selectional restrictions: verbs prefer certain argument

types (animate eaters, concrete objects). Lexical ontologies link senses to semantic classes.
Hand-crafted grammar filters reject impossible structures before semantic analysis.

• Corpus-based methods estimate preferences from annotated treebanks: how often does
verb "saw" take instrument prepositional phrases versus noun modifiers? Log-linear models score
candidate analyses using features of surrounding words, dependency distances, and constituent

Dept. of CSE, SoCI Module-I | Unit-I Page 44

Module-I | Unit-I VFSTR Deemed to be University

sizes.

Neural contextual embeddings from BERT and successors cluster appropriate senses and
attachments in hidden space without explicit feature engineering. Fine-tuned classifiers atop
encoders perform word sense disambiguation and coreference resolution at benchmark-leading
accuracy on some datasets.

Discourse-centered methods include centering theory, which ranks antecedents by salience in

prior sentences, and rhetorical structure parsers linking clauses with explicit relations.

Interactive resolution asks users to clarify when confidence falls below threshold. Clarification is
preferable to wrong action in banking, healthcare, and legal workflows.

• Safety-critical systems abstain rather than guess: a medical coding assistant that is unsure
of diagnosis sense should flag for human review. Calibration of confidence scores is an active
research area because raw softmax probabilities are often overconfident.

Students should compare strategies by interpretability, data requirement, and failure mode.
Rule-based methods explain decisions; neural methods generalize but may err silently.

Layer knowledge-based, corpus, neural, discourse, and interactive strategies in production

designs.

12.4 Case Study - Telescope Sentence

The sentence "I saw the man with the telescope" is the standard end-to-end case study linking
syntactic ambiguity to semantic and pragmatic resolution. Faculty expect complete analysis in
examinations.

• Syntactic analysis 1 (verb phrase attachment): The prepositional phrase "with the telescope"
attaches to the verb "saw." Structure: [I [saw [the man]] [with the telescope]].

• Interpretation: the observer used a telescope as an instrument to see the man at a distance.

• Syntactic analysis 2 (noun phrase attachment): The PP attaches to the noun "man."
Structure: [I saw [the man [with the telescope]]].

• Interpretation: the man possessed or was carrying a telescope; the observer may have used
unaided vision.

Both trees are grammatically licensed. Pure CFG acceptance cannot choose between them.
Disambiguation invokes verb semantics (perception verbs frequently take instrument adjuncts),
corpus statistics (treebank attachment counts for "saw"), and discourse context.

Contextual cues shift preference. Prefix "From the observatory deck, I saw the man with the

Dept. of CSE, SoCI Module-I | Unit-I Page 45

Module-I | Unit-I VFSTR Deemed to be University

telescope" primes instrument reading. Follow-up "I asked him to let me look through it" coherently
continues NP attachment where "it" refers to the man's telescope.

For NLP pipeline design, the case study motivates n-best parsing followed by semantic reranking
rather than greedy single-parse commitment. It also illustrates limits of grammar-only modules
and the necessity of world knowledge.

Students must draw both trees, state semantic consequences of each, and identify which
knowledge sources resolve ambiguity in a given mini-discourse.

• Telescope sentence: both trees, semantics, context - complete 8-mark answer structure.

12.5 Ambiguity in Indian Language Context

Indian languages exhibit all six ambiguity types discussed in universal terms, plus additional
challenges from morphology, script, and code-mixing that English-centric textbooks sometimes
underemphasize.

Segmentation ambiguity arises when agglutinative suffix chains allow multiple splits. A Telugu or
Tamil compound noun may be segmented at more than one morpheme boundary without spaces
to guide tokenizers.

Case marking reduces some syntactic ambiguities present in English because grammatical
relations are overt on nouns, but free word order reintroduces attachment questions for adverbs
and nested clauses.

Code-mixed sentences such as "Meeting cancel ho gayi" blend English nouns with Hindi verb
phrases. Language identification per span affects which lexicon and parser rules apply.

Homography in Devanagari may map different pronunciations to identical spellings until schwa
deletion rules disambiguate in speech processing. ASR and TTS pipelines must encode
phonological knowledge absent from raw graphemes.

Multilingual parsers pretrained predominantly on English treebanks may import attachment biases
inappropriate for Hindi SOV statistics. Evaluation on Indian treebanks reveals whether models
truly generalize or merely anglicize structure.

National multilingual platforms such as Bhashini must handle ambiguity at scale across scripts.
Students should illustrate each ambiguity type with at least one Indian language example in
revision notes, demonstrating local competence beyond translated English sentences.

Code-mixed Indian examples demonstrate local competence beyond translated English

sentences.

Dept. of CSE, SoCI Module-I | Unit-I Page 46

Module-I | Unit-I VFSTR Deemed to be University

Module-I, Unit-I - Typical NLP Tasks

13.1 Introduction - Task Taxonomy

Natural Language Processing is organized into numerous task types, each defined by input
representation, output representation, and standard evaluation metrics. Commercial product
names change yearly, but task definitions remain stable across textbooks and university syllabi.
VFSTR Module 1 Unit I expects mastery of seventeen typical tasks spanning low-level
morphology through speech and dialogue.

Understanding task taxonomy prevents category errors in system design. Machine translation is
not the same as information retrieval. Stemming is not lemmatization. Speech recognition is not
text-to-speech. Examinations reward precise definitions and penalize vague references to brand
names without task specification.

• Tasks cluster naturally: morphological and lexical tasks prepare text; syntactic and semantic
tasks build structure; document-level tasks classify or transform whole texts; speech tasks
interface with audio; generation and dialogue tasks produce language and manage interaction.

Modern practice often fine-tunes one pretrained transformer for multiple tasks by changing output
layers, but pedagogically students must still know classical task boundaries because debugging,
data annotation, and metric selection remain task-specific.

This chapter surveys all seventeen tasks with definitions, contrasts easily confused pairs, and
connects tasks to pipeline components from Topic 10.

• Revision strategy: flashcard each task name with one-line definition and one evaluation metric.

Seventeen tasks checklist grouped by word, sentence, document, and speech levels.

• Additional solved examples for this topic: Problem 13.1, Problem 13.2, Problem 13.3. Full
solutions appear at the end of this unit.

13.2 Morphological and Lexical Tasks

The first five typical tasks operate at word and subword level, preparing raw text for higher
analysis.

Task 1 - Tokenization segments character sequences into tokens (words, punctuation, subwords)
and often detects sentence boundaries. Input is raw text; output is token sequence. Evaluation
uses token-level precision and recall against gold segmentation, critical for Chinese and Indian
languages without clear whitespace.

Task 2 - Stemming applies heuristic rules to strip affixes and approximate a root form. Porter

Dept. of CSE, SoCI Module-I | Unit-I Page 47

Module-I | Unit-I VFSTR Deemed to be University

stemmer for English conflates "studies," "studied," and "studying" to "studi." Fast but coarse; may
merge unrelated words sharing prefix.

Task 3 - Lemmatization returns dictionary citation form using morphological analysis and POS
tags: "studying" becomes "study" as verb. More accurate than stemming, essential before
generating grammatically correct output.

Task 4 - Part-of-speech tagging assigns grammatical categories (noun, verb, adjective) to each
token. Sequence labeling task evaluated by per-token accuracy. Feeds parsers and lemmatizers.

Task 5 - Named entity recognition locates and classifies spans as person, organization, location,
date, etc. Evaluated with span-level F1. Distinct from POS tagging because entities are
multi-token phrases.

• Examination favorite: contrast stemming versus lemmatization with examples and state when
each is appropriate (IR indexing often stems; NLG and QA prefer lemmas).

Stem versus lemma contrast is guaranteed 2-mark question; prepare four examples.

13.3 Syntactic and Semantic Tasks

Tasks six through eleven build structural and meaning representations on tokenized input.

Task 6 - Dependency parsing outputs labeled arcs from heads to dependents (nsubj, obj, obl).
Evaluated with labeled attachment score (LAS). Preferred for multilingual pipelines and semantic
role labeling.

Task 7 - Constituency parsing outputs phrase structure trees with non-terminals NP, VP, PP.
Evaluated with bracketing F1 against treebank gold trees. Common in English-centric research
and grammar checking.

Task 8 - Semantic role labeling assigns roles such as agent, patient, instrument to predicate
arguments.

• Example: Rama=agent, window=patient, rock=instrument for "broke" predicate. Evaluated with

role F1.

Task 9 - Word sense disambiguation selects among dictionary senses for polysemous words
using context. Evaluated on Senseval benchmarks with accuracy per word.

Task 10 - Coreference resolution links pronouns and definite descriptions to antecedent entities
across sentences. Evaluated with MUC or CoNLL F1 on entity clusters.

Task 11 - Relation extraction identifies semantic relations between entity mentions: founded-by,
located-in, part-of. Often formulated as classification on entity pairs or as span prediction.

Dept. of CSE, SoCI Module-I | Unit-I Page 48

Module-I | Unit-I VFSTR Deemed to be University

These tasks form the bridge from syntax to applications such as knowledge graph construction
and structured question answering. Errors in parsing propagate to SRL and relation extraction,
motivating joint models or end-to-end pretraining.

Parsing output feeds SRL and relation extraction; note dependency path features.

13.4 Document and Text-Level Tasks

Tasks twelve through seventeen operate on documents, collections, or language pairs rather than
isolated sentences alone.

Task 12 - Text classification assigns a label or distribution over labels to an entire document or
sentence: spam detection, sentiment polarity, topic category.

• Metrics: accuracy, precision, recall, F1 per class.

Task 13 - Information retrieval ranks documents from a large collection by relevance to a keyword
or natural language query.

• Metrics: precision at k, mean average precision, nDCG. Returns documents, not direct
answers.

Task 14 - Question answering returns a specific answer string or span given a question and
context passage (reading comprehension) or corpus (open-domain QA).

• Metrics: exact match and F1 on answer spans.

Task 15 - Text summarization produces shorter text preserving salient content. Extractive
methods select sentences; abstractive methods generate new wording.

• Metrics: ROUGE overlap with reference summaries.

Task 16 - Topic modeling discovers latent thematic structure in document collections, classically
Latent Dirichlet Allocation assigning word distributions to topics without labels. Evaluated
qualitatively or with coherence scores.

Task 17 - Machine translation transforms text from source language to target language preserving
meaning.

• Metrics: BLEU, chrF, human adequacy and fluency ratings.

• Contrast for exams: information retrieval returns ranked documents; question answering
extracts or generates a concise answer. Summarization compresses one document; topic
modeling finds themes across many documents.

Dept. of CSE, SoCI Module-I | Unit-I Page 49

Module-I | Unit-I VFSTR Deemed to be University

• IR versus QA distinction: ranked docs vs answer string; do not confuse with search snippets
alone.

13.5 Speech, Generation, and Dialogue Tasks

Beyond text-in-text-out tasks, deployed language technology interfaces with speech audio and
interactive dialogue.

Automatic speech recognition (ASR) maps acoustic speech signals to transcript text. Evaluated
with word error rate (WER). ASR precedes textual NLP in voice assistants; errors in transcription
cascade to all downstream modules.

Text-to-speech (TTS) synthesizes intelligible speech waveforms from text input. Evaluated with
mean opinion score for naturalness and intelligibility tests. TTS is generation from structured
linguistic input, not understanding.

• Text generation broadly covers any system producing fluent language: dialogue replies,
story continuation, data-to-text weather reports. Subsumes machine translation target side and
abstractive summarization. Metrics include BLEU, human fluency, and task-specific checks for
factuality.

Dialogue management maintains state across conversational turns, decides next system action
(inform, request, confirm), and integrates NLU and NLG. Evaluated with task success rate and
user satisfaction in simulated or live studies.

• Distinctions frequently tested: ASR is perception (speech to text); TTS is production (text to
speech). NLU classifies incoming meaning; NLG generates outgoing text. Do not conflate speech
recognition with natural language understanding on the resulting transcript.

ASR WER and TTS MOS are standard metrics; define WER formula if asked.

13.6 Task Selection for System Design

Engineers decompose applications into constituent NLP tasks before selecting models and
annotation budgets.

• Consider voice banking for Hindi users: ASR transcribes spoken query; tokenizer and
morphological analyzer segment Devanagari text; intent classification (text classification task)
identifies transfer-balance versus check-balance intents; slot filling resembles NER for account
numbers and amounts; optional coreference links follow-up pronouns; retrieval queries
transaction FAQ knowledge base; NLG verbalizes confirmation; TTS speaks reply.

• Campus exam-schedule chatbot may omit ASR/TTS if text-only: document ingestion and
indexing (IR), question classification, extractive QA over schedule PDFs, summarization for long

Dept. of CSE, SoCI Module-I | Unit-I Page 50

Module-I | Unit-I VFSTR Deemed to be University

policy answers.

Task decomposition clarifies data needs. NER requires span annotations; text classification
needs document labels; parsing needs treebanks. Mixing annotation types without planning
wastes budget.

• Modern shortcut: fine-tune one multilingual transformer with task-specific heads for NER,
classification, and QA simultaneously after shared pretraining.

• Shortcut does not eliminate need to understand task metrics: NER uses span F1,
classification uses accuracy, QA uses exact match.

• Examination preparation: list all seventeen tasks from memory, group by level (word,
sentence, document, speech), and for each pair of similar tasks state one distinguishing
sentence.

Decompose voice banking and campus FAQ bot into tasks with metrics per task.

Key Points for Review

• Stemming is fast but approximate; lemmatization is slower but linguistically correct.

Dept. of CSE, SoCI Module-I | Unit-I Page 51

Module-I | Unit-I VFSTR Deemed to be University

Solved Numerical Examples

All worked problems for this unit are collected in this section. Each problem is labelled with the
same number cited in the corresponding topic (for example, Problem 2.3 refers to Topic 2,
worked example 3).

Problem 1.1: Problem 1.1 - Pipeline Token Count

• Topic: Module-I, Unit-I - Introduction to Natural Language Processing

Formula: Total tokens = sum of tokens per sentence

Problem 1.1 - Pipeline Token Count

• Sentence 1: 'I am feeling hungry.' -> 5 tokens

• Sentence 2: 'The students are submitting their assignments.' -> 7 tokens

Corpus of 1,000 sentences x 7 tokens = 7,000 tokens total.

Problem 1.2: Problem 1.2 - NLU Slot Extraction

• Topic: Module-I, Unit-I - Introduction to Natural Language Processing

Formula: Output = Intent + slot-value pairs

Problem 1.2 - NLU Slot Extraction

• Utterance: 'Book a flight to Hyderabad on Friday.'

Intent = BOOK_FLIGHT; Destination = Hyderabad; Date = next Friday

Filled slots = 3 of 4 = 75% completeness.

Problem 2.1: Problem 2.1 - Shannon Entropy

• Topic: Module-I, Unit-I - Neighbors of NLP - AI, ML, and Related Fields

Formula: H(X) = - Sum p_i log2(p_i)

Problem 2.1 - Shannon Entropy

p = [0.50, 0.25, 0.125, 0.125] for four intents

Dept. of CSE, SoCI Module-I | Unit-I Page 52

Module-I | Unit-I VFSTR Deemed to be University

H = -(0.5(-1) + 0.25(-2) + 0.125(-3) + 0.125(-3))

H = 1.75 bits (max = log2(4) = 2.00 bits)

Problem 3.1: Problem 3.1 - Model Scale Comparison

• Topic: Module-I, Unit-I - Origins and Evolution of NLP

Problem 3.1 - Model Scale Comparison

Word2Vec skip-gram with 300-dimensional vectors and a 50,000-word vocabulary uses approximately 30
million parameters. BERT-base has 110 million parameters. GPT-3 is reported at approximately 175 billion
parameters. Calculate the ratio of GPT-3 parameters to BERT-base parameters and discuss one
engineering implication.

Solution:

Ratio = 175,000,000,000 / 110,000,000 = 1,590.9 (approximately 1,591 times larger)

• Engineering implication: Serving GPT-3 requires distributed GPU clusters and incurs substantial
inference cost per query. BERT-base can be fine-tuned and deployed on a single high-end GPU for many
enterprise tasks. Scale therefore constrains which model class is feasible for a given product budget and
latency requirement.

Problem 3.2: Problem 3.2 - Corpus Growth and Statistical MT

• Topic: Module-I, Unit-I - Origins and Evolution of NLP

Problem 3.2 - Corpus Growth and Statistical MT

A research group collects parallel English-Hindi sentence pairs for statistical machine translation.

• Year 1: 100,000 pairs.

• Year 2: 500,000 pairs.

• Year 3: 2,000,000 pairs. BLEU score improves as 18.4 + 4.2 log10(N) where N is the number of pairs.
Calculate BLEU for each year and the marginal gain from Year 2 to Year 3.

Solution:

• Year 1: N = 100,000 = 10^5; log10(N) = 5; BLEU = 18.4 + 4.2(5) = 39.4

• Year 2: N = 500,000 = 5 x 10^5; log10(N) = 5.699; BLEU = 18.4 + 4.2(5.699) = 42.34

Dept. of CSE, SoCI Module-I | Unit-I Page 53

Module-I | Unit-I VFSTR Deemed to be University

• Year 3: N = 2,000,000 = 2 x 10^6; log10(N) = 6.301; BLEU = 18.4 + 4.2(6.301) = 44.86

Marginal gain Year 2 to Year 3 = 44.86 - 42.34 = 2.52 BLEU points

• Interpretation: Doubling corpus size yields diminishing returns on a logarithmic scale, consistent with
empirical observations in statistical NLP. Annotation cost grows linearly while performance gains slow.

Problem 3.3: Problem 3.3 - Rule Coverage Estimate

• Topic: Module-I, Unit-I - Origins and Evolution of NLP

Problem 3.3 - Rule Coverage Estimate

A symbolic parser has 1,200 context-free rules. Testing on 10,000 sentences reveals 3,400 failures due to
uncovered constructions. If each new rule costs 2 hours to write and test, estimate minimum additional
engineering hours to achieve zero failures on this test set (optimistic lower bound assuming one rule fixes
one failure type).

Solution:

Uncovered failure rate = 3,400 / 10,000 = 34%

Optimistic lower bound on new rules needed = 3,400 (if each failure is unique)

Engineering hours = 3,400 x 2 = 6,800 hours

At 40 hours per week, this is 170 weeks (approximately 3.3 years) for one engineer, illustrating why pure
rule expansion does not scale for open-domain NLP.

Problem 4.1: Problem 4.1 - TF-IDF for Spam Keyword

• Topic: Module-I, Unit-I - Applications and Case Studies

Problem 4.1 - TF-IDF for Spam Keyword

• A corpus contains 2,000 emails: 1,500 ham and 500 spam. The token "prize" appears in 400 spam
messages and 5 ham messages. In a test spam message of 200 words total, "prize" occurs 30 times, so
term frequency TF = 30/200 = 0.15. Compute IDF using natural log with IDF = ln(N / df) where N = 2000
and df = number of documents containing "prize". Then compute TF-IDF.

Solution:

df = 400 + 5 = 405

Dept. of CSE, SoCI Module-I | Unit-I Page 54

Module-I | Unit-I VFSTR Deemed to be University

IDF = ln(2000 / 405) = ln(4.938) = 1.596

TF-IDF = 0.15 x 1.596 = 0.239

The high IDF reflects that "prize" is rare in ham mail, making it a strong spam indicator in this corpus.

Problem 4.2: Problem 4.2 - Spam Classifier Metrics

• Topic: Module-I, Unit-I - Applications and Case Studies

Problem 4.2 - Spam Classifier Metrics

A spam filter is evaluated on 10,000 messages (2,000 spam, 8,000 ham).

• Results: 1,750 spam correctly detected; 250 spam missed; 7,600 ham correctly passed; 400 ham
wrongly blocked as spam. Compute precision, recall, and F1 for the spam class.

Solution:

TP = 1,750, FP = 400, FN = 250

Precision = 1750 / (1750 + 400) = 1750 / 2150 = 0.8140

Recall = 1750 / (1750 + 250) = 1750 / 2000 = 0.8750

F1 = 2 x 0.8140 x 0.8750 / (0.8140 + 0.8750) = 0.8430

The filter catches most spam but blocks 5% of ham (400/8000), which may be unacceptable for certain
enterprise mail gateways.

Problem 4.3: Problem 4.3 - BLEU Score from N-gram Overlap

• Topic: Module-I, Unit-I - Applications and Case Studies

Problem 4.3 - BLEU Score from N-gram Overlap

• A machine translation system produces: "the cat sat on mat".

• Reference translation: "the cat is sitting on the mat". Calculate modified unigram precision (count clipped
matches / output unigrams). Ignore brevity penalty for this simplified problem.

Solution:

• Output unigrams: the, cat, sat, on, mat (5 total)

Dept. of CSE, SoCI Module-I | Unit-I Page 55

Module-I | Unit-I VFSTR Deemed to be University

• Reference unigrams: the, cat, is, sitting, on, the, mat

• Clipped matches: the (1 in output, 2 in ref, clip to 1), cat (1), on (1), mat (1). "sat" has no match.

Modified unigram precision = 4 / 5 = 0.80

Full BLEU would apply brevity penalty because output is shorter than reference, reducing the score. This
illustrates why short fluent fragments can score misleadingly high without adequacy.

Problem 5.1: Problem 5.1 - Embedding Compression Ratio

• Topic: Module-I, Unit-I - Themes of NLP

Problem 5.1 - Embedding Compression Ratio

A vocabulary contains 50,000 word types. One-hot encoding uses one dimension per type. A dense
embedding uses 300 dimensions per word. Calculate the compression ratio of embedding dimension to
one-hot dimension and the memory to store one vector in bytes if each embedding dimension is 32-bit float.

Solution:

Compression ratio = 50,000 / 300 = 166.67 (approximately 167x fewer dimensions)

• One-hot: 50,000 bits if binary, or 50,000 x 4 bytes = 200,000 bytes as float (sparse storage is smaller in
practice)

• Dense embedding: 300 x 4 = 1,200 bytes per word vector

Dense storage is dramatically smaller per active word while capturing similarity structure unavailable in
one-hot.

Problem 5.2: Problem 5.2 - Ambiguity Counting

• Topic: Module-I, Unit-I - Themes of NLP

Problem 5.2 - Ambiguity Counting

A sentence has 2 lexical ambiguities (each word has 3 senses) and 2 independent syntactic attachments
(each with 2 choices). Assuming independence, how many combined analyses exist? If a uniform prior
assigns equal probability to each, what is P(correct) if only one analysis is correct?

Solution:

Lexical combinations = 3 x 3 = 9

Dept. of CSE, SoCI Module-I | Unit-I Page 56

Module-I | Unit-I VFSTR Deemed to be University

Syntactic combinations = 2 x 2 = 4

Total analyses = 9 x 4 = 36

P(correct) = 1/36 = 0.0278 (approximately 2.78%)

This illustrates combinatorial explosion motivating statistical or neural disambiguation rather than
enumeration.

Problem 5.3: Problem 5.3 - Latency Budget Allocation

• Topic: Module-I, Unit-I - Themes of NLP

Problem 5.3 - Latency Budget Allocation

An NLP pipeline must complete within 100 ms. Tokenization takes 2 ms, parsing 45 ms, semantic role
labeling 60 ms, all sequential. Which stage must be optimized first to meet budget, and what is the current
total?

Solution:

Total = 2 + 45 + 60 = 107 ms (exceeds budget by 7 ms)

SRL at 60 ms is the largest component; optimizing SRL or parallelizing SRL with parsing after partial trees
yields the greatest savings.

• Target: reduce SRL by at least 7 ms or overlap parsing and SRL on sentence chunks.

Problem 6.1: Problem 6.1 - Morphological Richness and Vocabulary

• Topic: Module-I, Unit-I - Language and Knowledge

Problem 6.1 - Morphological Richness and Vocabulary

Estimate surface vocabulary inflation for a morphologically rich language. A Hindi verb root "chal" (walk)
generates 32 inflected forms across tense, aspect, mood, person, and gender. English verb "walk"
generates 4 forms (walk, walks, walked, walking). If a corpus contains 500 distinct verb roots per language,
compare approximate verb-form counts.

Solution:

Hindi verb forms (approximate) = 500 x 32 = 16,000

English verb forms = 500 x 4 = 2,000

Dept. of CSE, SoCI Module-I | Unit-I Page 57

Module-I | Unit-I VFSTR Deemed to be University

Ratio = 16,000 / 2,000 = 8x

Actual vocabulary inflation is higher when nominal inflection and compounding are included. This motivates
morphological analysis before statistical estimation in Indian language pipelines.

Problem 6.2: Problem 6.2 - Layer Identification Drill

• Topic: Module-I, Unit-I - Language and Knowledge

Problem 6.2 - Layer Identification Drill

• Classify which primary knowledge layer resolves each item: (a) schwa deletion in Hindi TTS, (b)
choosing bank=finance in "deposited money at the bank", (c) detecting sarcasm in "Great job, you failed
again."

Solution:

(a) Phonological/phonetic layer

(b) Lexical-semantic (word sense disambiguation) with pragmatic context

(c) Pragmatic layer (literal semantic polarity is positive; intended meaning is negative)

• Examination tip: label layers precisely; do not write "semantics" for everything.

Problem 6.3: Problem 6.3 - Knowledge Base Lookup

• Topic: Module-I, Unit-I - Language and Knowledge

Problem 6.3 - Knowledge Base Lookup

A QA system stores 2 million triples. Average lookup latency is 5 ms per triple query. A question requires
chaining 4 triple lookups sequentially. What is knowledge retrieval time excluding language processing?

Solution:

Total = 4 x 5 ms = 20 ms

If lookups parallelize when independent, time could drop to 5 ms. Sequential dependency (answer to step 1
informs step 2) forces additive latency.

Problem 7.1: Problem 7.1 - Combinatorial Ambiguity

Dept. of CSE, SoCI Module-I | Unit-I Page 58

Module-I | Unit-I VFSTR Deemed to be University

• Topic: Module-I, Unit-I - Challenges in NLP

Problem 7.1 - Combinatorial Ambiguity

A sentence has 3 lexical ambiguities (2 senses each) and 2 PP attachments (2 choices each), independent.
How many combined readings? Uniform prior: probability of guessing the single correct reading?

Solution:

• Lexical: 2^3 = 8

• Attachments: 2^2 = 4

Total = 8 x 4 = 32

P(correct guess) = 1/32 = 0.03125 (3.125%)

Problem 7.2: Problem 7.2 - Paraphrase Coverage

• Topic: Module-I, Unit-I - Challenges in NLP

Problem 7.2 - Paraphrase Coverage

An intent dataset has 1,000 utterances covering 50 intents (20 per intent). A paraphrase augmentation
adds 5 variants per utterance. New total utterances?

Solution:

Original = 1,000

Augmented per utterance = 5 new variants

New variants = 1,000 x 5 = 5,000

Total = 1,000 + 5,000 = 6,000 utterances

Problem 7.3: Problem 7.3 - Language Resource Gap

• Topic: Module-I, Unit-I - Challenges in NLP

Problem 7.3 - Language Resource Gap

English has 3 billion tokens of labeled NER data; Telugu has 30 million (1% of English). A model needs 100
million tokens to reach 90% F1. Assuming linear data scaling is optimistic, is Telugu data sufficient?

Dept. of CSE, SoCI Module-I | Unit-I Page 59

Module-I | Unit-I VFSTR Deemed to be University

Solution:

Telugu has 30M < 100M required

Deficit = 70 million tokens

Transfer learning or multilingual joint training is necessary; monolingual Telugu-only training is insufficient
under stated assumption.

Problem 8.1: Problem 8.1 - CFG Derivation Length

• Topic: Module-I, Unit-I - Language and Grammar

Problem 8.1 - CFG Derivation Length

• Given rules: S->NP VP (1), NP->Det N (2), NP->Det Adj N (3), VP->V NP (4), with lexicon Det->the,
Adj->old, N->man|telescope, V->saw. Derive "I saw the man" using minimal rules if pronoun NP->I is
added. Count rule applications.

Solution:

S => NP VP (1)

NP => I (lexical)

VP => V NP (4)

V => saw

NP => Det N (2)

Det => the, N => man

• Rule applications (non-lexical): 1 + 1 + 4 + 2 = 4 productions to expand S, NP, VP, inner NP

• Including lexical expansions: 6 total rule applications if counting each CFG step.

Problem 8.2: Problem 8.2 - Parse Count

• Topic: Module-I, Unit-I - Language and Grammar

Problem 8.2 - Parse Count

A CFG parser produces 5 trees for a 12-word sentence. If each tree requires 40 ms to score with a
reranker, what is total reranking time?

Dept. of CSE, SoCI Module-I | Unit-I Page 60

Module-I | Unit-I VFSTR Deemed to be University

Solution:

5 x 40 ms = 200 ms

Problem 8.3: Problem 8.3 - Attachment Probability

• Topic: Module-I, Unit-I - Language and Grammar

Problem 8.3 - Attachment Probability

• Corpus statistics: verb-attached PP follows 'saw' 70% of time; noun-attached 30%. For ambiguous
sentence, what attachment does a maximum-likelihood model choose and what is its probability?

Solution:

Choose verb attachment (VP) with P = 0.70

Problem 9.1: Problem 9.1 - Morphological Forms per Root

• Topic: Module-I, Unit-I - Processing Indian Languages

Problem 9.1 - Morphological Forms per Root

A Hindi verb root generates 28 inflected forms; Telugu root generates 35. A lexicon lists 2,000 verb roots
per language. Compare approximate inflected verb entries if all combinations occur.

Solution:

Hindi = 2,000 x 28 = 56,000

Telugu = 2,000 x 35 = 70,000

Difference = 14,000 more Telugu surface verb forms

Problem 9.2: Problem 9.2 - Tokenization Error Impact

• Topic: Module-I, Unit-I - Processing Indian Languages

Problem 9.2 - Tokenization Error Impact

A whitespace tokenizer achieves 95% token accuracy on Hindi social media; 5% errors on 1 million tokens.
How many erroneous tokens? If 40% of errors propagate to wrong POS, how many POS errors expected
from token errors?

Dept. of CSE, SoCI Module-I | Unit-I Page 61

Module-I | Unit-I VFSTR Deemed to be University

Solution:

Erroneous tokens = 0.05 x 1,000,000 = 50,000

POS errors from token errors = 0.40 x 50,000 = 20,000

Problem 9.3: Problem 9.3 - Parallel Corpus Deficit

• Topic: Module-I, Unit-I - Processing Indian Languages

Problem 9.3 - Parallel Corpus Deficit

• English-Hindi parallel corpus: 10M sentence pairs.

• English-Telugu: 1M pairs. BLEU improves as 22 + 3 log10(N). Compute BLEU for each pair.

Solution:

• Hindi: log10(10^7)=7; BLEU = 22 + 3(7) = 43

• Telugu: log10(10^6)=6; BLEU = 22 + 3(6) = 40

Gap = 3 BLEU points from tenfold less data under model

Problem 10.1: Problem 10.1 - Pipeline Bottleneck

• Topic: Module-I, Unit-I - Components of NLP Systems

Problem 10.1 - Pipeline Bottleneck

Tokenizer processes 1,000,000 tokens/sec. Parser handles 5,000 sentences/sec averaging 20 tokens
each. SRL handles 2,000 sentences/sec. Which component limits throughput on continuous stream?

Solution:

Parser token demand = 5,000 x 20 = 100,000 tokens/sec (not limiting vs 1M tok/s)

Sentences into SRL = 2,000/sec vs parser 5,000/sec

Bottleneck = SRL at 2,000 sentences/sec

Problem 10.2: Problem 10.2 - Error Propagation

• Topic: Module-I, Unit-I - Components of NLP Systems

Dept. of CSE, SoCI Module-I | Unit-I Page 62

Module-I | Unit-I VFSTR Deemed to be University

Problem 10.2 - Error Propagation

Token accuracy 98%, morph accuracy 95% given correct token, parse accuracy 90% given correct morph.
Chain accuracy approx product?

Solution:

0.98 x 0.95 x 0.90 = 0.8379 (83.79% end-to-end parse accuracy)

Problem 10.3: Problem 10.3 - Component Latency Budget

• Topic: Module-I, Unit-I - Components of NLP Systems

Problem 10.3 - Component Latency Budget

• Total budget 250 ms: token 5 ms, morph 15 ms, parse 80 ms, semantics 120 ms, discourse 50 ms if run
sequentially. Total? Which components to parallelize?

Solution:

Sequential total = 5+15+80+120+50 = 270 ms (over budget by 20 ms)

Run discourse partial parallel with semantics after core parse, or optimize semantics by 20 ms

Problem 11.1: Problem 11.1 - Phase Trace for Simple Sentence

• Topic: Module-I, Unit-I - Phases of Processing

Problem 11.1 - Phase Trace for Simple Sentence

• Trace 'Cats chase mice' through all five phases with POS: cats/NNS, chase/VBP, mice/NNS.

Solution:

• Lexical: [Cats, chase, mice]

• Morphological: cat plural; chase present; mouse plural

• Syntactic: NP(Cats) VP(chase NP(mice)); SVO

• Semantic: Agent=cats, Theme=mice, Action=CHASE

• Pragmatic: Generic biological statement; no deixis

Dept. of CSE, SoCI Module-I | Unit-I Page 63

Module-I | Unit-I VFSTR Deemed to be University

Problem 11.2: Problem 11.2 - Phase Timing

• Topic: Module-I, Unit-I - Phases of Processing

Problem 11.2 - Phase Timing

Lexical 3 ms, morph 10 ms, syntax 40 ms, semantics 90 ms, pragmatics 30 ms sequential. Total? If syntax
and semantics parallelized after morph, new total?

Solution:

• Sequential: 3+10+40+90+30 = 173 ms

Parallel syntax+sem after morph: 3+10+max(40,90)+30 = 133 ms

Problem 11.3: Problem 11.3 - Pragmatic Misclassification Cost

• Topic: Module-I, Unit-I - Phases of Processing

Problem 11.3 - Pragmatic Misclassification Cost

• Intent bot: 10,000 requests/day; 5% pragmatic errors on indirect requests; each error costs Rs 8 support
call. Daily cost?

Solution:

Errors = 0.05 x 10,000 = 500

Cost = 500 x 8 = Rs 4,000 per day

Problem 12.1: Problem 12.1 - Word Sense Disambiguation

• Topic: Module-I, Unit-I - The Problem of Ambiguity

Problem 12.1 - Word Sense Disambiguation

• Corpus counts for 'bank': finance sense 900, river sense 100. Maximum likelihood sense for unseen
context with no other cues?

Solution:

P(finance) = 900/1000 = 0.90

Choose finance sense

Dept. of CSE, SoCI Module-I | Unit-I Page 64

Module-I | Unit-I VFSTR Deemed to be University

Problem 12.2: Problem 12.2 - Combined Ambiguity

• Topic: Module-I, Unit-I - The Problem of Ambiguity

Problem 12.2 - Combined Ambiguity

2 lexical (3 senses each), 1 syntactic (2 trees), 1 anaphoric (2 antecedents). Independent combined

readings?

Solution:

3 x 3 x 2 x 2 = 36 readings

Problem 12.3: Problem 12.3 - Parser Beam Search

• Topic: Module-I, Unit-I - The Problem of Ambiguity

Problem 12.3 - Parser Beam Search

Parser keeps 10-best trees; correct tree ranked 3rd with score 0.82; top tree 0.85. If reranker improves
correct tree score by 0.05, new ranking?

Solution:

Correct tree new score = 0.87; becomes rank 1 (beats 0.85)

Problem 13.1: Problem 13.1 - Stem vs Lemma

• Topic: Module-I, Unit-I - Typical NLP Tasks

Problem 13.1 - Stem vs Lemma

• Tokens: studies, studied, studying. Porter stemmer outputs studi for all. Lemmatizer with POS yields
study. If vocabulary reduction goal counts unique types before 3 and after stem vs lemma, compare.

Solution:

Before = 3 types

After stem = 1 type (studi)

After lemma = 1 type (study)

Both collapse count; stem may merge unrelated words sharing prefix in other examples

Dept. of CSE, SoCI Module-I | Unit-I Page 65

Module-I | Unit-I VFSTR Deemed to be University

Problem 13.2: Problem 13.2 - NER Span Count

• Topic: Module-I, Unit-I - Typical NLP Tasks

Problem 13.2 - NER Span Count

• Text: 'Rama works at VFSTR in Hyderabad.' Entities: Rama/PERSON, VFSTR/ORG, Hyderabad/LOC.

How many entity spans? If BIO tagging uses 2 tags per entity span on average, total entity tags?

Solution:

3 spans

BIO tags approximately 3 x 2 = 6 tags (B-PER, I-PER not needed for single token; adjust: 3 B-tags + 0
I-tags = 3 minimum; multi-token would add I-tags)

Problem 13.3: Problem 13.3 - Retrieval vs QA

• Topic: Module-I, Unit-I - Typical NLP Tasks

Problem 13.3 - Retrieval vs QA

Corpus 1 million docs; retrieval returns top 10. QA model reads 10 docs, finds answer in 1. If retrieval
recall@10 is 80%, what fraction of questions can QA theoretically answer?

Solution:

0.80 of questions have answer in top 10; QA upper bound 80% if reader perfect

Practice Exercises

E3.1. Name three representative systems or methods from the symbolic era and state one
limitation shared by all three.

• Answer: Examples: ELIZA, SHRDLU, early MT rule systems.

• Shared limitation: brittle coverage outside designed constructions.

E3.2. Explain why Hidden Markov Models were a natural fit for part-of-speech tagging.

• Answer: POS tagging is a sequence labeling problem with hidden grammatical states and
observed words; HMMs model transitions and emissions probabilistically.

Dept. of CSE, SoCI Module-I | Unit-I Page 66

Module-I | Unit-I VFSTR Deemed to be University

E3.3. Place BERT in the correct NLP era and justify with two distinguishing features.

• Answer: Neural era; uses transformer self-attention and pretrained contextual embeddings
fine-tuned on downstream tasks.

E4.1. List four NLP applications and map each to its primary underlying task.

• Answer: Search-indexing/IR; Translation-MT; Voice assistant-intent+slot filling; Sentiment

analysis-classification.

E4.2. Why is false positive rate especially costly in email spam filtering?

• Answer: Legitimate mail wrongly blocked may contain invoices, admissions, or medical results,
causing real harm.

E4.3. Outline the encoder-decoder flow for English-to-Telugu translation in three sentences.

• Answer: Encoder reads English tokens into hidden states; attention links each decoder step to
relevant English words; decoder emits Telugu tokens with appropriate morphology and SOV
order.

E5.1. Contrast one-hot and contextual embeddings for the word 'bank' in two sentences.

• Answer: One-hot: identical vectors regardless of financial vs river sense.

• Contextual: different vectors conditioned on surrounding words like 'money' vs 'river'.

E5.2. Give an example of deixis and state what contextual information resolves it.

• Answer: 'Come here tomorrow' requires speaker location (here), hearer position, and calendar
date (tomorrow).

E5.3. Why is TF-IDF still used when neural embeddings exist?

• Answer: Interpretable, cheap to compute, strong baseline for document classification without
GPU training.

E6.1. Explain why 'The chicken is ready to eat' requires world knowledge beyond syntax.

• Answer: Syntax allows agent and patient readings; world knowledge about food preparation
favors chicken as food.

Dept. of CSE, SoCI Module-I | Unit-I Page 67

Module-I | Unit-I VFSTR Deemed to be University

E6.2. List the six knowledge layers with one NLP task each.

• Answer: Phonetics-ASR; Morphology-analyzer; Syntax-parsing; Semantics-SRL;

Pragmatics-intent; World knowledge-QA.

E6.3. Analyze 'I saw her duck' at lexical and syntactic levels.

• Answer: Lexical: duck noun vs verb.

• Syntactic: simple NP object vs reduced relative clause with verb duck.

E7.1. Give original examples of variability and ambiguity.

• Answer: Variability: 'cancel my order' vs 'I want a refund'.

• Ambiguity: 'He ate pizza with anchovies' (pizza topping vs instrument).

E7.2. Why is code-mixing a challenge for Indian language NLP?

• Answer: Tokenizer and POS models trained on monolingual text fail on script switches and
mixed morphology.

E7.3. Map sarcasm detection to challenge type and module.

• Answer: Pragmatic non-literal language challenge; handled by sentiment/intent classifiers with

contextual models.

E8.1. Draw constituency and dependency representations for 'The cat sat on the mat'.

• Answer: Constituency: S[NP[Det The, N cat], VP[V sat, PP[on, NP[the mat]]]].

• Dependency: sat root; cat nsubj; mat obl; on case.

E8.2. Explain PP attachment ambiguity with a new example.

• Answer: 'She opened the door with the key' - key as instrument vs door having a key.

E8.3. Why are dependency parsers popular for multilingual NLP?

• Answer: Dependencies capture head-dependent relations without language-specific phrase

labels; SOV and SVO languages share framework.

Dept. of CSE, SoCI Module-I | Unit-I Page 68

Module-I | Unit-I VFSTR Deemed to be University

E9.1. Contrast SOV Hindi with SVO English for MT challenges.

• Answer: Reorder constituents; align verb inflections with subject gender/number; handle
postpositions.

E9.2. Why is stemming alone inadequate for Tamil verbs?

• Answer: Agglutinative suffix chains encode multiple features; stemmer may over-truncate or
under-segment.

E9.3. Describe code-mixing in a sample Hinglish tweet.

• Answer: Latin English content words in Devanagari Hindi frame; tokenizer needs mixed-script
handling.

E10.1. List six components of a classical NLP system.

• Answer: Tokenizer, morphological analyzer, parser, semantic analyzer, discourse/pragmatic

analyzer, knowledge base/reasoner.

E10.2. Explain error propagation with a concrete tokenization mistake.

• Answer: Split 'New York' wrongly yields two locations; NER and relation extraction fail.

E10.3. Where does ASR fit relative to NLP components?

• Answer: Before tokenizer; converts speech signal to text feeding lexical analysis.

E11.1. Apply five phases to 'Open the window please'.

• Answer: Lexical tokens; morph imperatives; syntax command VP; semantics action OPEN
object window; pragmatics polite request.

E11.2. Why is 'Can you pass the salt?' pragmatically a request?

• Answer: Dining context; speaker wants action not ability information.

E11.3. Which phase fixes 'their' binding in Example B?

• Answer: Syntactic-semantic interface and discourse coreference in pragmatic phase.

Dept. of CSE, SoCI Module-I | Unit-I Page 69

Module-I | Unit-I VFSTR Deemed to be University

E12.1. Give original examples for pragmatic and discourse ambiguity.

• Answer: Pragmatic: 'Nice weather' during storm (sarcasm).

• Discourse: two sentences without clear causal link.

E12.2. Classify ambiguity types in 'Flying planes can be dangerous'.

• Answer: Syntactic (modifier attachment) leading to semantic reading differences.

E12.3. How do transformers help lexical ambiguity?

• Answer: Contextual embeddings differ for each occurrence; classifier layers pick sense.

E13.1. Distinguish information retrieval and question answering.

• Answer: IR ranks documents; QA returns specific answer string or span.

E13.2. Distinguish ASR and TTS.

• Answer: ASR speech-to-text; TTS text-to-speech.

E13.3. Map sentiment analysis to task category.

• Answer: Document or sentence text classification.

Review Questions

[Remember] State the three parent disciplines of NLP.

• Answer: Linguistics, Computer Science, Statistics/Machine Learning.

[Understand] Why cannot NLP be taught as pure computer science?

• Answer: Language requires formal linguistic theory and probabilistic modeling, not algorithms
alone.

[Apply] Design NLU output for: 'Remind me to call Mom at 6 pm'.

• Answer: Intent=SET_REMINDER; contact=Mom; time=18:00.

Dept. of CSE, SoCI Module-I | Unit-I Page 70

Module-I | Unit-I VFSTR Deemed to be University

[Analyze] Compare pipeline vs end-to-end neural approach.

• Answer: Pipeline: modular, interpretable, error propagation.

• End-to-end: learns features jointly, less interpretable.

[Evaluate] Is 70% intent accuracy acceptable for hospital triage bot?

• Answer: No; high-risk misclassification cost demands higher recall on emergency intents.

[Create] Propose architecture for campus exam-schedule FAQ bot.

• Answer: Document ingestion, chunking, retrieval, NLU for query type, NLG answer with source
citation.

[Remember] List three neighbors of NLP besides CS.

• Answer: Linguistics, statistics, speech processing (any valid trio).

[Understand] Explain why regex spam filters fail on obfuscated spam.

• Answer: Rules do not generalize; ML learns variant patterns.

[Analyze] When is a rule-based system preferable to ML?

• Answer: Low data, high interpretability need, stable patterns.

[Remember] In which decade did ELIZA appear?

• Answer: 1960s (1966).

[Understand] Why did the symbolic era fail to solve open-domain machine translation?

• Answer: Rule coverage could not keep pace with lexical and syntactic variety; no robust ranking
among competing analyses.

[Apply] A startup has 5,000 labeled emails and no GPU cluster. Which era's techniques are most
practical initially?

• Answer: Statistical methods (Naive Bayes or logistic regression on TF-IDF) or small fine-tuned
models; full GPT-scale training is infeasible.

Dept. of CSE, SoCI Module-I | Unit-I Page 71

Module-I | Unit-I VFSTR Deemed to be University

[Analyze] Compare failure modes of rule-based parsers versus neural parsers.

• Answer: Rule-based: crash on unseen constructions.

• Neural: may produce fluent but structurally wrong analyses with high confidence.

[Evaluate] Should a hospital discharge-summary system rely solely on a generative LLM?

• Answer: No; hallucination risk and regulatory audit requirements favor retrieval grounding, rules
for protected health fields, and human review.

[Create] Design a three-stage email pipeline using one technique from each era.

• Answer: Regex header normalization (symbolic), TF-IDF spam scorer (statistical), transformer
phishing detector on body text (neural).

[Remember] What does TF-IDF stand for and what does each component measure?

• Answer: Term Frequency-Inverse Document Frequency; local prominence vs corpus-wide

rarity.

[Understand] Why does spam detection exemplify text classification?

• Answer: Input is document text; output is discrete class label spam/ham learned from labeled
examples.

[Apply] Choose a metric for a medical chatbot triage system and justify.

• Answer: High recall on emergency intents to avoid missed urgent cases, even at some
precision cost.

[Analyze] Why might BLEU underrate a paraphrastic correct translation?

• Answer: BLEU rewards n-gram overlap; valid synonyms not in reference score zero.

[Evaluate] Is bag-of-words sufficient for phishing detection in 2025?

• Answer: Partially; character n-grams and neural models catch obfuscation, but bag-of-words
remains a fast baseline.

[Create] Sketch data requirements for a Telugu news summarization service.

Dept. of CSE, SoCI Module-I | Unit-I Page 72

Module-I | Unit-I VFSTR Deemed to be University

• Answer: Large Telugu news corpus, reference summaries or extractive labels, evaluation for
fluency and factuality, domain coverage across topics.

[Remember] Name the four central themes of NLP from this unit.

• Answer: Representation, ambiguity, context, computation.

[Understand] Why does static Word2Vec fail on 'He went to the bank' without context?

• Answer: Single vector per type cannot distinguish financial vs river senses.

[Apply] Select representation for 10 million product reviews on a single server.

• Answer: TF-IDF or bag-of-words with linear classifier; full transformer training may be overkill.

[Analyze] How do ambiguity and context interact in pronoun resolution?

• Answer: Multiple syntactic antecedents possible; discourse salience and world knowledge
select one.

[Evaluate] Is O(n^3) parsing acceptable for 50-word sentences in batch MT?

• Answer: Yes for offline batch (50^3 = 125,000 steps is small); not for per-keystroke
autocomplete.

[Create] Propose representation for multilingual Indian social media text with code-mixing.

• Answer: Subword BPE on mixed script-normalized corpus plus language ID tags; contextual
embeddings fine-tuned on in-domain data.

[Remember] How many knowledge layers are taught in Unit I?

• Answer: Six.

[Understand] Distinguish semantic and pragmatic knowledge.

• Answer: Semantic: literal truth-conditional meaning.

• Pragmatic: speaker intent and indirect force.

Dept. of CSE, SoCI Module-I | Unit-I Page 73

Module-I | Unit-I VFSTR Deemed to be University

[Apply] Which layers does a spell checker primarily use?

• Answer: Orthographic/phonological and morphological; minimal semantics for homophone

disambiguation in advanced systems.

[Analyze] Why do LLMs fail on physical commonsense in Winograd schemas?

• Answer: Statistical association without grounded simulation; fragile pronoun resolution under
adversarial wording.

[Evaluate] Is WordNet alone sufficient for open-domain QA?

• Answer: No; coverage and recency limits require corpus retrieval or larger parametric
knowledge.

[Create] Design knowledge sources for a campus hostel FAQ bot.

• Answer: Structured timetable database, policy PDF retrieval, intent ontology for complaint
types, optional neural paraphrase matcher.

[Remember] List five major challenges in NLP from this topic.

• Answer: Ambiguity, variability, context dependence, linguistic diversity, non-standard input.

[Understand] Explain paraphrase challenge for intent classification.

• Answer: Same intent, many surface forms; model must generalize beyond training phrasing.

[Apply] Which challenge dominates OCR of handwritten Telugu forms?

• Answer: Script noise and non-standard spelling; preprocessing and lexicon constraints help.

[Analyze] How does error propagation amplify early tokenizer mistakes?

• Answer: Wrong split yields wrong POS, wrong parse, wrong semantic roles in cascade.

[Evaluate] Is English benchmark leadership sufficient to deploy Hindi social media moderation?

• Answer: No; domain and language shift require Hindi-specific evaluation and data.

Dept. of CSE, SoCI Module-I | Unit-I Page 74

Module-I | Unit-I VFSTR Deemed to be University

[Create] Propose mitigation plan for low-resource Kannada POS tagging.

• Answer: Multilingual BERT fine-tuning, transfer from Telugu, small active learning annotation
batch, morphological features.

[Remember] What does CFG stand for?

• Answer: Context-Free Grammar.

[Understand] Difference between constituency and dependency grammar?

• Answer: Constituency groups phrases; dependency links heads to dependents directly.

[Apply] Use corpus stats to resolve PP attachment in 'eat pizza with fork'.

• Answer: Instrument attachment to verb 'eat' is statistically preferred.

[Analyze] Why does CFG alone not resolve 'I saw the man with the telescope'?

• Answer: Multiple valid trees; needs semantic or statistical preferences.

[Evaluate] Is hand-written CFG sufficient for open-domain English parsing?

• Answer: No; needs broad coverage grammar or neural parser trained on treebank.

[Create] Write four CFG rules generating simple Hindi SOV sentences.

• Answer: S->NP VP; NP->N; VP->NP V; lexical entries for nouns and verbs.

[Remember] Name three structural features of Indian languages.

• Answer: SOV/free order, rich morphology, multiple scripts (any valid trio).

[Understand] Why whitespace tokenization fails for some Indian language compounds?

• Answer: Bound morphemes and conjuncts lack spaces marking word boundaries.

[Apply] Choose morphological tool type for Hindi lemmatization.

• Answer: Finite-state transducer or neural morphological analyzer with case/gender features.

Dept. of CSE, SoCI Module-I | Unit-I Page 75

Module-I | Unit-I VFSTR Deemed to be University

[Analyze] How does free word order affect dependency parsing?

• Answer: Must use case markers and head rules, not fixed position heuristics.

[Evaluate] Can English BERT be deployed for Telugu NER without fine-tuning?

• Answer: Poorly; needs Telugu fine-tuning or Indic multilingual model.

[Create] Outline pipeline for Telugu agricultural SMS classification.

• Answer: Script normalize, morph segment, Indic embedding, classifier, evaluate on

farmer-language test set.

[Remember] Name the first and last components in a text-only classical stack.

• Answer: Tokenizer/lexical analyzer; knowledge base/reasoner (or discourse before KB).

[Understand] Why pass n-best parses to semantics?

• Answer: Preserve ambiguity; semantic module may rank readings using wider context.

[Apply] Profile shows SRL dominates latency; propose fix.

• Answer: GPU batching, lighter model, or skip SRL on low-risk sentences.

[Analyze] Compare modular vs end-to-end component boundaries.

• Answer: Modular: clear interfaces, error trace; E2E: joint training, opaque failures.

[Evaluate] Should knowledge base be removed if using GPT-4?

• Answer: No for audited enterprise facts; parametric knowledge drifts and hallucinates.

[Create] Sketch component diagram for Hindi voice banking assistant.

• Answer: ASR, tokenizer, morph, intent+slots, fraud KB lookup, NLG, TTS.

[Remember] List the five processing phases in order.

• Answer: Lexical, morphological, syntactic, semantic, pragmatic/discourse.

Dept. of CSE, SoCI Module-I | Unit-I Page 76

Module-I | Unit-I VFSTR Deemed to be University

[Understand] Distinguish semantic and pragmatic phases for indirect requests.

• Answer: Semantic: literal ability question.

• Pragmatic: intended request act.

[Apply] Identify phase where PP attachment is resolved.

• Answer: Primarily syntactic with semantic/pragmatic disambiguation.

[Analyze] How does changing 'hungry' to 'tired' alter pragmatic output?

• Answer: Same structure; semantic predicate changes; pragmatic intent shifts food vs rest.

[Evaluate] Is skipping morph phase acceptable for English POS tagging?

• Answer: Sometimes with subword models; risky for Indian languages.

[Create] Write exam-style five-phase analysis for one Hindi sentence.

• Answer: Student provides SOV sentence with full phase trace and agreement features.

[Remember] How many ambiguity types in Unit I taxonomy?

• Answer: Six.

[Understand] Distinguish semantic scope and syntactic attachment ambiguity.

• Answer: Scope: quantifier/modifier logical interaction.

• Attachment: phrase structure bracketing.

[Apply] Resolve 'he' in 'Rama met Sita. He smiled.'

• Answer: Rama antecedent via salience and gender defaults; discourse centering.

[Analyze] Why PP attachment is syntactic yet needs world knowledge?

• Answer: Multiple valid trees; semantics selects plausible instrument vs modifier.

Dept. of CSE, SoCI Module-I | Unit-I Page 77

Module-I | Unit-I VFSTR Deemed to be University

[Evaluate] Is single-parse pipeline safe for legal contract analysis?

• Answer: No; needs n-best or human review for attachment and scope ambiguities.

[Create] Construct sentence with three ambiguity types simultaneously.

• Answer: Student sentence with lexical, syntactic, and anaphoric layers explained.

[Remember] Name any five of the seventeen typical NLP tasks.

• Answer: Tokenization, POS tagging, NER, MT, summarization (any five valid).

[Understand] Why is lemmatization harder than stemming?

• Answer: Requires POS and morphological analysis for correct dictionary form.

[Apply] Choose tasks for English-Hindi glossary extraction from parallel corpus.

• Answer: Tokenization, alignment, possibly NER for terms, MT evaluation.

[Analyze] How does dependency parsing support relation extraction?

• Answer: Dependency paths connect entity mentions to predicate relations.

[Evaluate] Is topic modeling sufficient for customer complaint routing?

• Answer: Partially; classification fine-tuned on labels likely superior for routing accuracy.

[Create] Decompose university exam-schedule chatbot into NLP tasks.

• Answer: ASR optional, intent classification, NER for course codes, QA over schedule KB, NLG
response.

Dept. of CSE, SoCI Module-I | Unit-I Page 78

Module-I | Unit-I VFSTR Deemed to be University

Appendix

Appendix - Unit I Formula Reference

| Quantity | Formula |

|----------|---------|

| Precision | P = TP / (TP + FP) |

| Recall | R = TP / (TP + FN) |

| F1-score | F1 = 2PR / (P + R) |

| Shannon entropy | H = -Sum p_i log2(p_i) |

| TF-IDF | TF-IDF(t,d) = tf(t,d) x log(N / df(t)) |

| Type-token ratio | TTR = |V| / total tokens |

| Bigram probability | P(w_i | w_{i-1}) = C(w_{i-1}, w_i) / C(w_{i-1}) |

Dept. of CSE, SoCI Module-I | Unit-I Page 79