MODULE 1

CHAPTER1
INTRODUCTION
1 WHAT IS NATURAL LANGUAGE PROCESSING (NLP)

 Language is the primary mode of human communication, shaping thoughts and emotions.
 Processing natural language is an innate ability of humans, but computational models are needed for
automatic processing.
 NLP involves developing computational models to process human language.
The main goals of NLP are:
 Developing automated tools for language processing – Enabling machines to process and understand
human language.
 Understanding human communication better – Studying how humans acquire, store, and use language.
Requirements for NLP Development:
 Understanding how humans acquire, store, and process language.
 Knowledge of linguistic structures and real-world contexts.
Approaches to NLP
There are two major historical approaches to NLP:
 Rationalist Approach
 Assumes the presence of an innate language faculty in the human brain.
 Argues that language is too complex to be learned only from sensory input.
 Associated with theories like Chomsky’s Universal Grammar.
 Empiricist Approach
 Does not assume a built-in language faculty.
 Believes language is learned through experience using pattern recognition, generalization, and
association.
 Emphasizes learning from data (aligned with modern machine learning and deep learning techniques).
Conclusion
 NLP is essential for automating language-related tasks and enhancing human-computer interaction.
 The field continues to evolve with advancements in artificial intelligence, machine learning, and deep
learning.
 Understanding NLP helps in developing intelligent applications such as chatbots, translation tools, and
sentiment analysis systems.

1
2 ORIGINS OF NLP

 NLP originated from machine translation research.

 It involves both understanding (interpretation) and generation (production) of language, including speech
processing.

Relation to Other Linguistic Fields

 Theoretical Linguistics: Studies language structure and general linguistic principles without concern for
actual sentence processing.
 Psycholinguistics: Investigates how humans understand and produce language, using empirical research.
 Computational Linguistics: Uses computational models to apply linguistic theories and process
language.

Challenges in Computational Linguistics

 Representing language computationally is difficult because full knowledge representation is nearly

impossible.
 Distinguishing between "knowledge" and "language" is crucial.

Two Main Approaches in NLP

 Knowledge-Driven Systems: Use handcrafted rules but are limited by the difficulty of encoding
linguistic knowledge.
 Data-Driven Systems: Learn from large datasets using machine learning, requiring less manual effort
and handling noisy data better.

NLP’s Modern Importance

 NLP has expanded beyond academic study to real-world applications, especially with web-based
information explosion.
 NLP plays a key role in information retrieval (e.g., extraction, summarization, question answering).

2
3 LANGUAGE AND KNOWLEDGE

 Language is the medium through which knowledge is expressed.

 The same knowledge can be conveyed in different languages, but separating language from its content is
challenging.
 Language processing involves converting natural language into a formal representation for computational
understanding.
 Text processing is viewed as a knowledge representation tool that evolves to accommodate new concepts.

Levels of Language Processing

1. Lexical Analysis (Word-Level Processing)

 Focuses on individual words as the fundamental syntactic and semantic units.

 Requires morphological knowledge (understanding word structures and formation rules).

Example:

"Run" → "Running" (morphological transformation)

2. Syntactic Analysis (Sentence-Level Processing)

 Analyzes sentence structure using grammatical rules.

 Ensures correctness in word order, number agreement, and case agreement.

Example:

✅ "I went to the market." (Correct structure)

❌ "Went the I market to." (Incorrect word order)

❌ "She are going to the market." (Incorrect agreement)

3. Semantic Analysis (Meaning Interpretation)

 Deals with understanding the meaning of words and sentences.

 Words can have multiple meanings, and context determines the correct interpretation.

Example:

"Colorless green ideas sleep furiously." (Grammatically correct but semantically meaningless—Chomsky, 1957)

Semantic structure is influenced by syntax and broader world knowledge.

4. Pragmatic Analysis (Context & Real-World Knowledge)

 Meaning is influenced by context, background knowledge, and speaker intention.

Example:

3
"Kabir and Ayan are married."

"Kabir and Suha are married."

The second sentence is often interpreted as "Kabir and Suha are married to each other," while the first sentence is
ambiguous.

5. Discourse Analysis (Paragraph and Document-Level Processing)

 Interprets larger text units by analyzing connections between sentences.

 Requires discourse knowledge (how previous sentences affect meaning).

Example:

"The district administration refused to give the trade union permission for the meeting because they feared
violence."

"The district administration refused to give the trade union permission for the meeting because they oppose
government."

The word "they" has different interpretations in each sentence, requiring pragmatic knowledge to resolve
ambiguity.

Highest Level: Pragmatic Analysis

Deals with how language is used in real-world situations.

Requires world knowledge beyond linguistic rules.

Conclusion

 Language processing involves multiple levels, each requiring different types of knowledge.
 While syntax and semantics are crucial, pragmatics and world knowledge play a vital role in
understanding meaning.
 Natural Language Processing (NLP) continues to evolve to bridge the gap between human
communication and computational understanding.

4 THE CHALLENGES OF NLP

4
  NLP faces challenges related to representation and interpretation of language.

 Human language is ambiguous and vague, making precise computational representation difficult.

 Capturing all required knowledge and context for language processing is nearly impossible.

Challenges in NLP:

A. Semantic Ambiguity
 Compositional semantics assumes sentence meaning is derived from the meanings of individual words.

 However, syntax and context also play a major role in meaning.

Example:

"The old man finally kicked the bucket."

The literal meaning is different from the idiomatic meaning ("died").

B. Context and Evolving Language

 Words gain new meanings and evolve over time.

Example:

 9/11 is widely understood to refer to the 2001 terrorist attack, not just a date.

 While initially meant "a short interval of time" but is now commonly used as a conjunction.

 Machines struggle to interpret such changes without explicit domain knowledge.

C. Idioms, Metaphors, and Ellipses

 Phrases often do not follow literal meaning.

Example:

“Kick the bucket” means to die, not literally kicking a bucket.

D. Quantifier Scoping Issues

 The scope of words like "each," "the," etc., is not always clear.

 Misinterpretation leads to incorrect processing.

E. Ambiguity in Language

5
 1. Word-Level Ambiguity

 Some words have multiple meanings.

Example:

Bank (financial institution or riverbank)

Bat (an animal or a cricket bat)

NLP techniques:

 Part-of-Speech (POS) Tagging helps determine word roles.

 Word Sense Disambiguation (WSD) determines the correct meaning based on context.

2. Structural Ambiguity

 Sentence structure creates multiple interpretations.

Example:

"Stolen rifle found by tree."

Was the rifle stolen, or was it found next to a tree?

Verb sub-categorization and probabilistic parsing help resolve such ambiguities.

3. Pragmatic and Discourse Ambiguities

 Meaning depends on speaker and listener knowledge, culture, and context.

Example:

Taj could mean:

Taj Mahal (monument)

Taj Hotels (business chain)

Taj Tea (brand)

Depends on the listener’s cultural background.

Complexity of Grammar in NLP

 Many grammars exist to describe sentence structures.

 Infinite sentence variations make it difficult to create exhaustive grammar rules.

6
  Humans understand even incorrect sentences, which makes NLP grammar design more complex.

Solutions to NLP Challenges

 Part-of-Speech Tagging: Identifies correct word roles.

 Word Sense Disambiguation: Resolves multiple meanings.

 Probabilistic Parsing: Helps determine correct sentence structure.

 Machine Learning & Deep Learning: Improve contextual understanding.

Conclusion

 NLP is difficult due to semantic ambiguity, evolving language, idioms, and context dependencies.

 Ambiguities occur at multiple levels (word, sentence, discourse).

 Grammar complexity makes NLP rule-based approaches less effective.

 AI and machine learning techniques are necessary to improve NLP accuracy.

5 LANGUAGE AND GRAMMAR

 Automatic processing of language requires explicit rules and exceptions to be explained to a computer.
7
  Grammar is a set of rules that defines a language, allowing parsing and sentence generation.
 These rules relate coding devices (language level) rather than world knowledge (Bharati et al., 1995).
 Syntax and semantics are interrelated, but grammar writing separates them for ease of processing.

Challenges in Language Specification

 Natural languages evolve constantly, making formal specification difficult.

 Exceptions in grammar rules make standardization complex.

Various grammatical frameworks have been developed to address these challenges.

Major Grammar Theories

Several grammar theories have been developed, each with a unique focus:

 Phrase Structure Grammar focuses on derivation.

 Dependency Grammar, Lexical Functional Grammar, Paninian Grammar, and Link Grammar
focus on relationships.

Chomsky’s Contributions to Grammar

 Noam Chomsky introduced a hierarchy of formal grammar based on complexity.

 Phrase Structure Rules (Rewrite Rules) form the basis of many grammar models.
 Generative Grammar: A rule-based system to generate well-formed sentences.
 Chomsky argued that phrase structure grammars are inadequate for natural language processing.

Transformational Grammar

 Introduced in Chomsky’s Syntactic Structures (1957).

 Proposes that each sentence has:

 Deep Structure: Represents core meaning.

 Surface Structure: The actual spoken/written form.

 Transformations convert deep structure into surface structure.

 Different sentences can share the same deep structure but have different surface structures.

Figure 1.1: Transformation from Deep Structure to Surface Structure

8
Example of Deep and Surface Structures

Sentences (1.4a) and (1.4b) share the same deep structure:

(1.4a) Pooja plays veena.

(1.4b) Veena is played by Pooja.

The transformation changes subject-object roles but retains meaning.

Components of Transformational Grammar

Transformational grammar consists of three main components:

1. Phrase Structure Grammar

 Generates natural language sentences and assigns structural descriptions.

Example of phrase structure rules:

S → NP + VP

VP → V + NP

NP → Det + Noun

V → Aux + Verb

Lexicon (word categories):

Det: the, a, an

Verb: catch, write, eat

9
Noun: police, snatcher

Auxiliary (Aux): will, is, can

Example sentence generated:

"The police will catch the snatcher." (1.5)

2. Transformational Rules

 Transform underlying structures into derived structures.

Example: Active to Passive transformation

NP₁ Aux-V NP₂ → NP₂ Aux + be + en V by NP₁

This rule converts an active sentence into a passive one by:

Adding "be" and "en"

Reordering sentence components

Example transformation:

Active: The police will catch the snatcher. (1.5)

Passive: The snatcher will be caught by the police.

Types of Transformations:

 Obligatory transformations: Ensure subject-verb agreement.

 Optional transformations: Modify sentence structure while keeping meaning.

3. Morphophonemic Rules

 Convert sentence structures into phonemes for pronunciation.

Figure 1.2: Phrase Structure Tree for "The police will catch the snatcher."

10
Passive Transformation Process

 The transformation of an active sentence into a passive follows these steps:

Apply passive transformation rules:

NP₁ Aux-V NP₂ → NP₂ Aux + be + en V by NP₁

Example transformation:

Active Sentence (1.5): The police will catch the snatcher.

Intermediate Passive Form: The snatcher will be en catch by the police. (Figure 1.3)

Figure 1.3: Passive Transformation Process for "The police will catch the snatcher."

Reordering rule: Changes "en catch" → "catch en".

11
Morphophonemic rule: Converts "catch en" → "caught".

Final Passive Sentence: The snatcher will be caught by the police.

Complex Noun Phrases in Sentence Structure

 Noun phrases are not always simple (e.g., "The culprit").

They may contain:

Adjectives (e.g., The clever culprit).

Modifiers (e.g., The culprit in the alley).

Relative Clauses (e.g., The culprit who escaped).

Long-Distance Dependencies

 Definition: When a verb and its subject or object are separated by long phrases.
 Phrase structure rules struggle to account for such dependencies.

Example:

The book that the professor recommended was difficult to understand.

 Recommended and book are linked, but they are far apart in the sentence.

Wh-Movement (A Type of Long-Distance Dependency)

 Wh-movement occurs when a question word ("who," "what," "where," etc.) moves to the front of a
sentence.

Example:

Statement: You saw whom?

Question (Wh-movement applied): Whom did you see?

12
6 PROCESSING INDIAN LANGUAGES

· Non-linear Structure:

 Indic scripts often include conjunct characters and diacritics, requiring specialized rendering techniques.
 Unlike English, where letters appear sequentially, Indic characters may have parts appearing before, above,
or below the main consonant.

· Subject-Object-Verb (SOV) Order:

 Unlike English (which follows Subject-Verb-Object), Indian languages naturally place the verb at the end.
 NLP models trained on English-centric datasets struggle to adapt to this order.

· Free Word Order:

 Many Indian languages allow flexible word placement without changing meaning, complicating rule-based
parsing.
 Dependency parsing must rely more on semantic and morphological markers rather than strict word order.

· Spelling Variations:

 Hindi and other Indic languages have less rigid spelling standardization, leading to variations in
transliterations and informal spellings.
 This complicates spell-checking and text normalization.

· Rich Morphology:

 Indian languages exhibit extensive inflections, compounding, and agglutination.

 Morphological analyzers must handle a large number of word forms.

· Complex Predicates (CPs):

 CPs combine a noun/adjective with a verb (e.g., "dhyaan dena" in Hindi = "pay attention").
 Simple word segmentation does not work well; context-aware models are needed.

· Post-positions instead of Prepositions:

 Unlike English prepositions, Indian languages use postpositions (e.g., "ke liye" in Hindi instead of "for").
 This alters dependency parsing and phrase structure analysis.

· Verb Complexes:

 Indian languages often use verb sequences (e.g., "jaata raha" = "kept going"), making verb phrase
recognition harder.
 Standard NLP tools need adaptation to recognize and process such structures.

13
7 NLP APPLICATIONS

 Machine Translation

Automatic translation between languages.

Requires understanding of words, grammar, semantics, and world knowledge.

 Speech Recognition

Converts speech signals into text.

Challenges include pronunciation variations, homonyms (e.g., dear vs. deer), and acoustic ambiguities.

 Speech Synthesis

Converts text into speech.

Used in applications like reading emails or audiobooks.

Involves text processing as a key NLP component.

 Natural Language Interfaces to Databases

Enables querying structured databases using natural language sentences.

 Information Retrieval (IR)

Identifies relevant documents based on a user’s query.

NLP techniques used include:

Stop word elimination

Stemming

Phrase extraction

Word sense disambiguation

Query modification

Use of knowledge bases (e.g., WordNet, LDOCE, Roget’s Thesaurus)

 Information Extraction (IE)

Captures factual information from documents.

Unlike IR, it extracts structured information based on pre-defined templates.

Focuses on retrieving specific data from within a document rather than entire documents.

14
 Question Answering

Extracts precise answers from documents rather than retrieving whole documents.

Utilizes information extraction and semantic analysis.

Requires background knowledge for certain types of questions.

 Text Summarization

Generates concise summaries of documents.

Involves syntactic, semantic, and discourse-level text processing.

15